Structured Outputs & JSON 8 min read Updated Jul 7, 2026

How to Stop AI From Inventing Missing Data

When a source is missing a field, AI tends to fill the gap with a plausible guess instead of saying it isn't there. Here's how to make the model mark missing data explicitly — and check the result before you trust it.

Try the Extraction Prompt Generator

When the AI fills in what isn't there

You ask the AI to pull a customer record out of an email thread — name, company, plan, budget, deadline, phone. The thread has some of that and not the rest. What comes back looks complete: a budget figure, a region, a firm deadline, all in a confident tone. The problem is that half of it was never in the email. The model didn't find those values; it produced them.

This is the quiet failure mode of AI extraction. Asked for a field, the model treats returning something as the job and a blank as failure, so when the data is missing it reaches for the most plausible value instead of saying it isn't there. The output reads as data; some of it is invention. This guide is a prompt-and-review path that gets the model to mark what's missing instead of guessing it — and check the result before you trust it. It reduces invented data; it can't guarantee the model never guesses, so verification stays part of the process.

Why AI invents missing data

It's less hallucination than permission. A model completes patterns, and a record with a labelled empty field is an unfinished pattern it wants to close. Unless you tell it otherwise, three defaults push it toward inventing:

  • A blank looks like failure. Nothing in the request says an empty answer is acceptable, so the model treats not knowing as the wrong response and fills the gap.
  • Plausible beats absent. Given a customer with a company but no budget, a mid-range number fits — the model optimizes for a coherent record, not a faithful one.
  • Missing and omitted blur together. If the format lets the model drop a field it can't fill, a real gap and a value it forgot look identical downstream.

Step 1: Write an explicit missing-data policy

Decide, before you prompt, what missing should look like in the output, and name the exact value for it: null, "unknown", "not_in_source", or a "needs_review" flag for anything ambiguous. Pick one and state it, so the model has an approved way to say a field isn't there instead of improvising one.

The Mark Missing Fields in AI Extraction resource is a ready-made extraction prompt built around exactly this: it sets a single missing-value behavior, treats ambiguous data as missing rather than a guess, and carries the instruction not to invent or guess a value for a field that isn't in the source. You adapt it to your fields and run it in your own assistant.

Step 2: Make missing status explicit — don't let the field vanish

A missing value and an omitted key look the same to the code that reads the output, but they mean different things: one is a known gap, the other is the model quietly dropping a field it couldn't fill. Require every field in the schema to appear every time, with the missing marker when there's no value — so a gap is visible instead of silent.

This is a discipline a structured contract enforces well. The Create Reliable JSON Responses resource is a JSON contract tuned for it: optional fields with no value are set to null rather than omitted, and null means unknown, never an invented stand-in. Keeping the key present, always, is what turns a silent drop into a reviewable gap.

Step 3: Forbid inference and normalization

Two habits turn absent data into invented data: inferring a value from context, and normalizing a loose value into a precise one. Both feel helpful and both lose the truth. State the rules plainly — extract only what the source says, don't infer a value from surrounding hints, and don't convert a vague phrase into a specific one.

So "sometime next month" stays "sometime next month," not a firm date; a region you could guess from an email domain stays missing unless the text names it; a budget "in the low six figures" isn't rounded to a number. The point isn't to make the model less useful — it's to keep the line between what the source stated and what the model would have to assume, because that line is the whole difference between data and a guess.

Step 4: Require evidence for every value it fills

The strongest guard against invention is making the model show its work: for each field it fills, ask for the exact source span or location that supports it. A value the model can't point to is a value it made up — and requiring the citation makes that visible instead of buried in a confident-looking record.

The Reduce AI Hallucinations with Grounding resource is a grounding prompt built on this idea: it permits "the source does not say" as a valid answer, forbids uncited claims, and treats anything not in the source as unsupported even when it's plausible. Grounding doesn't prove the model got a value right, but it turns a bare assertion into a claim you can check against the text.

Step 5: Assemble it into one extraction prompt

These rules only work if they travel together in the prompt — the schema, the missing-value marker, the no-inference rule, and the evidence requirement, applied to your actual fields. You can hand-write that, or generate it.

The Extraction Prompt Generator builds an extraction prompt from your field list with a dedicated missing-data setting — leave empty, null, "unknown", or skip — plus an ambiguity policy, and it writes in the source-only, no-guessing rules. It produces prompt text you run in your own AI assistant; it doesn't call a model, read your source, or prove a value is real. The setting to weigh hardest is the missing-data one — it's the difference between a record with honest gaps and one that's confidently wrong.

Step 6: Check the output — structure first, then source

Even a careful prompt can produce a bad record, so treat the output as a draft to verify. Two checks catch different problems. First, structure: is every expected field present, is the missing marker used where it should be, are the types right? The AI Output Validator checks that in your browser and points at what drifted — but it checks the shape, not the facts. It can tell you a field is missing or malformed; it can't tell you whether a filled value is actually in your source.

That second check — is each value really supported by the text — is yours to run. The Hallucination Detection Prompt resource helps: it's an audit prompt that classifies each claim in the output as supported, contradicted, or unsupported against the source you paste, and quotes the span for the ones it can back. You still read the flagged items and decide; the audit surfaces what to look at, it doesn't certify the data.

Common mistakes

A few habits keep invented data flowing:

  • Not saying what missing looks like. With no approved way to mark a gap, the model invents one — or fills it.
  • Letting fields be omitted. A dropped key and a real gap read identically, so you can't tell forgotten from absent.
  • Allowing inference to be helpful. A guessed region or a firmed-up date is invented data wearing a useful face.
  • Trusting confidence. Models state made-up values in the same tone as real ones; the tone is not evidence.
  • Skipping the source check. A structural validator confirms the shape, not that the values came from the text — that check is still yours.

A worked example: a customer record from an email thread

Say you're extracting a customer record from an email thread. The thread gives a name and a company, mentions wanting "something around next month," and nothing else. Here's the invented version next to the honest one:

The same email, extracted without and with a missing-data policy 
// Without a missing-data policy, the model returns:
{
  "name": "Dana Ruiz",
  "company": "Northwind",
  "requested_plan": "Pro",       // guessed — never stated
  "budget": 25000,                // invented
  "deadline": "2026-08-01",       // "next month" hardened into a date
  "region": "EU",                 // inferred from the email domain
  "confidence": "high"            // confident about made-up values
}

// With an explicit policy and no inference, it returns:
{
  "name": "Dana Ruiz",
  "company": "Northwind",
  "requested_plan": "not_in_source",
  "budget": null,
  "deadline": "around next month (verbatim; no specific date given)",
  "region": "not_in_source",
  "missing_fields": ["requested_plan", "budget", "region", "phone"]
}

Where this fits in NewPrompt

Controlling missing data on one extraction is a prompt-and-review habit. When extraction is a recurring job, the AI Data Extraction Workflow sequences it into a repeatable process — bound the source, define the fields and the missing-data rule, force a strict shape, and validate before the data flows on. And when extraction is a stage in a larger system — turning unstructured documents into structured records at volume — the Build an AI Document Processing System with AI project has an extraction stage where handling missing values and ambiguity explicitly is the point, so honest gaps rather than a confident guess are built into the path. Across all of it, NewPrompt structures the prompts, the contract, and the review; you run them in your own AI assistant or API, check each value against the source, and decide what's trustworthy enough to use.

Tools for this guide

Each generates the prompt described above — you run it in your own AI assistant.

Tool

Extraction Prompt Generator

Build prompts that extract defined fields from unstructured text — emails, invoices, tickets, résumés.

Structured Output
Tool

AI Output Validator

Paste an AI's output and validate it against the expected format — with a repair prompt for every failure found.

Structured Output

Ready-made resources

Reusable prompts and templates for the exact steps in this guide.

Resource

Create Reliable JSON Responses

Getting JSON once is easy; getting the same JSON shape on run 500 is the real problem. The consistency mechanics: stable schema, null discipline, and type pinning.

Prompt Engineering
Resource

Hallucination Detection Prompt

Catch the confident invention — check an AI output's claims against its source and flag every statement that isn't supported, with the unsupported span quoted.

AI Agents

Take it further

When this task is one step inside a larger workflow or build.

Workflow

AI Data Extraction Workflow

Turn messy text into structured data you can trust enough to feed another system — bound the source, extract the fields, force clean JSON, and validate before it flows downstream.

4 steps 25–45 minutes
Project

Build an AI Document Processing System with AI

The full path to an AI document processing system — define the use case, design the intake pipeline, extract fields from unstructured documents, classify and route them, pin the output contract, evaluate accuracy, then ship it monitored.

7 stages AI Systems

FAQ

Can a prompt stop the AI from ever inventing missing data?

No prompt can fully stop it — that's why this is a policy plus a review, not a single instruction. An explicit missing-value marker, a no-inference rule, and an evidence requirement make invention far less likely, but you still check the output against the source before you rely on it. The goal is honest gaps you can see, not a guarantee the model never guesses.

Should the AI omit a field it can't fill, or mark it missing?

Mark it, don't omit it. An omitted key and a real gap look identical to whatever reads the output, so you can't tell a value the model forgot from one that genuinely isn't there. Keep every field present and set a missing marker — null, "unknown", or "not_in_source" — so a gap is visible and reviewable.

Does the AI Output Validator check whether a value is actually in my source?

No. It runs structure and format checks in your browser — is the field present, is the type right, is the shape valid — and it can flag a missing expected field, but it can't tell whether a filled value came from your source or was invented. That source check is yours: read each value against the text, or use an audit prompt that classifies each claim as supported or unsupported.

What's the difference between "unknown" and null for a missing field?

Use whichever your downstream code expects, but pick one and define it. Some teams use null for no value present and a string like "not_in_source" or "needs_review" to separate absent from ambiguous. The important part isn't the exact token — it's that every gap gets a defined, consistent marker instead of a guessed value or a dropped key.