How to Stop AI From Mixing Facts From Different Sources
Give AI several sources and it pours them into one bowl — attributing Source A's claim to Source B, averaging a real conflict into a smooth "customers generally think," and calling it consensus. Here's how to keep each source separate and attributed so the facts stay traceable to where they came from.
Get the Research Synthesis AssistantWhen three sources become one blurred opinion
You paste three customer interviews into the AI and ask what customers think. Back comes a clean line: "Customers generally found onboarding easy but wanted better reporting and clearer pricing." It reads like a finding. It isn't. Only one of the three interviews said onboarding was easy — another described a two-week setup where the data import failed twice. "Better reporting" flattens a real disagreement: one interview found reporting too limited, another found it strong. And "clearer pricing" came from a single person. The model took three distinct voices, poured them into one bowl, and served you the average as if it were the consensus.
That's the failure this guide is about. When you hand a model several sources, it tends to treat them as one undifferentiated pile of text — and once the boundaries between sources are gone, it can't tell which fact came from where, so it blends them: a claim from Source A gets attributed to Source B, two sources get merged into a made-up middle, a conflict gets smoothed into "all agree." The fix is to stop giving it a pile and start giving it separate, labeled sources — each with its own boundary, each claim traced back to where it came from. Keeping sources separate and attributed makes this mixing far less likely and the real conflicts visible. It does not verify that any source is right or guarantee every claim is attributed correctly — you still check the result.
Why AI blends sources together
The blending isn't carelessness — it's what a model does with a context that has no seams. Pasted end to end, three sources look like one long passage, and the model answers about the passage, not the sources. Five things pull it toward a smooth, unattributed blur:
- The sources arrive without identities. Concatenated into one block, nothing tells the model where Source A ends and Source B begins, so "which source said this" has no answer to carry forward.
- A tidy average beats a messy truth. Three partly-conflicting views resolve more cleanly into one "customers generally think" than into an honest "one said X, one said the opposite."
- Conflicts read as friction to remove. Faced with two sources that disagree, the model reaches for a reconciled sentence rather than reporting the disagreement — so "all sources agree" appears when one plainly doesn't.
- A single source's claim becomes a general finding. Something one interview mentioned gets stated as what "customers" think, with nothing marking it as a lone data point.
- Attribution can't be recovered downstream. If the model loses track of which source said what while reading, no amount of prompting the output afterward puts it back — the tracking has to survive from the input on.
Step 1: Give each source its own boundary and label
The blur starts at the input, so fix it there. Before the model reads anything, package each source as its own clearly-bounded, labeled block — Source A, Source B, Source C — so that "which source is this from" is answerable for every line. Separate, delimited sources are the one thing that lets attribution survive all the way to the synthesis.
The Long Input Formatter is the tool for the boundary itself: it wraps a pasted source verbatim between explicit start/end delimiters and adds [§N] section labels so any claim can cite where it sits. One honest detail: it packages one source at a time — the [§N] labels mark sections within that single block — so the way you get several distinct, attributable sources is to package each one as its own delimited block rather than expecting a single pass to sort them into Source A, B, and C for you. The Package Research Material resource is a worked example of the result: several sources placed in delimited, labeled sections under rules that say to keep each finding attributed to its own source, never blend findings into one claim, and — when sources disagree — report the disagreement with both citations instead of resolving it silently.
Step 2: Summarize each source on its own before you compare anything
The instinct is to ask for one combined summary of everything. Resist it — a combined summary is exactly where the blending happens. Instead, summarize each source on its own terms first, one at a time, so each source's actual claims stay intact before any comparison starts. A summary that only ever sees one source can't import a fact from another, which makes per-source summaries the clean parts a trustworthy synthesis is built from.
The Structured Summary Prompt builds a summary prompt you run once per source, with a strict fidelity setting that keeps the summary to what's actually in that source and writes "not covered in the source" for anything it doesn't address. Run it against each source separately and you get a set of faithful, comparable summaries — same shape, different content — with no cross-contamination, because nothing was ever asked to hold two sources at once. This is the step that keeps summary and synthesis apart: you summarize now, and you reconcile later.
Step 3: Attribute every claim to its source, and forbid blending
Only now does the model reconcile the sources — and only under a standing rule: every claim it states names the source it came from, and it may never merge two sources into an anonymous "customers think X." That rule has to be set as the session's behavior, not tacked onto one request. The System Prompt Generator builds that session: a synthesizer role with a must-do — trace every claim to its labeled source — and a must-not-do — never assert a claim you can't attribute, never blend sources into an unattributed blob.
You don't have to write that from scratch. The Research Synthesis Assistant is a ready-made version of exactly this session: a two-pass prompt that analyzes each source on its own first, then synthesizes across the set — points of agreement, contradictions with the conflicting sources named, and gaps none of them cover — under the hard rule "attribute every claim to its source; do not blend sources into anonymous statements." If you want the attribution machine-checkable rather than prose, the JSON Output Prompt Builder shapes each claim into a record — the claim, a source_id, a verbatim source quote, whether it's an agreement or a conflict, and a review flag — with a strict rule that an unknown source becomes null rather than a guessed one, giving the model an explicit null to reach for instead of pressure to fabricate an attribution just to fill the field.
Step 4: Separate agreement, conflict, and source-only — don't force consensus
A good multi-source synthesis sorts claims; it doesn't average them. Ask for the sort explicitly: where the sources genuinely agree (and name which ones), where they conflict (name both, keep both, flag it), and what only a single source says — a source-only claim, not a general finding dressed up as one. The distinction between "two sources confirm this" and "one source mentioned this" is most of the value, and it's the first thing a blended summary destroys.
Two rules make the difference. First, treat any "all sources agree" as a claim that needs proof: require the source-by-source evidence behind it, because a global agreement statement is exactly where the model manufactures a consensus that isn't there. Second, when sources conflict, don't let the model pick a winner — have it flag the conflict for review and leave both positions standing. Reconciling a genuine disagreement between sources is a judgment call for you or someone who knows the domain, not something the model should quietly resolve by choosing the more confident-sounding source.
Step 5: Know what attribution proves — and what it doesn't
Source labels make attribution reviewable, not correct. A model can put "Source B" on a claim Source B never made, quote the right words under the wrong label, or attribute cleanly and still be repeating a source that's simply wrong. A citation is a pointer to check, not a proof — so the last step is to read the claim-and-source pairs against the actual sources, leaning hardest on the flagged conflicts and any "all agree" line.
Be clear about the limits of the tooling here too. A structural validator can confirm an attribution report is well-formed — that every row carries a source_id and a quote, that the JSON parses — but it never sees your sources, so it can't tell you the source_id points to the right source or that the quote is real. The separation and labeling reduce mixing and make conflicts visible; they don't certify a single attribution or catch every conflict, and for anything carrying legal, financial, or compliance weight, the sign-off stays with you or a qualified reviewer.
Common mistakes
A few habits pour the sources back into one bowl:
- Pasting all the sources as one block. With no boundaries, the model reads one passage, not several sources, and attribution is lost before it starts.
- Asking for the combined summary first. Summarize everything at once and the blending is already done; summarize each source alone, then reconcile.
- Letting "generally," "most," or "all" stand unchecked. Global agreement words are where invented consensus hides — demand the source-by-source evidence behind them.
- Treating a source-only claim as a general finding. Before a single-source point ships as what "customers" concluded, make the output say how many sources actually back it.
- Forcing a conflict to one answer. When sources disagree, keep both and flag it; a model picking the more confident source is not a resolution.
- Trusting the label as proof. A source_id makes a claim checkable, not correct — a well-attributed claim can still be a misread or a wrong source.
A worked example: three customer interviews
Say you're pulling themes from three customer interviews. Here's the blended summary that invents a consensus, next to the source-separated read that keeps each fact with its source and flags the conflicts.
Three interviews: a blended summary that invents consensus vs a source-separated readTHREE SOURCES (customer interviews):
[Source A] "The onboarding was easy, but reporting felt too limited."
[Source B] "Setup took two weeks because the data import failed twice."
[Source C] "Reporting is strong, but the pricing page was confusing."
BLENDED SUMMARY (what you don't want):
"Customers generally found onboarding easy but wanted better reporting
and clearer pricing."
- "generally ... easy": only Source A said onboarding was easy; Source B
describes a two-week setup with a failed import. Not a consensus.
- "better reporting": Source A found reporting limited, Source C found it
strong. A conflict, flattened into agreement.
- "clearer pricing": only Source C raised pricing. A source-only claim,
stated as a general finding.
SOURCE-SEPARATED READ (what to ask for):
onboarding / setup A: easy | B: two-week setup, import failed twice CONFLICT REVIEW
reporting A: too limited | C: strong CONFLICT REVIEW
pricing C: pricing page confusing SOURCE-ONLY (C)
agreement across all three: none of the themes is confirmed by more than one source.
(every claim tagged with its source_id; conflicts kept and flagged, not averaged away.)Where this fits in NewPrompt
The AI Research Synthesis Workflow exists because this is the recurring version of the same problem — a stack of sources a model wants to reduce to soup. It sequences the same three moves you'd otherwise do by hand — separate and label, summarize each, then synthesize with every point traced to a source — into a pass you can repeat. When the synthesis is one piece of something larger, the Build an AI Research Assistant with AI project sets it on a full path where pulling each claim back to its source and naming the agreements and conflicts are stages in their own right, not an afterthought bolted on at the end. What none of it does is the reading. The prompts get built here; the model runs in your own assistant, and every attribution is yours to confirm — a claim sitting next to "Source B" is a pointer you still check against Source B, not a fact the platform stands behind.