How to Split Long Documents for AI Without Losing Context
A document too long for the model has to be split — but a blind split makes the AI forget earlier parts, drift on definitions, and lean on whatever it saw last. Here's how to split and synthesize without losing the thread.
Try the Long Prompt SplitterWhen a document is too long for the model
You have an 80-page vendor contract — or a research report, or a migration plan — and you want the AI to analyze it. You paste it and the model refuses because it's over the context limit, or worse, it quietly works from a truncated version and answers confidently about the half it actually read. So you split the document into chunks and feed them one at a time. Now a different failure shows up: the model has forgotten chunk 2 by the time it reaches chunk 9, a term defined early gets used loosely later, and the final summary leans on whatever it saw last.
The problem isn't the model being lazy — it's that each chunk arrives without the context the others established. Splitting a long document is easy; splitting it without losing the thread is the actual work. This guide is a path for doing that: plan the size, cut on real boundaries, package the parts so they read as one source, and collect findings in a shape that survives synthesis. It's a process, not a guarantee — you still run the prompts and check the final answer against the source yourself.
Why splitting loses context
A model doesn't carry memory across messages the way you carry it across pages. Each chunk is judged mostly on what's in front of it, so anything that lived in an earlier chunk is gone unless you carry it forward on purpose. Three things break when you split blind:
- Boundaries land mid-thought. A blind character cut splits a sentence, a table, or a clause from the definition it depends on, so neither chunk is complete.
- Definitions drift. A term set in section 2 is re-encountered in section 8 with no reminder, so the model re-interprets it — often differently than it was defined.
- The last chunk wins. Without a structured way to gather findings, the final answer over-weights the most recent chunk and under-weights everything before it.
One limit worth naming first
Splitting solves the per-message size limit — it lets you deliver a document too big to paste at once — but it does not expand the model's total context window. If the whole document plus its analysis can't share one conversation at all, chunk-by-chunk won't save it; you summarize each part and synthesize from the summaries, or move to a retrieval approach. Knowing which situation you're in is the real first step, so start by measuring, not cutting.
Step 1: Check whether it fits, and size the chunks from that
Before you cut anything, find out what you're dealing with. Estimate the document's size against your model's context window, leaving room for the response, so you know whether this is a splitting job or a summarize-then-synthesize job.
The Context Window Estimator does this in your browser: paste the content, pick the model, reserve space for the answer, and it returns a token-range estimate and a fit verdict from Safe through Will Not Fit. Treat the figure as an estimate, not a measurement — your provider's tokenizer is authoritative. Use it to choose a chunk size that leaves headroom, not to prove an exact count.
Step 2: Map the document before you cut it
A blind split has no idea what matters; a map does. Skim the document into an outline first — the sections, the defined terms, the decisions or requirements, and where each one lives. It takes a few minutes and it changes every later step: you cut on the outline's seams instead of arbitrary offsets, and you know which definitions have to travel with which chunks.
The map is also what you hand the model alongside each chunk so it always knows where the current part sits in the whole. A chunk that says it is section 4 of 11, and that sections 2 and 7 define the terms it uses, is far harder to misread than a nameless block of text.
Step 3: Split on real boundaries, not character counts
Cut where the document already breaks — sections, headings, paragraphs, speaker turns — not at a fixed character offset. A boundary that respects structure keeps each clause with its context and each table intact; a blind cut is where half your context loss comes from.
The Long Prompt Splitter does this in your browser: you paste the text, pick a chunk size, and it splits at meaningful boundaries in preference order — section, then paragraph, then line, with a hard character cut only as a counted last resort — and wraps each part in "chunk i of N" markers that tell the model to wait, acknowledge, and hold its analysis until every part has arrived. The chunks are verbatim slices of your text; it doesn't send them anywhere, summarize them, or read them for you. For section-aware splitting specifically, the Chunk Long Documents resource is a worked example that starts each chunk at a heading and tells the model to answer by the document's original structure.
Step 4: Package the parts so they read as one source
Ordered chunks still aren't one document until you make them addressable. Wrap the source in clear delimiters and label each section so the model can cite a specific part instead of guessing, and state the grounding rule — answer from the source, and say so when the source doesn't cover it — outside the content itself.
The Package Long Documents for AI resource is a worked example of exactly this: delimited source, numbered section labels, and grounding rules that keep answers anchored to the text. To carry state across the parts as they arrive — the numbering, the wait-and-acknowledge rules, the all-parts-received signal — the Preserve Context Across Chunks resource shows the continuity contract that stops the model answering early or dropping a part. Both are patterns you paste and adapt; they structure how the parts hang together, they don't hold the context for you.
Step 5: Collect findings with one shared schema
If each chunk comes back in a different shape, the final synthesis is weak — the model can't line up what doesn't line up. Give every chunk the same output schema: the key points anchored to their section, the defined terms, the decisions or obligations, and — kept separate — anything uncertain or missing. Same fields, every chunk, so merging them is mechanical instead of guesswork.
The Summarize a Long Document resource is a summary prompt template with fidelity rules — compress only what's actually there, keep numbers, dates, and names exact — that you adapt into your per-chunk schema. Keeping the uncertain-or-missing field as its own line is what stops the model quietly filling gaps in the final pass.
Step 6: Reconcile and check before you trust the synthesis
The final answer is a merge, not a fresh read. Before you rely on it, run a reconciliation pass: does any term get used differently across sections, are there contradictions between chunks, and did the model keep the uncertain items open instead of inventing answers for them? Ask it plainly to list what it could not confirm from the source.
This is the step that catches the failure the whole process is trying to prevent — a confident summary that smoothed over a gap. The structure reduces context loss; it can't prove nothing was missed, so the last check is yours: spot-read the sections the answer leans on and confirm the citations point where they claim.
Common mistakes
A few habits reintroduce the context loss you split to avoid:
- Cutting at a fixed character count. It splits sentences, tables, and clauses from their definitions — the single biggest source of lost context.
- Not telling each chunk where it sits. A part with no map is a part the model can misread in isolation.
- Letting the model answer before all the chunks arrive. It will summarize what it has and treat it as the whole document.
- Mixing summary formats across chunks. Different shapes don't merge, so the synthesis leans on whichever chunk happened to be clearest.
- Not tracking uncertainty. If "missing or unclear" isn't its own field, the model fills the gap in the final answer and you can't tell what was real.
A worked example: an 80-page vendor contract
Say you're analyzing an 80-page vendor contract. It's over the window, so you split it — and these are the risks the split introduces if you're not careful:
- "Service credit" is defined in section 2 and used with a narrower scope in the payment terms 40 pages later.
- The termination clause references an SLA defined in an earlier chunk the model no longer has in front of it.
- The final summary over-weights the indemnity section because it was in the last chunk.
- Each chunk came back in a different format, so the merge is uneven and some obligations get lost.
- An ambiguous liability cap got a confident answer instead of being flagged as unclear.
For each chunk, return exactly this and nothing else:
- Chunk: <i of N> | Sections: <headings covered>
- Key points: <each anchored to the section it comes from>
- Defined terms: <term -> definition, with the section it is defined in>
- Decisions / obligations: <what must happen, who, by when>
- Open / uncertain: <anything ambiguous or not stated — do not guess>
Final synthesis (run only after every chunk summary exists):
Merge the chunk summaries into one answer. Flag any term used
differently across sections, list unresolved contradictions, and keep
the Open / uncertain items open — do not fill them in.Where this fits in NewPrompt
Splitting one document by hand is fine for a one-off. When reading documents too big for a single prompt is a recurring job, the AI Long Document Analysis Workflow sequences these moves into a repeatable pass — check the fit, split on real boundaries, package the parts as one source, then analyze and stitch the findings together. And when the document is one input to a larger effort — reading across a whole corpus to produce grounded, cited findings — the Build an AI Research Assistant with AI project has a source-organizing stage where this splitting work belongs, so it's part of the analysis path rather than a detour. Across all of it, NewPrompt structures the splitting, the packaging, and the prompts; you run them in your own AI assistant or API, do the final synthesis, and confirm the answer against the source yourself.