Fix AI Format Drift
Run 40 looks nothing like run 1: sections reordered, one came back empty. Detect the drift, repair the run, keep the pipeline honest.
Overview
Format drift is the recurring-workflow disease: the same prompt that produced perfect output in week one starts reordering sections and leaving them empty by week six. This setup validates a drifted incident summary — Timeline and Root Cause swapped, Impact present but empty — the two drift signatures that presence-only checks miss completely. The repair prompt fixes the specific run; the validation history tells you when the underlying prompt needs a strictness upgrade.
Workflow
-
Validate a sample of runs
Drift is statistical — one validation catches the run, a pattern of warnings catches the trend.
-
Note the drift signatures
Reordered sections and empty-but-present sections are drift; missing sections are breakage. Different fixes.
-
Repair the run, harden the prompt
The repair prompt fixes today's output; recurring warnings mean the generator prompt needs Strict consistency.
Why This Works
- Order and emptiness checks catch the drift presence-only QA misses
- Per-run repair keeps the pipeline flowing while you fix the root cause
- Validation history turns anecdotal "it got worse" into evidence
Best for
- Recurring workflows running the same prompt for months
- Outputs consumed by position-sensitive readers or parsers
- Teams that QA samples rather than every run
Not for
- Preventing drift at the source — strictness levels in the generator tools do that
- Comparing two prompt versions — that's the Prompt Version Diff
Use cases
- Spot-checking recurring outputs for structural drift
- Catching empty sections that presence checks pass
- Deciding when a prompt needs stricter consistency rules