Catch Invalid AI Labels
The model answered "Complaints" — your set says "Complaint". One character of drift, one broken dashboard. Caught before it counts.
Overview
Invalid labels poison aggregations silently: "Complaints" instead of "Complaint" creates a new category with a count of one, dashboards fragment, and nobody notices until the quarterly review. This setup validates a feedback classification that returned a near-miss label outside the defined set — the plural drift that exact string matching downstream will treat as a brand-new category. The validator flags it as a fail with the full allowed set in the repair, and the fix costs one message instead of one data-cleaning sprint.
Workflow
-
Validate at the gate
Between the model and the database — the only place a label fix is one message cheap.
-
Watch for near-misses
"Complaints" vs "Complaint" is the signature drift: too close for a human to flag, far enough to fragment data.
-
Repair with the full set
The repair prompt lists every allowed label — the model picks the right one with the set in view.
Why This Works
- Set-membership validation matches how aggregation actually breaks
- Near-miss labels are exactly what human review approves without noticing
- A one-message repair beats retroactive data cleaning every time
Best for
- Feedback pipelines aggregating labels over time
- Dashboards that fragment on novel label strings
- Anyone who has run SELECT DISTINCT label and wept
Not for
- Improving the label definitions so drift stops — that's the Data Classification Prompt
- Multi-label ordering checks — this validates set membership and format
Use cases
- Gating labels before they enter counts and dashboards
- Catching plural/singular and synonym drift
- Keeping category sets closed in long-running pipelines