Engineering

Investigate Runtime Errors

The error only happens for some inputs: find the implicit contract those inputs violate, with the runtime checklist as the suspect list.

Overview

Runtime errors that hit only some inputs are contract violations wearing exception costumes: somewhere, the code assumes a shape, an order, or a presence that those inputs don't deliver. This setup investigates an InvalidOperationException that fails only for certain tenants: the runtime strategy traces the failing path, the checklist walks the suspects — null paths, dependency failures, invalid state, configuration, environment drift, unchecked input contracts — and the framework demands the divergence point: what is different about the inputs that fail?

Workflow

Characterize the failing inputs
The investigation's first real question: what do the failing cases share that the passing ones lack?
Name the violated assumption
"Tenant created mid-month has no complete billing period" — the contract the code assumed but never checked.
Fix the contract, then the code
Decide whether the assumption should be enforced, relaxed, or handled — then the change writes itself.

Why This Works

Input characterization converts a mystery into a diff between datasets
The checklist keeps boring causes (config, drift) in view next to glamorous ones
Contract naming produces fixes that survive the next unusual input

Best for

Multi-tenant systems with data-shaped failures
Batch jobs that fail on specific records
Errors that survived one "fix" already

Not for

Errors that strike randomly on identical input — that's the intermittent strategy
Performance degradation without errors — that's the performance strategy

Use cases

Errors that hit some tenants, users, or files and not others
Exceptions whose message names the symptom, not the assumption
"Sequence contains no elements" and its empty-collection cousins

Customize This Resource

Opens this setup in Debugging Prompt Generator. Generate to get the full investigation contract — then adjust the problem type, mode, and environment.

Open in Debugging Prompt Generator

Prompt Template

DEBUGGING OBJECTIVE
Investigate why the report generator throws InvalidOperationException on some inputs.
Problem type: runtime error — an exception, crash, or stack trace marks the failure.
Determine what is actually happening. Do not jump directly to a solution before identifying the most likely root cause.

SYSTEM CONTEXT
Environment: shared development — check for collisions with other developers' changes and shared test data.
Reported severity: Medium.
Investigation strategy for this problem type:
- Locate the exact failure point: the line, the call, and the state at the moment of failure.
- Trace the execution path backward from the failure to its trigger — the throw site is rarely the cause site.
- Distinguish the proximate cause (what threw) from the root trigger (what made it throw).

KNOWN FACTS
Symptoms: Report generation fails for a subset of monthly reports.
Expected behavior: Every scheduled report renders to PDF.
Actual behavior: InvalidOperationException: "Sequence contains no elements" for some tenants.
Reproduction steps:
- Generate the monthly report for a tenant created mid-month
Logs / errors: none provided.

INVESTIGATION FRAMEWORK
Structure the investigation in exactly these stages:
1. SYMPTOMS — restate the observable symptoms, separating observations from interpretations.
2. KNOWN FACTS — list what is directly evidenced, each fact with its source.
3. REPRODUCTION STEPS — verify the provided steps reproduce the problem; reduce them to the minimal failing case.
4. POSSIBLE CAUSES — candidate causes, drawn from the checklist below and from the evidence.
5. ROOT CAUSE ANALYSIS — the most likely cause and the alternatives, per the root cause rules.
6. EVIDENCE REVIEW — which evidence is solid, which is weak, which is missing, and what contradicts what.
7. MISSING INFORMATION — what you need that you do not have. Ask for it; do not guess it.
8. VALIDATION PLAN — the tests or observations that will confirm or eliminate the leading cause.
9. FIX RECOMMENDATION — per the fix requirements below.
10. POST-FIX VERIFICATION — per the verification requirements below.

DEBUGGING CHECKLIST
Candidate causes to check for this problem type:
1. Null/undefined references on the failure path
2. Dependency failures (network, database, filesystem) surfacing as exceptions
3. Invalid state reached through an unexpected order of operations
4. Configuration differences from the environment where it works
5. Environment drift: versions, locales, time zones, OS differences
6. Input that violates an implicit contract the code never checks

ROOT CAUSE RULES
- Identify the MOST LIKELY root cause — and at least two ALTERNATIVE causes.
- For each candidate cause: list the evidence supporting it AND the evidence contradicting it.
- Define a validation step for each candidate: what observation would confirm or eliminate it.
- The symptom site and the cause site may be different places — finding where it hurts is not finding why.

EVIDENCE RULES
Label every statement as exactly one of:
- FACT: directly observed. Example: "The application returned HTTP 500."
- ASSUMPTION: believed without direct evidence. Example: "The database is probably down." Assumptions must be verified or discarded — never silently promoted to facts.
- HYPOTHESIS: a testable causal claim. Example: "A failed database connection may be causing the 500 response." Every hypothesis comes with its test.
- Prefer evidence over inference; where evidence is missing, label the assumption explicitly.
- Carry at least two hypotheses until evidence separates them.

VALIDATION REQUIREMENTS
- Before proposing any fix, state how the diagnosis was validated — or what validation is still pending.
- The gold standard: a reproduction that turns the failure on and off by toggling the suspected cause.
- If the root cause cannot be validated with the available information, say so and list what is needed.

FIX REQUIREMENTS
- Any fix proposal must include: the likely cause, the supporting evidence, the proposed change, and how to verify it worked.
- Frame it as "here is why this is likely the fix" — never an unexplained patch.
- Prefer the smallest fix that addresses the root cause; label symptom-only fixes as temporary mitigations.

POST-FIX VERIFICATION
- Define what to observe after the fix: the original reproduction passing, error rates at baseline, the affected metric recovered.
- Check for regressions in the paths the fix touched.
- State how long to observe before declaring the problem resolved — intermittent problems need longer windows.

Related Resources

Explore more resources that work well with this one.