Engineering

Investigate Intermittent Bugs

It fails once in twenty runs: the occurrence pattern IS the evidence. Hunt the difference between failing and passing runs — forensically.

Open in Debugging Prompt Generator

Overview

Intermittent bugs gaslight engineers: every look at the code says it's fine, and one run in twenty says otherwise. This setup investigates a flaky CI test under forensic mode: occurrence patterns treated as primary evidence (when it fails and when it doesn't is the strongest signal available), the difference hunted between failing and passing runs — timing, load, data, environment — and concurrency suspected first, per the checklist: races on shared state, timing dependencies, retry behavior, stale state, cache inconsistencies. Forensic mode matters here most: with evidence this thin, guesses are expensive.

How to use this resource

Log the pattern before the code

Failure rate, time of day, parallelism, data — the occurrence pattern is the dataset; collect it first.
Diff the runs

What is different about run 17? CI parallelism, shared state, timing — the difference IS the lead.
Demand a longer verification window

The contract's post-fix rule: intermittent problems need extended observation — one green run proves nothing.

Why This Works

Pattern-as-evidence framing extracts signal from a bug that hides from direct observation
The concurrency-first checklist starts where sometimes-bugs usually live
Forensic rules prevent expensive guesses when evidence is thinnest

Best for

Flaky tests that pass locally and fail in CI
Bugs that vanish when observed (and return after the standup)
Race conditions suspected but never caught

Not for

Deterministic failures — the standard strategies diagnose them faster
Fixing flaky tests by rewriting them — pin the cause first; the Test Case Prompt Generator's discipline prevents the next one

Use cases

Hunting the test that fails 5% of CI runs
Diffing failing runs against passing ones systematically
Catching the race that only fires under CI parallelism

FAQ

Why does this debugging prompt treat when the test fails as evidence instead of just reading the code?

Because the promptTemplate fixes 'Problem type: intermittent' and declares the occurrence pattern the strongest signal available, so it instructs hunting the difference between failing and passing runs across timing, load, data, and environment, and suspecting time and concurrency first per the 5-item checklist. With one failure in twenty, code-reading alone yields nothing to diff.

How do I prove the fix actually worked if the bug only shows up 5% of the time?

The template's VALIDATION REQUIREMENTS name the gold standard as a reproduction that toggles the failure on and off by toggling the suspected cause, and POST-FIX VERIFICATION demands a longer observation window because one green run proves nothing for intermittent problems. You run that validation yourself in your assistant; NewPrompt does not confirm the root cause or watch the CI window for you.

Will the debugging-prompt-generator find the root cause of my flaky OrderFlowTests failure for me?

It generates a structured diagnosis prompt built on the 10-stage framework and FACT/ASSUMPTION/HYPOTHESIS evidence labels, which you copy and run in your own ChatGPT, Claude, or Gemini. NewPrompt does not execute the investigation or guarantee the cause; the template even forbids assumptions in conclusions and requires each claim cite specific evidence, so you own the final diagnosis and fix.

Customize This Resource

Opens this setup in Debugging Prompt Generator. Generate to get the full investigation contract — then adjust the problem type, mode, and environment.

Open in Debugging Prompt Generator

Prompt Template

Copy it as-is, or use Open in Debugging Prompt Generator to load it pre-filled and customize it with your own context.

DEBUGGING OBJECTIVE
Pin down the test that fails roughly once in twenty CI runs.
Problem type: intermittent — it does not reproduce consistently; the occurrence pattern is the evidence.
Determine what is actually happening. Do not jump directly to a solution before identifying the most likely root cause.

SYSTEM CONTEXT
Environment: staging — verify parity with production configuration; any difference from production is itself a suspect.
Reported severity: Medium.
Investigation strategy for this problem type:
- Treat occurrence patterns as primary evidence: when it happens and when it does not is the strongest signal available.
- Hunt the difference between failing and passing runs: timing, load, data, environment.
- Suspect time and concurrency first — races, retries, and timing dependencies are the usual suspects for sometimes-bugs.

KNOWN FACTS
Symptoms: OrderFlowTests.CompletesCheckout fails about 5% of CI runs; it always passes locally.
Expected behavior: The test passes deterministically.
Actual behavior: Sporadic timeout waiting for the confirmation element.
Reproduction steps:
- Re-run the CI suite; the failure is not tied to any code change
Logs / errors: none provided.

INVESTIGATION FRAMEWORK
Structure the investigation in exactly these stages:
1. SYMPTOMS — restate the observable symptoms, separating observations from interpretations.
2. KNOWN FACTS — list what is directly evidenced, each fact with its source.
3. REPRODUCTION STEPS — verify the provided steps reproduce the problem; reduce them to the minimal failing case.
4. POSSIBLE CAUSES — candidate causes, drawn from the checklist below and from the evidence.
5. ROOT CAUSE ANALYSIS — the most likely cause and the alternatives, per the root cause rules.
6. EVIDENCE REVIEW — which evidence is solid, which is weak, which is missing, and what contradicts what.
7. MISSING INFORMATION — what you need that you do not have. Ask for it; do not guess it.
8. VALIDATION PLAN — the tests or observations that will confirm or eliminate the leading cause.
9. FIX RECOMMENDATION — per the fix requirements below.
10. POST-FIX VERIFICATION — per the verification requirements below.

DEBUGGING CHECKLIST
Candidate causes to check for this problem type:
1. Race conditions on shared state
2. Timing dependencies: timeouts, debounce, clock skew
3. Retry behavior masking or causing failures
4. Stale state surviving between runs
5. Cache inconsistencies between layers

ROOT CAUSE RULES
- Identify the MOST LIKELY root cause — and at least two ALTERNATIVE causes.
- For each candidate cause: list the evidence supporting it AND the evidence contradicting it.
- Define a validation step for each candidate: what observation would confirm or eliminate it.
- The symptom site and the cause site may be different places — finding where it hurts is not finding why.

EVIDENCE RULES
Label every statement as exactly one of:
- FACT: directly observed. Example: "The application returned HTTP 500."
- ASSUMPTION: believed without direct evidence. Example: "The database is probably down." Assumptions must be verified or discarded — never silently promoted to facts.
- HYPOTHESIS: a testable causal claim. Example: "A failed database connection may be causing the 500 response." Every hypothesis comes with its test.
- Every conclusion must cite the specific evidence that supports it — quote the log line, the stack frame, the reproduction result.
- Unknowns remain unknown: write "insufficient evidence" rather than a plausible guess.
- No unsupported certainty: every claim carries its confidence level and its evidence.
- Treat the absence of expected evidence as a finding in itself.
- Assumptions are forbidden in conclusions — they may appear only in MISSING INFORMATION, as questions to resolve.

VALIDATION REQUIREMENTS
- Before proposing any fix, state how the diagnosis was validated — or what validation is still pending.
- The gold standard: a reproduction that turns the failure on and off by toggling the suspected cause.
- If the root cause cannot be validated with the available information, say so and list what is needed.

FIX REQUIREMENTS
- Any fix proposal must include: the likely cause, the supporting evidence, the proposed change, and how to verify it worked.
- Frame it as "here is why this is likely the fix" — never an unexplained patch.
- Prefer the smallest fix that addresses the root cause; label symptom-only fixes as temporary mitigations.

POST-FIX VERIFICATION
- Define what to observe after the fix: the original reproduction passing, error rates at baseline, the affected metric recovered.
- Check for regressions in the paths the fix touched.
- State how long to observe before declaring the problem resolved — intermittent problems need longer windows.

More resources from Debugging Prompt Generator

Resource

Debugging Prompt — the Investigation Contract

"Fix this error" gets guesses. The investigation contract gets a ten-stage diagnosis: facts separated from assumptions, alternatives weighed, fixes justified.

Prompt Engineering

Resource

Error Analysis Prompt

An exception is a symptom, not a diagnosis: trace from the throw site back to the root trigger, with the runtime checklist on the table.

Engineering

Resource

Debug Performance Problems

Measure before reasoning: find WHERE the time goes, separate latency from memory from throughput, and think at production scale.

Engineering

Resources that pair well

Resource

Code Review Prompt — the Review Contract

"Review this code" gets shallow comments. The review contract gets findings with severities, a checklist, and a verdict.

Prompt Engineering

Resource

Playwright Test Prompt

getByRole over CSS chains, auto-wait over sleep, web-first assertions — Playwright tests written the way Playwright wants.

Engineering

Resource

Refactor Prompt — the Behavior Preservation Contract

"Refactor this code" invites silent behavior changes. The refactoring contract preserves business rules, outputs, and side effects — and flags uncertainty instead of deciding it.

Prompt Engineering

Related tools

Tool

Debugging Prompt Generator

Build debugging prompts from symptoms, environment, and reproduction steps — root cause first, fix second.

Coding Workflows

Guides for this resource

Guide

How to Reproduce a Bug With AI

You click the thing twenty times, it works every time, and you close the ticket "cannot reproduce" — while it keeps happening to users. Here is how to reproduce a bug with AI: turn a vague report into conditions that make the failure appear and disappear on demand, without inventing the ones you were not given.

Software Development with AI

Tip: Save time by exploring related resources and tools that integrate with this resource.