How to Reproduce a Bug With AI
You click the thing twenty times, it works every time, and you close the ticket "cannot reproduce" — while it keeps happening to users. Here is how to reproduce a bug with AI: turn a vague report into conditions that make the failure appear and disappear on demand, without inventing the ones you were not given.
Get the Bug Reproduction Prompt"Cannot reproduce" is not the same as "no bug"
A ticket comes in: sometimes, after an upload, the status is wrong. You open the feature, do the thing the reporter described, and it works. You do it again — works. You do it a dozen times, it works every time, and you close the ticket "cannot reproduce," which feels honest because you genuinely couldn't. Meanwhile it keeps happening to users, because "I could not make it happen" and "it does not happen" are different sentences, and the gap between them is where real bugs go to hide.
That gap is the whole problem this guide is about. A bug you cannot reproduce is a bug you cannot debug with any confidence and cannot confirm you fixed — every diagnosis is a guess and every fix is a hope, because you have no way to make the failure appear and then watch it stop. Reproduction is the step that turns "sometimes it breaks" into "it breaks exactly when these conditions hold," and that turn is worth more than it looks: it is the difference between chasing a ghost and cornering it.
It is also a different job from the ones next to it. It is not reading a stack trace, not finding the root cause, and not writing the test that guards the fix — it is the narrower, earlier task of finding the conditions under which the failure reliably happens, so that everything after it has something solid to stand on. An AI can help structure that search, but only from what you actually observed, and the discipline is refusing to let it fill the conditions you did not give it with plausible ones it made up.
What the AI helps with, and what only your system can show
The reproduction plan the AI produces is built from what you hand it: the report, the steps as given, the environment you can describe, the inputs, the logs, and whatever variations you have already noticed. From those it can structure a search — a control case, the variables worth changing, the order to change them in. What it cannot do is run any of it. It reaches no application, no browser, no device, no logs, and no database; it runs no code and reproduces no failure; it creates no environment and no account state; it changes no timing, no concurrency, and no load; and it measures no frequency, because measuring means running, and running is yours.
That boundary is exactly why the plan is a candidate and not a result. The AI can propose that timing might matter; it cannot tell you it does, because it never watched the failure happen. Every "it reproduces 7 times in 10" in a good plan came from you running it 10 times and counting — if a number like that appears without you having run anything, the model invented it, and an invented frequency is worse than none because it looks like evidence. You run the steps against the real system, you record what actually happens, and there is no guarantee the plan reproduces the bug, finds the cause, or confirms the fix.
So the method leans entirely on that split. The AI is good at the part that is structure — what to hold constant, what to vary, what a result would mean — and useless at the part that is fact, because the facts only exist once the code runs. Keeping those apart is what keeps a reproduction plan honest instead of confident.
Step 1: Pin the failure signature before you chase it
Before hunting conditions, define exactly what a hit looks like, because "the status is wrong" is too vague to recognize when you have caught it. The failure signature is the specific, observable thing that distinguishes a failure from a success: a particular wrong value on screen, a specific error type or message, a status code, a missing action, an incorrect state transition — precise enough that you can look at a run and say yes or no without interpreting. "The second tab shows Processing for a record whose stored status is Ready" is a signature; "sometimes it's wrong" is not.
Getting the signature exact does two things at once. It stops you from counting near-misses as hits or dismissing real hits as flukes, which is how an intermittent bug gets called "cannot reproduce" — you were reproducing it and did not recognize it. And it forces a first separation the report usually blurs: what was actually observed versus what the reporter believed. "Shows the old status" is a report; "shows Processing after the record flipped to Ready" is what you see when you look, and only the second is something you can test against.
Keep the signature free of anything sensitive. A correlation id, a log marker, or a specific value can sharpen it, but the record id, the user, and the payload get redacted before any of it is shared — the signature is what makes a hit recognizable, not a place to carry data that should not travel.
- Define the exact observable that separates a hit from a miss — a value, an error, a status, a state transition.
- A vague signature is how an intermittent bug gets closed "cannot reproduce" — you hit it and didn't know.
- Separate what was observed from what was reported; only the observed thing is testable.
- Redact ids, users, and payloads from the signature — sharpen it with a marker, not with sensitive data.
Step 2: Rebuild the reported conditions exactly as given
Start by trying to reproduce the failure under the conditions the report actually describes — no more, no less. Assemble what you were told: the steps, the input, the environment, the account or permission state, the sequence, and whatever preconditions were mentioned. The goal here is fidelity, not cleverness; the reproduction search only works if the first attempt is a faithful rebuild of what the reporter did, so that when it fails or succeeds you know which conditions produced that.
This is the step where an AI will quietly help you lose, because a report is always incomplete and the model will fill the holes with plausible specifics — a browser version nobody mentioned, an account state nobody described, a timing the report never gave. Those inventions feel harmless and they poison the search, because now you are reproducing a bug under conditions the reporter never had. Every gap in the report is an unknown to mark as unknown, not a blank to fill: the environment you cannot confirm, the exact timing you were not told, the frequency you have not yet measured all stay explicitly open.
Where the report is genuinely silent on something that might matter, that silence is itself a finding to carry forward, not a decision to make on the reporter's behalf. "The report does not say which browser" is the correct content, and it becomes one of the variables you will test deliberately in Step 4 rather than a value you guess at now.
- Rebuild the reported conditions faithfully first — the search only works if attempt one matches what the reporter did.
- Every gap in the report is an unknown, not a blank to fill. An invented browser or timing reproduces a different bug.
- Mark environment, timing, and frequency you were not given as explicitly open — do not let the model supply them.
- A silence in the report is a variable to test later, not a value to decide now.
Step 3: Run a control, then change one variable at a time
A failing run means nothing without a passing one to compare it against, so establish a control: the closest version of the scenario that is supposed to work, run under the same conditions. If the failing case is "open the record in a second tab," the control is "open it in the same tab" or "refresh after"; the control is what tells you that the thing you changed is the thing that matters, rather than something in the environment you did not notice.
Then move one variable at a time, and only one. This is the rule the whole search rests on, because if you change the tab and the timing and the input together and the failure appears, you have learned nothing about which of the three caused it. Change the timing alone; run it; record the result. Change the tab alone; run it; record. A reproduction matrix — the conditions down one side, the result of each across from it — is just the disciplined form of this, and it is where the pattern that a single run hides becomes visible.
Record every result, including the ones that pass, because the passes are evidence too. A negative result narrows the search — it tells you a condition is not necessary — but it rarely eliminates a hypothesis outright, and treating it as if it does is a common way to talk yourself out of the real cause. "It did not fail when I waited five seconds" narrows toward timing; it does not prove there is no caching involved, because waiting changes more than one thing at once. Narrow with negatives; do not close with them.
- Establish a control — the nearest passing case under the same conditions — or a failure proves nothing.
- Change exactly one variable per run. Change three at once and a failure tells you nothing about which mattered.
- A matrix of conditions and results makes the pattern a single run hides visible.
- Record passes as evidence; a negative narrows the search but rarely eliminates a hypothesis on its own.
Step 4: Narrow to a minimal failing case — and treat intermittent as its own problem
With the matrix showing which conditions matter, reduce to the smallest combination that still fails — the least input, state, sequence, and environment that reliably produces the signature. Minimal does not mean fewest lines of code; it means nothing left in that you can remove without the failure going away. And there is a hard limit on the reduction: the moment you simplify away the thing that actually causes the failure, you no longer have a reproduction of the real bug, you have a different, tamer bug that happens to look similar. A mock that removes the real dependency, a simplified input that drops the field that mattered — those change the conditions, and if they make the failure vanish, that is a finding about the cause, not a cleaner repro.
Intermittent failures need their own honesty, because they break the instinct that one run settles anything. One success does not mean fixed and one failure does not mean always; the evidence is the pattern across many runs, so you record the count — seven failures in ten attempts under one condition — and treat that number as the finding, never a single lucky or unlucky run. Timing, concurrency, ordering, and load are separate variables to vary deliberately, because "intermittent" usually means one of them is the hidden condition you have not pinned yet.
Two traps are specific to intermittent bugs and worth naming. A retry or a refresh can hide the failure rather than fix it — if the user who refreshes never sees the bug, the bug is underreported, not absent, and a reproduction that always refreshes will conclude "cannot reproduce" while the bug ships. And a failure that will not appear in your environment may still be real in production, because the test environment differs in timing, data, and load; "cannot reproduce here" is a fact about here, not a verdict on the bug.
- Minimal = nothing removable without the failure vanishing — not the fewest lines. Simplify away the cause and it is a different bug.
- A mock or a simplified input that removes the failure changed the conditions — note the difference, do not bury it.
- For intermittent bugs the count across runs is the evidence: "7 of 10," never a single run. Vary timing, concurrency, ordering, and load separately.
- A retry or refresh can hide a bug, not fix it; and "cannot reproduce here" is about your environment, not proof the bug is gone.
Step 5: Package the reproduction and hand it off — without naming a cause
A reproduction is finished when someone else can follow it and get the failure, so package it that way: the minimal failing sequence, the environment it needs, the exact signature to watch for, the observation points that show it happening, and the frequency you actually measured. That package is the deliverable — a way to make the failure appear on demand — and it is worth being precise that this is all it is. Reproducing a bug strengthens some hypotheses and weakens others, but it does not prove a cause; "reproduces" and "explains" are different claims, and a reproduction plan that ends by declaring the root cause has quietly done a different, less careful job than the one it was for.
So the handoff names the next check, not the answer. The single most discriminating test — the one observation that would separate the surviving hypotheses — is the bridge to debugging, where finding and confirming the cause is the actual work. That is a different task with its own guide, and the reproduction's job is to hand it a failure it can turn on and off, plus the evidence gathered so far, not a conclusion it has to un-believe later.
Downstream there is one more handoff worth marking: once the cause is known and the fix is in, the reproduction becomes the basis for a regression test — the permanent guard that fails before the fix and passes after. Writing that test is its own step and its own guide, and an intermittent reproduction usually needs a deterministic trigger before it can become a reliable test at all. Note the fix-validation condition while it is fresh: after a fix, the failing case goes to zero across runs, the control still passes, and — the check people skip — a case that should still fail still does, so the fix solved the bug rather than flattening the feature.
- Package the minimal sequence, environment, signature, observation points, and measured frequency — a failure someone else can trigger.
- Reproduction strengthens hypotheses; it does not prove a cause. "Reproduces" is not "explains" — do not end on a root cause.
- Hand debugging the next discriminating check and the evidence, not a conclusion.
- The reproduction becomes a regression test later (a different job); note the fix-validation condition, including what should still fail.
Common mistakes
The ways a reproduction attempt goes wrong, most of them by confusing "I could not make it happen" with something stronger:
- Reading "cannot reproduce" as "no bug." It means you have not found the conditions yet, not that the conditions do not exist.
- Filling the report's gaps with invented specifics. A browser, an account state, or a timing the reporter never gave reproduces a different bug under conditions they never had.
- Declaring a root cause because it reproduced. Reproduction narrows hypotheses; it does not prove one — "reproduces" is not "explains."
- Changing several variables at once. If the failure appears after you changed the tab, the timing, and the input together, you have learned nothing about which caused it.
- Concluding from a single run. One pass is not "fixed" and one fail is not "always" — for intermittent bugs the count across runs is the evidence.
- Letting a refresh or retry hide the failure. If the reproduction always refreshes, it will report "cannot reproduce" while the bug keeps shipping to users who don't.
- Over-minimizing until the bug is gone. A mock or a trimmed input that removes the real condition gives you a tamer look-alike, not the bug you were chasing.
A worked example: a status that goes stale in a second tab
Here is a vague, intermittent report worked into a reproduction — a web app where, after an upload, opening the same record in a second tab sometimes shows the old status. It is a good example precisely because so little is known at the start: the timing window, whether a cache is involved, and the root cause are all unknown, and the discipline is getting to a reliable reproduction without pretending otherwise.
Watch what the run counts do and do not license. The matrix reports counts from runs actually performed on this fictional scenario — seven failures in ten attempts under the failing condition, zero in the control — and every number is something the reproduction produced, not a figure assumed up front. The three hypotheses each carry evidence for and against, the reproduction strengthens one without proving it, and the example ends by handing debugging the single check that would settle it — not a root cause it has no standing to declare.
A vague intermittent report worked to a minimal failing case: a control, a one-variable-at-a-time matrix with real run counts, three hypotheses each with evidence for and against, and a handoff to debugging — with no root cause declaredTHE REPORT (as it arrived, nothing added):
"After I upload a document and then open the same record in a second
tab, sometimes the second tab still shows the old status. Not every
time."
OBSERVED vs REPORTED (kept apart):
reported (unverified) : "shows the old status" / "not every time"
observed (once I look): the second tab shows "Processing" after the
record has flipped to "Ready" elsewhere
FAILURE SIGNATURE (precise enough to recognize a hit):
second tab displays status = "Processing" for a record whose stored
status is "Ready". NOT "an error", NOT a crash -- a stale value on screen.
(record id redacted in anything shared: rec_******)
ENVIRONMENT -- facts vs unknowns:
fact : two tabs in the same browser session (report says "second tab")
fact : upload shows "Processing"; a background worker later sets "Ready"
repro : attempted in local dev -- MY environment, not a claim about the
env reporter (their browser, device, and build are UNKNOWN)
unknown : the exact timing window -> to be measured
unknown : whether the second tab polls or updates live -> to be checked
unknown : cache involvement (browser or server) -> NOT confirmed
CONTROL CASE (the path that is supposed to work):
upload, then refresh the SAME tab -> status shows "Ready". This is the
baseline a failing case has to differ from.
REPRODUCTION MATRIX (one variable changed at a time; 10 attempts each.
the counts come from actually running it, not from a guess):
# | tab | timing | refresh | stale shown
--+---------+---------------------+---------+------------
A | same | after upload | yes | 0 / 10 (control)
B | second | opened immediately | no | 7 / 10 (the failure)
C | second | opened after ~5s | no | 0 / 10
D | second | opened immediately | yes | 0 / 10
READING THE MATRIX (what each row narrows, not proves):
B is the failing condition. It is 7/10, not 10/10 -- INTERMITTENT,
consistent with a race: sometimes the worker finishes before the
second tab loads, sometimes after.
B -> C : the only change is WAITING. Waiting past the window removes it.
B -> D : the only change is a REFRESH. A refresh removes it.
both point the same way: the stale value is whatever was true when the
second tab loaded, and nothing updates it afterward -- UNLESS something
forces a re-read. That is an inference from the matrix, not yet a cause.
MINIMUM FAILING SEQUENCE (smallest thing that still fails):
1. upload a document (status shows "Processing")
2. immediately -- before the worker flips it -- open the same record in
a second tab (this is row B: "opened immediately")
3. do not refresh -> the second tab shows "Processing" and stays there
smaller does NOT work: wait too long (C) or refresh (D) and it passes.
those are not simplifications -- they remove the failure, so they are
the boundary of the minimal case, not inside it.
THREE HYPOTHESES (each with evidence FOR and AGAINST -- none confirmed):
H1 browser/client cache: the second tab served a cached page.
for : a refresh clears it (D: 0/10) -- cache-bust behavior.
against : waiting (C: 0/10) also clears it, which a plain URL-keyed
cache would not explain -- though a cache the first tab
warmed is still possible, so H1 is narrowed, not ruled out.
H2 server response cache: the server returned a stale "Processing".
for : possible if the status read is cached server-side.
against : a refresh (D) returns fresh instantly, which a TTL-based
server cache would not reliably do.
H3 read-after-write / worker timing: the second tab read the status
before the worker flipped it, and never re-read.
for : the ~5s wait (C) removes it; 7/10 fits a race with the
worker; refresh (D) forces the re-read that fixes it.
against : it assumes the tab does not poll -- which is UNKNOWN from
the facts, not established.
THE NEXT DISCRIMINATING CHECK (one check, separates the three):
capture, for a failing run: (a) did the second tab make a network
request at all; (b) what status did the server return and at what time;
(c) the worker-completion timestamp.
no request made -> points at H1 (served from cache)
request returned "Processing", -> points at H3 (the response was
and worker finished AFTER it correct for its moment; the tab
just never updated)
request returned "Processing", -> points at H2 (server handed back
but worker finished BEFORE it a stale value)
this narrows to one hypothesis. It still is not the root cause -- it is
the evidence a debugging pass would start from.
WHAT THIS REPRODUCTION DOES AND DOES NOT ESTABLISH:
established : a reliable way to make the failure appear (B) and disappear
(C, D) on demand -- the thing that did not exist before.
NOT established : the cause. The matrix strengthens H3 and weakens a
plain URL cache, but "reproduces" is not "explains."
HANDOFFS (where this ends, on purpose):
to debugging : the failing sequence + the discriminating check above
to a regression : once the mechanism is known, a forced-delay version
test of sequence B becomes a permanent test -- but writing
it is the test guide's job, and an intermittent repro
needs a deterministic trigger first
fix-validation : after a fix, B goes to 0/N, the control (A) still
passes, AND a record that is genuinely still
processing still shows "Processing" -- a fix that
always shows "Ready" is a new bug, not a fix.Where this fits in NewPrompt
The Reproduce Software Bugs Prompt is the starting point because it is built for exactly this gap — a vague report that nobody can make happen — and it carries the discipline this guide is about: it takes the symptoms, the environment, and the expected-versus-actual as given, treats "no reproduction steps provided" as a fact rather than a hole to fill, and reduces before it diagnoses. Its own notes draw the two boundaries that keep it honest: a bug that already reproduces goes straight to debugging, and writing the test after reproduction is a different tool. It is built with the Debugging Prompt Generator, which assembles the prompt in your browser and runs nothing; what comes back is prompt text you run in your own assistant, against a system only you can touch.
When the bug is the intermittent kind — a failure once in several runs rather than every time — the Investigate Intermittent Bugs Prompt is the sharper tool, because for those the occurrence pattern is the evidence and the work is hunting the difference between the failing and passing runs. It pairs with this guide's Step 4, where timing, concurrency, and load become the variables you vary on purpose. Reproduction and this are the same job at different resolutions: get the failure to happen reliably, then read the pattern in when it does.
The neighbors mark where this task ends. Reading a stack trace is its own step — it can suggest which conditions to reproduce, but it is about interpreting a trace, not producing a failure. General debugging is the arc this feeds: once you can make the failure appear and stop at will, finding and confirming the cause is the next job, and a reproduction that stopped short of naming a cause is doing its job, not falling short of it. And once the cause is fixed, the reproduction becomes the basis for a regression test — the Regression Test Prompt pins the behavior so the bug cannot come back unnoticed, which is a different tool for a different step. This whole sequence is the front of the AI Debugging Workflow, whose first move is "reproduce the failure first" — because everything after it, every diagnosis and every fix, is only as trustworthy as the reproduction it stands on.