How to Threat Model a Feature With AI
Threat model a feature while it is still a design: name what it protects, follow the data until it crosses a trust boundary, and write abuse cases against the real flow rather than a list of security-sounding nouns.
Get the Threat-Modelling StanceThere is nothing to look at yet
That is the point of doing it now, and it is also why the meeting keeps getting moved. Threat modelling happens while the feature is three paragraphs in a doc and a photo of a whiteboard: the exact window in which nobody can say "look at line 40," because there is no line 40. You are being asked what could go wrong with something that does not exist, by people who would rather be building it, and every objection you raise is unfalsifiable in the most literal sense.
Six weeks later there is plenty to look at. There is also a queue in the middle of it that four things now trust, and moving that trust has stopped being an afternoon's conversation. It is a migration. The cheapest hour on this feature is the one where the design is still an opinion. It is also the hour with the least to show for it, which is why it keeps losing to work that produces a diff.
Where the sentences come from
Ask anyone — a model, a room, yourself at 5pm — what could go wrong with a feature, and you will get plausible sentences. Data leakage. Injection. Rate limiting. Access control. Each names a real category, and none of them is about your feature; the same eight bullets fit the last four things you shipped, which is exactly why they generate no work and get nodded through.
The difference is where the sentences come from. Recall gives you categories. Derivation gives you findings. So you write down what the feature protects, who touches it, where it crosses from one level of trust to another, and what already stands in the way — and only then ask what goes wrong. Now the answers arrive attached to something: this asset, past this boundary, given this precondition, against this existing control, leaving this gap. Not more thorough. Arguable, because every line points at a piece of the design somebody drew.
AI is good at this once you have supplied the model. Hand it the feature, the actors, the flow, the boundaries and the controls you already have, and it will produce a covering set of candidate abuse cases, push on the assumptions you left implicit, and catch the boundary you drew two paragraphs ago and then reasoned past. What it cannot do is know your system. What NewPrompt does is assemble a prompt, in your browser, and that is the edge of it: it does not read your repository or your source control, does not reach your running application, your network, your cloud, your database or your identity provider, does not scan code, does not test for vulnerabilities, does not run a penetration test, does not confirm that anything is exploitable, and does not run monitoring or any kind of security testing. Every fact in the model is one you typed. What comes back is a set of candidates, and a candidate threat says only that nothing in the design appears to stop something. That is not the same claim as a vulnerability existing. Your security owner and the technical owners confirm the model, the mitigations and the risk you are accepting; anything high-risk still wants a qualified security engineer and real testing against the built thing. Nothing here makes a feature secure, threat-free, fully mitigated or ready for production.
Step 1: Draw the box, and say what is outside it
One paragraph: what is changing, and what is explicitly not in this model. The second half does more work than the first. Without it every threat model expands until it is about authentication, because authentication is upstream of everything and somebody always has a question about it, and you spend the hour re-modelling a system that already has a model and an owner.
So name the neighbours and hand them over out loud. The login that got the user here is a different model. Who is allowed to do what, once they are in, is access-control design, and it is one control family inside this one rather than a substitute for it: a permission matrix tells you who may press the button and nothing about what the button reaches afterwards. This box is the feature's own path. The boundary you draw here is what stops the document dissolving into every adjacent concern in the codebase.
Step 2: Name what the feature protects before naming what could go wrong
Assets first, threats second, and in that order for a reason: a threat with no asset behind it is a worry, and worries do not survive a planning meeting. The list is shorter than people expect and always contains something that is not data:
The two that get missed are the last two. Availability is an asset because a feature that lets one customer stall a shared queue produces an outage the other customers experience, and nobody files that as a security bug. Audit history is an asset because when the question is who did this, the answer has to be true, and a record that can be made to say the wrong thing is worse than no record: somebody will believe it.
Keep this list separate from the things that merely appear in the diagram. A queue, an object store and a log aggregator are dependencies — they do nothing on their own, they have no intent, and filing them as actors is how you end up with a threat model about infrastructure instead of about people.
- The data itself, including other customers' data, which is the one people forget because this feature is not about them.
- Credentials and secrets the path touches, holds, or logs.
- Availability of this feature, and of anything sharing infrastructure with it.
- The audit record: who did what, and whether it can be made to say something false.
- Tenant isolation, if you have tenants. An asset in its own right, not a property of the database.
- User trust — the quiet one. The reason a user believes what the feature shows them is theirs.
Step 3: Follow the data until it crosses something
Draw the flow in whatever ugly form you have; arrows in a text file are fine. Where does data enter, where does it go, where does it rest, who reads it later. Then mark every place it crosses from one level of trust to another. A trust boundary is not a network-diagram thing. It is any point where something stops being checked because whatever came before it was supposed to have checked.
That sentence is where most findings live. The boundary at the front door is the one everybody models, and it is usually the one already handled, because it looks like a boundary. The ones that pay are internal and invisible: a service that receives work from a queue has no session, no user, and no way to tell whether what it just received is true. The control at the front door was real. It does not extend past the hallway, and nothing announces that.
Then write the assumptions down as assumptions, and go check two of them. These are the load-bearing sentences of the whole model, they are usually said rather than verified, and one of them will be wrong in a way that quietly invalidates a chunk of your work — either by making a threat impossible, which is a relief, or by making it trivial, which is the finding. When one turns out wrong, redraw the diagram before you touch the threat list. An assumption that changes the flow has changed what everything downstream of it was resting on, and a threat list reconciled against an old diagram is the exact mistake this whole exercise exists to catch.
Step 4: Write each abuse case against the flow, not against your imagination
An abuse case earns its place by pointing at the diagram. The shape that keeps it honest:
The precondition does the filtering. "Someone sends a malicious file" has no precondition and can be neither argued with nor designed against. "Something other than the API can enqueue a job" is a precondition: it is either true or false, somebody can go and find out which, and until they do you know precisely what you do not know.
Frame these as properties rather than performances. The useful sentence says nothing in this design stops a given thing from happening — a claim about the design, checkable against the diagram, and the input to a fix. The alternative is a story about an intruder crafting something clever, which sounds more urgent, cannot be verified without building the thing, and invites a level of operational detail that nobody needs in order to close the gap. You are here to find out what to build. Keep the threat at the resolution that decides the mitigation, and no further.
- Preconditions — what has to be true first. If nothing does, it is not a case yet.
- Affected asset — from Step 2's list, by name. If it is not on the list, say why it is here anyway.
- Impact — what it costs, in the currency of that asset.
- Existing control — what is there, honestly, including "nothing".
- Gap — why the existing control does not cover this. Usually: it lives on the wrong side of a boundary.
- Candidate mitigation — what would close it, at the level of a requirement rather than a patch.
- Verification need — the test somebody writes later to find out whether any of this was true.
Step 5: Keep the control you have apart from the mitigation you want
Two columns, never one. The moment "we bound the file size" and "we should bound it in the worker too" share a cell, the model reads as though the gap is closed, and everyone who skims it will believe that. Present tense is what exists. Future tense is what somebody has to build, and it needs a name on it or it will not get built.
Then check each mitigation three ways. Against the feature: a fix that changes what the feature means is a product decision in disguise, and a security review is not where product decisions get made. Against itself: the natural fix for a data-in-logs finding is to log the thing so you can debug it. And against the other mitigations, which is the one everybody skips — two fixes written an hour apart, each sensible alone, can meet in the middle and produce a behavior neither author would have signed off on.
On severity, be plain and stop. Impact usually follows from the asset and you can reason about it. Likelihood usually depends on a fact nobody in the room has, and inventing one to fill a column produces a number that outlives its own uncertainty: six weeks on it is a 3 in a spreadsheet and nobody remembers it was a shrug. Write "unknown", and keep high-impact-unknown-likelihood visible as both rather than averaging it into a comfortable medium. What you leave open, leave open deliberately: a residual risk with an owner, a reason and a revisit trigger has been accepted, and the same risk without them has merely gone unnoticed. Only one of those is a decision, and it belongs to the security owner and the product owner together. The model does not get a vote.
Step 6: Hand it over with the unknowns still marked
It goes to somebody who can say the model is wrong: a security engineer if you have one, and the technical owners of whatever it depends on. What they are reviewing is not the threat list. It is the box and the assumptions, because everything else is downstream of those, and if the box is wrong the threats are wrong in a way no amount of care in the threat list will repair.
Mark the false positives instead of deleting them. Something gets raised in every one of these that turns out not to be a threat this model can act on: a real problem in the wrong document, a compliance question in a security question's clothes. Strike it, say why, name the person it went to. Delete it and it comes back next review, from the same person, with the same conviction, and gets re-litigated from scratch.
Then keep it, and give the unknowns names too. An open question with an owner is work. The same question with "TBD" against it is a decision nobody has noticed they are making by default.
Common mistakes
The failure mode is a document that reads like security and generates no work.
- Threats with no asset. "Injection" is a category. "A validation error puts a customer's phone number somewhere every engineer can read" is a threat, because you can name what it costs and who it costs it to.
- The threat model that was never reopened. It was written, it was reviewed, and then the design changed twice and nobody looked at it again. A model that is not reopened when the flow moves was a compliance exercise with good intentions.
- Boundaries drawn around teams. The line goes where the trust changes, not where the org chart does. Two services owned by the same group, one of which cannot verify what the other sent it, have a boundary between them whatever the reporting structure says.
- The assumption verified by asking the person who wrote it. "Can anything else reach this queue?" "No." That is the same sentence you started with, now wearing a witness. Verified means somebody looked at the configuration.
- Precision nobody earned. A severity of 7.5 on a feature that does not exist, computed from a likelihood that was a guess. The decimal point is doing rhetorical work. Say high impact, likelihood unknown, blocked on Q1, and the reader knows exactly as much without being misled about how much.
- Reading the list as findings. They are candidates against a design, written from what you typed. NewPrompt does not read your code, reach your systems, test anything, or establish that one of them is real. A security engineer decides which survive, and the built thing still needs testing.
Threat modelling a multi-tenant CSV import
A workspace admin uploads a CSV of contacts and the rows become contact records in their workspace. Small, dull, and it touches every tenant's data, an object store, a queue and a log aggregator. The ordinary kind of feature where this actually pays. The feature, the infrastructure and the people in it are invented.
One feature, three trust boundaries and six candidate threats: a wrong assumption that redraws the diagram and creates a threat, a control that stops at a door the file never uses, two mitigations that collide, a fix that reopens what it closes, and one struck as somebody else's problemTHREAT MODEL — Bulk CSV Contact Import draft v1, design stage, no code
THE BOX
In scope The import path: upload, parse, validate, insert, import report.
Out of scope The contacts feature itself. Export. The authentication that got
the admin here; that has its own model and its own owner.
Written first. In six months this paragraph is the part still true.
ACTORS
Workspace admin authenticated; starts the import
Workspace member sees the resulting contacts and the import report
Import worker internal service, runs async, holds no user session
A compromised admin not a villain. The same admin, whose session is under
someone else's control. Same permissions, different
intent. Modelled because the controls hold either way.
DEPENDENCIES (not actors: they do nothing on their own)
Object store holds the uploaded file
Queue carries the job from API to worker
Log aggregator third party; engineering-wide read access
ASSETS — what this path protects
Every workspace's contact data, not only this one's
The uploaded file, at rest in the object store
Availability of the import pipeline for the other tenants
The audit record of who imported what, and when
A member's trust that a contact in their list came from their workspace
DATA FLOW
browser --[B1]--> API ---> file record {file_key, workspace_id, uploaded_by}
| |
| +--> queue {file_key, workspace_id, actor_id}
| |
| [B2]
| v
+--[B3]--> object store <--------- import worker --> database
pre-signed PUT |
24h, no API in path +--> report --> browser
+--> log aggregator
TRUST BOUNDARIES
B1 browser -> API. Everything from the browser is untrusted. The tenant comes
from the session, never from the request body.
B2 queue -> worker. The worker holds no session. It has workspace_id because
the payload contains it.
B3 browser -> object store. Direct. The API is not in this path, so no
API-side control applies to it at all.
B1 is the one everybody models. B2 and B3 are the ones that pay.
ASSUMPTIONS (checked, and one was wrong)
A-1 The queue is reachable only from the API. UNVERIFIED — see Q1.
T-1 rests on this.
A-2 The bucket is private, so the file is reachable only by us.
WRONG. The upload uses a pre-signed URL handed to the browser, valid 24
hours. The true statement: reachable by anyone holding that URL for 24
hours, and writable by them too. This is B3, which was not on the first
diagram. Correcting it changed three lines: it created T-2, it removed
C-2 from the upload path entirely, and it is why the file record — not
the file — is what T-1's mitigation trusts.
EXISTING CONTROLS (what is there, not what we want)
C-1 API derives workspace_id from the authenticated session, at B1, and
writes it to the file record
C-2 10 MB limit on the upload request
C-3 Bucket is private; access by pre-signed URL
C-4 Logs are access-controlled to engineering
──────────────────────────────────────────────────────────────
T-1 Rows land in another workspace
IMPACT high LIKELIHOOD unknown — turns entirely on Q1
Preconditions Something other than the API can enqueue, or the API copies a
client-supplied field into the payload. Depends on A-1.
Asset Every workspace's contact data. Tenant isolation.
Impact Contacts written into a workspace that did not import them,
visible to its members, attributed by the audit record to an
actor who did nothing.
Existing control C-1 — at B1.
Gap C-1's guarantee is not re-established at B2. The worker cannot
tell whether the workspace_id it holds is the one C-1 derived
or one someone put there. The value crosses; the guarantee
does not.
Mitigation The worker reads the tenant from the file record it fetches by
file_key, and treats the payload's workspace_id as an assertion
to verify against it rather than a fact to use.
Verification A test that enqueues a job naming a different tenant, asserting
zero rows land. Nobody has written it.
^ high impact and unknown likelihood do not average to medium. Both stay
on the line, and the unknown is what makes Q1 the blocking question.
T-2 A file larger than the API would take
IMPACT medium LIKELIHOOD high — this needs no bad intent
Preconditions An admin writes a file through the pre-signed URL (B3) that is
larger than C-2 would have allowed. Depends on A-2.
Asset Availability of the pipeline for the other tenants.
Impact Workers pinned on one file; queued imports stall. One tenant
produces an outage the other tenants experience.
Existing control None. C-2 sits on an API path this file does not take.
^ this line only exists because A-2 got checked. On the first diagram the
file went through the API, C-2 capped it at 10 MB, and T-2 was
impossible. One wrong assumption was holding up the whole threat.
Gap Nothing bounds the file, and nothing bounds rows or line
length even if something bounded bytes.
Mitigation The worker enforces its own bounds — bytes, rows, line length
— and fails the job rather than the process. A per-tenant
concurrency cap would bound the blast radius further.
Verification A file over each bound, asserted to fail cleanly while a second
tenant's queued job completes.
T-3 The same import runs twice
IMPACT medium LIKELIHOOD high — retries are on by default
Preconditions The queue retries after a worker timeout.
Asset Contact data integrity; the audit record.
Impact Duplicate contacts. Two audit rows, reading as though the admin
imported twice.
Existing control None.
Gap The job carries no idempotency key and the insert is not
idempotent.
Mitigation An idempotency key on the JOB — the file_key, which is unique
per upload — recorded only on successful completion, so a retry
is a no-op and a re-run after a failure is not.
^ two traps here, and the first one is the interesting one. 'Upsert the
rows on email' was the first suggestion: it closes the threat and
silently merges two people who share an inbox. That is a feature change
wearing a mitigation's clothes, and it does not get decided in a
security review. The second: key on completion, not on receipt. Key on
receipt and a job that failed under T-2's new bounds can never be
re-run — the mitigations collide and the import silently never happens.
Verification Enqueue the same job twice, assert one set of rows. Then fail
one, re-enqueue it, and assert it runs.
T-4 Contact data leaves the workspace through the error path
IMPACT medium LIKELIHOOD high — bad rows are the common case
Preconditions A row fails validation. Every real import has some.
Asset Contact data, and who is allowed to see it.
Impact Two audiences widen at once. The report echoes the failing row
to every member of the workspace, not only the admin who
uploaded. The worker logs it to an aggregator all of
engineering can read — somewhere the workspace never agreed to.
Existing control C-4 — logs access-controlled to engineering.
Gap 'Engineering' is a much larger audience than 'this workspace'.
C-4 is a control against outsiders and was never a control
against this.
Mitigation Requirement, not a log line: no contact field value reaches the
aggregator by any path, including exception traces. The report
and the log carry a row number and a validation code.
^ the obvious fix is 'log the row so we can debug it', which is the threat.
And patching one log statement misses the stack trace, which is where a
parse failure puts the row anyway. Note this does not close everything:
row number plus file_key still points at the value for anyone who can
read the store. That is smaller, and it is not nothing.
Verification A failing import and an induced parse exception, both asserted
clean.
T-5 A stored cell is a formula somewhere downstream
IMPACT low LIKELIHOOD unknown — depends on the export path
Preconditions A contact field begins with a character a spreadsheet reads as
the start of a formula.
Asset Outside the ASSETS block above, deliberately: the risk lands on
whoever opens an export, and export is out of the box. It is
here because this is where the content enters the system.
Impact A cell that is data here may not be treated as data there.
Existing control None at import. Stored verbatim, which is correct.
Gap Not ours to close. Import is the entry point, not the exposure.
Mitigation Handed to Priya (export owner) with the entry point named.
Verification The export owner's call, in the export owner's model.
T-6 STRUCK — not a threat this model can act on
Raised 'An admin could import a contact list they have no right to.
That is a GDPR problem.'
Assessment Real problem, wrong document. The feature does exactly what it
says, for a user who is who they claim to be, with data they
already hold. No boundary fails and no control is missing:
there is nothing here to design.
Disposition Struck. Sent to Marta (data protection lead). Recorded so it
is not re-raised and re-litigated from scratch in the next
review.
──────────────────────────────────────────────────────────────
ACCEPTED RESIDUAL RISK
T-2's per-tenant concurrency cap is not being built this quarter. With the
worker's byte and row bounds in, one tenant can slow the queue but not stop
it. Residual: a determined admin degrades import latency for others.
Accepted by Ana (platform lead). Revisit when import volume doubles.
Accepted with a name and a trigger is a decision. The same risk without them
is just unnoticed.
OPEN QUESTION — blocking
Q1 Can anything other than the API enqueue an import job? Nobody in the room
knew. T-1 is high impact and its likelihood is unanswerable until this is.
Owner: Ana. This is the line the review was actually for.
NOTHING HERE HAS BEEN RUN
Every Verification line is a test somebody still has to write, against code
that does not exist. T-1 says nothing in this design prevents a job payload
from naming another tenant. It does not say the import is exploitable: nobody
has tried, and there is nothing yet to try it against. A security engineer
reviews this before any of it is treated as settled. Fictional feature,
fictional infrastructure, fictional people.Where this fits in NewPrompt
The Security Engineer Role Prompt is the stance, and stance is most of what separates a threat model from a list of nouns. It is a rendered role prompt, a standing instruction you paste at the top of the conversation, and its own copy names this job outright: threat-modelling a feature before it ships is the first thing it says it is for. What it puts in the model's head is the part you would otherwise have to argue for. Assume breach, so one compromised component does not hand over everything. Ask what is worth stealing and where the trust boundaries are. And the rule that keeps the exercise in its lane: stay defensive, recommend mitigations and detections, do not produce working exploit code. Its limits are declared too — it says no to offensive tooling by design. One place to depart from it deliberately: it tells the model to rank by likelihood times impact, and Step 5's position is that when you do not have the likelihood, saying so beats multiplying by a guess.
It is a role, which means it supplies the perspective and not the document. The structure — the box, the assets, the flow, the seven fields per abuse case — is yours, and it is the half that makes the output reviewable. Role Prompt Generator is where a variant of that prompt gets built, in your browser, if the stock seniority or focus areas are not yours. It assembles persona text and nothing else runs.
The clean neighbour is Security Code Review Prompt, and the boundary between them is time. It reviews code that exists against a fixed list of checks and returns findings on a file you paste. This model runs before any of that, on a design, and produces requirements rather than findings. Its own note applies here too: it raises the floor, it is not a substitute for a professional penetration test. The AI Security Review Workflow is where that half lives — anchor the model as a security engineer, run a threat-focused review of the code, back the findings with tests. Read its boundary rather than assuming it is this guide's parent: it says plainly that it is not for designing security architecture from scratch, because it reviews existing code. This guide is the step before there is anything to review. NewPrompt gives you the wording and the sequence. Running them is yours, and so is every risk decision that comes out.
In six months, most of what you wrote will be gone. The threats will have been fixed, accepted, or overtaken by a rewrite that made half of them irrelevant. Somebody will be sitting in front of the same four arrows asking whether the new queue changed anything, and they will answer it in twenty minutes, because the boundaries are drawn and the assumptions are written down where a changed one is visible. That is the whole return. Do it without them and the next person starts from a blank page, in front of a feature that now has a line 40 — which is how most teams end up threat modelling exactly once.