For Harborline Financial Group, a fictional customer. All data is synthetic.
A security review for every AI agent, and every decision is recorded with who made it.
Fixed rules and two AI models review each agent against the OWASP Top 10 for Agentic Applications. They show their evidence and where they disagree. Then a person decides, and the decision is audited.
The problem
Every agent needs a review, and the reviews take weeks
- Harborline plans to launch about 40 internal AI agents this year, heading toward 300: claims triage, KYC document review, payments operations, HR and IT helpers.
- Every agent needs a security review before launch. A small AppSec team does them by hand, and a review takes about 15 business days.
- CISO
- “How do I know the AI reviewer is not being fooled?”
- Model Risk Officer
- “Who is accountable for each decision?”
- AppSec lead
- “Will I drown in false alarms?”
- Platform engineering
- “Can I see and cap what this costs?”
How a review works
Five steps, and only the last one is a person
- 1
Submit a profile
The agent's tools, permissions, data, confirmation, credentials, memory, logging and kill switch, checked against a strict schema.
- 2
Rules set a floor
Eight fixed rules read the listed fields, never the prose, and set a minimum severity. Tier rules set the rule tier.
- 3
Reviewer model finds risks
Against the ten OWASP agentic risk categories. Every finding needs a quote copied exactly from the profile.
- 4
Checker model checks them
A second model from another provider gives a verdict on each finding, adds what was missed and proposes its own tier.
- 5
A person decides
The final tier is the highest of the three, so a model can raise it but never lower it. The decision is recorded and audited.
Four tiers
The tier decides who must approve
- Tier 1
Read-only, internal documents
AppSec approves
- Tier 2
Drafts changes or takes small actions a person confirms
AppSec approves
- Tier 3
Handles customer financial data
AppSec, then Model Risk, two different people
- Tier 4 Committee
Moves money with no person confirming, or acts on its own with power to write or run code
Never approved in the app: sent to a review committee with an evidence pack
Five decisions
What I chose, and why
- 1
A model can raise the tier, never lower it
Rules set a floor for severity and tier. The final tier is the highest of the rule, reviewer and checker tiers.
Why. Models can be fooled into lowering the severity or risk floor, which is a security risk. Erring on the lower side instead of the higher side can be dangerous.
- 2
Evidence quotes stay word for word
A finding whose quote is not found in the profile by exact match is dropped, so every quote on screen can be checked against the profile.
Why. So a reviewer can check every finding against the profile itself. A paraphrase could hide a model's mistake or quietly change the meaning.
- 3
The database records the decision
A decision is written to our own database first, then the paused workflow is resumed. If the run cannot be resumed, the decision still stands.
Why. The decision is the part that has to be true. It is saved to our own database first, with an audit entry, so it stands even if the paused run can't be resumed.
- 4
Providers restricted, no prompt training
Each call names the providers allowed to serve it and refuses any that may train on prompts. Zero data retention is not set: it needs a Pro or Enterprise plan.
Why. Today's profiles are fictional, but a real customer's would describe real systems. So each model call is limited to named providers that don't train on prompts.
- 5
Measure at a freeze, never tune on held-out profiles
Models, prompts and rules were frozen at a git tag before any evaluation run. Four of the 12 profiles were held out and never tuned on.
Why. If I tune on the profiles I score, the score flatters me. So I froze the system, then measured, and kept four profiles that were never used for tuning.
The evidence
Measured on 12 synthetic profiles, at a freeze
At the eval-v1 freeze
| Measure | Tuning set (8) | Held-out set (4) |
|---|---|---|
| Final tier inside the answer key's range | 6 of 8 | 2 of 4 |
| Final tier below the key (the dangerous direction) | 0 | 0 |
| Expected critical risks found, rules and models combined | 9 of 9 | 6 of 6 |
| Found by the rules alone / the models alone | 6 of 9 / 8 of 9 | 5 of 6 / 6 of 6 |
| Adversarial profiles that fooled it | 0 of 2 | none in this set |
| Cost per review, from AI Gateway records | about $0.008 | about $0.008 |
| Model time per review | about 20 s | about 22 s |
What did not go well
Held-out misses
At the freeze, 2 of the 4 held-out profiles ended one tier too high. Every tier error was too high, never too low.
Profile 03 still fails
Out of range in 5 of 6 runs: the reviewer model often calls this read-only research bot Tier 3. Left for the next round.
Profile 04 is Harborline's call
The rules give Tier 4, because it changes live claim records with no person confirming. The answer key says Tier 3. It is a policy question for Harborline, and the rule was not changed.
ASI01 or ASI06
In 7 of the 8 reviews where the models missed an expected Agent Goal Hijack (ASI01), they filed the same risk as Memory and Context Poisoning (ASI06).
The checker is not independent
It agreed with 45 of 45 reviewer findings. Without the reviewer's list it raised a clearly different set. A blind checker was not better: tiers did not change. See the comparison below.
A small set
12 profiles, one answer key, synthetic data. These are indications, not proof, and answers vary between runs.
Is the checker an independent second opinion?
The same checker model, run two ways on the same reviews: as built (it sees the reviewer's findings first) and blind-first (it sees only the profile and the rules output).
| Measure | Tuning set (8) | Held-out set (4) | ||
|---|---|---|---|---|
| As built | Blind-first | As built | Blind-first | |
| Overlap with the reviewer's categories | 75% | 35% | 79% | 44% |
| Reviewer findings not supported | 0 of 30 | 17 of 30 | 1 of 16 | 8 of 16 |
| Final tier in the key's range | 7 of 8 | 7 of 8 | 3 of 4 | 3 of 4 |
The blind checker's extra disagreements were mostly risks it did not list that run, not reasoned objections; it also differed from its own earlier runs. No production change. These runs came after the fixes, so the held-out results had already been seen.
After two checker fixes (labelled separately)
- Tuning set, final tier in range
- 7 of 8
- profile 03 counted from its one baseline run; it failed 5 of 6 runs
- Held-out set, final tier in range
- 3 of 4
- one rerun after the fixes, held-out results already seen
- Profile 07 (payments, a person confirms each payment): the checker called it Tier 4. After fix A, in range in 3 of 3 runs, from 0 of 3.
- Profile 02 (read-only HR bot): the checker counted “no person confirms” as a risk though it takes no actions. After fix B, 3 of 3 runs in range, from 2 of 3.
- Each fix is one sentence in the checker's instructions, measured on 3 runs before and 3 after.
Security tests
- Four prompt-injection attacks, including orders in the owner name and fake end-of-data markers: every tier held at 4 and each attempt was flagged with a verified quote.
- Every route test passed: signed out, wrong role, wrong stage, another site, decision replay and reuse, and the public workflow webhook.
- One real fault found and fixed, and four wrong test expectations corrected, with both results kept.
- A scan of every saved version of every file, run during security testing, found no passwords or keys; each change since is scanned before it is saved.
Full detail: evaluation results, the checker independence experiment and security tests.
Where things run
Vercel runs the review. Harborline keeps its systems of record.
On Vercel (built)
- The overview site and the reviewer app (Next.js)
- The durable review that pauses for a person (Vercel Workflow)
- Every model call, with provider allow lists and no prompt training (AI Gateway)
- Reviews, decisions and the append-only audit log (Neon Postgres from the Marketplace)
In Harborline's AWS (stays there; not built here)
- Corporate sign-in (OIDC or SAML)
- The agent registry
- The audit log and SIEM
- Ticketing
Vercel Secure Compute would connect the two over a private network. Designed, not built.
Built on Vercel
- Hosting and Git deploys
- One project; every push to main deploys the live site.
- AI Gateway
- One path to both models, with provider restrictions, no prompt training and the cost of every call on record.
- Workflow
- The review runs as a durable workflow that waits, for hours if needed, until a person decides.
- Neon from the Marketplace
- Postgres added from the Vercel Marketplace, holding reviews, decisions and the audit log.
Honest limits
What this build does not do yet
Limits
- Sign-in is a stand-in: anyone with the demo passcode can choose any role.
- A paused or running workflow keeps all its data with no time limit. Vercel's retention clock (1 day on Hobby, 7 on Pro, 30 on Enterprise) starts only after a run finishes. Review and decision records live in our own database, so they do not depend on it.
- Zero data retention is not switched on (it needs Pro or Enterprise).
- The evidence check proves a quote is in the profile, not that it supports the finding. The approver judges that.
What is next
- Fix profile 03's reviewer tier, measured the same way.
- Ask Harborline to settle profile 04's tier.
- Test a checker that runs several times and keeps only risks it raises consistently.
- Real corporate sign-in, the registry, SIEM forwarding and Secure Compute.
- Workflow's experimental zero-retention option for finished runs: described, not enabled.