Audn vs Codex vs Aikido on the Same Repo — 159 Findings, and All Three Agreed on Exactly 3

Every security vendor publishes a benchmark where they win.

So here's ours, with the part nobody publishes attached: the 63 findings we couldn't confirm, the 9 findings Codex caught that we structurally cannot see, the 23 findings Aikido found alone — including a class we don't scan for at all — and the blind spots that survive even with all three tools running.

The full interactive report is live, unedited, with every row from all three sides:

→ blog.audn.ai/audn-vs-codex-vs-aikido

(The original two-way Audn-vs-Codex A/B is still up at blog.audn.ai/audn-vs-codex.)

The setup was simple. We pointed Audn Agentic WhiteBox at a fork of OWASP Juice Shop — the deliberately-vulnerable app the whole industry uses as a punching bag. Then we ran the official OpenAI Codex security review over the same source. Then we ran Aikido, a commercial SAST + SCA + secrets + CI platform, over the same repo. Same target, ozgurozkan123/juice-shop. Three very different machines looking at it.

The headline numbers are 103, 20, and 36. Those are also the least interesting things in the report.

The Result That Should Worry You

Between the three tools there were 159 findings.

The three-way intersection — issues that all three independently flagged — is three issues:

JWT signature is not verified — Audn traces the hard-coded key and algorithm confusion in lib/insecurity.ts; Codex flags a newly-introduced global registration of forged JWTs; Aikido's SAST flags the same files by pattern.
Unpinned third-party GitHub Actions holding the repo token — image_actions.yml, all three, from three different angles.
Remote fetch without integrity check in CI — Audn's remote installer lead, Codex's lockfile/scripts findings, Aikido's ci.yml flag.

Three tools. 159 findings. Three unanimous. When a live red-team, a diff reviewer, and a commercial SAST/SCA platform all land on the same thing independently, treat it as certainly real — and treat the other 156 as evidence that these tools are not substitutes for each other.

	Audn WhiteBox	Codex	Aikido
What it is	SAST fused with a live red-team run	Commit-diff code review	SAST + SCA + secrets + CI (ASPM)
Findings	103	20	36
Severity mix	18 Crit / 24 High / 52 Med / 5 Low / 4 Info	8 High / 7 Med / 2 Low / 3 Info	3 Crit / 13 High / 8 Med / 12 Low
Reproduced live	18	0	0
Dependency CVEs (SCA)	No	No	Yes — 3
Secrets scanning	Partial (source)	No	Yes — 17
Diff-aware	No	Yes	No
Remediation	Prose + CVSS + attack paths	Generated git diff	Fix-time estimate + guidance
Runtime	2h 28m + live target	Light, diff-scoped	~52 seconds
Sweet spot	Pentest / exploitability proof	PR gate / regressions	Continuous baseline

Where Each One Actually Shined

Audn shined on: proving things

103 findings with CWE, file:line, CVSS vectors, and 32 derived attack paths. But the number that matters isn't 103 — it's 18, the findings the agent didn't just report but fired at the running app and reproduced:

server.ts:280 — unauthenticated public access logs containing change-password URLs with cleartext current and new passwords. Not a code smell. Pulled off the live box.
A spoofable X-Forwarded-For header bypassing the password-reset rate limit — an account-takeover enabler with no static signature at all. The agent found it by getting rate-limited and then working out how to not be. No pattern scanner will ever find this, because there's no pattern.
routes/search.ts:21 — SQL injection, fired and confirmed. NoSQL injection in trackOrder.ts and showProductReviews.ts. XSS via models/product.ts. Hard-coded credentials in users.yml, 7ms.yml, and routes/login.ts — four of them, all validated live.

The other Audn-exclusive lane is authorization logic, which pattern scanners structurally cannot reason about: 13 broken-access-control/IDOR findings (basket IDOR exposing other users' carts, coupon updates missing ownership checks, duplicate BasketId bypassing ownership), 3 race conditions (wallet debit permitting concurrent overspending), and the business-logic set (client clock controlling coupon validity, unanchored coupon validation accepting unbounded discounts). Aikido found zero IDOR findings. Not because it's bad — because no regex knows who's supposed to own a basket.

Codex shined on: what the change broke

Codex reviewed commit 160f306 as a diff. Nearly every finding is framed as "introduced by this commit." It caught nine things nobody else did — and adding Aikido didn't cover a single one of them:

2FA temporary JWTs accepted as bearer auth. A commit stopped inserting no-data JWTs into authenticatedUsers to fix a crash — but never started rejecting them. They still sail through as valid auth. A red-team walking the app from outside would never trip over this. A diff reader catches it instantly.
Unbounded complaint similarity scan → DoS. A per-request check running ComplaintModel.findAll() plus an O(n) similarity loop over every complaint. Introduced by the changeset.
A Junie AI-skill that curls untrusted reference URLs and feeds the response back to the agent — SSRF reaching localhost and link-local ranges. That's agent tooling in the developer environment, a supply-chain surface our exploit lens never scanned.
A new default theme hard-coding remote Google Fonts, leaking every visitor's IP, UA, and referer to a third party. A privacy class we don't look for.
Plus a Windows-only i18n path bug, a broken Hacking Instructor lookup, a dropped test boundary assertion, and a 500 on missing snippet metadata.

Codex also does something the other two deliberately don't: it separates intentional challenge vulnerabilities from real risk. On an app built to be broken, that calibration is worth a lot. And every finding arrives with a patch you can apply.

Aikido shined on: the hole the other two shared

This is the finding that made the three-way version of this report worth publishing. In the two-way Audn-vs-Codex writeup, blind spot #1 was dependency CVEs — neither tool ran SCA. Aikido closed it.

23 of Aikido's 36 issues are unique to it, and they cluster into exactly the classes we and Codex both skip:

3 real dependency CVEs — jsonwebtoken (missing input validation), express-jwt (improper authorization), sanitize-html (XSS). Nobody else ran SCA. Nobody else would have found these.
17 secrets findings across 40+ files, including test fixtures — one file alone holding 33 exposed secrets, plus exposed JWTs scattered through the tree. Our secrets coverage is partial and source-only; Codex has none.
5 SAST/CI findings the others missed — unsafe YAML load leading to RCE, document.write() XSS, XSS via window.location.href, file inclusion, and actions/checkout persisting Git credentials.

And it did the whole thing in ~52 seconds. Against our 2h 28m. That speed difference isn't a footnote — it's the entire reason these three tools belong at different points in the pipeline, which I'll get to.

Where Each One Is Weak, Stated Plainly

Audn:

Most findings are unconfirmed. 63 static leads + 18 probed-no-verdict. Only ~17% reproduced live.
No patches. We describe the fix in prose. Codex hands you a diff. We should hand you a diff.
Slow, and needs a deployed target. 2h 28m.
No SCA. We don't enumerate dependency CVEs at all — Aikido's three would have gone unfound.
No diff awareness. We can't tell a regression from a decade-old line.
We over-report intentional vulns, because we have no notion of "is this supposed to be here."

Codex:

Narrow. 20 findings; misses most of the app's real attack surface by design.
No live proof. Static only.
Scoped to one changeset — it won't tell you the posture of the whole app.
No SCA, no secrets sweep. Near-duplicate findings inflate the count.

Aikido:

Static only — no exploit proof, same as Codex.
No diff awareness — misses every one of Codex's introduced regressions.
Noisy. Many of those secrets are test fixtures. Counts are grouped, so "33 exposed secrets" is one row.
No business-logic or IDOR reasoning — zero findings in the largest category Audn covers.
Incomplete SCA. Aikido itself flagged a missing lockfile, so its own dependency view is partial. Transitive known-vulnerable packages can still hide.

Audn's honesty layer

One structural difference worth naming: Audn grades its own certainty and publishes the grade.

Confidence tier	Count	Meaning
Confirmed	16	Flagged in source and reproduced live
Observed live	2	Reproduced live with no static lead behind it
Probed, no verdict	18	Attacked, reached no conclusion
Static lead	63	Flagged in source; the run never got to it
Likely false positive	4	Exercised without reproducing

We could have shipped 103 rows of undifferentiated red and let you sort it out. Instead the report tells you which 18 are facts about your running system and which 63 are patterns we never got to prove. We also label four of our own findings as likely false positives, and flag where our SAST input got truncated rather than pretending the coverage was there.

Neither Codex nor Aikido has an equivalent tier. In Audn's terms, all 20 Codex findings and all 36 Aikido findings sit at the unverified level — as do 83 of our own.

Recommended stack — where each scanner runs

These three aren't competitors. They're gates at different points on the path from commit to main. Cost sets the cadence: Aikido is cheap enough for every push, Codex rides every PR, and Audn — heavy and needing a live target — runs at the merge-to-main gate.

On every commit / push → Aikido (~52s)

SAST + SCA + secrets + CI/IaC baseline.

Fail on a new dependency CVE
Fail on a committed secret
Flag obvious injection / XSS sinks

Fast enough to run on every push without slowing anyone down — and it's the only one that catches the dependency-CVE + secrets class.

When a PR is opened → Codex (light)

Diff-aware review of the changeset.

Block introduced regressions and logic bugs
Attach the generated patch to the review
Skip intentional / known vulns

Rides each pull request; tells reviewers what this change broke versus what was already there — and hands them a fix.

When the PR is ready to merge → main → Audn (~2h 28m)

SAST + live red-team against a deployed preview of the branch.

Block merge on a live-confirmed critical
Surface business-logic / IDOR chains
Rank by confidence (confirmed vs lead)

The last line before code hits main — the only tool that proves real exploitability. Too heavy for every commit; perfectly placed at the merge gate.

⚑ Prereq: Audn needs a running target. Deploy the PR to a preview/staging environment first, then point Audn at it.

On main / in production → Aikido + Audn

Keep the baseline live. Aikido re-scans continuously (new CVEs land daily); Audn re-runs periodically against prod or staging. Dependency CVEs surface after merge too, so the baseline scan never really stops.

Why this order — cheapest and most frequent first, exploit-proof last

Gate	Tool	Runs on	Cost	Blocks the merge when…
Baseline	Aikido	every commit / push	~52s	a new dependency CVE or a secret enters the tree
PR gate	Codex	PR opened	light	the diff introduces a regression or logic bug
Merge-to-main gate	Audn	PR ready to merge	~2h 28m	a vuln is live-confirmed exploitable on a preview deploy
Post-merge	Aikido + Audn	continuous / periodic	mixed	— monitoring for new CVEs & drift, not a gate

The logic: run the cheapest, highest-frequency check earliest and often; reserve the expensive live-target check for the merge-to-main milestone, where a false "ship it" is most costly. Codex sits in the middle as the human-review companion.

One caveat worth planning for: Audn's ~2h 28m run means the merge gate is not instant. Budget for it, or run Audn against the preview as soon as the PR is approved rather than at the moment of merge.

Why a company needs all three

Because each one is blind in a way the other two are not, and the blindness is structural — not a roadmap gap you can wait out.

Drop Aikido and you ship known-vulnerable jsonwebtoken and express-jwt to production, plus 17 secrets sitting in your tree. Neither of the other two runs SCA. Both would have missed all of it.
Drop Codex and every subtle regression your team introduces ships. The 2FA JWT bug is the perfect example: the code got safer-looking and less safe in the same commit. A whole-repo scanner sees a file it already knows; only a diff reader sees the change.
Drop Audn and you have 56 static findings across two tools and zero proof that any of them fire in your deployment — plus no coverage at all of the IDOR, race-condition, and business-logic layer that made up the largest slice of real findings here. You'd be triaging a pile of maybes and calling it a security posture.

They're layers, not substitutes. Aikido is the fast continuous baseline, Codex is the PR gate for regressions, Audn is the live pentest that proves exploitability.

What still isn't covered — by any of the three

We cross-referenced all three reports against Juice Shop's own 113-challenge catalogue. Adding Aikido closed the dependency-CVE blind spot and dented the secrets one. What survives:

End-to-end business-logic exploitation. Audn surfaces IDOR component leads; the multi-step chains (GDPR erasure flow, forged-coupon end-to-end, premium unlocks) still aren't demonstrated end to end by anyone.
Client-side / runtime front-end depth. All three are code-centric. Angular DOM sinks beyond those flagged, live CSP behavior, and client routing/auth remain under-explored.
Live infrastructure / edge posture. Codex and Aikido are static; Audn ran live but left its transport/header/container items either unreached or at no-verdict. The real TLS config, the actual security headers the edge returns, and container hardening are still unaudited.
Exploitability of the static majority. All 20 Codex + all 36 Aikido + 83 of Audn's findings are unverified against a live target.
Complete SCA / SBOM. Even with Aikido, there's no full transitive dependency graph, license posture, or signed SBOM — and the lockfile gap means known-vulnerable transitive packages can still hide.

That fourth item is the honest summary of the whole exercise: of 159 findings, 18 are demonstrated facts. Everything else is a hypothesis of varying quality. Anyone selling you a single tool as complete coverage is selling you the gap.

What's Under the Hood

WhiteBox runs on Necromicon — our frontier cyber model built on Kimi K3, abliterated so it doesn't flinch mid-engagement and fine-tuned on real pentesting sessions rather than CTF puzzles. The live-reproduction step is the whole reason it exists: a guardrailed model refuses to fire the payload, and a finding you never fired is a finding you can't grade.

Three ways to get the same engine:

audn.ai/whitebox — connect a GitHub repo, get the scan-attack-triage loop end to end. This is exactly what produced the Audn column above.
penclaw.ai — self-serve autonomous cyber harness; BlackBox against a live target with no source.
platform.audn.ai — OpenAI-compatible API if you want to build your own validation workflow.

And the base model is open. Free-tier weights are on HuggingFace at audnai/penclaw-Kimi-K3.0-abliterated-GGUF — run it on your own hardware, verify the abliteration, and check our claims without asking our permission.

The full interactive A/B/C — all 159 findings, the coverage matrix by vulnerability class, the three-way overlap clusters, the recommended pipeline, and every con listed against our own name — is at blog.audn.ai/audn-vs-codex-vs-aikido. Sources: Audn's report (103 findings, live target on Render), Codex's report (20 findings, commit 160f306) from the official OpenAI Codex security review, and the Aikido repo scan (36 issues) — all on ozgurozkan123/juice-shop. Overlap is mapped at issue-area level; blind spots are cross-referenced against Juice Shop's 113-challenge catalogue. If you think we scored something wrong, tell us: support@audn.ai.