One engine. Cross-validated.
Every audit runs the same 58 deterministic scanners, then the WPHK Engine reasons on top — security findings cross-validated by two independent models. And we show our workings: real plugins, real findings, published evaluation reports.
WPHK Engine · two-model security cross-validation · benchmarked monthly
One engine. Every audit.
Security engine
Two frontier models audit security independently. Findings both models corroborate are promoted to HIGH confidence; single-model claims are marked for review.
Quality & accessibility engines
Architecture, maintainability, WordPress best practices, documentation, error handling, and WCAG 2.1 AA / EAA accessibility.
Theme & performance engines
A seven-phase theme audit for theme ZIPs, plus optional server-load analysis when the performance engine is selected.
Live results live
A fixed corpus of plugins with known-flaw ground truth, run monthly against the frontier models. Click a model to see its per-fixture card; click a column to sort. Methodology and latest results are public via /api/v1/benchmark.
Corpus v1.0: 8 fixtures with known vulnerabilities (SQLi, XSS, CSRF, REST permission gaps, capability checks, hardcoded secrets, LFI, nonce-missing AJAX) + 3 clean fixtures with correct code. Public API: /api/v1/benchmark
Why these models, in this engine
Model composition isn't a marketing decision — it's the benchmark table above, applied. Models earn their slot on recall and false-positive discipline, and they can lose it the same way.
Kimi K3 + DeepSeek v4 Flash
Two models audit security independently. Findings both corroborate get HIGH confidence; single-model claims are marked for review. Cross-validation is how the engine kills hallucinations without slowing every audit.
DeepSeek v4 Flash
The disciplined workhorse: 100% recall AND zero false positives on clean code, at the lowest cost on the board. When the cheapest model is also the most careful, it runs everything that isn't security.
No permanent seats
A model that can't beat the incumbent on recall and FP discipline doesn't get promoted — no matter what its marketing says. Every candidate runs this corpus before it touches your audits.
Nobody gets promoted on reputation
When a new frontier model ships we run it through this page's corpus before it touches production — Claude Opus 5 (10 FPs on release week) and Claude Fable 5 (0% recall, cyber filter) are both still on the ladder as proof that nobody gets promoted on reputation.
Oiko plugins, every model, pinned commits live
The seeded corpus measures planted-bug recall. This is the other half: every model against five production plugins (boost, fields, forms, guard, sessions), pinned to a commit SHA per run. Severity chips are finding counts per model per plugin — the honest picture of how output varies on real code, and what a full audit costs at each plugin size.
oiko-boost · 24 auditable files · 17 models · pinned a0d76a6
5 production plugins · every model on the ladder · pinned commit SHA per run.
What the numbers mean
Every metric above comes from the same monthly run — same corpus, same prompts, raw engine output before our production false-positive filters.
Recall
Of the vulnerabilities we deliberately seeded into the corpus fixtures, what fraction did the model find? 100% means it caught everything — every SQL injection, XSS, open REST route, and hardcoded secret we planted. This is the headline quality number: a model with low recall is auditing blind.
Clean FPs (false positives)
Findings the model reported on fixtures that contain no bugs at all — correct, well-escaped WordPress code. Every FP is a false alarm a developer has to read, triage, and dismiss. High FP counts destroy trust in the tool, so we weight them as heavily as recall. Zero is the only good score here.
Cost (USD)
What the full corpus run cost at provider list prices, input + output tokens. This is what determines whether a model can run every audit or only makes sense as a security cross-validator. Costs on /models are shown in USD because that's how providers bill us.
Latency
Wall-clock seconds for the corpus run. Fast models (under ~10s) work for interactive uploads and CI gates; slow models (60s+) only work when the user is waiting for a deep review anyway. Reasoning models trade latency for recall — that's the roster decision in one number.
Corpus & fixtures
The benchmark runs a fixed set of 11 small plugins: 8 seeded with known vulnerabilities (SQLi, XSS, CSRF, REST permission gaps, capability checks, hardcoded secrets, file includes, nonce-missing AJAX) and 3 written correctly. Because the ground truth is known, we can measure exactly what each model catches and what it invents.
How the roster uses this
Models earn their slot in the WPHK Engine. A model that can't beat the incumbent on recall and FP discipline doesn't get promoted — no matter what its marketing says. When a new frontier model ships, we run it through this corpus and publish the results here before it touches your audits.