crimes v0.26.0 — one charge, one answer (and two broken instruments)

Theme: 0.25.0 left five debts. All five are closed. But the release’s own thesis — that reconciling the cross-language scoring gaps would produce the first real movement in the deterministic metric since 0.24.0 — turned out to be wrong, and finding out why produced the two changes that matter most.

schema_version stays at 0.7.0. No field is added, renamed or retyped.

Zero agent invocations across the entire release. Every claim below is from evals:ranking, a ten-repository corpus, or the stored responses of runs already paid for.

---

The headline is a bug nobody was looking for

crimes silently lost the entire churn signal when the scan root was a symlink.

git log -- <pathspec> only matches paths git has committed. A symlinked root produces a pathspec that exists on disk and not in history, so the log matched nothing, every entry was dropped, and the caller received gitAvailable: true with an empty file list. Churn 0 for every file — presented as a measurement rather than as a failure, which is why it survived.

scan  evals/fixtures/01-messy-ts-app   42 findings, churn>0:  0
scan  examples/messy-ts-app            42 findings, churn>0: 42  (0.05–0.20)

hotspots via the symlink    git_available: true, every change_count 0
hotspots via the real path  git_available: true, src/billing.ts: 4

Any user scanning through a symlinked checkout, a workspace link or a mounted path lost the signal the same way, and git_available: true told them nothing was wrong.

It also meant 70% of the project’s own headline eval metric was computed with churn pinned to zero, because fixture 01 is a symlink. Fixing it moved mean_ndcg_deep 0.3449 → 0.3468 — the first real movement since 0.24.0, and not from the work that was supposed to produce one.

The measurement that was measuring the calendar

evals:ranking is documented as “deterministic — there is no noise band, so any delta is real”. It was a function of wall-clock time.

rank_score = agent_risk * (1 + recency * 0.5), and recency is 1 within 7 days of a file’s last commit, decaying to 0 at 14. Same build, same fixture, varying only the calendar:

CRIMES_NOW=2026-08-11   recency [0.57, 1]
CRIMES_NOW=2026-08-12   recency [0.43, 1]     ← one day later, already different
CRIMES_NOW=2026-08-26   recency [0]           ← and the finding order changes

Two runs a fortnight apart would have reported a re-ranking no build produced. buildScoringContext now takes its reference date from CRIMES_NOW when set, the eval runner pins every fixture scan to a committed constant, and the report records reference_date so a comparison across two dates cannot pass as a scanner change. Product runs leave it unset — a user’s repo really is a function of today.

---

D1 — one charge, one answer

Eight charges are implemented in both language packs. Seven disagreed. Four independently-attributable changes, each measured on the corpus alone before landing.

Two detectors could not respond to their own evidence. circular_dependency and deep_import expressed no intrinsic on the universal side, so every finding took a flat declared value however much evidence there was. On hono a 4-file import cycle and a 2-file one both scored agent_risk 0.09; on cal.com, 478 deep_import findings scored identically whether the file reached into one deep package path or twenty-eight. Both now ramp: 204 corpus findings move, none appears or disappears.

Two “unported improvements” could not be ported. Python’s direct_date raises its base for a naive datetime — a value carrying no offset, which JavaScript has no equivalent of at all. Python’s sync_io_in_hotpath raises its base inside an async def, on the grounds that stalling the event loop is worse than stalling one worker in a pool — and Node has no pool, so a readFileSync blocks the one event loop either way. Copying that across would have scored a difference that does not exist. Both entries keep their place with the argument written down.

Four constant gaps closed, and it was not a matter of taste. detector-defaults.ts publishes 29 expressed bases that every entry in INTRINSIC_DEFAULTS was anchored against. The disputed charges all appear there with their universal value; the Python bases appear nowhere. The direction was decided by which number has dependents. 1,457 findings move, none added or removed.

And one pair was hiding something bigger. commented_out_code’s language-js twin gated at statementCount >= 5 and then scored 0.48 + statementCount * 0.04 — so its published base of 0.48 was unreachable, and all 463 corpus findings carried exactly 0.68 or 0.72. Every peer calibrated against “0.48 commented_out_code (js)” was calibrated against a number no report contained. Both twins now share one ladder over one unit; the intrinsic distribution goes from {0.35, 0.68, 0.72} to {0.48, 0.51, 0.57, 0.60} — one population, and it finally ramps.

KNOWN_SHAPE_GAPS and KNOWN_SAME_DIR_DISAGREEMENTS are both empty.

D5 — the noise band was wrong, and backwards

The documented band was ±6pp claude / ±3pp codex, from three samples at 0.12.1. A release that moved no finding on any fixture exceeded the codex figure on the first try.

Re-derived from two repeat pairs — release boundaries where the agents received byte-identical input — using per-scenario variance rather than aggregate samples, which is 48 estimates instead of 2:

agent	was	is
claude	±6pp	±5pp
codex	±3pp	±7pp

Codex is the noisier agent in aggregate, not the steadier one. Three points cannot tell σ 0.012 from σ 0.031.

A scorer defect was inside the band. Charge names were matched case-sensitively, so an agent writing “Permission IA drift” instead of “Permission IA Drift” scored zero on a finding it had named correctly. 0.25.0’s headline result — codex −5.1pp on identical input — is −3.2pp under the fixed scorer.

The honest conclusion: at 48 scenarios structural_pass_rate resolves a ~5pp move. Resolving 2pp needs roughly 283 scenarios for claude and 468 for codex. A full agent run is a smoke test on the wire output, not a release’s evidence.

D2 — every differentiated detector now has a scenario

Nine detectors were referenced by no scenario. Two already fired on an existing deep fixture. Six of the remaining seven already had fixture content — in examples/risky-service, the 0.16.0 correctness fixture, which was never registered as an eval fixture. Registering it plus one new file closed the debt.

Fixture 15 is deliberately shallow (22 findings), so seven scenarios were added without making the headline mean incomparable.

D3 — closed with a measurement, no code

.gitattributes linguist-vendored, tsconfig exclude and .eslintignore were costed on hono, cal.com, n8n and posthog before a reader was written. .eslintignore no longer exists (ESLint 9 moved to flat-config ignores), root tsconfig.exclude names nothing crimes already skips, and zero patterns corroborate on any of the four.

The corroboration rule turned out to be load-bearing rather than a formality. n8n’s .prettierignore ends with a # Handled by biome comment and a glob covering every .ts file — read as a maintenance claim, that excludes all 18,783 of its TypeScript files.

Which gives the rule: a tool’s preferences need corroborating; a provenance claim does not. So linguist-generated — the repository asserting who wrote its own file, the same claim an @generated banner makes — is now honoured from a single source and joins the existing generated-code policy. On posthog that removes 69 findings across four files the path heuristics could not recognise: 43 large_function, 8 exact_duplicate_block, 5 negative_flag_maze and more, on a machine-written validator accused of having a God Function and badly-named booleans.

D4 — the test-only bucket, decided by looking at it

sync_io_in_hotpath.py exempted guarded modules nothing references; the bucket referenced only by tests was left reported as an open judgement. Inspected rather than reasoned about, it is 63 findings in 36 modules across four repositories and every one is developer or CI tooling — pre-commit hooks, CI analysis scripts, release tooling — each referenced exactly once by its own unit test in a mirrored test tree. Now exempt.

---

Still unsettled

weak_test_signal is not a constant gap. The last entry in KNOWN_DISAGREEMENTS, carried for nine releases as “needs its own argument”. The two detectors do not emit the same thing: universal is one finding per hollow test, Python is one finding per file — 3.19 findings per affected file against exactly 1.00. Reconciling the constants first would compare a per-test judgement with a per-file one. The granularity has to be chosen before either number means anything, and that changes finding counts rather than scores.

The parity gate compares {base, step, cap} and was structurally unable to notice. Its doc comment now says so.

The recency multiplier is a bet, and it is now measured. With churn fixed, recency was 0 on all four deep fixtures — correctly, since they are months old — so rank_score on the deep set was agent_risk, and a multiplier larger than any single agent_risk input was invisible to the metric that validates ranking. On a real repository it is anything but invisible: toggling it on posthog moves 99.9% of 14,181 findings, median displacement 534 places, and makes the top-20 100% recency-boosted.

16-recency fixes the blindness. It is the only fixture with git history — evals:setup builds it from RANKING_REFERENCE_DATE, so its age tranches hold instead of decaying with the wall clock — and it is deep enough (34 findings) to count. Its two scenarios test the term’s premise from both sides rather than assuming it:

plan-16-checkout-rollout    answer in src/checkout/ (3 days old)
   recency on  0.844 (rank 1)   off 0.425 (rank 15)   +0.418
review-16-whole-repo-audit  answer in src/legacy/ (90 days old)
   recency on  0.327 (rank 12)  off 0.456 (rank 5)    -0.129

So the term is not a mistake. It is a strong, unlabelled bet that the reader cares about what the team is currently touching, and it pays about three times more when that is right than it costs when it is wrong. What is genuinely open is whether the default should make that bet silently, and whether --no-recency deserves to be more discoverable than a flag nobody knew to reach for.

The level 0.3 remains unvalidated, the class table still has zero standard members, and linguist-generated is the only JavaScript tooling signal read — the other three are closed by measurement rather than by implementation.