crimes v0.23.0 — the score's missing inputs

Theme: agent_risk is the score that makes crimes something other than a linter. Its heaviest term is the detector’s own judgement about how badly a finding will mislead an agent. 28 of 70 detectors were not making one.

They were not scored as unknown, either. They were scored below every deliberate judgement any detector author had ever made — including contract_drift, swallowed_error, duplicated_policy and permission_ia_drift, which are among the most agent-hostile charges the tool makes.

schema_version stays at 0.7.0. No field is added, renamed or retyped, and no finding is added or removed anywhere on the corpus. Scores and ordering move.

---

The defect

agent_risk = 0.40*intrinsic + 0.20*churn + 0.20*test_gap + 0.20*blast_radius

intrinsic is the detector’s own scores.agent_risk — the only genuinely agent-specific input in the system, and the place PRD §10’s “multiple sources of truth”, “misleading name” and “hidden side effect” are encoded. A detector that sets none falls through to NEUTRAL_INTRINSIC, which the source described as saying “what is actually known: nothing”.

Measured across the corpus, that is not what it said:

• the 29 expressed agent-signal bases run 0.35 to 0.80
• NEUTRAL_INTRINSIC is 0.30 — below all of them

So silence was not neutral. It was a verdict of “less agent-hostile than the most lenient charge in the tool”, entered by nobody, on 28 detectors.

Nothing enforced the field, which is why it accumulated quietly. The class it lands findings in, standard, has zero members across all 70 detectors — so the fallback path was invisible in the class table too.

The band the ceiling was fitted to does not exist

STRUCTURAL_CEILING = 0.3 is justified in the source like this:

the agent-signal band runs 0.31–0.53 (sync_io_in_hotpath 0.43–0.53, direct_date 0.51, commented_out_code 0.41, contract_drift ~0.36) … 0.3 puts the whole structural class at or below the bottom of the agent-signal band.

This release rebuilt ce0ccab — the commit that chose the constant — and scanned the exact tree its comment cites:

type	claimed	measured at ce0ccab
sync_io_in_hotpath	0.43–0.53	0.20–0.53
direct_date	0.51	0.18–0.51
commented_out_code	0.41	0.14–0.41
contract_drift	~0.36	0 findings — does not fire on that tree
the band itself	”0.31–0.53”	min 0.12, p50 0.35, max 0.53

Every quoted figure is that type’s maximum — the band was read off the head of each type’s distribution rather than off the distribution. 45% of the agent-signal population sat at or below 0.30 on the day the constant was chosen; it is 47–75% across the corpus today.

The anchor was circular as well. contract_drift expresses no intrinsic, so its position in “a large_file still outranked a genuine contract drift” was the fallback being argued about.

The constant is unchanged. Correcting a rationale does not choose a different number, and this release deliberately changed the inputs rather than the mechanism so the two are not confounded in one baseline.

What shipped

INTRINSIC_DEFAULTS — one table, a declared intrinsic for each of the 28, every value anchored to a named expressed peer with the anchor written beside it. One table is the point: intrinsics can only be calibrated against each other where they can be seen next to each other.

A standing gate. detector-defaults.test.ts reads the detector sources and fails when a registered detector expresses neither its own intrinsic nor a declared one. It reads source rather than carrying a hand-written list, because a list cannot see a detector added tomorrow — which is exactly the hole that let 28 accumulate.

NEUTRAL_INTRINSIC stays at 0.30, deliberately. Built-ins can no longer reach it, so reaching it now means a detector is missing from the table. Raising it to the expressed median would make that omission harder to notice.

Measured effect

Deterministic (evals:ranking — scan-only, no agent, no noise band), split by whether a scenario’s labelled answer is one of the previously-suppressed types:

bucket	n	mean nDCG	up	down
labels a previously-suppressed type	12	0.5441 → 0.6213 (+0.0772)	7	1
labels only always-expressed types	33	0.4510 → 0.4458 (−0.0053)	0	22

The second row is displacement, not regression: nothing went up, and the drops are uniform and tiny. 28 detectors that could not previously surface now do, so everything else shifts down a rank — and those labels were chosen while those 28 were suppressed.

On real repositories the change is conservative. It moves a head only where the suppressed detectors actually fire, and it reduces over-concentration where it does:

repo	top-20 dominant, before → after	concentration lift
hono	option_bag_junk_drawer 5/20 → swallowed_error 6/20	6.00 → 2.80
mlflow	sync_io_in_hotpath 11/20 → 10/20	2.85 → 2.59
pydantic, drf, zulip/zerver	unchanged	unchanged

Concretely, on hono’s largest file the two hidden-failure findings — swallowed_error and unsafe_retry — now sit above three God Function / God File findings that previously buried them.

Eval baseline

agent	0.22.0	0.23.0	move
claude	0.81	0.81	0pp
codex	0.58	0.63	+5pp

Do not read codex’s +5pp as this change working. It clears codex’s ±3pp band, but structural_pass_rate matches a detector’s literal id in the response text, and this release changed which findings rank highly — so it changed which ids an agent is most likely to quote. That is a plausible mechanism for a move that says nothing about whether the findings got better.

More importantly, the accidental repeat sample at 0.22.0 measured this directly: on provably identical input, 13 of codex’s 48 scenarios moved anyway, with full 0.00 ↔ 1.00 swings. One sample cannot separate +5pp from that. The scan input did change here, so this is a real measurement rather than a resample — but it is a single one.

The deterministic instrument is the one to read for this release: no agent, no noise band, and the split by label above.

per_scenario_kind is recorded in summary.json and deliberately not quoted — at 7–8 scenarios per kind, one flipped assertion moves it 8–15pp.

Two things measured and deliberately not changed

The mechanism. A monotonic squash of the structural class (min(scored, 0.3) → scored * 0.3) was implemented and measured: 13 up / 0 down on the differentiated bucket, and structural findings out of the top 20 entirely on four of five corpus repos — including pydantic, whose top 20 still leads with large_function under the ceiling, and drf’s 15/20. So the ceiling is not doing the job its comment claims. It was still not taken, on attribution grounds: two findings-moving changes in one baseline make neither attributable. The evidence is recorded for whoever picks it up.

Concentration. zulip’s “16 of 20 sync_io_in_hotpath” is stale — it is 12 of 20 — and mostly not the ranking’s doing. Against the population the head is drawn from, zulip’s lift is 1.20: the head is very nearly a faithful sample of a repo that genuinely has a lot of blocking I/O in Python. Where lift is high, the cause is an uncalibrated intrinsic, so the question collapses into the one this release fixed.

Also in this release

• files_partial_parse no longer overstates itself. The warning claimed “detectors that judge a whole file declined to run on it”, which is true of language-py and false of the JS pack, whose detectors deliberately keep running. The copy is now split per pack. Fixed on main after 0.22.0 shipped and held back rather than cut as a patch for one sentence.

Still open

• Ceiling vs monotonic squash — measured above, undecided.
• The class table is hand-maintained, and standard still has zero members: it is an unlabelled-default bucket, not a third category.
• The same charge carries different intrinsics in different packs — sync_io_in_hotpath is 0.55 in JS and 0.50 / 0.70 in Python.
• The 41 expressed intrinsics are still literals inside their detectors, so that half of the calibration is still spread across 41 files.

Full evidence in docs/calibration-followups.md § “0.23.0 — the intrinsics were never calibrated”.