crimes v0.19.0 — the backlog release

Theme: eight months of work reached main and stopped. This release is the queue draining. It is the largest span the project has published — 50 commits, ~30 defect fixes, four features, two schema_version bumps, and one change to how the package installs.

It also carries a lesson worth stating up front: 0.18.0 through 0.18.4 were never published. They are internal eval-baseline markers, which the versioning policy creates by design — a findings-moving change gets a patch bump and an eval re-run without cutting a release. That is a good policy and it worked exactly as written. What it does not do is push anything to npm. Five markers accumulated, and for that whole period the fix for a detector that reported 8,019 false findings on airflow existed only in this repository.

Nothing below is new work. It is work that was already done and that nobody could install.

---

Read this first if you consume the JSON

schema_version moves 0.4.0 → 0.6.0 — two bumps in one release.

0.5.0: the blast-radius integer was mislabelled

• Renamed: scores.blast_radius_importers → scores.blast_radius_transitive_importers.
• New: scores.blast_radius_direct_importers.

The old name promised “N files import this” and delivered the transitive closure — every file that can reach it. Those diverge by more than a rounding error: on hono, src/utils/mime.ts has 5 direct importers and a closure of 240, and six files in the core component all report exactly 197 while their direct fan-in ranges from 2 to 70.

If you read blast_radius_importers, rename the key — and look hard at whether you wanted blast_radius_direct_importers instead.

0.6.0: every finding carries its own fingerprint

• New required field: fingerprint.
• New optional field: score_rationale.

The fingerprint is the handle four commands accept — crimes ignore, crimes unignore, crimes feedback, crimes triage — and the JSON did not contain it. id is positional and only means something inside the report that produced it, so a consumer wanting to act on a finding had to rebuild the fingerprint from the other fields and hope its construction matched fingerprintFinding, discriminator rule included. That is precisely the part a reimplementation gets wrong.

Nothing was renamed or removed in this bump, so a consumer that ignores unknown keys needs no change. additionalProperties: false validators and schema_version === "0.5.0" hard-checks need updating.

Full field-by-field notes: Migrating from 0.4.0 to 0.5.0 and 0.5.0 to 0.6.0.

Pinned suppressions and baselines: twelve detectors need re-recording

The fingerprint-collision work continued past what 0.17.0 started. Entries naming a finding from any of these stop matching:

commented_out_code, weak_test_signal, anonymous large_function, unbounded_async_fanout, swallowed_error, contract_drift, logic_in_comments, duplicate_component_shape, name_behavior_mismatch, duplicated_role_status_plan_check, negative_flag_maze, return_shape_roulette.

For the symbol-bearing ones only the ambiguous findings move — pins on uniquely-named symbols are untouched.

$ crimes feedback recheck

surfaces the affected entries, and now carries a per-detector note for all fifteen changed detectors rather than falling back to “detector behaviour unchanged” for three of them.

Re-recording actually works now. Before 1499b5e, crimes ignore on a discriminated finding was a silent no-op by id and a hard reject by fingerprint — every finding 0.17.0 gave a discriminator to was unignorable by both handles it accepts. That is the kind of defect that only shows up when someone tries to use the feature.

And so does the hint. Found while cutting this release: feedback recheck looked up its per-detector note by the current minor exactly, so every note went unreachable the moment the next minor shipped. All fifteen 0.17 notes had been orphaned since 404e581 — anyone who ran recheck on any 0.18.x build was told “detector behaviour unchanged” about the fingerprint change that had just invalidated their pin.

A release is not the unit a user upgrades across; a span is, and this release is the proof — 0.18.0–0.18.4 were never published, so every real upgrade path crosses two minors of changes at once. The lookup now takes the pin as well as the current version and returns every note in between, oldest first:

$ crimes feedback recheck
[1/2] commented_out_code — src/a.ts
      Marked fp in 0.16: "licence header, not dead code"
      In 0.19: 0.17: Fingerprints now carry a hash of the comment block as a
      discriminator … 0.18: Matching now requires code syntax rather than bare
      code-ish words, so prose no longer fires. Likely resolved if your pin was
      on a licence header — airflow went 8,019 findings to 45.

Nine detectors that changed behaviour in the 0.18.x span gained notes at the same time; they had none, because the map had only ever been written for fingerprint changes.

This is the third instance of the same shape in this codebase — apparatus that fails closed on correct input — after the eval scorer’s extension list and the biome guard’s summary regex. When you write a lookup or a check, ask what a correct input that it rejects would look like.

---

The install is clean again

On npm ≥ 11.18 the only crimes-specific output on a fresh install was a security warning:

$ npm i crimes
added 2 packages in 990ms
npm warn install-scripts 1 package had install scripts blocked because they are not covered by allowScripts:
npm warn install-scripts   crimes@0.17.0 (postinstall: node ./scripts/postinstall.mjs)
npm warn install-scripts   Run `npm install-scripts ls` to review, or `npm install-scripts approve <pkg>` to allow.

npm now blocks install scripts by default and asks the user to approve arbitrary code execution. What crimes was spending that prompt on: seven lines of welcome text. Its own comment had already conceded the point — “npm 7+ swallows postinstall stdout/stderr … Most users will never see this message … The bare crimes invocation in the CLI itself is the reliable surface.”

So a tool whose entire pitch is trustworthiness was asking for an arbitrary-code-execution approval in exchange for a message nobody saw. The script is gone. crimes with no subcommand still prints the whole onboarding, which is what was doing the work all along.

Verified end to end under npm 12.0.2: the published 0.17.0 reproduces the warning, the 0.19.0 tarball installs as a bare added 2 packages in 889ms. The smoke test now reads the installed package.json and asserts no preinstall / install / postinstall / prepare — the warning is triggered by the manifest, not by the file list, so asserting only on tarball contents would not have held.

---

Detector fixes, in order of how much they were costing

Every one of these was reproduced on a real repository before it was touched, and the number below is measured, not estimated.

commented_out_code matched English prose — airflow 8,019 → 45

41.2% of a 17,745-finding report. 7,320 of the 8,019 were the Apache licence header, because the code-token list was bare words and the header contains three of them: “for additional information”, “may not use this file”, “the License for the specific language”. Matching now requires code syntax. The whole airflow report drops to 9,771.

680 further non-licence blocks stopped firing; two sampled at random were both false positives (a licence header behind a shebang, a Go package doc comment).

parallel_destination — 2,819 findings from 134 files, now gated off

52.8% of n8n packages/frontend/editor-ui. It is the first detector to ship behind Detector.defaultOff; that package’s report drops 5,342 → 2,523. A gated detector announces itself on stderr even under --no-color, because the user did not make that choice — we did.

pass_through_abstraction fabricated chains from method names

n8n reported a 0.98-confidence chain that starts at Set.prototype.has and joins four unrelated registries, each delegating to its own private Map. A member call (this.repo.delete(…)) is no longer followed at all — its tail names a method on an object whose type was never read — and a cross-file step now requires the target to be exported. n8n packages/cli: 13 findings / 7 chains, all 7 at confidence ≥ 0.9 → 6 findings / 0 chains.

Never resolve a symbol by name alone is now a standing rule in this codebase, and it came from here.

cross_language_route_drift was confidently wrong

One HIGH-severity finding on PostHog accusing /api/projects/* of drifting from a Stripe mock. All five decorator-routed Python files in PostHog live in services/stripe-mock, against 90 DRF router.register calls the detector cannot see. An orphan is now reported only when the backend declares at least one route sharing its first path segment.

Match count would have been the wrong test — the detector’s own canonical positive has zero matches too.

scope-class — 15 airflow files carrying 44 findings were never reportable

The serious half was not the classifier but the enforcement: ~50 detectors predate isNeverReportable and never asked it. The policy is now enforced once in scan, so a detector added tomorrow inherits it. airflow 44 → 0, drf 2 → 0. Classifier also widened for _pb2.py / *.pb.go and for minified bundles by name — drf’s prettify-1.0.js needed content sniffing because it is minified but not named minified.

boolean_naming_drift flagged published API as “quick” to rename

Confirmed on drf and pydantic: many (Serializer(many=True)), public, coerce_to_string, strip_whitespace, fail_fast, repr — all public API, all offered at effort: "quick". Scoped to unannotated locals inside a function, the only place its own rationale holds. drf 7 → 1, pydantic 34 → 20.

The annotated case was self-contradictory: the detector’s own suggested fix reads “rename it, or add a : bool annotation”.

sync_io_in_hotpath described the wrong span

Findings are now one per enclosing function, so symbol, lines and evidence describe the same code. airflow span median 8 → 1 line, max 4,196 → 185. Django management commands and @cache-decorated functions are exempt. Splitting per function multiplied volume 494 → 1,108, capped to the 3 worst per file at 811.

hotspots ranked manifest churn first

hono’s #1 went from package.json (risk 0.72, 29 changes, one low finding) to src/adapter/aws-lambda/handler.ts. The row set had been the union of every churned path with every file carrying a finding; it is now restricted to files scan can report on. A ranking_note states when the order is really localeCompare.

Python test detection, in three parts

• Nested test_* functions were counted as tests. zulip zerver/tests: 17 of 40 claimed silent tests were nested functions (42.5%), and two files left the report entirely. The exclusion is drawn at any enclosing function, because that is where pytest’s own collection boundary sits.
• pytest.warns(...) and @pytest.mark.xfail were not credited. pydantic tests: 22 files / 76 claimed silent tests → 15 / 45.
• Cross-file assertion helpers were unresolved — see the feature section below.

Also fixed

• dependency_provenance_gap and tsconfig path aliases. Two causes, both confirmed on cal.com: . and .. slipped a relative-path guard that required a trailing slash, and aliases were only read from a root tsconfig cal.com does not have. Now collected from every tsconfig under apps/ / packages/ / libs/ / services/.
• verdict and diff failed on master-default repos. refs/remotes/origin/HEAD is consulted first — guessing by name order would silently compare against the wrong branch on a repo with both. The two-scan cost is halved by reusing the base scan when both refs resolve to the same tree: hono verdict --base HEAD 12.3s/15.8s → 7.0s/7.1s.
• diff human output was three integers with no locations. It now says where.
• explain exited 2 on asset findings. Asset detectors live in their own registry explain never searched, so oversized_raster, raster_should_be_vector and svg_with_embedded_raster were all unexplainable. The crimes ignore line it prints is now single-quoted, and was run end-to-end against a my src/big file.ts fixture to prove the pasted command works.
• Findings-per-file inside a group were uncapped. n8n’s instance-ai.service.ts listed 41 numbered findings under one heading; now 8, with the true tally on the heading line.
• --all, --flat and --top were silent no-ops under --format json. They now say so on stderr, leaving stdout byte-identical. Teaching the machine contract to withhold findings by default would have been the wrong fix.
• Five source extensions could not fire a rule at all. .json, .yaml, .txt, .rst, .adoc were missing from DEFAULT_SOURCE_INCLUDES — two lists had drifted. airflow 9,874 → 9,981 findings; the docs budget now reaches 28 files including RELEASE_NOTES.rst. A test pins the two lists in step, since the drift was the defect.
• lines was absent on duplicate-block findings that had the range in their evidence all along (Item.tsx:23-33). 46 of 46 now set the field.

---

Features

agent_risk stops being a length ranking

PRD.md §10 says agent_risk must not collapse into severity. It had. Severity is no longer an input at all — the fallback intrinsic derived from it was the collapse, for the ~18 detectors that set none of their own — and findings are classified structural / agent_signal / standard with a ceiling on the first.

Measured on the top 20 by rank: zulip zerver structural 18 → 0, Python 0 → 20 of 20; hono structural 14 → 0 with distinct types 8 → 10.

The ceiling had to be measured, not guessed. 0.4 left large_file outranking contract_drift because it sat inside the agent-signal band (0.31–0.53); 0.3 is that band’s floor.

Recorded honestly: zulip’s top 20 is now 16/20 sync_io_in_hotpath, which is a concentration of its own. The score is no longer a length ranking, but its shape is unsettled. What is measured and what is merely believed are separated in docs/calibration-followups.md.

blast_radius on a log scale

The old score saturated at 1.0 on 47% of findings — a score that is pinned for half the corpus cannot rank. Now log-scaled with a direct-fan-in tiebreaker.

A Repo-level section, so whole-repo findings are visible

On n8n packages/cli this surfaces five findings that were in the JSON and nowhere in the human view, including dependency_provenance_gap on package.json and agent_permission_sprawl on AGENTS.md. A section rather than a scoring boost: the problem was the grouping, not the ranking.

A repo-wide Python symbol index

weak_test_signal can now follow a call into another module, resolving through the MRO and the importing file’s own imports — never by name.

	before	after
zulip weak_test_signal	162	152
zulip files reporting	48	38
zulip test_message_delete.py	1	0
airflow weak_test_signal	435	380 (−12.6%)
airflow files reporting	326	271 (−16.9%)
airflow claimed-silent tests	634	462 (−27.1%)

Cost: airflow 97.1s → 99.4s over three samples each, against a ~12s run-to-run spread — no measurable regression.

Caught before shipping, by opening the files named in the evidence: three zulip test files were being credited through zerver/actions/*.py, where do_set_realm_property asserts isinstance(raw_value, property_type) about its own argument. That is a precondition defending a production function from its caller, not a test checking a result — and a test that calls it and checks nothing else is exactly the hollow test this detector exists to report. Crossing a file boundary silently broke an assumption same-file resolution never had to state: a test file’s own functions are test infrastructure by construction. Cross-file credits now require the helper to live in test infrastructure.

score_rationale, so evidence goes back to being receipts

How confidence and severity were arrived at, as a base plus named deltas. 17 call sites across 10 detectors. evidence is for things a reader can check; arithmetic is not one of them.

Detector gating

Detector.defaultOff, and detectors.enable naming a gated detector is additive — see the safety fix below.

---

The worst-shaped defect in this release

crimes scan prints, when a gated detector sits out:

crimes: parallel_destination did not run (off by default).
        Enable with "detectors": { "enable": ["parallel_destination"] }.

enable was a pure allowlist. So following that advice verbatim turned off all 68 other detectors and the entire asset pass, with no warning. Measured on the 05-stress-ia-drift fixture: 13 findings become 1.

The tool’s own remediation advice silently gutted the scan. For a product whose entire value is being trustworthy about what it did and did not look at, that is the worst shape a defect can take.

Naming a gated detector in enable is now additive; only default-on ids form the allowlist. The original ordering comment in applyEnableDisable was right about the case it considered — an unrelated enable list must not resurrect a gated detector, and still does not. What it did not consider was a list naming nothing but a gated detector, which is exactly what the hint tells users to write.

Also found: nothing documented that any detector ships gated. docs/packs.md and docs/finding-types/ia.md now do.

---

Three things deliberately not changed

The queue is a list of hypotheses, not a list of tasks. Ten entries turned out to be wrong; two were wrong at the premise, where acting would have made things worse. Those are recorded in docs/dogfooding/2026-08-03-remediation.md §4 rather than quietly dropped.

mixed_utc_local_methods cannot fire on modern Python — no change made. The entry’s premise is wrong and acting on it would have been a large regression. On airflow, 728 of 740 utcnow() receivers are timezone, and airflow_shared.timezones.timezone.utcnow() returns dt.datetime.now(tz=utc) — timezone-aware. That is not the naive-UTC trap this detector charges; it is the fix it recommends. Matching any <x>.utcnow() would have produced ~728 high-confidence false positives. Airflow’s whole tree has exactly one datetime.datetime.utcnow() and it is inside a comment. Zero findings is the correct answer, and a regression test now pins that decision.

pnpm run build build-ordering — does not reproduce. 8 runs, 3 from a cleared dist and 5 incremental; 8 of 8 carried the change. One trap for whoever re-tests it: an unused exported constant is the wrong marker, because esbuild tree-shakes it out of the CLI bundle, which looks exactly like a build-ordering failure.

blast_radius score saturation was left alone while the reporting was corrected. The score’s calibration is a separate decision from the integer’s honesty.

---

Known misses, recorded rather than fixed

• if __name__ == "__main__" scripts classify as domain, so sync_io_in_hotpath.py fires on them. No signal distinguishes them from real domain code without reading module-level control flow.
• pydantic/v1/ is a bundled legacy copy scope-class does not recognise. No general rule separates it from any other v1/ API directory.
• large_file counts blank lines. Fixing it drops every number 15–25% and retunes thresholds repo-wide — calibration, not a bugfix. The evidence line says N lines rather than claiming non-blank, so nothing is currently lying.
• transitiveImporterCount counts a file as its own importer on a cycle. Left deliberately; it is the number blast_radius normalises.
• JS syntax errors have no coverage.warnings[] signal. ts.createSourceFile keeps parseDiagnostics off the public SourceFile type, and reaching it means an internal-API dependency — judged not worth it in a field whose entire value is being trustworthy.
• weak_test_signal fingerprint collisions: 2 of 3,585 on n8n packages/cli, 30 of 3,458 on zulip, 115 of 9,926 on airflow. Two tests with identical titles in one file, so the discriminator cannot separate them. Folding the line range in would fix it and invalidate every pinned weak_test_signal suppression — a migration, not a patch.

---

Tooling

pnpm verify’s lint step had not run for ~16 commits. Biome 2.5.6 aborts its worker pool under memory pressure, prints [warn] Linter process terminated abnormally as its only output — no diagnostics, no Checked N files summary — and still exits 0. verify reported green with lint having done nothing.

The environmental cause is not ours. scripts/biome.mjs handles the part that was: a zero exit without a Checked N files summary is now a hard failure. With lint actually running, those ~16 commits carried exactly one warning. The guard earned its place within the hour by catching a real error in new code.

A ranking-quality metric. pnpm run evals:ranking scores the scan alone — nDCG over the order the scan emitted, against each scenario’s expected findings as graded relevance labels. No agent, so no noise band: any delta is real. On 0.17.1 → 0.18.1 it moved 36 of 45 scenarios by up to ±0.47 where structural_pass_rate moved by noise:

deep scenarios	n	mean nDCG	up	down
expect large_function / large_file	6	0.459 → 0.406	0	5
expect anything else	22	0.325 → 0.347	19	2

That is exactly what the agent_risk change set out to do. The headline mean is +0.006 because it nets the two buckets against each other — the per-scenario table is the deliverable, not the aggregate.

It also says something about the scenarios: the six that got worse are the ones whose labelled right answer is a length finding, and the product has now decided length findings should not lead. Those labels encode the old ranking. Re-labelling them would improve the metric without improving the product, so it was not done.

The eval scorer could not see a correct answer. It knew a finding by its slug, its charge, or its crime_NNNNN id — but not by its own evidence. So CLAUDE.md’s “evidence before judgement” was not being applied to the measurement apparatus. Measured by replay, so the 96 responses are byte-identical and the whole delta is the scorer: 4 of 96 moved, all codex, all from a hard 0 to a full pass. Claude unchanged, which is the evidence nothing was over-credited.

A string earns a place in the evidence index only if it identifies exactly one detector type in that scan. Bare line references, strings under 12 characters, and pure prose are dropped — arrow declaration is a real large_function evidence line and a phrase an agent can write about unrelated code. The prose filter changed no score, which is the point of adding it.

---

Eval baseline

The published baseline for this release is evals/results/0.18.4/ — the last internal marker. Nothing findings-moving landed between it and the tag; the only change is the packaging fix above.

agent	0.17.0	0.17.1	0.18.1	0.18.2	0.18.3	0.18.4	2σ band
claude	0.84	0.82	0.85	0.85	0.82	0.82	±6pp
codex	0.57	0.56	0.54	0.59	0.57	0.61	±3pp

Read this table carefully, because the honest reading is not the flattering one. Every move across the whole span sits inside or beside the measured noise band. The one delta that is attributable is codex 0.54 → 0.59 at 0.18.2, and that was a measurement correction — the scorer learning to recognise an answer it had been marking zero — not a product improvement.

The work in this release removed 8,019 → 45 commented_out_code findings on airflow, 2,819 → 0 parallel_destination on n8n, and a high-severity PostHog finding comparing a frontend to a Stripe mock. None of those repos is in the fixture set, and none of those defects exists at fixture scale. A flat aggregate is not evidence the work did nothing; it is evidence the fixtures cannot see this class of work. The evidence is in the per-repo measurements above.

pnpm run evals:verify-scenarios is green for the first time since 20e4e52.

---

Verification

pnpm verify                # format:check + lint + build + typecheck + test
pnpm --filter crimes smoke # pack + install in a temp dir + run every command

2,117 tests across six packages, up from 1,873 at 0.17.0.

Two properties with standing tests, re-checked after everything that touched scoring, discovery or sort order: fingerprint uniqueness (a gate in scan.test.ts; n8n’s residual is 28 of 16,325, 0.17%, all content-identical pairs) and byte-identical re-scans of an unchanged tree (cmp clean on messy-ts-app and hono).