crimes v0.16.0 — the correctness and authority slate
Theme: code that is correct today and unsafe on a bad day, plus the places a repo keeps its truth twice.
The detector families before this one asked is this code hard to change? — big functions, tangled imports, drifting names. This release asks a different question: what does this code do when something goes wrong, and where does it disagree with itself?
Ten detectors, four shared subsystems, and one property that runs through all of it: every finding names what is missing, not just what is present. An empty catch is reported as “no rethrow; no logging call; the error is never inspected”, because that list is the fix.
$ crimes scan examples/risky-service
🚨 src/services/payments.ts 1. Double Jeopardy · submitPayment → post A retry wraps an HTTP POST to api.post with no idempotency or deduplication key visible at the call site. If an attempt succeeds but the response is lost, the retry applies the operation a second time.
🚨 src/routes/export.ts 2. Policy Doppelgänger · admin plan "admin" "free" rule The same guard clause appears in 2 production files with no shared definition. Each copy is maintained independently, so changing one leaves the other enforcing the old rule.What ships
Section titled “What ships”Release A — cross-file authority
Section titled “Release A — cross-file authority”| detector | charge | finds |
|---|---|---|
duplicated_policy | Policy Doppelgänger | The same business, authorization, eligibility, pricing, or state-transition rule implemented independently in two or more production locations — including near-clone families, where several variants of one rule differ by a single value. |
contract_drift | Contract Split-Brain | Two declarations describing one record that disagree about requiredness, nullability, type, value set, nesting, or a critical field’s presence. Reads TypeScript interfaces, object type aliases, Zod, and Valibot. |
Release B — false confidence
Section titled “Release B — false confidence”| detector | charge | finds |
|---|---|---|
mock_saturation | Mock Alibi | A test that replaces every meaningful collaborator with a behaviourless double and then asserts only on those doubles. It reports coverage of a path it never exercises. |
swallowed_error | Catch and Release | A failure caught and discarded, or converted into an ambiguous success — with no propagation and no record of what went wrong. |
Release C — production realism
Section titled “Release C — production realism”| detector | charge | finds |
|---|---|---|
unsafe_retry | Double Jeopardy | A retry around a potentially-mutating operation with no visible idempotency or deduplication key. |
config_drift | Environment Roulette | One environment variable parsed, defaulted, or required differently across call sites; read past a central config boundary; exposed to a client bundle; or used without being documented. |
unbounded_async_fanout | Concurrency Stampede | Promise.all over a runtime-sized collection doing per-element I/O with no visible concurrency bound. |
Release D — agent hygiene and structural erosion
Section titled “Release D — agent hygiene and structural erosion”| detector | charge | finds |
|---|---|---|
dependency_provenance_gap | Phantom Accomplice | An external import with no declaring manifest, a manifest/lockfile disagreement, or a specifier that resolves differently between installs. |
pass_through_abstraction | Abstraction Laundering | A chain or cluster of wrapper functions that forward their arguments and add nothing. |
agent_permission_sprawl | Loaded Agent | Repository-local agent settings, hooks, and MCP servers that grant unrestricted execution, run fetched code, or exfiltrate the environment. |
Full reference:
finding-types/authority.md,
finding-types/correctness.md,
finding-types/agent-hygiene.md.
What this release deliberately does not do
Section titled “What this release deliberately does not do”Four boundaries, each of which a weaker version of this release would have crossed.
No registry calls. dependency_provenance_gap never claims a
package is malicious, hallucinated, abandoned, or unknown to npm. Those
are claims about the world; answering them needs the network access
crimes promises not to use. Every finding is a statement about this
repository’s own records, and the evidence says so.
Nothing is executed. agent_permission_sprawl reads hooks, MCP
launch commands, and settings as text. A tool that ran a
repository’s hooks in order to analyse them would be a remote code
execution vector wearing a linter costume.
No values are reported. config_drift reports names, locations,
parsers, and defaults written as literals in committed source. A real
.env is never opened — the discovery glob excludes it and a second,
independent filter rejects it.
No generic clone detection. duplicated_policy gates on a narrow
tier of unambiguously-business vocabulary. items.length > 0 appears
in fifty files of any large codebase and is not a policy; repetition
alone can never promote a generic predicate into a finding. During
calibration this single rule took the detector from 188 findings on the
crimes repo itself to 1.
Shared infrastructure
Section titled “Shared infrastructure”Four subsystems introduced once rather than reimplemented ten times:
- Cross-file risk index
(
packages/core/src/risk/) — policy clones, object contracts, environment reads, and pass-through chains, all built in one parse pass. Every “does A relate to B?” question is answered inside a cheap-to-compute bucket, and every bucket is capped, so cost grows with the number of distinct shapes rather than with the square of the file count. A 300-file fixture indexes in about a second. - Domain vocabulary
(
packages/core/src/domain/vocabulary.ts) — a two-tier catalogue. The broad tier raises confidence; the narrow tier decides whether a finding is emitted at all. Separating them is what stopsstate,valid, andflagfrom turning a policy detector into a clone detector. - Confidence and severity ladders
(
packages/core/src/scoring/confidence.ts) — every score is a base plus named, signed deltas, rendered into evidence:confidence 0.88 = 0.60 base + 0.12 (domain vocabulary: …) + 0.10 (spans 2 layers). A number you cannot reconstruct is a number you cannot argue with. - Scope classification
(
packages/core/src/util/scope-class.ts) — one answer to “is this generated / vendored / a migration / a fixture / a test?”, so two detectors can never disagree about the same file.
The language-js pack gains eight parser surfaces
(policyExpressions, objectContracts, errorHandlers,
retrySites, envReads, fanOutSites, testCases +
mockDeclarations, passThroughFunctions), collected in the existing
single AST walk.
Compatibility
Section titled “Compatibility”schema_version is unchanged at 0.3.0. Every new detector emits
the existing Finding shape. New type values are an additive change
to a documented-open enumeration, and docs/json-schema.md already
instructs consumers to treat unknown type values defensively.
- Existing detector ids, charges, and finding meanings are unchanged.
- Existing baselines, suppressions, and triage files continue to load. A baseline written before 0.16.0 simply has no entries for the new types, so their findings classify as new on the first run — the same behaviour as any other detector addition.
crimes diff,verdict,hotspots,context, and every CI gate work with the new findings unchanged.- No CLI flags were added, removed, or changed.
One deliberate overlap resolution
Section titled “One deliberate overlap resolution”duplicated_role_status_plan_check (0.6.0) owns one specific shape:
the same role / status / plan literal compared with two or more
different expressions across three or more files. duplicated_policy’s
near-clone pass skips exactly that shape. Neither detector reports
the other’s territory, so one crime never produces two findings. The
older detector’s id, charge, and behaviour are untouched.
Fingerprint stability
Section titled “Fingerprint stability”Several new detectors use Finding.symbol to carry a stable rule
identity rather than a declaration name:
| detector | symbol |
|---|---|
duplicated_policy | admin plan "admin" "free" rule |
config_drift | REQUEST_TIMEOUT_MS |
swallowed_error | persistOrder → insert |
unsafe_retry | submitPayment → post |
unbounded_async_fanout | notifyEveryone → post |
dependency_provenance_gap | undeclared imports |
agent_permission_sprawl | permissions.allow |
The fingerprint is <type>::<file>::<symbol>. A declaration name alone
would collide whenever one function contains two instances of the same
crime — a function with two try blocks, a file with two fan-outs —
and every downstream surface would then treat them as one finding.
These identities survive line moves and function renames.
New fixture
Section titled “New fixture”examples/risky-service/ — a small service carrying at least one
instance of every new crime, plus interactions between them: the
duplicated entitlement rule disagrees with the plan values its contract
declares; REQUEST_TIMEOUT_MS is parsed three ways and the retry
depends on it; and the test file mocks away exactly the boundary that
carries the most risk.
npx crimes scan examples/risky-service --allexamples/messy-ts-app/ is untouched, so the existing eval baselines
are unaffected.
Eval baseline
Section titled “Eval baseline”Like-for-like with 0.15.0 — same 48 scenarios over 13 fixtures, same two agents, no scenario added or retired. So unlike the 0.15.0 write-up, the aggregate can be read directly.
| agent | 0.15.0 (n=2 samples) | 0.16.0 | delta |
|---|---|---|---|
| claude | 0.85, 0.85 | 0.84 | −0.01 |
| codex | 0.52, 0.55 | 0.58 | +0.03 |
Per scenario kind, 0.16.0:
| kind | claude | codex |
|---|---|---|
| bugfix | 0.95 | 0.33 |
| context | 0.92 | 0.46 |
| plan | 0.75 | 0.57 |
| refactor | 0.89 | 0.74 |
| review | 0.78 | 0.57 |
This delta is a product delta, not a measurement correction. Nothing in the scorer, the judge prompts, the scenario rubrics, or the fixture finding sets changed between 0.15.0 and 0.16.0. What changed is the product: ten new detectors. The movement is what the new slate did to agent behaviour on an unchanged measuring stick.
Neither move is large enough to claim as an improvement. The two 0.15.0 samples already spanned 3 points on codex (0.52 → 0.55), so the +0.03 sits at the edge of the noise band this harness has previously measured, and claude’s −0.01 is well inside it. The honest reading is no regression from a ten-detector release, which is the result that matters — the risk with a slate this size is that extra findings crowd out the ones agents were already acting on, and that did not happen.
Verification
Section titled “Verification”pnpm verify # format:check + lint + build + typecheck + test1828 tests across six packages, up from 1523. New coverage: 61 parser-surface tests, 20 risk-index tests (including a 300-file scale guard), 20 end-to-end pipeline tests spanning baseline / suppressions / triage / diff / verdict / hotspots, and a dedicated suite per detector covering positive cases, near-miss negatives, false-positive traps, scope exclusions, every configuration key, and determinism.