Friction log: dogfooding round 0.14 → 0.17

Written during the round, not reconstructed afterwards. One section per repo, plus an orchestrator section for cross-repo observations and for every defect verified centrally with a minimal reproduction.

Severity vocabulary: annoyance / real problem / blocker.

The report that draws conclusions from this is 2026-08-02-0.14-to-0.17.md. This file is the raw material and deliberately includes small observations the report does not have room for.

orchestrator

Friction log: orchestrator (round setup + cross-repo observations)

`packs_loaded` reports what was loaded, not what ran

repo: improbable (TS/TSX only, zero .py files)
command: crimes scan . --all --format json --no-init
what happened: coverage.packs_loaded = ["universal", "language-js", "language-py"] on a repo with no Python whatsoever.
why it matters: the protocol’s own robustness advice is “check packs_loaded, don’t just count findings”: but the field cannot distinguish loaded from ran from found nothing. A user debugging “why did the Python pack find nothing in my Python repo” gets no signal from the one field that is supposed to answer that. A packs_used or a per-pack file count would.
severity: real problem

`evidence` conflates code evidence with score arithmetic

repo: improbable
command: crimes scan . --all --format json --no-init
what happened: swallowed_error evidence entries include "confidence 0.78 = 0.72 base + 0.06 (protected operation touches external request)" and "severity raised by: external request" in the same array as genuine code evidence like "calls inside the protected region: fetch, res.json".
why it matters: evidence is documented as the receipts for the finding. Score derivation is meta about the finding, not evidence for it. Any consumer that wants to render or reason over the evidence has to string-match to filter it out: which is exactly what this round’s card builder had to do. A separate score_rationale field would keep evidence meaning one thing. Note this is a schema change, so it costs a minor bump.
severity: annoyance (but a schema-shaped one, so cheaper to fix now than later)

`crimes` is not installable as a global for testing without publishing

repo: n/a
command: which crimes
what happened: no global binary; the round needed a hand-written shim pointing at packages/cli/dist/index.js.
why it matters: minor for this round, but it means every “try it on your repo” instinct requires either npx crimes (network) or a build. Worth confirming npx crimes@latest scan is genuinely the frictionless path the docs claim, since that is the documented first-run command.
severity: annoyance

`--all` is a byte-for-byte no-op in `--format json`

repo: hono, p-limit (verified on both)
command: crimes scan . --all --format json vs crimes scan . --format json
what happened: identical MD5. hono both b8cac7b8813aed91268c77d3593b2d21; p-limit both ae256124a5331c55529fbbfe6c0fbfcf.
why it matters: --all is the documented way to opt out of “signal over exhaustiveness”, and the dogfooding protocol itself (and docs/) tell users to pass --all --format json. In JSON mode the flag changes nothing. That is defensible as a design: JSON is the full contract, --all only affects the human renderer: but it is not what the flag’s name or the docs imply, and a user filtering on it will conclude their config is broken. Either document JSON as always-complete and warn when --all is passed redundantly, or make the default JSON view actually summarised.
severity: real problem (documentation/contract clarity, not correctness)

Control result: p-limit is not silent

repo: p-limit (df47604), 8 files, ~200 lines of source
command: crimes scan . --format json
what happened: 10 findings: 0 high, 6 medium, 4 low. Types: swallowed_error x3, boolean_naming_drift x2, large_function, return_shape_roulette, direct_date, large_file, unbounded_async_fanout.
why it matters: p-limit is a few hundred lines of exhaustively reviewed code by a maintainer known for minimalism. Ten findings is more than zero. The mitigating and genuinely good news is 0 high: the tool declines to alarm. But unbounded_async_fanout firing on a library whose entire purpose is bounding concurrency deserves individual scrutiny, and large_file / large_function on an 8-file repo suggests thresholds that do not scale down.
severity: real problem

CONFIRMED BUG: `large_function` (Python pack) reports a size claim it just disproved

repo: pydantic and zulip, found independently by two blind judges
source: packages/core/src/detectors/py/large-function.ts:75-108
what happens: the detector fires on EITHER tooLong (lines > threshold) OR tooDeep (nesting >= DEEP_NESTING). But the summary is built unconditionally as: ${fn.name} is ${lines} lines[ and nests N levels deep]. At this size an agent must read the whole body before it can safely change any part of it. When a finding fires ONLY on nesting, the prose still leads with the line count and still says “At this size”: about a size that passed. Evidence line 83 then prints 14 lines (threshold 50 for shape "domain") directly beneath it.
real examples: pydantic get_strict: 14 lines, threshold 50, depth 4. deep_update: 9 lines. flatten_errors: 18 lines. zulip: 26, 19, 23 lines against a 50-line threshold.
why it matters: the underlying finding (deep nesting) is legitimate. The message destroys it. Both blind judges independently classified these as self-contradicting and rated the detector noise: one wrote “the finding is really about nesting, and the size prose contradicts it”. This is the single clearest case in the round of a message bug being indistinguishable from a correctness bug to the reader. large_function is the highest-volume detector in the corpus (2583 findings).
fix shape: branch the summary on tooLong / tooDeep; lead with whichever actually tripped. Suppress or reword the sub-threshold evidence line.
severity: blocker

CONFIRMED BUG: `pass_through_abstraction` builds chains from name collisions

repo: zulip (card zulip-0018)
what happens: reported get_services “forwards through 5 layers across 5 files”. The actual body is one line: return services.get(bot_id). The “chain” was assembled from unrelated get methods in markdown_config, message_list and sub_store: matching on the method NAME, not on a call path.
why it matters: this is a correctness bug, not a calibration one. The finding asserts a specific structural claim that is simply false, with fabricated evidence naming five real files. pass_through_abstraction fired 19 times, all in one repo.
severity: blocker

`sync_io_in_hotpath` has no working notion of “hotpath”

repos: zulip, pydantic
what happens: fires on code that is provably not a request path: a if __name__ == "__main__" setup CLI (zulip-0001), a Django management command class Command(ZulipBaseCommand) (zulip-0026), a release/push.py script (pydantic-0022), and an @cache-decorated once-per-process function (zulip-0039). Two more sit inside if settings.DEVELOPMENT: with a comment saying nginx serves the file in production (zulip-0009).
why it matters: the detector’s entire claim is about WHERE the I/O happens. 202 findings across 8 repos, and the “hotpath” half of the name is not being established at all.
severity: real problem

`boolean_naming_drift` flags names that cannot be renamed

repos: zulip, pydantic
what happens: four separate findings on atomic = False inside class Migration(migrations.Migration): a Django-defined attribute name. Also 18 names “bound to a boolean expression with nothing saying they hold true/false” that are Pydantic fields explicitly declared name: bool. Also flags names its own convention clause exempts (“or an idiomatic bare name”): nullable, plural, frozen, from_attributes, lock_acquired.
why it matters: 637 findings, third-highest volume in the corpus, and a large share target identifiers that are framework-owned (unrenameable), already type-annotated as bool, or explicitly exempted by the detector’s own stated convention.
severity: real problem

`swallowed_error` does not read the comment that explains the catch

repos: hono (20 of 43 cards), improbable, p-limit
what happens: in nearly every case the code carries an explanatory comment or an immediately visible fallback. One judge noted the detector recognised deliberateness on exactly one card while missing an identical comment two functions above. Also fires where the next line is throw err (hono-0011) and where the catch calls three recovery functions (improbable).
why it matters: 289 findings across 7 repos. It has a BEST_EFFORT_FUNCTION_RE recognition path, and the evidence suggests it is far too narrow.
severity: real problem

SYSTEMIC: detectors with OR-triggers print evidence for the condition that did NOT trip

repos: pydantic, zulip, drf: found independently by three blind judges
the pattern: a detector fires on A || B, but builds its summary and its leading evidence line from A unconditionally. When only B tripped, the finding presents a measurement that PASSED as though it were the reason, and the reader sees a self-refuting statement.
confirmed instances:
- py/large-function.ts:75-108: fires on tooLong || tooDeep, always says “is N lines … At this size”. 12 of drf’s 50 cards, plus pydantic and zulip.
- todo-density.ts:40: if (matches.length < 3 && density < threshold) return [], i.e. fires on count>=3 OR density>threshold, then prints “4.2 markers per 1k LOC (threshold 10)”.
why it matters: this is the round’s most damaging finding, because it is cheap to fix and expensive in trust. Every blind judge independently used the word “contradicted”. A reader who catches the tool stating something its own evidence disproves stops believing the findings that ARE correct: and the underlying signals here (deep nesting, TODO count) are legitimate.
fix shape: derive summary and evidence from whichever condition actually tripped. Audit every detector with a compound trigger for the same shape.
severity: blocker

`weak_test_signal` does not follow assertion helpers

repos: drf (7 cards), zulip (3), pydantic (5)
what happens: claims “N of M test functions assert nothing” where the tests delegate to a helper that asserts. drf-0033: 11 of 11 tests route through _resolve_urlpatterns, which the card itself prints asserting on three attributes. drf-0037: 10 tests call _assert_datetime_rendering, shown ending in self.assertInHTML. zulip-0004: tests call _verify_archive_data. Also counts self.assert* but not self.fail, and not model_dump_json(warnings='error'), and treats @pytest.mark.benchmark functions as assertion-free when measuring is their entire purpose.
why it matters: 619 findings across 8 repos: 4th highest volume. Extract- a-shared-assertion-helper is one of the most common test idioms there is, and the detector reads it as the absence of testing. It is most wrong on exactly the well-factored suites it should be quietest about.
severity: blocker

Vendored and framework-owned code is treated as domain code

repos: drf, zulip, pydantic
what happens: drf-0007/0009/0036 target vendored minified google-code-prettify. 10 pydantic cards target pydantic/v1/, the frozen vendored v1 tree. drf-0014/0028 flag Django’s own mandated package layout (management/commands/*) as reaching into internals. Django Meta.abstract, Meta.proxy, Migration.atomic flagged as badly-named.
why it matters: util/scope-class.ts is documented as the single answer to generated/vendored/production classification. On real repos it is not catching vendored trees, framework-mandated layouts, or frozen legacy subtrees. This inflates every downstream count.
severity: real problem

VERIFIED BLOCKER: source under a dot-directory is silently skipped, and reported as clean

source: packages/core/src/discovery/files.ts:25: dot: false, hardcoded, no config or CLI override anywhere in discovery/.
minimal repro (built and run by the orchestrator, not reported second-hand): identical Python file, 8 lines, nesting depth 5, one open(). as visible.py -> “2 findings across 1 file” as .agents/skills/bad.py -> “No crimes detected. Suspiciously clean.” EXIT 0
found in the wild: ebg keeps all 62 of its .py files under .agents/. coverage.files_by_language had NO py key, yet packs_loaded still listed language-py. An A/B against the same tree with .agents renamed to agents went from 759 findings to 967, and turned up a real cross_language_concept_alias_drift that the real scan reported as zero.
why it matters: this is the worst failure mode a scanner can have: a confident all-clear on code it never opened, with a cheerful message and a zero exit code. .agents/, .github/, .claude/, .server/, .config/ are all common. Nothing in the output hints that anything was skipped.
severity: blocker

VERIFIED BLOCKER: `crimes context` ids do not resolve in `crimes explain`

repro (orchestrator-run, repo improbable): $ crimes context src/MetricsGrid.tsx -> id=crime_00001 (a MetricsGrid finding) $ crimes explain crime_00001 -> file: src/server/migrations.ts A different file. context numbers its findings locally from 1; explain resolves against the global scan ordering. They only agree when the file happens to rank first.
why it matters: context <file> then explain <id> is the documented pre-edit agent workflow and the core of the agent-native positioning: the crimes skill and crimes init --agents both point at it. An agent following it gets a confident, detailed rationale for a finding in a different file, with exit 0 and no warning. Independently found on ebg and reproduced here.
severity: blocker

`cross_language_route_drift` can only ever see a Python backend

source: packages/core/src/detectors/cross/route-drift.ts:124: skips any file whose pack !== "language-py".
consequence: a TypeScript backend is structurally invisible to it. ebg has 22 app/api/**/route.ts handlers exporting 23 verbs against 9 client fetch("/api/…") sites: a textbook drift surface: and the detector cannot produce a finding on it in principle, not by calibration.
why it matters: combined with the corpus-wide zero, the 0.15 release’s headline detector has (a) never fired on real code and (b) excludes the most common polyglot shape in the JS ecosystem.
severity: blocker

`agent_risk` has collapsed into size: a stated product invariant

repo: ebg (measured), consistent with the corpus-wide ranking
what happens: 15 of the top 20 findings by agent_risk are large_function / large_file. Nothing from the 0.16 slate appears, despite parallel_destination firing 52x and orphaned_destination 39x in that repo.
why it matters: CLAUDE.md states agent_risk “is the differentiator and must not be collapsed into severity”. Observably it is a length ranking. This is the score the whole product thesis rests on.
severity: blocker

VERIFIED BLOCKER: the 0.17.0 fingerprint fix did not fix the bug class

claim under test: v0.17.0 release notes: “Self-scan collisions: 3 -> 0”, and “The bug is in the fingerprint function’s inputs, so it gets one general answer” (the discriminator field).
what is actually true: only 2 detector files set discriminator (exact-duplicate-block.ts, magic-domain-literal-scatter.ts). The general mechanism shipped; it was wired into the instances the self-scan happened to surface. At least four other detectors have the identical shape.

measured on real repos (orchestrator-verified from the scan JSON):

repo	findings	unique fingerprints	lost to collisions
hono	377	288	89 (24%)
zulip	3759	3433	326
buildfest	1556	1362	194 (12%)
pydantic	578	552	26
Worst groups: `weak_test_signal` x8 (empty symbol), `commented_out_code` x12
(empty symbol), `large_function` x9 (`describe callback`),
`duplicate_component_shape` x10.

why it matters: these are exactly the consequences 0.17.0 named as its reason for a wire-format break. crimes ignore on one commented_out_code finding silently suppresses up to 12. crimes diff is blind to 24% of hono’s findings. The self-scan was a biased sample: this repo has few test files and little commented-out code: and the fix inherited that bias.
fix shape: any detector that can emit more than one finding per (type, file, symbol) needs a discriminator. Empty symbol is the tell. Consider asserting uniqueness in the scan pipeline so this cannot regress.
severity: blocker

`crime_NNNNN` ids are not stable across runs of an unchanged repo

repo: hono, two back-to-back scans at the same SHA
what happens: identical finding multiset, different ORDER, and ids are positional: so 5 ids changed identity. Run A crime_00024 = large_file on secure-headers.ts; run B crime_00024 = hardcoded_localhost on a benchmark. Suspected cause: the clock-derived recency score drifting 0.71 -> 0.70 and flipping a sort tie.
why it matters: crimes explain <id> is the documented workflow and does not survive a re-run. 0.17.0 fixed evidence-string determinism; finding IDENTITY ordering is still clock-dependent.
severity: blocker

`TEST_FILE_RE` misses `test.js`: the ava / node:test default filename

repo: p-limit
what happens: a bare top-level test.js matches none of *.test.js, _test.js, __tests__/, tests/. Cascade: charged on the domain 300-line budget instead of the 1500-line test_file budget; tier: "domain"; test_gap degraded to the 0.50 no-signal midpoint; crimes context index.js reports “no test file matched” for a package whose 378-line suite sits beside it and imports it on line 8.
severity: real problem

The `docs` shape and the data-format exclusion are unreachable at default config

source: DEFAULT_SOURCE_INCLUDES in packages/core/src/config.ts
what happens: it contains no json, yaml, yml, txt, rst, adoc. So (a) the 0.17.0 note that “.json/.yaml stay on the domain budget: a 3000-line config file is a finding worth having” describes a policy that never runs: hono’s 735-line package-lock.json and a synthetic 1201-line .json both produce ZERO findings; and (b) 5 of the 7 extensions the docs shape claims (.txt, .rst, .adoc, .markdown, and the data formats) are dead code. Overriding include makes both fire immediately: the shape logic is correct, the files never reach it.
severity: real problem

`test_gap: 0.5` means “no signal” but renders as a confident causal sentence

repo: p-limit
what happens: explain renders the 0.50 midpoint as “a test file matches this module’s name but does not import it”: false on both clauses, and contradicting crimes context on the same file, which says no test matched.
severity: real problem

`explain` exits 2 on a finding the scanner itself emitted

repo: hono
command: crimes explain crime_00369
what happened: crimes: no detector registered for finding.type "oversized_raster", exit 2. 22 of 23 hono types explained cleanly.
severity: real problem

`hotspots` ranks manifest churn as the top risk, over a different file set

repo: hono
what happens: #1 risk is package.json at 72%, ~14 points clear of anything real, because it churned 29x in 90 days on version bumps. It also ranks files scan never scanned (package.json, readme.md, index.d.ts): hotspots and scan operate over different file universes (929 vs 546 files on ebg). On a quiet repo it pads the list with rows reading “0 findings - highest none” rather than saying nothing is hot.
severity: real problem

NOTE FOR ANYONE READING THE ARTIFACTS

Several artifacts/*.json were 0 bytes at various points because a scan was SIGTERMed (exit 143), not because crimes found nothing. Do not read a 0-byte artifact as a clean scan. This is itself a mild finding: a killed scan leaves a file that is indistinguishable from a successful empty one.

VERIFIED BLOCKER: `large_file` counts total lines + 1 and calls it “non-empty lines”

repo: pydantic (orchestrator-verified against the files on disk)

measured:

file	crimes claims	real total	real non-blank
`pydantic/fields.py`	1914	1913	1654
`pydantic/types.py`	3336	3335	2647
`pydantic/networks.py`	1371	1370	1084
The claim is exactly `total + 1` every time, labelled “non-empty lines”.
Overstatement vs the true non-empty count is 16-26%.

consequence: it manufactures findings. Of 109 large_file findings on pydantic, 11 (10%) would not fire at all on the true non-empty count: _migration.py (claimed 318, real 298, threshold 300), _dataclasses.py (316 / 253), v1/env_settings.py (351 / 290), five pydantic-core/*.rs files, and two markdown docs: so the 0.17.0 docs shape inherits the bug (docs/migration.md claimed 1168, real 889, threshold 1000).
why it matters: large_file is the second-highest-volume detector in the corpus (779 findings). A ~10% phantom rate is roughly 78 findings that exist only because of an off-by-one plus a blank-line miscount. It also means the lines field and every ratio (“2.9x the configured threshold”) are wrong.
severity: blocker

VERIFIED BLOCKER: `crimes scan` on a nonexistent path exits 0 and reports clean

command: crimes scan /tmp/this-path-does-not-exist-xyz
what happens: CRIME SCENE REPORT repo: this-path-does-not-exist-xyz · 0 findings No crimes detected. Suspiciously clean. EXIT=0 The JSON form is equally well-formed: valid schema_version, empty findings, files_total: 0, exit 0.
why it matters: a path typo in CI produces a passing, well-formed clean report. verdict and explain both correctly exit 2 on bad input; scan is the outlier. Combined with the dot-directory bug, there are now two distinct ways to get a confident all-clear on code that was never read.
severity: blocker

VERIFIED BLOCKER: test-scope classification is structurally inert on Python

source: packages/core/src/config.ts:17 DEFAULT_NON_DOMAIN_PATTERNS = scripts/**, examples/**, fixtures/**, public/**, **/__tests__/**, **/*.test.{ts,tsx,js,jsx}, **/*.spec.{ts,tsx,js,jsx}. No tests/**, no test_*.py, no *_test.py.
measured: every finding is tier: "domain": pydantic 578/578 (including 118 under tests/), drf 104/104 (21 under tests/), zulip 3725/3759 with 662 test-file findings all domain.
the sharp part: crimes ALREADY has the right classifier. util/test-files.ts:23 matches test_[^/]+\.py$ and .*_test\.(?:py|go)$. tier uses a third, glob-based, JS-only mechanism instead. classifyScope is imported by exactly 4 detectors, all 0.16.0 JS-side: zero Python and zero universal detectors consult it.
why it matters: CLAUDE.md presents util/scope-class.ts as “one answer to generated / vendored / migration / fixture / test / config / production”. There are three answers and the one wired into scoring cannot see a Python test. Test files are ~62-64% of both pydantic and drf, and every finding in them is scored as production code.
severity: blocker

PEP 696 type-parameter defaults fail to parse; one form loses the whole file

what happens: class M[T, U = int], def f[T = int], def f[*Ts = *tuple[int]] all produce parse errors. type Al[T = int] = list[T] loses the entire file’s symbol table. The same forms without defaults parse cleanly.
why it matters: Python 3.13 syntax. Combined with the next entry, a repo adopting it degrades silently.
severity: real problem

`hasSyntaxErrors` is computed and then never surfaced

what happens: consumed by exactly one detector (weak_test_signal.py). There is no coverage.parse_errors, nothing on stderr, nothing in the JSON. pydantic’s one unparseable file was found only by importing the parser directly.
why it matters: this is the same failure family as the dot-directory bug and the nonexistent-path bug: crimes knows something went wrong and does not say so. A repo where the grammar chokes reports as clean.
severity: real problem

`commented_out_code` matches English prose, including Rust doc comments

repos: pydantic (77 findings) vs drf (2)
what happens: 9 of a deterministic 10-sample are English prose. One is pydantic-core/src/definitions.rs:1-5, a Rust /// doc comment. The rule keys on tokens def/class/fn, so prose about classes and fields matches. This is why pydantic scores 77 and drf 2: a writing-style difference, not a code-quality one.
note: on airflow this detector is 8019 findings, 45% of the entire scan. It was not in the judged sample.
severity: real problem

Corpus-integrity lesson: a broken clone looks exactly like a clean repo

repo: drf
what happened: git-lfs is not installed, so git checkout aborted the whole worktree write at docs/theme/src/drf-logos.fig. The clone had 15 files and no rest_framework/ or tests/ at all.
why it matters: had the agent not counted .py files by hand first, drf would have scanned to ~0 findings and been written up as the round’s best control result. crimes gave no indication it was looking at an empty repo. This is the dot-directory bug wearing different clothes, and it is a methodological warning for any future round.
severity: real problem (for the product: silent zero; for us: process)

`mixed_utc_local_methods` fired zero on 296K lines of Python

across pydantic and drf combined. Add to the zero-firing list.
severity: annoyance (a finding about the detector)

VERIFIED BLOCKER: `.gitignore` is not respected

claim under test: CLAUDE.md § Stack decisions: “File walking: fast-glob
- ignore (respect .gitignore)”.
orchestrator repro, from scratch: /tmp/gitig3/.gitignore -> “generated/” git check-ignore generated/big.ts -> generated/big.ts (ignored) crimes scan . -> “2 findings across 1 file · 1 high” -> “generated/big.ts 1. God Function 2. God File” The gitignored file is the ONLY file in the report. Also confirmed via coverage.files_total, which counts ignored files.
in the wild (buildfest): migrate-output/ is line 8 of .gitignore and is a verbatim copy of apps/airquote. It produced 100 findings directly, 229 more citing it: 323 of 1556 = 20.8% of the whole report: and 136 of 158 (86%) exact_duplicate_block findings. It also appears in coverage.by_package.
why it matters: duplication detectors are maximally vulnerable to build output, and build output is exactly what .gitignore names. This has stayed invisible because the hardcoded default excludes cover node_modules/dist/.next: the common cases: so the missing .gitignore support only shows up on project-specific ignore rules. Together with the dot-directory bug, crimes both scans what it shouldn’t and skips what it should, and neither is reported.
severity: blocker

VERIFIED BLOCKER: silent, nondeterministic finding loss under fd pressure

repo: buildfest, fd sweep
what happens: at ulimit -n 48 and above, output is byte-identical to baseline (sha256 d694948ff7eb…): the 0.17.0 bounded pool is correctly sized and holds. Below ~48 the scan silently reports fewer findings, nondeterministically: three runs at ulimit -n 40 on an unchanged tree gave 1522 / 1535 / 1543 findings. Every one exited 0, with empty stderr, an identical coverage block, and summary.high pinned at 187 while ~11% of the report vanished.
root cause, located: it is NOT indexes.ts: packages/core/src/ast-hash/function-index.ts:72-79 try { source = await readFile(abs, “utf8”); } catch { return hits; } // EMFILE becomes “this file has no functions” Same shape at packages/core/src/jsx/shape-index.ts:65-66. The bare catch cannot distinguish ENOENT from EMFILE, so each exhaustion silently deletes one file from the duplicate corpus: partial and racy, which is strictly worse than the all-or-nothing undefined that indexes.ts was designed to produce.
margin between correct and silently wrong: 8 descriptors.
fix: re-throw or count EMFILE/ENFILE, and surface a read-failure count in coverage. Touches no detector.
severity: blocker

`churn` and `recency` score 0.00 on a quiet repo and render as a claim about the code

repo: buildfest (zero commits in the default 90-day window)
what happens: scores.churn and scores.recency are 0.00 on all 1556 findings, and scan renders that as “churn low”: an assertion about the code, when the truth is “not measured”. hotspots degrades further: every file scores an identical risk 0.40 · 0 changes, so ranking collapses to alphabetical and the #1 hotspot is a PNG. With --since 1y the same command is genuinely good (package.json 0.91, 17 changes).
why it matters: same family as test_gap: 0.5 rendering as a confident causal sentence. The tool has no way to say “no signal” and defaults to saying something reassuring instead.
severity: real problem

Positive results worth recording

Determinism holds. 9 buildfest runs produced the same 2,412,426 bytes, including id fields. The 0.17.0 reproducibility claim is real.
Performance is not the problem. 1177 files in 27s at 308 MB max RSS. zulip: 3588 files in 81s.
Both zero cases pass on buildfest, stated positively: language-py loaded and produced exactly 0 findings on a repo with 0 .py; all three cross_language_* produced exactly 0 on a single-language repo.
Both 0.17 surfaces confirmed live: the docs shape fired on 3 .md with shape: document and a distinct 1000-line threshold; discriminator is present on 225 findings.

VERIFIED BLOCKER: `dependency_provenance_gap` checks Python imports against npm manifests

repo: zulip. Severity high, on package.json.
verbatim: “450 external package(s) are imported but declared in no applicable package.json. They resolve today through hoisting; a clean install can fail.” Evidence lists: _typeshed: imported at zerver/lib/test_console_output.py abc: imported at corporate/lib/stripe.py aioapns: imported at zerver/lib/push_notifications.py
abc and _typeshed are the Python standard library. aioapns is declared, in pyproject.toml. Every cited import is from a .py file, and the manifest set searched is “package.json, starlight_help/package.json”.
why it matters: the detector has no notion that a Python import belongs to a Python manifest. On any repo containing Python it emits a HIGH-severity finding that is 100% wrong and whose implied fix is to add the Python stdlib to package.json. Note the second finding from the same detector on the same repo (medium, @typescript/native declared but absent from pnpm-lock.yaml) is plausibly correct: so the JS path works and the language dispatch is the defect.
severity: blocker

VERIFIED BLOCKER: `blast_radius_importers` prints a component size as a per-file count

repo: zulip, verified from the scan JSON
measured: 3,759 findings carry blast_radius_importers, and there are only 37 distinct values. 866 findings share the value 798; 825 share 324. blast_radius is pinned at 1.0 on 1,758 of 3,759 (47%).
in the human report: zerver/data_import/slack.py renders as “blast top-quartile (798 importers)”. Actual direct importers: 5. inbox_ui.ts prints 324; actual 4.
why it matters: hundreds of unrelated files cannot each have exactly 798 importers: this is the size of a connected component in the import graph being rendered as a direct fan-in count. It is a number a maintainer can disprove in five seconds, printed next to findings that are otherwise correct. explain renders it honestly (“50+ transitive importers (cap reached)”); scan and context do not: so the same score is truthful in one renderer and false in two.
severity: blocker

Cross-language: diagnosed, on the second ideal input

repo: zulip. route_drift 0, type_drift 0, alias_drift 0.
the diagnosis (both halves are blind):
- routeFromDecorator() is the ONLY route source and reads decorators only. Zulip declares 387 URLs as Django URLconf function calls: rest_path("realm/domains", GET=…, POST=…) in zproject/urls.py. Repo-wide @app.get-shaped decorators: 0. So route_drift early-returns on an empty backend set and never reaches its comparison. Secondary blocker: Django routes are relative ("realm") and the extractor rejects anything not starting with /.
- The frontend half fails too. Zulip makes 140 calls shaped channel.post({url: "/json/typing", data, …}). The receiver channel fails REQUEST_RECEIVERS = /(^|\.)(axios|http|client|api|request)$/, and the URL sits in an object-literal first argument, so literalText() returns undefined. swallowed_error.js parses that exact call fine and quotes it in the human report: two collectors in the same parser disagree about what an HTTP call is.
- type_drift did run, and honestly found no name matches: but its Python pool is junk: the allValued fallback accepts any class whose members are all string constants, so 14 of 19 “closed sets” are things like Integration (image-path templates). Zulip’s 29 real enums are IntEnum/Enum with integer values, which the extractor never records.
and you cannot tell any of this from the output: "cross-language" is never a member of packs_loaded: it is ["universal", ...registeredPacks()]. No JSON field distinguishes “ran and found nothing” from “never ran”.
severity: blocker (for the 0.15 release’s premise)

749 files of Zulip’s rendered UI were dropped at discovery, silently

.hbs (469), .html (224) and .erb are outside the default include globs. They appear in neither coverage histogram: not files_by_language, not universal_only_by_extension: so nothing in the output tells the user that the entire template layer was never opened. Same family as the dot-directory and .gitignore bugs.
severity: real problem

`docs_code_drift`: 12 of 12 verified false positives

root-relative documentation links are resolved against the repo root instead of docs/, plus extensionless Sphinx-style links are not understood. All six targets spot-checked exist on disk.
severity: real problem

Quantified: `large_function` self-refutes on 19% of its Python findings

zulip: 172 of 898 large_function.py findings quote a line count at or below the threshold in the same evidence string. Canonical case: zerver/tornado/event_queue.py:1602 process_realm_user_add_event: “God Function … 9 lines (threshold 50 for shape ‘domain’)” on a clean 9-line event fan-out. Visible in the default human output at #17 of the top file.
large_function is 1,461 findings = 39% of the entire zulip report, and agent_risk ranks it first: top 20 by agent_risk is 18 large_function / large_file, zero Python, on a repo that is 71% Python.
severity: blocker (already recorded; this entry adds the measurement)

The positioning check, measured

On zulip, 81% of the report (3,038 of 3,759 findings) falls in what the agent characterised as the “a linter already does this” bucket: line counts, naming, commented-out code, TODO density. CLAUDE.md states crimes is explicitly not trying to replace ESLint/Biome. The defensible remainder is 721 findings, ~1.2 per 1k lines: which is a genuinely reasonable number, and is the product that appears to be hiding inside this one.

VERIFIED BLOCKER: no size or line-length guard before parsing: 135x slowdown

repo: ortomate-studio
measured: scan --all --format json = 675 s. Same repo with 92 vendor bundles removed = 5 s. One 1.9 MB bundle alone accounts for 251 seconds and produces 23 findings. Longest line in the repo: 1,621,186 characters. diff HEAD~5...HEAD never completed (killed >540 s).
root cause, verified at source: packages/core/src/config.ts:216-217 excludes **/*.min.js and **/*.generated.*: filename patterns only. Modern bundlers emit name-HASH.js (mermaid.core-DKFhFOEB.js), which matches neither. No size guard, no line-length guard, and no --exclude or --max-file-size flag on scan, so under --no-init there is no workaround at all.
the compounding harm: util/scope-class.ts classified these bundles as production domain code. 444 of 851 findings (52.2%) are inside the vendored reference/ directory, and 15 of the top 20 by agent_risk are minified third-party bundles. All five entries in the default report’s “Top files by risk” are somebody else’s code, with findings reading Unprefixed Boolean 'So' @L1. The report’s closing call to action is → Start with crimes context chrome-extension/reference/claude-1.0.77/assets/mcpPermissions-CJK8I7C7.js.
severity: blocker

`.gitignore`: confirmed absent at source, and on a second repo

source check: grep for gitignore parsing across packages/core/src/discovery/ returns nothing. discovery/files.ts passes ignore: options.exclude to fast-glob and nothing else. CLAUDE.md lists “fast-glob + ignore (respect .gitignore)” as a locked stack decision; it was never implemented. The dot: false behaviour is incidental, not gitignore support.
second independent repo: mysidekick-cc: api/_server.mjs (34 findings) and public/call.bundle.js (12) are esbuild output, both gitignored, both untracked: 46 of 241 findings = 19.1%. _server.mjs is a 2,668-line bundle of src/*.ts, so every finding in it duplicates one already reported against the real source. The report’s single closing recommendation is → Start with crimes context api/_server.mjs. The tool’s one piece of advice points at a generated file.
severity: blocker (already recorded; this adds source confirmation and a second repo)

Absent signals are scored as zeros and rendered to the user as good news

repo: ortomate-studio
what happens: across all 175 Swift findings, blast_radius = 0, blast_radius_importers = 0, test_gap = 0: constant, not distributed, because no pack claims Swift. The schema has no “unknown”, so absence becomes the best possible score.
sharpest instance: crimes context ProjectsStore.swift: 4,949 lines, 97 commits in 90 days, no test file: prints test gap: bottom-quartile, the identical phrase a well-covered TS file receives. Swift test discovery looks for .test/.spec/__tests__ and misses Tests/OrtomateStudioTests/*Tests.swift (20+ files, standard SwiftPM). In the command designed to brief an agent before it edits, the tool reports the opposite of the truth.
same family as test_gap: 0.5 narrated as a causal sentence, and churn: 0.00 rendered as “churn low”.
severity: blocker

The honesty gap is one line of output wide

repo: ortomate-studio (429 Swift files, 85% of source LOC)
the good news, recorded properly: the JSON is completely honest. All 429 Swift files appear in files_total, and universal_only_by_extension names ".swift": 429 explicitly. --explain-coverage prints an unusually good sentence: “These files were not claimed by any loaded language pack. They received universal-pack checks only (file size, raster assets, duplicate filenames, hardcoded localhost/local paths, docs link checking).”
the gap: that flag is off by default and printed last, after the report it should be qualifying. In the 174-line default report the string “swift” appears zero times. The 175 Swift findings come from just 4 detector types; ~45 detectors ran on 16,012 lines of TS/JS and on none of the 91,778 lines of Swift.
fix: promote one header line: 429 of 619 files: universal checks only (.swift). Cheapest high-value fix identified in the entire round.
severity: real problem

`packs_loaded` is a constant: confirmed at source

packages/core/src/coverage.ts:58 builds it as ["universal", ...router.registeredPacks()], and registration happens at module load. So language-py appears on every scan of every repo, including one with 429 Swift files and zero Python. It cannot answer the question its name implies. The honest signal sits right beside it: files_by_language has no py key: but nothing marks packs_loaded as non-diagnostic, and this round’s own protocol told agents to read it.
severity: real problem

`hotspots` is the honest command and `scan` is the one the docs push

On mysidekick-cc, hotspots ranks src/server.ts, src/prep/compileBrief.ts, src/db.ts: and the gitignored bundles are absent, because churn-based ranking drops untracked files for free. On ortomate-studio it puts ProjectsStore.swift (97 changes) at #1. Both times it beats scan.
the cheap fix this suggests: scan could borrow the tracked-in-git check hotspots already has, which would fix a large part of the .gitignore and vendored-bundle problems at once.
severity: worth acting on

The default report is not a summary

repo: mysidekick-cc. The first block prints 34 findings for one file; items 24-34 are eleven consecutive Catch and Release entries, seven of them the identical res.text() shape. --top caps files at 5 but does not cap findings within a file, so the default already prints everything for the files it shows: contradicting the PRD’s “top findings only, --all is opt-in”. Collapsing repeats of one charge in one file (Catch and Release x11) would halve the report with no information loss.
Four unreconciled file counts across three commands on one repo: 74, 63, 239, 220.
severity: real problem

ebg

Friction log: ebg

Repo: ~/dev/ebg (polyglot: TS/TSX/Python/Swift) SHA: e3b7b1e3b15a5a9b7a0548fc9d86aae2c57f7bb2 Binary: ~/crimes-dogfood/bin/crimes 0.17.0 Started: 2026-08-02

Python pack loaded but zero `.py` files were ever walked

repo: ebg
command: crimes scan /Users/andrew/dev/ebg --all --format json --no-init
what happened: coverage.packs_loaded includes language-py, but coverage.files_by_language is {"js": 376}: there is no py key at all, and .py does not appear in universal_only_by_extension ({".md":109, ".swift":58, ".sh":3}). The repo has 62 tracked .py files. Zero findings have a .py file. The files were not parsed, not skipped- with-a-reason, not counted: they are simply invisible.
why it matters: This is exactly the failure mode the protocol warns about: packs_loaded reports what was loaded, not what ran. A user with a Python subtree sees “language-py” in coverage and reasonably concludes their Python was analysed. It was not. There is no signal anywhere in the JSON or the human output that 62 files were dropped.
severity: real problem

Coverage accounting does not add up to the files actually analysed

repo: ebg
command: same
what happened: files_total 546 = js 376 + files_universal_only 170 (.md 109 + .swift 58 + .sh 3). But 25 findings are on .jpg files (oversized_raster). Images are evidently walked and analysed yet appear in no coverage bucket. So files_total under-reports the true analysed set, and the language histogram is not a partition of it.
why it matters: coverage is the only thing that tells a user what was and wasn’t looked at. If it silently omits a whole category that produced 25 findings, it can’t be used to reason about blind spots.
severity: real problem

No `coverage.by_package` at all (0.15 surface absent)

repo: ebg
command: same
what happened: The JSON coverage object has exactly five keys: files_total, files_by_language, files_universal_only, universal_only_by_extension, packs_loaded. by_package is absent. The repo has a root package.json and a vendored ios/Vendor/NFCPassportReader/Package.swift.
why it matters: 0.15 shipped coverage.by_package as part of the contract. On a real repo it does not appear, so either it is gated on a multi-manifest condition this repo doesn’t meet, or it regressed. Either way a consumer cannot rely on the key existing.
severity: real problem

All three `cross_language_*` detectors fired zero times

repo: ebg: the only polyglot repo in the local corpus
command: same
what happened: cross_language_route_drift, cross_language_type_drift, cross_language_concept_alias_drift: zero findings each. Given that the Python side was never walked (see above), this is a vacuous zero, not an informative one.
why it matters: The one repo in the local round that could have exercised the 0.15 cross-language pack could not, because the input to half of each detector never existed.
severity: real problem

ROOT CAUSE: the file walker is `dot: false`, so any source under a hidden directory is invisible

repo: ebg
command: controlled A/B in scratchpad (nothing written to ebg): copied ~/dev/ebg/.agents/skills to $S/pytest_visible/skills and ~/dev/ebg/.agents to $S/pytest_dot/.agents, then scanned both.
what happened:
- pytest_visible (same 62 .py files, non-hidden parent): files_by_language: {"py": 62}, 190 findings, 178 of them from language-py (large_function 116, sync_io_in_hotpath 43, boolean_naming_drift 15, direct_date 4, large_file 12).
- pytest_dot (identical bytes, hidden parent): files_total: 0, files_by_language: {}, 0 findings, exit 0.
- Source: packages/core/src/discovery/files.ts:25: dot: false. There is no CLI flag to override it, and crimes.config.json include globs won’t help because fast-glob’s dot option, not the pattern, is what excludes them.
why it matters: The entire Python surface of the only polyglot repo in the local corpus was dropped for a reason unrelated to Python. Worse, the degraded case is indistinguishable from a clean one: scanning the hidden copy prints “No crimes detected. Suspiciously clean.” with exit 0 and packs_loaded: [universal, language-js, language-py]. A user whose source lives under .server/, .functions/, .agents/, .github/scripts/ etc. gets a confident all-clear on an unscanned repo.
severity: blocker

`--explain-coverage` prints a pack as “loaded” and then omits it entirely

repo: ebg
command: crimes scan ~/dev/ebg --no-init --explain-coverage
what happened: Output reads packs loaded: universal, language-js, language-py followed by files by language pack: with a single line for language-js. language-py has no line at all: not “0 files”, just absent. Same in the JSON. The only place that could have told the user “a pack you loaded claimed nothing” says nothing.
why it matters: --explain-coverage exists precisely to answer “what did you not look at”. It answers it wrongly by omission.
severity: real problem

Cross-language verdict, with evidence (counterfactual scan)

repo: ebg
command: crimes scan $S/ebg_unhidden --all --format json --no-init where $S/ebg_unhidden is git archive HEAD of ebg with .agents renamed to agents: same bytes, nothing hidden.
what happened: With the Python visible, files_by_language becomes {"py": 62, "js": 375}, total findings rise 759 → 967, and cross_language_concept_alias_drift fires once (agents/skills/docx/.../merge_runs.py:93, “delete” concept named remove/trash in Python vs delete in TS, 11 files). cross_language_route_drift and cross_language_type_drift remain zero.
Shape check, by hand, in ebg:
- route_drift: shape genuinely absent. Zero matches for @app/@router/@bp.(get|post|…) or from fastapi|flask|django| starlette|aiohttp in any .py. The Python import histogram is pathlib 39, sys 30, defusedxml 21, zipfile 20, argparse 18, lxml 6: these are Claude-skill CLI scripts, not a service.
- type_drift: shape genuinely absent. Zero occurrences of Enum anywhere in the .py. All 14 Python classes are validator/handler classes (BaseSchemaValidator, RedliningValidator, RectAndField, ReviewHandler), none a constant/closed set.
- alias_drift: shape present, and it fires. Zero in the real scan was purely the dot-directory exclusion, not the detector.
severity: (informational: but the zero for alias_drift was a false zero caused by the walker, which is the blocker above)

`cross_language_route_drift` is blind to the route contract this repo actually has

repo: ebg
command: read packages/core/src/detectors/cross/route-drift.ts
what happened: collectBackendRoutes() skips every file where entry.pack !== "language-py" (line 124). The backend half of the contract can only ever be Python. ebg has a textbook drift-able surface that is entirely TypeScript: 22 app/api/**/route.ts handlers exporting 23 verbs (13 GET, 10 POST), against 9 client fetch("/api/…") call sites. The detector cannot look at it. (I hand-checked all 9 fetch paths against the 22 route dirs: all resolve, so there is no drift to miss in this repo. The blindness is structural, not a missed finding here.)
why it matters: The detector is named cross_language_route_drift but its real contract is “Python backend ↔ JS frontend”. On a Next.js / Remix / SvelteKit / Express repo: a large share of the target market: the same class of bug exists and is invisible. The name and the whyItMatters copy (“the contract between a frontend and its backend is written down twice”) both over-promise relative to the implementation.
severity: real problem

Python parse robustness: PASS, with positive evidence

repo: ebg (via the unhidden counterfactual)
what happened: 51 of 62 .py files produced ≥1 finding. Specific modern-syntax files, all confirmed to produce findings:
- match (3 files, e.g. .../docx/scripts/office/validate.py): large_function main [25,107], nesting depth 3. The real file is 111 lines with def main at 25, the match at 80, if __name__ at 110. The parser bounded main correctly through the match block and counted the case arms as nesting: so it structurally understood match, not merely tolerated it.
- walrus := (5 files): 2–8 findings each, e.g. comment.py (7).
- PEP 604 X | None (8 files): 2–7 findings each.
ebg contains no PEP 695 generics and no except*, so I probed those in the scratchpad. class Registry[T], def parse_generic_fn[T], type Alias = …, and except* ValueError as eg all parse: findings are emitted with correct symbols and line ranges (verified A/B against class Registry: and class Registry(Generic[T]): identical results).
severity: none: this is the one thing that worked cleanly.

BUG: `sync_io_in_hotpath` is a file-level finding wearing a function’s name and a fabricated line range

repo: ebg (and reproduced minimally in scratchpad)
command: crimes scan … --all --format json --no-init
what happened: In ebg’s Python, sync_io_in_hotpath fires 43 times across exactly 43 files: one per file. Each is attributed to a single function symbol, but the lines range is [first blocking call, last blocking call] across the whole file. Example: agents/skills/docx/scripts/office/validators/base.py → symbol: repair_whitespace_preservation, lines: [120, 763]. repair_whitespace_preservation actually spans 115–141. Lines 756 and 763 are inside _validate_single_file_xsd (750–785). The finding’s range crosses 8 unrelated functions. Minimal repro ($S/pp_multi/m.py, three identical 62-line functions alpha/beta/gamma, each with 60 open() calls): sync_io_in_hotpath alpha [2, 187]: "180 blocking calls". There should be three findings of 60 calls each; instead there is one, named alpha, spanning to line 187 (inside gamma).
why it matters: Two separate harms. (1) The line range is wrong, not merely coarse: anything that renders a code window (explain, the judging cards, an editor integration) will show a 643-line span and attribute other functions’ I/O to repair_whitespace_preservation. (2) The count is inflated: “180 blocking calls” describes a file, while the symbol describes a function. A reader opening the named function finds 4 of the 17 calls claimed.
severity: real problem

`large_function` double-counts a function and the callback that is its whole body

repo: ebg
command: crimes context 'app/agency/[agency]/(protected)/cases/actions.ts' --no-init
what happened: The file’s context view lists 24 findings, 14 of them God Function. Five are the same function twice: applyRegistryTrust [881,1096] and <anonymous> [888,1095]; declareBuyer [381,516] / <anonymous> [389,515]; runEbg, savePrimaryContact, uploadTrustDeed likewise. The repo’s server actions all have the shape export async function f() { …3 lines…; return withTenant(id, async () => { …everything… }) } so the callback is 96–99% of the outer body and both are charged. Repo-wide: 10 of 125 large_function findings (8%) are an inner function covering ≥80% of an outer one. case-workflow.ts run/<anonymous> and ownership-stack.ts recordOpenCorporatesOwnershipStack/visit too.
why it matters: It is the top-ranked file in the scan, so this is the first thing a user reads, and a quarter of what they read is a repeat. The <anonymous> half also carries no actionable identity: “An anonymous function is 208 lines long” cannot be located without opening the file.
severity: real problem

`crimes verdict` takes 53 seconds to say “UNCHANGED”

repo: ebg
command: crimes verdict --no-init (cwd = repo)
what happened: real 53.28 / user 14.26 / sys 3.21. Output is 8 lines: Verdict: UNCHANGED / New: 0 / Fixed: 0. scan --all --format json on the same repo is 6.0s. So verdict is ~9× a full scan, and 39s of the 53s is not CPU: it is blocked on git.
why it matters: This is the command intended for a pre-commit / CI gate. A minute of wall clock for a no-op answer is the kind of thing people remove from their hook after the second time.
severity: real problem

`crimes hotspots` is 27s and disagrees with `scan` about how many files exist

repo: ebg
command: crimes hotspots ~/dev/ebg --no-init
what happened: real 27.19 / user 8.17 (again git-bound). Header says 929 files; scan on the same repo says files discovered: 546. Neither number is explained, and 929 ≠ 877 tracked files either. Also the top three entries all render risk 1.00 (100%): a three-way tie at the cap, so the ranking conveys nothing at the top where it matters most.
why it matters: Two commands reporting two different repo sizes with no note of why makes both numbers untrustworthy.
severity: annoyance

`crimes context` prints 270 lines for one file

repo: ebg
command: crimes context 'app/agency/…/cases/actions.ts' --no-init
what happened: 270 lines of output, of which 14 near-identical God Function blocks each repeat the same 2-sentence boilerplate (“Bodies this size usually mix unrelated responsibilities, and an agent editing one section often misses interactions in another”) verbatim. There is no --top on context (there is on scan).
why it matters: context is pitched as the pre-edit command for an agent. 270 lines with an 8× repeated paragraph is a poor use of a context window, and for a human it buries the five genuinely distinct findings (Return Shape Roulette, Logic in the Alibi, fan-in/out, Parallel Destination) under a wall of size complaints.
severity: real problem

`crimes context` prints finding ids that resolve to different findings under `crimes explain`

repo: ebg
command: crimes context 'app/agency/…/cases/actions.ts' --no-init then crimes explain crime_00001 --no-init
what happened: context labels its findings id=crime_00001 … crime_00024, numbered locally from 1 for that file. scan and explain agree with each other on a global numbering. So:
- context says crime_00001 = God Function applyRegistryTrust, app/agency/…/cases/actions.ts:881–1096
- explain crime_00001 returns God Function listAmlIdentifiedPeople, lib/portal-token.ts:478–663
- context says crime_00024 = Parallel Destination in actions.ts; explain crime_00024 returns God Function run in trigger/case-workflow.ts. Verified for three ids; explain is deterministic across two runs (same md5), so this is a collision, not flakiness.
why it matters: context is the documented pre-edit command and it ends every finding with id=…, which reads as an invitation to run crimes explain <that id>. Doing so silently returns a different file’s finding. Nothing errors. An agent following the obvious workflow will act on the wrong code.
severity: blocker

`crimes explain` on an unknown id exits 0

repo: ebg
command: crimes explain crime_99999 --no-init
what happened: prints crimes: no finding with id or fingerprint "crime_99999": re-run 'crimes scan' and check the output. and exits 0.
why it matters: A lookup failure that reports success cannot be scripted against. Combined with the id-collision above, a wrapper cannot even detect that it asked for something that does not exist.
severity: real problem

`crimes diff` (human) takes 62s and refuses to say what the new crimes are

repo: ebg
command: crimes diff HEAD~5...HEAD --no-init
what happened: real 62.50 / user 14.84. Entire output: New crimes: 2 / Fixed crimes: 9 / Unchanged crimes: 686. No file, no charge, no line. To find out which two, you must re-run the whole 62s command with --format json and parse it. There is no --all, --top, or verbose flag on diff.
why it matters: The single question diff exists to answer is “what did I just add”. It answers “two things” and stops. At a minute per run, needing two runs is a two-minute round trip for two lines of information.
severity: real problem

`high_fan_in_fan_out`: the summary and the score disagree about the importer count

repo: ebg
command: crimes diff HEAD~5...HEAD --no-init --format json
what happened: lib/jurisdiction.ts: summary and evidence both say fan-in: 12 importers (p95 cutoff: 11, p99: 38), while the same finding’s scores.blast_radius_importers is 124 and blast_radius is 1 (saturated). Two numbers for the same quantity in one object, off by 10×.
why it matters: blast_radius is one of the six contract scores and it is being computed from a different importer set than the evidence the user is shown. Whichever is right, one of them is wrong, and a reader who spot-checks against the evidence will conclude the score is fabricated.
severity: real problem

CORRECTION to the `by_package` entry above: it is gated, not regressed: but the gate excludes real polyglot repos

repo: ebg
what happened: Read packages/core/src/discovery/by-package.ts. by_package requires MIN_PACKAGES = 2 directories containing one of package.json, pyproject.toml, setup.py, Cargo.toml, go.mod. ebg has exactly one (the root package.json). So its absence is correct by design. But: the 62-file Python subtree has no pyproject.toml/setup.py: it is a directory of scripts, which is how a great many polyglot repos carry their Python: and Package.swift is not in MANIFESTS at all despite 58 Swift files in a proper SPM layout. So by_package is absent even in the unhidden counterfactual (62 py + 375 js).
why it matters: The feature’s own docstring gives the motivating case as “a repo that is 70% TypeScript with one Python service”. ebg is literally that repo and gets nothing, because the detection of “package” is manifest-file presence rather than language-directory structure.
severity: real problem

Honest read of the default human `scan` as a user

repo: ebg
command: crimes scan ~/dev/ebg --no-init
what happened: 109 lines. It says “Top files by risk” and then prints every finding for the top 5 files: 78 findings, not a top-N. The claim in the docs is “signal over exhaustiveness, default shows top findings only”; the default shows exhaustive findings for a truncated file list, which is a different thing.
- The #1 file’s list is 22 of 25 entries that are all size complaints (God Function ×20, God File, plus two Return Shape Roulette on the same symbol). The three findings that are not about size: Duplicated Role/Status/Plan Check, High Fan-In/Fan-Out, Parallel Destination: are at positions 23, 24 and 25, i.e. the last three lines before the block ends. The differentiated detectors are ranked below a 61-line function that is 1.0× the threshold.
- Items 1 and 3 are the same function (applyRegistryTrust and its withTenant callback). Items 5 and 6 are the same function (findContactDetails / its inner arrow). So the top of the top file repeats itself twice in the first six lines.
- lib/config.ts is 9 consecutive Environment Roulette lines, one per env var (ANTHROPIC_API_KEY, COMPANIES_HOUSE_API_KEY, …). One finding per variable where one per file would say the same thing.
- Same-file contradiction visible without opening anything: app/agency/[agency]/(protected)/cases/[id]/page.tsx gets Parallel Destination (“/agency/:agency/cases/:id and /analyst/cases/:id serve overlapping user intent”) at item 13 and Orphaned Destination (“/agency/:agency/cases/:id appears to be unreachable from any nav, doc link, or in-repo import”) at item 15. One says it is a live destination duplicated elsewhere; the other says nothing reaches it.
- The id line is unusable. Each block ends with id=crime_00004 … crime_00065: a range with an ellipsis. There is no way to get the id of item 23. explain takes an id. So the default output cannot be fed into the next command in its own workflow.
- <anonymous> appears 6 times as a symbol in the top file. “An anonymous function is 208 lines long” is not locatable from the summary.
why it matters: With ~70 detectors, the default view is dominated by the two oldest and least differentiated ones. A first-time user’s impression of the product is “it counts lines”, and the findings that justify the positioning (agent risk, cross-file authority, IA drift) are visually last.
severity: real problem

`agent_risk` collapses into “big file / big function” on a real repo

repo: ebg
what happened: Top 20 findings by scores.agent_risk: 15 of 20 are large_function or large_file. The remaining 5 are direct_date (2), return_shape_roulette (3-way, two of which are the same symbol) and logic_in_comments (2). Nothing from the 0.16 correctness/authority set, nothing from IA drift, appears in the top 20 despite parallel_destination firing 52× and orphaned_destination 39×.
why it matters: CLAUDE.md states agent_risk “is the differentiator and must not be collapsed into severity”. Ranked on a real repo it is almost perfectly correlated with function length. Whatever the formula intends, the observable behaviour is a size ranking.
severity: real problem

Positive statements (the zero case)

repo: ebg, as scanned (i.e. Python invisible)
Python detectors produced zero findings: correct, there were zero .py files in the walked set, though for the wrong reason (see the dot-directory blocker).
All three cross_language_* detectors produced zero: correct given one language in the walked set; and route_drift/type_drift remain correctly zero even when Python is made visible, because the shapes they need are genuinely absent (no web framework, no Enum).
Reproducibility: two consecutive scan --all --format json runs produced byte-identical finding sets: same 759 findings, same ids, same order. No nondeterminism observed.
No file was written into ~/dev/ebg. git status --porcelain before and after shows only the pre-existing untracked docs/aplyid-company-context.md. No .crimes/ was created.

Detector coverage on ebg

31 of 68 registered detectors fired; 37 fired zero times, including all three cross_language_*, circular_dependency, deep_import, layer_violation, todo_density, concept_alias_drift, missing_agent_context, client_exposed_secret, and the whole *_disagreement family (type_, unit_, default_, requiredness_).
severity: informational, but deep_import and circular_dependency firing zero on an 81k-line Next.js app with a 30-import file is surprising enough to be worth a look.

Follow-up on the `deep_import` zero: mostly justified

Import-depth histogram over app lib components trigger: 241 at depth 2, 603 at depth 3, 2 at depth 4, 1 at depth 7. The depth-7 one is @/app/agency/[agency]/(protected)/cases/actions. So a zero is defensible, though that single import is exactly the shape the detector describes and it was not flagged. circular_dependency zero is unverified either way.
severity: annoyance

End of log. Nothing was written to ~/dev/ebg; final git status --porcelain shows only the pre-existing untracked docs/aplyid-company-context.md, and no .crimes/ exists.

buildfest

Friction log: buildfest (scale / performance case)

Repo: ~/dev/buildfest SHA: e14ef7c6fa58d220e5b1ec7b754e5792202ccadc (2026-02-13, “update package publish process”) Binary: ~/crimes-dogfood/bin/crimes: 0.17.0 Host: darwin 25.5.0, 10 cores, 64 GB RAM, node v24.18.0 Default ulimit -n: 1048576 (kern.maxfilesperproc 245760)

Baseline before any crimes run: no .crimes/ directory; git status --porcelain had 3 pre-existing untracked entries (.playwright-mcp/, docs/supabase-cleanup-prompts/, docs/supabase-triage.md).

Repo shape: 1529 tracked files (485 .ts, 367 .tsx, 71 .js, 128 .md, 116 .json, 79 .sql), 181,093 LOC across ts/tsx/js/jsx/mjs/cjs, 166 commits. Working tree on disk is 18 GB (node_modules + playwright videos + test-results); .git is 346 MB.

Entries

Baseline runs completed, no writes into the repo

repo: buildfest
command: /usr/bin/time -l ~/crimes-dogfood/bin/crimes scan ~/dev/buildfest --all --format json --no-init
what happened: exit 0, clean stderr. 26.67s wall (20.51 user / 2.61 sys), max RSS 308 MB, peak footprint 331 MB. 1556 findings, 2.3 MB JSON. Human scan run: 23.48s, max RSS 365 MB, 108 lines out. git status --porcelain identical before and after both runs; no .crimes/ created. --no-init behaved.
why it matters: positive result: 1177 files / 181k LOC in 27s single-threaded-ish with a 300 MB ceiling is fine for a repo this size, and read-only really is read-only.
severity: (not friction: recorded as a positive control)

`.gitignore` is NOT respected: a gitignored build-output directory contaminates 21% of all findings

repo: buildfest
command: crimes scan ~/dev/buildfest --all --format json --no-init
what happened: migrate-output/ is line 8 of ~/dev/buildfest/.gitignore and is a verbatim copy of apps/airquote. crimes walked into it anyway. git ls-files migrate-output | wc -l = 0; git check-ignore -v migrate-output = .gitignore:8:migrate-output/.
- 100 findings are located in gitignored files.
- 229 more findings name migrate-output in their summary/evidence: i.e. real source files are flagged because they “duplicate” a directory git has been told to forget.
- 323 / 1556 findings = 20.8% of the whole report.
- exact_duplicate_block: 136 of 158 (86%) are this one directory.
- It even shows up as a first-class entry in coverage.by_package (migrate-output/airquote, 69 files) and in the human report’s evidence strings as “production files”.
- Cross-check: other gitignored dirs (node_modules, .next, dist, .turbo) were excluded: because they’re in the hardcoded default exclude list, not because .gitignore was read. Untracked-but-not-ignored files (docs/supabase-cleanup-prompts/) were scanned, which is correct. So the built-in defaults work and the .gitignore layer does not.
why it matters: CLAUDE.md states the file walker is “fast-glob + ignore (respect .gitignore)”. It isn’t respecting it. Every monorepo with a codegen/migration/scratch output directory will get a report where the loudest detector is measuring a directory the maintainer deliberately excluded. This is the single largest precision leak in this scan and it is not a tuning problem: it’s a walker bug.
severity: blocker

Human reporter truncates `evidence` at 2 items: and for duplicates the dropped item is the only useful one

repo: buildfest
command: crimes scan ~/dev/buildfest --no-init
what happened: every duplicate line reads e.g. Exact Duplicate Block hash 9473ef42a72c… across 2 file(s), 2 occurrence(s), packages/auth/src/providers/UnifiedAuthProvider.tsx:367-395 (checkBusinessAccess): it names the file you are already looking at and hides the counterpart behind a truncated hash. The JSON is fine: crime_00014 carries three evidence strings and the third is packages/core/src/auth.tsx:308-336 (checkBusinessAccess). The human renderer prints evidence[0] and evidence[1] and drops the rest. For exact_duplicate_block the dropped one is the answer.
why it matters: a duplicate finding is only actionable if you know the other location, and the terminal view is the default view. As shipped you have to re-run with --format json and grep the hash. Had the counterpart been printed, the migrate-output/ contamination above would have been obvious in ten seconds instead of needing a script. Reporter-only fix; the schema already has the data.
severity: real problem

251 of 1556 findings (16%) have no `lines` field at all

repo: buildfest
command: crimes scan ... --format json
what happened: lines is present on 1305 findings and absent on 251. Every exact_duplicate_block (158), high_fan_in_fan_out (41), oversized_raster (13), duplicate_component_shape (12), accessible_interaction_risk (6), duplicated_role_status_plan_check (6), near_duplicate_block (5), design_token_escape (3), concept_alias_drift (2), docs_code_drift (2), circular_dependency (1), duplicated_navigation_source (1) and svg_with_embedded_raster (1) lacks it. For duplicates the line range exists but only as free text inside an evidence string ("...:367-395 (checkBusinessAccess)"). Other optional keys: symbol 588/1556, related_files 369/1556, discriminator 225/1556.
why it matters: the JSON output is stated to be the product contract. Any consumer that wants to place a finding in an editor: an LSP shim, a GitHub annotation, an agent deciding what to read: has to string-parse evidence for a sixth of all findings, and only for some detectors. Machine-readable line data should not be optional on a location-bearing finding.
severity: real problem

The human report is file-grouped, so one root cause becomes a wall

repo: buildfest
command: crimes scan ~/dev/buildfest --no-init (108 lines, 12.8 KB)
what happened: the report shows 5 files; within each file it prints every finding for that file, unranked and uncapped. File 2 (quote-search.ts) is 17 findings of which 11 are consecutive Exact Duplicate Block lines; file 3 (credits.ts) is 21 findings of which 9 are consecutive duplicates. Roughly 40 of the 108 lines are one repeated sentence with a different hex string.
why it matters: “Signal over exhaustiveness” is a stated product constraint and the default view violates it. A user’s eye slides off the block, and the genuinely distinct findings above and below it (Policy Doppelgänger, Half-UTC, DST-Naive Day Math) get lost inside it. A per-file cap, or collapsing N duplicates of the same detector into one line with a count, would fix it.
severity: real problem

Two different file counts in the same eight lines of output

repo: buildfest
command: crimes scan ~/dev/buildfest --no-init
what happened: the header says 1556 findings across 558 files; the footer says Showing 5 of 481 files. Nothing explains the 77-file gap (it appears to be the tests/ + scripts/ files rolled into “Also flagged elsewhere”).
why it matters: small, but it’s the first and last number a user reads and they contradict each other. Costs trust for no benefit.
severity: annoyance

`coverage.files_by_language` collapses TS/TSX/JS into one bucket called `js`

repo: buildfest
command: crimes scan ... --format json
what happened: the repo is 485 .ts + 367 .tsx + 71 .js; coverage reports {"js": 992} and nothing else. Also 79 tracked .sql and 116 .json files are invisible: they aren’t in files_by_language or in universal_only_by_extension (which only lists .md 149 and .sh 36).
why it matters: the protocol asks us to verify from coverage that a pack actually saw what we think it saw. With one bucket you cannot tell whether the TSX side of the parser ran at all, and you cannot tell whether the 79 SQL files were skipped deliberately or dropped silently. files_total (1177) minus js (992) minus universal-only (185) = 0, so SQL/JSON simply aren’t counted anywhere: that’s an unexplained hole in the coverage contract.
severity: real problem

Zero case: positive statement

repo: buildfest (single-language: JS/TS, zero .py)
command: crimes scan ~/dev/buildfest --all --format json --no-init
what happened: coverage.packs_loaded = ["universal","language-js","language-py"]. Packs that actually produced findings = universal (181) and language-js (1375). language-py produced exactly 0 findings and there are 0 .py files: correct. The three cross_language_* detectors produced exactly 0 findings: correct for a single-language repo; the cross-language pack does not even appear in packs_loaded.
why it matters: positive control passed. packs_loaded listing language-py on a repo with no Python is mildly misleading (it reports load, not applicability, which the protocol already flags) but it did not produce a single false positive.
severity: annoyance (the packs_loaded wording only)

0.17 surfaces confirmed present

repo: buildfest
what happened: the large_file docs shape fired on 3 Markdown files with a distinct threshold and an explicit shape line, e.g. docs/specs/ALLURE_SPEC.md: 2327 non-empty lines, 2.3× the configured 1000-line document threshold, shape: document (matches .md/.mdx/.markdown/.rst/.adoc/.txt). discriminator fingerprints are present on 225 findings across exact_duplicate_block, magic_domain_literal_scatter and near_duplicate_block.
why it matters: both 0.17 features are real and visible in the wild, not fixture-only. The document threshold (1000) is clearly separated from the code threshold (300), and the summary text is genuinely different rather than the code copy with a noun swapped.
severity: (positive control)

FD-LIMIT BUG HUNT: the 0.17.0 bounded pool bounds the fd count but the failure mode is still silent, and it is now nondeterministic

repo: buildfest (1177 files, the largest in the corpus)

commands:

# baseline, unbounded descriptors
/usr/bin/time -l ~/crimes-dogfood/bin/crimes scan ~/dev/buildfest --all --format json --no-init

# the experiment — one run per limit, stdout captured, stderr captured
bash -c "ulimit -n <L>; cd ~; exec ~/crimes-dogfood/bin/crimes scan ~/dev/buildfest \
  --all --format json --no-init" > scan-<L>.json 2> scan-<L>.err

what happened:

`ulimit -n`	reps	exit	total findings	`exact_duplicate_block`	`duplicate_component_shape`	`near_duplicate_block`	stderr
1048576 (default)	1	0	1556	158	12	5	clean
1024	1	0	1556	158	12	5	clean
512	1	0	1556	158	12	5	clean
256	1	0	1556	158	12	5	clean
128	1	0	1556	158	12	5	clean
64	4	0	1556	158	12	5	clean
48	3	0	1556	158	12	5	clean
40	3	0	1522 / 1535 / 1543	126 / 137 / 145	12	3 / 5 / 5	clean
40	1	143	(empty stdout)	n/a	n/a	n/a	clean
32	2	0	1391 / 1393	6 / 10	0	4 / 2	clean
32	1	143	(empty stdout)	n/a	n/a	n/a	clean
28	2	0	1386 / 1386	5	0	0	clean
24	1	0	1383	2	0	0	clean
16	1	1	(empty)	n/a	n/a	n/a	`Error: spawn git EMFILE` + raw node stack
12	1	1	(empty)	n/a	n/a	n/a	`Error: spawn git EMFILE` + raw node stack

The good news, stated plainly: at every realistic descriptor budget the fix holds. 1024 (Linux default), 256 (classic macOS default), even 64 and 48 all produce output that is byte-identical to the unbounded baseline: sha256 = d694948ff7ebefbb8c978ce320c907ed3c5e694dd6c8290f8b671ee0bde97c34 across six independent runs at six different limits. DEFAULT_IO_CONCURRENCY = 32 is correctly sized and the pool really does keep the fd count flat.

The bad news: below ~48 the scan does not crash and does not warn. It silently reports fewer findings, and which findings it drops changes run to run. Three runs at ulimit -n 40 on a byte-identical tree returned three different answers (1522, 1535, 1543 findings; 126, 137, 145 duplicates). Every one exited 0 with empty stderr and an identical coverage block: files_total: 1177, packs_loaded: ["universal","language-js","language-py"], by_package 31 entries. There is nothing in the output a user or an agent could use to tell a degraded scan from a clean one. summary.high even stays pinned at 187 in every run, so the headline number is stable while a tenth of the report evaporates.

Root cause, located. It is not indexes.ts. safelyBuildFunctionHashIndex never fires, because the index build succeeds: it succeeds on a truncated corpus. In packages/core/src/ast-hash/function-index.ts:72-79:

const perFile = await mapWithConcurrency(candidate, async (abs) => {
  const hits: FileHits = { exact: [], shape: [] };
  let source: string;
  try {
    source = await readFile(abs, "utf8");
  } catch {
    return hits;          // <-- EMFILE becomes "this file has no functions"
  }

packages/core/src/jsx/shape-index.ts:65-66 has the identical shape. That bare catch is presumably there for binaries and races, but it cannot tell ENOENT (ignore it) from EMFILE (the run is invalid). So each descriptor exhaustion silently deletes one file from the duplicate corpus, the pool moves on, the index reports success, and the detector emits a random subset. That is why the loss is partial and nondeterministic rather than all-or-nothing: a strictly worse failure than the index-level undefined that indexes.ts was designed to produce.

Two more failure modes at the bottom end. At 32 and 40 the process was twice killed with exit 143 (SIGTERM), empty stdout, and completely empty stderr: no message at all. And at 16 and below the very first spawn git throws an unhandled 'error' event, so the user gets a raw node:events:487 throw er; stack trace with an errno: -24 object rather than a sentence. Neither path is handled.

Portability caveat worth flagging: the pool is a fixed 32 and the docstring reasons about macOS 256 / Linux 1024, which is sound. But this box reports ulimit -n 1048576, so the stock configuration was never anywhere near the cliff: I had to construct it. The margin between “works” (48) and “silently wrong” (40) is 8 descriptors. Anything else in the process holding fds: a language pack that opens a tree-sitter WASM blob, a git subprocess, a future parallel parser: eats directly into that margin.

why it matters: the 0.17.0 release note claims the fd fan-out is fixed. Half of it is: the descriptor count is genuinely bounded and the numbers above prove it at every realistic limit. The half that is not fixed is the one that actually costs users trust: a degraded scan is indistinguishable from a clean one, and now it is nondeterministically degraded, so two engineers on the same commit can disagree about what the tool said and neither can prove anything. The fix is small and does not touch detector behaviour: re-throw (or count and surface) EMFILE/ENFILE in those two catch blocks, and put a read-failure count into coverage so “the scan was complete” becomes an assertable fact instead of an assumption.
severity: blocker (the silent-degradation half); the descriptor bound itself: fixed, verified

FD sweep: final numbers (supersedes the reps counts in the table above)

repo: buildfest
what happened: completed sweep, 2-3 reps at each low limit. Confirms the variance, which is the point:
- 40: 1522 / 1535 / 1543 findings (edb 126 / 137 / 145): three different answers on an unchanged tree, plus one SIGTERM kill.
- 32: 1391 (edb 6, dcs 0) / 1393 (edb 10, dcs 0), plus one SIGTERM kill.
- 28: 1386 / 1386 (edb 5, dcs 0, ndb 0): stable but wrong.
- 24: 1383 (edb 2) / 1384 (edb 1) / 1388 (edb 6, dcs 1): three different answers again.
- 20, 16, 12: Error: spawn git EMFILE, unhandled 'error' event, raw stack trace, exit 1, empty output. At least this one is loud.
- Every exit-0 run above had empty stderr and an identical coverage block.
severity: blocker (same defect as the entry above; this is the reproducibility evidence for it)

DETERMINISM CHECK: clean pass, no caveats

repo: buildfest
command: ~/crimes-dogfood/bin/crimes scan ~/dev/buildfest --all --format json --no-init, run 3× on an unchanged tree
what happened: all three runs exit 0 with 0 bytes on stderr and are byte-identical, sha256 = d694948ff7ebefbb8c978ce320c907ed3c5e694dd6c8290f8b671ee0bde97c34, before and after stripping every id field (id-stripped sha256 = e8f14698482f…, also identical across all three). Because id is assigned in rank order, byte-identity with ids included is the stronger statement: the ranking is stable too, not just the finding set. Counting the six fd-limit runs at 64/128/256/512/1024/default, that is 9 independent runs producing the same 2,412,426 bytes. The three detectors named in the 0.17.0 note are covered directly: exact_duplicate_block (158 findings, 158 identical hash <12-hex>… across N file(s) evidence strings), near_duplicate_block (5), magic_domain_literal_scatter (62). No drift in the finding set, the evidence strings, the scores, or the ordering.
why it matters: the 0.17.0 reproducibility claim holds, and the sorted-insertion fix in function-index.ts is doing what its docstring says. This is the cleanest result of the run.
severity: (positive control: pass)

`churn` is 0.00 on all 1556 findings, and only `explain` ever tells you why

repo: buildfest
command: crimes scan ~/dev/buildfest --all --format json --no-init
what happened: scores.churn == 0 and scores.recency == 0 for every single one of the 1556 findings. The repo has 166 commits and is not shallow: but its last commit is 2026-02-13 and today is 2026-08-02, so the default 90-day window contains zero commits (git rev-list --count --since=90.days HEAD = 0; --since=365.days = 166). So the number is correct. The problem is that nothing says so. scan prints Risk: churn low on every file: “low” is a claim about the code; the truth is “not measured”. Only crimes explain ever renders the real sentence: churn: 0.00: no commits touched this file in the last 90 days. That string exists in the codebase and is never shown in the command a user actually runs.
why it matters: churn is one of the six scores and a stated differentiator. On any repo that has been quiet for a quarter: a library between releases, a repo you just cloned, anything you inherited: a sixth of the scoring model silently zeroes out and the ranking degenerates, and the default output actively misreports it as a low-risk signal rather than an absent one. scan should say “churn: not measured (no commits in window)” and, ideally, widen or suggest widening the window.
severity: real problem

`hotspots` default 90d window makes the command actively misleading: and its top hit is a PNG

repo: buildfest
command: crimes hotspots ~/dev/buildfest --no-init (54.6s: the slowest command in the suite)
what happened: with zero commits in the default window, every file scores an identical risk 0.40 (40%) with 0 changes, so the “ranking” collapses to alphabetical order. The #1 hotspot in a 181k-line TypeScript monorepo is:
```
  1. apps/airquote/public/images/airquote-header.png  risk 0.40 (40%)
     0 changes
     1 finding · highest high
```
Entries 2-10 are also all risk 0.40 · 0 changes, all under apps/airquote/ because “a” sorts first. Run with a window that actually contains history and it is a genuinely good command: crimes hotspots ~/dev/buildfest --since 1y (36.9s) returns package.json at risk 0.91 · 17 changes · latest 2026-02-13, then scripts/dev-health-check.ts at 0.84 · 23 changes, then apps/bossary/src/app/layout.tsx at 0.82 · 14 changes: real spread, real dates, obviously useful. Separately: the header file count changes with the window: since 90d · 558 files vs since 1y · 3101 files: so the same field means two different things depending on a flag.
why it matters: a churn-ranked command whose churn input is empty should refuse or warn, not emit a confidently-ordered list of constants. As shipped, the first thing a new user runs on a quiet repo tells them their header image is their biggest risk. That is the kind of first impression that ends the trial. Fix is cheap: detect zero commits in window, print one line, and suggest --since.
severity: blocker (for first-run credibility on any repo quiet for >90 days)

`crimes diff` human output is three integers and names nothing

repo: buildfest
command: crimes diff HEAD~20...HEAD --no-init (56.5s; 937 files changed in the range)
what happened: the entire output is 7 lines:
```
CRIMES DIFF
base: HEAD~20
head: HEAD

New crimes: 76
Fixed crimes: 37
Unchanged crimes: 1046
```
No file, no line, no charge, no way to see what the 76 new ones are. --help offers --format, --no-color, --root, --show-suppressed, --fail-on: there is no --all or -v. Meanwhile --format json returns new_findings as a fully populated 76-element array with types and paths ({"type":"large_function","file":"packages/auth/src/ui/sign-up-form.tsx"}).
why it matters: the whole point of diff is “what did this branch introduce”. The data is right there in the same process and the human renderer throws it away. This is the single highest value-per-line-of-code fix in the list: it is purely presentational and it turns a useless command into the one you would wire into a pre-push hook.
severity: real problem

`crimes verdict` spends 68 seconds to compare a commit with itself and doesn’t mention that

repo: buildfest
command: crimes verdict --no-init (cwd = repo), 67.8s: the slowest command measured
what happened: exits 0 with Verdict: UNCHANGED / New: 0 / Fixed: 0 / Reason: no new findings and no fixed findings. The actual reason is that origin/main and HEAD are the same commit (e14ef7c… both) and git diff --name-only origin/main...HEAD is empty. It ran two full scans of 1177 files to discover that.
why it matters: 68 seconds is a long time to be told nothing, and the stated reason is not the real reason. A git rev-parse comparison up front costs microseconds and would let it say “base and head are the same commit: nothing to compare” instantly. Users will read “UNCHANGED” as “I checked and it’s fine” rather than “I compared X to X”.
severity: real problem

`crimes context` is the best command in the tool

repo: buildfest
command: crimes context apps/airquote/src/lib/credits.ts --no-init (cwd = repo), 2.13s, 83 lines
what happened: fast, scoped, and led with the thing an agent actually needs: Agent guidance (3 imperative lines), Related files with a stated reason for each (shares domain token "credits"), Likely tests, then findings with full evidence. This is the one output I would keep in an agent loop. Two caveats. (a) Run from outside the repo it fails with crimes: file not found: apps/... and exit 2: --root exists and would have fixed it, but the error message doesn’t mention it. (b) It silently resolved the root to the nested package apps/airquote, not the monorepo root: the header reads file: src/lib/credits.ts and it reports 6 findings where the monorepo-wide scan reported 21 for the same file. Neither number is wrong, but nothing tells you the scope changed, and the id=crime_00001 it prints is local to that narrowed run and does not correspond to the crime_00477 the full scan gave the same finding.
why it matters: mostly a positive result, but (b) is a trap: ids printed by context look like ids printed by scan and are not interchangeable. Either qualify them or drop them from context.
severity: annoyance

The “stable fingerprint” is not in the JSON, and `explain --help` documents the wrong shape for a sixth of findings

repo: buildfest

commands:

crimes explain "exact_duplicate_block::packages/auth/.../UnifiedAuthProvider.tsx::checkBusinessAccess"   # exit 2
crimes explain "exact_duplicate_block::packages/auth/.../UnifiedAuthProvider.tsx::::9473ef42a72c"        # exit 0

what happened: explain --help says the fingerprint is <type>::<file>::<symbol>. That works for symbol-bearing detectors (verified on name_behavior_mismatch::…::getSupabaseClient). For the 251 findings with no symbol the real shape is four parts with an empty slot (<type>::<file>::::<discriminator>) which is documented nowhere except inside explain’s own suppression hint, i.e. you must already have run explain by id to learn how to call explain by fingerprint. And there is no fingerprint field in the scan JSON at all (keys are id, type, charge, severity, confidence, file, summary, evidence, effort, fix_shape, scores, suggested_actions, pack, detector_id, tier + optional lines, symbol, related_files, discriminator). My first attempt at the 3-part form also cost a 28-second full re-scan before failing, because explain without --from re-scans the repo and gives no indication it is about to.
why it matters: the fingerprint is sold as the durable handle: it’s what crimes ignore takes and what any suppression file or CI baseline would key on. A durable handle that isn’t emitted in the machine-readable output, and whose documented format is wrong for 16% of findings, isn’t durable. Emitting fingerprint on every finding is a small additive schema change and removes the whole class of problem.
severity: real problem

Honest read of the default `scan` output, as a user

repo: buildfest
command: crimes scan ~/dev/buildfest --no-init: 23.5s, 108 lines / 12.8 KB
what happened: the length is right. 108 lines is a screen and a half; I read the whole thing without scrolling fatigue, and the closing line: → Start with 'crimes context apps/dev-portal/src/app/admin/settings/page.tsx': it concentrates the most risk in this scan: is a genuinely good ending: one command, one file, no ambiguity. What it gets wrong is within the frame. The five files it picked are reasonable, but roughly 40 of the 108 lines are Exact Duplicate Block hash <hex>… across 2 file(s), 2 occurrence(s) repeated with a different hex string, and (per the entries above) 86% of those are a .gitignored directory and none of them name the other file. So the densest third of the report is simultaneously the least trustworthy and the least actionable part of it, and it sits directly on top of the findings I’d actually act on (Policy Doppelgänger · plan plans rule: normalised rule: ($expr || .free), 13 occurrence(s) across 5 file(s), Half-UTC, Half-Local, DST-Naive Day Math), which are individually excellent. It does lead with what matters at the file level and buries the lede at the finding level. My honest reaction: I trusted line 2, skimmed lines 30-70, and re-engaged at the last line.
why it matters: the ordering machinery is fine: it’s the per-file expansion that is uncapped. Collapsing repeated same-detector findings into one line with a count would cut ~35 lines of noise and surface ~10 distinct findings currently hidden inside the block, with no scoring change.
severity: real problem

`.gitignore` bug: minimal reproduction (6 lines, outside any real repo)

repo: synthetic, built in scratch: confirms the buildfest finding is a general walker defect, not a migrate-output/ quirk

command:

mkdir -p t/src t/generated && cd t && git init -q .
printf 'generated/\n' > .gitignore
printf 'export function alpha(x: number) {\n  const now = new Date();\n  return x + now.getTime();\n}\n' > src/a.ts
cp src/a.ts generated/a.ts
git add -A && git commit -qm init

git ls-files                       # -> .gitignore src/a.ts        (generated/ correctly untracked)
git check-ignore -v generated/a.ts # -> .gitignore:1:generated/    generated/a.ts

crimes scan t --all --format json --no-init

what happened:
```
files_total: 2
findings: 2
files with findings: src/a.ts, generated/a.ts
```
crimes walks straight into a directory that git check-ignore reports as ignored by the repo’s own .gitignore, counts it in coverage.files_total, and emits findings from it.
why it matters: this reduces the 21%-contamination result on buildfest to a one-line cause and makes it trivially testable. node_modules/dist/.next/.turbo are excluded by the hardcoded default list, which is why the bug has stayed invisible: those cover the common cases. Any project-specific ignore (codegen output, migration scratch, vendored snapshots, fixture corpora) is scanned as production code. Given exact_duplicate_block is inherently cross-file, one ignored copy of a directory poisons findings in the real files too, which is how 34 ignored files produced 323 contaminated findings.
severity: blocker

Top 5, ranked by how much they cost the product

.gitignore is ignored: blocker, one-line repro, 21% of this repo’s findings contaminated and 86% of exact_duplicate_block. Walker fix.
Silent, nondeterministic degradation under descriptor pressure: blocker. The 0.17.0 pool bound genuinely works at every realistic ulimit -n; the failure mode below ~48 is still exit 0, empty stderr, identical coverage, and a different answer each run. Cause is the bare catch around readFile in ast-hash/function-index.ts:77 and jsx/shape-index.ts:66.
hotspots on a repo quiet for >90 days: blocker for first impressions. Constant risk, alphabetical order, a PNG at #1. Needs a zero-commits-in-window guard.
crimes diff human output is three integers: real problem, pure reporter gap, the 76 new findings are already in the JSON.
The duplicate-block wall in scan, and the missing counterpart file: real problem. ~40 of 108 lines are one repeated sentence, and the one datum that would make them actionable (evidence[2], the other file) is truncated by the human renderer.

Honourable mentions: churn silently 0 on every finding with only explain explaining why; lines absent on 16% of findings; no fingerprint field in the JSON while crimes ignore requires one; verdict burning 68s to compare a commit with itself.

What worked, stated plainly: determinism is perfect (9 byte-identical runs). Performance is good (27s, 308 MB peak for 1177 files / 181k LOC). Read-only is genuinely read-only (--no-init honoured, no .crimes/, no stray worktrees, git status byte-identical before and after ~30 command invocations). The zero cases pass (language-py loaded, 0 findings, 0 .py; cross_language_* 0 findings on a single-language repo). Both 0.17 surfaces are real in the wild. crimes context (2.1s) and crimes explain --from are the two best outputs in the tool.

zulip

Friction log: zulip

repo: ~/crimes-dogfood/corpus/zulip
SHA: f6470bb378888dc5f274aafd426ee79880550dc5
build: crimes 0.17.0 (~/crimes-dogfood/bin/crimes)
ulimit -n: 1048576
size: 8919 tracked files; 2016 .py (418,474 lines), 471 .ts (150,490 lines), 12 .js (3,546), 224 .html (17,925), 469 .hbs, 411 .md + 464 .mdx. Working tree 296M (154M of it .git).
role in round: second independent test of the 0.15.0 cross_language_* detectors, and a partial CONTROL: Zulip has a famously strict review culture.

Entries appended in the order they happened.

Timed baseline

command: /usr/bin/time -l crimes scan $R --all --format json --no-init
what happened: exit 0, 81.27s wall (45.11 user / 5.04 sys), max RSS 397,197,312 B = 378.8 MiB, peak footprint 453,686,368 B. 5,234,432 B of JSON. Nothing on stderr.
why it matters: baseline. Memory is fine; 81s on a 570k-line repo is tolerable but not fast.
severity: n/a

The human `scan` took 1.8x longer than the `--all` JSON scan

command: crimes scan $R --no-init vs crimes scan $R --all --format json --no-init
what happened: JSON --all = 81.3s. Plain human scan = 147.8s. The human scan renders fewer findings (130 lines of output vs 3,759 findings) yet takes nearly twice as long. Run-to-run variance on the same machine, same repo, nothing else running.
why it matters: either detection is nondeterministically slow or the human renderer is doing real work per finding it then throws away. Either way “the default command is the slowest one” is a bad first impression: a new user’s very first crimes scan on a large repo is a 2.5-minute wait with no progress output past the spinner.
severity: real problem

First human `scan` was killed with exit 143 at 22.8s

command: /usr/bin/time -p crimes scan $R --no-init > out 2> err
what happened: EXIT=143 (SIGTERM), time: command terminated abnormally, 0 bytes of output written. Immediate identical re-run succeeded. Not reproduced on retry.
why it matters: could be environmental (sandbox), but crimes produced no partial output and no error line: a user who hits this sees an empty file and no explanation. Recorded, not diagnosed.
severity: annoyance (unreproduced)

* ZERO cross-language findings on a textbook client/server repo *

command: crimes scan $R --all --format json --no-init
what happened: cross_language_route_drift, cross_language_type_drift, cross_language_concept_alias_drift = 0 / 0 / 0. And coverage.by_package is absent from the JSON entirely, and "cross" is not in coverage.packs_loaded (["universal","language-js","language-py"]).
why it matters: Zulip is the canonical shape these detectors were written for: 2,016 Python files with Django URL routing in zproject/urls.py, consumed by 471 TypeScript files that call those endpoints via channel.get/post. If the detector can’t see this pair, the 0.15.0 pack has never fired outside its own fixtures on either dogfood repo. Diagnosed below.
severity: blocker (for the 0.15 surface)

Detector count table (3,759 findings, 572k lines of code = 6.6 findings / 1000 lines)

large_function.py 898, large_function.js 563 → 39% of the entire report is one detector.
large_file 355, deep_import.py 275, commented_out_code 267, boolean_naming_drift .py 265 + .js 190 = 455.
Those six lines are 2,813 / 3,759 = 75% of the report.
why it matters: on the most heavily-reviewed repo in the corpus. See control assessment.
severity: real problem

Cross-language diagnosis: three independent structural misses, all confirmed by running the parsers directly

command: custom probe over packages/language-py/dist + packages/language-js/dist on all 8,919 Zulip files
what happened:
- PY ROUTES total = 0 across 2,016 Python files, 0 parse errors.
- JS FETCHSITES total = 0 across all .ts.
- PY CLOSED SETS = 0; JS CLOSED SETS = 25; name matches = 0.
causes, in order:
1. Routes are only read from decorators. routeFromDecorator() in packages/language-py/src/parse/routes.ts is the sole route source. Zulip declares 387 URLs as Django URLconf function calls: rest_path("realm/domains", GET=…, POST=…) and path(…) inside v1_api_and_json_patterns in zproject/urls.py. Repo-wide count of @app.get-shaped decorators: 0. Django/Flask-add_url_rule/urlpatterns is the dominant Python web idiom and the extractor cannot see it at all. The FastAPI decorator shape is the only one supported.
2. Even if path() were parsed, routeFromDecorator rejects any path not starting with /. Every Zulip route is relative ("realm", "messages/<int:message_id>") because the prefix comes from include(). So a path() extractor would still need prefix composition to produce anything.
3. Frontend calls are only read from fetch(...) or axios|http|client|api|request receivers with a positional string URL. Zulip makes 140 calls through its own channel module: channel.post({url: "/json/typing", data, success}). Receiver channel fails REQUEST_RECEIVERS; and the URL lives in an object-literal first argument, which literalText() returns undefined for. jQuery-$.ajax({url}) / any options-bag HTTP wrapper is invisible. This is a very common shape.
4. cross_language_type_drift is structurally impossible here. The TS half only accepts string-literal unions (25 found). The Python half only records a member value when the RHS is a string node. All 29 of Zulip’s Enum/IntEnum classes use integer values (MEMBERS_ONLY = 1). String-enum-vs-string-union is a narrow slice of real Python.
5. cross_language_concept_alias_drift requires an IA alias group whose aliases split strictly disjointly by language. Zulip’s textbook case: backend Stream, product-facing channel: did not produce one.
why it matters: two dogfood repos, zero cross-language findings. The 0.15.0 pack has still never fired outside its own fixtures. The failure is not calibration, it is coverage of idioms: FastAPI-decorator + bare-fetch + string-enum is one narrow stack, and Zulip (Django + jQuery wrapper + IntEnum) misses all three.
severity: blocker

`packs_loaded` cannot tell you whether the cross-language pack ran

what happened: coverage.packs_loaded = ["universal","language-js","language-py"]. "cross-language" is never a member: buildCoverage() computes it as ["universal", ...router.registeredPacks()], and the cross slate is not a language router pack. Every cross finding is emitted with the pack of its anchor file anyway (there are none here to check).
why it matters: the protocol explicitly warns “packs_loaded reports what was loaded, not what ran”. For the cross pack it doesn’t even report that. There is no field in the JSON that distinguishes “cross-language ran and found nothing” from “cross-language never ran”. I had to read the source and re-run the parsers by hand to answer the round’s key question.
severity: real problem

`coverage.by_package` is absent on a textbook polyglot repo

what happened: Zulip has package.json + pyproject.toml both at the repo root, plus one nested starlight_help/package.json. buildByPackage() requires MIN_PACKAGES = 2 nested (non-root) package dirs; Zulip has 1, so the field is dropped.
why it matters: by_package is the 0.15.0 answer to “which part is the Python one”, and the single most common polyglot layout: one root manifest per language: never qualifies. The field only appears for pnpm/uv-style workspaces, not for the client/server repos the cross pack targets.
severity: real problem

`contract_drift` summaries name the same identifier twice and read as nonsense

example: “CustomProfileFieldData and CustomProfileFieldData appear to describe the same record but disagree on 3 fields.”
why it matters: the sentence is literally “X and X disagree”. A user cannot tell from the summary that these are two different declarations in different files; the disambiguation is only in evidence. Every same-name pair (most of the 15) reads this way.
severity: annoyance

`contract_drift` matched three pairs of local variables both called `parsed`

files: web/src/invite.ts:224 vs stream_edit_subscribers.ts:343, user_profile.ts:633, user_group_edit.ts:2209
what happened: three separate high/medium findings claiming parsed and parsed “describe the same record”. They are unrelated local zod parse results in unrelated modules that happen to share a throwaway variable name and both have a code field.
why it matters: the concept-matcher normalises on the identifier, and parsed/result/data/opts are throwaway names, not concepts. 3 of 15 contract_drift findings (20%) come from one such name.
severity: real problem

CORRECTION to the probe above: `parsePyFile` is async; my first two probes silently read a Promise

what happened: parsePyFile() returns a Promise. Reading .functions off it yields undefined; my probe printed PY CLOSED SETS = 0 when the real answer is 19. Re-ran with await.
why it matters: noting it because it is my error, not crimes’, and because the corrected numbers change one of the three diagnoses below from “structurally impossible” to “ran and legitimately found nothing”.
severity: n/a (self-correction)

Corrected cross-language numbers (all with `await`)

PY: files=2016  functions=13311  classes=2906  filesWithSyntaxErrors=0
PY ROUTES        = 0     <- route_drift early-returns on backend.length===0
PY CLOSED SETS   = 19
JS FETCHSITES    = 0     <- route_drift would early-return here too
JS CLOSED SETS   = 25
NAME MATCHES     = 0     <- type_drift RAN and found nothing

cross_language_route_drift: blocked, never reached its comparison. Two independent causes (Django URLconf path()/rest_path() not read; channel.post({url}) options-bag not read).
cross_language_type_drift: ran, both sides non-empty, zero name matches. Genuine zero. But 14 of the 19 Python “closed sets” are junk: the allValued fallback grabs any class whose members are all string constants, e.g. Integration (members are image-path templates images/integrations/logos/{name}.png), AppleAuthMixin (OAuth URLs), MissingEventHeaderTestCase (a test fixture URL). Only ~5 are actual enums. So even the input pool is mostly noise.
cross_language_concept_alias_drift: ran (aliasGroups non-empty: the single-language concept_alias_drift.js fired once), zero groups split strictly-disjointly across languages. Zulip’s own famous rename (backend Stream / product-facing “channel”) was not caught.
severity: blocker for route_drift; real problem for the type_drift input pool.

Python parser health: clean

2,016 .py files, 0 with hasSyntaxErrors, 13,311 functions + 2,906 classes extracted.
Spot checks with exact function counts: zproject/backends.py 177/177 defs, webhooks/github/view.py 76/76, lib/digest.py 25/25, lib/onboarding.py 8/8.
match (PEP 634, 7 files) parses: build_preset_config at webhooks/common.py:84–112 bounds the match at :92 correctly. Walrus parses: send_initial_realm_messages 326–589 spans the := at :462. PEP 604 -> int | None (355 files) parses.
Not exercised here: PEP 695 generics (def f[T]), type X = statements, except*: Zulip is requires-python = ">=3.10" and uses none of them. Zero occurrences repo-wide. So 3.12 syntax remains untested by this repo.
severity: n/a: positive result

`large_function.py` charges “God Function” on a clean 9-line function, and its own evidence says the threshold was not exceeded

finding: zerver/tornado/event_queue.py:1602 process_realm_user_add_event
summary: “process_realm_user_add_event is 9 lines and nests 4 levels deep.”
evidence[0]: "9 lines (threshold 50 for shape \"domain\")"
the code: a 9-line event fan-out: for user: for client: if accepts: if x: continue; add_event(). Idiomatic, reviewed, nothing wrong with it.
scale: 172 of 898 large_function.py findings (19%) state a line count at or below the threshold they quote. Every one is charged “God Function”.
why it matters: the detector is really firing on nesting depth >= 4, but the charge, the summary’s first clause, and the primary evidence line are all about size. A user reading “God Function: 9 lines (threshold 50)” concludes the tool is broken, and they are not wrong to. In Python, where every for/with/if/try is an indent level, depth 4 inside a function body is completely ordinary.
why it matters more: large_function (.py 898 + .js 563 = 1,461) is 39% of the whole report. A fifth of its Python half is self-contradicting.
severity: real problem

`blast_radius` is saturated for half the report, and the human output prints a number a user can disprove in 5 seconds

command: crimes scan $R --no-init
what happened: blast_radius_importers takes only 37 distinct values across 3,759 findings. Two of them account for most of it: 798 (866 findings) and 324 (825 findings). No file ever gets two different values. blast_radius is min(transitive_importers / 50, 1), so 1,758 findings (47%) are pinned at the maximum, 1.0, and another 1,471 are at 0. The score is effectively ternary.
the disprovable claim: the human report says of zerver/data_import/slack.py: “blast top-quartile (798 importers)”. Actual direct importers of zerver.data_import.slack: 5. For web/src/inbox_ui.ts it prints 324; actual: 4. For corporate/lib/stripe.py it prints 798; actual: 49.
why it matters: the number is a transitive closure, but nothing in the output says so, and in a monolith with import cycles the closure collapses to one giant number shared by everything. Two consequences: (a) blast_radius cannot rank anything in a repo of this shape, and it is one of the six scores the aggregate ranking depends on; (b) a maintainer who checks the one concrete number in the risk line finds it off by 160x and stops trusting the rest.
severity: real problem

The default human report is 26 lines of “God Function” before anything interesting

command: crimes scan $R --no-init
what happened: 130 lines total. The top file’s section is 37 lines, of which 26 are God Function. The genuinely interesting findings in that section: Blocking the Hot Path, Catch and Release, Abstraction Laundering, False Identity, Logic in the Alibi: are ranked below the twentieth God Function. In corporate/lib/stripe.py the God Function list runs from a 291-line function down to a 27-line one before the first non-large_function finding appears.
and the self-contradiction is right in the default output, at #17 of the top file: “God Function · get_price_per_license() 27 lines (threshold 50 for shape “domain”), nesting depth 4”.
why it matters: the lede is buried by an order of magnitude. A maintainer’s first impression of crimes on their repo is “it counts lines and calls me names about it”. The one detector that is 39% of the report is the least differentiated thing crimes does: every linter has a function-length rule, and Zulip has consciously chosen not to enable one.
severity: real problem

”Showing 5 of 1240 files”

The default view shows 0.4% of the files with findings. There is no ordering explanation, no “and here is the one thing to do”. The closing hint (→ Start with crimes context corporate/lib/stripe.py) is the single best line in the output and it is the last one.
severity: annoyance

Nice: the tool quotes the exact call shape its own cross-language detector cannot parse

Human output line 119: Catch and Release · check_thumbnail_status → get protected operation: const response: unknown = await channel.get({ url: `/json/thumbnail/status/${path_id}`, })
swallowed_error.js reads channel.get({url: ...}) fine. collectFetchSite does not. The two collectors disagree about what an HTTP call looks like inside the same parser.
severity: real problem (it is the fix hint for the cross-language miss)

`commented_out_code` charges Zulip for having the best explanatory comments in the corpus

command: crimes context zerver/lib/narrow.py --no-init
what happened: 9 of the 23 findings on that file are Commented-Out Code. Two examples, verbatim from the source:
- narrow.py:243–257: “This class has an important security invariant: None of these methods ever *add* messages to a query’s result…”: 15 lines of prose documenting a security invariant.
- narrow.py:1055–1071: 17 lines of prose explaining topic-name HTML escaping and why process_fts_updates needs no special handling.
the trigger: evidence says ">=3 lines contain code-shaped tokens (def/fn/function/class/let/var/const/etc.)". In the first block the tokens are the English words “class” (“This class has an important…”), add_term, query.filter. In the second, “function” (“The escape_html function is used to…”).
the summary: “Code left in comments stales: agents may revive incorrect snippets.” It is not code. Nothing to revive.
scale: 293 commented_out_code findings (267 universal + 26 js): 7.8% of the whole report.
why it matters: this is the single most backwards result of the run. Zulip’s long, careful, prose-heavy comments are the reason its code is maintainable, and they are exactly the artefact an agent benefits from most. crimes charges them as a defect. On the control repo, the detector is inverted.
severity: blocker (for this detector)

`todo_density` fires below its own stated threshold 62% of the time

evidence, verbatim: ["2× TODO", "10.1 markers per 1k LOC (threshold 10)"]: two TODOs in a ~200-line file.
45 of 73 todo_density findings (62%) quote a per-1k figure at or below the threshold in the same string.
why it matters: third detector in this repo whose evidence line reads as a self-refutation (after large_function.py at 19% and large_file). The per-1k normalisation means any short file with 2 markers trips a “too many” charge. The summary: “too many in one file”: is false for 2 TODOs.
severity: real problem

`context`’s “Related files” is `ls` output, alphabetical

command: crimes context zerver/lib/narrow.py --no-init

what happened:

Related files
  · zerver/lib/__init__.py  — same directory
  · zerver/lib/addressee.py — same directory
  · zerver/lib/alert_words.py — same directory
  · zerver/lib/async_utils.py — same directory
  · zerver/lib/attachments.py — same directory
  … and 5 more

zerver/lib/ has ~200 files. These are the first five alphabetically. narrow.py imports none of them. The reason given is literally “same directory”.

why it matters: context is the command whose whole value proposition is “give an agent the right context before it edits”. Handing an agent five arbitrary alphabetical neighbours is worse than handing it nothing: it burns context window on noise. Meanwhile the real answer (narrow.py imports zerver.models, is imported by zerver/views/message_fetch.py) exists in the import index and is not used.
severity: real problem

`context` on Python otherwise works and is genuinely agent-shaped

Positives worth recording: the Agent guidance block is concrete and correct; Likely tests → zerver/tests/test_message_fetch.py is exactly right; Clues (churn 2 commits / 90d, 2 authors, last touched 2 weeks ago) is the kind of thing an agent cannot get cheaply. 25s for a 1,657-line file is slow but tolerable.
The signal-to-noise is the problem, not the shape: 23 findings of which 12 are God Function, 9 are prose comments, 1 is TODOs. 1 (Boolean Naming Drift) is arguably novel.
severity: n/a: mixed positive

`dependency_provenance_gap` applies the npm manifest model to Python and charges Zulip HIGH for importing the standard library

command: crimes explain crime_01348 --from zulip.json

evidence, verbatim:

· 450 external package(s) imported with no declaring manifest
·   `_typeshed` — imported at zerver/lib/test_console_output.py
·   `abc`       — imported at corporate/lib/stripe.py
·   `ahocorasick` — imported at zerver/lib/alert_words.py
·   `aioapns`   — imported at zerver/lib/push_notifications.py
·   +442 more
· manifests searched: package.json, starlight_help/package.json

Suggested action: “Add _typeshed, abc, ahocorasick to the dependencies of the package that imports them.”

why it’s wrong: abc and _typeshed are the Python standard library. ahocorasick/aioapns are pip packages declared in pyproject.toml, which the detector does not read: it only searched the two package.json files. severity high, confidence 0.74, and the confidence ladder even credits itself “+0.14 (375 runtime imports affected)” for the size of the mistake.
why it matters: this is a 0.16 detector, rated high, on the control repo, and it is 100% wrong. Any polyglot repo gets it. The detector must either read pyproject.toml / requirements*.txt + a stdlib list, or exclude language-py imports entirely.
severity: blocker (for this detector on polyglot repos)

`docs_code_drift`: 12 findings, 12 false positives, one root cause

Every flagged link resolves fine; the detector resolves root-relative doc links against the repo root instead of the docs root:
- presenting-visual-changes.md:131 → /images/firefox-rwd-capture-mobile.png (not found): it is at docs/images/firefox-rwd-capture-mobile.png.
- slash-commands.md:15 → /subsystems/widgets.md (not found): it is at docs/subsystems/widgets.md.
- reviewable-prs.md:91 → ../contributing/presenting-visual-changes (not found): extensionless Sphinx/MyST link; docs/contributing/presenting-visual-changes.md exists.
- html-css.md:284 → ../testing/typescript (not found): docs/testing/typescript.md exists.
Verified all six targets on disk. 12/12 false positive.
severity: real problem (two sub-bugs: docs-root resolution, extensionless links)

`unsafe_retry` and `unbounded_async_fanout` fired ZERO times

Two of the ten 0.16 detectors produced nothing on a 590k-line repo with a Tornado event system, a push-notification worker, and an aioapns async client. Recorded as a negative result about the detectors, per protocol.
Also zero repo-wide: all three cross_language_*.
severity: observation

`explain` is the best surface in the tool

Charge / type / severity+confidence / “What this detector looks for” / “Why it matters” / itemised evidence / confidence ladder arithmetic / risk profile with units / suggested actions with risk levels / a ready-to-paste crimes ignore line. Nothing here was confusing.
It is also the only renderer that describes blast radius honestly: blast radius: 1.00: 50+ transitive importers (cap reached). scan and context render the same score as blast top-quartile (798 importers) with no “transitive” and no “cap reached”. Two renderers, two different stories about the same number.
severity: n/a positive / annoyance for the inconsistency

`verdict` costs 2m42s to print six lines

command: crimes verdict --no-init (cwd = repo)
what happened: 162s wall, 90s user. Output: Verdict: UNCHANGED / New: 0 / Fixed: 0. Exit 0. Correct: HEAD is origin/main at the pinned SHA: but it runs two full scans to establish that the two refs are the same commit.
why it matters: a no-op comparison should be free. As a pre-commit or CI gate at 3 minutes on a repo this size, nobody keeps it.
severity: real problem

`crimes diff` human output does not say what changed

command: crimes diff HEAD~50...HEAD --no-init → 176s, exit 0, and the entire output is:
```
New crimes: 2 / Fixed crimes: 2 / Unchanged crimes: 3431
```
The JSON has the answer (new_findings[]: a new sync_io_in_hotpath at zerver/lib/push_notifications.py:190, a new dependency_provenance_gap; fixed: a sync_io_in_hotpath at :205 and a duplicated_policy in web/src/hotkey.ts). The human renderer prints none of it, and there is no --verbose/--show-new flag.
why it matters: “what did these 50 commits introduce” is the whole question. The human renderer answers “2” and stops. A PR-review user has to re-run with --format json and read it themselves.
severity: real problem

`crime_NNNNN` ids are not stable across identical runs

Two scan --all --format json runs on the same SHA, 81.3s and 83.1s: identical totals (3759 / 502 / 1761 / 1496), identical coverage, set-identical findings (0 in one and not the other): but 139 of 3759 positions differ, and 139 findings change their crime_ id between runs (3,615 of 3,754 keep theirs).
First divergence at index 685: run 1 has large_function zerver/tornado/event_queue.py:1442-1580, run 2 has large_file zerver/lib/streams.py. Two findings with equal sort keys, tie-broken nondeterministically (parallel file processing order).
why it matters: crimes explain <id> takes a per-scan id, and the human scan output prints id ranges (id=crime_00144 … crime_01287) as the user’s handle on a finding. Copy an id from a report, re-run, and 4% of the time you explain a different finding. The stable fingerprint exists and is the right answer: but the human renderer never prints one.
detection quality itself is deterministic, which is the important half. Recorded as a ranking/ID bug, not a detection bug.
severity: real problem

CONTROL ASSESSMENT: the shape is wrong for a repo of this quality

3,759 findings / 590,435 lines of code = 6.37 findings per 1,000 lines. 1,254 of 3,588 scanned files (35%) carry at least one.
- Python: 1,635 / 418,474 lines = 3.91 / 1k
- TS+JS: 1,290 / 154,036 lines = 8.37 / 1k
Zulip runs Ruff + mypy strict + Prettier + ESLint + a mandatory two-reviewer process, and has deliberately not enabled a function-length rule. crimes’ verdict is 3,759 charges.
81% of the report (3,038 findings) is in the “a linter already does this” bucket: large_function 1,461, large_file 355, deep_import 275, commented_out_code 293, boolean_naming_drift 455, todo_density 73, hardcoded paths 59, image-asset charges 65. CLAUDE.md says crimes is “explicitly not trying to replace ESLint/Biome”. On the control repo it is 81% exactly that, and Zulip’s maintainers already looked at each of those rules and declined them.
19% (721 findings) is the differentiated surface: sync_io_in_hotpath 117, high_fan_in_fan_out 121, name_behavior_mismatch 63, direct_date 79, weak_test_signal 192, swallowed_error 32, pass_through_abstraction 19, duplicated_policy 15, contract_drift 15, docs_code_drift 12, plus singletons. That number: ~1.2 per 1,000 lines: is what a repo of this quality plausibly deserves, and it is a defensible report.
The detector to name: large_function at 1,461 findings (39% of everything) on the best-maintained repo in the corpus, with 19% of its Python half self-refuting (evidence quotes a line count at or below the threshold it names). If one calibration result comes out of this repo, it is that one.
Runner-up: commented_out_code at 293 (7.8%), which on this repo is inverted: it charges Zulip’s long prose comments, its single best maintainability asset.
Third: boolean_naming_drift at 455 (12%). Naming convention, entirely stylistic, and Zulip’s own conventions differ.
Verified-false-positive detectors: docs_code_drift 12/12, dependency_provenance_gap 2/2 (both HIGH-severity claims about Python imports made against package.json).
severity: real problem (calibration, top-line)

ZERO-CASE STATEMENT (positive, per protocol)

Zulip is polyglot (py + ts), so the cross-language zero-case is not the protocol’s single-language test. Stated explicitly anyway:
- cross_language_route_drift = 0, cross_language_type_drift = 0, cross_language_concept_alias_drift = 0.
- language-py findings = 1,635 on a repo that does contain .py: the Python pack demonstrably ran and produced findings, so packs_loaded is trustworthy for the language packs.
- language-js = 1,290, universal = 834. All three loaded packs produced findings.
- No cross-language false positive was emitted. That half of the zero-case passes.
The repo was not written to. git status --porcelain empty after the full round; no .crimes/, no crimes.config.json. Every command used --no-init.

Robustness summary

check	value
`ulimit -n`	1048576
`coverage.files_total`	3,588
`coverage.files_by_language`	`{py: 2017, js: 688}`
`coverage.files_universal_only`	883 (`.mdx` 464, `.md` 396, `.rb` 12, `.sh` 8, `.bash` 2, `.lua` 1)
`coverage.packs_loaded`	`["universal","language-js","language-py"]`: all three produced findings
`coverage.by_package`	absent (only 1 nested manifest; needs 2)
stderr	empty on every successful run
exit codes	0 everywhere, except one unreproduced 143 on the first human scan
wall clock	scan json 81.3s / 83.1s · scan human 147.8s · hotspots 62s · context 24–27s · verdict 162s · diff 176s
max RSS	378.8 MiB
Python parse errors	0 / 2,016 files
determinism	findings set-identical; 139/3,759 positions and ids unstable

Files never covered

469 .hbs Handlebars templates and 224 .html are not in files_by_language and not in universal_only_by_extension: they are simply not discovered. Zulip renders most of its UI through .hbs; every {{#if}} policy check and every UI string in the product is invisible to the scan. .py 2,016 + .js/.ts 688 + universal 883 = 3,587 ≈ files_total 3,588, so the templates are excluded at discovery, silently.
severity: annoyance (defensible scope decision, but nothing in the output says the templates were skipped)

CORRECTION: the “human scan is 1.8x slower” entry was measurement noise

Re-measured. Five runs of effectively identical work on the same SHA:
run wall user
--all --format json 81.3s 45.1s
--all --format json (2nd) 83.1s 46.7s
human (default) 147.8s 47.2s
human (default, 3rd) 68.7s 47.2s
human --all 66.1s 47.5s
User CPU is flat at 45–48s across all five. Wall clock ranges 66s → 148s, a 2.2x spread, entirely outside CPU. So the earlier claim (human renderer is slow) is withdrawn; the real observation is that a scan’s wall-clock time is not predictable within a factor of two on the same repo and machine, and there is no progress output during the wait. The 148s run was the outlier, not the rule.
severity: annoyance (downgraded from “real problem”)

run	wall	user
`--all --format json`	81.3s	45.1s
`--all --format json` (2nd)	83.1s	46.7s
human (default)	147.8s	47.2s
human (default, 3rd)	68.7s	47.2s
human `--all`	66.1s	47.5s

`.hbs` / `.html` / `.erb` templates are silently outside the scan

Default include globs (packages/core/src/config.ts:198–204) cover {ts,tsx,js,jsx,mjs,cjs,cts,mts}, {py,pyi}, {rs,go,rb,java,kt,…,lua,sh,bash,zsh}, {md,mdx} and an image set. No .hbs, no .html, no .vue, no .svelte, no .erb.
Zulip: 469 .hbs + 224 .html + 56 .erb = 749 files, the entire rendered UI, never seen. coverage does not mention them in either files_by_language or universal_only_by_extension: they are dropped at discovery, so nothing in the report tells the user a third of their frontend was not scanned.
why it matters: the coverage block is the tool’s honesty surface. A file that was never a candidate is invisible in it. A Zulip maintainer reading files_total: 3588 against git ls-files | wc -l = 8919 gets no explanation.
severity: real problem

`agent_risk` does not differentiate: the top 20 is one detector

The protocol’s judging sample includes “the top 20 findings by agent_risk per repo”. On Zulip that sample is: 18 of 20 are large_function or large_file. The other two are name_behavior_mismatch. All 20 are in web/src/; not one is Python, despite Python being 71% of the code.
Distribution across all 3,759: min 0.08, p25 0.30, median 0.37, p75 0.49, p90 0.58, max 0.89. The whole differentiated surface: swallowed_error, duplicated_policy, contract_drift, sync_io_in_hotpath: sits in the 0.25–0.75 band, below a 682-line function.
why it matters: CLAUDE.md: “agent_risk is the differentiator and must not be collapsed into severity.” On this repo it has collapsed into file size × churn, with blast_radius contributing nothing (47% of findings pinned at 1.0). The one score that is supposed to make crimes different from a linter is currently ranking the linter findings first.
severity: real problem (top-line)

Closing note

Nothing crashed. No stack traces, no hangs, no non-zero exits except one unreproduced SIGTERM. Memory is comfortable (379 MiB peak on 590k lines). The Python parser is clean: 2,016 files, zero syntax errors, match and walrus and PEP 604 all handled. explain is genuinely good.

The problems are calibration and coverage, not stability:

the 0.15 cross-language pack is blind to Django URLconf and to options-bag HTTP wrappers, which is most of the real world;
one size-based detector is 39% of the report and self-refutes 19% of the time;
the report’s own ranking signal puts that detector at the top.

airflow

Friction log: airflow

Repo: ~/crimes-dogfood/corpus/airflow @ acb4dde004fa39b0876923e0d39883bd3c2a2a60 Binary: ~/crimes-dogfood/bin/crimes (0.17.0) ulimit -n = 1048576 Size: 421M total (216M .git), 13,659 tracked files, 7,677 .py, 594 .tsx, 440 .ts, 40,206 commits.

`/usr/bin/time -l` reliably SIGTERMs the long scan; bare invocation does not

repo: airflow
command: /usr/bin/time -l ./bin/crimes scan .../airflow --all --format json --no-init
what happened: two attempts under /usr/bin/time -l ended EXIT=143 (SIGTERM) at 74.6s and 156.7s with a 0-byte JSON and time: command terminated abnormally. The identical command without /usr/bin/time completed cleanly (exit 0, 25.4 MB JSON) three times running. A short scan (task-sdk subtree, 7.4s) under /usr/bin/time -l was fine.
why it matters: most likely an environment/sandbox artefact, not a crimes bug: but the protocol prescribes /usr/bin/time -p for the canonical timing step, and on this host that measurement method destroys the artefact it is measuring. Anyone following the protocol literally on a repo this size gets a 0-byte JSON and no error from crimes.
severity: annoyance (protocol), not a crimes defect
workaround used: wall clock via date, max RSS via a 0.5s ps -o rss= -p <pid> poller.

Scan wall-clock varies ~2.3x run to run on the same input

repo: airflow
command: crimes scan <airflow> --all --format json --no-init
what happened: 134s, 173s across two clean runs (and a killed run had reached 157s). Max RSS 577 MB (PID-targeted poll), /usr/bin/time reported 434–444 MB on the truncated runs.
why it matters: the machine had sibling dogfooding agents on it, so this is probably contention rather than crimes. Worth recording because “airflow takes ~2.5 min” is not a stable number.
severity: annoyance

Machine-generated OpenAPI client code occupies 6 of the top 20 findings by agent_risk

repo: airflow
command: crimes scan <airflow> --all --format json --no-init
what happened: airflow-core/src/airflow/ui/openapi-gen/** produced 125 findings, all tier: "domain", including the #3 ranked finding overall (large_file on requests/services.gen.ts, agent_risk 0.79) plus queries.ts, prefetch.ts, suspense.ts, schemas.gen.ts, ensureQueryData.ts. services.gen.ts and schemas.gen.ts match scope-class.ts’s GENERATED_RE (\.(generated|gen)\.[cm]?[jt]sx?$) and so are isNeverReportable; the sibling files under openapi-gen/ do not match any rule at all (the directory is openapi-gen, not generated).
why it matters: two separate problems. (1) large_file / near_duplicate_block / commented_out_code do not consult scope-class.ts at all, so even a path the classifier already calls generated is reported at full weight. (2) openapi-gen/ is a common generator output convention that GENERATED_RE does not cover. Net effect: the first screen a maintainer sees is dominated by a directory nobody edits by hand.
severity: real problem

`scope_class` is not exposed on any finding; `tier` is domain/nonDomain and does not track it

repo: airflow
command: jq over the JSON contract
what happened: no finding carries a scope_class field. The only adjacent field is tier, which takes domain / nonDomain. All 230 findings inside migrations/ are tier: "domain"; so are all 26 on .pyi stubs, all 125 under openapi-gen/, and the single _vendor/ finding. Of 6,770 findings under a tests/ path only 222 are nonDomain.
why it matters: the protocol asks whether scope classification is working. From the JSON contract alone that is unanswerable: the classifier’s output is invisible to a consumer. A downstream agent cannot filter “don’t show me migrations” without re-deriving path rules the tool already has.
severity: real problem (schema gap)

`mixed_utc_local_methods` fires ZERO times on the most timezone-obsessed Python codebase in existence

repo: airflow
command: crimes scan <airflow> --all --format json --no-init
what happened: 0 findings. Airflow contains 775 utcnow() call sites and 354 datetime.now( call sites; 21 files contain both, 9 of them non-test production modules. The clean positive case is providers/cncf/kubernetes/.../executors/kubernetes_executor.py: naive datetime.now() at lines 464 and 630, timezone.utcnow() at line 1098, same module. direct_date DID fire on that file at [464, 630], so the Python parser’s dateCalls saw the naive now() reads: what it did not do is classify timezone.utcnow() as kind: "utcnow".
why it matters: root cause is the wrapper idiom. Every large Python codebase wraps clock access (timezone.utcnow(), utils.now(), clock.utcnow()); the detector’s kind === "utcnow" only recognises the bare datetime.utcnow() form, which is deprecated in 3.12 and therefore increasingly rare in exactly the modern code the pack targets. The detector is not merely quiet: it is structurally unable to fire on idiomatic Python.
severity: real problem (detector is a candidate for retune-or-sunset)

41% of all airflow findings are the Apache license header

repo: airflow
command: crimes scan <airflow> --all --format json --no-init
what happened: commented_out_code fired 8,019 times. 7,320 of those (41.1% of the repo’s entire 17,801-finding output) have line range exactly (1,16) or (1,17): the ASF Apache-2.0 header every airflow source file carries. Evidence text: "16 consecutive comment lines (lines 1–16)", ">=3 lines contain code-shaped tokens (def/fn/function/class/let/var/const/etc.)". The header contains no def/class/let/var/const; the likeliest triggers are the http://… URL (a //) and the quoted "License" / "AS IS" string literals.
why it matters: every Apache-licensed repo: a very large fraction of the open-source Python world: will see this. It is 8,000 findings that are literally identical, on every file, and it swamps the JSON contract for any downstream consumer. mlflow, which does not carry per-file license headers, produced only 151 commented_out_code findings, so the asymmetry is entirely explained by the boilerplate.
severity: blocker for this repo’s output being usable

`hotspots` on airflow returns generated OpenAPI client code for 6 of the top 7

repo: airflow
command: crimes hotspots <airflow> --no-init (164s)
what happened: ranks 1,2,3,5,6,7,8 are all airflow-core/src/airflow/ui/openapi-gen/**; rank 13 is airflow-ctl/src/airflowctl/api/datamodels/generated.py. Generated clients are regenerated on every API change, so they have both maximum churn and maximum finding count: exactly the two inputs hotspots multiplies.
why it matters: hotspots is the “where is the risk” command. On the largest repo in the corpus it points almost entirely at files no human edits. scope-class.ts already knows two of these paths are generated; hotspots does not ask it.
severity: blocker

Every top hotspot scores `risk 1.00 (100%)`: the ranking carries no information

repo: airflow
command: crimes hotspots <airflow> --no-init
what happened: ranks 1 through 14 all read risk 1.00 (100%). mlflow’s top 3 also read 1.00, then it degrades sensibly (0.85, 0.84, 0.82…).
why it matters: on a big repo the score saturates and the displayed order is decided by something the output never shows. A user cannot tell rank 1 from rank 14.
severity: real problem

`large_function` reports a threshold the finding does not meet

repo: airflow
command: crimes scan <airflow> --no-init (default human view)
what happened: _update_dag_run_state_for_paused_dags is charged “God Function”, summary “is 30 lines and nests 4 levels deep”, first evidence line 30 lines (threshold 50 for shape "domain"). Same for _executor_to_workloads (40 lines / threshold 50) and _execute (49 lines / threshold 50). The real trigger is nesting depth, which appears at evidence positions 2 and 5.
why it matters: the headline evidence line reads as self-refuting: it states a threshold and then a number below it. A reader who checks the receipt concludes the tool is broken. This is exactly the “summary did not explain itself” case.
severity: real problem

The default human view prints 33 findings for one file, 24 of them the same charge

repo: airflow
command: crimes scan <airflow> --no-init (no --all)
what happened: the top-ranked file, scheduler_job_runner.py, lists all 33 of its findings, of which 24 are “God Function”. 164 lines of output for 5 files.
why it matters: --all is documented as the opt-in for exhaustiveness, but the default already is exhaustive within a file. The signal (“this file has 24 oversized functions”) is one line; it is delivered as 24.
severity: annoyance

`context` uses a different `crime_NNNNN` numbering space than `scan` / `explain`

repo: airflow
command: crimes context airflow-core/src/airflow/jobs/scheduler_job_runner.py --no-init, then crimes explain crime_00002 --no-init
what happened: context labelled the God File on scheduler_job_runner.py as id=crime_00001 and _executable_task_instances_to_queued as id=crime_00002. crimes explain crime_00002 returned large_file on api_fastapi/common/parameters.py: the repo-wide numbering. Both are reproducible; they are simply different ID spaces, and nothing in either output says so.
why it matters: the documented agent workflow is context-before-edit then explain-the-id. Following it hands you a different finding, silently. Only 103 of 17,801 findings carry a discriminator, so id is the only handle available for the rest.
severity: blocker for the agent workflow

`context` finds the right test file but does not rank it, and `test gap` contradicts it

repo: airflow
command: crimes context airflow-core/src/airflow/models/dagrun.py --no-init
what happened: “Likely tests” lists 30 paths in alphabetical order. The true 1:1 pair, tests/unit/models/test_dagrun.py (5,065 lines), is 20th, indistinguishable from tests/unit/cluster_policies/__init__.py and tests/unit/listeners/dag_listener.py, which are not tests at all. For scheduler_job_runner.py the list is 5 entries, 4 of them wrong, and the “Clues” block simultaneously reports test gap: bottom-quartile for a file with a 13,678-line dedicated test.
why it matters: “which test covers this file” is the single highest-value thing context can tell an agent, and an unranked 30-item list is not an answer. The internal disagreement (likely-tests found it; test_gap says there is none) means two different pairing heuristics disagree and neither is surfaced.
severity: real problem

`sync_io_in_hotpath` and `boolean_naming_drift` aggregate across functions but wear a single-symbol label

repo: airflow
command: crimes explain crime_00062 --no-init
what happened: the finding carries symbol: _enqueue_executor_callbacks and lines: 1265–1934, but its evidence lists calls in three different functions (_enqueue_executor_callbacks, process_executor_events, _run_scheduler_loop). boolean_naming_drift on triggerer_job_runner.py spans lines 438–1283 across six bindings in different classes. Two of the three matched calls are SQLAlchemy session.get(), classified into family network.
why it matters: the lines range is not a region a reader can open: it is the bounding box of scattered hits. Anything consuming lines to build a code excerpt (including the judging protocol’s ±30-line card) gets the wrong window.
severity: real problem

`explain` and `context` re-scan the whole repo every invocation

repo: airflow / mlflow
command: crimes explain <id> --no-init, crimes context <file> --no-init
what happened: airflow explain 164s / 168s / 180s per call; airflow context 17–19s (it scopes to the airflow-core package); mlflow context 112s and 97s (no package split at the repo root, so it scans everything).
why it matters: the crimes skill tells agents to run context before every edit. Two minutes per pre-edit check is not a workflow anyone keeps. There is a .crimes/ cache location in the design and it is not being used here.
severity: real problem

Three commands report three different file counts for the same repo

repo: airflow
what happened: scan human header says “across 9089 files”; coverage.files_total in the JSON says 9149; hotspots says “3896 files”/“10971 files” (airflow: 10971). All from the same commit, same flags.
why it matters: small, but “how much did you actually look at” is the one number a user checks to decide whether to trust the rest, and it changes depending on which command they ask.
severity: annoyance

Output is deterministic: recording the positive

repo: synthetic probe
what happened: two consecutive scan --all --format json runs produced byte-identical id / type / file / lines / evidence tuples for all 16 findings. explain crime_00002 returned the same finding on two separate invocations.
severity: n/a (positive)

mlflow

Friction log: mlflow

Repo: ~/crimes-dogfood/corpus/mlflow @ 0489fb31d8a144ec3c80d15eb44dbe24b790465e Binary: ~/crimes-dogfood/bin/crimes (0.17.0) ulimit -n = 1048576 Size: 675M total (320M .git), 9,241 tracked files, 12,842 commits.

mlflow scans 2.3x slower than airflow despite being 30% smaller

repo: mlflow
command: crimes scan <mlflow> --all --format json --no-init
what happened: 402s wall, 594 MB peak RSS, exit 0, empty stderr, 9.6 MB JSON. Airflow: 13,659 tracked files / 7,677 .py / 40,206 commits: took 173s at 577 MB. mlflow is 9,241 files / 2,627 .py / 12,842 commits. The one dimension where mlflow is bigger is TS/JS: 3,426 .ts/.tsx vs airflow’s 1,034, and coverage.files_by_language confirms it (js: 3513 vs js: 1089).
why it matters: cost tracks the JS pack, not repo size or Python volume. A Python-dominant repo pays 6.7 minutes because it has a React frontend. hotspots is 418s, context on one file is 97–112s, and verdict (two scans) did not finish inside 10 minutes.
severity: real problem

`crimes verdict` exits 2 on a `master`-default repo

repo: mlflow
command: crimes verdict --no-init (cwd = repo root)
what happened: crimes: could not pick a default base ref (tried origin/main, main). Pass --base <ref> explicitly, e.g. --base main. exit code 2. mlflow’s default branch is master, and origin/master exists.
why it matters: the message is clear and actionable, which is good, but master is a two-word addition to the fallback list and it is still the default branch on a large share of long-lived repos. Every mlflow contributor hits this on first run.
severity: annoyance

`crimes verdict --base master` did not complete in 10 minutes

repo: mlflow
command: crimes verdict --base master --no-init
what happened: killed at the 10-minute mark with no output. It needs a scan of base and of head; a single scan is 402s, so ~13.5 minutes is the floor. No progress output of any kind while it runs.
why it matters: verdict is the CI-gate command. A silent 13-minute run with no progress indicator is indistinguishable from a hang, and it is above most PR-check budgets.
severity: real problem

Generated protobuf stubs are scanned and reported as production code

repo: mlflow
command: crimes scan <mlflow> --all --format json --no-init
what happened: mlflow/protos/ produced 28 findings: 17 large_file, 9 boolean_naming_drift (language-py), 2 large_function: all tier: "domain". Includes service_pb2.py (5,696 lines) and service_pb2.pyi (2,833 lines). scope-class.ts’s GENERATED_RE does not match _pb2.py / _pb2_grpc.py / _pb2.pyi, and the large_file detector does not call looksGeneratedSource either. A synthetic 60,002-line file whose first line is # @generated by protoc. DO NOT EDIT. also drew a large_file finding.
why it matters: _pb2.py is the single most common generated-Python convention there is. Both layers of the classifier: the path rule and the banner sniff: exist and neither is consulted.
severity: real problem

Alembic migrations under `db_migrations/` are classified as production

repo: mlflow
command: crimes scan <mlflow> --all --format json --no-init
what happened: mlflow/store/db_migrations/versions/ holds 63 alembic revisions; the tree produced 43 findings, all tier: "domain". MIGRATION_RE requires a path segment that is exactly migrations / migration (or alembic/versions, db/migrate, prisma/migrations); db_migrations matches none of them. In airflow the directory is called migrations/versions/ and matches: yet those 230 findings are also tier: "domain", so matching the rule made no observable difference to the output.
severity: real problem

Jupyter notebooks are silently outside coverage

repo: mlflow
what happened: 42 .ipynb files. Zero findings, and .ipynb appears in neither coverage.files_by_language nor coverage.universal_only_by_extension. Nothing anywhere says notebooks were skipped.
why it matters: defensible as a scope decision, but on a data/ML repo notebooks are where a lot of the real Python lives. coverage is the mechanism for telling the user what was not looked at, and it does not mention them.
severity: annoyance

`context` is genuinely good here: recording the positive

repo: mlflow
command: crimes context mlflow/store/tracking/sqlalchemy_store.py --no-init
what happened: the “Likely tests” list contains all seven real tests/store/tracking/sqlalchemy_store/test_*.py files; mlflow/server/handlers.py correctly surfaced tests/server/test_handlers.py. Still unranked and still padded with tests/conftest.py and dev/benchmarks/tracing/test_trace_perf.py, but the right answer is in there.
severity: n/a (positive)

`mixed_utc_local_methods` fires zero times here too

repo: mlflow
what happened: 0 language-py findings (1 language-js). Same as airflow. Across both repos: 10,304 real Python files: the detector produced zero Python findings.
severity: real problem (see the airflow log for the root cause)

Encoding robustness: latin-1 survives, UTF-16 and NUL bytes are silently dropped

repo: synthetic probe (~/crimes-dogfood/scratch/pyprobe-repo), built because neither corpus repo contains a non-UTF8 .py
command: crimes scan scratch/pyprobe-repo --all --format json --no-init
what happened: four files with byte-identical Python bodies. UTF-8 control → 3 findings. Same bytes as latin-1 (high bytes in the docstring) → the same 3 findings, so replacement decoding works. Same bytes as UTF-16 → 0 findings. Same bytes with a NUL in the docstring → 0 findings. All four are counted in coverage.files_by_language.py, stderr is empty, exit code 0.
why it matters: this is precisely the failure the protocol warns about: a degraded scan is byte-identical to a clean one. coverage claims the file was scanned. There is no files_failed counter and no warning channel.
severity: real problem (low real-world frequency, but the silence is the defect)

`sync_io_in_hotpath` attributes 15 calls spanning 8,173 lines to a single function

repo: mlflow
command: crimes scan <mlflow> --no-init
what happened: in mlflow/store/tracking/sqlalchemy_store.py (10,093 lines) the finding is labelled Blocking the Hot Path · _hard_delete_experiment() and lists 15 calls at lines 900 … 9073. The human renderer prints ten of them inline plus ”…+5 more” on one line. Twelve of the fifteen are SQLAlchemy session.delete() / session.get(), classified as blocking I/O.
why it matters: the symbol label is wrong for 14 of the 15 sites, and the lines range covers 81% of the file. Any consumer that uses lines to fetch surrounding code: including the blind judging card: gets a meaningless window.
severity: real problem

Modern Python syntax: no parse failures found, on real code or synthetic

repo: airflow + mlflow + synthetic probe
what happened: verified by presence of findings from inside files using each feature, never by absence of an error.
- match: airflow 10/16 files, mlflow 59/74 produced language-py findings.
- walrus :=: airflow 215/238, mlflow 297/379.
- PEP 604 X | None: airflow 1441/1646, mlflow 410/612.
- positional-only /: airflow 5/6, mlflow 1/1.
- async generators: airflow 98/106, mlflow 37/57.
- decorators with complex expressions: airflow 1205/1480, mlflow 288/585.
- nested f-strings: airflow 14/16, mlflow 6/11.
- All rates are at or above the repo-wide base rate (airflow 4104/7677 = 53%, mlflow 955/2627 = 36%).
- PEP 695 generics (class Box[T], def get[U], def f[T, *Ts, **P], type X = ...) and except* groups appear in neither corpus repo, so they were tested synthetically: both parsed, with findings produced from inside a PEP-695-generic function body and from statements after an except* block.
severity: n/a (positive: the tree-sitter Python parser is the strongest part of the 0.14 pack)

`crimes diff` on mlflow not captured: same cost as `verdict`

repo: mlflow
command: crimes diff 'HEAD~5...HEAD' --no-init
what happened: killed by the 10-minute harness ceiling; artifacts/mlflow.diff.txt is empty. Like verdict it scans both refs, so ~13.5 minutes minimum at 402s/scan. The equivalent on airflow completed (New crimes: 0 / Fixed crimes: 0 / Unchanged crimes: 15772).
severity: real problem (same root cause as the verdict entry)

n8n

Friction log: n8n

repo: ~/crimes-dogfood/corpus/n8n @ fa2d8a131cdd34cf4ed31f46c079ff066042235d
build: crimes 0.17.0 (~/crimes-dogfood/bin/crimes)
size: 375M on disk, 26,079 tracked files; 18,783 .ts, 1,226 .vue, 41 .js, 69 .py, 3,492 .json, 578 .md
workspace: pnpm, pnpm-workspace.yaml globs packages/*, packages/@n8n/*, packages/frontend/**, packages/extensions/**, packages/testing/** (no ! negations)
ambient ulimit -n: 1048576 (soft). NOTE: this is NOT the macOS default (256): the FD-exhaustion regression test is weakened at this limit, so a separate low-ulimit stress run is recorded below.

Entries

`scan --all --format json` exceeds 10 minutes wall clock; no progress output at all

repo: n8n
command: crimes scan ~/crimes-dogfood/corpus/n8n --all --format json --no-init
what happened: First foreground attempt was killed at the 10-minute mark still running. Zero bytes written to stdout, zero bytes on stderr, no progress indicator, no spinner, no “scanning N files” line. From the outside the process is indistinguishable from a hang. Re-launched detached to get a real number.
why it matters: 18.8k TypeScript files is a completely ordinary size for a monorepo of this class. A CLI that takes >10 min and prints nothing while doing it will be Ctrl-C’d by every user who tries it, and they will report it as “crimes hangs on my repo”, not “crimes is slow”. There is no --timeout, no --progress, and no way to tell a slow scan from a wedged one.
severity: blocker

No progress reporting and no way to bound a scan

repo: n8n
command: crimes scan --help
what happened: scan has no --progress, no --timeout, no --concurrency, and no --exclude/--ignore-path escape hatch. On a repo where the scan runs for minutes with a silent stdout, the only user-visible signal is a hot CPU fan.
why it matters: On big repos the first-run experience is “type command, nothing happens for several minutes”. There is no way to say “just do packages/cli” other than passing a subdirectory path (which then silently loses the monorepo/workspace context that dependency_provenance_gap and coverage.by_package depend on).
severity: real problem

Scan cost is superlinear in file count: measured, not guessed

repo: n8n
command: crimes scan <subtree> --all --format json --no-init on three nested scopes
what happened:

scope ts/js/vue files wall clock max RSS
packages/core 255 6.7 s 205 MB
packages/cli 2,977 234.0 s 386 MB
whole repo 20,050 >29 min and still running ~575 MB peak footprint

11.7x the files costs 35x the time. That is roughly O(n^1.6), not O(n). Linear extrapolation from the packages/cli point puts a full n8n scan somewhere around 80-90 minutes.
why it matters: This is the difference between “a tool you run in CI on every PR” and “a tool you run once and never again”. The per-file cost is fine (26 ms/file at 255 files); something cross-file is quadratic-ish. Candidates given the 0.16 slate: the cross-file risk index (policy clones, object contracts, pass-through chains) and the duplicate-block/function-hash indexes, all of which do pair matching. docs/ claims these are “bucketed matching, never global pairwise”: the measured curve says at least one of them is not bucketed tightly enough at this scale.
severity: blocker

scope	ts/js/vue files	wall clock	max RSS
`packages/core`	255	6.7 s	205 MB
`packages/cli`	2,977	234.0 s	386 MB
whole repo	20,050	>29 min and still running	~575 MB peak footprint

`duplicated_policy` “variants disagree” groups distinct permission scopes as one rule

repo: n8n (scoped run on packages/cli)
command: crimes scan .../n8n/packages/cli --all --format json
what happened: Highest-severity duplicated_policy finding says: “5 variants of the same guard clause exist across 5 files and disagree on a value (one side tests credential:manageInstance, the other tests project:create). At most one of them can be current.” The shared shape is !hasGlobalScope($0,?): i.e. five different call sites of the same permission helper with five different permission scope strings.
why it matters: Purely factual note for the judge: the normalisation collapses the argument of a shared helper, so any codebase with a single hasPermission(user, "<scope>") helper produces a high-severity “at most one can be current” finding for the entire permission surface. Whether that reads as useful is not mine to say, but the sentence “at most one of them can be current” is asserted, not evidenced.
severity: real problem

`dependency_provenance_gap` treats the relative specifier `"."` as an external package

repo: n8n (scoped run on packages/cli)
command: crimes scan .../n8n/packages/cli --all --format json
what happened: The single dependency_provenance_gap finding lists three “external packages imported with no declaring manifest”. One of them is: `.` (type-only): imported at src/eventbus/event-message-classes/abstract-event-message-options.ts as "." "." is a relative import of the sibling index.ts. The detector’s own evidence line claims “relative paths … are excluded”, and this one was not.
why it matters: One third of this finding’s payload is wrong, and it is wrong in exactly the category the evidence text promises is handled. A reader who checks the first item and finds nonsense will not check the other two (@n8n/eslint-config and eslint, both of which look real).
severity: real problem

`agent_permission_sprawl` classifies a sentence about eslint-disable as an instruction to disable verification

repo: n8n (scoped run on packages/cli)
what happened: The one agent_permission_sprawl hit is AGENTS.md:25, classified [disable_verification], quoting: “import: or an inline eslint-disable of the rule: fails CI. The same rule also”. The sentence is telling agents that eslint-disable fails CI, i.e. the opposite of an instruction to skip verification. The quote is also cut mid-sentence at both ends.
why it matters: The detector is doing keyword matching over prose and the quoted span is not even a complete clause, so the reader cannot judge it without opening the file. Its own evidence block says it is advisory and confidence-capped at 0.35, which is honest: but a truncated mid-sentence quote is not usable evidence.
severity: real problem

Six finding types carry no `lines` at all: the judging card cannot be built for them

repo: n8n (scoped run on packages/cli, 3,593 findings)
command: crimes scan .../n8n/packages/cli --all --format json
what happened: lines is absent (not [1,1]: absent) on every finding of these types:

type findings lines present
high_fan_in_fan_out 297 0
exact_duplicate_block 60 0
near_duplicate_block 33 0
duplicated_role_status_plan_check 23 0
deep_import 3 0
circular_dependency 2 0

418 of 3,593 findings (12%) have file but no line range. exact_duplicate_block does put the line ranges in prose inside evidence (“src/credentials-helper.ts:76-83”), so the information exists: it is just not in the schema field.
why it matters: PRD.md §9 says the JSON is the product contract and the human renderer is a view over it. Any consumer that wants to jump to the code: an editor integration, a PR annotation, the blind judging card this very protocol specifies (“repo name, file path, line range”): has to regex prose out of evidence for one finding in eight. symbol is likewise undefined on 20 of 34 types.
severity: real problem

type	findings	`lines` present
`high_fan_in_fan_out`	297	0
`exact_duplicate_block`	60	0
`near_duplicate_block`	33	0
`duplicated_role_status_plan_check`	23	0
`deep_import`	3	0
`circular_dependency`	2	0

Determinism: finding SET is stable, ORDER and `recency` are not

repo: n8n (packages/cli subtree, 2,977 files, 3,593 findings, three runs)
command: crimes scan .../n8n/packages/cli --all --format json --no-init, run 3x (once at ulimit -n 64)
what happened:
- Finding set is identical across all three runs, comparing {type, file, lines, symbol, summary, evidence, discriminator}: 3,593 findings, byte-equal after canonical sort.
- exact_duplicate_block evidence strings are identical across all three runs. The 0.17.0 reproducibility claim holds on this evidence.
- Finding order differs: 202 of 3,593 array positions hold a different finding between run 1 and run 2. Raw files differ at byte 920,905.
- scores.recency changed for 37 findings between runs minutes apart (e.g. 0.73 → 0.72). Recency is wall-clock derived, so any two runs on different days produce different rank_score and therefore a different top-N.
why it matters: crimes diff and baseline compare runs. A stable set is the important half and it holds. But a --format json consumer that diffs the array (the obvious thing to do) sees 202 spurious moves per run, and --top N / default human output can genuinely change between two runs of the same commit because recency drifts. Nothing in the output tells you that recency is time-dependent.
severity: real problem

Positive result: bounded FD pool holds at `ulimit -n 64`

repo: n8n
command: ( ulimit -n 64; crimes scan .../n8n/packages/cli --all --format json --no-init )
what happened: Exit 0. Identical finding set to the unrestricted run (3,593 findings, same exact_duplicate_block / near_duplicate_block counts: 60 / 33). No EMFILE, no stderr, no silent index degradation. Same at 255-file scope where output was byte-identical at ulimit 64, 256, and 1048576. Peak concurrent regular-file FDs observed on the full-repo scan was 27.
why it matters: This was the specific thing 0.17.0 claimed to fix, and on a real repo at 1/16000th the ambient FD limit it holds. Recorded as a positive.
severity: n/a (positive)

The full-repo n8n scan did not finish in 60 minutes

repo: n8n (whole repo, 20,050 ts/js/vue files)
command: /usr/bin/time -l crimes scan ~/crimes-dogfood/corpus/n8n --all --format json --no-init
what happened: Started 22:29:55, still running and CPU-hot (85-100% of one core, RSS oscillating 250-512 MB) when the 60-minute supervisor limit killed it at 23:29:5x. Zero bytes of output had been written in that hour: the JSON is emitted only at the end, so a killed scan leaves nothing at all. No partial results, no resume, no --output streaming. Relaunched fully detached to get a completion number.
why it matters: This is the single most important result of the round. n8n is not an outlier repo: it is a well-known open-source TypeScript monorepo, and 20k files is normal for the segment crimes targets. In its current state the tool cannot report on it at all within an hour, and because output is buffered to the end, an interrupted run costs the user 100% of the work with 0% of the value. Any CI integration, any pre-commit hook, any agent pre-edit check is off the table at this size.
severity: blocker

`crimes verdict` fails on any repo whose default branch is `master`

repo: n8n (default branch master)
command: crimes verdict --no-init --no-color run with cwd inside the repo
what happened: Exit with nothing on stdout and this on stderr: crimes: could not pick a default base ref (tried origin/main, main). Pass --base <ref> explicitly, e.g. --base main. The repo has refs/remotes/origin/HEAD -> refs/remotes/origin/master, which git already resolves for you: the tool never asks. With --base master it works fine (UNCHANGED, 11.2 s).
why it matters: verdict is one of the eight required v0 commands. It hard-fails on n8n, on every pre-2020 repo, and on anything using develop/trunk. The fix is one line: read origin/HEAD, or fall back to master. The error message even suggests --base main, the thing that just failed.
severity: real problem

Positive: Python pack works on n8n’s Python subtree; and a note on how to tell

repo: n8n (packages/@n8n/task-runner-python, 58 .py files)
command: crimes scan .../packages/@n8n/task-runner-python --all --format json --no-init: 3.9 s, exit 0
what happened: coverage.files_by_language: {"py": 58}, 30 findings. 7 of the 8 Python detectors fired: large_function.py (14), boolean_naming_drift.py (4), weak_test_signal.py (3), circular_dependency.py (1), direct_date.py (1), sync_io_in_hotpath.py (1); deep_import.py and mixed_utc_local_methods.py did not. Plus 6 universal findings on the same files.
Methodological warning for whoever reads the JSON: Python findings carry the same type strings as the JS ones (large_function, not py_large_function). The only discriminator is detector_id, which is suffixed .py. Anyone counting the Python pack by type prefix will report zero.
why it matters: Positive result on the 0.14 surface, plus a trap in the schema that will produce a wrong “Python pack fired zero times” conclusion if the count is done the obvious way.
severity: n/a (positive) / annoyance (schema)

`cross_language_*` produced zero on a genuinely mixed TS+Python package: recorded as the zero case

repo: n8n
what happened: packages/@n8n/task-runner-python contains a Python task runner alongside its TypeScript siblings; scanning it produced 0 cross_language_type_drift, 0 cross_language_route_drift, 0 cross_language_concept_alias_drift. cal.com (no .py at all) also produced 0 of all three, which is the required correct answer there.
why it matters: The cal.com zero is a positive: packs_loaded lists language-py on a repo with no Python and correctly emits nothing, so packs_loaded is not lying. The n8n zero is unresolved until the full-repo scan lands, since the TS and Python halves live in different subtrees and a subtree scan cannot see across them.
severity: n/a (observation)

Default human `scan` on n8n `packages/cli`: 4.8 minutes, 166 lines, 35% of it is one detector

repo: n8n (packages/cli, 2,977 files, 3,593 findings)
command: crimes scan . --no-init --no-color: real 286.34, 166 lines
what happened: Better lede than cal.com: the #1 file is src/modules/instance-ai/instance-ai.service.ts, genuinely 6,000 lines with an 852-line executeRun(). That is a true and useful first sentence. But of the 138 numbered finding lines printed, 58 are God Function and 5 more God File: 46% of the visible report is the size detector repeating itself, including entries at 1.0× and 1.1× the threshold (startInternalFollowUpRun() 62 lines vs a 60-line threshold, printed as a numbered finding). The 0.16 slate that this round exists to test contributes 24 Catch and Release, 6 Policy Doppelgänger, 4 Abstraction Laundering, 1 Return Shape Roulette: 35 lines out of 138: and contract_drift, config_drift, mock_saturation, unsafe_retry, unbounded_async_fanout, dependency_provenance_gap and agent_permission_sprawl appear zero times in the default view despite firing 15/3/5/7/2/1/1 times in the JSON. “Also flagged elsewhere: tests/ 1381 findings”: 38% of all findings are in the tests bucket, correctly separated here (n8n uses test/ and __tests__/, which the prefix matcher catches; cal.com’s playwright/ and *.e2e-spec.ts it does not).
why it matters: Two separate problems. (a) large_function at 1.0-1.1x threshold is noise that crowds out everything else in the one view most users will ever see. (b) The default view is organised by file, so every repo-level finding: the whole dependency_provenance_gap / agent_permission_sprawl / config_drift class, which is what 0.15-0.16 added: is structurally invisible unless you pass --all and read JSON.
severity: real problem

`mock_saturation` mis-identifies the subject under test

repo: cal.com (packages/trpc)
command: crimes explain crime_00327
what happened: For googleWorkspace.handler.test.ts the evidence says subject under test: ../../../types. The test’s subject is the googleWorkspace handler; ../../../types is a type-only import. The mocked-collaborator ratio (“100% of this test’s collaborators are replaced”) is computed after excluding that wrong subject.
severity: annoyance

Positive: `hotspots` is genuinely good when the churn window has data

repo: n8n (packages/cli)
command: crimes hotspots . --no-init --no-color: real 292.39
what happened: src/modules/instance-ai/instance-ai.service.ts: 117 commits in 90 days, 42 findings, risk 1.00. Then its test file (76 changes), then instance-ai.adapter.service.ts (68 changes, 38 findings). This is the correct answer: that module is n8n’s newest and most volatile subsystem and it is also the biggest file in the package.
why it matters: Direct contrast with cal.com, where the same command produced a 17-way tie at 0.43 because the clone had only 46 commits in the window. hotspots is only as good as the churn signal, and it does not tell you which of those two situations you are in.
severity: n/a (positive): see the cal.com log for the failure mode

`crimes diff HEAD~50...HEAD` exceeded 10 minutes on one package and was killed

repo: n8n (packages/cli only: 2,977 files, not the whole repo)
command: crimes diff "HEAD~50...HEAD" --no-init --no-color
what happened: Killed at 10 min with no output. diff runs two full scans (base and head); a single scan of this package is ~286 s, so two plus a git checkout of the base tree exceeds 600 s. On the whole n8n repo it would be roughly two hours. The equivalent on cal.com completed in real 890.13 (14 min 50 s) and reported New 9 / Fixed 4 / Unchanged 4050 over 131 changed files, which is the right shape of answer: it just costs a quarter of an hour.
why it matters: diff <range> is one of the required v0 commands and is the one that belongs in CI. Two full scans per invocation is the wrong algorithm at this size: only 131 of 5,000 files changed on cal.com and only a fraction on n8n, and the tool re-analysed everything twice anyway. --changed exists for scan but there is no equivalent narrowing for diff.
severity: blocker

`.vue` files are not scanned, and `coverage` does not say so

repo: n8n (packages/frontend/editor-ui)
command: crimes scan .../packages/frontend/editor-ui --all --format json --no-init
what happened: The package contains 2,090 .ts and 1,037 .vue tracked files. coverage reports files_total: 2104, files_by_language: {"js": 2100}, files_universal_only: 4 (.md 4). Findings by extension: ts 5,337, mts 2, mjs 2, json 1, svg 1, webp 1: zero .vue. Repo-wide n8n has 1,226 .vue files. .vue appears nowhere: not in files_by_language, not in files_universal_only, not in universal_only_by_extension. There is no way to learn from the output that a thousand files were skipped. Not even the universal detectors (todo_density, commented_out_code, hardcoded_localhost) ran on them, and those are language-agnostic.
why it matters: This is exactly the failure mode the protocol’s robustness section is built to catch, and coverage does not catch it. A Vue team runs crimes scan, gets 5,344 findings and a clean exit, and has no idea their entire component layer was never looked at. Silently dropping a file type is worse than refusing to scan it. At minimum coverage needs a files_skipped_by_extension field.
severity: blocker

`parallel_destination`: 2,819 findings from 134 files in one package: 53% of the entire scan

repo: n8n (packages/frontend/editor-ui)
what happened: Full count table for this package: parallel_destination 2,819, large_function 964, weak_test_signal 251, orphaned_destination 246, everything else under 200. parallel_destination alone is more than half of all 5,344 findings and comes from only 134 distinct files: i.e. it is emitting one finding per pair, unpruned. Sample (confidence 0.60, severity medium, and there are ~30 of these on useNodeHelpers.ts alone):

/composables/useNodeHelpers and /stores/workflowDocument/useWorkflowDocumentNodeGroups appear to serve overlapping user intent across different parts of the IA. routes: /composables/useNodeHelpers, /stores/workflowDocument/useWorkflowDocumentNodeGroups shared tokens: use, node It is treating Vue composables and Pinia stores as routes in an information architecture and pairing them on the shared tokens use and node — use being the mandatory prefix of every Vue composable and node being the domain of the entire product. One of the pairs is a .test.ts file matched against its own non-test sibling. This detector produced 0 on cal.com and 0 on n8n packages/cli, so it is dormant until it hits a directory whose naming convention trips it, then it drowns the report. These findings also have no lines — adding parallel_destination and orphaned_destination to the six no-line types already recorded.
why it matters: Any user whose first scan is a Vue/Nuxt frontend sees a report that is majority one detector matching on the word “use”. It would take one --all run to conclude the tool is broken. This is the strongest candidate on the corpus for a default-off gate.
severity: blocker

Final scale result: n8n whole-repo scan never completed

Second attempt, launched fully detached at 23:30:27 so no supervisor could kill it:

2 hours 50 minutes elapsed, still running, still CPU-hot, RSS oscillating 300-690 MB. 0 bytes written to stdout.
The first attempt was killed at exactly 60 minutes in the same state.
Left running at PID 77025 in case it eventually lands; ~/crimes-dogfood/logs/n8n.time will contain /usr/bin/time -l output if it does.

Because the whole-repo scan cannot be measured, every n8n number in this round comes from subtree scans, each of which completed cleanly:

scope	files scanned	wall clock	max RSS	findings
`packages/@n8n/task-runner-python`	59 (58 `.py`)	3.9 s	n/a	30
`packages/core`	255	6.7 s	205 MB	277
`packages/frontend/editor-ui`	2,104 (1,037 `.vue` silently skipped)	139.5 s	383 MB	5,344
`packages/cli`	2,995	234.0 s	386 MB	3,593
whole repo	~20,050	>2 h 50 min, did not finish	~690 MB peak	n/a

Zero cases and robustness checks: n8n, from the subtree scans

coverage.packs_loaded is ["universal","language-js","language-py"] on every subtree.
Python pack: fired. 7 of 8 py detectors produced findings on packages/@n8n/task-runner-python (large_function.py, boolean_naming_drift.py, weak_test_signal.py, circular_dependency.py, direct_date.py, sync_io_in_hotpath.py); deep_import.py and mixed_utc_local_methods.py did not.
All three cross_language_* detectors: zero on every n8n subtree scanned, including the mixed TS+Python one. Unresolved for the repo as a whole because the whole-repo scan never completed.
coverage.by_package is absent on every subtree scan: it appears to require the scan root to be the monorepo root. Whether n8n’s 77 workspace members would be enumerated correctly is therefore untested, because the only run that would have answered it is the one that never finished.
Duplicate detection alive, not degraded: exact_duplicate_block 60 / near_duplicate_block 33 on packages/cli, and 66 / 19 on editor-ui.
All ten 0.16 detectors fired on packages/cli alone. None is zero on n8n.
Nothing on stderr on any completed run. All completed runs exit 0.

Artefact index (n8n)

Because the whole-repo run never produced a file, ~/crimes-dogfood/artifacts/n8n.json is 0 bytes. The usable n8n artefacts are:

n8n-clipkg.json / n8n-clipkg.run2.json / n8n-clipkg.fd64.json: packages/cli, three runs (determinism + FD stress)
n8n-editor-ui.json: packages/frontend/editor-ui (the .vue skip and parallel_destination explosion)
n8n-core.json, n8n-task-runner-python.json
n8n-clipkg.human.txt, n8n-clipkg.hotspots.txt, n8n-core.human.txt, n8n-core.diff.txt, n8n-core.context.*.txt
n8n-core.verdict.txt is 0 bytes: see the master branch entry above
n8n-clipkg.diff.txt is 0 bytes: killed at 10 min

cal.com

Friction log: cal.com

repo: ~/crimes-dogfood/corpus/cal.com @ 038381aeca6261635357957d66b8ba85cdb29737
build: crimes 0.17.0 (~/crimes-dogfood/bin/crimes)
size: 398M on disk, 7,693 tracked files; 3,878 .ts, 1,147 .tsx, 31 .js, 601 .sql, 355 .md
workspace: yarn (yarn.lock), workspaces array in root package.json (10 globs, no ! negations)
ambient ulimit -n: 1048576 (soft)

Entries

Timing caveat: measured under contention

repo: cal.com
what happened: The cal.com full scan was run concurrently with the n8n full scan (which by then had been running >30 minutes without finishing) and with several other agents’ scans on the same machine. Wall clock for cal.com is therefore an upper bound, not a clean measurement.
why it matters: Recorded so nobody reads the cal.com number as a clean benchmark.
severity: annoyance

`dependency_provenance_gap`: 148 “undeclared external packages”, of which the first 8 shown are all path aliases or relative imports

repo: cal.com
command: crimes scan ~/crimes-dogfood/corpus/cal.com --all --format json --no-init
what happened: One severity: high, confidence: 0.74 finding: “148 external package(s) are imported but declared in no applicable package.json … a clean install can fail.” The 8 itemised examples (the only ones a reader sees; the rest is +140 more) are: _utils, ., .., @components/AddToHomescreen, @components/apps, @components/auth, @components/booking, @components/dialog. Two are relative specifiers (".", ".."). Five are @components/*, which is a TypeScript path alias declared in apps/web/tsconfig.json lines 5-15 ("@components/*": ["components/*"]). cal.com has no root tsconfig.json: every alias map lives in a per-app tsconfig. The finding’s own evidence says: “node built-ins, relative paths, path aliases, workspace packages, and subpaths of declared packages are excluded”. None of those three exclusions worked here.
why it matters: This is the flagship monorepo detector on the exact repo shape it exists for, and every visible example is wrong. It is high severity and it is the finding a reader most likely opens first, because “a clean install can fail” is scary. The root cause looks specific and fixable: alias resolution only reads a root tsconfig, and cal.com (like many Next.js monorepos) has none.
severity: blocker

Positive: workspace glob handling and `coverage.by_package` are correct on cal.com

repo: cal.com
what happened:
- coverage.by_package has 118 entries. Ground truth: 119 directories contain a tracked package.json; by_package lists all of them except the repo root. Nothing invented, nothing missed.
- The lockfile arm of dependency_provenance_gap reports “114 workspace manifest(s) compared; 5 manifest(s) outside the workspace were skipped (they are not installed by this lockfile)”. Ground truth: the 10 workspaces globs in the root package.json match 113 of the 119 dirs; plus the root itself = 114. The 5 skipped are exactly packages/app-store/templates/{basic,booking-pages-tag,event-type-app-card,event-type-location-video-static,general-app-settings}, which are template scaffolds and genuinely not workspace members.
- It correctly identified yarn.lock (yarn, 3,086 entries) and produced two other plausible sub-findings: 5 unpinned specifiers ("latest", "*") and 4 manifest/lock disagreements, all four of which are npm-alias specifiers ("dub-package": "npm:dub@^0.61.12"): i.e. it handles yarn aliases in the manifest but the lockfile comparison does not resolve them.
why it matters: The 0.17.0 workspace-glob work is doing its job. The failure above is in alias resolution, not in workspace parsing: worth separating so the right thing gets fixed.
severity: n/a (positive, with one narrow caveat about npm: alias specifiers)

`coverage` does not account for files it demonstrably scanned

repo: cal.com
command: crimes scan ... --all --format json
what happened: coverage.files_total = 5,386 (js 5,051 + files_universal_only 335, itemised as .md 300, .mdx 25, .sh 9, .php 1). But the findings array contains 146 findings on image files: 66 .png, 36 .jpeg, 30 .jpg, 12 .svg, 2 .gif: from oversized_raster (124), svg_with_embedded_raster (12), raster_should_be_vector (10). None of those files appear anywhere in coverage. The repo has 601 .sql files that appear in neither coverage nor findings, so there is no way to tell from the output whether SQL was skipped deliberately or silently failed.
why it matters: The protocol’s whole robustness premise is “a degraded scan and a clean scan look identical, so read coverage”. If coverage omits a whole class of files that produced 3% of the findings, it cannot be used for that purpose. Also files_by_language labels a repo of 3,878 .ts + 1,147 .tsx as {"js": 5051}: correct internally, misleading to read.
severity: real problem

Default human `scan` on cal.com: 5.7 minutes to produce 107 lines, four of the five top files are test scaffolding

repo: cal.com
command: crimes scan ~/crimes-dogfood/corpus/cal.com --no-init --no-color: real 344.61 (5 min 45 s), 107 lines of output
what happened: Read as a user would read it:
1. Header is good and true: “4562 findings across 2030 files · 485 high 2190 medium 1887 low”.
2. Then “Top files by risk”, 5 of 1,796. In order: packages/testing/src/lib/bookingScenario/bookingScenario.ts (test scenario builder), apps/web/playwright/fixtures/users.ts (Playwright fixture), apps/web/playwright/lib/testUtils.ts (Playwright util), apps/api/v2/.../user-bookings.e2e-spec.ts (e2e spec), apps/web/modules/users/components/UserForm.tsx. Four of five are test scaffolding.
3. The closing call to action is → Start with crimes context packages/testing/src/lib/bookingScenario/bookingScenario.ts: it points a first-time user at a test helper as “the most risk in this scan”.
4. The report does have a tests bucket: “Also flagged elsewhere: tests/ 666 findings”: so it knows to separate tests, but the bucket is matched on a literal tests/ path prefix and therefore catches none of playwright/, *.e2e-spec.ts, or packages/testing/.
5. File #5 spends 9 of its 13 lines on the same Option type declaration compared against 9 different files, one finding per pair, each printing the same left-hand side. It reads like a stuck record.
6. Of the 25 lines given to file #1, 17 are God Function and 1 is God File. The top of the report is one detector saying the same thing 18 times.
7. The scariest finding in the scan does not appear at all. dependency_provenance_gap: high severity, “148 external packages … a clean install can fail”: is anchored on package.json, which is not in the top-5 files, so a default reader never sees it. Same for the agent_permission_sprawl hit and all 33 mock_saturation hits.
why it matters: Honest reaction: on a repo this size the default view is readable (107 lines is fine) but it is not usable, because the ranking answers “which file has the most findings” and the user asked “what should I worry about”. The lede is buried twice over: once by test scaffolding outranking production code, once by per-file grouping hiding every repo-level finding. And it costs 5.7 minutes to get there.
severity: real problem

`hotspots` works but every rank below #3 is a 14-way tie at 0.43

repo: cal.com
command: crimes hotspots ~/crimes-dogfood/corpus/cal.com --no-init --no-color: real 407.73 (6 min 48 s)
what happened: Ranks 4 through 20 all read risk 0.43 (43%) with 1 change. Ranks 1-3 are 0.49/0.46/0.46 with 3/2/2 changes. The clone has 16,481 commits total but only 46 in the last 90 days, which is the window hotspots uses (“since 90d”), so churn carries almost no information and the ranking collapses into “which file has any commit at all in the window”, broken by finding count only incidentally: packages/app-store/server.ts with 1 finding outranks packages/features/bookings/repositories/BookingRepository.ts with 14.
why it matters: The command’s whole premise is churn x risk. When the churn window is nearly empty it degrades silently into a near-random list and still prints confident percentages. It should either widen the window automatically, say “churn signal is weak in this window”, or fall back to finding-count ranking and say so. Costing 6.8 minutes to produce a 17-way tie is the worst combination.
severity: real problem

`explain` is the best thing in the tool: but it re-scans, and its ids are positionally assigned

repo: cal.com (packages/trpc subtree, 428 files)
command: crimes explain <id> --no-init x 12, one per finding type
what happened: All 12 resolved and rendered: duplicated_policy, high_fan_in_fan_out, large_function, deep_import, return_shape_roulette, contract_drift, exact_duplicate_block, unbounded_async_fanout, near_duplicate_block, dependency_provenance_gap, swallowed_error, mock_saturation. The format (what the detector looks for / why it matters / evidence / risk profile / suggested actions / suppress command) is clear, specific, and the confidence arithmetic is shown. No errors, no missing types. Two problems:
1. explain re-runs the whole scan. There is no cache: 15 s on a 255-file scope, and on cal.com proper it would be ~7.5 minutes per explain. crimes scan prints id=crime_00076 … crime_01106 and invites you to explain them; doing so on a real repo is a 7-minute round trip each.
2. Ids are positional (crime_00107) and finding order is not stable between runs (202 of 3,593 positions moved between two runs on n8n). So crimes explain crime_00107 can resolve to a different finding than the one the earlier scan printed. The ignore command correctly uses a type::file::symbol fingerprint; explain’s advertised handle does not.
why it matters: The workflow the tool advertises: scan, then explain the interesting ids: is quadratically expensive and not referentially stable.
severity: real problem

`unbounded_async_fanout` mis-describes a synchronous array method as an awaited call

repo: cal.com
command: crimes explain crime_00107 (server/routers/loggedInViewer/eventTypeOrder.handler.ts:55)
what happened: Evidence says “collection source: an awaited call (query, request, or listing)” and “produced by: input.ids.reverse”. The collection is input.ids.reverse().map(…): Array.prototype.reverse on a tRPC input array, entirely synchronous. That mis-classification adds +0.14 and +0.08 to the confidence (0.60 → 0.88) and produces the suggested action “Add a page size or take to input.ids.reverse so the collection cannot grow”, which cannot be done to an array reverse.
why it matters: The fan-out itself is real and the finding may still be worth acting on: but two of the three confidence boosts and one of the two suggested actions come from a wrong reading of the source expression. Recorded factually; the judge can decide the rest.
severity: real problem

`pass_through_abstraction` builds “call chains” by method name across unrelated classes: reproduced on both repos

repo: cal.com (and n8n)
command: crimes scan ... --all --format json
what happened: The chain arm of the detector reports multi-file forwarding chains. On cal.com the highest-confidence example (0.92):
```
WatchlistAuditService.ts:32  deleteAuditEntry(…)  → this.deps.auditRepository.delete(…)
  CachedTeamFeatureRepository.ts:52  delete(…)    → this.prismaTeamFeatureRepository.delete(…)
    CachedUserFeatureRepository.ts:45  delete(…)  → this.prismaUserFeatureRepository.delete(…)
```
These are three unrelated classes. auditRepository is not a CachedTeamFeatureRepository, and prismaTeamFeatureRepository is not a CachedUserFeatureRepository. The hops are joined on the bare method name delete. Identical shape on n8n (packages/cli, confidence 0.98):
```
ephemeral-node-executor.ts:115  isAgentProviderNode(…) → AGENT_PROVIDER_NODE_WHITELIST.has(…)
  auth-handler.registry.ts:112   has(…) → this.handlerMap.has(…)
    breaking-changes.migration-registry.service.ts:36  has(…) → this.migrations.has(…)
      external-secrets.ee/provider-registry.service.ts:25  has(…) → this.providers.has(…)
```
Four different registries joined on has. The evidence then asserts “reading the chain end to end yields the same call the caller wrote”, and “spans 4 files” / “4 layers deep” each add confidence, so the deeper the accidental name collision, the more confident the finding gets.
why it matters: Both repos’ single most confident pass_through_abstraction finding (0.92 and 0.98) is of this shape, and confidence rises with the number of unrelated files pulled in. The single-file arm (“5 functions in this file exist only to forward to this.redis”) looks sound and is a different code path.
severity: blocker (for the chain arm; single-file arm unaffected)

`verdict` spends 13.8 minutes computing that nothing changed, when `git rev-parse` answers it instantly

repo: cal.com
command: crimes verdict --no-init --no-color (cwd = repo root): real 830.20 (13 min 50 s)
what happened: Output is 9 lines: base: origin/main, head: HEAD, Verdict: UNCHANGED, New: 0, Fixed: 0. HEAD and origin/main are the same commit on this checkout with a clean worktree. The tool ran two complete scans of a 5,000-file repo to discover that a commit is identical to itself.
why it matters: verdict is the CI-gate command. Fourteen minutes per invocation, on a question that a rev-parse comparison short-circuits, is the difference between a usable gate and an unusable one. A trivial guard (base SHA == head SHA and worktree clean → UNCHANGED immediately) removes the entire cost in the most common CI case where the branch has not diverged.
severity: real problem

Factual shape of the big 0.16 producers (characterisation only, no judgement)

swallowed_error: cal.com 114, n8n packages/cli 276.

Handler kind: cal.com 103 try/catch / 9 .catch() / 2 discarded rejection; n8n 229 / 33 / 14.
Not a test artefact. Test-shaped paths (__tests__, *.test.ts, playwright/, packages/testing/, *.e2e-spec.ts) account for 10 of 114 on cal.com and 1 of 276 on n8n. The overwhelming majority is production code.
What the handler does: “logs but never passes the error” 25/67, “returns null” 19/30, “returns undefined” 18/25, “contains only a comment” 15/53, “empty body” 15/36, “returns false” 10/22.
62/114 and 151/276 are catch { } with no error binding at all.
Boundary classification fires on a minority: 77/114 and 183/276 have no boundary label; the rest match on substrings like repository, delete, auth, fetch, webhook, session, cancel.
Severity: cal.com 70 low / 43 medium / 1 high; n8n 174 low / 102 medium / 0 high.

contract_drift: cal.com 307 (the single largest 0.16 producer anywhere in this pair).

277 of 307 compare two declarations with the same name.
Grouped by the “concept” the matcher reduced the names to: get 113, option 39, user 14, host 9, appkey 6. 131 of 307 are under packages/trpc.
The get cluster is the tRPC handler idiom: every handler file declares a local type GetOptions = { ctx: …; input: TSomeSchema }, and the detector pairs them off, reporting “disagree on 1 field: input is TListMembersSchema on one side, TGetAllByUserIdInputSchema on the other”.
The option cluster is one type Option = { label; value } in UserForm.tsx compared pairwise against nine other Option declarations: nine separate findings, all printed in the default human report.

duplicated_policy: cal.com 104, n8n 77. 100% production paths in both.

Summary shapes: “same conditional rule” 40/34, “same boolean rule” 31/5, “same guard clause” 19/23, “N variants … disagree on a value” 11/15, “same state-transition table” 3/0.
Domain vocabulary that drove the confidence bumps: credential/credentials (13 + 3 on cal.com, 7 + 2 on n8n), status, role, seats, organization, owner.
Clean examples exist (albyPaymentAppCredentials?.userCredentialIds || [-1] repeated verbatim across four payment-app Setup pages). The “variants disagree” arm is the one that groups distinct helper arguments: see the n8n log.

config_drift: cal.com 55, n8n 3.

The 55-vs-3 gap is a repo shape difference, not a detector inconsistency: n8n funnels env through src/config/index.ts, so the detector reports the 3 leaks; cal.com reads process.env directly all over, so the “used but not documented” (20), “server-shaped value read from client-reachable code” (8+4+3) and “different defaults” (7+5) arms all fire.
Scope: 47 production / 7 test / 1 fixture. It correctly declines to print values: “no configuration values are reported by this detector”.
The client-exposure arm is honest about its own limits: “whether a bundler actually ships this depends on the build; this is a path heuristic, not a bundling proof”.

mock_saturation: cal.com 33, n8n packages/cli 5. 100% in test files on both.

Every one is one test case, not one file. cal.com’s biggest reports 35 mocked collaborators, 95% of collaborators replaced, “2 mock-interaction, 0 behavioural” assertions.
n8n’s 5 all come from a single file (community-node-types.service.test.ts) in a repo with thousands of Jest/Vitest test files that use jest-mock-extended: so the trigger is narrow.

`context` is the one command that stays fast at this scale, but it silently re-roots and disagrees with `scan`

repo: cal.com
command: crimes context <file> --no-init --no-color (cwd = repo root)
what happened: Timings: 3.7 s, 11.2 s, 13.2 s: two to three orders of magnitude cheaper than scan (458 s) on the same repo, and the output is the most useful thing the tool produces (risk banner, agent guidance, related files, likely tests, churn/authorship clues, then the findings with evidence). Two wrinkles:
1. It re-roots to the nearest package and prints truncated paths. crimes context packages/features/bookings/lib/EventManager.ts prints file: bookings/lib/EventManager.ts and lists related files as bookings/lib/.... Copy-pasting those paths from the output does not work from the repo root.
2. Because the scope is reduced, it does not agree with scan: the full scan reports 14 findings on EventManager.ts, context reports 12. Nothing says the view is partial.
3. A path typo produces a clean crimes: file not found: <path> on stderr with an empty stdout and, notably, still writes a zero-byte file if you redirect. Fine, but exit code should be checked by any wrapper.
why it matters: This is the command that actually works on large repos and it is the one the tool recommends at the end of every scan. It deserves to agree with scan, or to say that it doesn’t.
severity: annoyance (paths) / real problem (silent disagreement with scan)

Zero cases and robustness checks: stated positively, per protocol

coverage.packs_loaded: ["universal", "language-js", "language-py"].
language-py was loaded and produced exactly zero findings on a repo with zero .py files. Verified by detector_id suffix, not by type string. Correct.
All three cross_language_* detectors produced zero on this single-language repo (cross_language_type_drift, cross_language_route_drift, cross_language_concept_alias_drift). Correct: this is the case where a false positive would be worse than a miss.
coverage.files_total 5,386; files_by_language {"js": 5051}; files_universal_only 335 (.md 300, .mdx 25, .sh 9, .php 1). See the coverage entry above for the 146 image findings that are missing from this accounting.
coverage.by_package present, 118 entries, verified correct against the repo’s 119 package.json directories.
Duplicate detection is not silently degraded: exact_duplicate_block 122, near_duplicate_block 48, duplicate_component_shape 1. JSX shape index is alive (1,147 .tsx files; duplicate_component_shape, responsive_fragility, accessible_interaction_risk, design_token_escape all fired).
Nothing on stderr. Exit 0. real 458.19, max RSS 419 MB.
All ten 0.16 detectors fired. None is zero on this repo.

ortomate-studio

Friction log: ortomate-studio

Repo: ~/dev/ortomate-studio SHA: 8ad20cea3b6e0c1889f62b8bc7578e64300c4139 (branch andrew-dev, clean tree) Build under test: crimes 0.17.0 Case: GRACEFUL DEGRADATION: 429 .swift files (91,778 LOC) vs 95 .js + 69 .ts + 4 .tsx (16,012 LOC). Swift is unsupported. Swift is 72% of tracked source files and 85% of source LOC.

Entries appended as they happen.

`crimes scan` never completes on this repo: killed twice, 358s and 731s

repo: ortomate-studio
command: crimes scan ~/dev/ortomate-studio --all --format json --no-init
what happened: Two independent runs. Run 1: SIGTERM at 358s real, output file 0 bytes. Run 2 (/usr/bin/time -l): SIGTERM at 731s real, 256s user, peak RSS 493 MB, output file 0 bytes. Non-deterministic wall clock, deterministic outcome: no JSON, ever. Exit 143 both times. Not an OOM: memory plateaus around 230–490 MB.
why it matters: For a user this is indistinguishable from a hang. There is no progress output, no spinner, no per-file logging, and nothing on stderr. You stare at a blank terminal for 12 minutes and then have nothing. The JSON contract is “the product” per CLAUDE.md, and on this repo the product does not emit.
severity: blocker

Root cause: minified-bundle exclusion is name-based only (`**/*.min.js`)

repo: ortomate-studio
command: per-subtree bisect: crimes scan <subdir> --format json --no-init
what happened: Bisected by subtree. Sources/ (Swift, 6.1 MB) = 3s. server/ = 4s. ios/ = 1s. chrome-extension/ = killed, never finishes. That subtree holds chrome-extension/reference/claude-1.0.77/assets/: 92 of the repo’s 95 tracked .js files, 12 MB, 812 total lines, with a single line of 1,621,186 characters. These are Vite-hashed vendor bundles (sidepanel-Bb4xbmwz.js, index-BYVE4JzA.js, mermaid.core-BAp6MO0B.js). DEFAULT_CONFIG.exclude in packages/core/src/config.ts only has **/*.min.js and **/*.generated.*: pure filename patterns. Modern bundlers do not emit .min.js; they emit name-HASH.js. Nothing in the default path looks at file size or line length before handing the file to the parser.
why it matters: Any repo that vendors a built frontend: extremely common: will hit this. The fix is cheap (bytes-per-line or max-file-size heuristic) but the failure mode is total.
severity: blocker

No CLI escape hatch for exclusions

repo: ortomate-studio
command: crimes scan --help
what happened: scan has no --exclude, no --max-file-size, no --timeout. The only way to exclude the vendor bundles is to write crimes.config.json into the repo: which a read-only evaluator (or anyone auditing a repo they don’t own) cannot do, and which --no-init explicitly exists to avoid.
why it matters: When the tool chokes on a directory there is no way to get past it from the command line. Broken and unworkaroundable is worse than broken.
severity: real problem

Timing caveat for this round

repo: ortomate-studio
what happened: ps showed up to 8 concurrent crimes scan processes from other agents in this dogfooding round on the same machine. All wall-clock numbers here are contended and should be treated as upper bounds. CPU% per process sat at 12–90%.
severity: annoyance (measurement hygiene, not a tool defect)

251 seconds to analyse ONE 1.9 MB minified file: and it emits 23 findings

repo: ortomate-studio (isolated repro, file copied to scratch)
command: crimes scan <dir-with-one-file> --format json --no-init
what happened: Copied a single vendor bundle (chrome-extension/reference/claude-1.0.77/assets/sidepanel-Bb4xbmwz.js, 1.9 MB, 6 lines, longest line 1.6 M chars) into an empty git repo and scanned it. rc=0 in 251 seconds. So the full-repo scan is not hung: it is ~4 minutes per big bundle × 92 bundles. It would need hours. Worse, it did not skip the file: it emitted 23 findings on minified vendor code: 8 swallowed_error, 5 name_behavior_mismatch, 5 return_shape_roulette, 1 direct_date (“94 direct uses of Date.now()/new Date()”), 1 dst_naive_arithmetic, 1 negative_flag_maze, 1 boolean_naming_drift, 1 pass_through_abstraction. One rated high.
why it matters: Two defects in one. (1) There is no size/line-length guard before parsing, so cost is unbounded. (2) util/scope-class.ts is supposed to be “one answer to generated / vendored / … / production” and it classified a Vite output bundle as production domain code. Every one of those 23 findings is noise a maintainer cannot act on: you do not refactor Date.now() out of a vendored build artifact.
severity: blocker

CORRECTION: the scan does complete, in 675 s. It is 135× slower than it should be.

repo: ortomate-studio
command: crimes scan ~/dev/ortomate-studio --all --format json --no-init
what happened: A fourth attempt, run detached, finished: real 675, EXIT=0, 1.3 MB of JSON, empty stderr, 851 findings. So the earlier SIGTERMs at 358 s and 731 s were the harness reaping a long-running job, not the tool. Retracting “never completes”; the accurate statement is 11 minutes 15 seconds. The same repo with the 92 vendored bundles removed scans in 5 seconds. Same file count otherwise, same detectors. That is a 135× penalty paid for 12 MB of files nobody wants analysed.
why it matters: Still disqualifying in practice: nothing in crimes init --agents’ pre-edit-hook story survives an 11-minute scan, and the CLI prints no progress for the entire time: but “slow” is a different bug from “hangs” and the report must say the right one.
severity: real problem (downgraded from blocker)

52% of all findings: and 15 of the top 20 by `agent_risk`: are in vendored bundles

repo: ortomate-studio
command: crimes scan … --all --format json --no-init
what happened: Of 851 findings, 444 (52.2%) are inside chrome-extension/reference/claude-1.0.77/: a checked-in copy of somebody else’s built Chrome extension, across 57 distinct minified files. 17 of them are rated high. Breakdown: 145 name_behavior_mismatch, 120 return_shape_roulette, 78 swallowed_error, 17 high_fan_in_fan_out, 15 direct_date, 14 boolean_naming_drift, 12 unsafe_retry, 9 duplicated_policy. Ranked by agent_risk, positions 6 through 20 are all vendored bundles: mermaid.core-*.js, cytoscape.esm-*.js, ganttDiagram-*.js. Only positions 1–5 are real code.
why it matters: Two things break at once. (1) The headline number is double what it should be: 851 vs 407: so severity counts are meaningless. (2) This round’s own sampling rule is “top 20 findings by agent_risk per repo”. Following it here sends 15 minified-bundle cards to the blind judge, who will correctly call them noise, and the resulting act rate will indict the wrong detectors. name_behavior_mismatch and return_shape_roulette will look catastrophic when the actual defect is scope classification.
severity: blocker

`crimes diff` inherits the pathology from history and cannot finish

repo: ortomate-studio
command: crimes diff HEAD~5...HEAD --no-init
what happened: Killed at >9 minutes, no output. diff re-scans the base ref, and every reachable ref in this repo contains the vendored bundles, so there is no range for which diff is usable. There is no way to scope it.
why it matters: diff main...HEAD is the CI story. On this repo the CI story does not run.
severity: real problem

Swift findings are structurally incapable of ranking: two of six scores are hard zero

repo: ortomate-studio
command: crimes scan … --all --format json --no-init, score analysis
what happened: Across all 175 .swift findings: blast_radius = 0 (175/175), blast_radius_importers = 0 (175/175), test_gap = 0 (175/175). Not distributed: constant. The 217 .ts findings have blast_radius_importers spread across 0–55 and test_gap = 0.5. Consequence: Swift agent_risk maxes at 0.56 (mean 0.25) while TS reaches 0.79 (mean 0.38). Ranking is by aggregate risk, so 429 files: 85% of the repo, 92% of its source lines: are pushed below the TS files by arithmetic, independent of their actual risk.
why it matters: These are not measurements, they are absences. There is no Swift import graph and no Swift test mapping, so the scorer substitutes 0. The schema has no way to say “unknown”, so a missing signal is indistinguishable from a measured-zero signal: and the ranking treats it as good news.
severity: real problem

A missing test signal is rendered to the user as a good test signal

repo: ortomate-studio
command: crimes context Sources/OrtomateStudio/Projects/ProjectsStore.swift --no-init
what happened: ProjectsStore.swift is 4,949 lines, 97 commits in 90 days, and has no test file. context prints · test gap: bottom-quartile: the exact same phrase a genuinely well-covered TypeScript file gets, because test_gap = 0 renders as “bottom-quartile” (i.e. least gap). It also prints Likely tests: (no test file matched the target basename (looked for siblings, __tests__/ and tests/ directories, and the .test / .spec / _test / test_ naming conventions)…): every one of those is a JS/TS convention. This repo has Tests/OrtomateStudioTests/*Tests.swift, 20+ files, the standard SwiftPM layout, and none of it is recognised.
why it matters: This is the sharpest form of the honesty problem. The tool does not merely omit Swift: on its most-changed Swift file it reports the opposite of the truth about test coverage, in the command specifically designed to brief an agent before it edits that file.
severity: real problem

`by_package` covers only the JS packages of a Swift-first repo

repo: ortomate-studio
command: crimes scan … --all --format json --no-init
what happened: coverage.by_package lists exactly two entries: chrome-extension (109 files, js 107) and server (62 files, js 60). The repo has three SwiftPM packages: root Package.swift, OrtomateRemoteKit/Package.swift, and ios/. None appear. by-package.ts maps only language-js→js and language-py→py, so package discovery is package-manifest-driven for npm only.
why it matters: A user reading by_package on this repo sees a two-package Node project. It is a 429-file Swift application with a small Node sidecar.
severity: real problem

THE HEADLINE: the default report on a 429-file Swift app is five minified vendor bundles and no mention of Swift

repo: ortomate-studio
command: crimes scan ~/dev/ortomate-studio --no-init (574 s)
what happened: Header: 851 findings across 255 files · 55 high 391 medium 405 low. “Top files by risk”: all five are chrome-extension/reference/claude-1.0.77/assets/*.js: mcpPermissions-…, useStorageState-…, sidepanel-…, mermaid.core-…, wardley-…. The findings read as you would expect from minified code: Return Shape Roulette · I 115 object-literal return shapes, Unprefixed Boolean 'So' @L1, Logic in the Alibi comment says: ",o=/^\/\//;function l(e,t){if(!e.scheme…", Abstraction Laundering · qn(). Closing call to action: → Start with crimes context chrome-extension/reference/claude-1.0.77/assets/mcpPermissions-CJK8I7C7.js. The string “swift” appears exactly once in the 174-line report, and only because I passed --explain-coverage. Without that flag it appears zero times.
why it matters: This is the product-integrity question stated plainly. A maintainer of this repo runs npx crimes scan: the canonical first-run command: waits ten minutes, and is told their biggest risk is a third-party file they have never opened, in a directory literally named reference/. Meanwhile 91,778 lines of Swift, 92% of the source they actually maintain, are never named.
severity: blocker

VERDICT on the honesty question: the JSON tells the truth, the default UI does not

repo: ortomate-studio
command: crimes scan … --format json vs crimes scan … vs … --explain-coverage
what happened: Nothing is silently dropped. coverage.files_total = 619 and includes every Swift file; coverage.files_universal_only = 450 with universal_only_by_extension = {".swift": 429, ".md": 19, ".sh": 2}; and --explain-coverage prints an unusually good paragraph: “These files were not claimed by any loaded language pack. They received universal-pack checks only (file size, raster assets, duplicate filenames, hardcoded localhost/local paths, docs link checking).” That is exactly the right sentence. It is behind an off-by-default flag, printed last, after the report it should be qualifying. The default human path emits no coverage signal of any kind. What “universal-pack only” costs in practice: the 175 .swift findings come from just 4 detector types (large_file 84, commented_out_code 79, hardcoded_localhost 8, hardcoded_local_path 4). The other ~45 detectors: every correctness, duplication, authority and agent-risk check: ran on the 16,012 lines of TS/JS and on none of the 91,778 lines of Swift.
why it matters: The honest answer to “is the tool truthful here” is yes in the contract, no in the interface. Promoting one line (429 of 619 files: universal checks only (.swift)) into the default header would close the gap entirely. It is the cheapest high-value fix in this log.
severity: real problem

Methodology note: which artifacts came from where

repo: ortomate-studio
what happened: scan --all --format json (ortomate-studio.json, 675 s) and scan human (ortomate-studio.human.txt, 574 s) were run against the real repo at 8ad20cea, read-only, untouched: verified by ls -A + git status --porcelain diff before and after. Those are the authoritative numbers. hotspots, verdict, context, explain and the .novendor.* artifacts were run against a local clone in scratch (git clone --local, same SHA) with one extra commit removing chrome-extension/reference/, because each of those commands re-scans and 11 minutes × 5 was not affordable. The clone shares the full git history, so churn/recency figures carry over; finding counts in the .novendor.* artifacts are for the 527-file subset, not the repo. ortomate-studio.diff.txt is 0 bytes: diff never completed (see above).
severity: n/a: measurement provenance

mysidekick-cc

Friction log: mysidekick-cc

Repo: ~/dev/mysidekick-cc SHA: 7e0372210adcebc0ca7601c977bb3b354136bd40 (branch main; vault/knowledge.md modified pre-existing, not touched by this round) Build under test: crimes 0.17.0 Case: ZERO CONTROL: 176 .ts + 1 .js (21,943 LOC), zero .py. Single-language TypeScript.

Entries appended as they happen.

`.gitignore` is not respected: 19% of all findings are from gitignored build artifacts

repo: mysidekick-cc
command: crimes scan ~/dev/mysidekick-cc --all --format json --no-init
what happened: api/_server.mjs (34 findings) and public/call.bundle.js (12 findings) are esbuild output, both explicitly listed in .gitignore, both untracked. That is 46 of 241 findings = 19.1%. _server.mjs is a 2,668-line bundle of src/*.ts, so every finding in it is a duplicate of a finding already reported against the real source. Same for the 33 gitignored vault/daily/*.md files: coverage.files_universal_only is 58 and the repo only tracks 25 .md: the other 33 are gitignored journal entries. Root cause is not subtle: packages/core/src/discovery/files.ts calls fast-glob with ignore: options.exclude and nothing else. There is no .gitignore parsing anywhere in discovery. dot: false is why .build/-style dot-dirs get skipped: that is incidental, not gitignore support.
why it matters: CLAUDE.md states the locked stack decision as “File walking: fast-glob + ignore (respect .gitignore)”. It is not implemented. The consequence is not cosmetic: the human report’s single closing call-to-action is → Start with crimes context api/_server.mjs: it concentrates the most risk in this scan. The tool’s one piece of advice points the user at a generated file they must not edit.
severity: blocker

The zero case passes cleanly, but `packs_loaded` lies

repo: mysidekick-cc
command: crimes scan … --all --format json --no-init
what happened: Zero .py files in the repo. Confirmed positively: cross_language_route_drift, cross_language_type_drift, cross_language_concept_alias_drift → 0 findings each. Python pack → 0 findings. pack field across all 241 findings is only language-js (223) and universal (18). No language-py, no cross. That is the right answer. But coverage.packs_loaded is ["universal", "language-js", "language-py"]. packages/core/src/discovery/coverage.ts:58 builds it as ["universal", ...router.registeredPacks()]: registration happens at module load, so language-py is listed on every scan of every repo on earth.
why it matters: The protocol’s own robustness check is “packs_loaded: and whether the pack actually produced findings”, precisely because the field cannot answer that. It is a constant. The honest signal is right next to it: files_by_language is {"js": 181} with no py key at all. So the data exists; the field named packs_loaded is just the wrong one to read, and nothing labels it as such.
severity: real problem

The default human report is a wall, not a summary

repo: mysidekick-cc
command: crimes scan ~/dev/mysidekick-cc --no-init (no --all)
what happened: 103 lines. --top caps files at 5 but does not cap findings within a file, so the very first block is a flat numbered list of 34 items for api/_server.mjs: items 24 through 34 are eleven consecutive Catch and Release findings, seven of which are the identical shape (res.text() inside a try). src/routes/settings.ts further down has 7. PRD principle is “Default crimes scan shows top findings only, not everything. --all is opt-in”: but the default already prints everything for the files it does show.
why it matters: The lede is buried by construction. Nothing distinguishes the one finding worth acting on from ten near-identical repeats of the same pattern. Collapsing repeats of one charge in one file into Catch and Release ×11 (lines …) would cut this report roughly in half with no information loss.
severity: real problem

Three different file counts in three commands, none reconciled

repo: mysidekick-cc
what happened: scan header: “241 findings across 74 files”; then “Showing 5 of 63 files”; JSON coverage.files_total is 239; hotspots header says “220 files”. Four numbers, no explanation of the relationship between any two of them. (74 is real: I verified distinct file values in the JSON. 63 is 74 minus the files rolled into the “Also flagged elsewhere” line. 239 is files scanned. 220 is files with git history in the window.)
why it matters: Every one of those is defensible on its own and the set is incoherent to read. A user cannot tell whether the tool is confused or they are.
severity: annoyance

`hotspots` is the honest view and `scan` is not: and the tool prefers `scan`

repo: mysidekick-cc
command: crimes hotspots … --no-init vs crimes scan … --no-init
what happened: hotspots ranks src/server.ts, src/prep/compileBrief.ts, src/db.ts: the real, high-churn, load-bearing files. The gitignored bundles are absent, because they have no git history and churn-based ranking drops them for free. scan puts one of those same bundles at #1. The two commands disagree about what matters most in the repo, and the one the docs push you to (npx crimes scan is “the canonical first-run command”) is the wrong one.
why it matters: The fix for scan may be as cheap as borrowing what hotspots already knows: a tracked-in-git check.
severity: real problem

`context` and `explain` are the best surfaces in the tool

repo: mysidekick-cc
command: crimes context src/prep/compileBrief.ts --no-init, crimes explain crime_00001 --no-init
what happened: Recording a positive. context gave agent guidance, related files with why they are related (“shares domain token ‘prep’”), 8 likely tests, churn/author/test-gap clues, then every finding with full evidence. explain showed the confidence arithmetic inline (confidence 0.78 = 0.72 base + 0.06 (handler contains only a comment)) and a copy-pasteable suppression command. Both ran in 3–5 s. Every summary I read explained itself without opening the file.
why it matters: These justify the product. They are also the two commands the human scan report does the least to lead you toward.
severity: n/a: positive

`crimes diff` returns three integers and nothing to act on

repo: mysidekick-cc
command: crimes diff HEAD~5...HEAD --no-init
what happened: Entire output: “New crimes: 18 / Fixed crimes: 3 / Unchanged crimes: 181”. No list, no ids, no files, and no hint that --format json would give more. --all is not accepted.
why it matters: “18 new crimes” without saying which ones is not actionable. verdict at least names a reason and a next action; diff, the lower-level command, gives less.
severity: annoyance

Misleading Commander error on a malformed command

repo: mysidekick-cc
command: crimes "hotspots /path" --no-init --no-color (my own zsh quoting mistake: zsh does not word-split unquoted $var)
what happened: Error was error: unknown option '--no-color', exit 1. The unknown thing was the command, and --no-color is a documented, working option for that command.
why it matters: Sent me looking for an option-parsing bug in hotspots that does not exist. Low value, but it cost real minutes.
severity: annoyance

pydantic

Friction log: pydantic

Repo: ~/crimes-dogfood/corpus/pydantic @ 63f2722f6bfedf0c5b645b348a62ca0ce0b5358c Binary: ~/crimes-dogfood/bin/crimes (crimes 0.17.0)

Size at scan time: 405 .py (171,350 LOC), 127 .rs (pydantic-core submodule, checked out), 99 .md, 57 .png, 24 .yml, 19 .toml. 289 of the 405 .py files are tests/conftest (110,559 LOC = 64% of Python LOC).

Note: this repo is NOT pure Python. The pydantic-core/ submodule is checked out with 127 Rust source files. crimes has no Rust pack, so this is a live test of files_universal_only.

`tier` is JS-only: 100% of Python findings are labelled `domain`, including tests

repo: pydantic (identical on drf)
command: crimes scan <repo> --all --format json --no-init
what happened: [...new Set(findings.map(f => f.tier))] == ["domain"] for 578/578 pydantic findings and 104/104 drf findings. 118 pydantic findings (20.4%) and 20 drf findings (19.2%) are physically inside tests/, and every one of them is tier: "domain". 21 pydantic findings are in docs/: also domain.
root cause (read from source, no change made): packages/core/src/config.ts:17 DEFAULT_NON_DOMAIN_PATTERNS is ["scripts/**","examples/**","fixtures/**","public/**","**/__tests__/**", "**/*.test.{ts,tsx,js,jsx}","**/*.spec.{ts,tsx,js,jsx}"]. Not one pattern can match a Python test. Python uses tests/, test_*.py, *_test.py, conftest.py; the glob list knows only __tests__/ and the .test./.spec. infix on {ts,tsx,js,jsx}.
the twist: crimes already has a correct Python-aware test classifier: packages/core/src/util/test-files.ts TEST_FILE_RE matches tests/, test_*.py and *_test.py, and its own doc comment brags about unifying this in 0.14.0 (“a file is a test file or it isn’t”). And util/scope-class.ts classifyScope() builds on it. But tier does not use either: context-helpers.ts:139 calls makeTierClassifier(config.scopeTiers.nonDomain), a third, glob-based, JS-only mechanism. Three test classifiers, and the one that reaches the JSON contract is the only one that can’t read Python.
and classifyScope barely matters anyway: grepping consumers, classifyScope is imported by exactly 4 detectors: unsafe-retry, unbounded-async-fanout, swallowed-error, dependency-provenance-gap: all 0.16.0 JS-side detectors. Zero Python detectors and zero universal detectors (large_file, large_function, commented_out_code, todo_density) consult it. So on a Python repo the shared exclusion policy is dead code.
why it matters: This is the exact mechanism the protocol names as the answer to “did crimes discount tests”. On Python it is inert. A consumer filtering tier == "domain" to get real code gets the whole test suite too. It also means weak_test_signal.py (52 findings, 100% in tests, by construction) competes for the default top-N against production findings.
severity: real problem

Parser stress test: the grammar is solid, with ONE real gap (PEP 696)

repo: pydantic
command: direct probe of packages/language-py/dist/index.js parsePyFile() over all 406 .py/.pyi files, plus a synthetic grammar matrix.
whole-repo sweep result: 406 files, 405 clean, 1 hasSyntaxErrors, 0 threw, 0 files with def lines but zero parsed functions. drf: 154/154 clean. So the answer to “did it actually parse” is essentially yes.

grammar capability matrix (synthetic files, one construct each, run through parsePyFile):

construct	result
`match`/`case` incl. mapping, sequence, class, or-patterns	clean, symbols correct
walrus `:=` (if and while)	clean
`except` / `except (A, B) as eg` (PEP 654, 3.11)	clean
PEP 701 nested-quote f-strings (3.12)	clean
PEP 695 `def f[T]()`	clean
PEP 695 `class C[T]`	clean, class+method extracted
PEP 695 `type Alias = int`, `type Pair[T] = ...`	clean
PEP 695 bounds `class M[T: int]`	clean
PEP 695 `class M[Ts]`, `class M[*P]`	clean
PEP 696 type-param defaults `[T = int]` (3.13)	`hasSyntaxErrors = true`
`@overload` chains	clean, all 3 arms extracted separately
metaclass `__new__` / `metaclass=`	clean, `Meta`+`C`+methods extracted
`if TYPE_CHECKING:` + string annotations	clean, imports counted
positional-only `/`, kw-only ``, `*kw`	clean
PEP 614 arbitrary decorator expressions	clean

the one gap, isolated to a 1-line repro: any = <default> inside a type parameter list is a hard grammar error. Verified across all four positions:
- class M[T, U = int]: → ERROR
- def f[T = int](x): ... → ERROR
- type Al[T = int] = list[T] → ERROR (and 0 functions, 0 classes extracted)
- def f[*Ts = *tuple[int]](x): ... → ERROR
The same declarations without defaults parse clean, as do bounds, *Ts and **P: so this is specifically PEP 696 (Python 3.13), not PEP 695. Bundled grammar is the vendored tree-sitter-python.wasm under packages/language-py/dist/; web-tree-sitter ^0.26.11.
blast radius today: small but non-zero. Recovery is graceful for the def/class forms (symbols still come out), but the type Al[T = int] form loses the entire file’s symbol table.
severity: real problem (will grow: 3.13 defaults are new but spreading)

`hasSyntaxErrors` is invisible: a degraded scan and a clean scan are identical

repo: pydantic / drf
command: grep -rn hasSyntaxErrors packages/*/src
what happened: hasSyntaxErrors is set correctly by language-py/src/parse/index.ts:50 and then consumed by exactly one detector: core/src/detectors/py/weak-test-signal.ts:38, which returns [] for the file. It is never aggregated into coverage, never printed, never in the JSON. There is no coverage.parse_errors / files_failed field.
why it matters: This is the precise failure mode the protocol warns about (“A degraded scan and a clean scan look identical”). A repo that leans on PEP 696, or that a future grammar regression breaks, would return a small, reassuring finding count. I only found pydantic’s one bad file by importing the parser directly: no CLI surface would have told me.
fix shape: add coverage.parse_errors: {count, files[]} (schema change, minor + migration) or at minimum a stderr warning (patch).
severity: real problem

Bonus: tree-sitter flags a file CPython accepts

repo: pydantic
file: tests/mypy/outputs/mypy-plugin_ini/frozen_field.py
what happened: the single hasSyntaxErrors file in pydantic. CPython ast.parse accepts it. Minimal repro (5 lines, valid Python):
```
class C:
    @property
# a comment at column 0
    def f(self) -> str:
        return "x"
```
A column-0 comment between a decorator and its def, inside an indented block → ERROR node. Move the comment to the decorator’s indent and it parses clean. Python’s tokenizer ignores comment-only lines for INDENT/DEDENT; the grammar does not.
severity: annoyance (upstream tree-sitter-python issue, not crimes’ code)

Negative result: `except*` and PEP 695 are NOT exercised by pydantic

what happened: I set out to confirm findings originated from files using each construct. Two constructs are absent from the corpus:
- except*: grep -rnE '^\s*except\*' over 405 files: zero hits. My first grep “found” one, but it matched the prose *except* inside a comment.
- PEP 695: all 14 real-looking sites (def f[T], class M[T], type X =) are inside triple-quoted strings passed to create_module(...) in tests/test_validate_call.py / tests/test_forward_ref.py, or inside a docstring example in pydantic/_internal/_generics.py:363. Not one is module-level code. The parser correctly extracts nothing there: so would CPython’s ast.
why it matters: worth stating plainly so the round does not claim coverage it does not have. pydantic is a weaker modern-syntax stress test than its reputation suggests; it targets 3.9+ and puts new syntax behind create_module + skipif(sys.version_info < (3,12)). The synthetic matrix above is the real evidence, not the corpus.
severity: annoyance (observation about the corpus)

`boolean_naming_drift.py`: 33 findings / 116 bindings, and 42% aren’t assignments at all

repo: pydantic (33 findings, 116 bindings, 72 distinct names) / drf (7 findings, 13 bindings)
command: crimes scan <repo> --all --format json --no-init
shape (characterisation only, no verdict):
- Initializer kinds, pydantic: boolean literal 55, other 49, comparison 5, membership 4, negation 2, identity 1.
- The other bucket is annotation-only declarations. Spot-checked every <other> in pydantic/v1/config.py (10 of them, lines 48–70): the source is anystr_lower: bool, validate_all: bool, orm_mode: bool … inside class ConfigDict(TypedDict, total=False). There is no = and no value. Same for pydantic/fields.py:85: the source is repr: bool inside a TypedDict. The finding renders it as `repr` = <other> and the summary reads “names bound to a boolean expression”. For 49 of 116 bindings that sentence describes something that does not exist in the file.
- Binding site, pydantic: module/class-body 65, local-in-function 36, instance-attribute 15. So the majority are class-body declarations, i.e. declared API surface, not local control flow.
- What the names are. pydantic: orm_mode, validate_all, arbitrary_types_allowed, populate_by_name, str_strip_whitespace, hide_input_in_errors, from_attributes, frozen, repr, init: these are pydantic’s documented public config keys and Field() kwargs; several (repr, init, init_var, frozen) are name-compatible with dataclasses.field() by design. drf: many (ListSerializer.many, DRF’s single most-used public kwarg), authenticated_users_only, public, ensure_ascii (mirrors json.dumps), and: at rest_framework/authtoken/models.py:25,60: abstract and proxy, which are Django class Meta reserved option names. Those two cannot be renamed; Django reads them by name.
- 0 of 40 findings across both repos are in test files.
why it matters: the fix the detector proposes (is_/has_/should_ prefix) is, for the majority of these, a breaking public-API rename or an impossible framework rename. The evidence string also misdescribes 42% of what it matched.
severity: real problem

`sync_io_in_hotpath.py`: 11 findings, 8 of which are in build/release tooling

repo: pydantic (11) / drf (1)

shape: bucketed by what the file is:

bucket	n	files
release automation scripts	3	`release/prepare.py`, `release/push.py`, `release/shared.py`
mkdocs build plugins	2	`docs/plugins/main.py`, `docs/plugins/using_update.py`
mypy plugins (compile-time)	2	`pydantic/mypy.py`, `pydantic/v1/mypy.py`
test/preview runner	1	`pydantic-core/wasm-preview/run_tests.py`
library runtime	3	`pydantic/_internal/_git.py`, `pydantic/v1/parse.py`, `pydantic/deprecated/parse.py`

8 of 11 are in code that runs once, offline, on a developer’s or CI’s machine. There is no worker and no event loop to hold. The summary nonetheless says “Each one holds the worker (or the event loop) for its full duration”, and every finding asserts the call runs inside <fn> (shape "domain"). Of the 3 “library runtime” hits, two are parse.py: functions literally named load_file/load_str_bytes whose stated job is to read a file. drf’s single hit is rest_framework/management/commands/generateschema.py handle(): a Django management command, i.e. a CLI entry point.

not all wrong: the async sub-claim is accurate where it fires. wasm-preview/run_tests.py main() genuinely is async def (it awaits micropip.install), and the evidence correctly escalates for it.
root cause is the same missing scope policy: release/**, docs/plugins/** and **/management/commands/** are not in DEFAULT_NON_DOMAIN_PATTERNS (scripts/** is; release/** is not), and sync-io-in-hotpath does not call classifyScope. The “hotpath” test is shape === "domain", which is a function-shape judgement made without reference to whether the file is ever on a request path.
severity: real problem

`direct_date.py`: 3 findings in pydantic, 0 in drf. Genuinely quiet.

shape: pydantic/v1/types.py:1161,1174 and pydantic/v1/_hypothesis_plugin.py:172,173 (both in the frozen v1 compat tree), and release/prepare.py:113 (a release script). All date.today(). Evidence is precise: exact call, exact lines, and a correct naive-vs-aware sub-claim. Zero in test files. Zero in drf.
observation: this detector did not over-fire. Of the three named as most-likely-to-over-fire, direct_date is the one that behaved. Note that 2 of 3 are in pydantic/v1/**: a deliberately-frozen vendored-in copy of pydantic v1 that upstream will never modernise. classifyScope’s vendored bucket doesn’t match v1/, and nothing consults it here anyway.
severity: annoyance

`docs_code_drift`: indented code fences are not recognised, so Python generics parse as markdown links

repo: pydantic
finding: crime_00411, docs/concepts/models.md: “16 local links … point at file(s) that do not exist on disk”, with evidence docs/concepts/models.md:731 → data=1 (not found), :733 → data='value' (not found), :735 → data='value' (not found) …
root cause, traced to one regex: packages/core/src/ia/extract.ts:233: const FENCE_RE = /^```/;: is anchored to column 0. pydantic’s docs put fences inside mkdocs-material admonitions and tab blocks, so they are indented: docs/concepts/models.md:715 is ```python {upgrade="skip"}. parseMarkdown never enters fence state, so the whole code block is scanned as prose by MD_LINK_RE = /\[([^\]]+)\]$([^)\s]+)(?:\s+"[^"]*")?$/g.
and the collision is exact: a Python generic subscript immediately followed by a call: Response[int](data=1): is character-for-character a markdown inline link [int](data=1). pydantic’s docs are about generics, so this is maximally unlucky.
scale: grep -rnE '^[ \t]+```' docs/ = 246 indented fences vs 1000 column-0 fences in pydantic’s docs. ~20% of the doc corpus is invisible to the fence tracker. Every generic example inside one is a candidate link.
fix shape: allow up to 3 leading spaces per CommonMark (/^ {0,3}```/), and ideally track the opening fence’s indent to find its close. Patch-level (detector tuning + eval re-run).
severity: real problem

Vendored / minified assets: `.min.js` is excluded, `-min.js` is not

repo: drf
what happened: rest_framework/static/rest_framework/js/prettify-min.js: a 13 KB minified copy of Google’s prettify: produced 2 findings (direct_date.js at :27, and exact_duplicate_block.js). Its siblings bootstrap.min.js (39 KB) and jquery-3.7.1.min.js (87 KB) produced zero.
root cause: packages/core/src/config.ts:216: the only minification pattern in the default exclude list is "**/*.min.js". The hyphen form *-min.js is not covered, and neither is *.min.css / *-min.css.
the duplicate finding is the worse half: crime_00053 exact_duplicate_block.js pairs docs/theme/js/prettify-1.0.js:2-28 with rest_framework/static/rest_framework/js/prettify-min.js:2-28 and proposes “extracting a shared helper”. Those are the pretty-printed and the minified build of the same vendored third-party file. There is no shared helper to extract. classifyScope’s vendored bucket doesn’t match static/ either: and exact_duplicate_block doesn’t call it.
severity: real problem (config default: cheapest fix in the round)

Schema: `lines` is optional, and 7 findings per repo omit it

repo: pydantic (7) / drf (7)
what happened: a consumer doing f.lines.join("-") crashes. File-level and repo-level findings carry no lines: circular_dependency.py, docs_code_drift, oversized_raster, raster_should_be_vector, exact_duplicate_block.js.
why it matters: the JSON output is “the product contract”. The judging protocol for this very round requires “file path, line range, the code ±30 lines” per card: for 14 findings across these two repos there is no line range to build a card from. Not wrong, but it needs to be documented, and card tooling needs a fallback.
severity: annoyance

`crimes context` prints finding IDs that `crimes explain` resolves to different findings

repo: pydantic and drf (both)

commands:

crimes context pydantic/_internal/_generate_schema.py --no-init
  →  2. pydantic/_internal/_generate_schema.py:738-875 (_model_schema)
        Charge: God Function  …  id=crime_00002

crimes explain crime_00002 --no-init
  →  charge: God File
     file:   pydantic/types.py
     lines:  1–3336

Different file, different detector, different charge. No error, no warning. Reproduces on drf: context rest_framework/fields.py labels fields.py:86-110 as crime_00002; explain crime_00002 returns fields.py:1-1966.

mechanism: crime_NNNNN is a positional index within whichever report was just produced. context renumbers from 1 within its file-scoped report; explain renumbers from 1 within a fresh repo-wide scan. The two numbering spaces are unrelated but look identical and share a prefix.
why it matters: scan → note the id → explain the id is the documented loop, and context is the command the crimes agent skill tells agents to run before editing. An agent that reads a context finding and calls explain on the printed id gets a confident, well-formatted explanation of the wrong finding, and nothing anywhere says so. The id= suffix reads like a stable handle; it is a line number in disguise.
note: 0.17.0 added discriminator fingerprints and explain accepts “id or fingerprint”, but only some detectors emit discriminator (of the 578 pydantic findings I saw it on exact_duplicate_block), so it is not yet a general substitute.
fix shape: either make the id content-derived and report-independent, or have context print the fingerprint instead of a positional id.
severity: real problem

Positive: output is deterministic

repo: pydantic
command: crimes scan . --all --format json --no-init run three times
what happened: all three byte-identical, including the run captured 20 minutes earlier as the artifact. cmp clean. Worth recording because the protocol asks specifically about evidence reproducibility across runs.
severity: n/a (positive)

`crimes context` “Related files” is just directory siblings, alphabetically

repo: pydantic / drf
command: crimes context pydantic/main.py --no-init
what happened: Related files = pydantic/__init__.py, alias_generators.py, aliases.py, annotated_handlers.py, _migration.py: ”: same directory” for every entry, and they are simply the first 5 filenames alphabetically in that directory. main.py actually imports _internal/_model_construction.py, _internal/_fields.py, etc., and the import graph is clearly built (circular_dependency.py uses it, blast_radius_importers: 285). The section presents an alphabetical ls as if it were derived relatedness.
severity: annoyance

`crimes context` “Likely tests” is unbounded and imprecise

repo: drf
command: crimes context rest_framework/serializers.py --no-init
what happened: 35 likely tests, printed in full in the human view: longer than the findings section. The list includes non-tests that merely live under tests/: tests/browsable_api/serializers.py, tests/browsable_api/views.py, tests/schemas/views.py, tests/importable/__init__.py. testBaseCovers() matches tests/browsable_api/serializers.py to rest_framework/serializers.py on basename alone, ignoring that the former is a test helper.
why it matters: basename matching across a deep test tree gets progressively worse the more conventional the naming. “Read the likely tests before editing” is not actionable at 35 entries.
severity: annoyance

`large_function.py` is 43% of the corpus, and ~42% of it isn’t about size

repos: pydantic 249, drf 47 = 296 of 682 total findings (43%)
split by what actually triggered it (comparing the finding’s own “N lines (threshold T)” evidence):

repo size-triggered (N > T) nesting-triggered (N ≤ T)
pydantic 148 (59%) 101 (41%)
drf 24 (51%) 23 (49%)

Of the 124 nesting-triggered findings, 81 are at nesting depth exactly 4: the trigger point.
the summary is wrong for all 124 of them. Example, drf rest_framework/serializers.py:1452-1467:

get_unique_together_constraints is 16 lines and nests 5 levels deep. At this size an agent must read the whole body before it can safely change any part of it.

“At this size”, for a 16-line function, against a 50-line threshold. The charge is “God Function”. The detector is really large-or-deeply-nested function, but the prose and the leading evidence line are both written for the size case only.

repo	size-triggered (N > T)	nesting-triggered (N ≤ T)
pydantic	148 (59%)	101 (41%)
drf	24 (51%)	23 (49%)

the nesting threshold is JS-calibrated and Python pays for it. Probing maxNestingDepth with matched synthetic files:

source	visual indent levels	reported depth
`def` → `for` → `if` → stmt	3	2
`def` → `for` → `try/except` → stmt	3	3
`def` → `with` → stmt	2	1
`def` → `if` → `if` → `if` → stmt	4	3
`def` → `for` → `with` → `try/except` → stmt	4	4: trips at 7 lines

try/except costs one more level than if/else at identical indentation, because the except handler is counted as its own level. Python is an EAFP language: try/except is the idiomatic guard where JS would use if. Add with (context managers, ubiquitous) and idiomatic Python reaches depth 4 a full indent level sooner than equivalent JS. The concrete case in drf, rest_framework/fields.py:86-110 get_attribute (a 25-line module-level function that does for / try / if) is charged “God Function” at depth 4. There is nothing unusual about it.

why it matters: this is the highest-volume detector in the round by a factor of 3, on two of the most-reviewed Python codebases in existence, and roughly 4 in 10 of its findings are produced by a nesting rule that Python’s own idioms inflate.
fix shape: per-language nesting threshold, and/or stop counting except and with as full levels; and fix the summary to say “nests N levels deep” rather than “at this size” when size was not the trigger.
severity: real problem

`todo_density` fires below its own stated threshold in 13 of 25 findings

repos: pydantic 24, drf 1
what happened: pydantic/main.py is charged “Unfinished Business” with evidence ["3× TODO", "1.6 markers per 1k LOC (threshold 10)"]. The finding states a measurement 6× under the threshold it cites, and fires anyway. 13 of the 25 todo_density findings across both repos are below their own quoted threshold.
root cause: packages/core/src/detectors/todo-density.ts:40: if (matches.length < 3 && density < threshold) return [];: it is an OR: ≥3 markers or density above threshold. The evidence renderer at line 67 only ever emits the density clause, so the absolute-count trigger is invisible and the finding reads as self-contradictory.
why it matters: a reader’s first move is to check the number against the threshold, see it is 6× under, and conclude the tool is broken. It isn’t: it just never says why it fired. Also: the detector is named todo_density and half its output is not about density.
fix shape: emit the clause that actually fired: "3 markers (floor: 3)": when the count trigger is what caught it. Patch-level, evidence text only.
severity: real problem (cheap to fix, disproportionately corrosive)

Human `scan` output buries the lede: 23 consecutive identical charges

repo: pydantic
command: crimes scan <repo> --no-init
what happened: 131 lines of output. The default view shows the top 5 files but prints every finding within each: _generate_schema.py gets 29 numbered lines, of which items 2 through 23 are all “God Function”, each a near-identical sentence differing only in name/line-count. The genuinely distinct signals in that file (Boolean Naming Drift, 4× Commented-Out Code, Unfinished Business) are items 24–29, below the fold.
also: the per-file footer prints id=crime_00047 … crime_00345: a range spanning ids that mostly belong to other files. It is not a usable handle for anything in the list, and given the context/explain id mismatch above, ids in the human view are doubly untrustworthy.
also: the header says 578 findings across 223 files but never says how many files were scanned (637). That number is the one that would have caught the broken clone.
why it matters: “Signal over exhaustiveness” is a stated product constraint. Top-5-files-but-all-their-findings is exhaustive within a truncated set: the worst of both.
severity: real problem

`commented_out_code`: ~90% of a random sample is English prose, incl. Rust doc comments

repos: pydantic 77, drf 2. The 38× asymmetry is itself the clue.
sampling: deterministic: every 8th of the 77 pydantic findings (n=10). 9 of 10 are explanatory prose comments, not commented-out code. Examples:
- pydantic/fields.py:545-558: “HACK: It is common for users to define ‘make model partial’ (or similar) utilities, that convert all model fields to be optional…” (14 lines of English)
- pydantic/_internal/_generate_schema.py:470-475: six lines explaining why an isinstance check is used for empty enums.
- pydantic-core/src/definitions.rs:1-5: a Rust /// doc comment. ”/// Definition / reference management…”. Rustdoc is the language’s documentation mechanism; flagging it as commented-out code is a category error, and it comes from the universal pack applying a JS/Python comment heuristic to .rs.
- The single code-looking hit is tests/mypy/outputs/.../plugin_fail_baseConfig.py: # MYPY: error: … expected-output fixture data.
mechanism: the stated rule is ”>=3 lines contain code-shaped tokens (def/fn/function/class/let/var/const/etc.)”. pydantic’s domain vocabulary is those tokens: its prose is unavoidably about class, def, type, field, subclass, dataclass. drf’s prose is about serializers and viewsets, so it scores 2. The detector is measuring how much a project’s comments talk about language constructs, not how much dead code they contain.
why it matters: 79 findings, third-highest volume in the round, on material that is the opposite of a crime: these are the comments a maintainer wrote so the next reader would not have to guess. The context “Agent guidance” then instructs: “Do not copy disabled code from comments; verify whether it should be deleted or explained as rationale”: pointing an agent at well-written rationale and suggesting deletion.
fix shape: require code structure (balanced delimiters, assignment or call syntax, trailing : / ;) rather than keyword presence; exclude /// and /** */ doc-comment forms; consider gating on the file’s language.
severity: real problem

`circular_dependency.py` cannot distinguish `TYPE_CHECKING` imports from runtime ones

repo: pydantic (3 findings, incl. a 46-module cycle rooted at pydantic/__init__.py, severity high, confidence 0.95)
what I verified: parsing this file with parsePyFile
```
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from b import B
def f(x: "B"): return x
```
yields two import records, and the from b import B record is indistinguishable from a module-level one: no typeOnly / typeChecking / guard flag on PyImport. So the import graph the cycle detector consumes cannot exclude type-only edges.
why it matters: a TYPE_CHECKING-guarded import never executes. The detector’s entire stated rationale is runtime behaviour: “Python executes a module top-to-bottom on first import… whichever module is imported first decides whether this raises, so it can pass under pytest and fail under a production server”. For a type-only edge that outcome is impossible by construction. 54 of 139 files in pydantic/ contain if TYPE_CHECKING: blocks, so the exposure is large, and modern typed Python (which is precisely what pydantic is) uses this pattern specifically to break import cycles. The tool may be reporting the cycle a maintainer already fixed.
what I did NOT verify: whether pydantic’s specific 46-module SCC survives removal of type-only edges. Proving that needs the graph rebuilt both ways. Flagging as unresolved rather than asserting a false positive.
severity: real problem (unconfirmed on this instance, but the detector provably cannot tell the difference)

hono

Friction log: hono (CONTROL)

repo: ~/crimes-dogfood/corpus/hono
SHA: 51db3131d5e97076327edaa0afdb60ebb77c264f
build: crimes 0.17.0
role: CONTROL. Well-maintained, heavily-reviewed TS web framework. Few findings is the correct answer.

Size baseline (pre-scan)

482 tracked files
.ts 324 files / 65,989 lines; .tsx 32 / 14,947; .mts 27; .mjs 2 / 106; .js 1 / 37
src/ alone: 308 files, 61,879 lines of .ts
.md 9 files / 677 lines total <- NOTE: no markdown file anywhere near the 1000-line docs threshold
.json 34 / 1,911; .yml 13 / 610

Timings / robustness

run	wall	user
`scan --all --format json`	34.55s	9.01s
`scan` (human)	19.17s	8.72s
`hotspots`	39.37s	9.15s
`verdict`	55.23s	15.68s
`diff HEAD~30...HEAD`	52.74s	15.48s
p-limit `scan --all --format json`	1.32s	0.70s

stderr empty on every run. No stack traces. No hangs.

coverage for hono: files_total 390, files_by_language {js: 381}, files_universal_only 9 (.md 8, .sh 1), packs_loaded ["universal","language-js","language-py"], by_package 5 entries (all five benchmarks/* workspaces, all dominant_language: js).

THE CONTROL NUMBER: 377 findings on hono

repo: hono
what happened: 377 findings: 22 high, 205 medium, 150 low: across 152 files. 81,962 lines of scanned JS/TS → 4.6 findings per 1000 lines. 27,658 non-test lines → 13.6 per 1000 non-test lines.
the distribution is top-heavy in a way that matters: 3 detectors produce 65% of the total. large_function 143 (38%), weak_test_signal 69 (18%), high_fan_in_fan_out 34 (9%). Add large_file 26 and boolean_naming_drift 24 and 5 detectors are 84% of output.
why it matters: hono is one of the most-reviewed TypeScript codebases on npm. 377 is not a quiet result, and the shape of it says the volume comes from a handful of size/count detectors rather than from a broad true signal.
severity: real problem

186 of 377 findings (49%) are in test files

repo: hono
what happened: the default human view’s last section reads Also flagged elsewhere / tests/ 186 findings / Run with --all to see them. tier split is 191 domain / 186 nonDomain.
why it matters: half the output is about test code. The default view does bucket it away, which is the right instinct: but it is still half of the number a user sees in the header line (“377 findings”), and all 377 are in the JSON with no --exclude-tests flag and no tier filter on the CLI.
severity: real problem

`--all` vs default: 35 of 377 shown (9.3%): the constraint holds on hono, not on p-limit

repo: both
what happened: hono default human view lists 35 numbered findings across 5 of 85 files, then Showing 5 of 85 files. Run with --all for every finding. p-limit default human view lists 9 of 10.
why it matters: the “signal over exhaustiveness” constraint is implemented as scan.topFiles: 5, i.e. a file cap, not a finding cap. On a big repo that happens to compress 10×; on a small repo it does nothing at all. The gap is an accident of repo size, not a designed budget.
also: --all does not affect --format json at all (byte-identical output).
severity: real problem

Finding ordering: and therefore `crime_NNNNN` ids: is NOT reproducible across runs

repo: hono
command: crimes scan <hono> --all --format json --no-init, run twice, same SHA
what happened: the finding multiset is identical (377, same severity split, identical coverage). The order is not, and ids are assigned by position, so:
- run A crime_00024 = large_file on src/middleware/secure-headers/secure-headers.ts
- run B crime_00024 = hardcoded_localhost on benchmarks/http-server/benchmark.ts 5 ids changed identity between two back-to-back runs. First order divergence at index 23. Likely cause: the recency score is wall-clock derived and drifted 0.71 → 0.70 between runs, flipping a sort tie. p-limit was byte-identical across two runs (too small to hit a tie).
why it matters: crimes explain crime_00024 is the documented workflow and it resolves to a different finding on a second run of an unchanged repo. The human scan output prints id=crime_00019 … crime_00153 ranges for the user to feed back in. Any ticket, PR comment, or agent transcript quoting a crime_ id is stale the moment anyone re-runs.
severity: blocker

Fingerprints are not unique: 97 collisions on hono

repo: hono
what happened: the stable fingerprint is <type>::<file>::<symbol>. hono’s 377 findings reduce to 280 unique fingerprints: 97 collisions. Worst offenders: 9× large_function::src/types.test.ts::describe callback, 8× weak_test_signal::runtime-tests/deno-jsx/jsx.test.tsx:: (empty symbol), 8× weak_test_signal::src/jsx/dom/intrinsic-element/components.test.tsx::.
consequence 1: crimes diff HEAD~30...HEAD reports New 4 / Fixed 12 / Unchanged 284 = 288 findings accounted for, against a scan total of 377. 89 findings (24%) are invisible to diff because they collapse into a colliding fingerprint.
consequence 2: crimes ignore <fingerprint> would silently suppress up to 9 findings.
consequence 3: crimes explain <fingerprint> is ambiguous by construction.
severity: blocker

`hotspots` ranks hono’s `package.json` as the #1 risk in the repo

repo: hono
command: crimes hotspots --no-init
what happened: 1. package.json risk 0.72 (72%) · 29 changes · latest 2026-07-31 · 1 finding · highest low. The #2 entry is at 0.58. Nothing else is close.
why it matters: package.json churns 29× in 90 days because of version bumps and dependency updates. The risk model reads churn as danger with no notion that a manifest’s churn is mechanical. The single most prominent line of output in the whole command is pointing a maintainer at their version field.
also: hotspots ranks files that scan never scanned: package.json, readme.md, index.d.ts all appear in p-limit’s hotspots list at risk 0.03 with 0 findings, while coverage.files_total for p-limit is 8 and does not include package.json. Two commands, two different file universes.
severity: real problem

`hotspots` on p-limit lists every file including 0-finding ones

repo: p-limit
what happened: repo: p-limit · since 90d · 7 files, and it lists all 7: including three entries with 0 findings · highest none at risk 0.03 ( 3%).
why it matters: on a clean repo this is the graceful-degradation case, and it half works: the numbers are honestly low (max 27%) and it does not manufacture urgency. But it still pads the list with rows whose only content is “nothing here”. There is no “nothing is hot in this repo” statement: the user has to infer it from the percentages.
severity: annoyance

`verdict` degrades correctly on both repos

repo: both
what happened: Verdict: UNCHANGED / New: 0 / Fixed: 0 / Reason: no new findings and no fixed findings / Recommended next action: no maintainability change vs. base: verdict is advisory only. Exit 0.
why it matters: this is the right answer and the right tone. Logged as a positive. Caveat: it took 55s on hono to say “nothing changed”, because it does a full double scan.
severity: annoyance (the 55s only)

`crimes explain` hard-fails with exit 2 on a finding type the scanner emits

repo: hono
command: crimes explain crime_00369 --from hono.json --no-init
what happened: crimes: no detector registered for finding.type "oversized_raster". Exit code 2. crime_00369 is a real finding in the report (docs/images/hono-logo.png, 436.2 KB, low).
why it matters: scan can produce a finding that explain refuses to explain. The asset-pack detectors are not in whatever registry explain consults. Any agent looping explain over every id in a report will hit a non-zero exit on a valid input.
severity: real problem

`unsafe_retry` HIGH on a virtual-DOM reconciler, reasoning about “database write”

repo: hono
finding: crime_00045: src/jsx/dom/render.ts:497-665, high, conf 0.68
what happened: evidence reads retry construct: build() calls itself from its catch block, retried mutation: update at line 625 (name suggests a write), and boundary: database write (matched on 'update'): a duplicate here is visible to the customer, not just to the logs.
why it matters: not judging correctness: but the evidence asserts a database write in a JSX DOM renderer, on the strength of the identifier update. This is the highest- severity finding in the file that the tool itself nominates as the repo’s top risk, and its supporting sentence is about a database that does not exist in this package.
severity: real problem

`accessible_interaction_risk` says a test file “appears to ship” UI

repo: hono
finding: crime_00304: src/jsx/dom/index.test.tsx, medium, tier: nonDomain
what happened: summary: “src/jsx/dom/index.test.tsx appears to ship 6 clickable non-button elements without accessibility metadata. Keyboard and screen-reader users will not be able to…”
why it matters: the detector already knows the file is nonDomain: the tier is right there on the finding: and still emits a medium-severity claim about shipped user-facing UI for a test fixture. The word “ship” is doing real work in that sentence and it is false.
severity: real problem

`contract_drift` summary names the same type twice

repo: hono
finding: crime_00262: src/helper/css/index.ts:217-223, medium
what happened: “DefaultContextType and DefaultContextType appear to describe the same record but disagree on 1 field.”
why it matters: the summary is unreadable without opening the file: the protocol explicitly asks for this. Two same-named types from different modules need qualifying.
severity: annoyance

`explain`’s “test gap” gloss states a fact that is false

repo: p-limit (also fires on hono)
what happened: every p-limit explain renders test gap: 0.50: a test file matches this module's name but does not import it. For index.js: no test file matched the name (crimes context index.js says so in the same breath: “no test file matched the target basename”), and test.js:8 is import pLimit, {limitFunction} from './index.js';: it does import it. So the two commands contradict each other, and the surviving sentence is wrong on both clauses.
why it matters: 0.50 is the no-signal midpoint, but it is rendered with a confident causal explanation instead of “no signal”. That is worse than printing nothing.
severity: real problem

Suggested `crimes ignore` commands are not shell-pasteable

repo: both
what happened: explain closes with a copy-paste block:
- crimes ignore swallowed_error::index.js::run → catch_clause --reason "…": unquoted fingerprint containing a space and a U+2192 arrow; the shell splits it.
- crimes ignore unbounded_async_fanout::benchmark.js::<module> → writeFile --reason "…": same, plus < and > which the shell treats as redirections.
- crimes ignore boolean_naming_drift::index.js:: --reason "…": trailing empty symbol.
why it matters: the one command the tool tells you to run is broken as printed for any finding whose symbol has a space, an arrow, or angle brackets: i.e. most of them.
severity: real problem

Zero case: CONFIRMED clean on hono

repo: hono
what happened: cross_language_* detectors: 0 findings (no detector_id containing cross_language). Python-pack detectors: 0 findings (pack == "language-py" count = 0). Correct for a TS-only repo.
BUT coverage.packs_loaded still lists language-py on a repo with zero .py files, and coverage.files_by_language correctly reports only {js: 381}. packs_loaded and files_by_language disagree and only the latter is informative.
severity: annoyance

`.json` / `.yaml` are never scanned at default config

repo: hono
what happened: hono’s 735-line benchmarks/query-param/package-lock.json and 698-line package.json produce no large_file finding. DEFAULT_SOURCE_INCLUDES has no json/yaml/yml/txt entry. Verified against a synthetic probe in the p-limit clone: with an include override, a 1201-line .json fires large_file high at 4.0× the configured 300-line threshold: so the intended behaviour is implemented and simply unreachable. See p-limit-friction.md for the full probe.
severity: real problem

Sandbox artefact (environment, not crimes)

command: first /usr/bin/time -p crimes scan <hono> --all --format json under the agent sandbox was SIGTERMed at 17.5s → exit 143, leaving a 0-byte hono.json. Re-ran unsandboxed: exit 0 in 34.55s. Recording it because three pre-existing artefacts in ~/crimes-dogfood/artifacts/ (airflow.json, zulip.json, ortomate-studio.json) are 0 bytes with 0-byte .time files: same signature. Anyone reading those as “crimes found nothing” would be wrong.
severity: annoyance (for this round: real problem for whoever inherits those files)

VERBATIM: default `crimes scan` on hono (no `--all`)

CRIME SCENE REPORT
repo: hono  ·  377 findings across 152 files  ·  22 high  205 medium  150 low

Top files by risk
src/jsx/dom/render.ts  11 findings · 3 high, 5 medium, 3 low
   1. God Function · build()   lines 497–665 (169 lines), 2.8× the domain function threshold (60 lines)
   2. God File   826 non-empty lines, 2.8× the configured 300-line threshold
   3. Double Jeopardy · <module> → update   retry construct: build() calls itself from its catch block (line 497),   retried mutation — update at line 625 (name suggests a write)
   4. God Function · applyProps()   lines 164–272 (109 lines), 1.8× the domain function threshold (60 lines)
   5. False Identity · buildNode()   name prefix suggests a pure transformation, side-effect-like calls: createContext
   6. False Identity · renderNode()   name prefix suggests a read, side-effect-like calls: createDocumentFragment
   7. God Function · applyNodeObject()   lines 387–481 (95 lines), 1.6× the domain function threshold (60 lines)
   8. Circular Dependency   cycle size: 2 files, member: src/jsx/dom/render.ts
   9. Catch and Release · updateSync → build   protected operation: build(context, node, undefined), calls inside the protected region: build
   10. High Fan-In / Fan-Out   fan-in: 12 importers (p95 cutoff: 11, p99: 29), fan-out: 9 imports (p95 cutoff: 9, p99: 10)
   11. Plural Mismatch   plural name, singular type: `children: Child` @L815, v1 detector — type aliases and generic types are silently skipped
     Risk: churn low · test gap bottom-quartile · blast top-quartile (48 importers)
     id=crime_00019 … crime_00153

benchmarks/http-server/benchmark.ts  7 findings · 1 high, 3 medium, 3 low
   1. Sync I/O in Hot Path   `existsSync` @L86 in `setupTemp` (domain function), `rmSync` @L87 in `setupTemp` (domain function)
   2. Dev-Server URL   `127.0.0.1:3000` @L137, `127.0.0.1:3000` @L142
   3. False Identity · buildVersion()   name prefix suggests a pure transformation, side-effect-like calls: writeFileSync, fetch
   4. God Function · main()   lines 234–324 (91 lines), 1.5× the domain function threshold (60 lines)
   5. God Function · buildVersion()   lines 93–178 (86 lines), 1.4× the domain function threshold (60 lines)
   6. God File   327 non-empty lines, 1.1× the configured 300-line threshold
   7. Catch and Release · buildVersion → runCommand   protected operation: const stashResult = await runCommand('git stash push -m "benchmark-temp"', HONO_ROOT) (+2 more statement(s)), calls inside the protected region: runCommand, stashResult.stdout.includes
     Risk: churn low · test gap top-quartile · blast bottom-quartile (0 importers)
     id=crime_00020 … crime_00138

src/middleware/jwk/jwk.ts  6 findings · 1 high, 4 medium, 1 low
   1. God Function · jwk()   lines 48–168 (121 lines), 2.0× the domain function threshold (60 lines)
   2. Option Bag Junk Drawer · jwk()   parameter: options, 7 distinct property reads: alg, allow_anon, cookie, headerName, jwks_uri, keys, verification
   3. God Function · <anonymous>   lines 75–167 (93 lines), 1.2× the function threshold (80 lines)
   4. Exact Duplicate Block   hash c56d4897f096… across 2 file(s), 2 occurrence(s), src/middleware/jwk/jwk.ts:170-183 (unauthorizedResponse)
   5. Policy Doppelgänger · authorization "Authorization" rule   normalised rule: ("Authorization" || .headerName), 2 occurrence(s) across 2 file(s)
   6. High Fan-In / Fan-Out   fan-out: 10 imports (p95 cutoff: 9, p99: 10)
     Risk: churn low · test gap top-quartile · blast bottom-quartile (1 importer)
     id=crime_00038 … crime_00090

src/utils/url.ts  5 findings · 2 medium, 3 low
   1. Commented-Out Corpse   3 comment lines, code-like tokens:
   2. God Function · _getQueryParam()   lines 219–300 (82 lines), 1.4× the domain function threshold (60 lines)
   3. God File   320 non-empty lines, 1.1× the configured 300-line threshold
   4. High Fan-In / Fan-Out   fan-in: 12 importers (p95 cutoff: 11, p99: 29)
   5. Unprefixed Boolean   `multiple` @L222, lines: 222
     Risk: churn low · test gap bottom-quartile · blast top-quartile (214 importers)
     id=crime_00003 … crime_00053

src/jsx/components.ts  6 findings · 1 high, 4 medium, 1 low
   1. God Function · ErrorBoundary()   lines 61–261 (201 lines), 3.4× the domain function threshold (60 lines)
   2. God Function · <anonymous>   lines 165–256 (92 lines), 1.1× the function threshold (80 lines)
   3. God Function · <anonymous>   lines 170–254 (85 lines), 1.1× the function threshold (80 lines)
   4. Circular Dependency   cycle size: 2 files, member: src/jsx/components.ts
   5. High Fan-In / Fan-Out   fan-out: 10 imports (p95 cutoff: 9, p99: 10)
   6. Unprefixed Boolean   `caught` @L135, lines: 135
     Risk: churn low · test gap bottom-quartile · blast top-quartile (48 importers)
     id=crime_00013 … crime_00148

Showing 5 of 85 files. Run with --all for every finding.

Also flagged elsewhere
  tests/  186 findings
  Run with --all to see them.

→ Start with `crimes context src/jsx/dom/render.ts` — it concentrates the most risk in this scan.

Honest first-time-user reaction: hono

repo: hono
command: crimes scan ~/crimes-dogfood/corpus/hono --no-init
what happened / my reaction: mixed, leaning toward distrust: but for different reasons than p-limit, and it is closer to salvageable.

What builds trust here. The default view is good. It does not dump 377 findings; it shows 5 files, buckets 186 test findings out of the way, and closes with one concrete next action. The per-file rollups (Risk: churn low · test gap bottom-quartile · blast top-quartile (48 importers)) are the kind of thing no linter gives you, and src/jsx/dom/render.ts genuinely is the scariest file in hono: 826 lines, a 169-line build(), a self-recursive catch-block retry, in a circular dependency, imported by 48 files. If the tool had shown me only that file I would have been impressed. verdict said UNCHANGED without inventing drama. Every evidence block shows its arithmetic.

What costs trust.
1. The header number is 377. Whatever the default view does downstream, the first thing on screen is a four-digit-adjacent count for a codebase that is famously tidy. And 84% of it comes from five detectors that mostly count lines and imports.
2. hotspots puts package.json first, at 72%, ~14 points clear of anything real. That is the answer to “what should I worry about” and it is the version field.
3. The high-severity findings do not hold up under their own evidence. The single unsafe_retry HIGH on render.ts argues from boundary: database write (matched on 'update') inside a virtual-DOM reconciler. accessible_interaction_risk says a .test.tsx file “appears to ship” clickable elements while carrying tier: nonDomain on the same object. Dev-Server URL HIGH is 127.0.0.1:3000 in a benchmark script.
4. Plural Mismatch volunteers its own unreliability in the output: plural name, singular type: children: Child @L815, v1 detector: type aliases and generic types are silently skipped. Shipping a caveat about detector immaturity inside the finding text is honest, but it reads as “we know this one is rough”.
5. Two runs, same SHA, different ids. Once I noticed that crime_00024 names a different finding on a re-run, everything in the report that references an id became untrustworthy to quote.
Net. I would keep using crimes context <file> before an edit: that command is genuinely good and it is the product’s actual wedge. I would not put crimes scan’s headline number in front of a team, and I would not wire hotspots into anything. The tool’s problem on hono is not that it is wrong about everything; it is that the loudest outputs (total count, hotspot #1, the HIGH findings) are the least reliable ones, and the quiet outputs (per-file context, evidence blocks, verdict) are the good ones. That ranking is exactly backwards from what earns trust on first contact.
severity: real problem

p-limit

Friction log: p-limit (CONTROL / null hypothesis)

repo: ~/crimes-dogfood/corpus/p-limit
SHA: df476048d023ff868cd45b35ee47f5fb0ca2b25a
build: crimes 0.17.0
role: THE null hypothesis. A few hundred lines of exemplary sindresorhus code. Honest expected total is ~0.

Size baseline (pre-scan)

16 tracked files, 1,637 lines total including docs and license.

 12 .editorconfig      234 benchmark.js
  1 .gitattributes     136 index.d.ts
  3 .github/security.md 127 index.js
 22 .github/workflows/main.yml  33 index.test-d.ts
  2 .gitignore           9 license
  1 .npmrc              58 package.json
                       173 readme.md
                       213 recipes.md
                       236 scripts/benchmarker.js
                       377 test.js

Source proper (index.js + index.d.ts) = 263 lines. Everything else is tests, benchmarks, docs, config.

No markdown file over 1000 lines (largest is recipes.md at 213), so the 0.17.0 docs shape cannot be exercised here on its own.

THE CONTROL NUMBER: crimes finds 10 findings in a 263-line library

repo: p-limit
command: crimes scan ~/crimes-dogfood/corpus/p-limit --all --format json --no-init
what happened: 10 findings. 0 high, 6 medium, 4 low. Across 4 of 8 scanned files. Scanned universe is 8 files (6 js + 2 md); index.js + index.d.ts = 263 lines. 10 findings / 1,144 lines of scanned JS = 8.7 per 1000 lines. 3 of the 10 are in index.js itself (127 lines) = 23.6 per 1000 lines of the actual library.
why it matters: p-limit is the null hypothesis. It has ~1.5 billion downloads/wk of scrutiny on it. Zero-to-two would have been a strong result. Ten is not quiet.
severity: real problem

Default `crimes scan` (no `--all`) shows 9 of the 10 findings on p-limit

repo: p-limit
command: crimes scan ~/crimes-dogfood/corpus/p-limit --no-init
what happened: the default human view prints 9 findings across 3 files and then says “Also flagged elsewhere / scripts/ 1 finding / Run with —all to see them.” So the “signal over exhaustiveness” default suppresses exactly one finding here.
why it matters: the PRD constraint is “default crimes scan shows top findings only”. On a small repo the default view is effectively --all. A first-time user running npx crimes scan on a tiny clean library sees a wall of nine charges.
severity: real problem

`--all` has NO effect on `--format json`: the JSON is byte-identical

repo: both
command: crimes scan $R --format json --no-init vs ... --all --format json --no-init
what happened: diff of the two JSON outputs is empty for p-limit and byte-identical for hono (529,405 bytes both ways).
why it matters: --all is documented and rendered as a scope flag (“Run with —all to see them”), but it is purely a human-renderer flag. Any consumer of the JSON contract who assumes --all widens the result set is wrong, and anyone measuring “the —all vs default gap” from JSON will measure zero. Not necessarily a bug, but it is undocumented in the output and it makes the two surfaces disagree about what the word “all” means.
severity: real problem

`test.js` at repo root is NOT recognised as a test file

repo: p-limit
finding: crime_00006: God File, test.js, 378 lines, medium, tier: "domain"
what happened: charged at 1.3× the configured 300-line threshold: i.e. the domain budget, not the 1500-line test_file budget. TEST_FILE_RE in packages/core/src/util/test-files.ts requires *.test.js / *_test.js / __tests__/ / tests/. A bare top-level test.js matches none of them. Same classifier fed tier: all 3 test.js findings are tier: "domain".
why it matters: test.js is the single most conventional test filename in the npm ecosystem (it is the ava/node:test default and what sindresorhus uses in ~every package). Missing it means every sindresorhus-shaped repo gets its test suite charged as production code, on both the large_file shape and the domain/nonDomain tier.
severity: real problem

`swallowed_error` fires on a `catch {}` whose reason is written on the line above it

repo: p-limit
finding: crime_00003: index.js:44, medium, confidence 0.86
what happened: the evidence says “the error is never inspected”, “no rethrow … no logging”. index.js:39-41 is a three-line comment reading: “Wait for the function to complete (success or failure) / We catch errors here to prevent unhandled rejections, / but the original promise rejection is preserved for the caller”. The detector reads the handler body but not the comment immediately above the protected region.
why it matters: not judging correctness here: but the evidence text asserts an absence (“there is no record the operation failed”) that a reader can disprove by looking three lines up. Evidence that overstates is worse than a bare flag.
severity: real problem

`boolean_naming_drift` flags a public API option name

repo: p-limit
finding: crime_00008: `rejectOnClear` @L4, low
what happened: rejectOnClear is p-limit’s documented public option (pLimit({concurrency, rejectOnClear})). The suggested action is “Rename to is*/has*”.
why it matters: the suggested fix is a breaking semver-major API change, offered at effort: "quick". The detector has no notion of “this identifier crosses the package boundary”.
severity: annoyance

`test_gap: 0.5` on every p-limit finding, including `index.js`

repo: p-limit
what happened: all 10 findings carry test_gap: 0.5 and the human view renders “test gap ~median” for every file including index.js.
why it matters: p-limit has a 377-line test.js covering index.js. 0.5 is the no-signal default. Because test.js is not matched as a test file (above), the test-gap scorer sees a repo with zero tests and falls back to the midpoint everywhere. So one naming-convention miss silently degrades a whole score dimension into a constant.
severity: real problem

0.17.0 `docs` shape: WORKS: but it is not visible in the `tier` field

repo: p-limit (synthetic probe, written into the disposable clone then reverted)
command: wrote DOCS_PROBE_1200.md (1279 lines) / DOCS_PROBE_2500.md (2557 lines), rescanned, then rm + git status --porcelain clean at df476048d023ff868cd45b35ee47f5fb0ca2b25a.
what happened: exactly as specified.
- 1279 lines → severity: low, conf 0.74, “Document is 1279 lines (document threshold 1000)”, evidence 1.3× the configured 1000-line document threshold + shape: document (matches .md/.mdx/.markdown/.rst/.adoc/.txt), agent_risk 0.13.
- 2557 lines → severity: medium, conf 0.93, 2.6×, agent_risk 0.22.
- fix_shape correctly switches to “split into per-topic pages; one subject per document”.
- BUT tier is "domain" on both. tier is the scope tier (domain/nonDomain) from scoring/tier.ts, a completely different axis. The large_file shape (domain/test_file/docs) is exposed only as an English sentence inside the evidence array. A JSON consumer cannot filter or group by shape without string-matching prose.
why it matters: the JSON is the product contract. A behavioural tier that only exists in a human-readable evidence string is not in the contract.
severity: real problem

The `.json`/`.yaml` “deliberate exclusion” is unreachable at default config

repo: p-limit (synthetic probe)
what happened: large-file.ts comments that “.json, .yaml, and friends are data, and a 3000-line config is a finding worth keeping”. A 1201-line DOCS_PROBE.json and a 1201-line DOCS_PROBE.yaml dropped into the repo produced zero findings at default config. coverage.universal_only_by_extension listed only .md. With include overridden to add {json,yaml,yml,txt}, both immediately fire large_file high, 4.0× the configured 300-line threshold: so the shape logic is correct. The files simply never reach the detector: DEFAULT_SOURCE_INCLUDES in packages/core/src/config.ts covers {ts,tsx,js,jsx,mjs,cjs,cts,mts}, {py,pyi}, a compiled-language list, and {md,mdx}: no json, no yaml, no yml. Real-world confirmation: hono’s 735-line benchmarks/query-param/package-lock.json and 698-line package.json produced no large_file finding.
also: .txt, .rst, .adoc, .markdown are in DOCS_EXT_RE but not in DEFAULT_SOURCE_INCLUDES either: a 1279-line .txt produced nothing at default config, and fired low / docs-shape once included. So 5 of the 7 extensions the docs shape claims to handle are dead code at default config.
why it matters: documented intent and shipped behaviour disagree. The docs shape as shipped applies to .md/.mdx only.
severity: real problem

Zero case: CONFIRMED clean

repo: p-limit
what happened: cross_language_* detectors: 0 findings. Python-pack detectors: 0 findings (pack == "language-py" count = 0; no *.py detector_id). Correct: the repo has no Python and one language.
BUT: coverage.packs_loaded = ["universal", "language-js", "language-py"]. language-py is reported as loaded on a repo with zero .py files. The protocol already warns that packs_loaded reports what was loaded, not what ran; this confirms it is uninformative as a health signal.
severity: annoyance

VERBATIM: default `crimes scan` on p-limit (no `--all`)

CRIME SCENE REPORT
repo: p-limit  ·  10 findings across 4 files  ·  0 high  6 medium  4 low

Top files by risk
index.js  3 findings · 2 medium, 1 low
   1. God Function · pLimit()   lines 3–115 (113 lines), 1.9× the domain function threshold (60 lines)
   2. Catch and Release · run → catch_clause   protected operation: await result;, handler kind: try/catch (line 44)
   3. Unprefixed Boolean   `rejectOnClear` @L4, lines: 4
     Risk: churn low · test gap ~median · blast bottom-quartile (3 importers)
     id=crime_00001 … crime_00008

benchmark.js  3 findings · 2 medium, 1 low
   1. Return Shape Roulette · <anonymous>   3 object-literal return shapes, lowest key overlap: 25%
   2. Temporal Recklessness   4× Date.now(), lines: 32, 59, 74, 140
   3. Concurrency Stampede · <module> → writeFile   fan-out: Promise.all at line 64, collection: fileData (← Array.from({length: 100}, (_, i) => ({ name: `data-${i}.jso…).map(…)
     Risk: churn low · test gap ~median · blast bottom-quartile (0 importers)
     id=crime_00002 … crime_00007

test.js  3 findings · 2 medium, 1 low
   1. Catch and Release · <module> → all   protected operation: await Promise.all(promises);, calls inside the protected region: Promise.all
   2. God File   378 non-empty lines, 1.3× the configured 300-line threshold
   3. Unprefixed Boolean   `ran` @L69, lines: 69
     Risk: churn low · test gap ~median · blast bottom-quartile (0 importers)
     id=crime_00004 … crime_00009

Also flagged elsewhere
  scripts/  1 finding
  Run with --all to see them.

→ Start with `crimes context index.js` — it concentrates the most risk in this scan.

Honest first-time-user reaction: p-limit

repo: p-limit
command: crimes scan ~/crimes-dogfood/corpus/p-limit --no-init
what happened / my reaction: I would distrust the tool.

I know this repo. It is 127 lines of library code that half the JavaScript ecosystem depends on, written and re-reviewed by one of the most careful maintainers in open source. The tool opens with CRIME SCENE REPORT and charges it with nine crimes, in a file-grouped layout that reads as a rap sheet.

The specific things that cost trust, in order:
1. “God Function · pLimit() lines 3–115 (113 lines), 1.9× the domain function threshold”: pLimit is a closure factory. Lines 3–115 are the whole module: the nested resumeNext/next/run/enqueue/generator arrows and one Object.defineProperties block. The detector is measuring the file and calling it a function. This is the first line of the report and it is the weakest one.
2. “God File · test.js · 378 non-empty lines, 1.3× the configured 300-line threshold”: a test file charged on the production budget because the filename is test.js and not something.test.js. Both the shape and the tier got it wrong.
3. “Catch and Release · index.js:44” with the error is never inspected: three lines above the flagged catch is a comment explaining exactly why the catch is empty. The evidence asserts an absence that the file itself refutes.
4. Three of the nine are in benchmark.js: a benchmark harness: charged with Temporal Recklessness for calling Date.now() (that is what a benchmark does) and Concurrency Stampede for Promise.all over Array.from({length: 100}), where the evidence says “N is a property of the data, not of this code” about a literal 100 three tokens away, and simultaneously claims the collection is “an awaited call (query, request, or listing)” while also saying “produced by: Array.from”. Two clauses of the same evidence block contradict each other.
5. Every file shows test gap ~median including index.js, which has a 377-line suite. And “Likely tests: (no test file matched…)” in crimes context index.js, for a package whose test file is sitting in the same directory importing it.
6. The closing nudge: → Start with crimes context index.js (it concentrates the most risk in this scan) points at the library’s entire public implementation.
What would keep me at the table: the evidence blocks are genuinely well built. Every finding shows its confidence arithmetic (0.86 = 0.72 base + 0.14 (handler body is empty)), line numbers, and a named fix shape. Nothing is a bare assertion. verdict correctly said UNCHANGED. The tool is honest about its reasoning: it is just reasoning from bad premises here.

But the headline is: on the cleanest small library I can think of, the default view fires nine charges and the very first one is wrong. A first-time user runs npx crimes scan on a repo they know well precisely to calibrate the tool. p-limit is that repo for a lot of people, and it fails the calibration.
severity: blocker (for adoption, not for correctness)

thickeningtime

Friction log: thickeningtime

Repo: ~/dev/thickeningtime (Next.js app, TS/TSX, 324 commits) SHA: baf0b22bbfc991b64d4838b556977ddd5c7a557b Binary: ~/crimes-dogfood/bin/crimes 0.17.0 Started: 2026-08-02 Focus: command surface beyond scan

`context` ids and `explain` ids are different namespaces: silent wrong answer

repo: thickeningtime
command: crimes context lib/pdf-generator.ts --no-init then crimes explain crime_00001 --no-init
what happened: context prints id=crime_00001 next to generatePersonalizedPrintPdf in lib/pdf-generator.ts. Feeding that exact id to explain returns Sync I/O in Hot Path in lib/chapters.ts: a different finding, different file, different detector. Exit 0, no warning. context renumbers findings 1..N within the file it was given; scan numbers them across the whole repo (the same finding is crime_00004 in scan).
why it matters: context → explain is the documented agent workflow and the tip printed by crimes --help. An agent that follows it gets a confidently-worded explanation of the wrong code, with no signal that anything went wrong. Worst failure class available: silent, plausible, wrong.
severity: blocker

No `fingerprint` field anywhere in `scan --format json`

repo: thickeningtime
command: grep -c fingerprint artifacts/thickeningtime.json → 0
what happened: explain, ignore, unignore and feedback all take a stable fingerprint (<type>::<file>::<symbol>), and explain even prints the crimes ignore <fingerprint> line for you. But the JSON contract: the thing the PRD calls the product: never emits a fingerprint field. The only machine-readable id is crime_NNNNN, which is scan-local and unstable.
why it matters: the stable identifier is undiscoverable from the machine-readable output. Any programmatic consumer (CI, an agent, a dashboard) must either reconstruct the fingerprint by string-concatenating type::file::symbol: undocumented, and symbol is absent on ~25% of findings: or shell out to explain per finding and scrape the suppress line.
severity: real problem

`explain` has no `--root`; with no `--from` it silently full-scans cwd and can hang forever

repo: thickeningtime (observed from a wrong cwd)
command: crimes explain crime_00004 --no-init run with cwd=~/crimes-dogfood
what happened: hung past a 600 s timeout with zero output: no “scanning…”, no progress, no file count. It was recursively scanning cwd (which happened to hold 3.3 GB of unrelated clones). Every other repo-touching command (scan, hotspots, context, diff, verdict) takes a path argument or --root; explain takes neither.
why it matters: explain is the one command you reach for while already deep in a session, and it is the one command that cannot be pointed at a repo. Wrong cwd = indefinite silent hang. There is also no progress output on any long scan, so “slow” and “wedged” are indistinguishable.
severity: real problem

`explain` fails outright on `oversized_raster`

repo: thickeningtime
command: crimes explain crime_00348 --from thickeningtime.json --no-init
what happened: crimes: no detector registered for finding.type "oversized_raster". exit 2. Tried one id of all 32 finding types present in this repo: 31 explained fine, oversized_raster is the only one that 500s on its own output.
why it matters: a detector can emit a finding the tool then refuses to explain. It is a registry gap, not a content gap, so it will hit every user of that detector 100% of the time.
severity: real problem

`weak_test_signal` cannot see inside Playwright test callbacks: 8/8 wrong in one file

repo: thickeningtime
command: crimes context e2e/social-reading.spec.ts --no-init
what happened: all 8 findings claim 0 expect/assert calls. e2e/social-reading.spec.ts:65-71 contains four expect(...) calls in the very first flagged test. The reported line range is 66-66: the declaration line only: which suggests the callback body was never walked. The shape is test("…", async ({ request }) => { … }): a destructured-object arrow param.
why it matters: this is a factual claim about the code that is checkable and false, stated at confidence 0.88. Playwright/{ page }/{ request } fixtures are the dominant e2e idiom in Next.js repos, so this is not an edge case.
severity: real problem

`context` on a test file does not know it is looking at a test file

repo: thickeningtime
command: crimes context e2e/social-reading.spec.ts --no-init
what happened: “Likely tests: (no test file matched the target basename … and no test file imports it)” and “Clues · test gap: bottom-quartile”: reported about a test file. Agent guidance was “Nearby tests may not protect behaviour”.
why it matters: context is pitched as the pre-edit briefing for an agent. Given a spec file it should say “this is a test file, here is what it covers”: instead it emits two lines of noise that are literally inapplicable and mildly misleading.
severity: annoyance

`hotspots` puts 8 zero-churn files in the top 20 of a churn ranking

repo: thickeningtime
command: crimes hotspots ~/dev/thickeningtime --no-init
what happened: top 10 tracks git well (positions 1–6 are exactly the git top-6 by 90-day commit count). But ranks 13–20 all read 0 changes, at a flat risk of 0.40, and they outrank real churn: app/page.tsx (4 commits in 90 d) sits at rank 33 with risk 0.36, below app/api/admin/chapters/route.ts (0 commits, 1 finding) at rank 14. 18 of the top 30 have zero commits in the window.
why it matters: the command’s own one-line description is “Rank files by change risk using Git churn and current scan findings”. Past the top ~10 it is a severity ranking wearing a churn ranking’s name, and it demotes the files a maintainer would actually recognise as hot. Risk also saturates at 0.58 and is heavily quantised for zero-churn files (0.40 / 0.24 / 0.12), so ties are everywhere.
severity: real problem

`scores.recency` is 0.00 on all 503 findings

repo: thickeningtime
command: crimes scan … --all --format json
what happened: every finding has scores.recency = 0. scan advertises --no-recency (“disable the recency multiplier on rank_score”), implying it is on by default and doing something. The repo’s newest commit is 34 days old.
why it matters: either the multiplier is dead code or its window is short enough that a repo idle for a month gets no signal from it at all. Either way --no-recency is currently a no-op flag on this repo, and rank ordering is not what the flag implies.
severity: real problem

`churn` is 0.00 on 316 of 503 findings in a 324-commit repo

repo: thickeningtime
command: crimes scan … --format json
what happened: 63% of findings score churn: 0, and the human report prints “Risk: churn low” under every single one of the top 5 files: including lib/pdf-generator.ts, which is the 5th-most-changed file in the repo’s whole history (19 commits). The window is hardcoded to 90 days in the scan path with no flag to widen it (hotspots has --since; scan does not).
why it matters: “churn low” printed under the repo’s genuinely hottest files is actively misleading, and there is no flag to fix it from the scan side.
severity: real problem

`scan` human output contradicts itself on the file count

repo: thickeningtime
command: crimes scan ~/dev/thickeningtime --no-init
what happened: header says “503 findings across 187 files”; the footer says “Showing 5 of 163 files”; hotspots on the same repo says “287 files”; coverage.files_total in the JSON says 298.
why it matters: four different denominators on one repo in two commands. None is labelled well enough to reconcile.
severity: annoyance

124 of 503 findings (25%) carry no `lines` field at all

repo: thickeningtime
command: crimes scan … --format json
what happened: lines is absent (not null: absent) on 124 findings, across orphaned_destination, design_token_escape, oversized_raster, route_metadata_drift, duplicate_component_shape, high_fan_in_fan_out.
why it matters: the schema is the product contract and a consumer cannot tell an optional field from a missing one. “Evidence before judgement… line ranges” is a stated PRD constraint.
severity: real problem

`explain` rejects a two-part fingerprint

repo: thickeningtime
command: crimes explain "large_file::lib/pdf-generator.ts" --from … --no-init
what happened: exit 2, “no finding with id or fingerprint”. The three-part form with an empty symbol (large_file::lib/pdf-generator.ts::) works.
why it matters: for file-level findings there is no symbol, so the natural thing a human types is the two-part form. Requiring a trailing :: is an unguessable ritual and the error message does not hint at it.
severity: annoyance

`crimes scan <path-that-does-not-exist>` reports a clean repo and exits 0

repo: thickeningtime (control)
command: crimes scan /Users/andrew/dev/nope-not-here --no-init
what happened: CRIME SCENE REPORT / repo: nope-not-here · 0 findings / No crimes detected. Suspiciously clean. exit 0. --format json is the same: files_total: 0, findings: [], packs_loaded populated, exit 0. Nothing anywhere says the directory does not exist.
why it matters: the single most likely CI misconfiguration: a typo’d or wrong-relative path: produces a permanently green gate. coverage.files_total: 0 is the only tell and no consumer is told to check it. Contrast with context, which correctly errors with file not found and exit 2.
severity: blocker

`crimes scan <a-file>` leaks a raw Node ENOTDIR with an internal path

repo: thickeningtime
command: crimes scan lib/pdf-generator.ts --no-init
what happened: crimes: ENOTDIR: not a directory, open '/Users/andrew/dev/thickeningtime/lib/pdf-generator.ts/.crimes/triage.json' exit 1.
why it matters: unhandled errno surfaced to the user, from the triage-config loader rather than from argument validation. scan <file> is a completely reasonable thing to type (that is what context takes) and the message does not say “scan takes a directory, you probably want crimes context”.
severity: annoyance

`crimes context <directory>` and `context <file outside the repo>` both answer “risk: NONE” with exit 0

repo: thickeningtime
command: crimes context lib --no-init / crimes context /etc/hosts --no-init
what happened: both exit 0. The directory case prints file: lib / risk: NONE (0 findings). The outside-the-repo case prints file: ../../../../private/etc/hosts / risk: NONE (0 findings): it happily renders a path that escapes the root four levels up. The “agent guidance” in both cases is the boilerplate “no language pack claims (no extension) files”.
why it matters: context is the pre-edit safety check. “risk: NONE” for something it never analysed is the exact answer that makes an agent proceed. Nonexistent files error correctly (exit 2): so the validation exists, it just does not cover directories or out-of-root paths.
severity: real problem

`crimes diff` human output is three integers and nothing else

repo: thickeningtime
command: crimes diff HEAD~30...HEAD --no-init
what happened: 7 lines total: base, head, New crimes: 33, Fixed crimes: 10, Unchanged crimes: 424. No list of the new crimes, no files, no severities. diff has no --all or --top. The JSON form does contain all 33 with full detail: it is purely a renderer gap.
why it matters: the stated use for diff is PR review. “33 new crimes” with no way to see them in human output means the reviewer must re-run with --format json and read raw JSON, or run a full scan and diff by eye. --fail-on new-high exists, so a CI job can block on findings it then cannot print.
severity: real problem

`diff`’s head-side scan disagrees with `scan --all` on the same working tree by 46 findings

repo: thickeningtime
command: crimes diff HEAD~30...HEAD --format json vs crimes scan --all --format json
what happened: new_findings + unchanged_findings = 457; scan --all on the same tree = 503. The gap is concentrated: weak_test_signal 6 vs 24, parallel_destination 4 vs 14, swallowed_error 63 vs 74, oversized_raster 4 vs 7, plus one each off contract_drift, duplicated_policy, duplicated_role_status_plan_check, duplicated_navigation_source. (Tree was dirty with 12 untracked paths, none of them .ts/.tsx, so that does not explain the test/error deltas.)
why it matters: two commands give two different answers for the state of HEAD. Whichever is right, diff’s “unchanged” count cannot be reconciled against a scan, so a user cannot verify a gate.
severity: real problem

`crimes verdict` takes 17 s to conclude base == head

repo: thickeningtime
command: crimes verdict --no-init (cwd=repo, HEAD identical to origin/main)
what happened: 17.4 s wall, then Verdict: UNCHANGED / New: 0 / Fixed: 0. git rev-list --left-right --count origin/main...HEAD = 0 0: the two refs are the same commit. It ran two full scans to discover that.
why it matters: the answer is right, but 17 s for a same-SHA comparison is 17 s of a pre-commit or PR hook nobody will keep. A ref-equality short-circuit is one git rev-parse away.
severity: annoyance

`hotspots --since` default of 90d is wrong for the common case, and the header “N files” is a git artefact

repo: thickeningtime
command: crimes hotspots --since 1y --no-init vs the 90d default
what happened: at --since 1y the ranking is visibly better: app/download/[token]/page.tsx (24 commits) and lib/personalization.ts (20) at risk 1.00, lib/pdf-generator.ts (19) at 0.97: and it matches the all-time git churn list. At the 90d default the top risk in the entire repo is 0.58 and 8 of the top 20 have zero commits. The header also flips from “287 files” to “4634 files”: larger than the repo has ever contained at one time, so that number counts historical/deleted paths from git log, not files that exist.
why it matters: any repo whose last burst of work was >90 days ago (i.e. most side projects and most maintenance-mode services) gets a degenerate, near-flat ranking by default, and the header number is not a repo fact.
severity: real problem

`scan --changed` costs the same as a full scan and recommends running `context` on a PNG

repo: thickeningtime (12 untracked paths, none of them source)
command: crimes scan --changed --no-init
what happened: 16.1 s wall (a full scan --all of this repo is 4.2 s) to return 7 findings, all oversized_raster on untracked PNGs and one public/ bucket. Closing line: → Start with crimes context generated_imgs/edited-2026-03-11T…png: it concentrates the most risk in this scan.
why it matters: --changed is the pre-commit/CI path, so it is the one that must be fast, and it is 4× slower than scanning everything. And the next-action line is generated without checking that the target is a file context can say anything about.
severity: real problem

Flags that exist on one command and not its sibling

repo: thickeningtime
commands: crimes scan --since 1y → error: unknown option '--since'; crimes hotspots --top 5 → unknown option '--top'; crimes hotspots --root . → unknown option '--root'; crimes explain --root … → not offered.
what happened: --since is on hotspots only, --top is on scan only, --root is on context/diff/verdict only, --all is on scan/hotspots but not diff. The churn window that hotspots --since controls is the same 90d window scan bakes into every scores.churn, but scan cannot change it.
why it matters: four commands, four different option vocabularies for the same three concepts (where, how far back, how much). Every one of these was a first-guess.
severity: annoyance

`diff` range validation is genuinely good

repo: thickeningtime
commands: diff HEAD~5..HEAD, diff HEAD~5, diff nosuchref...HEAD
what happened: double-dot → “double-dot ranges are not supported: use triple-dot (…) form”, with an example. Missing separator → “missing triple-dot separator”, with an example. Unresolvable ref → unknown git ref "nosuchref". Pass a ref Git can resolve, e.g. main, origin/main, or a commit SHA. All exit 2. Same for explain --from on a missing file and on a non-JSON file.
why it matters: noting the positive: this is the standard the rest of the surface should be held to, and it makes the scan <nonexistent-dir> exit-0 case look worse by contrast, not better.
severity: n/a (positive)

Confirmed: no writes into the repo

repo: thickeningtime
what happened: full recursive path listing before/after the whole session is byte-identical; no .crimes/ was created; git status --porcelain unchanged at 12 untracked paths. Every command was run with --no-init.
severity: n/a (positive)

choreograph.cc

Friction log: choreograph.cc

Repo: ~/dev/choreograph.cc (Next.js app, 228 .ts / 52 .tsx, 380 commits) SHA: 9e9720c9d47170f0fafc6dfd3fcc5b473c26dc6d (clean tree) Binary: ~/crimes-dogfood/bin/crimes 0.17.0 Started: 2026-08-02 Focus: command surface beyond scan

`scan` is not reproducible: `scores.recency` reads the wall clock, so ids and ordering change between runs

repo: choreograph.cc (clean tree, same SHA, same command)
command: crimes scan ~/dev/choreograph.cc --all --format json --no-init run twice, ~25 minutes apart
what happened: identical finding multiset (490 both times), but 51 findings changed scores.recency (e.g. 0.13 → 0.12) and the output order diverged from index 14 onwards. Since crime_NNNNN ids are assigned by rank position, crime_00015 is a different finding in run 2 than in run 1. recency is the only field that moved; evidence and all other scores were byte-identical.
why it matters: three things break at once. (1) You cannot diff or cache two scan artifacts: every run is a spurious diff. (2) crimes explain crime_00015 answers a different question depending on what time you run it, on the same commit. (3) scan (top-5 view) can show a different set of files morning vs afternoon. The tool ships a detector called Temporal Recklessness for exactly this pattern.
severity: blocker

`context` silently drops cross-file findings that `scan` reports for the same file

repo: choreograph.cc
command: crimes context src/components/admin/JobDetail.tsx vs crimes scan --all
what happened: scan reports 12 findings for JobDetail.tsx (3 high, 7 medium, 2 low); context reports 11 (3 high, 6 medium, 2 low). The missing one is duplicated_policy at lines 10–19: Policy Doppelgänger · status "completed" "failed" rule … 2 occurrence(s) across 2 file(s). Verified by diffing the two JSON outputs on (type, lines, symbol): exactly one finding in scan and not in context, nothing the other way.
why it matters: context is the pre-edit briefing, and the finding it drops is a cross-file one: precisely the class an agent cannot discover by reading the file it is about to edit. Dropping it inverts the command’s whole reason to exist. context on a file is also what crimes hook runs on PreToolUse, so this is the default agent path.
severity: blocker

`test_gap: 0.00: "a test file imports this module"` in a repo with zero test files

repo: choreograph.cc (no *.test.*, no *.spec.*, no __tests__/, no test script in package.json)
command: crimes explain crime_00482 --from choreograph.cc.json (agent_permission_sprawl on .mcp.json)
what happened: Risk profile prints test gap: 0.00: a test file imports this module. There is no test file in this repository. 12 findings carry test_gap: 0, all on non-source files: 4 .png, 6 .md plan documents, CLAUDE.md, .mcp.json. The remaining 478 findings all carry exactly 0.5, rendered as “test gap ~median”.
why it matters: the score’s prose is generated from the number without checking whether the number was ever measured. 0 means “not applicable / not computed” and gets narrated as an affirmative claim about test coverage that is provably false. And 0.5 on every source finding in a repo with no tests at all means the signal is a constant, not a measurement: while the renderer says “~median”, implying a comparison happened.
severity: real problem

`name_behavior_mismatch` matches prefixes by substring, not word boundary: `cancel` reads as `can`

repo: choreograph.cc (and thickeningtime)
command: crimes scan --all --format json
what happened: 4 findings whose evidence line is “name prefix suggests a predicate”: cancel ×2 (src/components/admin/JobDetail.tsx), cancelEdit (src/components/admin/QuotesAdmin.tsx), isoWeek (scripts/_backfill-song-type.ts). cancel/cancelEdit are matching on can; isoWeek on is. None is a predicate. cancel setting React state is the single most ordinary thing in the file.
why it matters: trivially checkable false positives at confidence 0.80, on a detector whose entire premise is that names should be trusted. Every one of the four is the same one-line fix (require a boundary: prefix followed by an uppercase letter or end-of-name).
severity: real problem

`crimes diff` misses findings that `scan` reports: including the repo’s only `agent_permission_sprawl`

repo: choreograph.cc (clean tree, so tree state is not the explanation)
command: crimes diff HEAD~30...HEAD --format json vs crimes scan --all --format json
what happened: new_findings + unchanged_findings = 479 vs 490 from scan. Missing: 4 swallowed_error, 3 duplicated_role_status_plan_check, 1 each of contract_drift, logic_in_comments, name_behavior_mismatch, and the repository’s only agent_permission_sprawl finding (.mcp.json). The same effect appeared on thickeningtime (457 vs 503).
why it matters: diff --fail-on new-high is the CI gate. A finding that diff cannot see can never be gated on, and the one it drops here is the agent-supply-chain finding: an unpinned npx -y n8n-mcp@latest MCP server. Repo-level findings (.mcp.json, package.json) appear to be outside diff’s scope entirely.
severity: real problem

`diff` is 5.5× slower than a full `--all` scan of the same repo

repo: choreograph.cc
command: crimes diff HEAD~30...HEAD --no-init = 26.4 s wall; crimes scan --all --format json = 4.8 s
what happened: 26 s to produce three integers. (thickeningtime: 13.1 s diff vs 4.2 s scan.)
why it matters: two scans should cost ~2×, not 5.5×. Whatever diff does to materialise the base ref is the dominant cost, and it is the command most likely to sit in a pre-push hook.
severity: annoyance

`hotspots` promotes five throwaway probe scripts over a genuinely-churning source file

repo: choreograph.cc
command: crimes hotspots ~/dev/choreograph.cc --no-init
what happened: rank 1 is correct (src/lib/daily-agents.ts, 12 commits, risk 0.76). Ranks 7–11 are scripts/_check-limitless-recent.ts, scripts/_melvin-fleetonomics-transcript.ts, scripts/_probe-github-events.ts, scripts/_probe-melvin-day.ts, scripts/_telos-andrew-conversations.ts: five underscore-prefixed one-off probe scripts, 1 commit each, all tied at exactly 0.43. Meanwhile src/lib/comedian/system-prompts.ts (6 commits in the window, 3rd-hottest file in the repo) sits at rank 138, risk 0.18, because it has 0 findings. 5 of the top 20 have zero commits.
why it matters: core ships util/scope-class.ts as “one answer to generated / vendored / migration / fixture / test / config / production”, and hotspots does not consult it. The command also cannot see a hot file that happens to be clean: which is the one thing churn is supposed to add over severity. scan compounds it by binning 149 of the repo’s 490 findings (30%) into a one-line “Also flagged elsewhere · scripts/” that you can only expand by re-running with --all.
severity: real problem

`verdict` output is identical whether the branch is genuinely unchanged or the comparison is degenerate

repo: choreograph.cc (HEAD == origin/main)
command: crimes verdict --no-init, 11.6 s
what happened: Verdict: UNCHANGED / New: 0 / Fixed: 0 / Reason: no new findings and no fixed findings. Correct: but it does not say that base and head are the same commit, which is the actual reason and a completely different situation from “I made changes and they were neutral”. Same output shape on thickeningtime.
why it matters: on a CI branch that failed to fetch its base ref, verdict would print a confident UNCHANGED and exit 0. The one line that would disambiguate (“base and head resolve to the same commit”) is missing.
severity: real problem

`explain`’s “Likely remedies” boilerplate is written for naming detectors and leaks into every other type

repo: choreograph.cc
command: crimes explain crime_00482 --from … (agent_permission_sprawl on .mcp.json)
what happened: remedy 2 reads “If this reflects a real project convention, configure the detector or record feedback instead of renaming code blindly.” The finding is about an unpinned MCP server in a JSON config file. There is no code and nothing to rename. The same sentence appears verbatim under all 25 explainable types in this repo.
why it matters: the first-party long-form explanation is otherwise the best thing on the surface: genuinely specific, with a confidence ladder broken out term by term. One templated sentence written for name_behavior_mismatch undercuts it on every finding that is not about a name.
severity: annoyance

Positive: `explain` is excellent on the 0.16 detectors

repo: choreograph.cc
command: crimes explain <id> --from … on one instance of each of the 26 finding types present
what happened: 25/26 succeeded. agent_permission_sprawl produced a genuinely useful writeup: identified npx -y n8n-mcp@latest as unpinned, named the environment variables passed through (N8N_API_KEY, N8N_API_URL, “names only; no values read”), explained that MCP servers inherit the agent’s environment, and broke out confidence 0.75 = 0.70 base + 0.05 (launch command read verbatim from committed configuration). swallowed_error, unsafe_retry, config_drift, duplicated_policy, contract_drift are similarly specific.
why it matters: this is the part of the product that most justifies the positioning, and it works. Recording it so the failures above are read in proportion.
severity: n/a (positive)

Zero case: confirmed clean

repo: choreograph.cc (single-language: 283 JS/TS files, 37 .md, no .py)
what happened: cross_language_contract_drift, cross_language_duplicate_logic, cross_language_orphan_endpoint produced zero findings, as required. All Python detectors produced zero findings. No cross-language pack appears in coverage.packs_loaded at all: the list is ["universal", "language-js", "language-py"]. language-py is loaded on a repo with no .py files and correctly produces nothing.
severity: n/a (positive)

`coverage.by_package` is absent from the 0.15 surface on both TS monorepo-ish repos

repo: choreograph.cc (and thickeningtime)
command: crimes scan --all --format json
what happened: coverage contains only files_total, files_by_language, files_universal_only, universal_only_by_extension, packs_loaded. No by_package key.
why it matters: the protocol lists coverage.by_package as a 0.15 robustness check “where present”; it is present on neither single-package repo, so there is nothing to check. Recording it as a negative result: the field appears to require a workspace/monorepo layout to materialise.
severity: n/a (observation)

Confirmed: no writes into the repo

repo: choreograph.cc
what happened: recursive path listing before/after identical; no .crimes/ created; git status --porcelain empty before and after.
severity: n/a (positive)

improbable

Friction log: improbable

Repo: ~/dev/improbable (130 .ts / 71 .tsx, 408 commits, clean tree) SHA: ec22855b2c8f820c914b91f32d98909c6819151a Binary: ~/crimes-dogfood/bin/crimes 0.17.0 Started: 2026-08-02 Note: artifacts/improbable.json pre-existed; not re-run. Everything else fresh. Focus: command surface beyond scan

Root cause found: fingerprints are not unique, and `diff` silently dedupes on them

repo: improbable (and reproduced on both other repos)
command: crimes diff HEAD~30...HEAD --format json vs crimes scan --all --format json
what happened: diff head-set = 424; scan --all = 488. 64 findings missing (13%), concentrated in weak_test_signal (−38), large_function (−16), logic_in_comments (−4), swallowed_error (−4); by directory, test/ loses 42 of 90. Computing type::file::symbol over the scan output explains it almost exactly: 488 findings collapse to 420 distinct fingerprints: 68 collisions. Worst offenders: large_function::src/MetricTreeSurface.tsx::<anonymous> ×12, weak_test_signal::test/server.test.ts:: ×7, weak_test_signal::test/connector.test.ts:: ×6, logic_in_comments::src/MetricTreeSurface.tsx:: ×5. The same arithmetic holds on the other two repos: choreograph.cc 490 findings / 477 fingerprints / 11-finding diff gap; thickeningtime 503 / 455 / 46.
why it matters: this is one root cause behind three separate symptoms. (1) diff and verdict undercount by 2–13%, so --fail-on new-high gates on an incomplete set. (2) crimes ignore <fingerprint> --reason "…" silences every colliding finding: one reason retires 12 distinct God Functions across 10 000 lines of MetricTreeSurface.tsx, or all 7 weak tests in server.test.ts. (3) explain <fingerprint> returns an arbitrary member of the set with no indication there are 11 others (verified: it returned lines 9864–10261 out of 12 candidates spanning lines 2948–13653, exit 0, silent). Anything without a symbol, or with <anonymous>, <module>, or a repeated → callee suffix, collides by construction.
severity: blocker

0.17’s `discriminator` fingerprints reach only 3 detectors, none of the colliding ones

repo: improbable / choreograph.cc / thickeningtime
command: grep -o '"discriminator"' *.json
what happened: discriminator is present on 20 / 18 / 17 findings respectively, and only ever on magic_domain_literal_scatter, exact_duplicate_block, near_duplicate_block. It never appears on large_function, weak_test_signal, logic_in_comments, swallowed_error: the four detectors that actually generate the collisions above.
why it matters: 0.17 shipped the mechanism that would fix fingerprint collisions and wired it to the duplication family, which was already the least collision-prone (its findings carry a natural discriminator). The uniqueness problem is untouched.
severity: real problem

`context` rewrites paths inside a workspace package and prints a path that does not exist

repo: improbable (pnpm workspace: root @ortomate/improbable + app package)
command: crimes context app/src/middleware.ts --no-init (cwd = repo root)
what happened: header reads file: **src/middleware.ts**. There is no src/middleware.ts in this repo: ls src/middleware.ts → No such file. It analysed the right file (the churn clue “1 commits over 90d · last touched 10 days ago” matches app/src/middleware.ts exactly), but re-rooted the display path into the app/ package and dropped the prefix. “Related files · src/instrumentation.ts” is the same rewrite of app/src/instrumentation.ts. crimes scan on the same repo reports these files correctly as app/… (67 findings under app/).
why it matters: scan and context disagree about what a file is called, and context’s version is unopenable. The repo also has a real top-level src/, so src/middleware.ts is not merely wrong, it is plausible: an agent will try to open it, or worse, create it.
severity: real problem

`weak_test_signal`: a second, distinct failure mode: assertions behind a helper

repo: improbable
command: crimes context test/server.test.ts --no-init
what happened: test/server.test.ts:50-52, “rejects creation without edit rights, using the configured message” → “0 expect/assert calls”, confidence 0.88. The test body is one line: await expectStatus(server.createNode(...), 403, "editors only");, and expectStatus is a local helper defined 5 lines above that calls expect(...). This is different from the thickeningtime failure (where expect( was literally inside the callback and still missed): here the detector’s premise is simply that assertions must be lexically inline.
why it matters: weak_test_signal is 59 of improbable’s 488 findings (12%) and 24 of thickeningtime’s. A shared expectStatus/assertOk helper is standard practice in any test suite past a few hundred lines, so the detector penalises exactly the suites that have been refactored. Two independent failure modes on one detector across two repos.
severity: real problem

`hotspots` is genuinely good on an actively-developed repo: the 90d default is the whole problem

repo: improbable (408 commits in the last 90 days, last commit 2 days ago)
command: crimes hotspots ~/dev/improbable --no-init
what happened: top 10 is git’s top 10 by 90-day commit count, near-perfectly ordered, and zero zero-churn files appear in the top 30. src/MetricTreeSurface.tsx (117 commits, 57 findings) at 1.00 is unarguably the right #1.
why it matters: the ranking failures logged against the other two repos are not a ranking-model failure, they are a window failure: a repo idle for >90 days has no churn signal in the default window and the formula falls back to severity. Recording it here so the recommendation is “widen or auto-select --since”, not “rewrite the ranker”. One residual issue: 7 files tie at exactly 1.00, and src/index.ts (45 commits, 2nd-hottest in the repo) sits at rank 10 / 0.84 while src/types.ts (34 commits) is rank 3 / 1.00: the clamp makes the top tier arbitrary.
severity: real problem (window), annoyance (saturation)

`context` on the hot file costs 13.5 s: and `crimes hook` runs it before every edit

repo: improbable
command: crimes context src/MetricTreeSurface.tsx --no-init → 13.5 s wall, 651 lines of output for 57 findings
what happened: the output is high quality: 8 likely tests correctly identified, churn: 117 commits over 90d · last touched 4 days ago · 3 authors, 7 lines of specific agent guidance. It is also 651 lines long and takes 13.5 s.
why it matters: crimes hook registers context as a Claude Code / Codex PreToolUse hook. A 13.5 s stall plus 651 lines injected into the context window before every edit to the repo’s most-edited file is not a thing anyone leaves switched on. context has no --top/--max-findings (scan has --top), so there is no way to ask for the short version.
severity: real problem

`diff` and `verdict` are the two slowest commands and the two most likely to be in a hook

repo: improbable
command: diff HEAD~30...HEAD = 27.8 s; verdict = 23.8 s; scan --all --format json = 11.7 s; scan human = 11.7 s
what happened: verdict spent 23.8 s to report UNCHANGED on a HEAD identical to origin/main. Across the three repos: verdict 17.4 s / 11.6 s / 23.8 s, all on same-SHA comparisons.
why it matters: consistent with the other repos: diff costs ~2.4× a full scan and verdict ~2× even when the answer is decidable from git rev-parse alone.
severity: annoyance

`explain` clears every finding type in this repo: 27/27

repo: improbable
command: crimes explain <id> --from improbable.json on one instance of each of the 27 types present
what happened: all 27 succeeded with full “What this detector looks for” / “Why it matters” / evidence / confidence-ladder / suggested-actions sections, including the 0.16 detectors agent_permission_sprawl (×2, on .claude/settings.json::hooks and .agents/skills/clerk-cli/SKILL.md::agent instructions), unsafe_retry (trigger.config.ts), return_shape_roulette, option_bag_junk_drawer, duplicated_policy, config_drift. Fingerprint form works both with --from and against a fresh scan.
why it matters: across all three repos, one instance of each of 32 / 26 / 27 types was explained and only oversized_raster ever failed. explain is the most solid command on the surface.
severity: n/a (positive)

Id stability: intermittent, which is worse than consistently broken

repo: improbable
command: crimes explain crime_00100 --from <artifact captured 90 min earlier> vs crimes explain crime_00100 (fresh scan)
what happened: both returned large_function / buildKpiColumns / src/KpiTreeGrid.tsx:399–677. Stable here: whereas on choreograph.cc two runs 25 minutes apart reordered from index 14 onwards because scores.recency crossed a rounding boundary (0.13 → 0.12).
why it matters: ids are stable most of the time and silently shift when a recency value happens to round differently. Intermittent id drift is harder to notice than total instability, and it lands on explain, ignore, and feedback: all of which take an id.
severity: real problem

Zero case: confirmed clean

repo: improbable (183 JS/TS files, 105 .md, no .py)
what happened: cross_language_contract_drift, cross_language_duplicate_logic, cross_language_orphan_endpoint → zero findings. All Python detectors → zero findings. coverage.packs_loaded = ["universal","language-js","language-py"]; language-py is loaded and correctly silent. No cross-language pack is listed in packs_loaded on any of the three repos.
severity: n/a (positive)

`coverage.by_package` does not appear even on a real pnpm workspace

repo: improbable (pnpm-workspace.yaml declaring packages: [app], app/package.json present, 67 findings under app/)
command: crimes scan --all --format json
what happened: coverage has exactly five keys: files_total, files_by_language, files_universal_only, universal_only_by_extension, packs_loaded. No by_package. grep -c by_package = 0 across all three repo artifacts.
why it matters: the protocol names coverage.by_package as the 0.15 robustness check. This is the strongest workspace case in my three repos and the field is absent, so the 0.15 coverage surface is unverifiable in the wild: it may require an explicitly-configured layout.
severity: real problem (negative result on the 0.15 surface)

0.17 `large_file` `docs` shape works

repo: improbable / choreograph.cc / thickeningtime
what happened: 1 / 6 / 8 large_file findings on .md, each with a correct shape line: 1.7× the configured 1000-line document threshold · shape: document (matches .md/.mdx/.markdown/.rst/.adoc/.txt). Documents get their own 1000-line budget rather than the 300-line code budget.
severity: n/a (positive)

Confirmed: no writes into the repo

repo: improbable
what happened: recursive path listing before/after identical; no .crimes/; git status --porcelain empty. Every command run with --no-init.
severity: n/a (positive)

posthog

Friction log: posthog

Repo: ~/crimes-dogfood/corpus/posthog @ 96e243726db1ca001c2aad520e281719f2e3cbd6 Binary: ~/crimes-dogfood/bin/crimes → crimes 0.17.0 Host: macOS 26.5.1, 10 cores, 64 GB RAM, ulimit -n = 1048576 Started: 2026-08-02

Size / language mix (git ls-files)

ext	files
.py	17690
.ts	8473
.tsx	6430
.rs	1342
.json	1493
.md	1025
(total tracked)	41747

Notes on measurement conditions

Between 5 and 8 other crimes processes (other repos in the same dogfooding round) were running concurrently on this 10-core box for the whole duration of the PostHog scan. All wall-clock timings below are contaminated by CPU contention and should be read as an upper bound, not a clean measurement. Max RSS is less affected.

`--no-init` is a global option but is not listed in `scan --help`

repo: posthog
command: crimes scan --help
what happened: the protocol’s canonical invocation is crimes scan "$R" --all --format json --no-init. scan --help lists 13 options and --no-init is not among them. It only appears under crimes --help as a top-level option. First reaction was “this flag doesn’t exist on scan” and I nearly dropped it.
why it matters: the flag that every CI/agent invocation needs is invisible from the help of the command everyone actually runs.
severity: annoyance

`scan` on a 1.6M-LOC repo prints nothing for >10 minutes

repo: posthog
command: crimes scan . --all --format json --no-init
what happened: no progress output of any kind. No file count, no “parsing 32k files”, no spinner, nothing on stderr. For the first eight minutes the only way to tell it was alive was ps. RSS oscillated between 50 MB and 340 MB, so it was clearly working, but from the outside a hung process and a working one are indistinguishable.
why it matters: the first thing a new user does is point it at their monorepo. If it looks hung for ten minutes they Ctrl-C and never come back. There is no --verbose, no --progress, no ETA.
severity: real problem

The scan spends most of its wall clock inside one `git log` subprocess

repo: posthog
command: crimes scan . --all --format json --no-init
what happened: sampling the stuck-looking process showed the main thread blocked and one child: git log --since=90 days ago --pretty=format:CRIMES_COMMIT %cI %ae --name-only --no-merges That subprocess alone ran for >5 minutes. Run standalone the same command takes ~88 s and emits 138,253 lines (PostHog’s 90-day churn window). Git also warns exhaustive rename detection was skipped due to too many files: that warning goes to the subprocess’s stderr and is swallowed; nothing appears in crimes’ own stderr.
why it matters: the churn window is a fixed 90 days with no config knob and no cap on output size. On any repo with PostHog’s commit rate this dominates the scan. Node also has to buffer the whole 138k-line blob in memory before parsing it.
severity: real problem

Run 1 of the JSON scan: killed externally at 773 s, no partial output

repo: posthog
command: /usr/bin/time -l crimes scan . --all --format json --no-init
what happened: EXIT=143 (SIGTERM) after 773.52 s real, 164.29 s user, 51.67 s sys, max RSS 678 MB / peak footprint 1.09 GB. The SIGTERM came from outside crimes (a sibling dogfooding session running pkill), so this is not a crimes crash: but the consequence is: posthog.json was 0 bytes. Thirteen minutes of work produced nothing at all, because the JSON reporter buffers the entire document and writes it in one go at the very end.
why it matters: on a repo this size any interruption: Ctrl-C, CI step timeout, OOM killer: costs the whole run with zero salvage. There is no incremental/streaming output and no cache, so a retry starts from zero.
severity: real problem

Run 2: >22 minutes and still running, RSS ~580 MB, still zero output

repo: posthog (41,747 tracked files; 17,690 .py, 8,473 .ts, 6,430 .tsx; ~1.6M LOC of py+ts; 1.0 GB working tree, 401 MB .git)
command: /usr/bin/time -l crimes scan . --all --format json --no-init
what happened: second attempt, this time with no other crimes processes competing. Past the git log phase by ~6 min; then a long silent parse/detect phase with RSS climbing to 577 MB.
why it matters: scan-detect.ts keeps a PackedFile: the full parsed AST summary: for every JS and Python file in the repo, for the whole scan, purely so the three cross-language detectors can run at the end. That is ~32,600 retained parses on PostHog. The cost is paid on every scan whether or not the cross-language detectors will produce anything, and it is the reason peak footprint hits ~1 GB.
severity: real problem

THE HEADLINE: `cross_language_route_drift` cannot see Django/DRF at all

repo: posthog
command: read-only probe running the shipped parsers (packages/language-py/dist, packages/language-js/dist) over all 17,704 .py and 15,133 .ts/.tsx files: i.e. exactly what runCrossLanguageDetectors would be handed.
what happened:
- Backend routes extracted: 28. Every single one comes from services/llm-gateway/ (a FastAPI LLM proxy) and services/stripe-mock/ (a mock of Stripe’s API, for tests). Zero routes from PostHog’s own API.
- PostHog’s real API surface is Django + DRF: 206 router.register(...) calls, 183 path()/re_path() entries in urlpatterns, and 205 @action(detail=…, methods=[…]) viewset actions. routes.ts only recognises a decorator whose callee is <app|router|api|bp|blueprint|mod|server>.<verb> with a positional string literal starting with /. Django declares none of that shape. The dominant Python web framework is invisible.
- Frontend fetch sites extracted: 42, of which 0 match any backend route: a 100% orphan rate. Composition of the 42: 11 from frontend/src/lib/api.test.ts (unit-test assertions, several of them deliberately invalid: /api/projects/null, /api/projects/089908?x), 4 from playwright/e2e/*.spec.ts, 1 from a semgrep rule fixture (.semgrep/rules/devex/toolbar-no-raw-fetch.test.ts), and 22 from products/desktop/packages/api-client/src/posthog-client.ts, a generated client whose paths carry {uuid} / {project_id} placeholders.
- PostHog’s ~2,000 actual frontend calls are invisible for two independent reasons: they are written api.get('api/environments/@current') with no leading slash (104 such literal call sites: rejected by if (!url.startsWith("/")) return;), and the rest are template literals with substitutions (rejected by literalText).
- Net effect: the detector emits exactly one finding: “42 frontend calls reach a path no backend route declares”: produced by comparing PostHog’s test suite and desktop client against a Stripe mock’s route table. Every listed call is a false positive, and the evidence block leads with test files.
why it matters: this is the flagship 0.15.0 detector meeting the exact repo shape it was designed for, and it is not merely silent: it is confidently wrong. The backend.length === 0 guard that exists to prevent this is defeated by 28 routes from two unrelated sidecar services. Worse, the failure is invisible: nothing in the output says “I found 28 routes and they all came from services/”.
severity: blocker

`cross_language_type_drift` is the one that works: 22 candidate findings

repo: posthog
what happened: same probe: 870 Python closed sets, 1,622 TS string-literal unions, 22 name-matched pairs that diverge. Several look substantive rather than cosmetic, e.g. QuotaResource: ee/billing/quota_limiting.py:78 vs nodejs/src/common/services/quota-limiting.service.ts:8, drifting in both directions (12 Python-only members, 2 TS-only), which is a real Python↔Node billing contract. MetricAggregation, RunStatus, LLMProvider are the same shape across products/*/backend vs products/*/frontend. Counterweight: ~11 of the 22 pair posthog/schema_enums.py against products/desktop/packages/api-client/src/generated.ts, which is a generated client. Those are all one-directional (Python-only extras) and are really one finding: “the desktop client’s codegen is stale”: reported eleven times. generated.ts does not match the default **/*.generated.* exclude (the dot is in the wrong place), so it is scanned as hand-written source.
severity: annoyance (dedupe / generated-file handling), but the detector itself is the success story of the release

Python pack parses PostHog cleanly, including modern syntax

repo: posthog
what happened: 17,704 .py files, 0 parse exceptions, 151,558 functions and 33,099 classes extracted. Positive syntax confirmation by sampling files known to contain each construct and checking that symbols came out (not merely that nothing threw): match statements 25/25 produced symbols; walrus 24/25; PEP 604 X | None 25/25; PEP 695 3.12 generics (def f[T], class C[T], type X = …) 23/25; except* 5/5.
severity: n/a: positive result

`crimes init` silently amputates Python: and `packs_loaded` still says it ran

repo: posthog
command: crimes init then crimes scan
what happened: crimes init wrote "include": ["**/*.{ts,tsx,js,jsx,mjs,cjs}"]. That is hardcoded in packages/cli/src/init-detect.ts:85; detectRepoShape() looks only for pnpm-workspace.yaml / turbo.json / next.config.* / vite.config.* and a .js file. It has no notion of Python, Rust, Go, or Markdown. On PostHog: 17,690 .py files, 42% of the repo: the config it writes drops every one of them, plus .md (which docs_code_drift needs) and every universal-only language.

Measured on products/metrics (a subtree with both halves):

files_total by_language findings by pack
default config 100 {py: 48, js: 37} 75 py 31 / js 29 / universal 14 / cross-language 1
after crimes init 37 {js: 37} 33 js 27 / universal 6 / cross-language 0

The whole Python pack, both cross-language detectors, and 8 universal findings vanish. And in both runs coverage.packs_loaded reports ["universal","language-js","language-py"]. The report asserts the Python pack is loaded while zero Python files were even discovered.
also: the init command’s own help text (init.ts:45-49) documents the eight language-py detectors in the same breath as writing a config that disables all of them. scopeTiers.nonDomain likewise only lists TS/JS test globs, so on a polyglot repo Python tests get treated as domain code. And $schema is pinned to 0.1.0 on a 0.17.0 tool emitting schema_version 0.4.0.
why it matters: crimes init is the documented first command and crimes.config.json is meant to be committed. On any polyglot repo the onboarding step permanently disables the two headline capabilities of 0.14 and 0.15, and the coverage block is not just silent about it, it affirmatively misreports.
severity: blocker

	files_total	by_language	findings	by pack
default config	100	`{py: 48, js: 37}`	75	py 31 / js 29 / universal 14 / cross-language 1
after `crimes init`	37	`{js: 37}`	33	js 27 / universal 6 / cross-language 0

Config is resolved from the scan root, not the cwd

repo: posthog
command: crimes scan products/metrics run from the repo root
what happened: the repo-root crimes.config.json written by crimes init was not applied; the scan used built-in defaults. Only after copying the config into products/metrics/ did it take effect.
why it matters: the natural workflow: crimes init at the root, then crimes scan some/subdir: silently ignores your config, and nothing in the output names the config file that was (or wasn’t) loaded. There is no --config flag and no “using config: …” line.
severity: real problem

Scan cost is strongly super-linear in file count

repo: posthog
command: crimes scan <subtree> --all --format json --no-init, timed
what happened:

subtree source files wall
products/surveys 62 4.4 s
products/metrics 90 5.2 s
products/error_tracking 407 8.4 s
ee 906 9.5 s
frontend 5,361 > 600 s (killed at 10 min)
whole repo 32,837 > 6,000 s and still running

Fixed overhead is ~3.5 s and the marginal cost between 90 and 906 files is ~6.6 ms/file. Extrapolating linearly the whole repo should take ~4 minutes. It has been running for 1 h 40 m. Something in the cross-file index or detector layer is super-linear: the per-file loop cannot explain a 25× miss.
calibration point: a standalone probe that runs the shipped parsers over the same 17,704 .py + 15,133 .ts files completes in ~5 minutes, so parsing is not the cost. The cost is in the indexes and detectors built on top of it.
why it matters: the tool is effectively unusable on a large monorepo, which is precisely the population that needs “where is the risk?” most. Every scan-backed command inherits this: baseline save, baseline check, triage, verdict, hotspots, ignore: all pay the full scan.
severity: blocker

subtree	source files	wall
`products/surveys`	62	4.4 s
`products/metrics`	90	5.2 s
`products/error_tracking`	407	8.4 s
`ee`	906	9.5 s
`frontend`	5,361	> 600 s (killed at 10 min)
whole repo	32,837	> 6,000 s and still running

The super-linear cost is on the JavaScript side, not Python

repo: posthog
command: crimes scan <subtree> --all --format json --no-init
what happened:

subtree files dominant language wall
posthog/ 4,759 Python 55 s
frontend/ 5,361 TypeScript 1,245 s (20 m 45 s)

Near-identical file counts, 22.6× difference. The Python pack scales fine. Whatever is quadratic lives in the JS pack or in the cross-file indexes it feeds (function-hash / JSX-shape / import graph / risk index). PostHog has 15,133 JS/TS files (2.8× frontend/) which is consistent with the whole-repo scan still running after 1 h 53 m.
why it matters: this narrows the fix to one side of the codebase and means the 0.14 Python pack is not the performance problem, despite being the newest pack.
severity: blocker

subtree	files	dominant language	wall
`posthog/`	4,759	Python	55 s
`frontend/`	5,361	TypeScript	1,245 s (20 m 45 s)

Root cause of the blow-up, named: `exact_duplicate_block` + `near_duplicate_block`

repo: posthog
command: node --cpu-prof … crimes scan frontend/src/lib --all --format json --no-init (1,430 JS files, 69 s: vs frontend/ 5,361 files, 1,245 s: 3.75× the files, 18× the time ⇒ ~O(n²·¹))
what happened: two functions account for 36.1% of sampled CPU (12.41 s + 12.23 s of a 68.2 s profile), plus another 9.9% in their enclosing run. They are the anonymous comparator inside:
- packages/core/src/detectors/exact-duplicate-block.ts
- packages/core/src/detectors/near-duplicate-block.ts
Both are per-file detectors that do this in run(ctx):
```
for (const [hash, hits] of [...ctx.functionHashIndex.byExact].sort(
  (a, b) => a[0].localeCompare(b[0]),
)) {
  …
  const anchor = [...distinctFiles].sort()[0]!;
  if (anchor !== ctx.file) continue;   // ← discards ~everything
}
```
The entire global function-hash index is materialised into an array and re-sorted with localeCompare once per file, and then all but the handful of groups anchored on the current file is thrown away. That is O(files × functions × log functions) with localeCompare (one of the slowest string comparators in V8) in the inner loop. near_duplicate_block additionally calls anyExactKeyShared against byExact inside that loop (another 0.9%). PostHog yields ~150k Python functions and a comparable number of JS ones, over 15,133 JS files.
fix shape (not applied, per protocol): group the index by anchor file once when the index is built, and have each detector look up byAnchor.get(ctx.file). Sorting for determinism can happen once, not once per file. This should be a straight O(n) rewrite with identical output.
why it matters: this single pattern is the difference between a 4-minute scan and a >2-hour one on a large monorepo, and it is entirely mechanical to fix.
severity: blocker

Write-command results (step 6)

Exercised in the disposable clone. The repo-root baseline save was still running after 56 minutes, so the full suite was exercised at products/metrics (100 source files) where each command completes in seconds. Every command below was run for real and its artefacts captured in ~/crimes-dogfood/artifacts/posthog.writecmds.txt.

command	result
`init`	ok (but see the amputation entry above)
`baseline save`	ok: “Recorded 33 findings”
`baseline check`	ok exit 0: but the arithmetic doesn’t close
`ignore <fp> --reason …`	ok, wrote well-formed `.crimes/suppressions.json`
`audit-suppressions`	ok, clear output
`unignore <fp>`	ok, removed the entry
`triage --list`	ok
`triage --format json`	exit 2: refuses interactive in non-TTY (correct)
`triage --apply <array>`	exit 2: schema undocumented, see below
`triage --apply <document>`	ok
`feedback <fp> --verdict known --note …`	ok
`feedback list` / `summary` / `recheck`	all ok
`hotspots`	ok, genuinely useful ranking
`verdict`	exit 2: no `main`/`origin/main` on a detached pinned SHA
`context <file>`	ok
`diff HEAD~5...HEAD`	ok

`baseline save` and `baseline check` disagree by 3 on an unchanged tree

command: crimes baseline save . then immediately crimes baseline check .
what happened: save reported 33 findings; the very next check on an untouched tree reported New 0 / Fixed 0 / Unchanged 30. 33 ≠ 30. Cause: fingerprints are not unique. In .crimes/baseline.json, 33 findings collapse to 30 distinct fingerprints: weak_test_signal::frontend/metricsIngestionLogic.test.ts:: appears 3 times and weak_test_signal::frontend/components/metricsUsageTrackingLogic.test.ts:: twice, all with an empty symbol segment.
why it matters: the 0.17.0 “discriminator fingerprints” work does not cover weak_test_signal. Three consequences: baseline arithmetic silently loses findings; crimes ignore <fp> suppresses all three at once with no warning; crimes feedback <fp> records one verdict for three different findings. Fingerprints are the identity key for every stateful command, and they collide.
severity: real problem

`--show-suppressed` includes the finding but never annotates it

command: crimes scan . --no-init --no-color --show-suppressed
what happened: the suppressed God File on frontend/components/metricsViewerLogic.tsx reappears, but with no suppression marker anywhere in the output: the flag’s own help text promises “annotated as suppressed”. Worse, the footer still prints 1 finding suppressed; run with --show-suppressed to see. while you are running with --show-suppressed. There is no way to tell which of the listed findings is the suppressed one.
severity: real problem

`--show-triaged` likewise shows no disposition

command: crimes scan . --show-triaged after applying a wont-fix
what happened: help says “annotated with disposition + reason + owner + date”. Grepping the whole output for wont-fix, triag, or the owner returns nothing. The finding renders identically to any other.
severity: real problem

`triage --apply` requires an undocumented strict schema

command: crimes triage --apply /tmp/triage.json
what happened: the obvious payload (an array of entries) is rejected with (root): Invalid input: expected object, received array. The accepted shape is a full document: schema_version, report_type, created_at, updated_at, entries[], each entry requiring fingerprint, type, file, disposition, reason, owner, date (YYYY-MM-DD), .strict() so any extra key fails. None of this is in --help, and the error names only the first violation. I had to read packages/core/src/triage.ts to construct a valid file.
why it matters: --apply is the only non-interactive path into triage, i.e. the only one an agent or CI job can use, and it is undiscoverable from the CLI. An --example/--init emitter, or a tolerant “bare array” form, would close this.
severity: real problem

`triage --apply` silently writes a `feedback.jsonl` entry too

what happened: after applying one triage entry, crimes feedback list showed two entries: mine, plus a [known] entry auto-created from the triage disposition, using the triage reason as the feedback note. Nothing in the triage --apply output mentions writing to the feedback log.
why it matters: the feedback log is the calibration input. Triage dispositions leaking into it as known verdicts silently biases the corpus, and a user reviewing feedback list sees entries they never recorded.
severity: real problem

`feedback <fingerprint>` accepts fingerprints that don’t exist

what happened: I recorded a verdict against cross_language_type_drift::backend/facade/enums.py::MetricAggregation: a finding that no longer exists under the current config (Python is excluded). It was written with no warning and no validation.
severity: annoyance (arguably correct for offline workflows, but a typo’d fingerprint is unrecoverable and silent)

`crimes diff` ignores the local `crimes.config.json`

command: crimes diff HEAD~5...HEAD
what happened: reported Unchanged crimes: 72 in a tree where crimes scan reports 33. The config is untracked, so scanning a git ref never sees it and silently falls back to defaults. Nothing in the output explains the 2× discrepancy.
severity: real problem

`context` “Related files” reasoning is vacuous in a deep tree

command: crimes context frontend/components/metricsViewerLogic.tsx
what happened: all five related files are justified with same directory; shares domain token "frontend"; matches domain "frontend". frontend is a path segment of the scan root, not a domain token. The same three clauses repeat for every entry, so the section carries no information beyond “same directory”.
severity: annoyance

Step 7: reading the default human view as a first-time user

The whole-repo human scan had not returned after 2 h 45 m, so this is the default view of the posthog/ Django package alone (4,776 files):

CRIME SCENE REPORT
repo: posthog · 4497 findings across 1963 files · 603 high 2473 medium 1421 low

The default view is a wall of God Function

what happened: the default (non---all) view shows 5 files and 107 findings. The top file, dags/events_backfill_to_duckling.py, lists 31 numbered findings of which 23 are “God Function”: items 2 through 11, then 13, 15–20, 22–25: each a near-identical line reading N lines (threshold 50 for shape "domain"), nesting depth D. Six more are “Commented-Out Code”. The second file, models/integration.py, is 22 findings of which 18 are God Function.
the lede is buried: the single most interesting thing on that page is models/integration.py item 2: 36 blocking requests.* calls inside one OAuth function: and it renders as a ~600-character evidence blob enumerating every call site before truncating at …+26 more. Right below it, Temporal Recklessness: 21× time.time(). Those two are real, specific, and actionable. They are sandwiched between twenty restatements of “this function is long”.
the God Function threshold is the problem, not the finding: 1,822 of 4,497 findings (40.5%) are large_function, firing at 50 lines for Python “domain” shape. A 52-line Django method is unremarkable; item 15 in the top file is literally run() 52 lines (threshold 50). Two lines over is not a crime and it crowds out everything that is.
honest reaction: I would not use this output. There is no way to read it top-to-bottom and come away with an action, because the ranking optimises for concentration of findings per file and the most concentrated findings are the cheapest ones. Collapsing repeated charges per file into one line (“23 God Functions, longest 235 lines”) would free the page to lead with the blocking-IO and time.time() findings that actually differentiate this tool.
severity: real problem

Whole-repo finding tables (from tractable slices)

The whole-repo scan never returned, so these are the two largest slices, each scanned end-to-end with the shipped binary.

slice	files	findings	wall	max RSS
`posthog/` (Django backend)	4,776 (4,727 py)	4,497	76 s	433 MB
`frontend/` (Kea/React app)	5,358 (5,339 js)	6,490	1,140 s	689 MB

10,134 files → 10,987 findings ≈ 1.08 findings per file. Extrapolated across 32,837 discovered source files that is ~35,000 findings for the repo.

Zero-case, stated positively

posthog/ slice (Python only): all three cross_language_* detectors produced 0 findings. Correct.
frontend/ slice (JS only): all three cross_language_* detectors produced 0, and the Python pack produced 0. Correct.
But both slices report coverage.packs_loaded = ["universal","language-js","language-py"] regardless. packs_loaded is a static list of registered packs and carries no information about what ran. On the frontend/ slice it names a pack that saw zero files.
coverage.by_package was absent on both slices (neither has ≥2 nested manifests), so the 0.15 surface went unexercised at slice scope.

WHOLE-REPO SCAN COMPLETED: 3 h 42 m

/usr/bin/time -l crimes scan . --all --format json --no-init

13321.56 real   10509.01 user   491.22 sys
maximum resident set size   1,704,280,064   (1.70 GB)
peak memory footprint       2,358,501,360   (2.36 GB)
EXIT_CODE=0     WALL_SECONDS=13322

Output: 56.2 MB of JSON, 38,063 findings, exit 0, nothing on stderr. Coverage: files_total 35,108 · files_by_language {py: 17662, js: 15055} · files_universal_only 2,391 (.rs 1342, .md 801, .go 125, .sh 76, .mdx 22, .cpp 12, .h 8, .c 4, .rb 1) · packs_loaded ["universal","language-js","language-py"] · by_package present with 209 entries and it is correct: it picks up products/warehouse_sources (6,281 files, dominant py), frontend (5,355, js), nodejs, services/mcp, the products/desktop/packages/* tree, and correctly returns dominant_language: null for genuinely mixed packages (products/posthog_ai 149 py / 201 js, products/metrics 48/37). Rust crates appear with files_by_language: {}: right answer, they are universal-only.

The cross-language answer, from the real scan

detector	findings
`cross_language_route_drift`	1
`cross_language_type_drift`	22
`cross_language_concept_alias_drift`	0

This matches the parser probe exactly (probe predicted 1 / 22 / 0), which confirms the diagnosis above is the whole story.

The single route-drift finding is anchored on frontend/src/layout/navigation/ImpersonationNotice/adminLoginAs.ts:4, severity high, and reads “41 frontend calls target a path that no backend route declares”. Its evidence ends with 28 backend routes declared across 10 files: all 28 from services/llm-gateway and services/stripe-mock. Five of the eight quoted call sites are frontend/src/lib/api.test.ts lines asserting on deliberately invalid URLs (/api/projects/null, /api/projects/089908?x). It is a high-severity, zero-value finding.

The 22 type-drift findings are the opposite: substantive. The best of them is IntegrationKind: posthog/schema_enums.py:2829 vs products/desktop/packages/api-client/src/generated.ts:5185, drifting in both directions (9 Python-only values, and slack-posthog-code only in TypeScript). ChartDisplayType, TaxonomicFilterGroupType, WebStatsBreakdown, PropertyOperator are the same shape.

`cross_language_concept_alias_drift` fired zero, and here is why

PostHog uses every alias in the built-in tenant group on both sides: team, workspace, organization, account, tenant, company all appear as Python class/def names and as TS function/type names. The detector requires onlyPy.length > 0 && onlyJs.length > 0, i.e. a clean vocabulary split. A large mature product uses all the words everywhere, so the clean-split condition is essentially unsatisfiable at scale. This detector may only ever fire on small codebases.

21 of 69 registered detectors fired zero times on PostHog

action_label_drift, boundary_bypass, client_exposed_secret, client_reachable_secret, command_drift_docs_code_drift, copy_ia_drift, cross_language_concept_alias_drift, default_disagreement, duplicated_navigation_source, finder_duplicate_filename, layer_violation (config-driven, expected), magic_domain_literal_scatter, missing_agent_context, orphaned_destination, parallel_destination, permission_ia_drift, requiredness_disagreement, route_metadata_drift, type_disagreement, undocumented, unit_disagreement.

`magic_domain_literal_scatter` is non-monotonic: 194 on a subtree, 0 on the repo

what happened: scanning frontend/ alone produces 194 magic_domain_literal_scatter findings; scanning products/metrics alone produces 3; scanning the whole repo, which strictly contains both, produces 0.
why it matters: a detector whose output disappears as you add files is measuring the corpus, not the code. Something in it is a ratio or percentile against total corpus size. A user who scans a package and then scans the monorepo will see findings vanish with no explanation, and crimes diff / baseline check will read that as “fixed”.
severity: real problem

Full finding table: whole repo, 38,063 findings

By pack: language-py 18,690 · language-js 13,543 · universal 5,807 · cross-language 23. By severity: high 9,177 · medium 16,703 · low 12,183.

Top types: large_function 9,899 · deep_import 9,448 · large_file 4,065 · boolean_naming_drift 3,454 · weak_test_signal 1,759 · high_fan_in_fan_out 1,435 · swallowed_error 1,417 · direct_date 1,260 · commented_out_code 1,189 · sync_io_in_hotpath 833. Four detectors produce 70% of all output. deep_import alone is 9,448 findings, 100% from the Python pack, and PostHog’s import depth is a deliberate products/<name>/backend/… convention: it is charging the repo for its own directory layout.

Every scan-backed command is unusable at this size

baseline save . at the repo root was still running after 2 h 34 m (and that is with the crimes init config in place, i.e. JS-only, half the corpus). crimes scan, hotspots, verdict, diff, triage, ignore and baseline check all pay the same cost. A pre-commit hook or CI gate is out of the question.

drf

Friction log: drf (Django REST Framework)

Repo: ~/crimes-dogfood/corpus/drf @ 0e25e53582df802da4fcc329bd57e4ba5f61010b Binary: ~/crimes-dogfood/bin/crimes (crimes 0.17.0)

Size at scan time: 154 .py (40,450 LOC), 127 .png, 77 .md, 60 .po/.mo, 53 .html, 8 .js, 8 .css. 81 of the 154 .py files are tests/conftest (25,065 LOC = 62% of Python LOC).

Corpus clone was broken on arrival (environment, not crimes)

repo: drf
command: git -C ~/crimes-dogfood/corpus/drf status --short
what happened: The clone had an empty git index (0 files) and only 15 files on disk: rest_framework/ and tests/ were entirely absent. Cause: git-lfs is not installed on this machine, and git checkout aborts the whole worktree write at docs/theme/src/drf-logos.fig (an LFS pointer): git-lfs filter-process: git-lfs: command not found … fatal: smudge filter lfs failed. The abort left the checkout half-done and the index blown away.
fix applied: set filter.lfs.{smudge,clean}=cat, filter.lfs.process="", filter.lfs.required=false in the clone’s local config, then git reset --hard <SHA>. Restored 154 .py files.
why it matters: Not a crimes bug, but a corpus-integrity one. Had I scanned as-received, crimes would have reported ~0 findings on a repo with 0 source files and looked beautifully calibrated. Any dogfooding round needs a source-file count assertion before it trusts a low finding count.
severity: blocker (for the round, until fixed)

BLOCKER: `crimes scan` on a nonexistent path exits 0 and reports “Suspiciously clean”

repo: drf (reproduces anywhere)
command: crimes scan ~/crimes-dogfood/corpus/drff --no-init ← note the typo
what happened:
```
CRIME SCENE REPORT
repo: drff  ·  0 findings

No crimes detected. Suspiciously clean.
```
Exit code 0. Nothing on stderr. --format json is equally confident: summary.total = 0, coverage.files_total = 0, and packs_loaded: ["universal","language-js","language-py"]: a full, well-formed, apparently-successful report for a directory that does not exist. crimes scan /no/such/path behaves identically.
why it matters: this is the most dangerous output the tool can produce, because it is indistinguishable from a genuine pass and it praises you for it. It fails in the direction of false confidence.
- It would have silently absorbed the broken clone logged above: had I not counted .py files by hand first, drf would have scored “0 findings” and been written up as crimes’ best control result of the round.
- coverage.files_total: 0 is the tell, but it is buried in JSON and absent from the human view entirely: the human renderer never prints how many files it looked at.
- CI wrapping crimes scan as a gate passes green forever on a path typo, a moved directory, or a bad --root.
- Note ~/crimes-dogfood/artifacts/zulip.json and ortomate-studio.json are 0 bytes from earlier sessions in this same round. Worth checking whether they are the same class of silent no-op.
contrast: crimes verdict --base <bad-ref> and crimes explain <bad-id> both get this right: clear stderr message, exit 2. scan is the outlier.
fix shape: error + non-zero exit when the root does not exist or is not a directory; and print files scanned: N in the human header so 0 is visible. Both patch-level.
severity: blocker

`large_file` evidence says “N non-empty lines” but N is total lines + 1. 127/127 wrong.

repo: drf (18/18) and pydantic (109/109): every single large_file finding in the round
command: crimes scan <repo> --all --format json --no-init
what happened: the finding for rest_framework/decorators.py reads:

File is 301 lines (threshold 300) … evidence: "301 non-empty lines", "1.0× the configured 300-line threshold"

Ground truth: wc -l = 300, and non-empty lines = 223. The claimed number is source.split("\n").length, i.e. total lines plus the phantom empty string after the trailing newline. Verified across all 127 findings in both repos: claimed value is totalLines + 1 every time, and the string “non-empty lines” is false every time.

it is not just cosmetic: it manufactures findings. 13 of the 127 large_file findings are on files that are under the 300 threshold on the metric the evidence claims to measure:

repo	file	claimed	real total	real non-empty
drf	`rest_framework/decorators.py`	301	300	223
drf	`rest_framework/permissions.py`	315	314	237
drf	`rest_framework/utils/field_mapping.py`	329	328	268
drf	`rest_framework/validators.py`	355	354	295
pydantic	`pydantic/_internal/_dataclasses.py`	316	315	253
pydantic	`pydantic/v1/env_settings.py`	351	350	290
pydantic	`pydantic/_migration.py`	318	317	298
pydantic	`pydantic-core/src/tools.rs`	321	320	279
pydantic	`.../validators/decimal.rs`	306	305	277
pydantic	`.../validators/validation_state.rs`	307	306	261
pydantic	`.../validators/shared/lookup_tree.rs`	311	310	278
pydantic	`.../serializers/mod.rs`	314	313	296
pydantic	`.../input/input_string.rs`	349	348	299

decorators.py is 26% under the threshold. Docstring- and blank-line-rich Python is exactly the style this hits hardest: the gap between total and non-empty runs 20–35% in both repos.

the sharpest version of it: crimes diff HEAD~5...HEAD on drf reports exactly one new crime in five upstream commits. That one crime is large_file on decorators.py: a file that is not large. The single highest-signal output the tool produced on this repo is an artefact of this bug.
large_function is unaffected: spot-checks match the source.
fix shape: either count non-empty lines as advertised, or change the string. Counting non-empty removes 13 findings and re-scores 127: a patch bump + eval re-run.
severity: real problem (highest-value fix in my two repos)

`crimes diff` human output reports counts and nothing else

repo: drf
command: crimes diff HEAD~5...HEAD --no-init
what happened: entire output was New crimes: 1 / Fixed crimes: 0 / Unchanged crimes: 99. It does not say what the new crime is. You have to re-run with --format json and read new_findings[] to learn it was large_file on rest_framework/decorators.py. With one new finding there is no length excuse for eliding it.
why it matters: diff is the PR-gate command (--fail-on new-high). Failing a build with a count and no location is the least useful place to omit detail.
severity: real problem

`large_file` on Markdown (0.17.0 `docs` shape): flags an auto-generated changelog

repo: drf / pydantic
what happened: docs/community/release-notes.md (2708) and docs/api-guide/serializers.md (1232) in drf, plus .md files in pydantic, are charged “God File” with fix_shape “split by responsibility; one concern per module”. release-notes.md is an append-only changelog; it grows by one section per release forever and splitting it is not a thing anyone would do.
severity: annoyance (characterisation for the judge, not a verdict)

Positive: `hotspots`, `explain`, `verdict` error paths, determinism

hotspots was the most immediately readable output on both repos and needed no explanation. (Minor: drf’s #6 hotspot is docs/img/api-docs.gif with “0 changes”, and #5 is a test file.)
explain output is genuinely good: detector rationale, evidence, the churn/test-gap/blast breakdown, and remedies including “configure the detector or record feedback instead of renaming code blindly”.
crimes verdict --base origin/master (a ref that does not exist in either repo) printed a clear message and exited 2. crimes explain crime_99999 likewise exited 2. Both correct: which is what makes scan’s exit-0 on a nonexistent path stand out.
Three consecutive scan --all --format json runs on pydantic were byte-identical to each other and to the artifact captured earlier.

Friction log: dogfooding round 0.14 → 0.17

orchestrator

Friction log: orchestrator (round setup + cross-repo observations)

packs_loaded reports what was loaded, not what ran

evidence conflates code evidence with score arithmetic

crimes is not installable as a global for testing without publishing

--all is a byte-for-byte no-op in --format json

Control result: p-limit is not silent

CONFIRMED BUG: large_function (Python pack) reports a size claim it just disproved

CONFIRMED BUG: pass_through_abstraction builds chains from name collisions

sync_io_in_hotpath has no working notion of “hotpath”

boolean_naming_drift flags names that cannot be renamed

swallowed_error does not read the comment that explains the catch

SYSTEMIC: detectors with OR-triggers print evidence for the condition that did NOT trip

weak_test_signal does not follow assertion helpers

Vendored and framework-owned code is treated as domain code

VERIFIED BLOCKER: source under a dot-directory is silently skipped, and reported as clean

VERIFIED BLOCKER: crimes context ids do not resolve in crimes explain

cross_language_route_drift can only ever see a Python backend

agent_risk has collapsed into size: a stated product invariant

VERIFIED BLOCKER: the 0.17.0 fingerprint fix did not fix the bug class

crime_NNNNN ids are not stable across runs of an unchanged repo

TEST_FILE_RE misses test.js: the ava / node:test default filename

The docs shape and the data-format exclusion are unreachable at default config

test_gap: 0.5 means “no signal” but renders as a confident causal sentence

explain exits 2 on a finding the scanner itself emitted

hotspots ranks manifest churn as the top risk, over a different file set

NOTE FOR ANYONE READING THE ARTIFACTS

VERIFIED BLOCKER: large_file counts total lines + 1 and calls it “non-empty lines”

VERIFIED BLOCKER: crimes scan on a nonexistent path exits 0 and reports clean

VERIFIED BLOCKER: test-scope classification is structurally inert on Python

PEP 696 type-parameter defaults fail to parse; one form loses the whole file

hasSyntaxErrors is computed and then never surfaced

commented_out_code matches English prose, including Rust doc comments

Corpus-integrity lesson: a broken clone looks exactly like a clean repo

mixed_utc_local_methods fired zero on 296K lines of Python

VERIFIED BLOCKER: .gitignore is not respected

VERIFIED BLOCKER: silent, nondeterministic finding loss under fd pressure

churn and recency score 0.00 on a quiet repo and render as a claim about the code

Positive results worth recording

VERIFIED BLOCKER: dependency_provenance_gap checks Python imports against npm manifests

VERIFIED BLOCKER: blast_radius_importers prints a component size as a per-file count

Cross-language: diagnosed, on the second ideal input

749 files of Zulip’s rendered UI were dropped at discovery, silently

docs_code_drift: 12 of 12 verified false positives

Quantified: large_function self-refutes on 19% of its Python findings

The positioning check, measured

VERIFIED BLOCKER: no size or line-length guard before parsing: 135x slowdown

.gitignore: confirmed absent at source, and on a second repo

Absent signals are scored as zeros and rendered to the user as good news

The honesty gap is one line of output wide

packs_loaded is a constant: confirmed at source

hotspots is the honest command and scan is the one the docs push

The default report is not a summary

ebg

Friction log: ebg

Python pack loaded but zero .py files were ever walked

Coverage accounting does not add up to the files actually analysed

No coverage.by_package at all (0.15 surface absent)

All three cross_language_* detectors fired zero times

ROOT CAUSE: the file walker is dot: false, so any source under a hidden directory is invisible

--explain-coverage prints a pack as “loaded” and then omits it entirely

Cross-language verdict, with evidence (counterfactual scan)

cross_language_route_drift is blind to the route contract this repo actually has

Python parse robustness: PASS, with positive evidence

BUG: sync_io_in_hotpath is a file-level finding wearing a function’s name and a fabricated line range

large_function double-counts a function and the callback that is its whole body

crimes verdict takes 53 seconds to say “UNCHANGED”

crimes hotspots is 27s and disagrees with scan about how many files exist

crimes context prints 270 lines for one file

crimes context prints finding ids that resolve to different findings under crimes explain

crimes explain on an unknown id exits 0

crimes diff (human) takes 62s and refuses to say what the new crimes are

high_fan_in_fan_out: the summary and the score disagree about the importer count

CORRECTION to the by_package entry above: it is gated, not regressed: but the gate excludes real polyglot repos

Honest read of the default human scan as a user

agent_risk collapses into “big file / big function” on a real repo

Positive statements (the zero case)

Detector coverage on ebg

Follow-up on the deep_import zero: mostly justified

`packs_loaded` reports what was loaded, not what ran

`evidence` conflates code evidence with score arithmetic

`crimes` is not installable as a global for testing without publishing

`--all` is a byte-for-byte no-op in `--format json`

CONFIRMED BUG: `large_function` (Python pack) reports a size claim it just disproved

CONFIRMED BUG: `pass_through_abstraction` builds chains from name collisions

`sync_io_in_hotpath` has no working notion of “hotpath”

`boolean_naming_drift` flags names that cannot be renamed

`swallowed_error` does not read the comment that explains the catch

`weak_test_signal` does not follow assertion helpers

VERIFIED BLOCKER: `crimes context` ids do not resolve in `crimes explain`

`cross_language_route_drift` can only ever see a Python backend

`agent_risk` has collapsed into size: a stated product invariant

`crime_NNNNN` ids are not stable across runs of an unchanged repo

`TEST_FILE_RE` misses `test.js`: the ava / node:test default filename

The `docs` shape and the data-format exclusion are unreachable at default config

`test_gap: 0.5` means “no signal” but renders as a confident causal sentence

`explain` exits 2 on a finding the scanner itself emitted

`hotspots` ranks manifest churn as the top risk, over a different file set

VERIFIED BLOCKER: `large_file` counts total lines + 1 and calls it “non-empty lines”

VERIFIED BLOCKER: `crimes scan` on a nonexistent path exits 0 and reports clean

`hasSyntaxErrors` is computed and then never surfaced

`commented_out_code` matches English prose, including Rust doc comments

`mixed_utc_local_methods` fired zero on 296K lines of Python

VERIFIED BLOCKER: `.gitignore` is not respected

`churn` and `recency` score 0.00 on a quiet repo and render as a claim about the code

VERIFIED BLOCKER: `dependency_provenance_gap` checks Python imports against npm manifests

VERIFIED BLOCKER: `blast_radius_importers` prints a component size as a per-file count

`docs_code_drift`: 12 of 12 verified false positives

Quantified: `large_function` self-refutes on 19% of its Python findings

`.gitignore`: confirmed absent at source, and on a second repo

`packs_loaded` is a constant: confirmed at source

`hotspots` is the honest command and `scan` is the one the docs push

Python pack loaded but zero `.py` files were ever walked

No `coverage.by_package` at all (0.15 surface absent)

All three `cross_language_*` detectors fired zero times

ROOT CAUSE: the file walker is `dot: false`, so any source under a hidden directory is invisible

`--explain-coverage` prints a pack as “loaded” and then omits it entirely

`cross_language_route_drift` is blind to the route contract this repo actually has

BUG: `sync_io_in_hotpath` is a file-level finding wearing a function’s name and a fabricated line range

`large_function` double-counts a function and the callback that is its whole body

`crimes verdict` takes 53 seconds to say “UNCHANGED”

`crimes hotspots` is 27s and disagrees with `scan` about how many files exist

`crimes context` prints 270 lines for one file

`crimes context` prints finding ids that resolve to different findings under `crimes explain`

`crimes explain` on an unknown id exits 0

`crimes diff` (human) takes 62s and refuses to say what the new crimes are

`high_fan_in_fan_out`: the summary and the score disagree about the importer count

CORRECTION to the `by_package` entry above: it is gated, not regressed: but the gate excludes real polyglot repos

Honest read of the default human `scan` as a user

`agent_risk` collapses into “big file / big function” on a real repo

Follow-up on the `deep_import` zero: mostly justified

`.gitignore` is NOT respected: a gitignored build-output directory contaminates 21% of all findings

Human reporter truncates `evidence` at 2 items: and for duplicates the dropped item is the only useful one

251 of 1556 findings (16%) have no `lines` field at all

`coverage.files_by_language` collapses TS/TSX/JS into one bucket called `js`

`churn` is 0.00 on all 1556 findings, and only `explain` ever tells you why

`hotspots` default 90d window makes the command actively misleading: and its top hit is a PNG

`crimes diff` human output is three integers and names nothing

`crimes verdict` spends 68 seconds to compare a commit with itself and doesn’t mention that

`crimes context` is the best command in the tool

The “stable fingerprint” is not in the JSON, and `explain --help` documents the wrong shape for a sixth of findings

Honest read of the default `scan` output, as a user

`.gitignore` bug: minimal reproduction (6 lines, outside any real repo)

The human `scan` took 1.8x longer than the `--all` JSON scan

First human `scan` was killed with exit 143 at 22.8s

* ZERO cross-language findings on a textbook client/server repo *

`packs_loaded` cannot tell you whether the cross-language pack ran

`coverage.by_package` is absent on a textbook polyglot repo

`contract_drift` summaries name the same identifier twice and read as nonsense

`contract_drift` matched three pairs of local variables both called `parsed`

CORRECTION to the probe above: `parsePyFile` is async; my first two probes silently read a Promise

Corrected cross-language numbers (all with `await`)

`large_function.py` charges “God Function” on a clean 9-line function, and its own evidence says the threshold was not exceeded

`blast_radius` is saturated for half the report, and the human output prints a number a user can disprove in 5 seconds

`commented_out_code` charges Zulip for having the best explanatory comments in the corpus

`todo_density` fires below its own stated threshold 62% of the time

`context`’s “Related files” is `ls` output, alphabetical

`context` on Python otherwise works and is genuinely agent-shaped

`dependency_provenance_gap` applies the npm manifest model to Python and charges Zulip HIGH for importing the standard library