Calibration · latest run

How well does pkgxray call it?

An at-scale, code-only static scan of published packages, measured against a committed known-malware corpus. Four numbers, adjudicated by hand, reproducible from the inputs below.

5,000

packages scanned

published npm packages, one static pass · source data

0.0%

false blocks, top 1,000

0 of the 1,000 most-downloaded packages · source data

95.2%

known-malware catch rate

20 of 21 corpus samples blocked outright · source data

2026-07-21

run date

pkgxray 1.0.6 · build 76c0768 · source data

Methodology & how to reproduce → · Raw JSON · Top-1000 target list

Aggregate calibration on the 1.0.6 engine. packagesScanned counts the distinct popular packages validated this cycle: the top-1000 calibration list plus two deeper, non-overlapping download-ranked sweeps (1,000 and 3,000). The top-1000 false-block figure is the re-measured heuristic count on the fixed engine (0/1000). Catch rate is against the committed reconstructed known-malware corpus (npm removes live malware, so live recall is untestable). This run's scan environment had the OSV endpoint unavailable, so it is heuristic-scoped; the false-block metric is heuristic-only by definition and is unaffected, and known-CVE blocks are by-design true positives, never false blocks.

What the 5,000 counts: 1,000 top-1000 calibration list + 1,000 deeper download-ranked sweep (first) + 3,000 deeper download-ranked sweep (second). Counted separately, not in this denominator: 300 MCP cohort, 21 reconstructed known-malware corpus.

Corrections

Published runs are immutable. If a number is wrong we publish a new dated run and note it here — we never silently edit a number in place. Contest a figure at the tracker.

2026-07-21 — Widened validation past the top-1000 calibration list into two deeper, non-overlapping download-ranked sweeps (1,000 and 3,000 packages) to hunt false blocks the head never exercised. The deeper sweeps surfaced 16 wrongly-blocked popular packages: 14 were genuine heuristic false positives across six detectors (OID/version dotted-quads read as IPs, benign public DNS, HTTP-cookie identifiers read as the browser cookie DB, the metaMask web3 connector name read as a wallet read, shell-rc paths named in doc/comments, Sentry's getTransaction read as an on-chain loader, and env/require co-location inside bundled megafiles), and 2 (bun, firebase-tools) genuinely perform the flagged high-risk operation and are recorded as defensible true positives. Every fix strictly relaxes a heuristic, so nothing new can block. The top-1000 calibration list was then re-scanned on the fixed engine (1.0.6): 0 / 1,000 heuristic false blocks, unchanged.
2026-07-26 — Metadata-only clarification (no measured number changed): (1) recorded that engine 1.0.6 (commit 76c0768) was a pre-release build never published to npm — the public npm release at run time was 1.0.4 — and added a commit-checkout reproduction command, since `npx pkgxray@1.0.6` would not resolve; (2) added the explicit 5,000-package composition (top-1000 calibration list + two deeper download-ranked sweeps of 1,000 and 3,000) and recorded that the 300-package MCP cohort and the 21-sample reconstructed known-malware corpus are separate sets, not part of the 5,000 denominator; (3) noted that the 4,000 extended-sweep targets were selected from the committed download-rank pool but the resolved snapshot was not committed.

Run history

Every published run stays up at a stable URL.

Pinned version: this page is the 2026-07-21 snapshot at pkgxray 1.0.6 · 76c0768. Engine 1.0.6 was a pre-release build, not published to npm; the public npm release at run time was 1.0.4. Latest run →