Inspect the evidence. Measure what actually works.
ShipCheck now separates advisory predictions from demonstrated regressions, makes incomplete verification visible, and provides a comparison lab for real, source-backed measurements.
Advisory predictions, static rules, demonstrated regressions, pre-existing failures, and incomplete checks retain distinct labels through scan reports and GitHub summaries. Observation-only contracts remain nonblocking.
The installed Vitest workflow checks immutable revisions, records actual assertions and failures, and requires matching test identity. Changed tests, setup problems, missing reports, and unsupported execution remain inconclusive.
Repository owners can select the stored claims a linked test actually exercises. AI plans and matching text no longer stand in for runtime coverage. Verifier changes become captured contract versions.
Reports retain verification coverage, omitted guards, engine configuration, and the actual comparison base. Guard-loading and evidence-persistence failures cannot silently produce a clean verification result.
Add reviewed historical cases and real tool observations, compare shared cases across configurations, test memory-enabled and memory-disabled reviews, and export JSON or spreadsheet-safe CSV. Version history, concurrent-save protection, and recoverable form drafts preserve the study.
On one public Zod change with two documented regression causes and one within-change negative control, both advisory-review variants identified one cause and missed the other. Memory did not improve detection in this single-run pilot. The reports, label-validation evidence, limitations, timings, and token usage are published; no competitor was run.
Measured comparisons read a separately reviewed public dataset. Saving a private study does not publish repository links or evidence. Missing measurements stay missing, and software test-suite results are reported separately from detection accuracy.
The verification and benchmark migration has been applied, including rollback-only tenant-isolation and revision checks. Verification and isolated public-label validation workflows are installed through protected pull-request checks.
Documentation covers the comparison lab, historical information boundaries, runtime prerequisites, supported execution, metric denominators, and publication. Regression and DOM tests cover verdict policies, evidence reports, metrics, private saves, CSV exports, and draft recovery.