Skip to content

What changed.Why it matters.

A running record of the systems ShipCheck adds to help engineering teams preserve behavior, verify evidence, and govern the path to production.

Built in public, one system at a time.

Every release strengthens the same loop: remember what matters, test the change, explain the evidence, and retain the outcome.

Inspect the evidence. Measure what actually works.

ShipCheck now separates advisory predictions from demonstrated regressions, makes incomplete verification visible, and provides a comparison lab for real, source-backed measurements.

Evidence strength stays separate from severity

Advisory predictions, static rules, demonstrated regressions, pre-existing failures, and incomplete checks retain distinct labels through scan reports and GitHub summaries. Observation-only contracts remain nonblocking.

Scoped base/head runtime evidence

The installed Vitest workflow checks immutable revisions, records actual assertions and failures, and requires matching test identity. Changed tests, setup problems, missing reports, and unsupported execution remain inconclusive.

Deliberate test-to-contract mappings

Repository owners can select the stored claims a linked test actually exercises. AI plans and matching text no longer stand in for runtime coverage. Verifier changes become captured contract versions.

Visible gaps and durable evidence

Reports retain verification coverage, omitted guards, engine configuration, and the actual comparison base. Guard-loading and evidence-persistence failures cannot silently produce a clean verification result.

A private comparison lab

Add reviewed historical cases and real tool observations, compare shared cases across configurations, test memory-enabled and memory-disabled reviews, and export JSON or spreadsheet-safe CSV. Version history, concurrent-save protection, and recoverable form drafts preserve the study.

Our first public historical pilot

On one public Zod change with two documented regression causes and one within-change negative control, both advisory-review variants identified one cause and missed the other. Memory did not improve detection in this single-run pilot. The reports, label-validation evidence, limitations, timings, and token usage are published; no competitor was run.

Publication with an explicit boundary

Measured comparisons read a separately reviewed public dataset. Saving a private study does not publish repository links or evidence. Missing measurements stay missing, and software test-suite results are reported separately from detection accuracy.

Live schema and workflow setup

The verification and benchmark migration has been applied, including rollback-only tenant-isolation and revision checks. Verification and isolated public-label validation workflows are installed through protected pull-request checks.

Updated guides and validation

Documentation covers the comparison lab, historical information boundaries, runtime prerequisites, supported execution, metric denominators, and publication. Regression and DOM tests cover verdict policies, evidence reports, metrics, private saves, CSV exports, and draft recovery.

An upgraded reasoning engine behind every review

We've upgraded ShipCheck's behind-the-scenes reasoning engine across pull request reviews, behavior verification, repository history, and regression-test generation.

Upgraded review and investigation

Code reviews, verification planning, and evidence investigations now run on the upgraded engine to reason through changes and their surrounding repository context.

Connected repository workflows

Historical-fix discovery, behavior-origin analysis, suggested fixes, and regression-test generation use the same upgraded engine.

Updated usage accounting

Token and cost tracking now reflect the upgraded engine, including cached context and reasoning usage.

A consistent workspace and public site

ShipCheck's interface now follows one dark, electric-blue design system across the product, with clearer navigation and more readable evidence.

Unified workspace navigation

A collapsible dashboard sidebar and shared top bar provide consistent access to overview, repositories, scan history, teams, and settings.

Readable scan and repository reports

Repository controls, branch comparisons, pull requests, scan evidence, and Production Sentinel screens share dark surfaces, stronger text contrast, and a 12px minimum text size.

Refreshed product pages

Features, pricing, documentation, comparisons, security, and policy pages use consistent typography and navigation. Changelog is now accessible from the main navbar.

Focused authentication

Login and account creation use focused GitHub sign-in forms with distinct repository-verification and behavior-memory visuals.

Govern every repository from one organization policy

Team Workspaces now provide a centralized organization control plane for repository access, Behavior Contract defaults, merge enforcement, approvals, and protected deployment environments.

Organization-wide enforcement

Apply criticality, enforcement, approval, GitHub merge-gate, waiver, and permission requirements across every shared repository in one versioned operation.

Repository-scoped RBAC

Switch an organization to restricted access and grant individual members read, contribute, or manage access for each repository.

Two-person contract approval

Organization policy changes can leave affected Behavior Contracts pending until a different administrator reviews the new immutable contract versions.

Deployment control policy

Declare protected environments and whether deployment integrations should observe, require verified behavior, or require verification plus approval.

Organization audit evidence

Policy versions, synchronized contract changes, access grants, actors, and resulting decisions are retained in the workspace audit history.

Turn production incidents into permanent protection

Production Sentinel incidents can now become reviewable learning proposals that strengthen the Behavior Contract responsible for preventing the same regression.

Incident learning proposals

Capture the cause, resolution, permanent invariant, and a focused regression scenario directly from an open, acknowledged, or resolved production incident.

One-step guard approval

Approve a proposal to append its invariant and regression scenario to the source Behavior Guard as a new immutable Behavior Contract version.

Atomic promotion

Contract changes, version history, approver identity, and the incident-to-guard link are committed together so partial protection cannot be recorded.

Permanent audit trail

Every proposal retains its incident snapshot, review status, resulting guard, and approved contract version for later investigation and compliance review.

Sentinel dashboard workflow

Recent incidents now expose capture, review, approval, and dismissal controls beside the runtime evidence that created them.

Purchase and manage paid plans with Stripe

Paid individual and team plans now connect pricing choices to Stripe checkout and subscription management.

Plan-aware checkout

Pricing buttons route new purchases into Stripe checkout and account for existing subscriptions when changing plans.

Subscription changes

Billing controls handle upgrades, downgrades, cancellation, and resuming a subscription through dedicated account workflows.

Team billing

Workspace owners can manage team subscriptions and seats from the shared workspace.

Protect behavior across repository boundaries

Team Workspaces can now map provider Behavior Contracts to the repositories that consume them, then carry downstream risk into every provider scan and ship verdict.

Cross-repository dependency map

Connect a provider contract to API consumers, SDK dependencies, event consumers, and data-contract consumers across repositories shared with a workspace.

Automatic downstream verification

Provider scans classify every mapped dependency as preserved, at risk, violated, or inconclusive from the contract evidence produced for that revision.

Risk-aware downstream findings

At-risk, violated, and inconclusive dependencies become scan findings, so cross-repository impact contributes to the final ship verdict instead of living in a separate diagram.

Shared dependency evidence

Workspace scan reports show the affected consumer repository, relationship type, status, message, and evidence used for each downstream decision.

Workspace-scoped governance

Dependency creation and removal use workspace roles, audit events, row-level security, and automatic cleanup when a repository is unshared.

Base-versus-head proof and shared Team Workspaces

ShipCheck can now execute the same protected scenario against both sides of a change and give engineering teams a shared place to govern repositories, contracts, policies, evidence, and decisions.

Differential Verification Lab

Eligible runtime Behavior Guards execute against the base and head commits so ShipCheck can distinguish a confirmed regression from a preserved behavior, pre-existing failure, improvement, or inconclusive run.

Paired execution evidence

Scan reports retain base and head status, commit SHAs, output, duration, and transition classification for the exact scenario that ran.

Team Workspaces

Create shared workspaces with owner, admin, member, and viewer roles; invite teammates by verified email; and share repositories without transferring repository ownership.

Shared contracts and policies

Teams can assign contract owners, save organization policy templates, apply governed settings, and require a second administrator to approve pending contract changes.

Shared reports and audit history

Workspace members can inspect verification across shared repositories while role changes, invitations, repository sharing, policy changes, and contract decisions remain auditable.

Team billing foundation

The backend now supports workspace-linked subscriptions, seat counts, pooled allowances, and billing-owner controls ahead of public Team checkout.

GitHub merge gates and audited exceptions

Risk-aware verdicts can now become enforceable GitHub checks, with repository policy controlling when a change passes, requires review, or is blocked.

GitHub merge checks

Completed pull-request scans publish a check conclusion derived from the underlying ShipCheck verdict and repository merge policy.

Repository enforcement policy

Repositories can choose whether review-required verdicts block, which GitHub permission may approve an exception, how long it lasts, and whether regulated contracts may be waived.

Time-bounded exceptions

Authorized maintainers can approve a reasoned temporary exception without changing or hiding the underlying Behavior Contract verdict.

Exception revocation and audit

Approvals, revocations, policy snapshots, verdict snapshots, actors, reasons, and expiry are retained as governance evidence.

Governed behavior verification and risk-aware ship verdicts

ShipCheck now turns protected behavior into a governed, versioned contract; investigates relevant changes with bounded evidence tools; and makes ship decisions from both code findings and the risk of the behavior at stake.

Behavior Contracts V2

Every Behavior Guard can now carry explicit lifecycle, criticality, enforcement, ownership, approval, evidence freshness, preconditions, invariants, scenarios, and observable outcomes.

Immutable contract versions

Contract changes create immutable snapshots with the responsible developer and change reason, preserving the exact expectation and governance policy that applied at each point in time.

Behavior Verification Planner

Relevant protected behaviors are converted into small, falsifiable verification plans with invariants, scenarios, expected outcomes, and requested evidence before verification begins.

Bounded read-only investigation

The verification agent can retrieve files, look up references, discover tests, and inspect historical evidence under a strict call budget without mutating the repository.

Governed evidence policy

Coverage scoring, citation validation, contradiction handling, source diversity, and contract-specific evidence requirements now determine whether a behavior can be called verified.

Explicit uncertainty

When runtime proof is unavailable or required evidence is missing, ShipCheck reports an explicit inconclusive result with remaining questions instead of presenting code presence as proof of behavior.

Risk-aware ship verdicts

Ship decisions now distinguish Ready to ship, Ready with uncertainty, Review before shipping, and Do not ship based on findings plus each affected contract's criticality and enforcement mode.

Risk-prioritized verification

The Behavior Impact Graph selects relevant contracts while critical and regulated behavior receives priority; deprecated and retired contracts are excluded from active verification.

Auditable scan decisions

Each verification run stores the exact contract snapshot, evidence policy result, investigation trail, uncertainty, and verdict reasons used for that commit.

Behavior Contract controls

Repository pages now let developers configure contract governance and structured claims, while scan reports expose the contract version, evidence, enforcement, and reasons behind the final verdict.

Behavior Impact Graph

ShipCheck can now map a protected behavior to the implementation, downstream code, APIs, tests, and workflows that depend on it, giving pull request review a clearer picture of behavioral blast radius.

Behavior Impact Graph

Protected behaviors can now be represented as a persistent graph connecting Behavior Guards to implementation code, API entrypoints, regression tests, and downstream workflow impact.

Symbol-level behavior analysis

ShipCheck can move beyond file-level associations and identify specific functions that implement or depend on protected behavior, making behavioral relationships substantially more precise.

Deterministic dependency discovery

Bounded static analysis can follow supported imports and function calls around protected behavior without requiring ShipCheck to send an entire repository to an AI model.

Direct and downstream impact

Behavior Impact can distinguish the implementation directly responsible for a protected contract from downstream code and potential impact reached through that implementation.

Behavior-aware pull request targeting

Static dependency evidence can identify Behavior Guards connected to files and symbols changed by the pull request, helping ShipCheck prioritize the most relevant repository behavior during analysis.

Evidence-backed graph relationships

Graph connections carry their relationship type, confidence, evidence, and source so ShipCheck can distinguish proven implementation and verification links from weaker inferred impact.

Interactive impact exploration

The scan report now presents protected behavior, implementation, and impact or verification as an interactive graph with pan, zoom, auto-fit, relationship labels, stage boundaries, and node details.

Persistent behavior knowledge

Behavior Impact relationships are stored as repository knowledge instead of existing only for a single scan, allowing future pull requests to build on previously established behavioral context.

More efficient AI review context

ShipCheck's review prompts were tightened to remove repeated instructions, empty sections, and redundant metadata while preserving the code, behavioral evidence, repository context, and safeguards used during analysis.

Run ShipCheck directly from GitHub

The /shipcheck pull-request workflow is now surfaced across ShipCheck's website and documentation, making it clear how an activated repository can be rescanned without leaving GitHub.

Expanded GitHub workflow documentation

Documentation now explains repository activation, the /shipcheck pull-request command, permission requirements, on-demand rescanning, and how ShipCheck publishes results back into GitHub.

Stronger Behavior Guards and historical regression detection

ShipCheck can now do more than preserve known behavior. It can learn from previously fixed problems, verify protected behavior against real test suites, and classify exactly what kind of behavioral change a pull request introduces.

Historical regression detection

ShipCheck can use previously fixed repository problems during pull request analysis and flag changes that recreate an old failure mode or break behavior established by a past fix.

Mine Git history for past fixes

Repository history can be inspected for meaningful bug fixes and safeguards, turning established behavior from previous fixes into candidate Behavior Guards that developers can review and protect.

Behavior categories

Behavioral findings can now distinguish removed safeguards, authorization changes, validation changes, API contract changes, state-transition changes, error behavior, data semantics, user behavior, and other behavioral changes.

Runtime-verified Behavior Guards

Behavior Guards can be connected to executable regression tests so protected behavior can be verified against the actual repository instead of relying only on static reasoning.

Expanded language and test framework support

Runtime verification now supports Vitest, Jest, Node Test, Playwright, Pytest, Python unittest, Go test, Cargo test, JUnit, .NET test, RSpec, and PHPUnit.

More precise test execution

ShipCheck can carry the relevant test name or verification pattern into runtime verification, allowing generated and protected tests to be executed more precisely.

Behavior outcomes

Pull request scans surface whether repository behavior was preserved, intentionally changed, violated, or newly introduced so developers can understand behavioral impact without interpreting every finding individually.

Improved scan experience

Dashboard and scan interfaces received readability, layout, and navigation improvements while scan processing and verification were made more resilient for larger real-world repositories.

ShipCheck BETA documentation

A rebuilt documentation experience now explains ShipCheck's core workflow, Behavior Guards, behavioral regression detection, Behavior History, historical fixes, runtime verification, scanning, GitHub integration, and repository configuration with searchable navigation.

Behavior History

ShipCheck can now remember how protected repository behavior evolves over time and trace where that behavior originally came from.

Behavior lifecycle tracking

Protected behaviors can record when they were introduced, preserved, violated, restored, or intentionally changed.

Git history origin tracing

ShipCheck can inspect repository history to find the earliest supported commit where a protected behavior became true.

Cross-file evidence

Origin analysis can follow related implementation files, helpers, authorization logic, validation, and tests instead of relying only on the file where the Behavior Guard was discovered.

Origin evidence

Historical introductions include the source commit, strongest supporting file, confidence score, and an explanation of why that commit established the behavior.

Intentional vs accidental changes

Developer-confirmed behavior changes can be distinguished from regressions, creating a more accurate record of what the repository is expected to do.

More reliable scans

ShipCheck's scan infrastructure became more resilient for real-world repository workloads.

Safer background processing

Improved scan queue and worker behavior so interrupted scans can recover more reliably.

Retry handling

Failed or interrupted scan jobs can be retried instead of becoming permanently stuck.

Stale job recovery

ShipCheck can detect abandoned scan work and recover it without requiring manual intervention.

Behavior Guard

Repositories can preserve important behavioral expectations and make them part of future pull request review.

Protected behavior

Protect repository behaviors and safeguards that future pull requests should preserve.

Behavior-aware review

Pull requests can be evaluated against previously protected behavior instead of being reviewed only as isolated code changes.

Stronger regression detection

Behavior Guard gives ShipCheck additional context for recognizing when a reasonable-looking change weakens behavior the repository depends on.

Regression tests from findings

ShipCheck can turn behavioral findings into regression tests that preserve the behavior developers actually care about.

Generate Regression Test

Generate a test directly from a ShipCheck finding to capture the behavior that should continue working.

Behavior-focused tests

Generated tests target the behavioral regression identified by ShipCheck instead of producing generic test coverage.

Better prevention

Once a regression is fixed, its expected behavior can be encoded in the test suite so the same class of issue is less likely to return.

GitHub-native workflow

ShipCheck became increasingly centered around the pull request workflow developers already use.

GitHub App installation

Connect ShipCheck through the GitHub App and choose which repositories it can access.

Pull request scanning

Review connected repositories and their pull requests directly through ShipCheck.

No personal access token required

Repository access is handled through the GitHub App installation flow instead of requiring personal access tokens.

Per-repository settings

Different codebases need different review behavior, so ShipCheck added repository-specific controls.

AI review controls

AI-enhanced behavioral analysis can be controlled independently for individual repositories.

Category controls

Security, Reliability, Code Quality, Accessibility, and Testing analysis can be enabled or disabled independently.

Ignored paths and rules

Exclude generated files, documentation, vendored code, snapshots, and rules that do not apply to a repository.

Safer Apply Fix

ShipCheck's automated fixes became more careful about the exact pull request revision they modify.

Revision-aware fixes

Before applying a fix, ShipCheck verifies that the pull request has not changed since the finding was produced.

Atomic edits

Related file changes are applied together so fixes do not leave the repository in a partially updated state.

Automatic stale detection

After code changes, previous scan results are marked stale so developers know the pull request should be verified again.

Track findings across revisions

ShipCheck began tracking how findings change while a pull request evolves.

New findings

See which issues were introduced by the latest pull request revision.

Still present

See which previously detected findings remain unresolved.

Resolved

See which findings disappeared after the pull request was updated.

Reopened

Recognize when a previously resolved finding appears again later.

Scan-to-scan changes

Revision metadata, verdicts, Ship Score changes, and scan deltas make it easier to understand what changed between runs.

Ship Score and risk categories

Scan results became easier to prioritize with a single risk score and clearer finding categories.

Ship Score

Pull requests receive a score that summarizes the level of risk ShipCheck identified.

Ship verdict

Scans surface a clear recommendation about whether a pull request appears safe to ship or needs review first.

Five review categories

Findings are organized into Security, Reliability, Code Quality, Accessibility, and Testing.

Severity prioritization

Critical findings, warnings, and informational findings make it easier to focus on the changes that matter most.

Behavioral regression review

ShipCheck's core review model shifted from identifying questionable code to understanding how a pull request changes existing behavior.

Before and after behavior

Findings explain what behavior existed before the pull request and what behavior exists after it.

Removed safeguards

ShipCheck looks for validation, authorization, normalization, error handling, and other protections that became weaker or disappeared.

Broken contracts

Changes to APIs, callers, state transitions, assumptions, and repository conventions can be identified as behavioral risk.

Repository context

Analysis uses relevant code outside the changed lines to understand the contracts and assumptions surrounding a pull request.

Testing gaps

Behavior changes that should be protected by regression tests are surfaced during review.

ShipCheck beta

ShipCheck opened its beta for developers who want an additional verification layer before merging AI-generated and human-written code.

Pull request scanning

Connect a repository and analyze pull requests for behavioral regressions and risky changes.

Built for AI-assisted development

ShipCheck is designed for a world where tools like Cursor, Claude Code, Codex, and Copilot can produce large amounts of plausible-looking code quickly.

Pre-ship verification

Catch unintended behavior changes before they reach production.

See what the next change proves.

Bring durable behavior memory and governed evidence into your repositories.