ShipCheckBeta
ShipCheck DocumentationBeta

Everything you need to use ShipCheck.

Learn how ShipCheck analyzes pull requests, remembers expected software behavior, governs important safeguards as contracts, verifies them with evidence, and makes risk-aware ship decisions.

Core concept

Software should remember what it is supposed to do.

ShipCheck turns repository behavior into persistent memory. Each future change can then be evaluated against what the software has already established as important.

Discover

Find important existing behavior.

Protect

Create persistent Behavior Guards.

Verify

Evaluate future behavioral changes.

Remember

Build history across future changes.

Section

Getting started

What is ShipCheck?

Understand ShipCheck's behavioral-memory approach to code review.

ShipCheck is behavioral memory for software. It analyzes code changes against the behaviors, safeguards, assumptions, and contracts that a repository is expected to preserve.

Traditional code review primarily asks whether newly changed code looks correct. ShipCheck adds a different question: did this pull request change something the software was already supposed to keep doing?

  • Detect meaningful behavior changes.
  • Find security and authorization regressions.
  • Detect weakened safeguards and validation.
  • Identify test and regression-coverage gaps.
  • Discover important existing repository behavior.
  • Protect important behavior with Behavior Guards.
  • Govern protected behavior with versioned Behavior Contracts.
  • Require specific evidence before a behavior can be verified.
  • Preserve uncertainty when the available proof is incomplete.
  • Compare future pull requests against repository memory.
  • Track how behavior changes over time.
  • Mine Git history for important fixes worth protecting.
  • Generate focused fixes and regression tests.
  • Track findings across rescans.
  • Run scans directly from GitHub with the /shipcheck pull-request command.
  • Produce a risk-aware Ready, Review, or Do not ship verdict.
ShipCheck BETA

ShipCheck is currently in beta. Features and limits may continue to evolve while the behavioral-memory model is developed.

Connect GitHub

Install the ShipCheck GitHub App and give ShipCheck access to repositories.

ShipCheck connects through its GitHub App. The integration allows ShipCheck to discover repositories, inspect pull requests, read repository context, respond to repository events, accept the /shipcheck pull-request command, and publish scan information back into the GitHub workflow.

  1. 01Create or sign in to your ShipCheck account.
  2. 02Choose the option to connect GitHub.
  3. 03Install the ShipCheck GitHub App on your GitHub account or organization.
  4. 04Select the repositories ShipCheck should be allowed to access.
  5. 05Return to ShipCheck and synchronize your repositories.
  6. 06Activate the repositories you want ShipCheck to analyze.

Only repositories made available through the GitHub App installation can be synchronized into ShipCheck.

Repository sync

Understand how installed GitHub repositories appear inside ShipCheck.

After the GitHub App is installed, ShipCheck synchronizes repositories from that installation into the Repositories page.

Repository synchronization and ShipCheck activation are separate concepts. A repository may be visible because GitHub granted ShipCheck access while remaining inactive for scanning until you enable it.

  1. 01Open Repositories from the dashboard.
  2. 02Synchronize GitHub if a newly authorized repository is not yet shown.
  3. 03Select a repository.
  4. 04Enable ShipCheck for that repository.
  5. 05Configure repository-specific scan settings if necessary.

Run your first scan

Analyze a pull request from ShipCheck or trigger the scan directly from GitHub.

After connecting and activating a repository, you can start a pull-request scan from either the ShipCheck dashboard or directly from the GitHub pull request.

  1. 01Connect and activate the repository.
  2. 02Open one of the repository's pull requests inside ShipCheck.
  3. 03Start the ShipCheck scan.
  4. 04ShipCheck queues the analysis and displays scan progress.
  5. 05Open the completed report to review its risk-aware verdict, contract decisions, evidence, uncertainty, behavior changes, findings, and available actions.
Prefer to stay in GitHub?

On an open pull request, comment /shipcheck to request a ShipCheck scan of the current pull-request revision. ShipCheck will queue the scan and publish its GitHub result as the analysis progresses and completes.

/shipcheck

Run ShipCheck from GitHub

Trigger a ShipCheck scan by commenting /shipcheck on an open pull request.

ShipCheck can be triggered directly from a GitHub pull request. You do not need to leave the pull request and open the ShipCheck dashboard just to request another scan.

/shipcheck
  1. 01Open an active pull request in a repository connected to ShipCheck.
  2. 02Add a new pull-request comment containing /shipcheck.
  3. 03GitHub sends the comment event to the ShipCheck GitHub App.
  4. 04ShipCheck verifies that the repository is connected and that the commenter has sufficient repository permission.
  5. 05ShipCheck queues a scan for the open pull request.
  6. 06A ShipCheck GitHub comment is created and updated with the scan result.
Who can run the command?

ShipCheck only accepts the command from GitHub users with write, push, maintain, or admin permission on the repository. Read-only and triage users cannot consume scan capacity through /shipcheck.

The command must be on a pull request

The /shipcheck command is handled from pull-request comments. The same comment on a normal GitHub issue does not trigger a scan.

The command is useful after pushing new commits, when requesting an on-demand behavioral review, or whenever you want to rerun ShipCheck while remaining inside the GitHub workflow.

Section

Pull request scanning

How a scan works

See what ShipCheck considers when analyzing a change.

ShipCheck combines deterministic checks with repository-aware reasoning. It does not analyze a pull-request diff as an isolated block of text.

  • Changed files and pull-request diffs.
  • Relevant surrounding implementation.
  • Authentication and authorization boundaries.
  • Validation logic.
  • Callers and related helpers.
  • Existing tests.
  • Error handling.
  • State transitions.
  • Data semantics.
  • Repository-specific safeguards.
  • Protected Behavior Guards.
  • Behavior Contract versions and evidence requirements.
  • Relevant Git history.

This additional repository context helps ShipCheck distinguish meaningful behavior changes from ordinary implementation changes.

Scan progress

Track an analysis while ShipCheck processes the pull request.

ShipCheck processes scans asynchronously and reports progress while analysis is running. While processing, the scan progress page reports the state of the analysis instead of requiring the browser request to remain open for the entire scan.

When processing completes, ShipCheck redirects you toward the completed report where the full analysis can be inspected.

Risk-aware ship verdict

Understand how findings and Behavior Contracts govern the final ship decision.

A completed scan combines conventional findings with the verification result, criticality, enforcement, lifecycle, and approval state of every affected Behavior Contract. The numeric score remains useful context, but the verdict is no longer derived from score alone.

Completed ShipCheck pull request scan report showing verdict, score, findings, and behavioral analysis
A completed scan combines the overall verdict with the evidence and behavioral changes behind it.
Ready to ship

No blocking findings or enforced contract concerns remain.

Ready with uncertainty

A non-blocking contract remains inconclusive and its uncertainty is shown explicitly.

Review before shipping

A warning, pending contract decision, or high-risk inconclusive result requires human review.

Do not ship

A critical finding or enforced contract violation should block the change.

The report shows the policy version, contract decisions, evidence, citations, contradictions, and remaining uncertainty behind the verdict. The verdict is still a review signal rather than a substitute for engineering judgment.

Finding severity

Understand how ShipCheck communicates the importance of a finding.

Critical

A high-impact issue that may make the pull request unsafe to ship.

Warning

A meaningful concern that deserves review.

Info

Useful analysis that does not necessarily represent a blocking problem.

Passed

A check or protected condition was successfully preserved.

Finding categories

The affected-area categories used throughout repository settings and scan reports.

CategoryWhat it covers
SecurityAuthorization, authentication, secrets, unsafe boundaries, and security-sensitive behavior.
ReliabilityFailure paths, null assumptions, cleanup, normalization, and runtime reliability.
QualityContracts, implementation assumptions, incomplete changes, callers, and maintainability-related correctness.
AccessibilityAccessible names, semantics, labels, keyboard behavior, and user interaction accessibility.
TestingCoverage for changed behavior, important failure paths, edge cases, and regression protection.
OtherFindings that do not cleanly belong to one of the repository-configurable categories.

Behavior change categories

The behavior-specific taxonomy used when ShipCheck detects an actual behavioral transition.

Affected-area categories and behavior-change categories are intentionally separate. The first says what area of the software is involved; the second says what kind of behavior changed.

Behavior categoryMeaning
Removed safeguardA previously existing defensive check or safeguard disappeared.
Changed authorizationWho is allowed to perform an action changed.
Changed validationAccepted or rejected inputs changed.
Changed API contractAn API, caller, helper, or integration contract changed.
Changed state transitionThe conditions or result of a state transition changed.
Changed error behaviorErrors are raised, returned, handled, or suppressed differently.
Changed data semanticsThe meaning, normalization, storage, or interpretation of data changed.
Changed user behaviorUser-visible functionality or interaction behavior changed.
OtherA meaningful behavior change outside the predefined categories.
Section

Behavioral memory

Behavior Discovery

Discover important expectations already encoded in a repository.

Behavior Discovery identifies meaningful rules that already exist inside the repository without requiring developers to manually document everything first.

  • Authorization guarantees.
  • Validation rules.
  • Business rules.
  • Error-handling expectations.
  • Data semantics.
  • Existing safeguards.
  • User-facing behavioral expectations.

Discovered behaviors appear as candidates. A developer can protect a useful candidate or dismiss one that should not become persistent repository memory.

Behavior Guards

Protect an expected repository behavior across future code changes.

A Behavior Guard is a persistent expectation about what the repository is supposed to keep doing.

Only administrators may delete a tenant.

Expired sessions must always be rejected.

Course codes are normalized before duplicate detection.
ShipCheck Behavior Guards showing protected repository behaviors
Protected behaviors become persistent expectations that ShipCheck can verify against future changes.

Once protected, future pull requests can be evaluated against the behavior and its supporting evidence rather than relying on each reviewer to rediscover the rule from scratch.

Protected behavior
        ↓
Pull request changes code
        ↓
ShipCheck verifies the behavior
        ↓
Preserved / Changed / Violated

Behavior Contracts V2

Govern a protected behavior as a versioned, auditable software contract.

A Behavior Guard identifies what the repository must preserve. Behavior Contracts V2 adds the governance needed to decide how strongly that expectation should affect verification and the final ship decision.

  • Lifecycle: proposed, active, changing, deprecated, or retired.
  • Criticality: advisory, important, critical, or regulated.
  • Enforcement: observe, warn, require review, or block merge.
  • Owner and approval state.
  • Structured preconditions, invariants, scenarios, and observables.
  • Required runtime, test, static, historical, or manual evidence.
  • Evidence freshness and supersession metadata.
Behavior Contract v3

Criticality     Critical
Enforcement     Block merge
Approval        Approved
Invariant       Unauthorized cancellation never changes state.
Required proof  Runtime + test
Contract changes are versioned

Saving a governance or claim change creates an immutable contract snapshot with its change reason. Verification runs also retain the exact contract version used for that commit.

Behavior Verification Agent

See how ShipCheck plans, investigates, and evaluates protected behavior.

The Behavior Impact Graph selects contracts connected to a pull request. Higher-criticality contracts receive priority, while deprecated and retired contracts are not scheduled for active verification.

  • The planner converts the contract into falsifiable invariants and scenarios.
  • Read-only tools can retrieve files, look up references, discover tests, and inspect historical evidence.
  • A strict call budget bounds the investigation.
  • Runtime, code-presence, test-presence, and investigation results are normalized into evidence.
  • Coverage, citations, source diversity, contradictions, and required evidence are evaluated by policy.
Inconclusive is a real result

Code or test presence is not automatically proof that behavior executed successfully. If decisive or contract-required evidence is unavailable, ShipCheck reports the remaining uncertainty instead of manufacturing a verified result.

Behavioral Diff

Understand what the software does differently after a pull request.

A source-code diff describes which lines changed. Behavioral Diff describes what those changes mean for the software.

BEFORE

Only project owners can update a project.

AFTER

Any authenticated user can update the project.

RESULT

Authorization behavior changed.
Preserved

A known behavior still works after the pull request.

Changed

A known behavior is different after the pull request.

Violated

A protected expectation no longer holds.

New

The pull request introduces meaningful behavior that was not previously present.

Intentional behavior changes

Teach ShipCheck when changed behavior is deliberate rather than a regression.

Not every behavior change is a bug. Product requirements change, old restrictions are intentionally removed, and APIs evolve.

ShipCheck distinguishes likely unintentional changes, unclear changes, and likely intentional changes. Developers can explicitly mark supported behavior changes as intentional.

Repository memory evolves

An intentional change can update the expected behavior instead of leaving the repository permanently anchored to an outdated rule.

Behavior History

Track how protected repository behavior evolves across commits and pull requests.

Behavior History gives important software expectations their own timeline, separate from the raw Git history of files.

Introduced
    ↓
Preserved
    ↓
Changed intentionally
    ↓
Violated
    ↓
Restored

Git remembers how source code changed. Behavior History is designed to remember how expected software behavior changed.

Historical origin tracing

Trace protected behavior back to the Git history that established it.

ShipCheck can inspect Git history to determine where a protected behavior appears to have first become true.

  • Source commit.
  • Historical date.
  • Relevant implementation file.
  • Related implementation and test evidence.
  • Cross-file origin evidence.
  • Confidence score.
  • Explanation of why the commit established the behavior.

Origin tracing is not limited to the exact file where a Behavior Guard was discovered. ShipCheck can inspect related helpers, authorization logic, tests, and implementation history.

Mine historical fixes

Turn important past bug fixes into candidate Behavior Guards.

Some of the strongest evidence for what a repository must preserve is hidden inside bugs the team already fixed.

fix duplicate charges

prevent unauthorized access

reject expired sessions

stop duplicate submissions

ShipCheck can mine repository history for these types of fixes and convert supported fixes into candidate behaviors.

ShipCheck Historical fixes interface showing a behavior candidate mined from Git history
Historical fixes surface behaviors established by previous bug and safeguard fixes so they can become Behavior Guards.
  1. 01Open a repository in ShipCheck.
  2. 02Find the Historical fixes section.
  3. 03Choose Mine git history.
  4. 04Review the candidate behavior, evidence, source commit, and confidence.
  5. 05Protect the candidate if it represents behavior the repository should continue preserving.
  6. 06Dismiss candidates that should not become repository memory.
Section

Findings & repairs

Finding lifecycle

Follow a finding across repeated scans of the same pull request.

ShipCheck fingerprints findings so the same issue can be tracked between revisions instead of appearing as an unrelated new finding after every scan.

New

The finding was introduced in the current revision.

Still present

ShipCheck matched the issue to a finding from the previous completed scan.

Resolved

A previously detected finding is no longer present.

Reopened

A previously resolved issue appeared again later.

Evidence and source context

Review why ShipCheck produced a finding and where it applies.

ShipCheck findings can include the affected file, source lines, code snippets, behavioral before/after information, impact, and supporting repository evidence.

Review the evidence before acting on a finding. A finding should be treated as a reasoned engineering claim that can be inspected, not as an unexplained AI score.

Suggested and applicable fixes

Use ShipCheck to turn a concrete finding into a bounded code repair.

Eligible findings can include a suggested repair. Some repairs can be applied directly when ShipCheck has enough concrete context to construct a bounded edit plan.

ShipCheck finding showing evidence and available repair actions
Eligible findings expose actions directly beside the evidence that produced the finding.
  • Repairs remain tied to a specific finding.
  • Machine-applicable repair plans are bounded.
  • A repair can include multiple related edits when the account plan supports multi-file fixes.
  • ShipCheck limits generated repair plans to a small number of files and edits instead of allowing unrestricted repository rewrites.
Current safety bound

Machine-applicable repair plans are intentionally bounded to a small number of related files and edits.

Regression tests

Generate a focused test that protects behavior exposed by a finding.

A meaningful regression is more valuable when it produces durable protection. Eligible findings can be used to generate focused regression tests.

  1. 01Open a finding that supports regression-test generation.
  2. 02Ask ShipCheck to generate the regression test.
  3. 03Review the proposed test and the environment information ShipCheck used.
  4. 04Apply the generated test when it accurately captures the intended behavior.

This helps move the repository from detecting a bug once toward preventing the same class of regression in the future.

Mark a finding intentional

Record when a flagged change is actually the intended product change.

When ShipCheck correctly detects a behavior change but that change is intentional, it can be marked accordingly rather than being treated as an accidental regression.

This distinction is particularly important for behavioral memory because expected behavior must be allowed to evolve as the product evolves.

Finding feedback

Give feedback on individual ShipCheck findings.

Completed scan findings include feedback controls. Feedback provides a direct way to communicate whether an analysis was useful or inaccurate.

Finding feedback is separate from marking a behavior change intentional. An intentional change can still be correctly detected by ShipCheck.

Section

Repository tools

Repository dashboard

Use the repository page as the home for pull requests, Behavior Contracts, history, and repository health.

Each connected repository has its own ShipCheck page. It brings together repository status, pull requests, protected behaviors, Behavior Contract governance, scan activity, historical fixes, and repository-specific tools.

From a protected behavior, developers can configure lifecycle, criticality, enforcement, owner, approval, structured claims, evidence freshness, and required evidence. Every saved contract change requires a reason and produces a new immutable version.

ShipCheck repository dashboard showing repository status, protected behaviors, pull requests, scan activity, and historical fixes
The repository dashboard is the central workspace for a connected repository.

Compare branches

Run ShipCheck analysis between repository branches without depending on a specific PR workflow.

ShipCheck includes a repository branch-comparison workflow. This is useful when you want to analyze the behavioral difference between two branches directly.

  1. 01Open the repository.
  2. 02Choose Compare branches.
  3. 03Select the base branch.
  4. 04Select the branch containing the changes.
  5. 05Start the comparison.
  6. 06Review the resulting ShipCheck analysis.

Repository settings

Control how ShipCheck analyzes an individual repository.

Repository settings allow ShipCheck to adapt to the conventions of individual codebases instead of applying exactly the same scanning configuration everywhere.

ShipCheck repository settings showing scanning categories and repository-specific configuration
Repository settings control how ShipCheck analyzes an individual codebase.
SettingPurpose
AI analysisEnable or disable repository-aware AI analysis for this repository.
SecurityEnable or disable Security findings.
ReliabilityEnable or disable Reliability findings.
QualityEnable or disable Quality findings.
AccessibilityEnable or disable Accessibility findings.
TestingEnable or disable Testing findings.
Ignored pathsExclude files or directories from scan findings.
Ignored rulesDisable deterministic ShipCheck rules that are not useful for the repository.

Ignored paths

Exclude generated or irrelevant repository paths from findings.

Ignored paths support simple safe glob matching. They are useful for generated files, vendored code, snapshots, documentation, or other repository-specific content that should not produce findings.

docs/**
generated/**
vendor/**
src/**/*.snapshot.ts
coverage
  • * matches any characters except a path separator.
  • ** matches characters across directories.
  • ? matches one non-separator character.
  • A plain directory name matches that directory and files beneath it.

ShipCheck currently stores up to 50 ignored path patterns per repository.

Ignored rules

Disable deterministic scanner rules that do not fit a repository.

A deterministic check that is useful for most repositories may not make sense everywhere. Add its rule ID to Ignored rules to suppress that rule for the selected repository.

ShipCheck currently stores up to 100 ignored rule IDs per repository.

Section

GitHub workflow

GitHub pull-request events

Keep ShipCheck connected to pull-request activity through GitHub webhooks.

The ShipCheck GitHub App receives supported GitHub webhook events so repository and pull-request activity can flow into ShipCheck without requiring every action to begin from the dashboard.

This allows ShipCheck scanning to fit the normal pull-request workflow rather than requiring teams to maintain a separate code-review process.

Pull-request scan comments

Surface ShipCheck analysis inside GitHub.

ShipCheck can publish scan information associated with a pull request so developers can move between GitHub and the full ShipCheck report.

Generated code changes

Understand how applicable fixes and regression tests reach GitHub.

When ShipCheck applies an eligible machine-generated fix or regression test, its GitHub integration can create the required repository changes as focused commits rather than asking the user to manually copy generated code.

Generated code should still be reviewed like any other code change before merge.

Section

Plans & usage

ShipCheck BETA limits

Current beta usage entitlements for each ShipCheck plan.

ShipCheck's product entitlements are enforced by the application. During the current beta, the Free plan receives expanded beta usage.

PlanRepositoriesAI scansRegression testsHistoryMulti-file fixesPriority
Free · BETA3301030 daysNoStandard
Pro51002590 daysYesFaster
MaxUnlimited500100365 daysYesHighest
TeamUnlimited250 / seat50 / seat365 daysYesHighest
Team plan

Team entitlements scale with seats and use a minimum of three seats.

Billing management

Change, cancel, or resume a paid ShipCheck subscription.

Billing controls are available from the account settings area. ShipCheck supports starting checkout, changing a paid plan, scheduling cancellation, and resuming an eligible subscription.

  1. 01Open Settings.
  2. 02Open Billing.
  3. 03Review your current plan and usage.
  4. 04Choose the relevant plan or subscription action.
  5. 05Complete any Stripe-hosted checkout step when required.
Section

Account & support

Account settings

Manage your ShipCheck account separately from repository settings.

Account settings are distinct from repository settings. Repository settings control scanning behavior; account settings control the ShipCheck account itself.

The account settings area also includes account-deletion controls for users who want to remove their ShipCheck account.

Send ShipCheck feedback

Send general beta feedback directly from the product.

ShipCheck includes an in-product feedback control on the public experience. During beta, feedback is especially useful for reporting confusing findings, missing behavior, workflow issues, or product suggestions.

Security information

Find ShipCheck's public security documentation.

For information about ShipCheck's own security posture and handling of repository-related access, see the dedicated Security page.

Open Security
ShipCheck BETA

Protect the behavior your code already got right.

Connect GitHub, discover important repository behavior, and prevent future pull requests from silently breaking it.