Skip to content

Gates

Gates are concrete verification checks that run during lexicon verify. Each gate executes a shell command as a subprocess and reports pass, fail, skip, or error.

The core idea: make no-regression gates enforceable. Gates are hard enforcement boundaries. They protect against both human and AI attempts to game the system.

Examples of what gates protect against:

  • deleting tests to make CI pass
  • weakening assertions without acknowledgment
  • silently loosening score thresholds
  • rewriting contract semantics without updating contract status/history
  • weakening required gates without policy approval
  • gaming performance baselines
  • hiding failures behind skipped tests

The architecture assumes that AI can make “locally clever but globally bad” choices and defends against that.

The gates model is stored at specs/gates.toml.

Each gate has:

FieldTypeDescription
idstringUnique identifier (e.g., fmt, clippy)
labelstringHuman-readable label
commandstringShell command to execute
categoryenumrequired, scored, or advisory
timeout_secsu64?Timeout in seconds (default: 300s if not specified)
allow_skipboolWhether this gate may be skipped
  • Required — Must pass. Failure blocks the overall verification. Required gates default to allow_skip = false.
  • Scored — Contributes to the numeric score but does not block verification on its own.
  • Advisory — Informational only. Does not affect pass/fail.

Gates run sequentially via sh -c <command> in the repository root. Each gate produces:

  • Pass — Command exited with status 0
  • Fail — Command exited with non-zero status
  • Skip — Gate was in the skip list
  • Error — Command could not be executed (e.g., binary not found)

Stdout and stderr are captured for each gate, along with execution duration in milliseconds.

Gates can be skipped by passing their IDs to the skip list. However, policy enforcement prevents skipping gates that should not be skipped:

  • Required gates with allow_skip = false cannot be skipped — the policy validator rejects the request.
  • Only gates with allow_skip = true may be skipped.

Lexicon detects when a gate configuration change constitutes a “weakening”:

  • Lowering the category (e.g., Required to Scored, or Scored to Advisory) is a weakening
  • Making a non-skippable gate skippable (allow_skip: false to true) is a weakening

When gate_weakening_requires_approval is enabled in the manifest policy (the default), weakening changes are flagged and audited. This is a critical safety boundary — AI must not weaken gates to “fix” verification failures.

The gate system supports a wide range of no-regression checks:

  • fmt — Code formatting
  • clippy — Lint quality
  • unit tests — Core correctness
  • conformance tests — Contract verification
  • behavior tests — Scenario-level assertions
  • property tests — Randomized invariant checking
  • fuzz smoke — Crash-resistance validation
  • benchmark smoke — Performance regression detection
  • snapshot validation — Output stability
  • contract drift validation — Contract vs. code consistency
  • public API drift checks — API surface stability
  • required file integrity checks — Critical file protection

The default model created by lexicon init:

schema_version = "1.0"
[[gates]]
id = "fmt"
label = "Format Check"
command = "cargo fmt -- --check"
category = "required"
timeout_secs = 60
allow_skip = false
[[gates]]
id = "clippy"
label = "Clippy Lints"
command = "cargo clippy -- -D warnings"
category = "required"
timeout_secs = 120
allow_skip = false
[[gates]]
id = "unit-tests"
label = "Unit Tests"
command = "cargo test"
category = "required"
timeout_secs = 300
allow_skip = false
[[gates]]
id = "doc-tests"
label = "Documentation Tests"
command = "cargo test --doc"
category = "scored"
timeout_secs = 120
allow_skip = true

Edit specs/gates.toml to add project-specific gates:

[[gates]]
id = "miri"
label = "Miri Safety Check"
command = "cargo +nightly miri test"
category = "advisory"
timeout_secs = 600
allow_skip = true