> Verification that cannot fail — from The Handover, the-handover.org/docs/verification-that-cannot-fail
> Authors: Leon Mallett (captivated.online) with Claude Code · Last confirmed working: 2026-08-11
> © Captivated Ltd — free to use in your own work, not to redistribute as a collection. the-handover.org/licence

An agent that runs a check and reports the result is only as trustworthy as the
check. This document is about three ways a check reports success while being
structurally incapable of reporting anything else — all three found in one
project, all three by something other than the check itself.

The common shape: **nobody had ever seen the check fail.** A green result was
read as evidence, when it was only ever evidence that the check ran.

## 1. Grepping for words instead of asserting exit codes

A typecheck was being verified like this:

```bash
npm run check 2>&1 | grep -E "errors|warnings|hints"
```

The output ends with three lines: `- 0 errors`, `- 0 warnings`, `- 0 hints`.
Seeing them printed reads like confirmation.

**It matches `- 0 errors` and it does not match `- 1 error`.** The singular has
no "s". One error is precisely the case the pattern cannot see, and one error is
the overwhelmingly common case.

```bash
printf -- '- 1 error\n' | grep -E "errors|warnings|hints"   # no output
printf -- '- 0 errors\n' | grep -E "errors|warnings|hints"  # matches
```

The error had been present locally for days. CI, which does not grep, failed
correctly and was not looked at.

**The lesson is not "write better patterns".** It is that a tool already
reported success or failure precisely, in its exit code, and the pattern replaced
that signal with a worse one derived from its prose.

## 2. Pipes discard the exit code

The same command has a second defect, independent of the pattern. In a pipeline,
the shell reports the exit status of the **last** command, and `grep` succeeds.

```bash
false | cat; echo $?     # 0 — the failure is gone
```

So even a correct pattern would not have failed the run. Two options, and they
differ by shell:

```bash
set -o pipefail          # portable across bash and zsh; pipeline fails if any stage fails
false | cat; echo $?     # 1
```

Or read the stage directly — but note the syntax is **not** portable:

```bash
# bash: zero-indexed
false | cat; echo "${PIPESTATUS[0]}"     # 1
# zsh: a different variable, one-indexed
false | cat; echo "${pipestatus[1]}"     # 1
```

Using bash's form in zsh yields an empty string, which in a conditional is
falsy — failing open, again silently.

**The general fix is to stop post-processing and start chaining.** Commands
joined with `&&` stop at the first failure and return non-zero:

```bash
npm run canon:verify && npm run test && npm run check && npm run build
```

There is nothing to pattern-match, and no way for a failure to be reported as
success.

## 3. A guard nobody has watched fail

A build-time check compared two copies of a document and was supposed to fail the
build when they drifted. It passed. It passed every time.

That is exactly what a check with a broken file path, an unreachable branch or an
inverted condition also does.

**A guard you have only ever seen pass is not yet a guard.** It is a line of code
that has never been observed doing its job. The fix is cheap and takes a minute:
deliberately introduce the condition it exists to catch, watch it fail, then undo
that.

Doing this found a related problem in the same session — a command-line tool
whose usage error exited with the same code as a real verdict, so "I could not
parse my arguments" and "I rejected this document" were indistinguishable to
anything downstream. That was only visible because the failure path was
exercised on purpose.

This generalises past guards to anything with an error branch: **an error path
that has never executed is an assumption, not a behaviour.**

One case this remedy does not reach: a check that is correctly written and simply
has nothing to examine. You cannot introduce the condition it catches without
inventing a subject for it, and a fabricated subject is worse than the silence it
replaces. That needs a different fix — see
[Nothing was wrong, or nothing was examined](/docs/nothing-was-wrong-or-nothing-was-examined).

## What to do instead

- **Assert exit codes, not output.** If a tool cannot signal failure properly,
  that is worth fixing before building on it.
- **Chain with `&&`** rather than piping into a matcher. One command, one exit
  code, no interpretation.
- **Set `pipefail`** where a pipeline is genuinely needed.
- **Make each guard fail once, on purpose**, and record in the commit that you
  did. A guard's first failure should be one you caused.
- **When local and CI disagree, believe CI.** It has no muscle memory and no
  helpfully-worded grep. A red CI against a green local check almost always means
  the local check is the broken one.

## Why this matters more for agents

An agent runs verification commands far more often than a person does, reports
their results in prose, and is trusted partly because it says it checked. Every
one of these failures produces a confident, accurate-sounding report of a check
that could not have failed.

The report is not the verification. The exit code is.