`list_procs` matched on `readlink -f /proc/PID/exe`. Once the binary is
rebuilt -- which happens constantly here, `cargo test` alone is enough -- the
link reads "<path> (deleted)" and -f resolves it to something that matches
nothing. The scan then finds zero processes, so `start`'s orphan check passes
and a second instance can be launched alongside a stray.
Not theoretical. Two orphans were running undetected tonight:
pid 592731 :42327 a stale-cert redirector left from testing check-tls-parity,
still serving F9:16:1A -- the exact certificate whose
mismatch cost three live gates
pid 542693 :42230 a Blaze sidecar debug build from 03:12
Neither was in a client path, so neither was doing harm, but a stray listener
serving the known-bad certificate is precisely what should never sit around
unnoticed.
Fixed by using plain readlink and stripping the " (deleted)" suffix. Shown both
ways: with the bug `status` reports no processes at all for a live pid; with the
fix it reports 604454. The pidfile path was unaffected, which is why `stop` kept
working and hid this.
`verify` inspects `$BIN --identity`, which is the file on disk. That is not
necessarily what is serving. Caught during gate 14 setup: the live process had
been started from 5bc39e9, then `cargo test` re-ran build.rs (the branch ref
moved when an unrelated script was committed) and restamped the on-disk binary
to fc411bb. `verify` then reported "build identity OK" about an artifact that
was not the running service.
`start` now records the stamped commit to $RUNDIR/redirector.commit, and
`verify-running` compares THAT against HEAD, refusing when they differ. The
existing on-disk check stays -- it is the right gate for "may I start this" --
but only the recorded stamp answers "is the thing currently serving the thing I
think it is", which is the question a live gate's evidence depends on.
The binary records only the commit it was built from -- no dirty-tree flag.
Cargo will not re-run a build script because another crate's source changed, so
a compiled-in 'clean' claim can be stale and is not a safeguard; that was
verified on the Blaze host.
scripts/verify-build-identity.sh establishes both facts at LAUNCH, where they
cannot go stale: the stamped commit equals HEAD, and the migration crates are
clean. It REFUSES rather than warns, because for a migration gate a warning on
stderr is something to scroll past.
--identity prints the stamp without valid configuration. The launcher must be
able to establish which commit a binary came from BEFORE deciding whether to
run it; requiring a correct environment first would invert the check.
redirector.sh mirrors sidecar.sh: refuses to start with an orphan present or
the port busy, matches the resolved executable rather than the command line
(pgrep -f matches any shell mentioning the name), and stop PROVES the process
is gone and the port free.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
funman300