funman300
d7c0a5521d
Three times now the same sequence has broken the client path: a build guard correctly refuses to start the Rust replacement, and the `switch on` that follows in the same script arms anyway, because it never checked whether anything was listening. The redirect then lands on a closed socket and the working Python service is bypassed for no benefit. `on` now refuses unless the target port is listening. ALLOW_DEAD_TARGET=1 overrides it for arming ahead of a service that is about to start, but that has to be deliberate. Verified both ways: rc=2 and nothing installed against a dead port, rc=0 and two rules with the override. The watchdog now writes a pidfile. Stopping it by command-line match is unsafe -- any shell whose arguments merely mention the script name matches too, which has now killed the wrong process twice here (once via `pkill -f`, once via a /proc/*/cmdline substring loop).
openfut-blaze-host
A deliberately thin TCP host for the FIFA 17 Blaze RPC surface. Runs beside the working Python backend, never instead of it.
listener → Fire2 stream framing → per-connection Session
│
openfut-adapter-fifa17::dispatch
│
write returned frames, in order
Scope
Owns: a socket, a read buffer, one Session per connection, diagnostics.
That is the complete list.
Must never acquire: coins, club state, packs, profiles, market state, UTAS logic. Those belong to OpenFUT Core, reached later through the adapter. A transport host that starts holding game state becomes a second backend — the architecture this migration exists to avoid.
No TLS
The Blaze main port is plaintext. Verified against the running Python
backend by sending a raw Fire2 Util::ping and getting a plaintext
PingResponse back; blaze_responder_v3b.py::blaze_handle uses the raw socket
and only redir_handle wraps ssl. TLS belongs to the redirector phase.
Running it
Both critical settings are required — there is no default port anywhere in this crate, so it cannot silently collide with the Python container.
cargo build -p openfut-blaze-host
OPENFUT_ADVERTISE=<backend LAN ip> \
OPENFUT_BIND=0.0.0.0 \
POW_CONTENT_HOST=<backend LAN ip>:8085 \
OPENFUT_BLAZE_HOST_BIND=0.0.0.0 \
OPENFUT_BLAZE_HOST_PORT=<free port> \
OPENFUT_BLAZE_TRACE=/tmp/rust-blaze.trace \
./target/debug/openfut-blaze-host
OPENFUT_BIND is the advertised-config bind, not the listener's. The
adapter derives nucleusConnect from it (reproducing the oracle — see the
adapter README and the vault's known-issue entry), so it must mirror whatever
the Python container runs with, or the two will not compare. The listener has
its own OPENFUT_BLAZE_HOST_BIND.
For a FIFA test from another machine, OPENFUT_BLAZE_HOST_BIND must be 0.0.0.0
(or the LAN address); the default follows OPENFUT_BIND.
Live A/B against Python
./check-live-parity.sh 127.0.0.1:42130 127.0.0.1:<rust port>
Replays the recorded conversations against both endpoints over real sockets and diffs the normalized traces. Session keys and server timestamps are masked, so anything that differs is a real behavioural difference.
The diff is the test, not the probe's exit code. The probe only detects anomalies visible live — missing frames, an early close. A same-length content change deep inside a notification body shows up only as a digest difference in the trace. Both cases were verified by mutation.
For the comparison to mean anything both servers must run the same config —
same OPENFUT_ADVERTISE, OPENFUT_BIND, POW_CONTENT_HOST and active persona
— or legitimate config differences read as parity failures.
Current result against the live container:
main: IDENTICAL (82 frames)
fallbacks: IDENTICAL (12 frames)
locale: IDENTICAL (7 frames)
LIVE PARITY OK — 101 frames, identical normalized traces.
Tests
cargo test -p openfut-blaze-host — 18 tests. The integration suite runs the
real host on an ephemeral port and replays the recorded conversations over TCP,
covering what fixtures cannot:
- byte-for-byte replay over a socket
- requests dribbled one byte at a time (fragmentation)
- several requests in one write (coalescing)
- the four-frame login burst arriving in order on the wire
- session state persisting across frames, and not leaking between connections
- an absurd payload length closing the connection instead of allocating
- an undecodable body still getting a reply
Byte-exactness is possible only because Hooks injects the session key and
clock — they appear inside response bodies, so with the real ones no live run
could reproduce a recording. That is the crate's only test seam.
Promotion gates
Automated, re-runnable now:
- ✅ All migration tests pass (116 across the three crates).
- ✅ Python contract suite still 446/446.
- ✅ Scripted connection to the Rust host succeeds.
- ✅ Recorded conversations work through the real TCP host, and the live A/B against Python is identical over 101 frames.
Requiring a FIFA client on the game machine — not yet done:
- ⬜ FIFA reaches FUT with Rust Blaze.
- ⬜ Open a pack — exercise a known-working FUT action, not just bootstrap.
- ⬜ Close FIFA completely.
- ⬜ Repeat 5–6 with Rust Blaze.
- ⬜ Switch back to Python Blaze and verify FUT still works.
- ⬜ Switch to Rust once more and verify again.
Gates 9 and 10 matter as much as 5: Python → Rust → Rust → Python → Rust
proves the rollback path rather than asserting one exists.
Process and switch safety
Both were built after real incidents, not speculatively.
- Orphaned sidecars. A previous session's mutation runs left four sidecars
listening, two serving deliberately broken builds.
sidecar.shrefuses to start when any sidecar is already running, andstopverifies both that the PID is gone and that the port is free — failing if either check does not hold. Orphan detection matches the resolved executable, not the command line:pgrep -fwas tried first and matched any shell whose arguments merely mentioned the name. - A rollback that lied.
blaze-switch.sh offonce reported success while two rules remained active, because it matched--comment "tag"with quotes that this iptables does not emit — and the verification used the same broken matcher, so it confirmed its own failure. Rules are now matched on the bare tag string, andoffverifies withiptables-saveplus a tag-independent check that nothing still redirects the port.
The general lesson, now applied throughout: a verification must not share the failure mode of the thing it verifies.
Running gates 5–10
The Python redirector advertises a hardcoded BLAZE_PORT = 42130
(blaze_responder_v3b.py:173), so redirecting Blaze by reconfiguring it would
mean editing the frozen oracle and rebuilding the container. blaze-switch.sh
does it with a scoped NAT rule instead: no Python change, instant rollback.
Rules match only <LAN_IP>:42130. Traffic to 127.0.0.1:42130 is deliberately
left alone, so Python stays directly reachable on loopback and the A/B keeps
comparing real Python against real Rust.
cargo build -p openfut-blaze-host
export OPENFUT_ADVERTISE=<LAN_IP> OPENFUT_BIND=0.0.0.0 \
POW_CONTENT_HOST=<LAN_IP>:8085 \
OPENFUT_BLAZE_HOST_BIND=0.0.0.0 OPENFUT_BLAZE_HOST_PORT=<FREE_PORT> \
OPENFUT_BLAZE_TRACE=/tmp/rust-blaze.trace
./sidecar.sh start # refuses if an orphan or the port is busy
./blaze-switch.sh on <LAN_IP> <FREE_PORT> # Blaze -> Rust
./blaze-switch.sh status # confirm before launching FIFA
# … launch FIFA, reach FUT, OPEN A PACK, close FIFA, repeat …
./blaze-switch.sh off # Blaze -> Python (rollback)
./sidecar.sh stop # refuses while the switch is on
sidecar.sh stop refuses while the switch is on: stopping the sidecar then
would leave Blaze pointed at a dead port. Use stop --force only deliberately.
Evidence to keep from each live run:
/tmp/rust-blaze.trace— the normalized trace, which begins with the build banner, so a session is attributable to an exact binary and config table- the sidecar log — connection accepted, preAuth, login, the three notifications, subsequent commands, close reason
- whether the pack opened, not just whether the hub loaded
Then compare the Rust trace against a Python session trace. Message numbers and timestamps are session-dependent and masked; the semantic sequence and payload shapes must match.
Diagnostics
The log mirrors the Python responder's shape so the two can be read side by side: connection id, peer, frame number, route, msgType, msgNum, userIndex, options, payload and metadata sizes, every response emitted, and a close reason.
OPENFUT_BLAZE_TRACE=<path> additionally writes the normalized structural
trace — the same format the probe emits, so a live FIFA session against Rust can
be diffed against one against Python.
No credential or token is logged. Volatile values are replaced before they reach the line, not truncated after, and a test asserts a known secret never appears in trace output.
Evidence capture per gate
./gate-evidence.sh <gate-label> # after each gate; never modifies anything
Writes a timestamped bundle (switch rules, sidecar status, log, trace, Python contract result) and reports configured and observed state separately.
That separation is the point. blaze-switch.sh status = ON is an assertion from
the same tooling that performs the switch — and that tooling reported a
successful rollback once when it had not happened. The observed half comes from
a different source: the sidecar's own record of which peers connected to it. A
non-loopback peer in the sidecar log is proof the client's Blaze traffic landed
on Rust that does not depend on reading an iptables rule correctly.
The script says so explicitly, in one of two forms:
REMOTE peer(s) reached the Rust sidecar: 10.10.0.x
=> the client's Blaze traffic observably landed on Rust
no remote peer connected — only loopback (or nothing) reached Rust
=> a FIFA session did NOT land here
Raw frame capture (opt-in)
Evidence infrastructure, off unless OPENFUT_BLAZE_CAPTURE names a file.
OPENFUT_BLAZE_CAPTURE=/home/alex/OpenFUT/captures/gate7.ofcap ./sidecar.sh start
Two layers, deliberately:
live FIFA traffic
├── raw capture exact RX/TX bytes, mode 0600, gitignored — NEVER commit
└── blaze-sanitize → repository-safe, replayable fixtures
The raw file is forensic evidence and does contain session material; that is what makes it worth keeping and why it never leaves the machine unsanitized.
Format. Deterministic, big-endian: a 20-byte file header, then per-frame records with connection id, a global monotonic sequence, timestamp, direction and the exact frame bytes. RX is recorded as received; TX only after a successful write, so a record means the bytes were sent, not intended.
Component, command, msgNum, msgType and payload length are not stored beside
the frame — they are already in its 16-byte header, and a redundant copy can
disagree with the bytes, leaving a reader unable to tell which is true.
Record::header() derives them.
Sanitizing
./target/debug/blaze-sanitize captures/gate7.ofcap -o gate7.jsonl --report gate7.txt
Redacts only the named tags (KEY, AUTH, SESS, MAIL, PML) and reports
every substitution with path, kind and byte length. Replacement is
length-preserving, so the TDF varint, payload length and Fire2 header are
unchanged and the sanitized frame is exactly the size of the captured one —
asserted per frame, failing rather than emitting a subtly different
conversation. Frames with nothing sensitive keep their exact wire bytes.
Payloads that will not decode are passed through and reported, so a reader
knows they were never inspected rather than assuming they were checked clean.
Real run: 101 frames in, 9 redacted, 13 redactions; two live session keys present in the raw capture, zero in the sanitized output.
What the tests do and do not cover
Nine cases, all passing: capture off produces no artefact; RX and TX captured exactly; ordering preserved; fragmented input (one byte at a time) reconstructs the same frames as a single write; coalesced input is captured per frame rather than per read; capture does not alter wire output; sanitization removes a real session key from a real captured login; malformed/truncated/wrong-version captures fail clearly; every listed sensitive tag is provably reachable.
Mutation-tested: dropping TX capture, truncating captured frames to their
header, and removing KEY from the sensitive list each turn the suite red.
One mutation is not caught: moving the TX capture above the write. It is indistinguishable while writes succeed and diverges only when one fails, where it would record a frame the client never received. That invariant is held by code placement and a comment, not by a test.