mutation-battery.sh injects each of the 14 required wrong behaviours into
the committed source, runs the one invariant test that must catch it, and
requires a non-zero exit (mutant killed), reverting via git after each.
Kills: kit-by-slot, captain-as-resourceId, chemistry-reconciled,
custom-regenerated, index-derived, stale-accepted, missing-fabricated,
faked-asset-id, duplicate-instance-collapse, PUT-as-slot-diff,
wire-id-in-canonical, projector-bypasses-shared-shaper, schema-version-
ignored, player-state-keyed-by-definition. 14/14 killed.
fut::squad_projection is the ONE projector for every squad read shape.
project_squad(canonical squad + Fresh extension + owned items) -> the FIFA
17 squad wire object; user_mass_info_squad and squad_list are envelope-only
wrappers over the same output (no per-endpoint domain model).
Design guarantees exercised by tests:
- purity / no N+1: consumes a host-assembled input (read_squad_with_ext +
one batch owned-cards fetch + in-memory card defs); no per-slot lookup
- shared shaper: every occupied slot is shaped by fut::item::shape_item,
so squad items and /club items cannot drift
- Fresh -> full projection; Stale -> never applied (verdict surfaced);
Missing -> explicit, never fabricated
- captain projects as the resolved WIRE id (never resourceId); index and
formation round-trip verbatim; kit follows the player; two owned copies
of one definition stay distinct
Adds committed sanitized fixtures decoded from the squad session capture
(swap, f433, persisted userMassInfo.squad read, squad/list) and
tests/squad_projection.rs: baseline / swap / formation-change / persisted
read-after-write round-trips asserted by ownership class (canonical,
extension, shadow, derived identity), plus one-projector no-divergence.
Add fut::squad_ext::Fifa17SquadExtensionV1 — the versioned, adapter-owned
payload Core stores opaquely alongside the canonical squad. Carries the
FIFA-only wire state that is not Core-canonical:
- custom[] opaque 33-int string, round-tripped verbatim
- squad_type observed FIFA token
- kit_numbers keyed by owned_card_id (kit follows the PLAYER, proven
by the swap/formation captures), never by slot/definition
- manager opaque item ref (not a squad player; not shaped)
- kicktakers opaque role refs; relationship to captain UNKNOWN, so
preserved verbatim and never normalized to the captain
- client_reported chemistry/rating/starRating shadow, never authoritative
from_payload enforces the payload schema version first (distinct from Core's
DB schema); an unknown version is rejected, never coerced.
build_squad_write turns a parsed PUT + host wire->owned resolver into a
canonical ProposedSquad + extension, refusing on unresolved ids or a
duplicate owned item. Identity resolution is explicitly NOT authorization.
Refactor the 550a59d parser scaffold: ProposedSquad is now pure canonical
(FIFA-only + shadow fields moved to the extension); the canonical formation
is the FIFA wire token verbatim (drop the lossy f442->"4-4-2" map that
could not even represent f433) so formation and index round-trip exactly
with no derivation. Bench split is the fixed 23-slot array convention.
Move the per-item card shaper (CoreOwnedItem, Fifa17Identity,
ItemIdentityResolver, ShapeStats, shape_item) out of club_response into
fut::item so /club and the upcoming squad projection emit byte-identical
items from one source of truth. club_response keeps only the /club
{itemData:[...]} envelope and re-exports the moved types for API
stability. shape_item is now pub; no behavior change (all /club and
oracle-parity tests unchanged and green).
Adds item-shaper tests: full-field identity mapping and the duplicate
owned-copy invariant (two instances of one definition keep distinct wire
ids, share one asset id).
scripts/seed_fifa17_cards.py: deterministic pipeline from committed FIFA17 data
(pool.json + roster.json + leagues/nations/teams tables) -> the card-definition
identity catalog. CardDefinitionId is opaque + deterministic (fifa17_<asset>),
version 0 (base cards only; resource_id == asset_id). --check mode diffs against
committed output (drift-detection mutation-proven). Provenance embedded.
Generated openfut-adapter-fifa17/data/fifa17-card-identities.json: all 17,563
base assets. Semantic definition coverage (to /tmp, not committed here): 17,547
resolvable; 16 skipped for missing roster name (reported, never fabricated).
Adapter loads the committed catalog (test: 17,563 entries, Ronaldo fifa17_20801
-> asset 20801 v0). Phase commit 3/5. NOT owned inventory: this is 'which cards
exist', not 'which the user owns'. Core content seeding + dev-owned set next.
fut::catalog — Fifa17CardCatalog maps a semantic CardDefinitionId to a FIFA 17
render identity (resource_id = (version<<24)|asset_id; version 0 => resource==
asset). Versioned JSON (schema_version=1, game=fifa17); validates schema/game,
rejects asset_id > 24 bits, and rejects two card ids claiming one resource_id.
Unknown definitions resolve to None (callers drop, never fabricate).
Fifa17WireItemIdPolicy carries the owned-item namespace (base 100_000_000,
first id 100_000_001, per the oracle) supplied to the generic store.
Adds serde derive to the adapter. 8 catalog tests; 3/3 mutations killed
(resourceId-drops-version, conflict-detection-off, asset-range-off).
Phase commit 2/5. No card->asset DATA shipped: the synthetic Core catalogue is
unmappable (see seed plan); the loader + format land now, population later.
openfut-identity: durable, reversible (game_id, entity_kind, core_id) <->
external wire id mapping. Game-independent infrastructure (adapters supply the
numeric policy via base_floor; the store guarantees stable/unique/reversible/
game-scoped/persistent/atomic/explicit). JSON-file backed behind an
ExternalIdentityStore trait (SQLite can drop in later); parking_lot-guarded,
atomic temp+rename persist, rejects a torn reverse-duplicate on open.
Core never learns FIFA integers; only the host/adapter that owns a game
boundary uses this. 6 tests, 4/4 mutations killed (same-id-for-two-items,
lost-on-restart, broken-reverse, dropped-game-scope). Phase commit 1/5.
openfut-utas-host: the first live UTAS host. Serves GET /ut/game/<title>/club
from OpenFUT Core via the FIFA17 adapter and reverse-proxies every other UTAS
route verbatim to the Python oracle. Plaintext HTTP/1.1 keep-alive (no TLS);
route classification before execution; a Core error on /club degrades to an
empty page and never falls back to Python. CoreAccess is a host-owned boundary
(the adapter stays transport-agnostic).
openfut-adapter-fifa17::fut: owned_query (wire parse + FIFA id->name mapping,
unknown id = hard error), entities (id<->name from committed tables), and
club_response (FIFA _item shaping; drops items lacking a real FIFA asset id,
never fabricates one).
openfut-core submodule advanced to the reconciled trunk (6acae54 = 8c8a4116
multi-game + eab522a replace_squad/SquadRules + the /club semantic query).
11 host tests + adapter fut tests; 10/10 host mutations killed. rare=SP UNKNOWN.
Retail rendering of Core inventory still blocked on the Core-card->asset-id
identity decision (next phase).
Controlled retail capture, one criterion at a time, cleared between each.
47 transactions. Every filter the My Squad picker sends is now known from
the wire rather than guessed.
Route: GET /ut/game/fifa17/club -- the picker hits UTAS and reuses the
general club-inventory route.
level=any|gold quality lowercase, ALWAYS present
rare=SP "Special" uppercase, OMITTED when off
position=ST position uppercase, omitted when off
nation=52 entity id numeric
league=13 entity id numeric
team=5 entity id numeric, NESTED under league
sort=desc client constant; the UI has no sort control
start=/count=11 pagination
Two encoding families: short string enums, and numeric FIFA ids. The ids
must never reach Core. Filters compose as plain ANDs in one query --
string and id filters alike -- so each maps independently.
THE ROOT CAUSE IS SELF-AMPLIFYING.
club_route honours type, team and league; it never reads start, count,
level, sort or year. Because start is ignored, every page returns the
same full set, so the client concludes the page was full and asks for the
next one. One scroll produced 22 requests and 6.2 MB, stopping at
start=200 only because the client gave up -- against a filtered set of 32
items that should have been three pages.
That also explains why the bug reads as erratic rather than broken:
league=13&position=ST returns every Premier League player instead of
Premier League strikers. Plausible, wrongly sized, hard to notice.
Measured filtered sets, from the real cluttered club -- these are the
acceptance test for the fix:
unfiltered 1962
league=13 350
league=13&team=5 32
Two client behaviours worth carrying forward: the picker fires a query
per highlighted entry, not per selection (two requests for one club
pick), and parameter ORDER is not stable, so parsing must be key-value.
FIXTURE SIZE: bodies over 4 KB are truncated in the committed fixture,
with body_full_len and body_full_sha256 retained, because the same 1.1 MB
club response repeats ~25 times and its hash already proves identity.
13.3 MB -> 247 KB. The raw .ofcap keeps every byte, privately and
gitignored. Truncation is recorded per transaction so a trimmed fixture
is never mistaken for a whole response.
Audited across all three identifier surfaces -- headers, JSON bodies,
query strings -- before and after the size change: no leaks. 6/6
sanitiser mutations still killed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
24 transactions across 11 connections from a retail session: login,
hub, one pack open, two squad saves, a quick-sell, with before/after
state manifests. Raw .ofcap stays gitignored at 0600; the sanitized
corpus is committed as adapter fixtures.
TWO GAPS FOUND BY AUDITING THE OUTPUT, NOT BY TRUSTING THE SANITISER.
1. `POST /ut/auth` carries `macAddress` and `deviceId`. Session tokens
were being redacted correctly and these were not. A committed fixture
is a published fixture.
2. Then, with those fixed, the audit fired AGAIN on the file about to be
committed: `GET .../phishing/trusteddevice?deviceId=...` puts the id in
the QUERY STRING. Three input surfaces carry identifiers -- headers,
JSON bodies, and query strings -- and the sanitiser knew about two.
Both fixed in the tool rather than by editing the file, with a
regression test and a mutation for the query path.
AND A THIRD ARTEFACT MIX-UP, in the mutation harness itself. It reported
the query-redaction mutation as SURVIVED while a hand-run of the same
mutation killed it. Cause: the harness pointed at a stale scratchpad copy
of the test that pre-dated the query assertion, so it was faithfully
testing the mutated tool against a test that could not detect the
mutation. That is the same class as the build guard checking the wrong
binary and cargo reusing a binary compiled from mutated source -- the
third instance today of measuring the wrong artifact. The harness now
resolves ROOT from its own location and runs the COMMITTED test; the
stale copy is deleted.
Harness committed as scripts/mutate-utas-observe.py so this is repeatable
rather than a thing that happened once in a scratch directory. 6/6 killed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
UTAS needs a real request/response corpus before any Rust is written: it
is where protocol shape and FUT state start being coupled, so guessing is
worse here than it was for Blaze. The oracle truncates logged bodies at
~200 chars, and raising that cap would mean editing the behavioural
specification to make it easier to copy -- backwards. A proxy gets the
same evidence and leaves the oracle untouched.
THE DESIGN RULE: TEE, DO NOT REBUILD.
UTAS is plaintext HTTP/1.1 on ThreadingHTTPServer, so keep-alive,
pipelining and chunked transfer are all live. A proxy that parses a
request and re-emits it can corrupt the traffic it exists to observe --
and that corruption would present as a UTAS bug, pointing the
investigation in exactly the wrong direction. So bytes are copied
verbatim in both directions and a second copy goes to disk; transactions
are reconstructed later, offline, from that copy. A parser bug therefore
spoils the record and never the session.
Standalone, NOT in the container, so the same tool can later sit in front
of a Rust UTAS host and replay an identical captured request against both.
Two layers, as with the Blaze captures: raw/*.ofcap is exact bytes at mode
0600 and gitignored; sanitized/transactions.jsonl is the committed
artefact. Bodies are preserved EXACTLY and sanitised second -- only
known-secret headers and JSON keys are replaced, structure is never
reshaped, and every redaction is recorded in the transaction so a reader
knows what was touched.
Captured per transaction: connection id, sequence, relative and wall
time, elapsed ms, method, path, query, HTTP version, headers IN RECEIVED
ORDER as pairs (a dict would drop duplicates and ordering), raw body and
length for both directions, status, and observed keep-alive.
Verified as two independent properties, because they fail differently:
transparency (bytes through the proxy identical to bytes direct, Date
masked, with the mask asserted to have fired) and fidelity (parsed
transactions match what was sent, including a dechunked response and a
300-byte POST body). 5/5 mutations killed, including "record but do not
forward", "drop the last byte of every chunk" and "stop redacting".
scripts/test-utas-observe.py is committed alongside it: a capture tool
nobody can re-verify is not evidence.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Queued cleanup, run only AFTER the roster gate closed in both directions,
so the live A/B changed exactly one thing.
The two `drain_body` implementations were character-for-character
identical, so the extraction is a move. What it guards is not cosmetic:
answering while the client is still sending leaves unread data in the
receive queue and Linux turns the close into an RST rather than a FIN --
invisible in any comparison of the response, and worth two live gate
attempts to find. Behaviour that must be identical across hosts gets one
implementation, the same reasoning that produced openfut-tls.
SCOPE IS DELIBERATELY NARROW. Only the byte-identical part moved. The two
head-reading loops are NOT identical and stay where they are:
redirector roster
head cap 65536 16384
read chunk 4096 1024
on error abort proceed if any bytes arrived
Those differences are probably accidental, but each host is gate-proven
with the values it has. Unifying them would be a behaviour change wearing
a refactor's clothes -- exactly the mistake this project has already paid
for. They converge later as their own change with their own gate, or not
at all.
Purity shown, not asserted: every existing test in both hosts still
passes (426 workspace tests), and 7/7 mutations are killed, including
three in the SHARED crate that must break both hosts at once and one per
host that skips the drain call.
Three test cases neither host had now exist, because the extracted code
finally had somewhere to be tested directly: a malformed Content-Length,
an unterminated head, and a lookalike header. That last one matters --
`X-Original-Content-Length: 99` would drain 99 bytes that were never sent
if the match were `contains` rather than `starts_with`, and a mutation
confirms the test catches it.
Also fixes a race this run exposed in openfut-tls's own tests: keypair()
returned early if the certificate file existed, but wrote the certificate
BEFORE the key, so a parallel test could observe a cert whose key had not
landed. It failed one run and passed the next -- the kind of flake that
gets rerun instead of fixed. Now generated once per process via OnceLock,
key written first, and the suite was repeated five times to confirm.
Nothing deployed and nothing restarted: the running redirector and roster
are still the gate-proven binaries.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
redirector.sh and the coming roster.sh needed the same five rules, each
of which cost something to learn:
* resolve /proc/PID/exe; never match a command line. `pkill -f` /
`pgrep -f` match any shell whose ARGUMENTS mention the name, including
the shell running the command. That has killed this session's own
shell twice, and is now banned in migration tooling -- the helper
contains no `-f` matching and the header says why.
* `readlink`, not `readlink -f`. After a rebuild the link reads
"<path> (deleted)" and -f resolves it to nothing, so the orphan check
goes blind to exactly the long-lived processes it exists to find. Two
orphans hid there, one serving the wrong certificate.
* stop PROVES the process is gone and the port free.
* an ambiguous binary is an error for start/verify but NOT for
stop/status: rollback must never be blocked by a question about the
build tree.
* verify the RUNNING process's commit, not the artifact on disk, which
a rebuild can silently advance past.
Copying those into a second script would have been the same mistake as
copying the TLS setup. Instead scripts/host-lifecycle.sh owns them and a
service supplies four facts: name, crate, executable, port variable.
redirector.sh goes from 178 lines to 26 and roster.sh is 24, with no
behaviour change -- the refactored redirector.sh still sees the live
armed process (pid 830736, port 42227) and still refuses correctly
because HEAD has moved past it.
Paths are unchanged (rundir, pidfile, portfile, commit stamp, log), so
the currently running redirector stays manageable across this refactor.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Second consumer of openfut-tls, and the reason it was extracted first.
This host contains no roster content and no cipher choice: the adapter
owns the 67 bytes and the observed TLS profile, openfut-tls owns the
acceptor, and this crate owns accept/read/drain/write/close.
Lifecycle was MEASURED, not inherited. The obvious mistake here would
have been copying the redirector's 300ms dwell because the other host has
one. A probe against the oracle says otherwise:
dwell after responding 0 ms (redirector: 300 ms)
request body drained POST answered only once it arrives
close clean FIN, never RST
keep-alive none one request per connection
The probe ran against a REPLICA of roster_server.py loaded from its own
source, not against :8081 -- http.server.HTTPServer is single-threaded
and FIFA was mid-session, so holding a connection open to measure the
close would have stalled the game's poll and could have surfaced as the
squad-update error. The replica was then confirmed byte-identical to the
live oracle under masking, the 1-byte delta being the container's Python
version in the Server header.
Differential against the live oracle, every field identical, with the
Server header compared UNMASKED:
GET 230B HEAD 163B POST 163B
drained=True reset=False answered_before_body=False
keepalive: second request accepted by the socket, never answered
Testing follows the redirector's hard-won rule: where a property is
visible both to the client and inside the host, it is asserted inside the
host via ConnOutcome. A client-side check cannot tell "drained" from "not
drained" -- it reads the buffered response either way -- and that exact
mistake let a mutation survive once already.
9 parity tests, 6 unit tests, 5/5 mutations killed, including "answer
before draining", "hold the connection open like the redirector" and
"inherit the redirector's 300ms default".
drain_body is duplicated from the redirector deliberately. Unifying it
means editing the redirector, and the roster A/B must change exactly one
thing. Extraction is scheduled for after the roster gate closes.
Not deployed and not switched: Python still serves :8081.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The redirector was the only host that spoke TLS, so its TLS lived inside
it. The roster host needs the same listener, and that made the choice
explicit: share this code or copy it.
Copying it is what already went wrong. On 2026-08-11 the Rust redirector
served one certificate while the container served another. ProtoSSL
caches the server certificate per backend, so the redirector -- the first
TLS connection of a session -- decided what the client expected, and
every later service failed its handshake. Silently: Python's socketserver
swallows ssl.SSLError as OSError. Three gates went to it. One place to
configure TLS is the structural fix, so it exists before the second host
does rather than after.
Split along the line the architecture already draws:
openfut-tls how to build an acceptor. Game-independent.
Knows nothing about which suites any client
offers.
adapter-fifa17::tls what FIFA 17 was OBSERVED to offer: the six
enabled suites, the two refused, the TLS 1.2
window, the EA SNI. Plain strings, so the
adapter keeps its lean dependencies -- reading
a card table should not build OpenSSL.
redirector-host joins the two. Chooses no cipher of its own.
Behaviour is unchanged, and shown to be:
* tests/fifa17_tls_profile.rs carries over every case from the deleted
module -- FIFA's eight suites negotiate AES256-GCM-SHA384, each enabled
suite works alone, RC4-only is refused, ECDHE-only is refused. Deleting
a module must not quietly delete its evidence.
* one test pins the composed values literally against the host as it was
when gates 1-14 passed. A "pure refactor" that cannot fail is not a
claim, it is an assumption.
* the rebuilt binary self-tests to the same TLSv1.2 / AES256-GCM-SHA384
the retail client negotiated at 17:09 today.
Two improvements fall out of having one place to look:
* the startup banner now prints cert_sha256. The mismatch above raised no
error at startup and broke the client much later with nothing logged;
it is now the first line of the log.
* tls_min/tls_max print as TLSv1.2 rather than SslVersion(771). This line
is gate evidence and gets read by people.
Nothing deployed and nothing restarted: FIFA is mid-session on the
running redirector, which is untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The old check was `grep easw /etc/hosts && echo ok`. It passed on ANY
matching line -- including a line that shadows ours. glibc returns the
first match, and the sed above only deletes lines this script wrote
(`# openfut`), so a foreign entry earlier in the file wins forever and
re-running the script never helps.
Observed today: a leftover `127.0.0.1 easw.easports.com` from the
single-machine era, before the backend moved to its own host. Every arm
reported "/etc/hosts ok" while the name resolved to loopback.
Now it resolves the name -- the same call the game makes -- and compares
address to address, so a server given as a hostname is handled too. On a
mismatch it prints the offending lines with line numbers and says how to
fix them.
It does NOT delete them. This script writes one tagged line and owns only
that line; silently removing entries a user put there by hand is a bigger
hazard than the shadowing it would cure.
Reported as a warning, not an error, because it is survivable: the
responders advertise the server address, so the game stops using this
hostname after the first redirected contact. FIFA reached the FUT hub
today with this exact misconfiguration in place. Claiming it is fatal
would be wrong, and a check that overstates its findings gets ignored.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two defects, both found by the guards misfiring rather than by reading:
1. BIN preferred target/debug and fell back to release only when debug was
absent. `cargo build --release` therefore produced a correct binary while
the script kept inspecting a stale debug one, and the build guard refused
with a message naming a commit nobody was trying to run. The guard was
right that something was stale — it just pointed at the wrong artifact.
Disagreement between the two is now an explicit refusal naming both, with
OPENFUT_REDIRECTOR_BIN as the deliberate override.
The refusal is recorded at load and raised only by `verify` and `start`.
`stop` and `status` must work in any build-tree state: rollback can never
be blocked by a question about which artifact would have been started.
2. `verify-running` read the stamp file without checking the process still
existed. The stamp outlives the process, so after a stop it reported on a
corpse — either "identity OK" or a REFUSAL naming a commit, both implying
something was running when nothing was.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`cmd_status` has two paths. The `--name` path filters on an exact tag and
works. The no-name path — the "show me every switch on this box" survey,
which is how an orphan switch under a different name would be found —
built its python with shell quote-juggling and never closed the string
literal, so it died with a SyntaxError every time.
It failed loudly (rc=1, a traceback) rather than reporting "no rules", so
it never lied about the state. But it also meant the survey path had
never once run, which is the more useful lesson: every branch of a safety
tool needs exercising, not just the branch the happy path takes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`list_procs` matched on `readlink -f /proc/PID/exe`. Once the binary is
rebuilt -- which happens constantly here, `cargo test` alone is enough -- the
link reads "<path> (deleted)" and -f resolves it to something that matches
nothing. The scan then finds zero processes, so `start`'s orphan check passes
and a second instance can be launched alongside a stray.
Not theoretical. Two orphans were running undetected tonight:
pid 592731 :42327 a stale-cert redirector left from testing check-tls-parity,
still serving F9:16:1A -- the exact certificate whose
mismatch cost three live gates
pid 542693 :42230 a Blaze sidecar debug build from 03:12
Neither was in a client path, so neither was doing harm, but a stray listener
serving the known-bad certificate is precisely what should never sit around
unnoticed.
Fixed by using plain readlink and stripping the " (deleted)" suffix. Shown both
ways: with the bug `status` reports no processes at all for a live pid; with the
fix it reports 604454. The pidfile path was unaffected, which is why `stop` kept
working and hid this.
It is a TOTAL -- every packet reaching the chain counts there, including ones
already counted by a named-port rule. The old wording invited reading the number
as a remainder, which is how 15 unexplained attempts got misread earlier.
Next component in the migration order (Roster -> LSX -> UTAS). Adapter layer
only: no host, no runtime replacement, nothing armed.
The response is shaped as much by http.server.BaseHTTPRequestHandler as by the
oracle's handler code, so it is captured over the wire rather than reasoned
about:
* HTTP/1.0 status line -- protocol_version is left at its default, so the
reply is 1.0 even though the client asks for 1.1
* send_response injects Server: and Date: BEFORE the handler's own headers
* POST answers with headers only: the handler writes the body `if method ==
"GET"`, so a POST advertises Content-Length: 67 and then sends nothing
That last one is preserved, not corrected. It looks like a bug, but "obviously a
bug" has been the wrong call before in this port, and a test now asserts it so a
future cleanup has to argue with something.
Date and Server are volatile and are MASKED in the fixture rather than dropped,
so their presence and position are still asserted. Server is additionally
recorded verbatim: it carries the container's Python version, so a drift away
from roster::ORACLE_SERVER fails a test instead of silently changing every byte
we emit.
generate_roster.py --check FAILS when it cannot reach the oracle rather than
passing, and mutation-testing the mutation harness itself caught two "surviving"
mutations that were really sed no-ops. With application verified, all four
mutations (header order, Content-Length, XML body, Connection) are killed.
Three times now the same sequence has broken the client path: a build guard
correctly refuses to start the Rust replacement, and the `switch on` that
follows in the same script arms anyway, because it never checked whether
anything was listening. The redirect then lands on a closed socket and the
working Python service is bypassed for no benefit.
`on` now refuses unless the target port is listening. ALLOW_DEAD_TARGET=1
overrides it for arming ahead of a service that is about to start, but that has
to be deliberate. Verified both ways: rc=2 and nothing installed against a dead
port, rc=0 and two rules with the override.
The watchdog now writes a pidfile. Stopping it by command-line match is unsafe
-- any shell whose arguments merely mention the script name matches too, which
has now killed the wrong process twice here (once via `pkill -f`, once via a
/proc/*/cmdline substring loop).
`openfut-switch.sh on` prints "the service MUST stay up" -- true, and useless
when nobody is at the terminal. An armed switch pointing at a dead port means
the client hits a closed socket with no fallback.
This turns the documented rollback into an automatic one, failing toward the
Python oracle. The worst case of a spurious trip is a gate needing re-arming;
it can never leave the client broken.
It only ever REMOVES a switch. It does not install one, restart the Rust
service, or touch Python, and it does not re-arm after tripping -- an
unexplained rollback should be a finding to read, not something hidden by
flapping the switch back on.
The probe goes through the switch and speaks TLS, because a bare TCP connect
would succeed against a process wedged mid-handshake.
Tested both directions, not just the happy path: quiet for 45s against a
healthy service, and against a stopped one it failed 3/3 in 9s, rolled back,
and left Python serving -- verified by re-reading all four tables and by which
implementation's log grew.
`verify` inspects `$BIN --identity`, which is the file on disk. That is not
necessarily what is serving. Caught during gate 14 setup: the live process had
been started from 5bc39e9, then `cargo test` re-ran build.rs (the branch ref
moved when an unrelated script was committed) and restamped the on-disk binary
to fc411bb. `verify` then reported "build identity OK" about an artifact that
was not the running service.
`start` now records the stamped commit to $RUNDIR/redirector.commit, and
`verify-running` compares THAT against HEAD, refusing when they differ. The
existing on-disk check stays -- it is the right gate for "may I start this" --
but only the recorded stamp answers "is the thing currently serving the thing I
think it is", which is the question a live gate's evidence depends on.
Two live gates were lost to a second variable I had been asked to eliminate.
The Rust redirector was pointed at the repo's fifa17-recon/tools/redir_cert.pem
(fingerprint F9:16:1A...), while the running container serves a different cert
baked into its image (E7:F9:46...) which the Python redirector, roster and the
rest of the stack all share. So the A/B compared TLS implementation AND
certificate identity at once.
FIFA 17's ProtoSSL caches the server certificate for a backend. The redirector
is the first TLS connection of a session, so its cert becomes the one the client
expects; the next service presenting a different cert fails its handshake. That
is why the redirect itself always succeeded and the failure surfaced later, on
the roster fetch -- "An error occurred downloading the FUT Squad Update".
It stayed invisible because Python's socketserver swallows it: a handshake
failure at accept() raises ssl.SSLError, which subclasses OSError and is
discarded by _handle_request_noblock. No request log, no stderr. Every server
looked healthy while the client could not talk to any of them.
Confirmed on the wire: tls-observe in front of the roster server captured four
ClientHellos from the client, correct SNI and the same 8 static-RSA suites it
offers the redirector, none of which produced a request.
The check is mutation-tested against the real bug: with a redirector started on
the stale repo cert it exits 1 and names the mismatch.
openfut-observe.sh answers the one question no server log can: when a gate
fails and a service logged nothing, did the client try and fail, or never try?
Both look like silence. Two redirector gates were lost to that ambiguity --
"roster server logged nothing" was equally consistent with a broken roster
service, a wrong roster URL, and a client that never asked.
Built on iptables packet counters because this box has no tcpdump, no
conntrack, and no readable kernel log. That last one is verified rather than
assumed: an initial LOG-based version installed correctly and its rules matched
(counters proved it), but the output went nowhere -- journalctl -k has no
entries and dmesg is empty. Counters are also lower volume and record only SYNs,
so no payload can be captured even in principle.
Validated against the live client, not a loopback stand-in: an initial
self-test using this host's own address counted almost nothing, because
locally-generated packets never traverse PREROUTING. Against the real remote
client it counts 8081 at ~4/min, matching the roster server's own log.
Known gap, recorded rather than hidden: the catch-all TOTAL runs well above the
sum of the named ports, so the client makes steady background attempts to ports
not tracked here. It is present during a working session, so it is not the
failure signature, and it is not chased further here.
verify-build-identity.sh now rejects an argument that is not a commit hash.
Passing the binary path instead of its stamp previously produced a plausible
"REFUSING: binary was built from ./target/release/... but HEAD is <sha>", which
reads as a real stale-build finding rather than a caller mistake -- and a
safeguard that cries wolf is one people learn to route around. Usage error is
now exit 2, distinct from a genuine stale build (1) and success (0).
Two live gate attempts failed with "An error occurred downloading the FUT
Squad Update" while the redirect response was verified byte-identical to the
Python oracle. Rolling back to the Python redirector fixed it, so the response
bytes were never the whole contract.
Log archaeology found the discriminator: the client polls
/fifa17/fut/rosterupdate.xml ~4x/min in every successful FUT session, and the
only gap in 300 recorded fetches is 03:47-03:58 -- exactly the two
Rust-redirector sessions. The Blaze RPC sequence over those sessions is
identical (msgNum 0-53), so the divergence is entirely outside Blaze.
A differential lifecycle probe against both redirectors found the two
behaviours this host never reproduced:
* the oracle drains the request body per Content-Length; this host stopped
at the header terminator, leaving unread data in the receive queue, which
makes Linux close with RST rather than FIN
* the oracle holds the connection open ~300ms before closing
(time.sleep(0.3)); this host closed at 0ms
Both are now reproduced. The dwell is a named constant, ORACLE_CLOSE_DWELL,
overridable only so the causal experiment -- set it to 0, confirm the failure
returns -- can be run without a rebuild.
The suite could not have caught either: it sent Content-Length: 0, so there
was never a body to drain. It now POSTs a body, and asserts a split-write body
is fully consumed.
Testing the drain via client-visible symptoms does NOT work -- verified by
mutation: with the drain removed the client still reads the buffered response
and sees close_notify before any reset. So the host records a per-connection
ConnOutcome and the test asserts on that. Both mutations (no-dwell, no-drain)
are now each caught by exactly one test.
This does not yet prove causation for the FUT Squad Update failure; it removes
the only two measured divergences. Gate 6 is the test.
The binary records only the commit it was built from -- no dirty-tree flag.
Cargo will not re-run a build script because another crate's source changed, so
a compiled-in 'clean' claim can be stale and is not a safeguard; that was
verified on the Blaze host.
scripts/verify-build-identity.sh establishes both facts at LAUNCH, where they
cannot go stale: the stamped commit equals HEAD, and the migration crates are
clean. It REFUSES rather than warns, because for a migration gate a warning on
stderr is something to scroll past.
--identity prints the stamp without valid configuration. The launcher must be
able to establish which commit a binary came from BEFORE deciding whether to
run it; requiring a correct environment first would invert the check.
redirector.sh mirrors sidecar.sh: refuses to start with an orphan present or
the port busy, matches the resolved executable rather than the command line
(pgrep -f matches any shell mentioning the name), and stop PROVES the process
is gone and the port free.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
TLS DEPENDENCY, as directed: the openssl crate directly with the `vendored`
feature. NOT native-tls. native-tls abstracts over whatever the platform
provides; here the requirement is the opposite -- precise, evidenced behaviour
for one legacy client -- which needs explicit control of the cipher list,
protocol floor/ceiling and security level. Vendored so a distro libssl update
cannot silently change whether FIFA 17 can connect.
Scoped to this crate alone. Neither OpenFUT Core nor the generic protocol
crates gain an OpenSSL dependency.
CIPHERS driven by the captured retail ClientHello, not by generic legacy
assumptions. The six RSA+AES suites it offers are enabled; RC4 and MD5 are
deliberately NOT, even though the client offers them -- it already negotiates
AES256-GCM-SHA384, so resurrecting RC4 for completeness would weaken the
service for nothing. TLS 1.2 floor and ceiling, matching the observed client;
the floor is not dropped to 1.0 pre-emptively because "the oracle permits it"
is not "the client requires it".
SECURITY LEVEL IS NOT LOWERED. Tried the default policy first, as directed,
and OpenSSL 3.6.3 accepts static-RSA/AES without weakening. No SECLEVEL change
was needed and none is applied; it remains overridable per-listener with
evidence.
CERTIFICATE: the proven Python redirector's material is reused, so the TLS
implementation stays the only variable in an A/B. Verified RSA-2048, CN
winter15.gosredirector.ea.com, cert/key modulus match; the key stays
gitignored.
SHARED CONFIG. New openfut-host-config is now the only crate that reads the
environment, and both hosts resolve endpoints through it. Two hosts each
parsing OPENFUT_ADVERTISE would be exactly the "separate helpers constructing
endpoints from different sources of truth" the address audit forbids.
VERIFICATION BY REAL HANDSHAKE, not by enumeration. The crate exposes no
accessor for a context's configured suites at this version, which turned out
better: the host now rehearses the retail handshake at startup with a client
restricted to exactly FIFA's eight suites and REFUSES TO SERVE if it fails, so
a cipher/version misconfiguration surfaces at boot rather than as an
unexplained failure during a live gate.
Gates 1-5 pass: TLS config unit tests; a FIFA-suite-only client negotiates
TLSv1.2/AES256-GCM-SHA384; each enabled RSA+AES suite negotiable alone; an
RC4-only client is refused; an ECDHE-only client is refused (proving no modern
policy was silently inherited); a full HTTPS round-trip returns bytes
IDENTICAL to the Python oracle's recorded response.
Cargo.lock committed for reproducibility: openssl 0.10.81, openssl-sys 0.9.117,
openssl-src 300.6.1+3.6.3 (OpenSSL 3.6.3). Updating openssl-src is NOT a
routine bump -- it requires re-running the FIFA compatibility gates.
Gates 6-14 need the retail client and are next.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Blaze switch was hardwired to 42130 and could not intercept the redirector.
Rather than clone it, the iptables logic now lives in one place:
openfut-switch.sh generic: --server-ip --intercept-port --target-port
--name [--client-ip] [--legacy-tag]
blaze-switch.sh thin wrapper, CLI and output UNCHANGED so the validated
gate runbook and sidecar.sh's cross-check keep working
No deployment IP or port literal in the generic tool; 42130 is supplied by the
wrapper, 42127 by the redirector experiment.
VERIFICATION IS INDEPENDENT OF REMOVAL. Rules are created and deleted by their
comment tag; they are verified by parsing the kernel's own FIELDS (chain,
destination, dport, to-ports) with no reference to the comment. Status detects
duplicates, incomplete pairs, conflicting targets under one name, and foreign
redirects on the same port -- which it reports but never deletes. `off` removes
only rules bearing this switch's exact tag, then re-reads the table to confirm.
THREE BUGS FOUND WHILE BUILDING IT, all in the same family as the original
lying rollback:
1. Renaming the tag ORPHANED live rules. Gate 10 deliberately ended with the
switch on, so rules carrying the old tag were still installed and the
renamed tool could not see them -- `off` would have reported success while
traffic stayed redirected. Hence --legacy-tag: a rename must not strand
rules it owns.
2. Deleting by re-feeding the raw `iptables-save` line through the shell fails
on this iptables, which prints `--comment "tag"` WITH quotes; word-splitting
leaves the quotes inside the value so nothing matches. Bare-comment rules
deleted fine, which is exactly what made it look like it worked. Deletes are
now rebuilt from parsed fields and passed as argv elements.
3. `IFS=$'\t' read` collapsed consecutive tabs because tab is IFS *whitespace*,
so an absent `-s` shifted every later field left and produced
`-s <dport> --dport <to_ports> --to-ports ''`. Harmless here, but a shifted
spec that matched a real rule would delete the wrong one. Now uses \x1f.
Mutation-tested against all seven required cases: wrong intercept port, wrong
target port, missing rule, duplicate rule, changed comment representation
(bare vs quoted), and a rollback that leaves a foreign redirect installed --
which exits non-zero rather than claiming success.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The redirector TLS question cannot be answered from the cipher OpenSSL
selected: its server follows client preference by default, so FIFA preferring
static RSA does not prove ECDHE was unavailable. Choosing a TLS stack on that
inference would be a guess. This reads the actual ClientHello.
PASSIVE BY CONSTRUCTION. Bytes relay verbatim both ways, nothing is injected
or rewritten, and the handshake is still terminated by the untouched Python
redirector. A parse failure logs and relays anyway -- observation must never be
able to break the path it observes.
Reports record/client version, supported_versions, SNI, every offered suite by
name, extensions, and a verdict on whether ANY forward-secret suite is offered,
which is exactly the rustls question. Unknown suites print as hex rather than
being dropped.
Verified end to end against the live Python redirector with openssl s_client:
31 offered suites parsed, 18 classified forward-secret, and Python logged the
relayed request and served its 406B serverinstanceinfo -- proving observation
AND pass-through in one run.
Unit-tested on truncated and non-TLS input; the verdict is asserted in both
directions so a static-RSA-only hello reports RULED OUT rather than defaulting
to the permissive answer.
NOTE: that 18-suite result is from openssl s_client, NOT from FIFA. It proves
the instrument works. The actual question is still open until a retail FIFA
ClientHello is captured.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Cheap insurance, explicitly not the real check -- the semantic tests in
deployment_config.rs are what prove propagation, using two TEST-NET addresses
and bind != advertise. This grep only stops the lab subnet reappearing months
from now when the reasoning has been forgotten.
Deployment config legitimately contains real addresses and lives in gitignored
files, so it is never scanned. The frozen baseline doc is allowlisted BY PATH:
it records what a past deployment actually was, and rewriting it would falsify
the record.
Also swapped the lab IP for a TEST-NET placeholder in the usage examples and
error messages of compose/entrypoint/client_arm. Those were already correct
architecture -- every one requires the address via ${VAR:?} -- but using the
real lab IP as the example is the same 'happens to match our lab' smell, and
placeholders keep the tripwire allowlist near-empty.
Mutation-tested: adding a lab address to a source file makes it exit 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Mandatory OpenFUT architecture audit. Two real defects found and fixed, plus
the config surface tightened so neither class can recur.
DEFECT 1 -- hidden localhost fallback. The Rust host defaulted POW hosts to
127.0.0.1 while every other URL followed OPENFUT_ADVERTISE, so a remote
deployment would emit loopback POW URLs and fail far from the cause. It also
diverged from the deployed Python entrypoint, which derives them
(POW_HOST="${POW_HOST:-$ADV:8094}"). POW endpoints now derive from the
advertised address; explicit overrides still win.
DEFECT 2 -- Default gave loopback silently. `Endpoints::default()` and
`AdapterConfig::default()` supplied 127.0.0.1, so anything constructing a
config by omission got loopback with no signal. Both `Default` impls are
REMOVED. Loopback is now `Endpoints::loopback()` / `AdapterConfig::loopback()`:
an explicit, greppable decision. Production uses `advertising(host)`.
CONFIGURABILITY. `blaze_port` and `utas_port` are now config, not literals.
The advertised Blaze port is our choice -- the client goes wherever
<serverinstanceinfo> sends it -- and 8099 is the client's own built-in default
but still deployment config. A bad port value is an error, not a silent
fallback to the previous one.
TEST-NET EVERYWHERE. Committed fixtures and tests used the lab's real LAN
address; a test that passes because its constant matches the current lab
proves nothing about relocatability. Redirector fixtures regenerated on
RFC 5737 TEST-NET-1/2/3 plus loopback. Harness scripts no longer default the
client IP to the lab address -- client-state.sh now requires it.
SEVEN REQUIRED TESTS in tests/deployment_config.rs plus host-side coverage:
remote config never silently becomes localhost; missing advertise fails
clearly; bind may differ from advertise; changing the Blaze port changes the
redirect; changing the host updates all 200+ generated URLs with no
stragglers; no helper bypasses central config; mutations are detectable.
MUTATION TESTED, and it found a hole in the audit tests themselves. Hardcoding
utas_base, reverting the POW derivation and re-hardcoding the Blaze port were
all caught. Making the redirector read `bind` instead of `advertise` was NOT:
`advertising()` sets bind == advertise, so the two sources were
indistinguishable. That is the single most likely bypass -- the oracle really
does read bind for nucleusConnect -- so the test now forces bind != advertise
and asserts the bind address never reaches the wire. Re-mutated: caught.
Wire behaviour unchanged: oracle fixtures still current, 153 tests green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
REDIRECTOR. The first hop's <serverinstanceinfo> XML, byte-for-byte against
the oracle across three advertised addresses. Owns the response only; TLS and
HTTP transport belong to a host, exactly as the Blaze adapter owns dispatch
while the sidecar owns the socket.
The <secure>0</secure> field is the client being told the second hop is
plaintext -- independent corroboration of the plaintext Blaze finding, now
expressed in code.
NUCLEUS IS NOT PORTED, and that is a finding rather than an omission.
Instrumented across every live session:
listener bound YES 0.0.0.0:42131 since 00:21:32
handler logs on connect YES unconditional, before any parsing
client received the URL YES OSDK_NUCLEUS fetched 10+ times
client connected NO zero requests, including 4 full FUT flows
So the long-standing nucleusConnect=0.0.0.0 anomaly is explained: FIFA never
follows that URL on this path. The invalid address has never mattered because
nothing dials it. Porting the stub would add an untested component for no
parity gain.
TLS CONSTRAINT RECORDED, NOT RESOLVED. All 9 observed handshakes negotiated
AES256-GCM-SHA384 = TLS 1.2 with STATIC RSA key exchange. rustls supports only
forward-secret (EC)DHE suites and cannot serve that. Whether the client also
OFFERS ECDHE is unknown -- OpenSSL follows client preference by default, so
preferring static RSA does not prove it is the only option. This must be
instrumented from a real ClientHello before a TLS stack is chosen; the module
docs say so rather than guessing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Host-side ss cannot see the Python backend's connections: the responders run in
a container, so a client session terminates at 172.20.0.2:42130 inside its
namespace and the host only sees the NAT'd flow. 'ss | grep <client>' on the
host therefore reports nothing while a session is very much alive.
That produced a wrong precondition: 'no .105 Blaze session -- closed' was
reported while FIFA was mid-session on Python, and gate 9 was armed against a
client that had never exited. Python's own log had the answer -- it logs closes
reliably and there was no close for that session.
client-state.sh looks in both namespaces, reports Rust and Python separately,
and exits non-zero while any session is live. An unreachable container counts
as 'cannot confirm', not as 'clear'.
Fourth measurement bug in this tooling, and the most consequential: the other
three mis-COUNTED, this one mis-STATED a precondition and caused an action.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
'pack opens recorded: 45' appeared in the gate 8 report. The UTAS log is
cumulative across the entire deployment, so a bare count reads as if 45 packs
were opened during that gate; the real number was 1.
Now reports both, labelled, windowed from the sidecar's start time (it is
restarted per gate, so that is the gate boundary). A bare count in a gate
report will be read as belonging to that gate, so it has to be the one that
does.
Third counting bug in this tooling: the trace frame counter matched OPEN/CLOSE
markers, the capture and trace were read seconds apart during a live session,
and now this. Evidence tooling gets the same scrutiny as the code under test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three fixes, all found while closing out gate 7.
1. Frame count was wrong. It counted lines matching '^conn-', which also
matches the OPEN/CLOSE lifecycle markers, inflating the figure by one or
two. Compared against the capture's record count that looked like a
capture/trace divergence (89 vs 88) when there was none: read at the same
instant, both report 99. Evidence tooling that miscounts is exactly what
this project cannot afford.
2. The raw capture is now copied into the evidence bundle (0600), so a gate's
forensic bytes travel with its report.
3. FUT actions are now observed on the UTAS side. 'Known FUT action succeeded'
is a client-side fact, but FUT actions go over UTAS -- which is never
switched -- so the UTAS log confirms them independently of anyone's
recollection. Gate 7's pack open shows up as:
STORE: opened pack Special Players Pack -> 11 items
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
'Rust did not receive it' is weaker than 'Python did'. The redirector always
runs on Python and is never switched, so it advertises the Blaze endpoint on
every run; whether Python then receives the Blaze CONNECT it just advertised
says where the hop actually went.
This is already visible in the existing logs and settles gate 5-6 more firmly
than the sidecar record alone:
02:02:13 Python REDIR SENT -> 10.10.0.120:42130 (to .105)
02:02:13 Rust conn-0005 CONNECT from 10.10.0.105
Python received NO Blaze CONNECT
Same second, both sides: Python advertised the endpoint and did not get the
connection; Rust did. It is also the mechanism gate 8 needs in reverse.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Committing updates refs/heads/<branch>, not the HEAD file, so watching HEAD
alone left the stamp one commit behind -- observed live, the banner read
a84a72e immediately after 2337431 was committed. build.rs now also watches the
resolved branch ref.
Belt and braces, since cargo still cannot see every source change: sidecar.sh
compares the binary's stamped commit against the tree's real HEAD at launch and
says so loudly on a mismatch. An evidence artefact that names the WRONG commit
is worse than one that names none.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Evidence infrastructure, not protocol functionality. Built before gates 7-10
because those sessions cannot be reproduced -- a later run is a different
session, and the migration-validation runs happen once. Gates 5-6 already went
past without their bytes being recorded.
TWO LAYERS
live FIFA traffic
├── raw capture exact RX/TX bytes, mode 0600, gitignored
└── blaze-sanitize → repository-safe, replayable fixtures
CAPTURE. Off unless OPENFUT_BLAZE_CAPTURE names a file. Deterministic
big-endian container: 20-byte file header, then per-frame records carrying
connection id, a global monotonic sequence, timestamp, direction and the EXACT
frame bytes. RX is recorded as received; TX only AFTER a successful write, so a
record means the bytes were sent rather than intended.
Component/command/msgNum/msgType/payload length are deliberately NOT stored
beside the frame: they are already in its 16-byte header, and a redundant copy
can disagree with the bytes, leaving a reader unable to tell which is true.
Record::header() derives them, so every field the requirements name is
available without duplicating it.
SANITIZER. Redacts only the named tags in SENSITIVE_TAGS (KEY, AUTH, SESS,
MAIL, PML) and reports every substitution with path, kind and length.
Replacement is LENGTH-PRESERVING, so the TDF varint, payload length and Fire2
header are unchanged and the sanitized frame is exactly the size of the
captured one -- asserted per frame, failing rather than emitting a subtly
different conversation. Frames with nothing sensitive keep their exact wire
bytes. Payloads that will not decode are passed through and REPORTED, so a
reader knows they were never inspected rather than assuming they were checked.
TESTS. 39 in this crate. All nine required cases: capture disabled produces no
artefact; RX and TX captured exactly; ordering preserved; fragmented input
(one byte at a time) reconstructs the same frames as a single write; coalesced
input is captured as separate frames, not per-read; capture does not alter wire
output; sanitization removes a real session key from a real captured login;
malformed/truncated/wrong-version captures fail clearly; every listed sensitive
tag is provably reachable.
MUTATION TESTED. Dropping TX capture, truncating captured frames to their
header, and removing KEY from the sensitive list were each verified to turn the
suite red. One mutation was NOT caught: moving the TX capture above the write.
It is indistinguishable while writes succeed and only diverges when one fails.
That invariant is held by code placement and a comment saying so, not by a
test, and the code says as much rather than implying coverage it does not have.
Python oracle unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
funman300