openfut-import-fifa17/tests/durable_import.rs drives the REAL import pipeline
(analyze -> Report dry-run; emit_content; plan_apply -> GenericImportRequest +
deterministic identity mappings + watermark; the staging/preflight/seed/
post-validate gates over a real JsonIdentityStore; openfut_core::services::
import::apply_profile_import in one Core SQLite tx) against a disposable
temp-file Core DB, from a small sanitized in-test fixture (750000 coins, 3
resolvable base players, unopenedPackIds [70,70,101], one squad).
Five ordered steps on one durable target, all green:
A dry-run: report exposes persona/coins/inventory/unopened/fingerprint; ZERO
DB mutation (all Core tables COUNT=0, identity store empty).
B apply: coins=750000 exact, owned=3, packs=3 (opened=0), squad_players=2,
deterministic owned ids, import_fingerprint recorded, identities reverse-
resolve both ways, watermark=100000600.
C restart: close+reopen the SAME sqlite file -> identical state.
D re-apply same source -> AlreadyImported (fingerprint), no doubling.
E conflict (coins 750000->750001 flips the fingerprint for the same game)
-> apply fails closed ('different source'); DB unchanged.
Fingerprint = FNV-1a-64 hex of the source snapshot, carried into
ProfileImportRequest.source_fingerprint = Core profiles.import_fingerprint, the
per-game rerun-identity key. dev-deps added to openfut-import-fifa17
(openfut-core path, tokio, sqlx). No production/live data.
Spawn Core (axum) on an ephemeral loopback port backed by a disposable temp-file
SQLite, seed a fifa17 profile via the real Core HTTP API, then drive the HOST's
REAL transport (HttpCoreClient: CoreEconomy) + handlers against it — no fakes:
credits reads Core balance; match-reward writer credits via Core grant_reward;
purchasegroup full-gen renders the owned pack from a Core entitlement (no
sentinel); userMassInfo overlay derives coins from the same Core state
(credits==massinfo==Core invariant). Restart phase reboots Core from the same
on-disk DB and proves coins + entitlements persist. Temp dir + 127.0.0.1:0 only;
no prod DB/ports/containers/.105. dev-deps: openfut-core, tokio, axum.
FIFA17-specific orchestration that installs the real profile across BOTH durable
stores (openfut-identity + Core SQLite) recoverably and idempotently, handing
Core only a GENERIC request (all FIFA17 semantics stay in this adapter layer).
apply module:
- owned_item_id(persona, wire) = deterministic UUIDv5 from a private namespace;
identical in BOTH stores (identity core_id AND Core owned_cards.id), so the
running host's wire->owned reverse lookup resolves exactly what Core stored.
- plan_apply(report, raw_profile, fp): pure translation to a GenericImportRequest
(card_id = fifa17_<resourceId>) + the preserved (owned_item_id <-> source wire)
mappings + watermark. Canonical squad + Fifa17SquadExtensionV1 are built by the
SAME adapter code (parse_squad_put + build_squad_write) the retail-validated
live squad path uses. Refuses if the report has blockers.
- Two-store protocol (apply): staging gate -> local Core-preflight mirror ->
identity dry-preflight -> idempotent seed (insert_existing_mapping per instance
+ set_watermark) -> ONE generic Core import transaction (spawned binary) ->
cross-store post-validation -> completion record. A crash after identity
seeding re-converges on re-run (idempotent mappings + Core already_imported):
no cleanup, no reminting.
- gate_staging: deferred players are ABSENT from an import; allowed only for a
staged run behind --allow-deferred-players-for-staging (never a silent default;
prints an INCOMPLETE banner). Production requires zero deferred instances.
CLI: --apply (with --emit-content, --core-bin, --core-db, --core-data,
--identity-store, --allow-deferred-players-for-staging). Depends on
openfut-adapter-fifa17 + openfut-identity + uuid(v5).
Proven end-to-end on the real 33068179/CAGE profile (staged): first apply
imports 1949 supported instances (293 base + versioned), 11/11 f433 squad +
opaque extension, coins 28,112,944, fingerprint 8dc5582d2414af28; re-run is an
idempotent no-op (already_imported, DB unchanged); staging-not-default refuses
before any write; every OwnedItemId is an opaque UUID; identity wire-id set ==
source supported set exactly (0 minted, 0 dropped); 13 deferred instances leak 0.
10 new apply tests (determinism, request/mapping/squad translation, blocker
refusal, staging gate both ways, local preflight, identity seed/dry/postvalidate/
idempotency, conflict detection, graceful spawn failure). clippy -D warnings
clean; crate suite 23 tests green.
Fold entity resolution (nation/league/club id->name via committed tables,
mirroring seed_fifa17_cards.py) and quality-tier rarity into the analysis, so
the supported set is honest about unresolved entities too. Add an explicit
--defer-conflict <rid> allowlist: a reviewed conflict (169193) defers, any NEW
conflict still hard-fails (defer never becomes a silent conflict suppressor).
--emit-content writes three files, PUBLIC content separated from PRIVATE account
state: fifa17-production-cards.json (Core CardDefinition[] keyed fifa17_<resourceId>,
base+versioned, tier rarity, profile-derived, no promo labels), a versioned host
identity catalog {card_id:{asset_id,version}}, and a private import manifest
(supported instances' wire ids + deferred set with reasons + preserved watermark
+ target profile + snapshot fingerprint). Emit refuses while blockers exist.
Real profile (33068179/CAGE): 1681 supported defs (150 base + 1531 versioned),
9 NoName deferred, 1 approved-deferred conflict (169193, 4 copies), 1949
importable instances, watermark 100004617 -> next 100004617, active squad f433
11/11 supported. fmt + clippy -D warnings clean; 13 tests.
openfut-identity:
- insert_existing_mapping(game,kind,core_id,external_id): preserve an existing
external wire id instead of minting; idempotent for an identical mapping,
rejects conflicting forward/reverse with IdError::Conflict, persists atomically.
- persisted per-scope allocator watermark (set_watermark/watermark_for) so a
future mint continues past the source high-water even across burned-id gaps;
next id = max(base_floor, live_max+1, watermark). Backward-compatible on-disk
format (legacy bare [Row] still loads). +4 tests (10 total).
openfut-import-fifa17 (new): read-only dry-run analysis of a real FIFA17 Python
profile for a faithful Core import. Enforces disjoint item-class balance;
proposes profile-derived CardDefinitions keyed fifa17_<resourceId> (base vs
versioned never collapse) with a resourceId-group consistency gate (hard-fail on
disagreement, never pick a winner) and honest buildability (roster name +
version formula + metadata, never fabricated); plans owned-instance identity
(preserve Python wire ids, preserve nextItemId watermark); checks active-squad
coverage. --apply/--emit-content refuse to write in this phase. 11 tests.
Real profile (33068179/CAGE) dry-run: 1982 items balance (1962 players + 17
consumables + 3 staff); 1681 supported defs (155 base + 1535 versioned), 9
NoName unsupported, 1 hard conflict (resourceId 169193: one of 4 copies has a
divergent nation/team/league); 1949 importable player instances, watermark
100004617 -> first new alloc 100004617; active squad f433 fully supported.
fmt + clippy -D warnings clean.
fut::catalog — Fifa17CardCatalog maps a semantic CardDefinitionId to a FIFA 17
render identity (resource_id = (version<<24)|asset_id; version 0 => resource==
asset). Versioned JSON (schema_version=1, game=fifa17); validates schema/game,
rejects asset_id > 24 bits, and rejects two card ids claiming one resource_id.
Unknown definitions resolve to None (callers drop, never fabricate).
Fifa17WireItemIdPolicy carries the owned-item namespace (base 100_000_000,
first id 100_000_001, per the oracle) supplied to the generic store.
Adds serde derive to the adapter. 8 catalog tests; 3/3 mutations killed
(resourceId-drops-version, conflict-detection-off, asset-range-off).
Phase commit 2/5. No card->asset DATA shipped: the synthetic Core catalogue is
unmappable (see seed plan); the loader + format land now, population later.
openfut-identity: durable, reversible (game_id, entity_kind, core_id) <->
external wire id mapping. Game-independent infrastructure (adapters supply the
numeric policy via base_floor; the store guarantees stable/unique/reversible/
game-scoped/persistent/atomic/explicit). JSON-file backed behind an
ExternalIdentityStore trait (SQLite can drop in later); parking_lot-guarded,
atomic temp+rename persist, rejects a torn reverse-duplicate on open.
Core never learns FIFA integers; only the host/adapter that owns a game
boundary uses this. 6 tests, 4/4 mutations killed (same-id-for-two-items,
lost-on-restart, broken-reverse, dropped-game-scope). Phase commit 1/5.
openfut-utas-host: the first live UTAS host. Serves GET /ut/game/<title>/club
from OpenFUT Core via the FIFA17 adapter and reverse-proxies every other UTAS
route verbatim to the Python oracle. Plaintext HTTP/1.1 keep-alive (no TLS);
route classification before execution; a Core error on /club degrades to an
empty page and never falls back to Python. CoreAccess is a host-owned boundary
(the adapter stays transport-agnostic).
openfut-adapter-fifa17::fut: owned_query (wire parse + FIFA id->name mapping,
unknown id = hard error), entities (id<->name from committed tables), and
club_response (FIFA _item shaping; drops items lacking a real FIFA asset id,
never fabricates one).
openfut-core submodule advanced to the reconciled trunk (6acae54 = 8c8a4116
multi-game + eab522a replace_squad/SquadRules + the /club semantic query).
11 host tests + adapter fut tests; 10/10 host mutations killed. rare=SP UNKNOWN.
Retail rendering of Core inventory still blocked on the Core-card->asset-id
identity decision (next phase).
Queued cleanup, run only AFTER the roster gate closed in both directions,
so the live A/B changed exactly one thing.
The two `drain_body` implementations were character-for-character
identical, so the extraction is a move. What it guards is not cosmetic:
answering while the client is still sending leaves unread data in the
receive queue and Linux turns the close into an RST rather than a FIN --
invisible in any comparison of the response, and worth two live gate
attempts to find. Behaviour that must be identical across hosts gets one
implementation, the same reasoning that produced openfut-tls.
SCOPE IS DELIBERATELY NARROW. Only the byte-identical part moved. The two
head-reading loops are NOT identical and stay where they are:
redirector roster
head cap 65536 16384
read chunk 4096 1024
on error abort proceed if any bytes arrived
Those differences are probably accidental, but each host is gate-proven
with the values it has. Unifying them would be a behaviour change wearing
a refactor's clothes -- exactly the mistake this project has already paid
for. They converge later as their own change with their own gate, or not
at all.
Purity shown, not asserted: every existing test in both hosts still
passes (426 workspace tests), and 7/7 mutations are killed, including
three in the SHARED crate that must break both hosts at once and one per
host that skips the drain call.
Three test cases neither host had now exist, because the extracted code
finally had somewhere to be tested directly: a malformed Content-Length,
an unterminated head, and a lookalike header. That last one matters --
`X-Original-Content-Length: 99` would drain 99 bytes that were never sent
if the match were `contains` rather than `starts_with`, and a mutation
confirms the test catches it.
Also fixes a race this run exposed in openfut-tls's own tests: keypair()
returned early if the certificate file existed, but wrote the certificate
BEFORE the key, so a parallel test could observe a cert whose key had not
landed. It failed one run and passed the next -- the kind of flake that
gets rerun instead of fixed. Now generated once per process via OnceLock,
key written first, and the suite was repeated five times to confirm.
Nothing deployed and nothing restarted: the running redirector and roster
are still the gate-proven binaries.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Second consumer of openfut-tls, and the reason it was extracted first.
This host contains no roster content and no cipher choice: the adapter
owns the 67 bytes and the observed TLS profile, openfut-tls owns the
acceptor, and this crate owns accept/read/drain/write/close.
Lifecycle was MEASURED, not inherited. The obvious mistake here would
have been copying the redirector's 300ms dwell because the other host has
one. A probe against the oracle says otherwise:
dwell after responding 0 ms (redirector: 300 ms)
request body drained POST answered only once it arrives
close clean FIN, never RST
keep-alive none one request per connection
The probe ran against a REPLICA of roster_server.py loaded from its own
source, not against :8081 -- http.server.HTTPServer is single-threaded
and FIFA was mid-session, so holding a connection open to measure the
close would have stalled the game's poll and could have surfaced as the
squad-update error. The replica was then confirmed byte-identical to the
live oracle under masking, the 1-byte delta being the container's Python
version in the Server header.
Differential against the live oracle, every field identical, with the
Server header compared UNMASKED:
GET 230B HEAD 163B POST 163B
drained=True reset=False answered_before_body=False
keepalive: second request accepted by the socket, never answered
Testing follows the redirector's hard-won rule: where a property is
visible both to the client and inside the host, it is asserted inside the
host via ConnOutcome. A client-side check cannot tell "drained" from "not
drained" -- it reads the buffered response either way -- and that exact
mistake let a mutation survive once already.
9 parity tests, 6 unit tests, 5/5 mutations killed, including "answer
before draining", "hold the connection open like the redirector" and
"inherit the redirector's 300ms default".
drain_body is duplicated from the redirector deliberately. Unifying it
means editing the redirector, and the roster A/B must change exactly one
thing. Extraction is scheduled for after the roster gate closes.
Not deployed and not switched: Python still serves :8081.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The redirector was the only host that spoke TLS, so its TLS lived inside
it. The roster host needs the same listener, and that made the choice
explicit: share this code or copy it.
Copying it is what already went wrong. On 2026-08-11 the Rust redirector
served one certificate while the container served another. ProtoSSL
caches the server certificate per backend, so the redirector -- the first
TLS connection of a session -- decided what the client expected, and
every later service failed its handshake. Silently: Python's socketserver
swallows ssl.SSLError as OSError. Three gates went to it. One place to
configure TLS is the structural fix, so it exists before the second host
does rather than after.
Split along the line the architecture already draws:
openfut-tls how to build an acceptor. Game-independent.
Knows nothing about which suites any client
offers.
adapter-fifa17::tls what FIFA 17 was OBSERVED to offer: the six
enabled suites, the two refused, the TLS 1.2
window, the EA SNI. Plain strings, so the
adapter keeps its lean dependencies -- reading
a card table should not build OpenSSL.
redirector-host joins the two. Chooses no cipher of its own.
Behaviour is unchanged, and shown to be:
* tests/fifa17_tls_profile.rs carries over every case from the deleted
module -- FIFA's eight suites negotiate AES256-GCM-SHA384, each enabled
suite works alone, RC4-only is refused, ECDHE-only is refused. Deleting
a module must not quietly delete its evidence.
* one test pins the composed values literally against the host as it was
when gates 1-14 passed. A "pure refactor" that cannot fail is not a
claim, it is an assumption.
* the rebuilt binary self-tests to the same TLSv1.2 / AES256-GCM-SHA384
the retail client negotiated at 17:09 today.
Two improvements fall out of having one place to look:
* the startup banner now prints cert_sha256. The mismatch above raised no
error at startup and broke the client much later with nothing logged;
it is now the first line of the log.
* tls_min/tls_max print as TLSv1.2 rather than SslVersion(771). This line
is gate evidence and gets read by people.
Nothing deployed and nothing restarted: FIFA is mid-session on the
running redirector, which is untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TLS DEPENDENCY, as directed: the openssl crate directly with the `vendored`
feature. NOT native-tls. native-tls abstracts over whatever the platform
provides; here the requirement is the opposite -- precise, evidenced behaviour
for one legacy client -- which needs explicit control of the cipher list,
protocol floor/ceiling and security level. Vendored so a distro libssl update
cannot silently change whether FIFA 17 can connect.
Scoped to this crate alone. Neither OpenFUT Core nor the generic protocol
crates gain an OpenSSL dependency.
CIPHERS driven by the captured retail ClientHello, not by generic legacy
assumptions. The six RSA+AES suites it offers are enabled; RC4 and MD5 are
deliberately NOT, even though the client offers them -- it already negotiates
AES256-GCM-SHA384, so resurrecting RC4 for completeness would weaken the
service for nothing. TLS 1.2 floor and ceiling, matching the observed client;
the floor is not dropped to 1.0 pre-emptively because "the oracle permits it"
is not "the client requires it".
SECURITY LEVEL IS NOT LOWERED. Tried the default policy first, as directed,
and OpenSSL 3.6.3 accepts static-RSA/AES without weakening. No SECLEVEL change
was needed and none is applied; it remains overridable per-listener with
evidence.
CERTIFICATE: the proven Python redirector's material is reused, so the TLS
implementation stays the only variable in an A/B. Verified RSA-2048, CN
winter15.gosredirector.ea.com, cert/key modulus match; the key stays
gitignored.
SHARED CONFIG. New openfut-host-config is now the only crate that reads the
environment, and both hosts resolve endpoints through it. Two hosts each
parsing OPENFUT_ADVERTISE would be exactly the "separate helpers constructing
endpoints from different sources of truth" the address audit forbids.
VERIFICATION BY REAL HANDSHAKE, not by enumeration. The crate exposes no
accessor for a context's configured suites at this version, which turned out
better: the host now rehearses the retail handshake at startup with a client
restricted to exactly FIFA's eight suites and REFUSES TO SERVE if it fails, so
a cipher/version misconfiguration surfaces at boot rather than as an
unexplained failure during a live gate.
Gates 1-5 pass: TLS config unit tests; a FIFA-suite-only client negotiates
TLSv1.2/AES256-GCM-SHA384; each enabled RSA+AES suite negotiable alone; an
RC4-only client is refused; an ECDHE-only client is refused (proving no modern
policy was silently inherited); a full HTTPS round-trip returns bytes
IDENTICAL to the Python oracle's recorded response.
Cargo.lock committed for reproducibility: openssl 0.10.81, openssl-sys 0.9.117,
openssl-src 300.6.1+3.6.3 (OpenSSL 3.6.3). Updating openssl-src is NOT a
routine bump -- it requires re-running the FIFA compatibility gates.
Gates 6-14 need the retail client and are next.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OpenFUT Agent