From a93a8bcdd7be9cf86e29778556d9fbbb9ae2c41e Mon Sep 17 00:00:00 2001 From: funman300 Date: Tue, 4 Aug 2026 10:31:37 -0700 Subject: [PATCH] fifa17-recon: decouple contract suite from the Python implementation test_fut_contract.py no longer imports ACCOUNT from fut_account. The expected persona comes from FUT_TEST_PERSONA_ID and the target from FUT_TEST_BASE, so the suite now imports nothing but stdlib and talks to a server at a URL. That is what lets these 380 checks certify ANY implementation of the reversed spec, a future Rust openfut-core included, without replaying the reverse engineering. The original reason for reading ACCOUNT still holds and is preserved in the comment: suite and server must not each hold a private copy of the constant, or the identity-consistency checks would only prove two copies matched. Also adds pileSizeClientData(0x227) behind FUT_PILESIZES (default off). A probe run with 16 uniquely-valued entries did NOT move the MY CLUB counter, so that member is eliminated as its source; the code is kept for the record and flagged off. Docs: OPENFUT_PROJECT_REPORT.md and OPENFUT_HANDOFF.md. The report now separates "built but untested" from "never requested by the client" -- the server log records User-Agent, and splitting real client traffic (ProtoHttp) from this project's own probes shows /season, /tournament, /champion, /match, /clubUser and /user/list are at ZERO client requests. /clubUser (0 client, 93 probe) and /user/list (0 client, 180 probe) are the starkest: work was done on both assuming the client wanted them. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01VUT92pz6RWKih9dSr8ZpxW --- OPENFUT_HANDOFF.md | 246 ++++++++++++++++++++ OPENFUT_PROJECT_REPORT.md | 297 ++++++++++++++++++++++++ fifa17-recon/tools/test_fut_contract.py | 26 ++- fifa17-recon/tools/utas_server.py | 43 ++++ 4 files changed, 604 insertions(+), 8 deletions(-) create mode 100644 OPENFUT_HANDOFF.md create mode 100644 OPENFUT_PROJECT_REPORT.md diff --git a/OPENFUT_HANDOFF.md b/OPENFUT_HANDOFF.md new file mode 100644 index 0000000..186465b --- /dev/null +++ b/OPENFUT_HANDOFF.md @@ -0,0 +1,246 @@ +# OpenFUT — project handoff + +Self-contained briefing for an assistant with **no access to this repo or machine**. +Everything needed to understand the project and reason about its open problems is here. + +--- + +## 1. What this is + +**OpenFUT** is a clean-room, fully offline re-implementation of the server backend for +**FIFA 17 Ultimate Team (FUT)**. EA's servers for FIFA 17 are long dead. The goal is to +make the retail game's FUT mode fully playable again — open packs, build squads, use the +transfer market, play matches and earn rewards — by emulating every server the client +talks to, on localhost. + +Nothing is decompiled *into* the project. The game's binaries are read to learn the +**wire format** (which JSON keys, of which types, each response must contain), and the +servers are written from scratch in Python against that spec. + +The client is unmodified retail FIFA 17 running under Wine/Proton on Linux. + +--- + +## 2. Architecture — four independent servers + +FIFA 17 does not talk to one backend. It talks to four, on different protocols, and all +four must be satisfied in sequence before FUT loads. + +``` +FIFA 17 (Wine/Proton) + │ + ├─ LSX / Origin :4216 XML over TCP. Local Origin client emulation. + │ Login, entitlements, persona. + ├─ Blaze :42127 redirector (TLS) → :42130 game server, :42131 nucleus + │ EA's binary "Fire2/TDF" RPC protocol. Session, auth, + │ and — critically — the CLIENT-CONFIG STORE. + ├─ UTAS / RS4 :8099 The FUT REST API. JSON over HTTP. ~45 endpoints. + │ Club, squads, packs, market, matches. The bulk of it. + └─ POW / EASFC :8094 (+ :8080 content) A third HTTP API, discovered late. + Online status bar, level, EASFC credits, catalogue. +``` + +Plus two helpers: a **roster** server (:8081) serving a roster-update XML the FUT +loading screen blocks on, and **autopatch**, which patches the running process's +ProtoSSL certificate verification so the client accepts our self-signed TLS. + +### How the client is redirected + +Three mechanisms, in decreasing order of preference: + +1. **Blaze client-config keys** — the cleanest. Blaze serves a key/value config store + and the client reads its own service URLs from it. `FUT_RS4_APIURL_` and + `FUT_RS4_URL_` point the FUT API at `127.0.0.1:8099`; `FIFA_POW_URL` points + the EASFC layer at `127.0.0.1:8094`. **No root, no DNS games.** +2. **`/etc/hosts`** — for hosts baked into the binary (`easw.easports.com`, + `gosredirector.ea.com`). +3. **iptables DNAT** — for a hardcoded IP (`159.153.51.20` → the Blaze redirector). + +--- + +## 3. The wire format, and why it is unforgiving + +FUT responses are JSON, but the client does **not** use a general JSON object model. Each +response class has a hand-written SAX-style deserializer that walks tokens and dispatches +on a **hashed key id** (an "atom"). This has three consequences that dominate the project: + +**Atoms.** Every JSON key name maps to a 16-bit atom id via FNV-1a. There is a recovered +table of ~900 (id → name). A response is really "which atoms does deserializer X read, +and of what type". + +**Type fidelity is fatal.** Feeding a scalar where the parser expects an object or array +does not error — it **desyncs the token reader and the game hard-freezes** in a busy loop +at `0x1801c7f1a`. This is the single most common way to break the game, and it has bitten +this project repeatedly. Arrays must be arrays; nested objects must be objects. + +**Unknown keys are usually skipped safely** — most deserializers route an unrecognised +atom to a value-skip handler (`FUN_180135ff0`), so extra fields are inert. **But not +always**: at least one deserializer (`FutMoveCard`, `0x180128600`) has *no* skip handler +at all, so any unexpected key leaves its value unconsumed and desyncs the reader. + +### Working method + +For any endpoint: find the response class's deserializer, extract the atom ids it +compares against, map them to names, note the getter used for each (int / string / bool / +nested), and build the minimal body. Omit nested members unless their shape is known — +omission is skip-safe, a wrong shape freezes the game. + +--- + +## 4. What works today (live-verified) + +| Area | Status | +|---|---| +| Boot to the FUT hub | ✅ | +| Club identity, coins, W/D/L record | ✅ | +| Active squad — renders 11 real players, chemistry links | ✅ | +| Squad building — `PUT /squad/` fires, persists across relaunch | ✅ | +| Squad roster ("MY SQUADS") | ✅ | +| Transfer market — browse, bid, buy-now, sell, watchlist | ✅ | +| Packs — buy, cards land in club | ✅ (via a workaround, see §5) | +| Quick sell — destroys cards, credits coins | ✅ | +| Match loop — create/ready/play/destroy + coin rewards | ✅ implemented, **never played in-game** | +| Online/EASFC status bar (no "servers unreachable") | ✅ when enabled | +| Seasons / tournaments / leaderboards / champions | ⚠️ routed with reversed schemas, never live-tested | + +Current save state: 99 club items, 8,400 coins, 0-0-0 record. + +**Design convention:** every risky change ships behind an environment flag, default set +to whatever is live-proven (`FUT_MASSINFO`, `FUT_USERINFO`, `FUT_MODES`, +`FUT_PACK_AUTOCLUB`, `FUT_POW`, `FUT_STORE_GROUPS`, …). This exists because two working +screens were broken by shipping "corrections" on by default. + +--- + +## 5. Open problems + +### 5a. "Send to Club" kills the FUT session — UNSOLVED + +Opening a pack shows the cards correctly. Choosing **Send to Club** results in: + +> *"We are sorry but there has been an error connecting to FIFA 17 Ultimate Team. You +> will be returned to the FIFA 17 Main Menu."* + +The client then POSTs `ut/delete/auth` (logout) about a second later. The cards **do** +move correctly server-side every time — only the acknowledgement is rejected. + +**Eliminated by live test:** + +| Hypothesis | Result | +|---|---| +| Missing `chemistry` field | Failed without it too | +| Extra keys + no skip handler in the deserializer | Real finding; sending *only* the parsed key still failed | +| The response body at all | **A bare `{}` also fails** | +| A network failure ("error connecting") | Zero non-loopback connections during a failure — that string is FIFA's *generic* session-error text | +| Missing per-call service URLs | 146 were genuinely missing and are now served; unchanged | +| The POW layer saturating HTTP | Same failure with POW fully disabled | +| The reveal screen's exit path | **"Quick Sell All" from the same screen works** | +| The club not being loaded | Loading MY CLUB first, then moving, still failed | + +**The key asymmetry:** quick sell (`POST ut/delete//item`) and move +(`PUT ut//item`) both receive an identical bare `{}` from the same server, with the +same headers and status. Quick sell succeeds; move kills the session. Whatever decides +this is **client-side state, not the wire**. + +**Workaround in place:** pack contents are deposited straight into the club at open time +and the pending pile is kept empty, so the client is never offered a move. Packs are +fully usable; the cost is that the reveal screen shows nothing to assign. + +### 5b. "MY CLUB" counter always reads 0 — UNSOLVED + +The hub tab bar shows `MY CLUB 0` despite the club holding 99 items. + +**Eliminated by live test:** +- **Not the item list** — the user opened MY CLUB, the client fetched the club and + **displayed all 99 players correctly**, and the counter still read 0. +- **Not `pileSizeClientData`** — the massinfo member that carries pile sizes as + `{"entries":[{"key":int,"value":int}]}`. A probe sent 16 entries with uniquely + identifiable values; the counter stayed 0. +- **Not lazy loading** — the club endpoint was fetched 6 times that session. + +**Unexplored contrast:** the `ACTIVE SQUAD` tab in the same bar correctly shows `11/23`. +So some counters work. Whatever differs between that one and the club one is likely the +answer. + +### 5c. Store tiles render "unknown" — DIAGNOSED, NOT FIXED + +Pack tiles show `unknown` with zero item counts. The `"unknown"` string is an +unconditional **default** in a string constructor — the field simply never gets written. +The store renders *display groups*, and `displayGroup` is parsed **recursively by the same +element parser**. Sending it populated **froze the store** (the type-desync busy loop), so +it is behind a flag, default off. Doing it properly needs the group's own field set worked +out rather than a self-referential copy of the pack. + +--- + +## 6. Notable reverse-engineering findings + +- **Class → deserializer resolution.** A response class's name literal is preceded by a + **4-byte header**, and the constructing factory's `lea` points at *the header*, not the + text. Lookups must use `name_address - 4`. Six attempts failed on this off-by-four; + four of them returned zero results and nearly got recorded as "this class has no + deserializer". +- **The request-template table is a floor, not a ceiling.** Several real endpoints are + built by the caller appending a suffix and therefore never appear in the binary's URL + table: `squad/list`, `user/club`, `club/stats/*`, `clientdata/`. Only live traffic + reveals them. This has caught the project three separate times. +- **The documentation lies.** The project's own `ENDPOINT_MAP.md` (~1,360 lines, ~100 + reversed structs) has been wrong repeatedly: it claimed `FutMoveCard` parses + `chemistry` (it does not); it called seven store pack fields "skipped no-ops" (all are + parsed); it gave price-object keys as `amount`/`currency` (the parsers read + `externalPriceId`). **Verify against the decompiler before relying on any row.** +- **The online layer was hiding in an unpacked DLL.** "EA FC servers are unreachable" + comes from a third HTTP API implemented in a *loose, unpacked, string-rich* library — + not the packed executable, and not any layer previously emulated. It is redirectable + purely through the Blaze config store. +- **The main executable is Denuvo-packed.** Its code exists only in a live process. Live + memory is readable via `/proc/PID/mem` (the PE is mapped flat), which is how a crash + site was disassembled. Any logic living there cannot be reversed statically. + +--- + +## 7. Tooling built + +- A **PyGhidra harness** with helpers for decompiling, xrefs, vtables, byte scanning and + class→deserializer resolution. (Ghidra's Java/OSGi scripting is broken on this machine; + PyGhidra bypasses it entirely.) +- A **minidump reader** — exception record, fault-time registers, module map, and a stack + walk that recovers a usable backtrace. +- A **live code grabber** that reads and disassembles unpacked code out of a running + process. +- A **read-only live probe** pattern for polling client model state while playing. +- **Two test suites**: 380 live contract checks (freeze-safety: asserts every response + field's type against the reversed schema) and 51 pure unit checks for match rewards. +- A **traffic-replay audit** that diffs current responses against a known-good session — + this is what proves a change did not alter what the client sees. + +--- + +## 8. Documentation in-repo + +| File | Contents | +|---|---| +| `ENDPOINT_MAP.md` | ~100 reversed response structs, atoms, types, freeze risks | +| `FUT_RESPONSE_REBUILD_PLAN.md` | Squad family, massinfo, the service-layer call graph | +| `REBUILD_RESEARCH.md` | Complete API surface, gap analysis, POW layer, and every eliminated hypothesis with its evidence | +| `CARD_SYSTEM.md` | How card identity resolves locally from the game's own database | +| `REPACK_INTEL.md` | Origin/LSX emulation notes | + +--- + +## 9. Where help would be most valuable + +1. **The move-vs-quick-sell asymmetry (§5a).** Two sibling endpoints, identical responses, + one works. What client-side precondition could a "move item between piles" operation + have that a "discard item" operation does not — a loaded destination collection, a + known pile capacity, a valid target index? +2. **The MY CLUB counter (§5b).** Given the client demonstrably *has* the items and + *renders* them, what else could a tab counter read from? Note that a sibling counter in + the same bar works correctly. +3. **Whether either is fixable server-side at all**, or whether the honest answer is that + the deciding logic lives in the packed executable and only live instrumentation can + settle it. + +Useful framing: this project's failures have almost always come from proposing a fix +before testing the assumption under it. Hypotheses that come with a cheap way to +disconfirm them are worth far more than plausible ones. diff --git a/OPENFUT_PROJECT_REPORT.md b/OPENFUT_PROJECT_REPORT.md new file mode 100644 index 0000000..ff83df4 --- /dev/null +++ b/OPENFUT_PROJECT_REPORT.md @@ -0,0 +1,297 @@ +# OpenFUT — Project Report + +**Goal:** make FIFA 17 Ultimate Team fully playable offline, forever, by re-implementing +every server the game talks to. + +**Status:** FUT boots, loads, and is playable. Packs, squads, the transfer market, coins +and progression all work. Two cosmetic/flow problems remain open. + +**Timeline:** 2026-06-25 → 2026-08-04 · 44 commits · ~12,900 lines of Python across 39 +tools · ~3,600 lines of reverse-engineering documentation. + +--- + +## 1. What the project is + +EA shut down FIFA 17's servers years ago, which kills Ultimate Team — the mode is entirely +server-driven. Your club, squads, packs, market and progression all live server-side, so +without a backend the mode is dead even though the game still installs and runs. + +OpenFUT replaces that backend with local servers. The game is **unmodified retail +FIFA 17** running under Wine/Proton on Linux; nothing is patched into the game except a +single runtime tweak so it accepts our TLS certificate. + +This is **clean-room work**. No EA code is copied or redistributed. The game's own +binaries are read to learn the *wire format* — which JSON keys, of which types, each +response must carry — and the servers are written from scratch against that specification. + +--- + +## 2. Project history + +The project changed target twice before finding its footing. That arc matters, because +each pivot was driven by hitting a hard wall. + +**Phase 1 — FIFA 23 (June 2026).** Began as an offline FUT backend for FIFA 23: a Rust +core (`openfut-core`, Axum + SQLite), a protocol bridge (`openfut-bridge`), and a GUI +launcher. All three built and passed tests. The architecture was sound but the client +never got far enough to exercise it. + +**Phase 2 — the FIFA 23 wall.** FIFA 23 refused to go online at all. Extensive reverse +engineering of the connection state machine, live-memory probing, and forcing the +"go online" gate directly all failed — the client's internal coherence checks were the +wall, not any single flag. Documented as a dead end rather than fought. + +**Phase 3 — the FIFA 17 pivot (late July).** FIFA 17 turned out to be a far better target: +its network library is **unprotected and fully symboled**, exposing 979 RPC names. The +insight was to crack FIFA 17 first and port the understanding back. + +That worked, quickly: +- **TLS pinning defeated** — the client's certificate verification is patched at runtime + in memory, so a self-signed cert is accepted. +- **Origin/LSX emulation** — the local Origin client protocol was reverse engineered, + including the repack's own crypto layer, beating "log in to Origin" and + "title version outdated". +- **Blaze cracked end to end** — EA's binary RPC protocol: redirector, second-hop + handshake, and the encoded pre-auth exchange. This was the big one. +- **Full FUT API mapped** — ~100 response structures reverse engineered to field level. + +**Phase 4 — building the FUT backend (August).** With the protocol understood, the work +became making FUT actually *play*: card rendering, packs, the store, the transfer market, +squads, and match rewards. This is where the project stands. + +--- + +## 3. Architecture + +FIFA 17 does not talk to one backend. It talks to **four**, on different protocols, and +all four must be satisfied in sequence before FUT loads. + +``` +FIFA 17 (Wine/Proton) + │ + ├─ LSX / Origin :4216 XML/TCP — local Origin client emulation + │ login, entitlements, persona + ├─ Blaze :42127 redirector (TLS) → :42130 game, :42131 nucleus + │ EA's binary Fire2/TDF RPC + │ session, auth, and the CLIENT-CONFIG STORE + ├─ UTAS / RS4 :8099 the FUT REST API — JSON/HTTP, ~45 endpoints + │ club, squads, packs, market, matches + └─ POW / EASFC :8094 a third HTTP API (+ :8080 content) + online status, level, credits, catalogue +``` + +Plus **roster** (:8081), serving an XML file the FUT loading screen blocks on, and +**autopatch**, which patches certificate verification in the running process. + +### Redirection, in order of preference + +1. **Blaze client-config keys** — the client reads its own service URLs from a key/value + store that Blaze serves. Pointing FUT and EASFC at localhost needs **no root and no + DNS manipulation**. This is the clean mechanism and most redirection uses it. +2. **`/etc/hosts`** — for hostnames baked into the binary. +3. **iptables DNAT** — for one hardcoded IP address. + +--- + +## 4. The wire format — and why it is unforgiving + +FUT responses are JSON, but the client does not use a general JSON object model. Each +response class has a hand-written SAX-style deserializer that walks tokens and dispatches +on a **hashed key id** ("atom"). Three consequences dominate the project: + +**Atoms.** Every JSON key maps to a 16-bit id via FNV-1a. A recovered table of ~900 +id→name pairs is the Rosetta stone. A response spec is really "which atoms does this +deserializer read, of what type". + +**Type fidelity is fatal.** A scalar where an object or array is expected does not error — +it **desyncs the reader and hard-freezes the game** in a busy loop. This is the primary +failure mode of the entire project. + +**Unknown keys are usually skipped — but not always.** Most deserializers route +unrecognised atoms to a skip handler, making extra fields inert. At least one does not, +so any unexpected key desyncs it. + +**Working method:** locate the deserializer, extract its atom set and per-atom getter +types, build the minimal body, and omit nested members whose shape isn't known — omission +is safe, a wrong shape freezes the game. + +--- + +## 5. What works + +| Capability | State | +|---|---| +| Boot: Origin → Blaze → FUT hub | ✅ | +| Club identity, coins, W/D/L record | ✅ | +| Active squad — 11 real players, ratings, chemistry | ✅ | +| Squad building — saves and survives relaunch | ✅ | +| Squad roster ("MY SQUADS") | ✅ | +| Transfer market — browse, bid, buy-now, list, watchlist | ✅ | +| Store — buy packs | ✅ | +| Packs — cards land in the club | ✅ via workaround (§6a) | +| Quick sell — credits coins | ✅ | +| Match loop — create/ready/play/destroy + rewards | ⚠️ built, **never requested by the client** (see below) | +| Online/EASFC status bar | ✅ when enabled | +| Seasons, tournaments, leaderboards, champions | ⚠️ routed, **never requested by the client** (see below) | +| Card identity (names, faces, ratings) | ✅ resolves from the game's own local database | + +### "Untested" is two different things, and the difference matters + +The server log records the User-Agent of every request. The real client identifies as +`ProtoHttp`; this project's own curl and Python probes do not. Separating them shows that +several endpoints previously filed as "built but untested" have in fact **never been +requested by the game at all**, and everything recorded against them was self-inflicted +traffic: + +| endpoint | client requests | project probes | +|---|---|---| +| `/leaderboards/options` | 5 | 1 | +| `/clientdata/userHubData` | 13 | 2 | +| `/user/accountinfo` | 23 | 49 | +| `/season`, `/season/user` | **0** | 2 | +| `/tournament`, `/tournament/user` | **0** | 3 | +| `/leaderboards` (bare) | **0** | 2 | +| `/champion` | **0** | 2 | +| `/match` | **0** | 2 | +| `/clubUser` | **0** | 93 | +| `/user/list` | **0** | 180 | +| `/sbs` (SBC), `/draft/mode` | **0** | 0 | + +`/clubUser` and `/user/list` are the starkest: 273 requests between them, none from the +game. Work was done on both on the assumption the client wanted them. + +A zero in the client column does **not** mean the client never wants that endpoint. In +most cases it means **nobody has navigated to that part of the game yet**. It does mean no +claim about those endpoints has been tested against the client, and any analysis that does +not apply this filter is misleading by default. + +**Requirement:** every capture and analysis tool in this project should apply the +User-Agent split by default rather than as an afterthought. + +**Design convention.** Every risky change ships behind an environment flag whose default +is whatever is live-proven. This exists because shipping "corrections" on by default broke +two working screens — once freezing the store outright. + +--- + +## 6. Open problems + +### 6a. "Send to Club" ends the FUT session + +Opening a pack displays the cards correctly. Choosing **Send to Club** produces *"there +has been an error connecting to FIFA 17 Ultimate Team"* and a logout. The cards **do** +move correctly server-side; only the acknowledgement is rejected. + +Eliminated by live test: the response body (a bare `{}` fails identically), a missing +field, unknown-key desync, a network failure (zero external connections during a +failure — that error string is FIFA's *generic* session error), missing service URLs, +the online layer, the reveal screen's exit path, and the club not being loaded. + +**The asymmetry:** quick sell and move receive an *identical* bare `{}` from the same +server. Quick sell succeeds. Move ends the session. The deciding factor is client-side +state, not the wire. + +**Workaround:** pack contents are deposited straight into the club at open time, so the +client is never offered a move. Packs are fully usable; the reveal screen just doesn't +offer assignment. + +### 6b. "MY CLUB" counter reads 0 + +The hub shows `MY CLUB 0` despite 99 items. Not the item list (the client *displays* all +99), not the pile-size data (a 16-entry probe changed nothing), not lazy loading (fetched +6 times). Unexplored: the neighbouring `ACTIVE SQUAD` counter works correctly — the +difference between them is likely the answer. + +### 6c. Store tiles read "unknown" + +`"unknown"` is an unconditional default in a string constructor — the field is never +written. The store renders *display groups*, and that member is parsed recursively by the +same parser; sending it populated **froze the store**, so it is flagged off pending a +correct group schema. + +### 6d. Large parts of the game have never been opened + +Distinct from 6a to 6c, which are things that misbehave. Per the User-Agent table in +section 5, entire modes have never issued a single client request: Seasons, Tournaments, +FUT Champions, Draft, SBC, and the match loop itself. Their endpoints are routed and their +schemas are reversed, but no claim about any of them has been tested against the game. + +This is not a bug list. It is unmeasured surface, and it is the cheapest information +available to the project because most of it costs nothing but navigating menus. It is +recorded here because "routed from a reversed schema" reads like a stronger claim than it +is, and section 7 warns that this project's notes have described things differently from +what is true. + +*A multi-agent investigation into 6a and 6b is currently running.* + +--- + +## 7. Notable findings + +- **Class → deserializer resolution.** A response class's name literal is preceded by a + 4-byte header and the factory points at *the header*. Six attempts failed on that + off-by-four; four returned nothing and were nearly recorded as "no deserializer exists". +- **The URL table is a floor, not a ceiling.** Several real endpoints are built by + appending a suffix at the call site and never appear in the binary's template table. + Only live traffic reveals them — this caught the project three separate times. +- **The project's own documentation has been wrong repeatedly** — fields described as + inert turned out to be parsed, and documented key names didn't match the parsers. + Verify against the decompiler, not the notes. +- **The online layer was hiding in plain sight.** "EA FC servers unreachable" comes from a + third HTTP API in a *loose, unpacked, string-rich* library — not the protected + executable, and not any previously emulated layer. Redirectable purely by config. +- **The main executable is Denuvo-packed**, so its code exists only in a live process. + Live memory is readable, which is how a crash site was disassembled — but logic living + there cannot be reverse engineered statically. + +--- + +## 8. Tooling and quality + +- **PyGhidra harness** with decompile / xref / vtable / byte-scan / class-resolution + helpers (Ghidra's own Java scripting is broken on this machine). +- **Minidump reader** — exception record, fault-time registers, module map, stack walk. +- **Live code grabber** — reads and disassembles unpacked code from a running process. +- **Read-only live model probes** for watching client state while playing. +- **Test suites:** 380 live contract checks (type/freeze safety per reversed schema) plus + 51 pure unit checks. Both green. +- **Traffic-replay audit** — diffs current responses against a known-good session to prove + a change didn't alter what the client sees. +- **~3,600 lines of RE documentation** across six files, including every eliminated + hypothesis with its supporting evidence. + +--- + +## 9. Roadmap + +**Immediate (free, no code):** play a match — the reward loop is built and unit-tested but +has never run in-game. Enable the game-mode endpoints and see whether four more modes +light up. + +**Near term:** close the two open problems, most likely via live instrumentation rather +than more static analysis. Implement SBC and Draft (schemas already recovered). Fix the +store display groups properly. + +**Longer term:** the stated destination is porting this into the Rust `openfut-core` +behind a FIFA-17 bridge. Everything currently lives in Python prototypes; the +documentation is now good enough to write the port against. + +--- + +## 10. Honest assessment + +**What went well.** The FIFA 17 pivot was the decisive call — recognising that an +unprotected binary was worth more than persisting against a hardened one. Blaze, Origin +and the FUT API were all cracked end to end. The freeze-safety test suite has repeatedly +caught regressions before they reached the game. + +**What went badly.** Progress has been slowest where fixes were proposed before the +assumption under them was tested. Both open problems absorbed many attempts built on +plausible but unverified theories; several were disproved in a single measurement that +could have been taken first. Two working screens were broken by shipping unverified +"corrections" on by default — which is precisely why the flag convention exists now. + +**The most reliable technique** has been comparing a working case against a failing one: +diffing live traffic against a known-good session, and contrasting a succeeding endpoint +with its failing sibling. That has produced more answers than any amount of decompilation. diff --git a/fifa17-recon/tools/test_fut_contract.py b/fifa17-recon/tools/test_fut_contract.py index e4e3bcf..0c856f3 100644 --- a/fifa17-recon/tools/test_fut_contract.py +++ b/fifa17-recon/tools/test_fut_contract.py @@ -20,16 +20,26 @@ Exit 0 = all pass. No pytest dependency (stdlib only). """ import json, os, sys, urllib.error, urllib.request -sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) -from fut_account import ACCOUNT # the identity every layer must agree on - -BASE = "http://127.0.0.1:8099" +BASE = os.environ.get("FUT_TEST_BASE", "http://127.0.0.1:8099") G = "/ut/game/fifa17" V2 = "/ut/v2/game/fifa17" -# No PERSONA_ID literal here any more. This suite and the server MUST read the -# same source or the "identity is consistent" checks below would only be proving -# that two copies of a constant were copied correctly. -PERSONA_ID = ACCOUNT.persona_id + +# IMPLEMENTATION-INDEPENDENT BY CONSTRUCTION. +# This suite talks to a server at a URL and imports NOTHING from the server's own +# code. That is what lets it verify ANY implementation of the reversed spec -- a +# future Rust openfut-core included -- without replaying the reverse engineering. +# +# It used to do `from fut_account import ACCOUNT` for the persona, which was a +# Python import against the Python implementation and quietly made the suite +# unable to certify a non-Python server. The expected persona now comes from the +# environment, defaulting to the value every layer has agreed on all along. +# +# The original reason for reading ACCOUNT still stands and is preserved: the suite +# and the server must not each hold their own copy of the constant, or the +# "identity is consistent" checks would only prove that two copies were copied +# correctly. Point FUT_TEST_PERSONA_ID at whatever the server under test is +# configured with; the default matches the shipped default. +PERSONA_ID = int(os.environ.get("FUT_TEST_PERSONA_ID", "33068179")) _fail = [] _pass = 0 diff --git a/fifa17-recon/tools/utas_server.py b/fifa17-recon/tools/utas_server.py index 0f84473..6726743 100755 --- a/fifa17-recon/tools/utas_server.py +++ b/fifa17-recon/tools/utas_server.py @@ -499,6 +499,45 @@ SETTINGS = {"configs": []} _MI = os.environ.get("FUT_MASSINFO", "full") +# ---- pileSizeClientData: the MY CLUB counter -------------------------------- +# LIVE EVIDENCE (2026-08-04): the user opened MY CLUB, the client fetched GET /club +# and DISPLAYED all 99 players -- and the MY CLUB counter still read 0. So that +# counter is NOT derived from the item list; it is a PILE SIZE, delivered +# separately. massinfo's pileSizeClientData(0x227) is that member and we have never +# sent it. Parser 0x18013adb0: {"entries":[{"key":,"value":}]} -- key and +# value BOTH read with the int getter 0x1801c79d0, and the parser IS skip-safe. +# +# The pile-id enum is not recoverable from the strings (the "club"/"tradepile" +# literals are just atom names in the alphabetical key table). So rather than guess: +# +# FUT_PILESIZES=probe -> emit one entry per candidate key 0..15 with a UNIQUE +# recognisable value (100+key). Whatever number MY CLUB then displays names the +# club pile's key: 103 means key 3. One launch identifies the enum. +# FUT_PILESIZES=1 -> emit the REAL counts once PILE_KEY_CLUB below is known. +# +# Default OFF: this adds a member to boot-critical massinfo. It is a documented +# member of that parser and carries only ints, so the risk is low -- but "low" is +# what I said about displayGroup before it froze the store, so it ships behind a flag. +_PILESIZES = os.environ.get("FUT_PILESIZES", "") +PILE_KEY_CLUB = int(os.environ.get("FUT_PILE_KEY_CLUB", "-1")) # set once probed + + +def pile_size_body(): + """massinfo.pileSizeClientData -- see the note above.""" + if _PILESIZES == "probe": + return {"entries": [{"key": k, "value": 100 + k} for k in range(16)]} + counts = { + "club": len(STORE.items()), + "purchased": len(STORE.purchased()), + "tradepile": len(STORE.listings()), + } + if PILE_KEY_CLUB >= 0: + return {"entries": [{"key": PILE_KEY_CLUB, "value": counts["club"]}]} + # No verified key yet -> announce the club count on every candidate key. Crude, + # but every value is truthful, so no pile can be told a wrong number. + return {"entries": [{"key": k, "value": counts["club"]} for k in range(16)]} + + def massinfo(): if _MI == "empty": return {} # old known-hub-reaching body @@ -518,6 +557,10 @@ def massinfo(): # body and adding a member to it is the change class behind the last two # live regressions. Instant fallback: FUT_MASSINFO=squad. body["clubUser"] = club_user_body() + if _PILESIZES: + # pileSizeClientData(0x227) -> parser 0x18013adb0, int-only, skip-safe. + # This is what the MY CLUB counter reads (see pile_size_body above). + body["pileSizeClientData"] = pile_size_body() return body # ---- FUT item-definition serving (wf_e41070d8) -------------------------------