OpenFUT/OPENFUT_HANDOFF.md

# OpenFUT — project handoff

Self-contained briefing for an assistant with **no access to this repo or machine**.
Everything needed to understand the project and reason about its open problems is here.

---

## 1. What this is

**OpenFUT** is a clean-room, fully offline re-implementation of the server backend for
**FIFA 17 Ultimate Team (FUT)**. EA's servers for FIFA 17 are long dead. The goal is to
make the retail game's FUT mode fully playable again — open packs, build squads, use the
transfer market, play matches and earn rewards — by emulating every server the client
talks to, on localhost.

Nothing is decompiled *into* the project. The game's binaries are read to learn the
**wire format** (which JSON keys, of which types, each response must contain), and the
servers are written from scratch in Python against that spec.

The client is unmodified retail FIFA 17 running under Wine/Proton on Linux.

---

## 2. Architecture — four independent servers

FIFA 17 does not talk to one backend. It talks to four, on different protocols, and all
four must be satisfied in sequence before FUT loads.

```
FIFA 17 (Wine/Proton)
  │
  ├─ LSX / Origin      :4216    XML over TCP. Local Origin client emulation.
  │                             Login, entitlements, persona.
  ├─ Blaze              :42127 redirector (TLS) → :42130 game server, :42131 nucleus
  │                             EA's binary "Fire2/TDF" RPC protocol. Session, auth,
  │                             and — critically — the CLIENT-CONFIG STORE.
  ├─ UTAS / RS4         :8099   The FUT REST API. JSON over HTTP. ~45 endpoints.
  │                             Club, squads, packs, market, matches. The bulk of it.
  └─ POW / EASFC        :8094 (+ :8080 content)   A third HTTP API, discovered late.
                                Online status bar, level, EASFC credits, catalogue.
```

Plus two helpers: a **roster** server (:8081) serving a roster-update XML the FUT
loading screen blocks on, and **autopatch**, which patches the running process's
ProtoSSL certificate verification so the client accepts our self-signed TLS.

### How the client is redirected

Three mechanisms, in decreasing order of preference:

1. **Blaze client-config keys** — the cleanest. Blaze serves a key/value config store
   and the client reads its own service URLs from it. `FUT_RS4_APIURL_<MODULE>` and
   `FUT_RS4_URL_<CALL>` point the FUT API at `127.0.0.1:8099`; `FIFA_POW_URL` points
   the EASFC layer at `127.0.0.1:8094`. **No root, no DNS games.**
2. **`/etc/hosts`** — for hosts baked into the binary (`easw.easports.com`,
   `gosredirector.ea.com`).
3. **iptables DNAT** — for a hardcoded IP (`159.153.51.20` → the Blaze redirector).

---

## 3. The wire format, and why it is unforgiving

FUT responses are JSON, but the client does **not** use a general JSON object model. Each
response class has a hand-written SAX-style deserializer that walks tokens and dispatches
on a **hashed key id** (an "atom"). This has three consequences that dominate the project:

**Atoms.** Every JSON key name maps to a 16-bit atom id via FNV-1a. There is a recovered
table of ~900 (id → name). A response is really "which atoms does deserializer X read,
and of what type".

**Type fidelity is fatal.** Feeding a scalar where the parser expects an object or array
does not error — it **desyncs the token reader and the game hard-freezes** in a busy loop
at `0x1801c7f1a`. This is the single most common way to break the game, and it has bitten
this project repeatedly. Arrays must be arrays; nested objects must be objects.

**Unknown keys are usually skipped safely** — most deserializers route an unrecognised
atom to a value-skip handler (`FUN_180135ff0`), so extra fields are inert. This document
previously named `FutMoveCard` (`0x180128600`) as an exception with no skip handler at
all. **That was wrong**, and the retraction is in §5a: it has two skip-handler call sites
and parses seven atoms. No deserializer in this project is currently known to lack one.
Treat any future "this class has no skip handler" claim as unproven until the search is
shown to have covered the whole function.

### Working method

For any endpoint: find the response class's deserializer, extract the atom ids it
compares against, map them to names, note the getter used for each (int / string / bool /
nested), and build the minimal body. Omit nested members unless their shape is known —
omission is skip-safe, a wrong shape freezes the game.

---

## 4. What works today (live-verified)

| Area | Status |
|---|---|
| Boot to the FUT hub | ✅ |
| Club identity, coins, W/D/L record | ✅ |
| Active squad — renders 11 real players, chemistry links | ✅ |
| Squad building — `PUT /squad/<id>` fires, persists across relaunch | ✅ |
| Squad roster ("MY SQUADS") | ✅ |
| Transfer market — browse, bid, buy-now, sell, watchlist | ✅ |
| Packs — buy, reveal, Send to Club | ✅ (the workaround is retired, see §5a) |
| Quick sell — destroys cards, credits coins | ✅ |
| Match loop — create/ready/play/destroy + coin rewards | ✅ implemented, **never played in-game** |
| Online/EASFC status bar (no "servers unreachable") | ✅ when enabled |
| Seasons / tournaments / leaderboards / champions | ⚠️ routed with reversed schemas, never live-tested |

Current save state: 99 club items, 8,400 coins, 0-0-0 record.

**Design convention:** every risky change ships behind an environment flag, default set
to whatever is live-proven (`FUT_MASSINFO`, `FUT_USERINFO`, `FUT_MODES`,
`FUT_PACK_AUTOCLUB`, `FUT_POW`, `FUT_STORE_GROUPS`, …). This exists because two working
screens were broken by shipping "corrections" on by default.

---

## 5. Open problems

### 5a. "Send to Club" kills the FUT session — SOLVED 2026-08-04

Opening a pack shows the cards correctly; choosing **Send to Club** used to produce
*"there has been an error connecting to FIFA 17 Ultimate Team"* and a logout, seven
attempts running. The cards always moved server-side; only the acknowledgement was
rejected.

The cause was the response body. `PUT ut/%s/item` returns per-item **verdict** records,
not an acknowledgement, and the completion handler raises
`EVENT_CARDS_MOVE_CARD_FAILURE` when the record vector is empty or when `success != 1`.
Every body the project returned, `{}` included, reported the move as failed. The fix is
the real shape:

```json
{"itemData":[{"id":100000125,"pile":"club","success":true}, ...]}
```

Live: five cards to the club, session survived, cards persisted, no `ut/delete/auth`.
The autoclub workaround is retired.

Two corrections this closed, both worth carrying forward:

- The repo's claim that this deserializer had **no skip handler** and parsed only two
  keys was false. It came from searching a truncated decompile. It implied the body
  could not be at fault, which is what sent seven attempts after client-side state.
  Never conclude an absence from a truncated or unverified-length extraction.
- The quick-sell asymmetry was not evidence of client state. Quick sell's callbacks
  read only the transport status code and never touch the body.

### 5b. "MY CLUB" counter always reads 0 — UNSOLVED

The hub tab bar shows `MY CLUB  0` despite the club holding 99 items.

**Eliminated by live test:**
- **Not the item list** — the user opened MY CLUB, the client fetched the club and
  **displayed all 99 players correctly**, and the counter still read 0.
- **Not `pileSizeClientData`** — the massinfo member that carries pile sizes as
  `{"entries":[{"key":int,"value":int}]}`. A probe sent 16 entries with uniquely
  identifiable values; the counter stayed 0.
- **Not lazy loading** — the club endpoint was fetched 6 times that session.

**Unexplored contrast:** the `ACTIVE SQUAD` tab in the same bar correctly shows `11/23`.
So some counters work. Whatever differs between that one and the club one is likely the
answer.

### 5c. Store tiles render "unknown" — DIAGNOSED, NOT FIXED

Pack tiles show `unknown` with zero item counts. The `"unknown"` string is an
unconditional **default** in a string constructor — the field simply never gets written.
The store renders *display groups*, and `displayGroup` is parsed **recursively by the same
element parser**. Sending it populated **froze the store** (the type-desync busy loop), so
it is behind a flag, default off. Doing it properly needs the group's own field set worked
out rather than a self-referential copy of the pack.

### 5d. Seasons and Draft refuse — UNSOLVED, and not obviously server-side

Selecting **single-player Seasons** raises *"There was a problem communicating with the
FIFA Ultimate Team servers"* while making **zero requests to any layer**. UTAS, Blaze and
POW logs show only pings and one census subscription across the whole failure window. No
response can be wrong because no request was made. POW is eliminated (same failure with it
enabled and disabled).

**Online Draft** hangs the client rather than crashing it (process alive, no dump). The
one suspicious thing on the wire is `GET ut/%s/squad/mode/draft/state`, which our generic
`/squad` route answers with a full active-squad object: 23 slots, nested `itemData`, a
33-integer formation string. The real class wants `roundsInfo` plus a state enum, so this
is a textbook type-desync candidate and the timing matches. **Nothing has isolated it**;
it is a suspect, not a cause.

Both matter beyond themselves, because they are the only two routes into a match, and the
`/match` request shape has therefore never been captured.

---

## 6. Notable reverse-engineering findings

- **Class → deserializer resolution.** A response class's name literal is preceded by a
  **4-byte header**, and the constructing factory's `lea` points at *the header*, not the
  text. Lookups must use `name_address - 4`. Six attempts failed on this off-by-four;
  four of them returned zero results and nearly got recorded as "this class has no
  deserializer".
- **The request-template table is a floor, not a ceiling.** Several real endpoints are
  built by the caller appending a suffix and therefore never appear in the binary's URL
  table: `squad/list`, `user/club`, `club/stats/*`, `clientdata/<key>`. Only live traffic
  reveals them. This has caught the project three separate times.
- **The documentation lies.** The project's own `ENDPOINT_MAP.md` (~1,360 lines, ~100
  reversed structs) has been wrong repeatedly: it claimed `FutMoveCard` parses
  `chemistry` (it does not); it called seven store pack fields "skipped no-ops" (all are
  parsed); it gave price-object keys as `amount`/`currency` (the parsers read
  `externalPriceId`). **Verify against the decompiler before relying on any row.**
- **The online layer was hiding in an unpacked DLL.** "EA FC servers are unreachable"
  comes from a third HTTP API implemented in a *loose, unpacked, string-rich* library —
  not the packed executable, and not any layer previously emulated. It is redirectable
  purely through the Blaze config store.
- **The main executable is Denuvo-packed.** Its code exists only in a live process. Live
  memory is readable via `/proc/PID/mem` (the PE is mapped flat), which is how a crash
  site was disassembled. Any logic living there cannot be reversed statically.

---

## 7. Tooling built

- A **PyGhidra harness** with helpers for decompiling, xrefs, vtables, byte scanning and
  class→deserializer resolution. (Ghidra's Java/OSGi scripting is broken on this machine;
  PyGhidra bypasses it entirely.)
- A **minidump reader** — exception record, fault-time registers, module map, and a stack
  walk that recovers a usable backtrace.
- A **live code grabber** that reads and disassembles unpacked code out of a running
  process.
- A **read-only live probe** pattern for polling client model state while playing.
- **Two test suites**: 380 live contract checks (freeze-safety: asserts every response
  field's type against the reversed schema) and 51 pure unit checks for match rewards.
- A **traffic-replay audit** that diffs current responses against a known-good session —
  this is what proves a change did not alter what the client sees.

---

## 8. Documentation in-repo

| File | Contents |
|---|---|
| `ENDPOINT_MAP.md` | ~100 reversed response structs, atoms, types, freeze risks |
| `FUT_RESPONSE_REBUILD_PLAN.md` | Squad family, massinfo, the service-layer call graph |
| `REBUILD_RESEARCH.md` | Complete API surface, gap analysis, POW layer, and every eliminated hypothesis with its evidence |
| `CARD_SYSTEM.md` | How card identity resolves locally from the game's own database |
| `REPACK_INTEL.md` | Origin/LSX emulation notes |

---

## 9. Where help would be most valuable

1. **The MY CLUB counter (§5b).** Given the client demonstrably *has* the items and
   *renders* them, what else could a tab counter read from? Note that a sibling counter in
   the same bar works correctly.
2. **Seasons refusing with zero requests to any server (§5d).** The client raises a
   "problem communicating with the FIFA Ultimate Team servers" without contacting
   anything. Nothing on the wire can be wrong because nothing went on the wire.
3. **Whether the remaining problems are fixable server-side at all**, or whether the
   deciding logic lives in the Denuvo-packed executable and only live instrumentation can
   settle it. Note that this question was asked about `Send to Club` too, and there the
   answer turned out to be a plain wire fix, so treat "it must be client-side" as a
   hypothesis needing evidence rather than a fallback explanation.

Useful framing: this project's failures have almost always come from proposing a fix
before testing the assumption under it. Hypotheses that come with a cheap way to
disconfirm them are worth far more than plausible ones.

## FIFA 17 network-redirect milestone (2026-08-09)

Hook now installs a GENERIC network redirect on the fifa17 feature path (fifa17.rs
install_network_redirect): getaddrinfo IAT patch + inline connect detour + WSAConnect
IAT, with a configurable destination (connect_hook::set_target_ipv4) read from
openfut.cfg (single-line IP). Deployed DLL md5 bc9e0bc6, cfg=10.10.0.120.

RESULT of live launch (client 105 -> server 120):
- Error changed: "servers shut down" -> "Unable to connect to EA servers / check
  network". Redirect IS firing (progress).
- BLOCKER A: getaddrinfo IAT patched 0+0 -> FIFA 17 does NOT resolve via IAT
  getaddrinfo in the main exe or EAWebKit.dll. Names resolved via another path
  (gethostbyname or internal DirtySDK resolver). So no hostname reached 120.
- BLOCKER B (architectural): FIFA 17 online = Blaze binary TCP on high ports. Log
  shows connect 20.51.153.159:42230 sock_type=1 -> wsa_err=10035 (WOULDBLOCK->dead).
  Port 42230 is NOT in the remap set (443,10041,42127,3216) so it was not redirected.
  Even if redirected, the Docker bridge only speaks HTTPS on 8443 -- no Blaze
  listener exists for FIFA 17. This is a server-side build, not a hook tweak.

NEXT (evidence-first): add gethostbyname (and possibly a DirtySDK resolver) capture
to learn the hostname behind 20.51.153.159; widen Blaze port remap; then scope a
Blaze-speaking bridge listener before expecting the error to clear.

## DNS/getaddrinfo fix — RESOLVED (2026-08-09, hook md5 67e3639b)

Added src/resolver_hook.rs: INLINE detours at ws2_32 export addresses for
getaddrinfo + GetAddrInfoW + gethostbyname (same unhook/rehook pattern as
connect_hook). Replaces the IAT approach that patched 0 slots on FIFA 17.
Wired into fifa17.rs install_network_redirect; hooks.rs gained redirect_ip_cstr()
and redirect_ip_str() helpers.

LIVE RESULT (client 105 -> server 120):
- resolver detours 3/3 installed.
- getaddrinfo(winter15.gosredirector.ea.com) -> redirect. Game now dials
  10.10.0.120 (was 20.51.153.159 before). DNS BLOCKER A = SOLVED.

REMAINING BLOCKER B (architectural, NOT DNS): FIFA 17 online = EA Blaze binary
TCP. Game connects 10.10.0.120:42230 (gosredirector/Blaze redirector) ->
wsa_err=10035 (nothing listening). Two gaps: (1) connect_hook remap set lacks
42230; (2) even remapped, the Docker bridge only serves HTTPS on 8443 — no Blaze
listener exists. Clearing Unable to connect requires a Blaze redirector+main
server on the bridge side (real server build), not a hook change.
NOTE: the 3s TLS-handshake-EOF spam in bridge logs on :8443 is the LAUNCHER health
poller, not the game.