diff --git a/fifa17-recon/docs/priority-2026-08.md b/fifa17-recon/docs/priority-2026-08.md new file mode 100644 index 0000000..bdd7d31 --- /dev/null +++ b/fifa17-recon/docs/priority-2026-08.md @@ -0,0 +1,483 @@ +# OpenFUT priorities, August 2026 + +Written 2026-08-04. Supersedes the ordering implied by `REBUILD_RESEARCH.md` §14. + +This document exists to answer one question: **what is the next thing worth doing, and +how would we know if it worked?** Everything here is ordered by the cost of the +measurement that would settle it, not by how important the outcome feels. + +Two rules govern every entry: + +- A hypothesis without a cheap way to disconfirm it is worth less than a measurement. +- Omission is safe; a wrong shape freezes the game. When a body's shape is unknown, the + correct action is to send less, not to guess more. + +Values that have not been confirmed against the decompiler or against live traffic are +marked `TODO/CONFIRM`. Where I do not know something and cannot find out without running +the game, the entry says so rather than reporting something adjacent. + +--- + +## 1. What changed since the assessment brief was written + +Three findings landed after the brief and they reorder the plan. + +### 1a. The observation session ran, and did not produce the match shape + +The combined session (play a match, then navigate to Seasons / Tournaments / FUT Champions +with `FUT_MODES` still off) executed on 2026-08-03 under the stated constraints: one +launch, zero config changes, pure observation. It returned three things, none of which was +the target: + +| finding | evidence | +|---|---| +| Seasons refuses with **zero requests to any layer** | UTAS, Blaze and POW logs show only pings and one census subscription during the failure | +| `squad/mode/draft/state` exists and is answered wrongly | Real client (`ProtoHttp`) requested it; our generic `/squad` route returned a full active-squad object | +| Draft hung the client (process alive, no dump written) | Not a crash. Nothing has isolated the cause; the wrong body is the obvious suspect with exact timing, but it is **not** logged as the cause | + +The consequence for planning: **both known routes into a match are blocked**, so the +`/match` request shape cannot be captured by observation alone any more. It now sits behind +a fix rather than in front of one. That is a real demotion and it is why the match shape is +no longer the next task. + +`squad/mode/draft/state` is the fourth endpoint this project has found that is invisible to +the binary's URL template table because the caller composes it by appending a suffix. The +template table is a floor, not a ceiling. Treat it as such permanently. + +### 1b. The User-Agent split + +The request log records User-Agent, so real client traffic (`ProtoHttp`) can be separated +from this project's own curl and python probes. Doing that changed the project's picture of +itself materially: + +| endpoint | real client | our probes | +|---|---|---| +| `/user/accountinfo` | 23 | 49 | +| `/clientdata/userHubData` | 13 | 2 | +| `/leaderboards/options` | 5 | 1 | +| `/season`, `/season/user` | **0** | 2 | +| `/tournament`, `/tournament/user` | **0** | 3 | +| `/champion` | **0** | 2 | +| `/match` | **0** | 2 | +| `/clubUser` | **0** | 93 | +| `/user/list` | **0** | 180 | + +Endpoints this project had treated as exercised were exercised only by itself. `/clubUser` +and `/user/list` in particular have 93 and 180 recorded hits and **not one of them came +from the game**. Any claim of the form "the client asks for X" that predates this split is +unsupported until re-checked with the filter on. + +This has been propagated into `OPENFUT_PROJECT_REPORT.md` §5 and §6. + +### 1c. The `FutMoveCard` retraction + +This is the largest correction in the project's history and it is documented in full in +`REBUILD_RESEARCH.md` §16 and in commit `f488793`. + +The repo claimed `FutMoveCard` (`0x180128600`) had **no** skip handler and parsed only +`itemData` and `dreamSquads`. It has two skip-handler call sites and parses seven atoms. +The claim came from writing `src[:4000]` of a 6,193-character decompile and then searching +the truncation. Both skip-handler sites and four of the seven atoms lay past the cut. + +The cost was not the wrong fact. The cost was the inference built on it: "any extra key +desyncs this parser, so the response body cannot be the problem, so the deciding factor is +client-side state, not the wire." That premise directed seven consecutive attempts at +`Send to Club` and it was wrong. + +What the correct read gives us is a **necessary condition that had never been satisfied**: + +``` +PUT ut/%s/item returns per-item VERDICT records, not an acknowledgement. + id(0x15c) INT -> record+0x00 + pile(0x226) STRING -> enum 0x180142650 (club=7, purchased=6, trade=5) + success(0x2fa) BOOL -> record+0x0c, initialised to '\0' per element + reason(0x279) STRING -> "Destination Full" = 0xf + dreamSquads(0xe9) INT array; anything else -> skip handler +``` + +The completion handler raises `EVENT_CARDS_MOVE_CARD_FAILURE` when the record vector is +empty **or** when `record+0x0c != 1`. Every body this project has ever returned from that +endpoint, `{}` included, therefore reported the move as failed. Quick sell survives an +identical `{}` because its callbacks read only the transport code and ignore the body. +That is the whole asymmetry, and it was on the wire the entire time. + +The correct shape is implemented and staged behind `FUT_MOVE_BODY=ack`. The default remains +`empty` because **sufficiency is untested**: a necessary condition being satisfied does not +prove there is no second one. + +--- + +## 2. The next task + +**One launch. Set `FUT_MOVE_BODY=ack`. Buy a pack, reveal it, press Send to Club.** + +Everything about this is cheap. One flag, one existing menu path, one binary outcome +observable without any instrumentation: either the session survives or the client returns +to the main menu with the FUT connection error. + +It is first for four reasons: + +1. It is the only open problem where the decompiler has produced a **verified necessary + condition that has never been satisfied**. Every other open item is a hypothesis. +2. The failure it targets is the single largest user-visible defect. The current workaround + (deposit pack contents straight into the club and keep the pending pile empty) makes the + reveal screen show nothing to assign. +3. It disconfirms cheaply in both directions. If the session survives, `Send to Club` is + fixed and seven attempts' worth of client-state theorising is closed. If it still fails, + we have eliminated the wire as a *sufficient* explanation while keeping the necessary + condition satisfied, which is a genuinely narrower position than before. +4. It carries no freeze risk that the current default does not already carry. The response + is a three-field flat record vector against a deserializer whose full atom set is now + known, with two confirmed skip handlers. + +### Pre-launch checks + +- `FUT_MOVE_BODY=ack`, everything else at its default. In particular leave + `FUT_PACK_AUTOCLUB=1` alone; the ack path was written for that configuration. +- Confirm the running server is the new code. Query `PUT /item` with a scratch profile and + check the response is a record vector, not `{}`. A silent restart failure has already + cost this project one live test and one pack. +- Back up the profile with `fut_admin.py --backup` first. + +### What to record + +- Whether the session survives, verbatim error text if not. +- Whether the card actually appears in MY CLUB afterwards. +- Whether `ut/delete/auth` follows (the logout that has accompanied every previous failure). +- The exact request body the client sent, filtered to `ProtoHttp`. + +### Do not + +- Do not change a second flag in the same launch. If ack fails while something else is also + different, the result explains nothing. +- Do not fix anything else noticed during the session. +- Do not run a probe against the server while the client is running; the User-Agent table + above is the reason. + +--- + +## 3. The measurement queue + +Ordered by cost of the measurement, cheapest first. Each entry states what would disconfirm +it, because an entry that cannot be disconfirmed does not belong on this list. + +### 3.1 `Send to Club` under `FUT_MOVE_BODY=ack` + +Covered in §2. Cost: one launch, one flag. Status: staged, unrun. + +### 3.2 The MY CLUB counter, `GET /club/stats/` + +The hub tab bar reads `MY CLUB 0` while the club holds 99 items and renders all of them +correctly when opened. Eliminated already: the item list itself, `pileSizeClientData` (a +16-entry probe with uniquely identifiable values changed nothing), and lazy loading. + +The counter is server-supplied by `GET /club/stats/`, whose response class is +`FutStickerBookStats2` (`0x180130150`), body shape +`{"stat":[{contextId, contextValue, type, typeValue}]}`. The factory **wipes the map before +parsing**, which is why the current `{}` route is not neutral: it actively clears the +counter every poll. The paired list endpoint `FutStickerBookSearch` has no count atom at +all, and that asymmetry is the reason the list can be right while the number is zero. + +Cost: one flag, one glance at the hub. Disconfirmed by: the counter staying at 0 with a +populated stat set. + +`TODO/CONFIRM`: which `contextId` and `type` values correspond to the club bucket. The +enum has been read but the mapping to the specific tab has not been verified against a +screen. A reviewer flagged that the current plan probably populates the wrong bucket for +the screens actually opened, and that is unresolved. Expect to need a second iteration +varying only the ids. + +Also worth taking for free during any launch that reaches the hub: **`ACTIVE SQUAD` in the +same tab bar reads `11/23` correctly.** Whatever differs between that counter and the club +one is a direct lead and it costs nothing to look at both side by side. + +### 3.3 Route `squad/mode/draft/state` away from the generic `/squad` handler + +Not a measurement, a fix; it is here because it is the cheapest thing that could unblock a +route into a match. Right now that endpoint receives a full active-squad object: 23 slots, +nested `itemData`, the 33-integer `custom` string. The recovered `FutGetDraftCurrentState` +schema wants `roundsInfo` plus a state enum (`CAPTAIN_DRAFT`, `PLAYER_DRAFT`, +`READY_FOR_MATCH`). Feeding that parser a squad object is a textbook type-desync candidate +and the timing of the hang matches. + +Per the omission rule, the first version should route it to a **minimal** body, not a +guessed-complete one. Behind a flag. + +Disconfirmed by: Draft still hanging at the same point with the correct minimal body. That +would be genuinely informative, because it would mean the hang is not the response. + +### 3.4 `FUT_ACCOUNTINFO=1` + +The real client requests `/user/accountinfo` 23 times and we return `{}`. It is the +highest-traffic endpoint we deliberately stub, and after the User-Agent split it is one of +only three endpoints with meaningful real-client traffic that we do not serve properly. + +Risk: medium, and honestly stated. This is a populated body on a path the client genuinely +parses, which is the exact class of change that broke the store and broke pack opening. +One flag, one launch, revert on any freeze. + +### 3.5 Seasons + +Currently unactionable from the server side and I will not pretend otherwise. The client +raised "There was a problem communicating with the FIFA Ultimate Team servers" **without +contacting any server**. UTAS, Blaze and POW logs are all clean for that window. POW is +eliminated (the same failure occurs with POW fully disabled and with it enabled). + +This is the third time FIFA's error text has described a network failure that did not occur. +The strings are generic session-error text, not diagnostics; stop reading them as evidence +about the network. + +**I cannot tell what gates Seasons without instrumenting the running client.** The decision +logic is in the Denuvo-packed executable, whose code exists only in a live process. The +only avenue that would produce evidence is the live-memory probe pattern +(`/proc/PID/mem`, the PE maps flat at `0x140000000`), used read-only against the FUT state +machine while the menu is open. That is a real session of work, not a cheap measurement, +and it should not be started until §2 through §3.4 are done. + +### 3.6 The `/match` request and response shape + +Still uncaptured. `/match` has never been requested by the real client. `_match_result()` +falls back to `draw` because the score fields are unknown; the reward path has never +executed against a real match. + +This is now **blocked behind §3.3 or §3.5**, since both routes into a match are refusing. +It is not a task, it is the payoff of one. + +### 3.7 Store tiles reading "unknown" + +Diagnosed, not fixed, and deliberately parked. The `"unknown"` string is an unconditional +default in a string constructor: the field is never written. The store renders *display +groups*, and `displayGroup` is parsed recursively by the same element parser. Sending it +populated **froze the store**, so it sits behind `FUT_STORE_GROUPS`, default off. + +Fixing it properly requires working out the display group's own field set rather than +copying the pack's, which is decompiler work with no cheap live measurement attached. It is +cosmetic; packs are buyable and openable. Park it. + +--- + +## 4. The Rust port + +### 4.1 Timing: after, not now + +The port should happen after the core loop is live-proven, and the argument that leads is +the **match result shape**. + +`_match_result()` currently falls back to `draw` because nobody knows which fields carry +the score. That is not a detail of the match endpoint; it is the shape of the data the +reward economy consumes. Porting an economy whose input schema is a placeholder means +porting the placeholder, then porting the correction. The Python is where cheap schema +iteration happens: a wrong guess costs a restart, not a recompile and a type refactor. + +The same argument applies to `Send to Club`, to draft state, and to club stats. Three of the +four items in §3 will change a response schema. Freeze the schemas, then port. + +### 4.2 The contract suite can certify a port + +`test_fut_contract.py` (452 lines, 380 checks) was decoupled from the Python implementation +this cycle. It now imports `json` only, manipulates no `sys.path`, and reaches the server +over HTTP with `FUT_TEST_BASE` and `FUT_TEST_PERSONA_ID` configurable by environment. + +That means it is an **implementation-independent conformance suite**. A Rust server that +passes all 380 checks is wire-compatible with the Python one to the extent the suite +covers, and the suite's checks are type assertions against the reversed schemas, which is +exactly the property that matters here (a scalar where an object belongs is what freezes +the game). This is the single most valuable de-risking asset the port has, and it now +exists. + +`test_match_rewards.py` (51 pure unit checks) is not portable in the same way; it tests +Python functions directly. Its 51 assertions would need re-expression against the ported +economy. + +### 4.3 How much of `openfut-core` survives: measured + +The proposed narrowing was **"core owns ownership and economy, keyed by opaque integer item +ids; core knows nothing about card identity."** That boundary is correct, and every +per-item prediction made about it held up: + +| divergence | prediction | verdict | +|---|---|---| +| int item id / `assetId` / `resourceId` | deleted from core | confirmed | +| numeric nation / league / teamid | deleted from core | confirmed | +| `rareflag` / `cardsubtypeid` vs the Hero/Icon enum | deleted from core | confirmed | +| 33-int formation array, kicktakers, personaId | slots relational in core, encoded by bridge | confirmed | +| the pile model | genuine work, survives | confirmed | + +Under narrowing the clearly-deleted identity surface is `models/card.rs` (62 lines), +`models/chemistry_style.rs` (26), `services/card_db.rs` (52) and `routes/cards.rs` (173), +so 313 lines, plus the identity-dependent parts of `squad.rs` chemistry and `sbc.rs` +validation. About 10 percent deleted outright. On that framing, narrowing looks survivable. + +**Narrowing is not what guts core. FIFA 17 relevance is.** Mapping each core service +against the 45-template FIFA 17 URL table, six services model features that have no FIFA 17 +endpoint at all: + +| service | lines | +|---|---| +| `achievement` | 215 | +| `objective` | 175 | +| `event` | 172 | +| `checkin` | 133 | +| `upgrades` | 125 | +| `notification` | 49 | +| **total** | **869** | + +These are modern live-service concepts. FIFA 17 has no achievements endpoint, no events, no +notifications, no daily check-in, no card upgrades. `objective` is not merely unmapped: +this project searched the binary and found **no `FutGetObjectives` class exists**, which is +why the hub's Manager Tasks tile has no known feed. + +Those 869 lines are identity-free, so the narrowing boundary keeps every one of them. They +survive perfectly and are worth nothing here. + +The integration suite reclassifies the same way: + +| | tests | +|---|---| +| target features with no FIFA 17 endpoint | 26 | +| assert identity or presentation being deleted | 14 | +| ownership and economy, survive | **61** | + +An earlier assessment in this project reported "81 of 101 survive" and used it to argue the +suite was an asset. The number was right and the implication was wrong: 26 of those 81 test +features this game does not have. The real figure is **61 of 101, about 60 percent**. + +To name that plainly: core's reusability was argued before its FIFA 17 relevance was +measured, and the 6,707-line total was doing persuasive work it had not earned. Reusing +existing code is attractive partly because it exists and represents effort already spent, +and that is not a technical reason. + +**Conclusion.** Reuse `openfut-core` for its scaffolding (migrations, pool, app wiring, +error model) and for the 61 relevant tests, not for its service layer. What genuinely +transfers is the economy services that have a real FIFA 17 endpoint (`club`, `market`, +`pack`, `match_service`, `profile`, `squad` storage, `statistics`), roughly 1,400 lines. +Writing a FIFA-17-shaped core fresh against the existing scaffolding is comparable effort +to adapting the current one, and adapting carries a specific risk: dragging 869 lines of +live-service machinery along because deleting working code feels wasteful. + +`TODO/CONFIRM`: `season`, `fut_champs`, `draft` and `sbc` (1,013 lines) do have FIFA 17 +endpoints but materially different schemas, and none has ever been exercised live. Whether +their logic survives is unknown and cannot be settled until §3 produces live traffic for +them. + +`TODO/CONFIRM`: the pile model's cost. It needs a migration adding a pile column to owned +items, a move operation enforcing the "never in two piles" invariant, and per-pile queries. +I will not estimate lines without reading the ownership tables properly. + +### 4.4 The Python side of the port is cleaner than expected + +`utas_server.py` holds 48 handler functions. Twenty touch `STORE.*`, and only 44 lines in +total reference it. State access is shallow and concentrated rather than smeared through +serialization, so an HTTP seam between a Rust core and the existing Python wire layer is +viable. This would be an incremental extraction, not a rewrite. + +The wire and protocol layers (Blaze, LSX, POW, roster, seed schema; 3,083 lines) should be +ported last or not at all. They are working, they are protocol-shaped rather than +logic-shaped, and they are the least likely part of the system to change. + +--- + +## 5. Standing requirements + +These are not tasks with a completion date; they are rules that apply to future work. + +**The User-Agent filter is the default.** Every capture and analysis tool must filter to +`ProtoHttp` by default and require an explicit flag to include this project's own probes. +The unfiltered view has already produced at least one wrong picture of what the client +asks for. This is recorded as a requirement and **is not yet implemented**; it was outside +the read-only scope of the assessment cycle. + +**Every risky change ships behind a flag whose default is the live-proven value.** This +convention exists because two working screens were broken by shipping corrections on by +default: `FUT_STORE_GROUPS` froze the store, and `FUT_STORE_FIELDS` stopped packs opening. + +**"The parser reads X" is not "changing X is safe."** That inference is what broke pack +opening. + +**Never conclude an absence from a truncated or unverified-length extraction.** The +`FutMoveCard` retraction and the earlier `Memory.getBytes` bytearray scan (which silently +read nothing and returned zero hits) are the same failure. Assert that the searched region +covers the whole function before reporting that something does not appear in it. + +**Verify against the decompiler and against live traffic, not against the notes.** +`ENDPOINT_MAP.md` has been wrong repeatedly: it claimed `FutMoveCard` parses `chemistry` +(it does not), called seven store pack fields skipped no-ops (all are parsed), and gave +price-object keys as `amount`/`currency` when the parsers read `externalPriceId`. + +**Clean-room only.** Wire format is learned from the game's own binaries. No EA source. + +`TODO/CONFIRM`: two `.rdata` addresses flagged as wrong during review have not been +corrected and must be fixed before they propagate into `ENDPOINT_MAP.md`. + +--- + +## 6. Flags: current defaults and their justification + +| flag | default | why | +|---|---|---| +| `FUT_MASSINFO` | `full` | live-proven; boot-critical | +| `FUT_USERINFO` | `roster` | live-proven; `packs` and `full` are untested rungs | +| `FUT_PACK_AUTOCLUB` | `1` | the `Send to Club` workaround; revisit after §2 | +| `FUT_MOVE_BODY` | `empty` | `ack` is correct-by-decompiler but sufficiency is unproven | +| `FUT_MARKET` | `sample` | live-proven | +| `FUT_MODES` | off | changes `/season`, whose array-root freeze risk cannot fire unless the client asks, and it has never asked | +| `FUT_ACCOUNTINFO` | off | queued at §3.4 | +| `FUT_SQUAD_LIST` | `off` | live-proven off | +| `FUT_PILESIZES` | off | probe showed no effect on the counter | +| `FUT_STORE_GROUPS` | off | **froze the store** | +| `FUT_STORE_FIELDS` | off | **stopped packs opening** | +| `FUT_CLUB_RENAME` | off | `clubNameChangeAllowed: true` crashes the client at `FIFA17.exe+0x71b8651`; a contract check guards this | +| `FUT_CLUB_IDENTITY` | `off` | the real client has never requested those paths (0 of 93, 0 of 180) | + +Do not enable the bottom five without a new reason. "To see what happens" is not one. + +--- + +## 7. What could make this plan wrong + +The plan's ordering rests on assumptions that could each fail, and it is worth stating how +each failure would show itself. + +**The ack could satisfy a necessary condition and still fail.** This is the most likely bad +outcome and it is not a refutation of the analysis; `EVENT_CARDS_MOVE_CARD_FAILURE` firing +on an empty or `success != 1` vector is confirmed, but nothing establishes it is the *only* +gate. If ack fails, the next question is whether the failure event still fires (which would +mean a second wire condition) or whether the client fails somewhere else entirely (which +would revive, on much narrower grounds, the client-state line the retraction closed). The +plan should not treat a negative ack result as "back to square one." + +**The retraction may not be the last one.** Every atom table and schema in this repo was +produced by the same extraction tooling that produced the false `FutMoveCard` claim. The +specific bug (a truncated decompile) has been identified, but the general risk (an absence +inferred from an incomplete search) applies to every "this class does not parse X" statement +in `ENDPOINT_MAP.md`. Any of them could be wrong in the same direction. Treat every negative +claim in the docs as weaker than the corresponding positive claim. + +**Seasons might be gated by something no server can supply.** If the gate is an entitlement +check, a persona attribute or a Blaze session property evaluated entirely inside the packed +executable, then no amount of response-shape work reaches it and §3.5's live-probe session +is the only path. It is also possible that path shows the gate is unreachable, in which case +Seasons is out of scope and the match loop has to be reached through Draft or not at all. + +**The draft-state hang might not be the response body.** The timing is suggestive and the +type-desync mechanism is plausible, but nothing has isolated it. If a correct minimal body +still hangs, §3.3 does not unblock the match shape and §3.6 stays blocked behind §3.5, +which is the expensive branch. + +**The port conclusion assumes the narrowed core is the right architecture at all.** The +measurement in §4.3 answers "how much of core survives narrowing", not "is a Rust core the +right destination". The Python is working, live-proven on the paths that matter, and cheap +to iterate. If schema churn continues at the current rate through the whole of §3, the +honest answer might be that the port should be deferred again rather than scheduled. +The trigger to revisit is: schemas stop changing for a full cycle of live sessions. + +**The User-Agent finding might understate the problem rather than overstate it.** The split +distinguishes `ProtoHttp` from our probes, but it does not prove every `ProtoHttp` request +was made by the screen we assumed. Attributing requests to specific menu actions still rests +on timing correlation, which is weaker evidence than the User-Agent field itself. + +**Some of this game has never been opened.** Large parts of the FUT surface (SBCs, most of +the store, tournaments, leaderboards, champions) have never been navigated to by a human in +this project. Endpoints reading "0 client requests" mean "never visited", not "never +wanted". The next unvisited screen could produce a fifth suffix-endpoint surprise and +reorder everything below §2.