fifa17-recon: repair_club keeps dead cards unless asked, plus the 2026-08-05 plan
Deleting cards from someone's club is their call, not the tool's. The nine unrepairable blanks are now KEPT unless --delete-dead is passed. A blank card is ugly, not harmful, and the 175 stale cards were never the deletion candidates anyway: they are real players wearing old invented numbers and they get repaired in place. Also records the build round's synthesis as docs/plan-2026-08-05-families.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VUT92pz6RWKih9dSr8ZpxW
This commit is contained in:
funman300
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,172 @@
|
||||
# OpenFUT live-test script — 2026-08-05, non-player card families
|
||||
|
||||
Read this once end to end before touching anything. Total cost: one utas restart, one fresh FIFA launch, one MY CLUB visit.
|
||||
|
||||
---
|
||||
|
||||
## 1. What shipped
|
||||
|
||||
Six commits on `fifa17-fut-squad-and-userinfo`, nothing pushed. All three families default **OFF**. Two changes are **unflagged and already live in the working tree** (they take effect at the next restart whether or not you set a flag): the rareflag guard and the club_route mirror filter.
|
||||
|
||||
### The rareflag trap — no flag, always on (`d8ef9d4`)
|
||||
`fut_store._item()` takes `cardsubtypeid=`/`rareflag=` and forces `rareflag 0` on subtype 219.
|
||||
- MEASURED: `FUN_1801a88c0` is `*(rec+0x58) == 1`, and `FUN_1801bfac0` case 5 turns a rare Player Fitness card into a Squad Fitness card. rec+0x58 is read a second time, unconditionally, at line 108 for every cardtype, so the guard also stops a 219 being drawn as rare. That is intended and is stated in the source comment.
|
||||
- MEASURED: the player dict is byte-identical (20-key order and full value dict asserted in `tools/test_card_families.py`, 414 checks; mutation-checked).
|
||||
|
||||
### Consumables — `FUT_CONSUMABLES=1` / `=all` (`340c31f`, `0e4bc13`)
|
||||
126-item shelf over 69 subtypes, built from EA's own `fcc_*` rows.
|
||||
- MEASURED: the 172-subtype enum (144 live, 28 dead) re-derived twice independently from `FUN_1800d8330` + `FUN_18013f4d0`; EA's authored `(amount, rating)` per subtype; the four builder refusals (dead zone, missing `amount`, missing `contract`, rare 219).
|
||||
- Serves `type=contract` 13, `type=training` 42, `type=healing` 27, `type=development` 44. `=all` also answers an untyped club fetch with no `team=`/`league=` with all 126.
|
||||
- INFERRED: that any of those four `?type=` names is what the consumables screen asks for. **None has ever been on the wire.** `healing = categories {4,5}` is a guess (fitness has no arm of its own). `itemType: "player"` on a consumable is unobserved. `contract: 7` is **invented** — `fcc_contractcards` has no `amount` column.
|
||||
|
||||
### The four coach families — `FUT_COACHES=1` / `=all` (`9feb577`, `0e4bc13`)
|
||||
24-card shelf: 6 head coach, 6 GK coach, 6 physio, 6 fitness coach, ratings 55..80.
|
||||
- MEASURED: 411 real ids from read-only dumps with `rowcount == rows_emitted`; the merge key is the **raw** u32 at rec+0x18 (only players mask); each family's table-unique `DB Error` miss-fill (assetid 2000148 / 9000258 / 4000146 / 3000259, rating 0x32, rare 1); no row in any of the four tables has `value == 50`, so rating 50 can only ever be a miss. The four miss-fill ids and any row whose own stat write equals its family's miss-fill are excluded from the shelf.
|
||||
- A coach item sends 8 keys and **no invented geography**: no nation, no leagueId, no teamid, no preferredPosition, no attributeList, no rating, no rareflag, no assetId. (Diverges from the design agent's probe-shaped item_json, deliberately.)
|
||||
- Serves `type=headcoach|gkcoach|physio|fitnesscoach|staff`. `=all` also answers `type=manager`.
|
||||
- INFERRED: that the staff screen asks `type=manager` (observed exactly once, 2026-08-04) and nothing else.
|
||||
|
||||
### Managers — `FUT_MANAGERS=1` (`9b22435`, `0e4bc13`)
|
||||
10-card shelf on `type=manager|staff`: Luis Enrique 88, Conte 87, Wenger 86, Klopp 84, Koeman 82, Pardew 80, Di Francesco 78, Maes 75, Wdowczyk 70, Canning 65.
|
||||
- MEASURED: 417 rows, `carddbid == managerid + 1000000`, 297 join real names; `FUN_1801356c0` has no else-branch, so a wrong manager id is **completely silent**; `nation` lands at rec+0xde and `leagueId` at rec+0xe0 for a manager only, neither is written by the merge, and both have an identified reader (`FUN_1800e5940` publishing `NATIONALITY_ASSET_ID` and `LEAGUE_ID`); `negotiation` is published as `ATTRIB_CONTRACT_NEGOTIATION` from rec+0xe3 (Enrique is 3).
|
||||
- INFERRED: that the manager card **template** actually paints the flag, the league logo and the negotiation field. The readers exist in CardsDLL; the layout lives in the packed FIFA17.exe and cannot be read statically. TODO/CONFIRM.
|
||||
|
||||
### Architecture, all three
|
||||
Served as a **synthetic overlay, never granted into the save**. Clearing the flag restores the real club on the very next fetch. Overlay cards cannot be moved or quick-sold. The five staff sub-counters (stat ids 0xb..0xf, which `FUN_180094ce0` sums into STAFF_EMPLOYED) are computed from the overlay — that is the second, independent oracle.
|
||||
|
||||
### The mirror filter — no flag, always on (`0e4bc13`)
|
||||
`club_route` used to filter **nothing** on the `type=player`, `type=custom` and untyped branches. It now keeps `cardsubtypeid in (0,1,2,3)` there. Verified no-op: all 194 items in the live save are cardsubtypeid 0.
|
||||
|
||||
---
|
||||
|
||||
## 2. The test script
|
||||
|
||||
**Constraint that sets the shape:** env flags only take effect at process start, and restarting utas under a live client is what once produced "error connecting to Ultimate Team". So: FIFA closed, one restart, all three flags at their widest setting, then one fresh launch.
|
||||
|
||||
I dry-ran the armed server against a copy of the live profile at 10:11. The step-2 numbers below are what it actually returned, not predictions.
|
||||
|
||||
### Step 1 — Close FIFA, restart utas with all three flags (terminal only)
|
||||
|
||||
Quit FIFA from the game's own menu (`pkill -9 -f FIFA17.exe` if it will not).
|
||||
|
||||
```
|
||||
pkill -9 -f utas_server.py
|
||||
cd /home/alex/Documents/OpenFUT/fifa17-recon/tools
|
||||
setsid env FUT_CONSUMABLES=all FUT_COACHES=all FUT_MANAGERS=1 \
|
||||
python3 -u utas_server.py </dev/null >/tmp/utas.log 2>&1 & disown
|
||||
sleep 2 && ./openfut-fut.sh status
|
||||
```
|
||||
|
||||
Only utas restarts; lsx / blaze / roster / pow keep running. Do **not** use `openfut-fut.sh restart` — it restarts everything and drops the flags.
|
||||
|
||||
Positive: status shows `utas ✓ (8099)` and all other servers still up.
|
||||
Negative: utas down or 8099 not bound means a syntax or import error in the new code. Read `/tmp/utas.log`, fix, do not proceed. You have lost nothing but a terminal command.
|
||||
|
||||
### Step 2 — Prove the arms before spending a game launch (terminal only)
|
||||
|
||||
```
|
||||
for t in player manager staff training healing contract development; do
|
||||
printf "%-12s " $t
|
||||
curl -s "http://127.0.0.1:8099/ut/game/fifa17/club?type=$t&count=200" \
|
||||
| python3 -c "import sys,json;print(len(json.load(sys.stdin)['itemData']))"
|
||||
done
|
||||
printf "%-12s " untyped
|
||||
curl -s "http://127.0.0.1:8099/ut/game/fifa17/club?year=2017&count=200&level=any&sort=desc" \
|
||||
| python3 -c "import sys,json;print(len(json.load(sys.stdin)['itemData']))"
|
||||
python3 test_fut_contract.py | tail -2
|
||||
```
|
||||
|
||||
Positive, exactly: `player 194`, `manager 34`, `staff 34`, `training 42`, `healing 27`, `contract 13`, `development 44`, `untyped 320`, and `439 checks passed / 0 failed`.
|
||||
|
||||
Negative: any number off means the flags did not take or a module failed to import, and every in-game reading afterwards would be uninterpretable. `player` not 194 specifically means the mirror filter is wrong — stop, see §4(e). This step is the reason step 5 is readable at all, and it costs no menus.
|
||||
|
||||
### Step 3 — Launch FIFA fresh
|
||||
|
||||
`~/Desktop/launch-fifa17.sh` → Ultimate Team → security question (any answer) → the FUT hub. A fresh launch is required; stale in-process state produces the "FUT Squad Update" error.
|
||||
|
||||
### Step 4 — The players tab (you land on it; zero extra navigation)
|
||||
|
||||
Open MY CLUB. Read the default players list and click one club drill-down (Chelsea / team=5).
|
||||
|
||||
Positive: 194 footballers as before, no consumables and no staff mixed in; the Chelsea drill-down shows 26 items, not the whole club.
|
||||
Negative: **any** change here is a regression from the unflagged mirror filter, and it must be fixed before anything else is believed. See §4(e). This is the only step that can catch it, and it is free.
|
||||
|
||||
### Step 5 — The staff screen (the highest-information action in the run)
|
||||
|
||||
From MY CLUB open the **STAFF** tab. Read out, in this order:
|
||||
|
||||
1. The **STAFF EMPLOYED** number on the club stats panel.
|
||||
2. How many cards the staff tab shows.
|
||||
3. Whether **any** card reads `DB Error` with rating 50.
|
||||
4. For Luis Enrique: the name, the rating number, whether a **Spain flag** drew, whether a **LaLiga logo** drew, and the **CONTRACT NEGOTIATION** value.
|
||||
5. Whether **any** staff card shows a position label (ST/CM/...) or a row of six attribute numbers.
|
||||
|
||||
Positive: STAFF EMPLOYED reads **34**. 34 cards, real names, ratings 55..88, no `DB Error` anywhere. Enrique reads **88**, draws the Spain flag and the LaLiga logo, and shows negotiation **3**. No position label, no attributes.
|
||||
|
||||
That one screen chains, in order: the response ingested; cardsubtypeid reached rec+0x50 and `FUN_1800d8330` dispatched; five separate merges ran and hit; rec+0xde/rec+0xe0 are ours alone and are rendered (nothing else on the wire could have supplied that flag or that logo); and negotiation 3 is an independent second confirmation that the managercards row itself was read.
|
||||
|
||||
Negatives, each separately actionable:
|
||||
- **34 on the panel but the tab is empty.** Our club really holds the staff; the failure is downstream. The tab asks a `?type=` we do not answer, or staff items are not rendered. Step 7 says which.
|
||||
- **0 on the panel and the tab is empty.** The stats path never reached `_club_stat_set`. This is not a staff question at all and the coach ids must not be blamed.
|
||||
- **`DB Error` cards at rating 50.** The whole chain ran and the failure is specific to the key. The staff branches read rec+0x18 **raw**, so suspect a version byte, not the item shape. This is exactly why coaches are on this screen: a manager failing the same way is completely silent, so the coaches turn a blank manager card from uninterpretable into "the merge missed".
|
||||
- **Enrique reads 88 with a real name but no flag and no league logo.** The merge is fine and rec+0xde/rec+0xe0 are not what the manager template paints. Nothing else in the shipped shape changes; that is the TODO/CONFIRM above, resolved.
|
||||
- **Any staff card shows a position or six attributes.** Something upstream is still injecting `preferredPosition`/`attributeList` into staff items, because neither module sends them.
|
||||
|
||||
### Step 6 — The consumables screen (same visit, one tab press)
|
||||
|
||||
Walk to the consumables screen and every other MY CLUB tab you can reach. If any consumables appear, read the **title and the number** on three specific cards: subtype 219 (amount 20), subtype 220 (amount 10), subtype 61 (amount 5).
|
||||
|
||||
Positive: 219 titles Player Fitness (or the raw key `FUT_CONSUMABLE_NAME_PLAYERFITNESS`) and reads **20**; 220 titles Squad Fitness / `FUT_CONSUMABLE_NAME_SQUADTRAINING` and reads **10** — same family, different title, the trap seen from the safe side; 61 titles Player Training (Pace) and reads **+5**. The numbers are positive.
|
||||
|
||||
Negatives:
|
||||
- **Cards appear but every number reads `-1`** (it will be -1, not 255: `FUN_1801a8040` sign-extends). Atom 0x1b never reaches rec+0xbf and the entire amount-bearing half of the family is unshippable until that is found. The title still being right is what separates this from "consumables do not render" — title and number have independent sources.
|
||||
- **Nothing appears.** Do not conclude anything here; go to step 7. The item list for this screen has never been observed on the wire, so absence is expected-ish and only the log distinguishes "we answered the wrong arm" from "the screen is not a club fetch at all".
|
||||
|
||||
### Step 7 — Read the log (terminal only, and it is what makes step 6 worth doing)
|
||||
|
||||
```
|
||||
grep 'family overlay armed' /tmp/utas_server.log | tail -60
|
||||
grep -o 'GET /ut/game/fifa17/club[^ ]*' /tmp/utas_server.log | tail -60
|
||||
grep -o '/club/stats/[a-z0-9/]*' /tmp/utas_server.log | tail -20
|
||||
```
|
||||
|
||||
Positive: the census shows a `type=` we serve, and the `+N item(s)` on that line is non-zero.
|
||||
|
||||
Negatives:
|
||||
- **A `type=` we do not answer appears** (anything outside player/manager/custom/staff/headcoach/gkcoach/physio/fitnesscoach/contract/training/healing/development). Aim the flag at that arm at the next start. The request reached us; this is a one-word fix.
|
||||
- **Only `type=player`, `type=custom`, `type=manager` and untyped-with-`team=` appear, exactly as before.** Then the consumables item list is not a `club?type=` fetch under any name we serve, and the next round is a route hunt, not an item-shape hunt. Note that `/club/stats/consumables` fires 35 times in the existing log and `/club/stats/staff` 9 times, so the screens exist and do talk to us — the item list simply arrives by some other door.
|
||||
|
||||
**When done:** leave it armed if you want; nothing is written to the save. To disarm, restart utas with no flags (step 1 without the `env` prefix) and the real club returns on the next fetch.
|
||||
|
||||
---
|
||||
|
||||
## 3. What was skipped, and why
|
||||
|
||||
- **The rareflag trap, live.** The shelf builder now **refuses** to construct a rare 219 — that is the fix. Testing it live means hand-building the exact item we shipped code to prevent. The static chain is verified end to end (atom 0x271 → rec+0x58 → `FUN_1801a88c0` → the squad branch, and rec+0x58 is not overwritten for cardtype 6). Not worth a menu trip.
|
||||
- **The four-card manager probe** (`fut_staff.probe_items()`, including resourceId `(1<<24)|1000509`). Not wired into the overlay; wiring it costs a **second** restart. Its only unique leg is raw-vs-masked key, and that question only becomes live if managers come back blank while coaches resolve. Fire it in that branch, not before.
|
||||
- **Club items** (badge / kit / stadium / ball). `data/club_items.json` is on disk, uncommitted, with no reviewer verdict. Also `sweep_items()` answers **before** the type filter and keeps `itemType: "player"`, so any club-item experiment through the sweep is double-confounded.
|
||||
- **Manager formation modifiers 71-86** — zero rows in the 143-row `fcc_trainingcards`, and `FUN_1801bfac0` case 6 calls `FUN_1801a0100` on the formations result without the rowcount guard its twin case 7 has. A crash candidate. **Formation 121-136** — artwork constant is -1. **Manager league 300-341** — case 10 formats literally `"ML: %d"`, so even a perfect card reads "ML: 13". None are on the shelf.
|
||||
- **The dead-zone canary (subtype 89).** Its positive changes no decision (we only ship subtypes from `consumables.json` either way), and `consumables.json` predicted the wrong title for the dead zones until this round, so scoring it against the file was itself a trap.
|
||||
- **Coach nation/league echo.** The shipped coach item omits nation/leagueId/teamid entirely, so there is nothing left to echo. That test died with the item shape.
|
||||
- **Equipping a manager into the squad slot.** `FUN_18013d1f0` was never read for a staff member and PUT /squad's staff shape is unobserved.
|
||||
- **Packs.** Deliberately not in the same round as the shelf.
|
||||
|
||||
---
|
||||
|
||||
## 4. What could break, and the instant fallback
|
||||
|
||||
**(a) The restart drops the client** → "error connecting to Ultimate Team". This is why step 1 happens with FIFA closed. If it happens anyway: `./openfut-fut.sh start`, then relaunch FIFA fresh.
|
||||
|
||||
**(b) The STAFF tab chokes on 34 mixed items, or coaches confuse a manager-only tab.** Fallback at the next start: `FUT_COACHES=1` (removes coaches from `type=manager`, leaving the 10 managers alone) or `FUT_MANAGERS=` (leaves the 24 coaches alone). Both need a restart, so decide from the step 5 reading, do not iterate blind.
|
||||
|
||||
**(c) The untyped arm buries the consumables.** With `=all`, an untyped fetch returns 194 players **then** 126 consumables, and the players tab pages at `count=11&start=0,10,20,...`. If the client truncates each page, the consumables sit past the end. TODO/CONFIRM. Fallback: `FUT_CONSUMABLES=1` — drops the untyped arm entirely, and under the typed names consumables are served from position 0.
|
||||
|
||||
**(d) Any flag typed as `0`.** Already handled: `_family_flag()` treats `""`, `"0"`, `"off"`, `"no"`, `"false"` as OFF (`21b2263` — before that fix, `FUT_CONSUMABLES=0` would have turned the family **on**). Prefer unsetting the variable outright.
|
||||
|
||||
**(e) The players tab changes (step 4).** This is the mirror filter, and it has **no flag** — it is unconditional in `club_route`'s else-branch. Verified a provable no-op against the live save (all 194 items are cardsubtypeid 0), so this is unlikely, but if it fires the fallback is to restore that one line to a no-op in `tools/utas_server.py` and restart. Do not `git revert 0e4bc13`; that also removes the overlay wiring and the log line you need.
|
||||
|
||||
**(f) Step 2 disagrees with the expected numbers.** Unset all three flags and restart: you are back to exactly today's server. Nothing was written anywhere.
|
||||
|
||||
**(g) You try to move or quick-sell an overlay card.** Do not, during this run. Overlay items are not in the save, so the client is holding an id the server cannot find. TODO/CONFIRM what it does. If it happens: clear the flag, refetch, the card is gone.
|
||||
|
||||
**(h) The save.** The overlay never writes to it. The one write in this whole script is `test_fut_contract.py`'s club-rename round-trip, which restores the original name in a `finally` block.
|
||||
@@ -13,8 +13,10 @@ Two different faults, two different remedies:
|
||||
|
||||
DEAD the playerid does not exist in the roster at all. The client misses on it
|
||||
and stamps its generic card (rating 50, teamid 1933, nation 14, blank
|
||||
name). These are the blanks on screen and there is nothing to repair, so
|
||||
they are REMOVED.
|
||||
name). There is nothing to repair, because there is no player to repair it
|
||||
to. They are LEFT ALONE unless --delete-dead is passed: removing cards from
|
||||
someone's club is their call, not the tool's, and a blank card is ugly
|
||||
rather than harmful.
|
||||
|
||||
RUN THIS WITH utas_server STOPPED. The server holds the profile in memory and
|
||||
rewrites it on its own schedule, so an edit made underneath a running server gets
|
||||
@@ -24,8 +26,9 @@ cards were removed and reappeared a few hours later.
|
||||
Squad safety: a card referenced by a saved squad is never removed. Repair is safe
|
||||
for squad members because the item id does not change.
|
||||
|
||||
repair_club.py dry run (default)
|
||||
repair_club.py --fire write, after taking a timestamped backup
|
||||
repair_club.py dry run (default)
|
||||
repair_club.py --fire repair stale cards, keep dead ones
|
||||
repair_club.py --fire --delete-dead also remove the unrepairable blanks
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
@@ -43,6 +46,8 @@ POOL = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "data", "p
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--fire", action="store_true")
|
||||
ap.add_argument("--delete-dead", action="store_true",
|
||||
help="also remove cards whose playerid is not a real player")
|
||||
a = ap.parse_args()
|
||||
|
||||
truth = {p["id"]: p for p in json.load(open(POOL))}
|
||||
@@ -88,13 +93,18 @@ def main():
|
||||
continue
|
||||
repaired.append((it, changes, t))
|
||||
|
||||
keep_dead = [d for d in dead if d.get("id") in squad_ids]
|
||||
drop_dead = [d for d in dead if d.get("id") not in squad_ids]
|
||||
if a.delete_dead:
|
||||
keep_dead = [d for d in dead if d.get("id") in squad_ids]
|
||||
drop_dead = [d for d in dead if d.get("id") not in squad_ids]
|
||||
else:
|
||||
keep_dead, drop_dead = dead, []
|
||||
|
||||
print("club %d: %d already correct, %d to repair, %d dead to remove"
|
||||
% (len(items), untouched, len(repaired), len(drop_dead)))
|
||||
if keep_dead:
|
||||
print(" %d dead card(s) KEPT because a squad references them" % len(keep_dead))
|
||||
why = ("a squad references them" if a.delete_dead
|
||||
else "--delete-dead was not passed")
|
||||
print(" %d dead card(s) KEPT (%s)" % (len(keep_dead), why))
|
||||
for it, ch, _ in repaired[:8]:
|
||||
print(" repair id=%-11s asset=%-7s %s" % (it.get("id"), it.get("assetId"),
|
||||
"; ".join(ch)[:80]))
|
||||
|
||||
Reference in New Issue
Block a user