From 88da16a11e046726491149a713c645de3fddc732 Mon Sep 17 00:00:00 2001 From: funman300 Date: Tue, 11 Aug 2026 22:57:02 +0000 Subject: [PATCH] feat(fifa17): curated dev content pack generator + Core dev seed (submodule 36abd4b) --- docs/PROJECT_STATE.md | 131 +++++++++++++++++++++++++++++++++++ openfut-core | 2 +- scripts/seed_fifa17_cards.py | 118 ++++++++++++++++++++++++++++--- 3 files changed, 240 insertions(+), 11 deletions(-) create mode 100644 docs/PROJECT_STATE.md diff --git a/docs/PROJECT_STATE.md b/docs/PROJECT_STATE.md new file mode 100644 index 0000000..639a909 --- /dev/null +++ b/docs/PROJECT_STATE.md @@ -0,0 +1,131 @@ +# OpenFUT — Project State + +> **Canonical source:** `../OpenFUT-Vault/06 Agent Memory/Project State.md` +> This file is a mirror. If the two disagree, the vault wins. Update the vault first. + +Factual snapshot. Prefer this over the stale root `README.md`/`CLAUDE.md` status tables (FIFA 23). +Last compiled from repository evidence during context initialization. + +## Working + +- **FIFA 17 offline FUT stack, end-to-end.** Proven 2026-08-01: auth → Blaze login → device-trust → + the FUT hub. Brought up by `fifa17-recon/tools/openfut-fut.sh start`. Evidence: `FUT-RUNBOOK.md`, + `fifa17-recon/README.md`, the five responder scripts, gate-ladder troubleshooting table. +- **ProtoSSL cert-pin defeat** — two live `/proc/PID/mem` patches (`autopatch.py`), VAs stable + across launches. gdb-verified which gate was the wall. +- **LSX / Origin layer** — crypto handshake reversed byte-exact and confirmed against the repack's + own emu disassembly (`docs/REPACK_INTEL.md`); Origin login gates cleared. +- **Blaze redirector + Fire2/Heat2** — both hops defeated; preAuth/login/personas answered. +- **UTAS/RS4 FUT API** — `ut/auth` + boot calls + device-trust reach the hub with hand-authored JSON. +- **Persistent FIFA 17 account selection** — the launcher synchronizes one configured EA persona to + the Python backend before starting LSX/FIFA. LSX, Blaze, POW/EASFC, and UTAS then share that + identity, while FUT coins, inventory, squads, progression, and unopened packs persist in an + isolated save beneath `fifa17-recon/docker/state/accounts//`. The POW level/XP/funds + shown in FIFA's general account bar are account-scoped but remain distinct from FUT club coins. + A reversible server test on 2026-08-09 verified profile switching, POW values, a 400-coin pack + debit, five awarded items, and restoration of the original profile. +- **Account-scoped FUT security compatibility** — launcher account synchronization initializes a + persisted `securityQuestion` verification record in that persona's FIFA 17 profile. The UTAS + PHISHING handler returns the complete CardsDLL trusted-console response (`changed`, `exists`, + `locked`, `trusted`), accepts only well-formed legacy setup/validate requests under `X-UT-SID`, + and never stores or logs the client-transformed answer. This is server-side emulation; the hook + and launcher do not contain an answer or add UI automation. Automated contract coverage is in + `fifa17-recon/docker/ctx/tools/test_security_question.py`; live first/repeat-launch acceptance is + partially complete: the first launch entered FUT without a security dialog on 2026-08-09; a + second fresh-process FUT entry is still required to close persistence acceptance. +- **Safe responder diagnostics** — ordinary LSX logs redact challenge/session/auth-code attributes; + ordinary Blaze logs redact auth/session keys and no longer emit raw Fire2 hex, decoded TDF, or + config values. Forensic Blaze capture remains available only with the explicit + `OPENFUT_BLAZE_DUMP_FRAMES=1` opt-in. LSX and Blaze self-tests cover the new defaults. +- **OpenFUT Core** — Rust FUT economy backend, feature-complete for its scope and tested: profiles, + clubs, coins, packs, cards, squads, chemistry styles, SBCs, objectives, matches, market (NPC), + draft, FUT Champs, seasons, statistics, achievements, events, daily check-in, division + leaderboard, market trade history. 13 migrations. Integration suite (`tests/integration_test.rs`, + 96 test fns) runs against in-memory SQLite; CI (fmt/clippy/build/test) green on `openfut-core`. + +## Partially implemented + +- **FIFA 17 FUT hub depth** — reaching the hub is proven, but how much of FUT is fully navigable + beyond it (playing matches, pack opening, SBC submission through the *game* UI vs. spinner/error + states) is not documented as complete. The runbook's gate ladder lists failure modes still + guarded against. Treat "past the hub" as unverified. +- **Pack opening through the game UI** — proven live on 2026-08-09 with the recovered CAGE test + profile: purchase, reveal, item assignment/quick-sell, wallet refresh, and return from the reveal + all completed. The Python transaction path also passes its 446-check contract suite. FIFA's + hardcoded post-reveal `mypacks` return is supported by a short-lived active grace record for every + opened pack; it is excluded from unopened-pack counts and retired at the next hub request. +- **FIFA 17 FUT match lifecycle** — CardsDLL static analysis and isolated responder tests now cover + CREATE→READY→PLAY→END. Bare `/match` requests carrying body `matchId` are classified as PLAY + instead of accidentally allocating another match; READY returns the verified scalar `matchId` + and `opponentPersonaId` fields; END persists W/D/L, matches played, and coin rewards per account. + The implementation is deployed and `test_match_lifecycle.py` passes, but no football match has + started or completed in FIFA yet. The READY opponent `items` contract and client mode-entry gate + remain unresolved; `FUT_MODES` therefore stays off by default. +- **Core ↔ emulation integration** — the two halves exist and wiring has **started**. First + slice (2026-08-11): the My Squad owned-player search. `openfut-core` gained a semantic, + game-independent owned-inventory query (`services::inventory::{OwnedItemQuery, apply_query}` + + a `Quality` tier) that filters (AND) → orders deterministically → paginates, wired into + `GET /collection`; `openfut-adapter-fifa17::fut::owned_query` parses the FIFA17 `/club` wire + query and resolves numeric league/nation/team ids → semantic names (unknown id = hard error, + no raw-id passthrough). Intentional fix, not parity: Python applies only `league`+`team` and + ignores `level`/`rare`/`position`/`nation`/`start`/`count` (the request-amplification bug); + Core applies all proven filters and paginates. `rare=SP` semantics UNKNOWN, unimplemented. + Slice 2 (2026-08-11): `openfut-utas-host` — the first live UTAS host. Serves `GET …/club` + from Core through the adapter and reverse-proxies every other UTAS route verbatim to the + Python oracle (`utas_server.py`); plaintext HTTP/1.1 keep-alive, classify-before-execute, + no python-fallback after a Core error. `CoreAccess` is a host-owned boundary (the adapter + stays transport-agnostic). 11 host + 22 adapter tests; 10/10 mutations killed; fmt/clippy + clean. Slice 3 (2026-08-11, `3ef3bc3`): the real `Fifa17IdentityResolver` — catalog + (card id → real asset id) + persistent `openfut-identity` store (owned instance → + stable/reversible wire int) + wire-id policy, replacing all placeholders (one + production path). Wire-id namespace is globally monotonic within `(fifa17, owned-item)`, + not per-account (Core owned ids are UUIDs → unambiguous reverse). Slice 4 (2026-08-11, + core `36abd4b`): a curated 32-card real FIFA17 dev content pack + (`data/games/fifa17/dev/cards.json`, ids `fifa17_`), loaded only via opt-in + `Config.dev_content_games`; `seed-dev` grants a `game_id=fifa17` profile+club real + `OwnedCard`s (no FIFA wire ids — the resolver mints those at request time), idempotent, + default content untouched. **Remaining for a rendering retail `/club`:** Commit 6 — the + host sends `X-OpenFUT-Game: fifa17` so `/club` queries the all-mapped fifa17 profile + (no post-pagination drops), then the live retail A/B (no FIFA client in the build env). + See the vault UTAS Endpoint Map + Known Issues (incl. "identity resolution is NOT + authorization" for later mutations). + +## Stubbed / planned + +- **`fifa-blaze`** (Rust) — Milestone 1 capture stub only. Two TLS listeners that log packets; no + FIFA 23 component/command handlers. Its own README says IDs are unknown. Superseded in practice by + the Python FIFA 17 responders, kept as the intended FIFA 23 implementation surface. +- **`openfut-launcher` legacy controls** — core/bridge and FIFA 23 setup controls belong to a + superseded plan. The launcher now also owns the live FIFA 17 client flow: server/hook config, + account synchronization, local LSX, privileged autopatch, and game launch. +- **`tools/`** (file-watch-diff, squad-injector, exporters) — helpers for the FLE-Lua-bridge idea in + `docs/direction.md`. Not part of the live FIFA 17 path. +- **`docs/foundational-xi-injection-test.md`** — a planned (not executed) test procedure for the FLE + bridge route. + +## Stubbed / blocked (FIFA 23 lineage) + +- **`openfut-bridge`** — in-process `version.dll` hook on ProtoSSL. Git history: injection works but + the effort hit an "architectural wall" (async event-driven gate, not a poll). Superseded first by + the FLE-bridge pivot, then by the FIFA 17 route. Its `CLAUDE.md` task list is historical. + +## Unknown / requires investigation + +- Whether the FIFA 17 hub supports actually **playing a FUT match** offline and getting results back. +- Whether FUT actions beyond the now-verified pack reveal/assignment flow (submit SBC, transfer + market buy/sell, matches) round-trip correctly through `utas_server.py`. +- The exact division of FUT state ownership once Core is wired in (who is source of truth). +- Degree of FIFA 23 wire-format identity — asserted ("identical wire format") but the FIFA 23 client + has not been re-tested against these responders in this repo's evidence. + +## Known technical debt / hazards + +- **Root docs are stale.** `README.md`, `CLAUDE.md`, `openfut-bridge/CLAUDE.md` all describe FIFA 23 + as the target and mark FIFA 23 integration as the open item — they predate the FIFA 17 success. +- **All host state is volatile** across reboot except the `/etc/hosts` line — re-run + `openfut-fut.sh start`. Requires `ptrace_scope=0` + root arming (security-relevant). +- **Whole stack rides on EAAC staying neutralized** and game updates being off; a client update can + break the memory patches (VAs) and cert bypass. +- **`fifa17-recon/tools/lsx_responder_v2.py` is currently modified in the working tree** (uncommitted). +- `33068179` / `CAGE` remains the responder fallback, but the launcher now blocks one-button launch + until an explicit persona is configured and synchronized across LSX, Blaze, POW, and UTAS. diff --git a/openfut-core b/openfut-core index 6acae54..36abd4b 160000 --- a/openfut-core +++ b/openfut-core @@ -1 +1 @@ -Subproject commit 6acae54f80d61526fda2c6efaf4a9c7243659b93 +Subproject commit 36abd4b6fb6830cf7bac5f216cf09b6b7c50c38a diff --git a/scripts/seed_fifa17_cards.py b/scripts/seed_fifa17_cards.py index 869a50a..9547886 100755 --- a/scripts/seed_fifa17_cards.py +++ b/scripts/seed_fifa17_cards.py @@ -118,6 +118,85 @@ def build(recon): definitions.sort(key=lambda d: d["id"]) return catalog, definitions, skipped, len(pool) +# ── Curated development subset ─────────────────────────────────────────────── +# The dev pack is a small, deterministic slice of the full base-card set, chosen +# to exercise the retail /club UI (quality/position/nation/league/team filters + +# pagination) without loading all ~17.5K definitions. Selection is reproducible +# from the sorted source; no asset is hand-picked. +MY_SQUAD_PAGE = 11 # evidence: the client requests count=11 per page +DEV_PRIMARY_GOLD_CAP = 18 # > one page → quality=gold and league= span 2+ pages +DEV_SECONDARY_GOLD = 4 +DEV_SILVER = 6 +DEV_BRONZE = 4 + + +def select_dev(definitions): + """Deterministically curate a dev slice from the full definition set. + + Criteria (all reproducible): the two leagues with the most Gold cards are the + primary/secondary leagues (tie-break by name); take the primary league's top + Gold cards (capped), guaranteeing >1 page for both `quality=gold` and + `league=`; ensure a GK is present; add the secondary league's top + Golds (multiple leagues); add the top Silvers and Bronzes (quality spread). + """ + by_league = {} + for d in definitions: + by_league.setdefault(d["league"], []).append(d) + golds = lambda ds: [d for d in ds if d["overall"] >= 75] + ranked = sorted(by_league, key=lambda L: (-len(golds(by_league[L])), L)) + primary, secondary = ranked[0], ranked[1] + key = lambda d: (-d["overall"], d["id"]) + chosen = {} + for d in sorted(golds(by_league[primary]), key=key)[:DEV_PRIMARY_GOLD_CAP]: + chosen[d["id"]] = d + if not any(d["position"] == "GK" for d in chosen.values()): + gks = sorted((d for d in definitions if d["position"] == "GK" and d["overall"] >= 75), key=key) + if gks: + chosen[gks[0]["id"]] = gks[0] + for d in sorted(golds(by_league[secondary]), key=key)[:DEV_SECONDARY_GOLD]: + chosen[d["id"]] = d + for d in sorted((d for d in definitions if 65 <= d["overall"] < 75), key=key)[:DEV_SILVER]: + chosen[d["id"]] = d + for d in sorted((d for d in definitions if d["overall"] < 65), key=key)[:DEV_BRONZE]: + chosen[d["id"]] = d + return sorted(chosen.values(), key=lambda d: d["id"]), primary + + +def dev_coverage(dev): + """Coverage stats + the self-check gate the dev pack must pass.""" + q = lambda lo, hi=256: sum(1 for d in dev if lo <= d["overall"] < hi) + positions = sorted({d["position"] for d in dev}) + teams = {} + for d in dev: + teams[d["club"]] = teams.get(d["club"], 0) + 1 + cov = { + "count": len(dev), + "gold": q(75), + "silver": q(65, 75), + "bronze": q(0, 65), + "positions": positions, + "has_gk": "GK" in positions, + "gold_st": sum(1 for d in dev if d["position"] == "ST" and d["overall"] >= 75), + "leagues": sorted({d["league"] for d in dev}), + "nations": sorted({d["nation"] for d in dev}), + "max_same_team": max(teams.values()) if teams else 0, + "pages_gold": (q(75) + MY_SQUAD_PAGE - 1) // MY_SQUAD_PAGE, + } + # The pack is only useful if it can exercise the filters + pagination. + checks = { + "gold_over_one_page": cov["gold"] > MY_SQUAD_PAGE, + "has_gk": cov["has_gk"], + "has_st": any(d["position"] == "ST" for d in dev), + "multi_league": len(cov["leagues"]) >= 2, + "multi_nation": len(cov["nations"]) >= 2, + "has_silver": cov["silver"] >= 1, + "has_bronze": cov["bronze"] >= 1, + "same_team_group": cov["max_same_team"] >= 2, + } + cov["checks"] = checks + cov["ok"] = all(checks.values()) + return cov + def dumps(obj): return json.dumps(obj, indent=2, ensure_ascii=False, sort_keys=False) + "\n" @@ -127,13 +206,18 @@ def main(): ap = argparse.ArgumentParser() ap.add_argument("--recon", default=os.path.normpath(RECON)) ap.add_argument("--catalog-out", required=True) - ap.add_argument("--defs-out", help="optional semantic CardDefinition JSON (Core content)") + ap.add_argument("--defs-out", help="optional semantic CardDefinition JSON (full Core content)") + ap.add_argument("--dev-out", help="curated game-scoped dev CardDefinition pack") ap.add_argument("--check", action="store_true", help="fail if committed output would change") args = ap.parse_args() catalog, definitions, skipped, pool_rows = build(args.recon) catalog_bytes = dumps(catalog) defs_bytes = dumps(definitions) + dev, primary = select_dev(definitions) + dev_bytes = dumps(dev) + cov = dev_coverage(dev) + cov["primary_league"] = primary reasons = {} for _, why in skipped: @@ -145,26 +229,40 @@ def main(): "skipped_definitions": len(skipped), "skip_reasons": reasons, "catalog_sha256": hashlib.sha256(catalog_bytes.encode()).hexdigest()[:16], + "dev_coverage": cov, } print(json.dumps(report, indent=2), file=sys.stderr) + # The dev pack is worthless if it cannot exercise the filters + pagination. + if args.dev_out and not cov["ok"]: + print("DEV PACK FAILED COVERAGE GATE: %s" % cov["checks"], file=sys.stderr) + sys.exit(2) + # Every dev definition must have a catalog (FIFA render) identity. + missing = [d["id"] for d in dev if d["id"] not in catalog["cards"]] + if missing: + print("DEV DEFS WITHOUT CATALOG IDENTITY: %s" % missing, file=sys.stderr) + sys.exit(2) + + outputs = [(args.catalog_out, catalog_bytes)] + if args.defs_out: + outputs.append((args.defs_out, defs_bytes)) + if args.dev_out: + outputs.append((args.dev_out, dev_bytes)) + if args.check: drift = False - for path, want in [(args.catalog_out, catalog_bytes)] + ( - [(args.defs_out, defs_bytes)] if args.defs_out else [] - ): + for path, want in outputs: have = open(path, encoding="utf-8").read() if os.path.exists(path) else None if have != want: print("DRIFT: %s is stale (regenerate)" % path, file=sys.stderr) drift = True sys.exit(1 if drift else 0) - with open(args.catalog_out, "w", encoding="utf-8") as f: - f.write(catalog_bytes) - if args.defs_out: - with open(args.defs_out, "w", encoding="utf-8") as f: - f.write(defs_bytes) - print("wrote %s (%d cards)" % (args.catalog_out, len(catalog["cards"])), file=sys.stderr) + for path, data in outputs: + os.makedirs(os.path.dirname(os.path.abspath(path)), exist_ok=True) + with open(path, "w", encoding="utf-8") as f: + f.write(data) + print("wrote catalog=%d dev=%d cards" % (len(catalog["cards"]), len(dev)), file=sys.stderr) if __name__ == "__main__":