funman300/OpenFUT
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fifa17-recon: the envelope rule, and the one key of the auth body that is silently eaten

Browse Source 
The open question was whether a response deserializer DESCENDS a wrapper or PROBES for
one. Three roots spend an identical three tokenizer calls before dispatch, yet we serve
some bodies wrapped and some flat, and not all of those could be right.

TOKEN ENUM, decoded from the class table at DAT_18023dd40 and the switch in the
classifier FUN_1801c67a0 (the push/pop arms key off container state 2 = object,
3 = array):

  9  START_OBJECT      case 0x64, pushes state 2
  10 END_OBJECT        case 0x65, pops state 2
  11 FIELD_NAME        confirmed independently: FUN_18013bd40 tests +0xd0 == 0xb then
                       atom-hashes the string at +0xf8
  12 START_ARRAY       case 0x66, pushes state 3
  13 END_ARRAY         case 0x67, pops state 3
  1  error             the caseD_78 sink

So the three tokens are `{`, the first FIELD_NAME, and the token opening that field's
value. The envelope is structurally required and its name is NEVER hashed, which is why
FutCreatePack's ladder has no arm for createPackResponse (0xbe) and does not need one.
Coverage for that absence: the ladder has exactly four arms (0xec, 0x16e, 0x1dd, 0x264)
and 0xbe does not occur anywhere in the full 4702-char decompile, printed in full.

The competing reading rested on a factual error. It claimed the /purchased root spends
the same three tokens. FUN_180124ee0 spends TWO and hands off to FUN_18013bd40, which
spends the third. Same total, split across two functions. /purchased never was a
counterexample.

THE BUG THIS FOUND IS NOT THE ONE THAT WAS PREDICTED. The doc expected starterPack,
squad and userData to be swallowed on POST /user. They are not. FutCreateUser
(0x18014cc60) has ladder arms for exactly the five keys we send, and four of them
dispatch correctly at the outer level. The one that does not is `login`: its name is
eaten as the anonymous envelope and its value as the third token. It has an arm, so the
client wants it, and it has never once been delivered.

The second-order consequence matters more than the first. The key order of that dict is
load-bearing and nothing said so. Put userData first and the client loses the entire
user record, silently, with no error and no log line. That warning now sits in the code
next to the dict, which is the only place someone about to reorder it would look.

No behaviour change here. The utas_server.py edit is a comment. 439 contract checks
still pass. The probable proper fix, wrapping all five keys one level down inside a
single envelope key, is a hypothesis with a mechanism rather than a proven fix, and it
touches the login path, so it is not made here and would go behind a flag defaulting
off.

Writeup is section 2 of docs/plan-2026-08-05-pack-opening.md, added by the previous
commit. Opened by this and still UNKNOWN: GET /hub is answered with a flat two-key
body, which under this rule a three-token root would silently truncate, but there is no
FutGetHubServerResponse class and neither atom has a code xref, so /hub may not go
through a generated root at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
funman300
2026-08-05 19:25:01 -07:00
parent afdbb364ca
commit 1605e6effd
4 changed files with 267 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
fifa17-recon/tools/ghidra_queries/q_envelope_1.py
+88
View File
@@ -0,0 +1,88 @@
"""Settle the envelope rule: does a response deserializer DESCEND a wrapper or PROBE for one?
HYPOTHESIS UNDER TEST (from docs/plan-2026-08-05-pack-opening.md section 2):
FUN_180162880 (FutCreatePackServerResponse) calls the next-token primitive three times
unconditionally at 0x180162942/94b/954, tests only the third against token 10, and has no
ladder arm for `createPackResponse` (atom 0xbe). Reading A says that is a DESCEND, so the
envelope is structurally required and its name is irrelevant. Reading B says the token count
decides nothing, because FutCreateUser / FutDiscardCard / the /purchased root all spend the
same three tokens, and we serve POST /user and GET /purchased UNWRAPPED and both work.
Both cannot be right. The tokenizer decides it, so read the tokenizer.
CONTROLS, and why each one is here:
* 0x180124ee0 the /purchased root. We serve it UNWRAPPED and it demonstrably works, so
whatever the three tokens mean, its behaviour must be consistent with an unwrapped body.
This is the control that can actually refute reading A.
* 0x18014cc60 FutCreateUser. Same, served unwrapped, works.
* 0x180162880 FutCreatePack itself, the one nothing has ever parsed.
Printing all three side by side is the point: a claim about "three tokens" is only meaningful
if the same three tokens behave differently in a case we know works.
COVERAGE DISCIPLINE: this file prints every decompile IN FULL and prints len() first. No
absence claim may be made from anything below unless the printed length matches the claim.
"""
import traceback
TOKENIZER = 0x1801C7F10 # next-token primitive
BEGINOBJ = 0x1801C8270 # begin-object primitive
INTGET = 0x1801C79D0 # int getter, for token-value cross-reference
DESERS = [
(0x180162880, "FutCreatePackServerResponse (WRAPPED per reading A, never parsed live)"),
(0x180124EE0, "FutGetPurchasedItems root (CONTROL: served UNWRAPPED, works live)"),
(0x18014CC60, "FutCreateUser (CONTROL: served UNWRAPPED, works live)"),
]
def dump(va, title):
try:
f = func(va)
src = dec(va)
n = f.getBody().getNumAddresses() if f else -1
print("\n" + "=" * 78)
print("%#x %s" % (va, title))
print("body %d bytes / decompile %d chars (PRINTED IN FULL)" % (n, len(src)))
print("=" * 78)
print(src)
except Exception:
print("!! failed on %#x" % va)
traceback.print_exc()
try:
# 1. The two primitives. Everything else is interpretation of these.
dump(TOKENIZER, "next-token primitive -- what IS a token, and what is token 10?")
dump(BEGINOBJ, "begin-object primitive")
# 2. The three deserializers, so the token triples can be compared directly.
for va, name in DESERS:
dump(va, name)
# 3. Who else calls the tokenizer, and how many times each. A function that calls it
# exactly 3 times unconditionally is the pattern under test; the distribution tells
# us whether 3 is special or merely common.
print("\n" + "=" * 78)
print("CALLERS OF THE TOKENIZER %#x, with call count per caller" % TOKENIZER)
print("=" * 78)
counts = {}
for frm, typ, fn, ent in xrefs_to(TOKENIZER):
if not ent:
continue
counts.setdefault((ent, fn), []).append(frm)
for (ent, fn), sites in sorted(counts.items(), key=lambda kv: -len(kv[1])):
print(" %#x %-34s %2d calls %s"
% (ent, fn, len(sites), " ".join("%#x" % s for s in sorted(sites))))
print("\ntotal distinct callers: %d" % len(counts))
hist = {}
for k, v in counts.items():
hist[len(v)] = hist.get(len(v), 0) + 1
print("call-count histogram (calls -> how many functions):",
dict(sorted(hist.items())))
# 4. The begin-object primitive's callers, for the same reason.
print("\ncallers of begin-object %#x: %d" % (BEGINOBJ, len(set(
e for _, _, _, e in xrefs_to(BEGINOBJ) if e))))
except Exception:
traceback.print_exc()
fifa17-recon/tools/ghidra_queries/q_envelope_2.py
+91
View File
@@ -0,0 +1,91 @@
"""Pin the token enum, so the three-token pattern can be read rather than guessed.
Query 1 established the shapes but not the vocabulary:
FUN_180162880 FutCreatePack 3 tokenizer calls, then a key loop, exits on token 10
FUN_18014cc60 FutCreateUser 3 tokenizer calls, then a key loop
FUN_180124ee0 /purchased root TWO tokenizer calls, then FUN_18013bd40, no loop at all
That two-versus-three difference is the whole answer, but only if we know what a token IS.
Note it already refutes the claim in plan-2026-08-05-pack-opening.md section 2 that the
/purchased root "spends the same three tokens". It spends two.
FUN_1801c7f10 is only a pushback wrapper: it returns param_1[0x35] if a token was pushed
back, else classifies one byte via FUN_1801c67a0. So FUN_1801c67a0 holds the enum.
Known so far, from usage rather than from the enum: 10 ends the key loop, 0xd ends an array
(`while (iVar7 != 0xd)` around the itemList element parser), 8 is set when the input is
exhausted cleanly, 1 on error, 7 on a first-call special case.
WHAT WOULD FALSIFY THE DESCEND READING: if token 2 in the /purchased root is a
START_ARRAY rather than a key or a START_OBJECT, then `itemData` is not being consumed as a
wrapper and the two-call pattern means something else entirely.
"""
import traceback
TARGETS = [
(0x1801C67A0, "token CLASSIFIER -- the enum lives here"),
(0x18013BD40, "/purchased body sub-parser (what the 2-token root hands off to)"),
(0x180141EE0, "key reader used by the CreatePack/CreateUser loops"),
(0x1801C63E0, "parser init (called before begin-object in every root)"),
]
def dump(va, title):
try:
f = func(va)
src = dec(va)
n = f.getBody().getNumAddresses() if f else -1
print("\n" + "=" * 78)
print("%#x %s" % (va, title))
print("body %d bytes / decompile %d chars (PRINTED IN FULL)" % (n, len(src)))
print("=" * 78)
print(src)
except Exception:
print("!! failed on %#x" % va)
traceback.print_exc()
try:
for va, name in TARGETS:
dump(va, name)
# Cross-check the two-versus-three count mechanically over every response root we can
# name, rather than trusting three hand-picked examples. A root is recognised by calling
# the parser-init, begin-object and the tokenizer.
print("\n" + "=" * 78)
print("TOKEN CALLS BEFORE THE FIRST KEY READ, across all begin-object callers")
print("=" * 78)
roots = sorted({e for _, _, _, e in xrefs_to(0x1801C8270) if e})
print("begin-object callers: %d" % len(roots))
for ent in roots:
try:
f = func(ent)
if f is None:
continue
# order the call sites by address and count tokenizer calls that precede the
# first key-reader call, which is what "descend depth" actually means here
toks, keys, begin = [], [], []
for ad in f.getBody().getAddresses(True):
ins = listing.getInstructionAt(ad)
if ins is None:
continue
for r in ins.getReferencesFrom():
t = int(r.getToAddress().getOffset())
a = int(ad.getOffset())
if t == 0x1801C7F10:
toks.append(a)
elif t in (0x180141EE0,):
keys.append(a)
elif t == 0x1801C8270:
begin.append(a)
if not begin:
continue
b = min(begin)
first_key = min(keys) if keys else None
pre = [a for a in toks if a > b and (first_key is None or a < first_key)]
print(" %#x %-22s begin@%#x tokens_before_first_key=%d keyreads=%d"
% (ent, f.getName(), b, len(pre), len(keys)))
except Exception:
print(" %#x <failed>" % ent)
traceback.print_exc()
except Exception:
traceback.print_exc()
fifa17-recon/tools/ghidra_queries/q_envelope_3.py
+74
View File
@@ -0,0 +1,74 @@
"""Decode the token enum from the character-class table, and settle the /hub problem.
WHERE QUERY 2 LEFT IT. 67 of 86 response roots spend exactly 3 tokenizer calls before their
first key read. FUN_18013bd40 gives one anchor for free: it tests `*(int *)(param_1 + 0xd0)
== 0xb` and then atom-hashes the string at param_1+0xf8, so +0xd0 is the CURRENT TOKEN TYPE
and 0xb is FIELD_NAME. Usage gives two more: 10 exits a key loop, 0xd exits an array loop.
THE PROBLEM THAT STOPS ME CONCLUDING. If the three tokens were `{`, wrapper-key, open-bracket,
then every body would need exactly one top-level key. But we serve
GET /hub -> {"clubPlayers": 205, "auctionCount": 0}
with TWO top-level keys, and it is live-proven working (utas.log, 14:39:28). So either /hub's
root is one of the non-3 outliers, or the third token is a VALUE and the first key/value pair
is consumed without being dispatched. Those two readings imply very different rules for what
we may serve, so this is not a detail.
METHOD. Read the class table at DAT_18023dd40 for the structural characters, then walk the
switch in FUN_1801c67a0 to map class -> returned token. Then look up /hub's own root and
count its tokens, which discriminates the two readings directly.
CONTROL: '"' (0x22) must classify differently from '{' (0x7b), and a digit must differ from
both. If all three come back identical the table has been misread and nothing below stands.
"""
import traceback
CLASS_TABLE = 0x18023DD40
CHARS = [
(0x7B, "{ open-object"), (0x7D, "} close-object"),
(0x5B, "[ open-array"), (0x5D, "] close-array"),
(0x22, '" quote'), (0x3A, ": colon"), (0x2C, ", comma"),
(0x30, "0 digit (control)"), (0x41, "A letter (control)"),
(0x20, "space (control)"), (0x74, "t of true"), (0x6E, "n of null"),
]
try:
print("=" * 78)
print("CHARACTER CLASS TABLE at %#x" % CLASS_TABLE)
print("=" * 78)
tbl = read_bytes(CLASS_TABLE, 0x80)
print("read %d bytes" % len(tbl))
for c, label in CHARS:
print(" %-22s 0x%02x -> class %#x" % (label, c, tbl[c] if c < len(tbl) else -1))
distinct = {tbl[c] for c, _ in CHARS if c < len(tbl)}
print("\nCONTROL: %d distinct classes across the sampled characters. "
"If this is 1 the table is misread." % len(distinct))
# Full table, so nothing is hidden by the sample above.
print("\nfull 0x00-0x7f class map (index: class), nonzero only:")
print(" " + " ".join("%02x:%x" % (i, b) for i, b in enumerate(tbl) if b))
# The switch that turns a class into a token.
print("\n" + "=" * 78)
print("FUN_1801c67a0 classifier, IN FULL (the class -> token switch)")
print("=" * 78)
src = dec(0x1801C67A0)
print("decompile %d chars" % len(src))
print(src)
# /hub. Find its root and count tokens, which discriminates the two readings.
print("\n" + "=" * 78)
print("THE /hub DISCRIMINATOR")
print("=" * 78)
for lit in (b"RS4:FutGetHubServerResponse", b"clubPlayers", b"auctionCount"):
hits = find_all(lit)
print("\n%-32s %d hit(s): %s" % (lit.decode(), len(hits),
" ".join("%#x" % h for h in hits)))
for h in hits:
for frm, typ, fn, ent in xrefs_to(h):
print(" xref %#x in %s (entry %#x)" % (frm, fn, ent))
for frm, typ, fn, ent in xrefs_to(h - 4):
print(" xref-4 %#x in %s (entry %#x)" % (frm, fn, ent))
except Exception:
traceback.print_exc()
fifa17-recon/tools/utas_server.py
+14
View File
@@ -296,6 +296,20 @@ def user_post(h=None):
# serve the real schema-correct squad rather than {} -- a create-club response
# carrying an empty squad leaves the client with a 0-slot squad model, which is
# precisely the state that makes AddPlayerToSquad no-op (see REBUILD_PLAN S9c).
#
# !! KEY ORDER IS LOAD-BEARING. DO NOT REORDER THIS DICT. !!
# CreateUser is a three-token root: the parser consumes `{`, the FIRST field name,
# and the token that opens that field's value, all three WITHOUT dispatching them,
# and only then starts its key ladder. So whichever key is listed first here is
# silently discarded. Today that is `login`, which costs us nothing visible.
# Put `userData` first and the client loses the entire user record, with no error
# and no log line anywhere. Established 2026-08-05 by decoding the token enum in
# FUN_1801c67a0 (9=START_OBJECT 10=END_OBJECT 11=FIELD_NAME 12=START_ARRAY
# 13=END_ARRAY); see docs/plan-2026-08-05-pack-opening.md section 2 and
# tools/ghidra_queries/q_envelope_{1,2,3}.py.
# The probable proper fix is to wrap all five keys one level down inside a single
# envelope key, whose NAME the parser never checks. Untested, and it touches the
# login path, so it is not done here.
return {"login": True, "userData": user_info(),
"squad": current_squad(), "starterPack": {}, "bonusPacks": []}