fifa17-recon: /hub refutes yesterday's envelope conclusion, and two ENDPOINT_MAP freezes
Three things: the envelope rule was wrong and is corrected, /hub is settled, and two
documented response shapes that would freeze the client are fixed.
THE CORRECTION. The previous commit concluded that a three-token root consumes `{`, the
first field name and that field's value without dispatching them, so the first key of a
flat body was silently eaten, and that `login` had therefore never been delivered on
POST /user. That is WRONG and is withdrawn, along with the claim that the key order of
the auth dict is load-bearing.
The first call to FUN_1801c7f10 returns token 7 and consumes NO input. It is a
once-only start-of-document token, guarded by the flag at parser+0xda together with the
zero character counter at parser+0x30. So the three tokens are BOF, `{`, and the FIRST
FIELD NAME, and the key loop dispatches from that first key onward. The `== 10` test on
the third token is not an envelope check, it is the empty-object early-out: for `{}` the
third token is END_OBJECT and the root exits with its constructor defaults intact, which
is why answering `{}` has always been safe.
Corrected enum: 7=BOF 9=START_OBJECT 10=END_OBJECT 11=FIELD_NAME 12=START_ARRAY
13=END_ARRAY. The enum itself was right before; the inference from it was not.
HOW IT WAS CAUGHT, which is the part worth keeping. Not by more decompiling. /hub is
served flat and the wrong model predicted its first key would be discarded, so the
prediction was checked against the client's own memory: clubPlayers read back as 205,
the value the server sent, at model+0x1fd70+0x3c with the slide proven against the FNV
prologue first. One live read refuted a chain of otherwise sound static reasoning in
about a minute. tools/hub_counter_probe.py keeps it repeatable.
Consequence worth flagging: a wrapper is not just unnecessary for these roots, it would
be harmful, since a wrapper key hashes to an atom with no arm and the whole object is
skipped. That makes the createPackResponse envelope DOUBTFUL rather than confirmed.
Atom 0xbe has no arm in FUN_180162880. There is no live evidence either way because
nothing has ever parsed that body, so the buy path is left exactly as it is.
TWO ERRORS OF MINE ON THE WAY, both recorded in the doc because both are cheap to
repeat. I searched for RS4:FutGetHubServerResponse, found nothing and reported that no
hub class existed; the class is FutGetHubDataServerResponse (literal 0x18022ce40,
vtable 0x18022cd48, deser 0x1801738b0, control FutSquadSave -> 0x180171a60 matched in
the same run). Then I scanned 152 deserializers for clubPlayers, got zero hits and a
passing control, because the guard is `!= 0x90` and my pattern only matched `== 0x`.
The control passed only because auctionCount happens to use `==`. A control that does
not exercise the same code shape as the target is not a control. The comment already at
utas_server.py:1076 had the hub chain right the whole time.
ENDPOINT_MAP corrections, both freeze-risky as written, neither affecting what we serve
today:
* duplicateItemIdList is an ARRAY OF OBJECTS (element deser 0x180138e10), not the int
list at :1095. Bare ints where the element parser expects objects is a tokenizer
desync, i.e. a hard freeze at 0x1801c7f1a. Control that this is not a misread:
dreamSquads 0xe9 in FutMoveCard genuinely is a bare int array.
* FutDiscardCardServerResponse is {"items":[{"id":N}],"totalCredits":N}. There is no
top-level id.
No behaviour change. utas_server.py is comment-only. 439 contract checks pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
funman300
@@ -0,0 +1,120 @@
|
||||
"""Does GET /hub go through a three-token root, and is clubPlayers being silently eaten?
|
||||
|
||||
THE WORRY. The envelope rule (q_envelope_{1,2,3}.py, and section 2 of
|
||||
docs/plan-2026-08-05-pack-opening.md) established that a three-token root consumes
|
||||
`{`, the FIRST field name, and the token opening that field's value, all WITHOUT
|
||||
dispatching them. 67 of 86 roots are three-token. We answer GET /hub with a flat
|
||||
two-key body:
|
||||
{"clubPlayers": 205, "auctionCount": 0}
|
||||
If /hub is a three-token root then `clubPlayers` (atom 0x90) is being discarded on
|
||||
every hub load and only `auctionCount` (atom 0x33) is read.
|
||||
|
||||
WHY IT IS NOT ALREADY SETTLED. There is no RS4:FutGetHubServerResponse literal in the
|
||||
image, and both atoms appear only as atom-table entries with no code xref, so /hub may
|
||||
not be parsed by a generated root at all. Absence of the class literal is NOT evidence
|
||||
that no deserializer exists: class_deser() is known to produce false negatives, and the
|
||||
project rule is that an empty result means UNKNOWN.
|
||||
|
||||
METHOD, three independent angles so no single failure decides it:
|
||||
A. the UTAS route template array at 0x18021df80..0x18021e278, resolved to strings,
|
||||
which is the authoritative list of paths the client can request
|
||||
B. a direct hunt for whichever deserializer ladder tests atom 0x90 or 0x33, by
|
||||
decompiling every begin-object caller and grepping. This is the one that actually
|
||||
answers the question, because it finds the consumer regardless of class naming.
|
||||
C. token count for whatever B turns up, using the same rule as q_envelope_2
|
||||
|
||||
CONTROL: the same scan must find atom 0x16e (itemList) in FUN_180162880, which we know
|
||||
it is in. If the scan cannot find a known-present atom, its negative results are void.
|
||||
"""
|
||||
import traceback
|
||||
|
||||
ROUTE_TABLE = (0x18021DF80, 0x18021E278)
|
||||
A_CLUBPLAYERS = 0x90
|
||||
A_AUCTIONCOUNT = 0x33
|
||||
CONTROL_ATOM = 0x16E # itemList, known present in 0x180162880
|
||||
CONTROL_FUNC = 0x180162880
|
||||
|
||||
try:
|
||||
# ---- A. the route table -------------------------------------------------
|
||||
print("=" * 78)
|
||||
print("A. UTAS ROUTE TEMPLATE ARRAY %#x..%#x" % ROUTE_TABLE)
|
||||
print("=" * 78)
|
||||
lo, hi = ROUTE_TABLE
|
||||
n = 0
|
||||
for va in range(lo, hi, 16):
|
||||
try:
|
||||
p0, p1 = qword(va), qword(va + 8)
|
||||
s0 = rd_str(p0) if 0x180000000 <= p0 < 0x181000000 else ""
|
||||
if not s0:
|
||||
continue
|
||||
n += 1
|
||||
print(" %#x %-46r tok=%#x" % (va, s0, p1 & 0xFFFFFFFF))
|
||||
except Exception:
|
||||
pass
|
||||
print("resolved %d entries" % n)
|
||||
|
||||
print("\nliteral 'hub' occurrences in the image:")
|
||||
for h in find_all(b"hub\x00"):
|
||||
print(" %#x %r" % (h, rd_str(h - 24, 60)))
|
||||
|
||||
# ---- B. who tests these atoms ------------------------------------------
|
||||
print("\n" + "=" * 78)
|
||||
print("B. WHICH DESERIALIZER LADDER TESTS clubPlayers %#x / auctionCount %#x"
|
||||
% (A_CLUBPLAYERS, A_AUCTIONCOUNT))
|
||||
print("=" * 78)
|
||||
roots = sorted({e for _, _, _, e in xrefs_to(0x1801C8270) if e})
|
||||
# include sub-parsers too: a root may delegate, as /purchased does
|
||||
subs = sorted({e for _, _, _, e in xrefs_to(0x180135FF0) if e})
|
||||
cands = sorted(set(roots) | set(subs))
|
||||
print("scanning %d functions (%d begin-object callers, %d skip-handler callers)"
|
||||
% (len(cands), len(roots), len(subs)))
|
||||
|
||||
pats = {
|
||||
"clubPlayers": "0x%x" % A_CLUBPLAYERS,
|
||||
"auctionCount": "0x%x" % A_AUCTIONCOUNT,
|
||||
"CONTROL itemList": "0x%x" % CONTROL_ATOM,
|
||||
}
|
||||
hits = {k: [] for k in pats}
|
||||
scanned = 0
|
||||
for ent in cands:
|
||||
try:
|
||||
src = dec(ent)
|
||||
except Exception:
|
||||
continue
|
||||
scanned += 1
|
||||
for label, lit in pats.items():
|
||||
# match a comparison against the atom, not any incidental use of the number
|
||||
if ("== " + lit) in src or ("," + lit + ")") in src:
|
||||
hits[label].append(ent)
|
||||
print("scanned %d decompiles\n" % scanned)
|
||||
for label in ("CONTROL itemList", "clubPlayers", "auctionCount"):
|
||||
v = hits[label]
|
||||
print(" %-18s %d hit(s): %s" % (label, len(v), " ".join("%#x" % a for a in v[:12])))
|
||||
ok = CONTROL_FUNC in hits["CONTROL itemList"]
|
||||
print("\n CONTROL: itemList found in %#x ? %s" % (CONTROL_FUNC, "YES" if ok else "NO"))
|
||||
if not ok:
|
||||
print(" !! CONTROL FAILED. Every negative result above is VOID.")
|
||||
|
||||
# ---- C. token count for any consumer found ------------------------------
|
||||
print("\n" + "=" * 78)
|
||||
print("C. TOKEN COUNT for any function that consumes either hub atom")
|
||||
print("=" * 78)
|
||||
for ent in sorted(set(hits["clubPlayers"]) | set(hits["auctionCount"])):
|
||||
f = func(ent)
|
||||
toks, begin = [], []
|
||||
for ad in f.getBody().getAddresses(True):
|
||||
ins = listing.getInstructionAt(ad)
|
||||
if ins is None:
|
||||
continue
|
||||
for r in ins.getReferencesFrom():
|
||||
t = int(r.getToAddress().getOffset())
|
||||
if t == 0x1801C7F10:
|
||||
toks.append(int(ad.getOffset()))
|
||||
elif t == 0x1801C8270:
|
||||
begin.append(int(ad.getOffset()))
|
||||
print(" %#x %-22s begin-object=%s tokenizer calls=%d"
|
||||
% (ent, f.getName(), ("%#x" % min(begin)) if begin else "NONE", len(toks)))
|
||||
print(" callers: %s" % " ".join("%#x" % a for a, _ in callers(ent)[:8]))
|
||||
|
||||
except Exception:
|
||||
traceback.print_exc()
|
||||
@@ -0,0 +1,113 @@
|
||||
"""The hub response class exists after all. Resolve it and settle the clubPlayers question.
|
||||
|
||||
q_hub_1.py searched for RS4:FutGetHubServerResponse and found nothing, and the
|
||||
pack-opening doc recorded that as "there is no hub class". Wrong search, not absent
|
||||
class: the literal scan turned up `tHubDataServerResponse` at 0x18022ce61, i.e. the
|
||||
class is FutGetHubData..., not FutGetHub.... This is the same absence-from-a-bad-search
|
||||
trap the project has hit before, so this file re-derives it from the literal outward.
|
||||
|
||||
q_hub_1 also found ZERO direct `== 0x90` comparisons for clubPlayers across 152
|
||||
deserializers, with a passing control (itemList in 0x180162880). That is suggestive but
|
||||
it is NOT a proven absence, because dispatch here is frequently a running-sum sub/dec
|
||||
ladder, which a source grep for `== 0x90` cannot see. Settle it by reading the actual
|
||||
hub deserializer instead of by scanning.
|
||||
|
||||
QUESTIONS
|
||||
Q1 exact class literal, its vtable, its deserializer (RS4 rule: the literal is
|
||||
"RS4:<Name>", so search the full string and take vtable slot +0x08)
|
||||
Q2 how many tokens the deserializer spends before its first key read, which decides
|
||||
whether the flat two-key body we serve loses its first pair
|
||||
Q3 which atoms its ladder actually handles, read from the decompile IN FULL rather
|
||||
than grepped, so clubPlayers 0x90 and auctionCount 0x33 get a real answer
|
||||
|
||||
CONTROL: resolve FutSquadSave the same way in the same run. Its deserializer is known
|
||||
to be 0x180171a60. If that does not come back correct, nothing else here is trustworthy.
|
||||
"""
|
||||
import traceback
|
||||
|
||||
CANDIDATES = [b"RS4:FutGetHubDataServerResponse", b"RS4:FutSquadSaveServerResponse"]
|
||||
KNOWN = {"RS4:FutSquadSaveServerResponse": 0x180171A60}
|
||||
|
||||
|
||||
def resolve(lit):
|
||||
"""literal -> [(deser, vtable, factory)] via the RS4 rule."""
|
||||
out = []
|
||||
for a in find_all(lit):
|
||||
print(" literal at %#x: %r" % (a, rd_str(a)))
|
||||
for frm, typ, fn, ent in xrefs_to(a):
|
||||
if not ent:
|
||||
continue
|
||||
f = func(ent)
|
||||
if f is None:
|
||||
continue
|
||||
for ad in f.getBody().getAddresses(True):
|
||||
ins = listing.getInstructionAt(ad)
|
||||
if ins is None:
|
||||
continue
|
||||
for r in ins.getReferencesFrom():
|
||||
t = int(r.getToAddress().getOffset())
|
||||
if not (0x1801E5000 <= t <= 0x1802891FF):
|
||||
continue
|
||||
try:
|
||||
v0, v1 = qword(t), qword(t + 8)
|
||||
except Exception:
|
||||
continue
|
||||
if fm.getFunctionAt(addr(v0)) and fm.getFunctionAt(addr(v1)):
|
||||
out.append((v1, t, ent))
|
||||
return out
|
||||
|
||||
|
||||
try:
|
||||
found = {}
|
||||
for lit in CANDIDATES:
|
||||
name = lit.decode()
|
||||
print("=" * 78)
|
||||
print(name)
|
||||
print("=" * 78)
|
||||
res = resolve(lit)
|
||||
uniq = sorted({d for d, _, _ in res})
|
||||
for d, vt, ent in res:
|
||||
print(" deser %#x vtable %#x factory %#x" % (d, vt, ent))
|
||||
print(" -> distinct deserializers: %s"
|
||||
% (" ".join("%#x" % d for d in uniq) or "NONE"))
|
||||
found[name] = uniq
|
||||
if name in KNOWN:
|
||||
exp = KNOWN[name]
|
||||
print(" CONTROL: expected %#x, %s"
|
||||
% (exp, "MATCH" if exp in uniq else "*** MISMATCH, results are void ***"))
|
||||
print()
|
||||
|
||||
for d in found.get("RS4:FutGetHubDataServerResponse", []):
|
||||
src = dec(d)
|
||||
f = func(d)
|
||||
print("=" * 78)
|
||||
print("HUB DESERIALIZER %#x body %d bytes / decompile %d chars (IN FULL)"
|
||||
% (d, f.getBody().getNumAddresses() if f else -1, len(src)))
|
||||
print("=" * 78)
|
||||
print(src)
|
||||
|
||||
toks, begin, keys = [], [], []
|
||||
for ad in f.getBody().getAddresses(True):
|
||||
ins = listing.getInstructionAt(ad)
|
||||
if ins is None:
|
||||
continue
|
||||
for r in ins.getReferencesFrom():
|
||||
t = int(r.getToAddress().getOffset())
|
||||
a = int(ad.getOffset())
|
||||
if t == 0x1801C7F10:
|
||||
toks.append(a)
|
||||
elif t == 0x1801C8270:
|
||||
begin.append(a)
|
||||
elif t == 0x180141EE0:
|
||||
keys.append(a)
|
||||
print("\n tokenizer calls : %d at %s" % (len(toks), " ".join("%#x" % a for a in toks)))
|
||||
print(" begin-object : %s" % (" ".join("%#x" % a for a in begin) or "NONE"))
|
||||
print(" key-reader calls: %s" % (" ".join("%#x" % a for a in keys) or "NONE"))
|
||||
if begin and keys:
|
||||
b, k = min(begin), min(keys)
|
||||
print(" tokens before first key read: %d"
|
||||
% len([a for a in toks if b < a < k]))
|
||||
print("\n callers: %s" % " ".join("%#x %s" % (a, n) for a, n in callers(d)[:10]))
|
||||
|
||||
except Exception:
|
||||
traceback.print_exc()
|
||||
@@ -0,0 +1,102 @@
|
||||
"""What does the hub body actually contain? Read the sub-parser the hub root delegates to.
|
||||
|
||||
CHAIN ESTABLISHED SO FAR:
|
||||
FutGetHubDataServerResponse literal 0x18022ce40 -> vtable 0x18022cd48 -> deser
|
||||
0x1801738b0 (control FutSquadSave -> 0x180171a60 MATCHED in the same run).
|
||||
0x1801738b0 spends TWO tokenizer calls and then tail-delegates to FUN_180139610.
|
||||
That is the /purchased shape: root spends 2, the sub-parser spends the 3rd and owns
|
||||
the key loop.
|
||||
|
||||
We answer GET /hub with a FLAT two-key body {"clubPlayers":205,"auctionCount":0}.
|
||||
q_hub_1 scanned 152 deserializers for a direct comparison against clubPlayers (0x90)
|
||||
and found ZERO, with a passing control. auctionCount (0x33) got 7 hits, one of which
|
||||
is FUN_180139610 itself.
|
||||
|
||||
WHY THAT IS NOT YET AN ANSWER. A source grep for `== 0x90` cannot see running-sum
|
||||
sub/dec ladder dispatch, which is common in this binary, so zero hits is suggestive and
|
||||
not conclusive. Read the ladder instead of grepping it.
|
||||
|
||||
QUESTIONS
|
||||
Q1 print FUN_180139610 IN FULL and enumerate every atom its ladder handles, including
|
||||
any expressed as a running-sum sub/dec chain rather than an equality test
|
||||
Q2 is clubPlayers 0x90 among them, in ANY dispatch form
|
||||
Q3 does it consume a token before its loop (making the total 3, and therefore eating
|
||||
the first key/value pair of a flat body) or does it start looping immediately
|
||||
Q4 FUN_1801c8210(parser, 3, 1) is called by the hub root and not by the other roots
|
||||
read so far. Find out what it configures, since it may change the token semantics
|
||||
|
||||
CONTROL: FUN_180139610 must show auctionCount 0x33 somewhere, since the q_hub_1 grep
|
||||
already found it there. If the atom enumeration below cannot see 0x33, the enumeration
|
||||
is broken and its verdict on 0x90 is void.
|
||||
"""
|
||||
import traceback
|
||||
|
||||
SUB = 0x180139610
|
||||
CFG = 0x1801C8210
|
||||
A_CLUB = 0x90
|
||||
A_AUCTION = 0x33
|
||||
|
||||
try:
|
||||
for va, title in ((SUB, "hub body sub-parser"), (CFG, "parser config called by the hub root")):
|
||||
f = func(va)
|
||||
src = dec(va)
|
||||
print("=" * 78)
|
||||
print("%#x %s body %d bytes / decompile %d chars (IN FULL)"
|
||||
% (va, title, f.getBody().getNumAddresses() if f else -1, len(src)))
|
||||
print("=" * 78)
|
||||
print(src)
|
||||
print()
|
||||
|
||||
# Enumerate atoms mechanically from the instruction stream, which unlike a source
|
||||
# grep also catches sub/dec ladder steps.
|
||||
print("=" * 78)
|
||||
print("ATOMS REACHABLE IN %#x, read from the instruction stream" % SUB)
|
||||
print("=" * 78)
|
||||
f = func(SUB)
|
||||
imms, runsum = [], 0
|
||||
for ad in f.getBody().getAddresses(True):
|
||||
ins = listing.getInstructionAt(ad)
|
||||
if ins is None:
|
||||
continue
|
||||
m = ins.getMnemonicString().lower()
|
||||
txt = str(ins)
|
||||
if m in ("cmp", "sub", "dec", "add", "mov"):
|
||||
for i in range(ins.getNumOperands()):
|
||||
for o in ins.getOpObjects(i):
|
||||
try:
|
||||
v = int(o.getValue())
|
||||
except Exception:
|
||||
continue
|
||||
if 0 < v <= 0x400:
|
||||
imms.append((int(ad.getOffset()), m, v, txt))
|
||||
print(" %d candidate immediates in range 1..0x400" % len(imms))
|
||||
# running-sum reconstruction: consecutive sub/dec on the same register accumulate
|
||||
acc = 0
|
||||
print("\n addr mnem imm running-sum disasm")
|
||||
for a, m, v, txt in imms:
|
||||
if m in ("sub", "dec"):
|
||||
acc += v
|
||||
print(" %#010x %-4s %#-6x %#-11x %s" % (a, m, v, acc, txt))
|
||||
else:
|
||||
print(" %#010x %-4s %#-6x %-11s %s" % (a, m, v, "", txt))
|
||||
|
||||
seen = {v for _, m, v, _ in imms}
|
||||
sums = set()
|
||||
acc = 0
|
||||
for _, m, v, _ in imms:
|
||||
if m in ("sub", "dec"):
|
||||
acc += v
|
||||
sums.add(acc)
|
||||
print("\n distinct raw immediates : %s" % " ".join("%#x" % v for v in sorted(seen)))
|
||||
print(" distinct running sums : %s" % " ".join("%#x" % v for v in sorted(sums)))
|
||||
print("\n CONTROL auctionCount %#x present? %s"
|
||||
% (A_AUCTION, "YES" if (A_AUCTION in seen or A_AUCTION in sums) else "NO -- enumeration broken, verdict void"))
|
||||
print(" clubPlayers %#x present? %s"
|
||||
% (A_CLUB, "YES" if (A_CLUB in seen or A_CLUB in sums) else "NO"))
|
||||
|
||||
print("\n callees of %#x:" % SUB)
|
||||
for a, n in callees(SUB):
|
||||
print(" %#x %s" % (a, n))
|
||||
|
||||
except Exception:
|
||||
traceback.print_exc()
|
||||
Reference in New Issue
Block a user