funman300
1605e6effd
The open question was whether a response deserializer DESCENDS a wrapper or PROBES for
one. Three roots spend an identical three tokenizer calls before dispatch, yet we serve
some bodies wrapped and some flat, and not all of those could be right.
TOKEN ENUM, decoded from the class table at DAT_18023dd40 and the switch in the
classifier FUN_1801c67a0 (the push/pop arms key off container state 2 = object,
3 = array):
9 START_OBJECT case 0x64, pushes state 2
10 END_OBJECT case 0x65, pops state 2
11 FIELD_NAME confirmed independently: FUN_18013bd40 tests +0xd0 == 0xb then
atom-hashes the string at +0xf8
12 START_ARRAY case 0x66, pushes state 3
13 END_ARRAY case 0x67, pops state 3
1 error the caseD_78 sink
So the three tokens are `{`, the first FIELD_NAME, and the token opening that field's
value. The envelope is structurally required and its name is NEVER hashed, which is why
FutCreatePack's ladder has no arm for createPackResponse (0xbe) and does not need one.
Coverage for that absence: the ladder has exactly four arms (0xec, 0x16e, 0x1dd, 0x264)
and 0xbe does not occur anywhere in the full 4702-char decompile, printed in full.
The competing reading rested on a factual error. It claimed the /purchased root spends
the same three tokens. FUN_180124ee0 spends TWO and hands off to FUN_18013bd40, which
spends the third. Same total, split across two functions. /purchased never was a
counterexample.
THE BUG THIS FOUND IS NOT THE ONE THAT WAS PREDICTED. The doc expected starterPack,
squad and userData to be swallowed on POST /user. They are not. FutCreateUser
(0x18014cc60) has ladder arms for exactly the five keys we send, and four of them
dispatch correctly at the outer level. The one that does not is `login`: its name is
eaten as the anonymous envelope and its value as the third token. It has an arm, so the
client wants it, and it has never once been delivered.
The second-order consequence matters more than the first. The key order of that dict is
load-bearing and nothing said so. Put userData first and the client loses the entire
user record, silently, with no error and no log line. That warning now sits in the code
next to the dict, which is the only place someone about to reorder it would look.
No behaviour change here. The utas_server.py edit is a comment. 439 contract checks
still pass. The probable proper fix, wrapping all five keys one level down inside a
single envelope key, is a hypothesis with a mechanism rather than a proven fix, and it
touches the login path, so it is not made here and would go behind a flag defaulting
off.
Writeup is section 2 of docs/plan-2026-08-05-pack-opening.md, added by the previous
commit. Opened by this and still UNKNOWN: GET /hub is answered with a flat two-key
body, which under this rule a three-token root would silently truncate, but there is no
FutGetHubServerResponse class and neither atom has a code xref, so /hub may not go
through a generated root at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
75 lines
3.4 KiB
Python
75 lines
3.4 KiB
Python
"""Decode the token enum from the character-class table, and settle the /hub problem.
|
|
|
|
WHERE QUERY 2 LEFT IT. 67 of 86 response roots spend exactly 3 tokenizer calls before their
|
|
first key read. FUN_18013bd40 gives one anchor for free: it tests `*(int *)(param_1 + 0xd0)
|
|
== 0xb` and then atom-hashes the string at param_1+0xf8, so +0xd0 is the CURRENT TOKEN TYPE
|
|
and 0xb is FIELD_NAME. Usage gives two more: 10 exits a key loop, 0xd exits an array loop.
|
|
|
|
THE PROBLEM THAT STOPS ME CONCLUDING. If the three tokens were `{`, wrapper-key, open-bracket,
|
|
then every body would need exactly one top-level key. But we serve
|
|
GET /hub -> {"clubPlayers": 205, "auctionCount": 0}
|
|
with TWO top-level keys, and it is live-proven working (utas.log, 14:39:28). So either /hub's
|
|
root is one of the non-3 outliers, or the third token is a VALUE and the first key/value pair
|
|
is consumed without being dispatched. Those two readings imply very different rules for what
|
|
we may serve, so this is not a detail.
|
|
|
|
METHOD. Read the class table at DAT_18023dd40 for the structural characters, then walk the
|
|
switch in FUN_1801c67a0 to map class -> returned token. Then look up /hub's own root and
|
|
count its tokens, which discriminates the two readings directly.
|
|
|
|
CONTROL: '"' (0x22) must classify differently from '{' (0x7b), and a digit must differ from
|
|
both. If all three come back identical the table has been misread and nothing below stands.
|
|
"""
|
|
import traceback
|
|
|
|
CLASS_TABLE = 0x18023DD40
|
|
CHARS = [
|
|
(0x7B, "{ open-object"), (0x7D, "} close-object"),
|
|
(0x5B, "[ open-array"), (0x5D, "] close-array"),
|
|
(0x22, '" quote'), (0x3A, ": colon"), (0x2C, ", comma"),
|
|
(0x30, "0 digit (control)"), (0x41, "A letter (control)"),
|
|
(0x20, "space (control)"), (0x74, "t of true"), (0x6E, "n of null"),
|
|
]
|
|
|
|
try:
|
|
print("=" * 78)
|
|
print("CHARACTER CLASS TABLE at %#x" % CLASS_TABLE)
|
|
print("=" * 78)
|
|
tbl = read_bytes(CLASS_TABLE, 0x80)
|
|
print("read %d bytes" % len(tbl))
|
|
for c, label in CHARS:
|
|
print(" %-22s 0x%02x -> class %#x" % (label, c, tbl[c] if c < len(tbl) else -1))
|
|
|
|
distinct = {tbl[c] for c, _ in CHARS if c < len(tbl)}
|
|
print("\nCONTROL: %d distinct classes across the sampled characters. "
|
|
"If this is 1 the table is misread." % len(distinct))
|
|
|
|
# Full table, so nothing is hidden by the sample above.
|
|
print("\nfull 0x00-0x7f class map (index: class), nonzero only:")
|
|
print(" " + " ".join("%02x:%x" % (i, b) for i, b in enumerate(tbl) if b))
|
|
|
|
# The switch that turns a class into a token.
|
|
print("\n" + "=" * 78)
|
|
print("FUN_1801c67a0 classifier, IN FULL (the class -> token switch)")
|
|
print("=" * 78)
|
|
src = dec(0x1801C67A0)
|
|
print("decompile %d chars" % len(src))
|
|
print(src)
|
|
|
|
# /hub. Find its root and count tokens, which discriminates the two readings.
|
|
print("\n" + "=" * 78)
|
|
print("THE /hub DISCRIMINATOR")
|
|
print("=" * 78)
|
|
for lit in (b"RS4:FutGetHubServerResponse", b"clubPlayers", b"auctionCount"):
|
|
hits = find_all(lit)
|
|
print("\n%-32s %d hit(s): %s" % (lit.decode(), len(hits),
|
|
" ".join("%#x" % h for h in hits)))
|
|
for h in hits:
|
|
for frm, typ, fn, ent in xrefs_to(h):
|
|
print(" xref %#x in %s (entry %#x)" % (frm, fn, ent))
|
|
for frm, typ, fn, ent in xrefs_to(h - 4):
|
|
print(" xref-4 %#x in %s (entry %#x)" % (frm, fn, ent))
|
|
|
|
except Exception:
|
|
traceback.print_exc()
|