"""Decode the token enum from the character-class table, and settle the /hub problem.

WHERE QUERY 2 LEFT IT. 67 of 86 response roots spend exactly 3 tokenizer calls before their
first key read. FUN_18013bd40 gives one anchor for free: it tests `*(int *)(param_1 + 0xd0)
== 0xb` and then atom-hashes the string at param_1+0xf8, so +0xd0 is the CURRENT TOKEN TYPE
and 0xb is FIELD_NAME. Usage gives two more: 10 exits a key loop, 0xd exits an array loop.

THE PROBLEM THAT STOPS ME CONCLUDING. If the three tokens were `{`, wrapper-key, open-bracket,
then every body would need exactly one top-level key. But we serve
    GET /hub -> {"clubPlayers": 205, "auctionCount": 0}
with TWO top-level keys, and it is live-proven working (utas.log, 14:39:28). So either /hub's
root is one of the non-3 outliers, or the third token is a VALUE and the first key/value pair
is consumed without being dispatched. Those two readings imply very different rules for what
we may serve, so this is not a detail.

METHOD. Read the class table at DAT_18023dd40 for the structural characters, then walk the
switch in FUN_1801c67a0 to map class -> returned token. Then look up /hub's own root and
count its tokens, which discriminates the two readings directly.

CONTROL: '"' (0x22) must classify differently from '{' (0x7b), and a digit must differ from
both. If all three come back identical the table has been misread and nothing below stands.
"""
import traceback

CLASS_TABLE = 0x18023DD40
CHARS = [
    (0x7B, "{ open-object"), (0x7D, "} close-object"),
    (0x5B, "[ open-array"), (0x5D, "] close-array"),
    (0x22, '" quote'), (0x3A, ": colon"), (0x2C, ", comma"),
    (0x30, "0 digit (control)"), (0x41, "A letter (control)"),
    (0x20, "space (control)"), (0x74, "t of true"), (0x6E, "n of null"),
]

try:
    print("=" * 78)
    print("CHARACTER CLASS TABLE at %#x" % CLASS_TABLE)
    print("=" * 78)
    tbl = read_bytes(CLASS_TABLE, 0x80)
    print("read %d bytes" % len(tbl))
    for c, label in CHARS:
        print("  %-22s 0x%02x -> class %#x" % (label, c, tbl[c] if c < len(tbl) else -1))

    distinct = {tbl[c] for c, _ in CHARS if c < len(tbl)}
    print("\nCONTROL: %d distinct classes across the sampled characters. "
          "If this is 1 the table is misread." % len(distinct))

    # Full table, so nothing is hidden by the sample above.
    print("\nfull 0x00-0x7f class map (index: class), nonzero only:")
    print("  " + " ".join("%02x:%x" % (i, b) for i, b in enumerate(tbl) if b))

    # The switch that turns a class into a token.
    print("\n" + "=" * 78)
    print("FUN_1801c67a0 classifier, IN FULL (the class -> token switch)")
    print("=" * 78)
    src = dec(0x1801C67A0)
    print("decompile %d chars" % len(src))
    print(src)

    # /hub. Find its root and count tokens, which discriminates the two readings.
    print("\n" + "=" * 78)
    print("THE /hub DISCRIMINATOR")
    print("=" * 78)
    for lit in (b"RS4:FutGetHubServerResponse", b"clubPlayers", b"auctionCount"):
        hits = find_all(lit)
        print("\n%-32s %d hit(s): %s" % (lit.decode(), len(hits),
                                         " ".join("%#x" % h for h in hits)))
        for h in hits:
            for frm, typ, fn, ent in xrefs_to(h):
                print("      xref %#x in %s (entry %#x)" % (frm, fn, ent))
            for frm, typ, fn, ent in xrefs_to(h - 4):
                print("      xref-4 %#x in %s (entry %#x)" % (frm, fn, ent))

except Exception:
    traceback.print_exc()