#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""Read FIFA 17's live FUT club-stat store -- STRICTLY READ-ONLY.

WHY THIS EXISTS
---------------
Static analysis produced two contradictory positions about the "MY CLUB / 0 TOTAL
PLAYERS" bug:

  (A) our /club/stats response body IS parsed and IS stored, and the panel simply
      reads a bucket (mode) we never populate  -> the bug is SELECTION;
  (B) our body never lands in the store at all -> the bug is DELIVERY/SCHEMA.

Those two differ by one observable fact: what the store actually holds while the
game is sitting on the FUT hub. This tool reads it out of the running process.

THE CHAIN, RESOLVED STATICALLY THIS SESSION (CardsDLL, image base 0x180000000)
-----------------------------------------------------------------------------
  FUN_18011a830:                 return DAT_1802e6398;          <- CardsDb singleton
  CardsDb vtable = 0x18021c2a0
  vt+0x7f0 -> 0x18011bbb0:       `lea rax,[rcx+0x1f8b0]; ret`   <- THE STORE IS A
                                                                  SUBOBJECT, not a
                                                                  separate alloc.
      So   store == CardsDb + 0x1F8B0.  No virtual call is needed to reach it,
      which is what makes this probe possible from outside the process.
  vt+0x7f8 -> 0x18011bb10:       stat_get(this, contextValue, typeId)
  vt+0x800 -> 0x18011bba0:       `jmp vt+0x7f8(this, 0, typeId)` <- the panel's getter;
                                                                   contextValue is
                                                                   HARD-WIRED to 0.

  0x18011bb10 (getter_7f8, 47 lines, read in full) walks:
      outer map head = this+0x1f8e8 (== store+0x38)
      outer map root = this+0x1f8f8 (== store+0x48)
      outer node: child0 +0x00, child1 +0x08, parent +0x10,
                  key = uint32 @ +0x20                    (== contextValue)
                  inner map head = node+0x30, root = node+0x40, size = int32 @ node+0x50
      inner node: child0 +0x00, child1 +0x08, parent +0x10,
                  key   = int32 @ +0x20                   (== type id)
                  value = int32 @ +0x24                   (== typeValue)
      returns 0 when either key is absent -- so a MISSING entry and a STORED ZERO
      are indistinguishable to the panel, but NOT to this tool.

  0x180130150 (FutStickerBookStats2 deserializer, 6.7 KB, read in full) writes into
  exactly those two maps (`lVar12+0x30` outer base, node+0x28 inner base, inner
  node alloc size 0x28 with key@+0x20 / value@+0x24), and on END_OBJECT does
  `*(byte *)(store + 0x28) = 1` -- a PARSE-COMPLETED flag this tool reports.

  FUN_18012fbe0 (request completion) writes store+0x78/+0x7c/+0x80 from
  request+0xc4/+0xc8/+0xcc -- the mode tag the panel provider FUN_180043b90
  switches on.

WHICH BUCKET EACH PANEL MODE READS (FUN_180043b90, 11675 chars, read in full)
----------------------------------------------------------------------------
This corrects an earlier, wrong case map. Verified line by line:

  mode 1 club        vt+0x800  -> bucket 0 only.  PLAYERS_EMPLOYED, BALLS_EARNED,
                                 KITS_AVAILABLE, STADIA_OWNED, STAFF_EMPLOYED,
                                 TROPHIES_WON.
  mode 2 year        rows with IS_TEAM_CATEGORY=false -> bucket 0 (balls, stadia,
                                 managers, headcoaches, physio, gkcoaches, fitness,
                                 4 trophy variants);
                     rows with IS_TEAM_CATEGORY=true  -> bucket = row's NATION_ID
                                 (bronze/silver/gold, PLAYERS = their SUM, rare,
                                 kits, badges).
  mode 3 country/id  bucket = each row's LEAGUE_ID.  NEVER bucket 0.
  mode 4 league/id   bucket = each row's TEAM_ID.    NEVER bucket 0.
  mode 5 newcards    bucket 0 only.
  mode 6 consumables bucket 0 only.

  So in modes 2 (team rows), 3 and 4, everything we put in bucket 0 is invisible:
  the panel asks for buckets keyed by league / team / nation ids. The URL id
  (store+0x7c) is NOT the bucket key either -- the key comes from the row list the
  UI passes in as param_2.

READ-ONLY GUARANTEE
-------------------
/proc/PID/mem is opened "rb" and the only operations performed on it are seek()
and read(). assert_read_only() re-checks the handle's mode at startup and aborts
if anything ever made it writable. There is no ptrace attach, no write path, and
no code that constructs one.

HOW TO READ THE OUTPUT
----------------------
  bucket 0 holds type 1 (players) = 114
      -> our body LANDED. The store has the number. The panel showing 0 is then a
         SELECTION problem (wrong mode tag / wrong bucket / stale panel), not a
         delivery or schema problem.
  bucket 0 exists but type 1 is absent
      -> the type string in our JSON is not mapping to id 1 (FUN_18012fd40 maps the
         atom for "players", 0x238, to 1). Schema bug in the `type` field.
  a bucket exists but it is not 0
      -> contextId/contextValue guard put us in the wrong bucket; the panel getter
         only ever asks bucket 0.
  outer map is EMPTY while parsed=1
      -> the body parsed but every entry was dropped or the map was wiped after
         parse (the factory wipe). Delivery problem.
  outer map is EMPTY and parsed=0
      -> our response never reached this deserializer at all.
  mode tag != 1
      -> corroborates the "case 1 is never selected" verdict, but ONLY if the store
         does hold the values; on its own it proves nothing.

USAGE
-----
  python3 tools/probe_club_stats.py              # one snapshot, then exit
  python3 tools/probe_club_stats.py --watch      # poll until Ctrl-C, print on change
  python3 tools/probe_club_stats.py --raw        # + hexdump of store[0x00:0x90]
  python3 tools/probe_club_stats.py --get 1      # emulate vt+0x800(typeId) exactly

FIFA does not have to be in MY CLUB when you start it; --watch is the intended way
to see the store fill as you navigate.

Needs read access to /proc/PID/mem (kernel.yama.ptrace_scope=0, or run as root).
"""

import argparse
import glob
import os
import struct
import sys
import time

# ---------------------------------------------------------------- constants --
IMG_BASE = 0x180000000
DLL = "CardsDLL"
PROC_NAME = "FIFA17.exe"

G_CARDSDB = 0x1802E6398          # FUN_18011a830 returns this (VERIFIED this session)

STORE_OFF = 0x1F8B0              # vt+0x7f0: lea rax,[rcx+0x1f8b0]  (VERIFIED)

# offsets RELATIVE TO THE STORE
ST_PARSED = 0x28                 # set to 1 by the deserializer on END_OBJECT
ST_MAP_BASE = 0x30               # outer std::map base
ST_MAP_HEAD = 0x38               # == CardsDb+0x1f8e8, the getter's sentinel
ST_MAP_ANCHOR = 0x40
ST_MAP_ROOT = 0x48               # == CardsDb+0x1f8f8
ST_MAP_SIZE = 0x58               # base+0x28, same layout as the CardsDb item tree
ST_MODE = 0x78                   # request+0xc4 (FUN_18012fbe0)
ST_MODE_ARG1 = 0x7C              # request+0xc8
ST_MODE_ARG2 = 0x80              # request+0xcc

# node layout, shared by both levels
N_C0, N_C1, N_PARENT, N_KEY = 0x00, 0x08, 0x10, 0x20
IN_VALUE = 0x24                  # inner node only

# inner map, relative to an OUTER node
ON_INNER_BASE = 0x28
ON_INNER_HEAD = 0x30
ON_INNER_ROOT = 0x40
ON_INNER_SIZE = 0x50

MAX_NODES = 20000                # a corrupt tree terminates instead of hanging us
PTR_LO, PTR_HI = 0x10000, 0x00007FFFFFFFFFFF   # plausible user-space range

# URL builder FUN_18012f4f0
MODE_NAMES = {1: "club", 2: "year", 3: "country+id", 4: "league+id",
              5: "newcards", 6: "consumables"}

# FUN_18012fd40: atom -> type id, cross-referenced against docs/fut_atoms.tsv
TYPE_NAMES = {
    0x01: 'players', 0x02: 'playersBronze', 0x03: 'playersSilver',
    0x04: 'playersGold', 0x05: 'rarePlayers', 0x0a: 'staff',
    0x0b: 'staffManager', 0x0c: 'staffHeadCoach', 0x0d: 'staffGKCoach',
    0x0e: 'staffPhysio', 0x0f: 'staffFitnessCoach', 0x14: 'stadia',
    0x1e: 'balls', 0x28: 'kits', 0x29: 'kitsHome', 0x2a: 'kitsAway',
    0x2d: 'badges', 0x2e: 'badgeDBid', 0x2f: 'leagueLogos', 0x32: 'trophies',
    0x33: 'trophiesOffline', 0x34: 'trophiesOnline',
    0x35: 'trophiesFeaturedOffline', 0x36: 'trophiesFeaturedOnline',
    0x37: 'trophiesSeasonOffline', 0x38: 'trophiesSeasonOnline',
    0x3c: 'consumables', 0x41: 'consumablesHealing',
    0x42: 'consumablesContractPlayer', 0x43: 'consumablesTrainingPlayer',
    0x44: 'consumablesFitnessPlayer', 0x45: 'consumablesPosition',
    0x46: 'consumablesTrainingGk', 0x47: 'consumablesContractManager',
    0x48: 'consumablesFormationManager', 0x49: 'consumablesTrainingManager',
    0x4a: 'consumablesFitnessTeam',
    0x4b: 'consumablesTrainingPlayerPlayStyle',
    0x4c: 'consumablesTrainingGkPlayStyle',
    0x4d: 'consumablesTrainingManagerLeagueModifier',
}

PLAYERS_TYPE_ID = 1


# ------------------------------------------------------------------ process --
def find_pid():
    for d in glob.glob("/proc/[0-9]*"):
        try:
            with open(d + "/comm") as f:
                if f.read().strip() == PROC_NAME:
                    return int(d.rsplit("/", 1)[-1])
        except Exception:
            pass
    return None


def module_base(pid, name=DLL):
    """Live load base of CardsDLL. It is NOT 0x180000000 in the Wine process."""
    try:
        with open("/proc/%d/maps" % pid) as f:
            for line in f:
                if name in line:
                    return int(line.split("-", 1)[0], 16)
    except Exception:
        return None
    return None


class Mem(object):
    """Read-only /proc/PID/mem accessor. Failures return None, never raise."""

    def __init__(self, pid):
        self.pid = pid
        self.fails = 0
        self.f = open("/proc/%d/mem" % pid, "rb")   # "rb": read-only, by design
        self.assert_read_only()

    def assert_read_only(self):
        """Abort rather than continue if this handle could ever write."""
        mode = getattr(self.f, "mode", "")
        if not self.f.readable() or self.f.writable() or "+" in mode or "w" in mode:
            raise SystemExit("REFUSING TO RUN: /proc/%d/mem handle is not read-only "
                             "(mode=%r). This tool must never write to the game."
                             % (self.pid, mode))

    def read(self, va, n):
        if va is None or va < PTR_LO or va > PTR_HI:
            self.fails += 1
            return None
        try:
            self.f.seek(va)
            b = self.f.read(n)
        except Exception:
            self.fails += 1
            return None
        if b is None or len(b) != n:
            self.fails += 1
            return None
        return b

    def q(self, va):
        b = self.read(va, 8)
        return struct.unpack("<Q", b)[0] if b else None

    def u32(self, va):
        b = self.read(va, 4)
        return struct.unpack("<I", b)[0] if b else None

    def i32(self, va):
        b = self.read(va, 4)
        return struct.unpack("<i", b)[0] if b else None

    def u8(self, va):
        b = self.read(va, 1)
        return b[0] if b else None

    def alive(self):
        return os.path.exists("/proc/%d" % self.pid)


def plausible(p):
    return p is not None and PTR_LO <= p <= PTR_HI and (p & 7) == 0


# ------------------------------------------------------------- tree walking --
def walk(mem, head, root, key_signed, want_value):
    """DFS a red-black tree. Returns (list_of_(key, value_or_None, node_va), note).

    Both children are followed with a visited set, so the left/right convention
    does not matter and a cycle or a garbage pointer stops the walk instead of
    hanging it. `head` is the sentinel; it is never treated as a real node.
    """
    if root is None:
        return [], "root unreadable"
    if root == 0 or root == head:
        return [], "empty"
    out, seen, stack, note = [], set(), [root], "ok"
    while stack:
        p = stack.pop()
        if p == 0 or p == head or p in seen:
            continue
        if not plausible(p):
            note = "hit an implausible pointer -- walk partial"
            continue
        if len(out) >= MAX_NODES:
            note = "TRUNCATED at %d nodes" % MAX_NODES
            break
        seen.add(p)
        k = mem.i32(p + N_KEY) if key_signed else mem.u32(p + N_KEY)
        if k is None:
            note = "node key unreadable -- walk partial"
            continue
        v = mem.i32(p + IN_VALUE) if want_value else None
        out.append((k, v, p))
        for slot in (N_C0, N_C1):
            c = mem.q(p + slot)
            if c is None:
                note = "child pointer unreadable -- walk partial"
                continue
            if c and c != head and c not in seen:
                stack.append(c)
    return out, note


def read_store(mem, cdb):
    """Snapshot the whole two-level stat store. Never raises."""
    st = cdb + STORE_OFF
    s = {
        "cardsdb": cdb,
        "store": st,
        "parsed": mem.u8(st + ST_PARSED),
        "mode": mem.i32(st + ST_MODE),
        "mode_arg1": mem.i32(st + ST_MODE_ARG1),
        "mode_arg2": mem.i32(st + ST_MODE_ARG2),
        "outer_size": mem.i32(st + ST_MAP_SIZE),
        "buckets": None,
        "note": "",
    }
    head = st + ST_MAP_HEAD
    root = mem.q(st + ST_MAP_ROOT)
    s["outer_root"] = root
    outer, note = walk(mem, head, root, key_signed=False, want_value=False)
    s["note"] = note
    if note in ("root unreadable",):
        return s
    buckets = []
    for ctx, _v, node in sorted(outer):
        ihead = node + ON_INNER_HEAD
        iroot = mem.q(node + ON_INNER_ROOT)
        isize = mem.i32(node + ON_INNER_SIZE)
        inner, inote = walk(mem, ihead, iroot, key_signed=True, want_value=True)
        buckets.append({
            "contextValue": ctx,
            "node": node,
            "size_field": isize,
            "entries": sorted((k, v) for k, v, _ in inner),
            "note": inote,
        })
    s["buckets"] = buckets
    return s


def stat_get(store_snapshot, ctx, type_id):
    """Exactly what vt+0x7f8 returns: the value, or 0 when either key is absent.

    Returns (value, found) so a stored 0 can be told apart from an absent key --
    the game itself cannot make that distinction.
    """
    for b in store_snapshot.get("buckets") or []:
        if b["contextValue"] == ctx:
            for k, v in b["entries"]:
                if k == type_id:
                    return v, True
            return 0, False
    return 0, False


# ---------------------------------------------------------------- reporting --
def tname(t):
    return TYPE_NAMES.get(t, "type_%#x" % t)


def fmt_ptr(v):
    return "UNREADABLE" if v is None else "%#x" % v


def report(s, raw=None):
    lines = []
    a = lines.append
    a("CardsDb            %s" % fmt_ptr(s["cardsdb"]))
    a("store (cdb+0x1f8b0) %s" % fmt_ptr(s["store"]))
    p = s["parsed"]
    a("store+0x28 parsed   %s%s" % ("UNREADABLE" if p is None else p,
                                    "   <== a FutStickerBookStats2 body completed parsing"
                                    if p == 1 else ""))
    m = s["mode"]
    a("store+0x78 mode     %s  (%s)   +0x7c=%s +0x80=%s"
      % ("UNREADABLE" if m is None else m,
         MODE_NAMES.get(m, "unknown/never-set"),
         s["mode_arg1"], s["mode_arg2"]))
    a("outer map           root=%s  size_field=%s  walk=%s"
      % (fmt_ptr(s.get("outer_root")), s["outer_size"], s["note"]))

    buckets = s["buckets"]
    if buckets is None:
        a("  (outer map unreadable)")
    elif not buckets:
        a("  NO BUCKETS -- the stat map is empty.")
    else:
        for b in buckets:
            a("  bucket contextValue=%d  node=%#x  size_field=%s  entries=%d  (%s)"
              % (b["contextValue"], b["node"], b["size_field"],
                 len(b["entries"]), b["note"]))
            if b["size_field"] is not None and b["size_field"] != len(b["entries"]):
                a("      WARNING: size field disagrees with the walk -- walk suspect")
            for k, v in b["entries"]:
                a("      %-3d %-42s = %s" % (k, tname(k), v))

    a("")
    v, found = stat_get(s, 0, PLAYERS_TYPE_ID)
    a("vt+0x800(typeId=1 'players')  ->  %d      [%s]"
      % (v, "PRESENT in bucket 0" if found
         else "ABSENT -- the getter returns 0 by fallthrough"))
    if found and v:
        a("VERDICT INPUT: the store HOLDS players=%d. Delivery and schema are FINE;"
          % v)
        a("               a panel reading 0 is then a SELECTION failure.")
    elif s["parsed"] == 1 and not found:
        a("VERDICT INPUT: a body parsed (parsed=1) but bucket 0 / type 1 is absent.")
        a("               Either our contextValue is not 0 or our type string is not")
        a("               mapping to id 1. That is a SCHEMA failure, not selection.")
    elif not buckets:
        a("VERDICT INPUT: the store is empty. Our /club/stats body is NOT landing.")

    if m in (2, 3, 4) and buckets is not None:
        keyed = [b["contextValue"] for b in buckets if b["contextValue"] != 0]
        a("NOTE: mode %d reads PER-ROW buckets (%s), never bucket 0."
          % (m, {2: "NATION_ID for team rows", 3: "LEAGUE_ID", 4: "TEAM_ID"}[m]))
        a("      non-zero buckets present: %s"
          % (keyed if keyed else "NONE -- every per-row lookup falls through to 0"))
    if raw is not None:
        a("")
        a("raw store[0x00:0x90]:")
        for off in range(0, 0x90, 16):
            chunk = raw[off:off + 16]
            a("  +%#04x  %s" % (off, " ".join("%02x" % c for c in chunk)))
    return "\n".join(lines)


def signature(s):
    """Change key for --watch: everything a human would notice."""
    return (s["parsed"], s["mode"], s["mode_arg1"], s["mode_arg2"],
            tuple((b["contextValue"], tuple(b["entries"]))
                  for b in (s["buckets"] or [])))


# --------------------------------------------------------------------- main --
def main():
    ap = argparse.ArgumentParser(
        description="Read FIFA 17's live FUT club-stat store (READ-ONLY)")
    ap.add_argument("--watch", action="store_true",
                    help="poll and print whenever the store changes (Ctrl-C to stop)")
    ap.add_argument("--interval", type=float, default=0.5, help="poll seconds")
    ap.add_argument("--raw", action="store_true",
                    help="also hexdump store[0x00:0x90]")
    ap.add_argument("--get", type=lambda x: int(x, 0), default=None, metavar="TYPEID",
                    help="emulate vt+0x800(TYPEID) and print just that value")
    ap.add_argument("--ctx", type=lambda x: int(x, 0), default=0,
                    help="contextValue bucket for --get (default 0, what the panel uses)")
    args = ap.parse_args()

    pid = find_pid()
    if pid is None:
        print("%s is not running. Start FIFA, reach the FUT hub, then run this." % PROC_NAME)
        return 1
    base = module_base(pid)
    if base is None:
        print("%s (pid %d) is running but %s is not mapped yet." % (PROC_NAME, pid, DLL))
        print("Wait for the FUT layer to load (main menu / Ultimate Team) and re-run.")
        return 1
    try:
        mem = Mem(pid)
    except SystemExit:
        raise
    except Exception as e:
        print("cannot open /proc/%d/mem: %s" % (pid, e))
        print("Need:  sudo sysctl -w kernel.yama.ptrace_scope=0")
        return 1

    gva = base + (G_CARDSDB - IMG_BASE)
    print("pid=%d  %s base=%#x  (image base %#x)" % (pid, DLL, base, IMG_BASE))
    print("CardsDb global @ %#x  (static %#x)" % (gva, G_CARDSDB))

    def snap():
        cdb = mem.q(gva)
        if cdb is None:
            return None, "CardsDb global unreadable"
        if cdb == 0:
            return None, "CardsDb singleton is NULL -- the FUT layer is not constructed"
        if not plausible(cdb):
            return None, "CardsDb global holds an implausible pointer %#x" % cdb
        st = cdb + STORE_OFF
        if mem.read(st, 0x90) is None:
            return None, "store window at %#x is not mapped" % st
        return read_store(mem, cdb), None

    if args.get is not None:
        s, err = snap()
        if err:
            print(err)
            return 1
        v, found = stat_get(s, args.ctx, args.get)
        print("stat_get(ctx=%d, type=%d %s) = %d   [%s]"
              % (args.ctx, args.get, tname(args.get), v,
                 "present" if found else "ABSENT (getter fallthrough 0)"))
        return 0

    def once():
        s, err = snap()
        if err:
            print("[%s] %s" % (time.strftime("%H:%M:%S"), err))
            return None
        raw = mem.read(s["store"], 0x90) if args.raw else None
        print(report(s, raw))
        return s

    if not args.watch:
        print()
        s = once()
        print("\nfailed reads: %d" % mem.fails)
        return 0 if s else 1

    print("watching -- navigate FIFA into MY CLUB now. Ctrl-C to stop.\n")
    last = None
    try:
        while True:
            if not mem.alive():
                print("[%s] %s exited." % (time.strftime("%H:%M:%S"), PROC_NAME))
                break
            s, err = snap()
            if err:
                if last != err:
                    print("[%s] %s" % (time.strftime("%H:%M:%S"), err))
                    last = err
            else:
                sig = signature(s)
                if sig != last:
                    print("=" * 68)
                    print("[%s] STORE CHANGED" % time.strftime("%H:%M:%S"))
                    print(report(s, mem.read(s["store"], 0x90) if args.raw else None))
                    print()
                    last = sig
            time.sleep(args.interval)
    except KeyboardInterrupt:
        print("\nstopped")
    print("failed reads: %d" % mem.fails)
    return 0


if __name__ == "__main__":
    sys.exit(main())