fifa17-python: commit working FUT backend deployment (client/server split)
Freeze the running offline FUT backend into version control as fifa17-recon/docker/fifa17-python/ - declarative and rebuildable from a fresh checkout: * OPENFUT_BIND / OPENFUT_ADVERTISE client/server split in the responders (lsx, blaze, roster, utas, pow) + entrypoint.sh; OPENFUT_ADVERTISE is required for remote mode (compose and entrypoint fail without it) * docker-compose.yml reproducing the frozen baseline container exactly (env, ports incl. the 8085->8080 POW-content remap, /state bind, restart) * .env.example / .env for site config - the LAN IP is never hardcoded in source * tools/ + data/ staged from openfut-fut-backend:python-baseline-2026-08-10, verified byte-identical to the running container at freeze time * client_arm.sh (the 105 client-side arming counterpart) * Dockerfile bakes /app/SHA256SUMS.txt so any image is self-identifying * docs/BASELINE-python-2026-08-10.md: frozen image/container/hash record, restore instructions and rebuild-equivalence procedure Secrets (redir key/cert, .env) and runtime state (docker/state) stay gitignored. The live container is untouched pending the .105 launcher audit.
This commit is contained in:
root
+325
@@ -0,0 +1,325 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Extract FIFA 17's real player roster (playerId -> name + rating) from a LIVE
|
||||
FIFA17.exe process. READ-ONLY: /proc/PID/mem is opened 'rb' and only ever
|
||||
seek()/read(). There is no write path in this file.
|
||||
|
||||
WHY THIS FILE EXISTS, AND WHY IT IS NOT "dbdata.dll"
|
||||
---------------------------------------------------
|
||||
The plan of record was to pull the roster out of `/mnt/games/FIFA 17/dbdata.dll`
|
||||
(2,686,152 bytes, one export `getTableData`, 2.5 MB `.xdata` payload). That DLL
|
||||
is NOT a database. Verified this session by building tools/dbdata_probe.c with
|
||||
x86_64-w64-mingw32-gcc and running it under Wine:
|
||||
|
||||
base=00006FFFFA980000 getTableData=00006FFFFA9816B0 (rva 0x16b0)
|
||||
call 0: ret=00007FFFFEBF5DB0 len=1012 -> 1012 chars of base64url
|
||||
decoded: 759 bytes, md5 dc97c0dfd5edea5fb379dc14d8017980, entropy ~7.9
|
||||
|
||||
`.xdata` measures 7.52 bits/byte of entropy uniformly across its whole 2,515,528
|
||||
bytes (sampled at 0x0/0x1000/0x100000/0x200000/0x260000, zero 16-byte NUL runs),
|
||||
i.e. it is encrypted/packed, and the single export hands back a ~759-byte
|
||||
attestation blob, not tables. There is no table selector argument. So the
|
||||
roster cannot be read out of dbdata.dll without breaking its packer.
|
||||
|
||||
The roster IS, however, fully resident in the running game. FIFA 17 builds a
|
||||
flat, rating-sorted index of every player in the base DB and keeps it on the
|
||||
heap. That is what this tool reads.
|
||||
|
||||
THE STRUCTURE (resolved live, 2026-08-04, pid 11864, game sitting at the menu)
|
||||
-----------------------------------------------------------------------------
|
||||
Two heap regions cooperate:
|
||||
|
||||
* a "name pool" region (seen at 0x0d790000..0x0dc40000, 4.8 MB, rw-p) holding
|
||||
~17.5k individually-allocated, NUL-terminated UTF-8 strings in the form
|
||||
"<firstName>|<lastName>|<commonName>"
|
||||
e.g. "Cristiano|Ronaldo|", "Neymar|da Silva Santos Jr.|Neymar".
|
||||
commonName is usually empty (the string then ends in "||").
|
||||
|
||||
* an index table (seen at 0x0b8450d40 .. 0x0b8562fc0, rw-p) of 17,547 entries
|
||||
at a constant stride of 0x40 bytes, no gaps, sorted by rating DESCENDING:
|
||||
|
||||
+0x00 u32 playerId (20801 = Cristiano Ronaldo)
|
||||
+0x04 u32 rank (0..16546, dense, == entry index)
|
||||
+0x08 u32 rating (73..94 at the head, down to 40s at tail)
|
||||
+0x0c u32 aux (0 for most entries; a 32-bit hash for
|
||||
some -- purpose unresolved)
|
||||
+0x10 char* name begin -> into the name pool
|
||||
+0x18 char* name end == begin + strlen
|
||||
+0x20 char* name end + 1
|
||||
+0x28 u64 0x2c020e50 (constant across every entry)
|
||||
+0x30 u64 1, or a 32-bit hash in the low dword
|
||||
+0x38 u64 0x6ffffc32a968 (constant across every entry -- a vtable
|
||||
or allocator handle in the Wine range)
|
||||
|
||||
The {begin, end, end+1} triple at +0x10 is the reliable signature: it is
|
||||
self-validating (end-begin == strlen, and end+1 == the third pointer), which is
|
||||
why this tool anchors on it instead of on any hard-coded address. Nothing here
|
||||
is a fixed VA: run it against any FIFA17.exe and it re-locates the table.
|
||||
|
||||
WHAT THIS GIVES YOU AND WHAT IT DOES NOT
|
||||
----------------------------------------
|
||||
GIVES: playerId, rank, rating, firstName, lastName, commonName -- for the
|
||||
complete 17,547-player FIFA 17 roster.
|
||||
DOES NOT: position, nationality, teamId, or the six face attributes. Those
|
||||
are NOT in this table. See the "STILL MISSING" note at the bottom
|
||||
of this file for the leads that were found for them.
|
||||
|
||||
ANCHOR CHECK (the one the task asked for): playerId 20801 must be
|
||||
"Cristiano|Ronaldo|" rated 94. --check enforces it and exits non-zero if the
|
||||
parse disagrees.
|
||||
|
||||
USAGE
|
||||
-----
|
||||
./dbdata_extract.py # extract, write players_fifa17.json
|
||||
./dbdata_extract.py --check # extract + assert the Ronaldo anchor
|
||||
./dbdata_extract.py -o /tmp/roster.json
|
||||
./dbdata_extract.py --pid 11864
|
||||
./dbdata_extract.py --top 40 # print the top 40 and exit
|
||||
|
||||
Requires ptrace access to the FIFA process (this project already runs with
|
||||
kernel.yama.ptrace_scope=1 and the same uid, which is sufficient).
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import struct
|
||||
import sys
|
||||
from collections import Counter
|
||||
|
||||
ENTRY_STRIDE = 0x40
|
||||
NAME_TRIPLE_OFF = 0x10 # offset of {begin,end,end+1} inside an entry
|
||||
MAX_NAME_LEN = 120
|
||||
RONALDO_ID = 20801
|
||||
RONALDO_RATING = 94
|
||||
RONALDO_NAME = "Cristiano|Ronaldo|"
|
||||
|
||||
|
||||
def find_pid():
|
||||
for d in os.listdir('/proc'):
|
||||
if not d.isdigit():
|
||||
continue
|
||||
try:
|
||||
if open('/proc/%s/comm' % d).read().strip() == 'FIFA17.exe':
|
||||
return int(d)
|
||||
except OSError:
|
||||
pass
|
||||
raise SystemExit("FIFA17.exe is not running (this tool needs the live game)")
|
||||
|
||||
|
||||
def read_maps(pid):
|
||||
"""Readable, non-file-backed-or-anon RW regions, small enough to slurp."""
|
||||
out = []
|
||||
for line in open('/proc/%d/maps' % pid):
|
||||
parts = line.split()
|
||||
lo, hi = parts[0].split('-')
|
||||
lo, hi = int(lo, 16), int(hi, 16)
|
||||
perms = parts[1]
|
||||
if 'r' not in perms:
|
||||
continue
|
||||
if hi - lo > (1 << 31):
|
||||
continue
|
||||
out.append((lo, hi, perms, parts[5] if len(parts) > 5 else ''))
|
||||
return out
|
||||
|
||||
|
||||
class Mem(object):
|
||||
def __init__(self, pid):
|
||||
self.f = open('/proc/%d/mem' % pid, 'rb', 0)
|
||||
self.cache = {}
|
||||
|
||||
def read(self, va, n):
|
||||
self.f.seek(va)
|
||||
return self.f.read(n)
|
||||
|
||||
def region(self, lo, hi):
|
||||
if (lo, hi) not in self.cache:
|
||||
try:
|
||||
self.f.seek(lo)
|
||||
self.cache[(lo, hi)] = self.f.read(hi - lo)
|
||||
except OSError:
|
||||
self.cache[(lo, hi)] = b''
|
||||
return self.cache[(lo, hi)]
|
||||
|
||||
|
||||
def find_name_pools(mem, maps):
|
||||
"""Regions containing many '<a>|<b>|<c>\\0' strings = the player-name pool."""
|
||||
pat = re.compile(rb'[^\x00|][^\x00|]{0,44}\|[^\x00|]{0,49}\|[^\x00|]{0,49}\x00')
|
||||
pools = []
|
||||
for lo, hi, perms, name in maps:
|
||||
if 'w' not in perms or name:
|
||||
continue
|
||||
if not (0x100000 <= hi - lo <= 0x4000000):
|
||||
continue
|
||||
d = mem.region(lo, hi)
|
||||
if not d:
|
||||
continue
|
||||
n = len(pat.findall(d))
|
||||
if n >= 2000:
|
||||
pools.append((lo, hi, n))
|
||||
return pools
|
||||
|
||||
|
||||
def scan_entries(mem, maps, pools):
|
||||
"""Anchor on the self-validating {begin,end,end+1} name triple."""
|
||||
lows = [(lo, hi) for lo, hi, _ in pools]
|
||||
|
||||
def in_pool(va):
|
||||
for lo, hi in lows:
|
||||
if lo <= va < hi:
|
||||
return True
|
||||
return False
|
||||
|
||||
def pool_bytes(va, n):
|
||||
for lo, hi in lows:
|
||||
if lo <= va and va - lo + n <= hi - lo:
|
||||
return mem.region(lo, hi)[va - lo:va - lo + n]
|
||||
return mem.read(va, n)
|
||||
|
||||
found = {}
|
||||
for lo, hi, perms, name in maps:
|
||||
if 'w' not in perms or name:
|
||||
continue
|
||||
d = mem.region(lo, hi)
|
||||
if len(d) < ENTRY_STRIDE:
|
||||
continue
|
||||
for off in range(0, len(d) - ENTRY_STRIDE, 8):
|
||||
b, e, c = struct.unpack_from('<QQQ', d, off)
|
||||
if not (b < e < b + MAX_NAME_LEN and c == e + 1):
|
||||
continue
|
||||
if not in_pool(b):
|
||||
continue
|
||||
s = pool_bytes(b, e - b)
|
||||
if s.count(b'|') != 2:
|
||||
continue
|
||||
base = off - NAME_TRIPLE_OFF
|
||||
if base < 0:
|
||||
continue
|
||||
pid_, rank, rating, aux = struct.unpack_from('<IIII', d, base)
|
||||
found[lo + base] = (pid_, rank, rating, aux,
|
||||
s.decode('utf-8', 'replace'))
|
||||
return found
|
||||
|
||||
|
||||
def build(found):
|
||||
addrs = sorted(found)
|
||||
rows = []
|
||||
for a in addrs:
|
||||
pid_, rank, rating, aux, s = found[a]
|
||||
f0, f1, f2 = (s.split('|') + ['', '', ''])[:3]
|
||||
rows.append({
|
||||
'playerId': pid_,
|
||||
'resourceId': pid_, # version 0; resourceId = playerId | version<<24
|
||||
'rank': rank,
|
||||
'rating': rating,
|
||||
'firstName': f0,
|
||||
'lastName': f1,
|
||||
'commonName': f2,
|
||||
'aux': aux,
|
||||
'addr': '0x%x' % a,
|
||||
})
|
||||
return addrs, rows
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__.split('\n')[0])
|
||||
ap.add_argument('--pid', type=int)
|
||||
ap.add_argument('-o', '--out', default='players_fifa17.json')
|
||||
ap.add_argument('--check', action='store_true',
|
||||
help='assert playerId 20801 == Cristiano Ronaldo, 94')
|
||||
ap.add_argument('--top', type=int, default=0)
|
||||
args = ap.parse_args()
|
||||
|
||||
pid = args.pid or find_pid()
|
||||
mem = Mem(pid)
|
||||
maps = read_maps(pid)
|
||||
sys.stderr.write("pid %d, %d readable regions\n" % (pid, len(maps)))
|
||||
|
||||
pools = find_name_pools(mem, maps)
|
||||
if not pools:
|
||||
raise SystemExit("no player-name pool found -- is the game past the "
|
||||
"main menu with the player DB loaded?")
|
||||
for lo, hi, n in pools:
|
||||
sys.stderr.write("name pool 0x%x-0x%x %d name strings\n" % (lo, hi, n))
|
||||
|
||||
found = scan_entries(mem, maps, pools)
|
||||
addrs, rows = build(found)
|
||||
if not rows:
|
||||
raise SystemExit("index table not found")
|
||||
|
||||
strides = Counter(addrs[i + 1] - addrs[i] for i in range(len(addrs) - 1))
|
||||
sys.stderr.write("index table 0x%x-0x%x %d entries strides=%s\n"
|
||||
% (addrs[0], addrs[-1], len(rows), strides.most_common(3)))
|
||||
ranks = [r['rank'] for r in rows]
|
||||
sys.stderr.write("rank %d..%d (%d unique) playerIds %d unique rating %d..%d\n"
|
||||
% (min(ranks), max(ranks), len(set(ranks)),
|
||||
len(set(r['playerId'] for r in rows)),
|
||||
min(r['rating'] for r in rows),
|
||||
max(r['rating'] for r in rows)))
|
||||
|
||||
if args.top:
|
||||
for r in rows[:args.top]:
|
||||
print('%6d %2d %s' % (r['playerId'], r['rating'],
|
||||
'|'.join([r['firstName'], r['lastName'],
|
||||
r['commonName']])))
|
||||
return 0
|
||||
|
||||
by_id = {r['playerId']: r for r in rows}
|
||||
ok = True
|
||||
cr = by_id.get(RONALDO_ID)
|
||||
if cr is None:
|
||||
sys.stderr.write("ANCHOR FAIL: playerId %d absent\n" % RONALDO_ID)
|
||||
ok = False
|
||||
else:
|
||||
got = '|'.join([cr['firstName'], cr['lastName'], cr['commonName']])
|
||||
sys.stderr.write("anchor: playerId %d -> %r rating %d\n"
|
||||
% (RONALDO_ID, got, cr['rating']))
|
||||
if cr['rating'] != RONALDO_RATING or got != RONALDO_NAME:
|
||||
sys.stderr.write("ANCHOR FAIL: expected %r / %d\n"
|
||||
% (RONALDO_NAME, RONALDO_RATING))
|
||||
ok = False
|
||||
|
||||
with open(args.out, 'w') as fh:
|
||||
json.dump(rows, fh, ensure_ascii=False, indent=1)
|
||||
sys.stderr.write("wrote %s (%d players)\n" % (args.out, len(rows)))
|
||||
|
||||
if args.check and not ok:
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
# STILL MISSING: position / nationality / teamId / the six attributes.
|
||||
#
|
||||
# They are NOT in the index table above. Two leads were located live and are
|
||||
# recorded here so the next pass does not have to re-find them:
|
||||
#
|
||||
# (1) Materialised FUT card records. In the 40 MB heap region at 0x0b63b0000
|
||||
# the squad's resolved cards sit at a 0x180 stride, e.g. 0x0b840c2c0 =
|
||||
# Lewandowski and 0x0b840c440 = Luis Suarez. Layout relative to the record
|
||||
# word at +0x00c (0xf0, 0xf1 -- consecutive, an index):
|
||||
# +0x010..+0x02c six u32 attributes then two more u32
|
||||
# Lewandowski: 77 88 75 82 42 82 | 99 90
|
||||
# Suarez : 83 90 79 87 42 80 | 99 92
|
||||
# the last u32 is the overall rating (90 / 92, both correct for
|
||||
# FIFA 17), the 99 is constant across both.
|
||||
# +0x030 char[16] firstName ("Robert", "Luis")
|
||||
# +0x040 char[16] lastName ("Lewandowski", "Suárez")
|
||||
# Only ~a hundred of these exist process-wide -- they are built per card
|
||||
# that the client actually materialises, not a table. So this is a
|
||||
# VERIFICATION oracle for attribute values, not a bulk source.
|
||||
#
|
||||
# (2) A 32-byte-stride keyed table in the 238 MB heap region 0x37440000..,
|
||||
# seen at 0x42e8dbe8, carrying u64 fields keyed by playerId:
|
||||
# 20801 -> 27, 94, 77
|
||||
# 41236 -> 25, 90, 80 (41236 = Zlatan Ibrahimovic, rating 90)
|
||||
# The rating column is right in both rows; the 27/25 and 77/80 columns were
|
||||
# NOT identified. A 32-bit hash is interleaved in the high dword of a
|
||||
# rotating slot, so it is a hash container, not a flat array. Worth one
|
||||
# focused pass.
|
||||
#
|
||||
# Neither of these was pushed to completion in the session that wrote this file.
|
||||
# Do not cite them as resolved.
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user