funman300/OpenFUT
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(fifa17): complete kit stats, restore red squad tests, unrot prod gate

Browse Source 
Four defects found by running the suites and the staging lifecycle end to end
after the kit milestone.

1. club-stats kits were half-implemented. The global `kits` counter was real
   but `kitsHome`/`kitsAway` and every per-team `kits` bucket stayed hardcoded
   0, so the same screen reported two owned kits and zero home/away kits.
   `kits` is a total with a family split, exactly like players/playersGold and
   staff/staffManager. The split key is `fcc_kitcards.assetid`: 14 is the home
   family and 15 the away family, verified across all 1482 rows of the kit
   table (assetid 14 covers exactly the 63xxxxx carddbids, 828 rows; assetid 15
   exactly the 64xxxxx ones, 654 rows; no exceptions either way).
   ClubStatInput now carries `asset_id`, and a kit buckets onto the team that
   wears it -- including a team the club owns no player from, the normal case
   for a kit won from a pack. The host reads both from the catalog through new
   NON-MINTING accessors: `resolve`/`resolve_kit` allocate a wire id, which a
   read-only stats query must never do as a side effect.

2. host_test.rs had 10 tests red since the squad-manager work (25f4ad1 /
   d37a9d5); 56bd9dd updated the squad_projection integration test and stopped
   there. `put_body` hardcoded the captured manager ref 100000427 into EVERY
   save, including tests with no manager fixture, so each one was refused with
   `unresolved_wire_ids` -- the tests were reporting a real invariant against a
   fixture that could not satisfy it. The manager is now an explicit
   `Option<i64>` per test, and FakeCore models Core's manager persistence
   instead of inheriting the "not implemented" default that 502'd every save.
   Added the coverage whose absence let this rot: a manager assignment
   round-trips as a Core owned id, a later save without one CLEARS it, and an
   unowned manager ref refuses the whole save with nothing committed.

3. `club_route_maps_query_and_shapes_core_items` pinned `offset`/`limit`
   forwarding to Core, which the kit commit deliberately replaced with
   host-side pagination. It only ever passed because FakeCore ignored the
   window -- against a real Core, `start=10` over a one-item club was always an
   empty page. Retargeted to the real contract (Core gets semantic filters and
   NO window) plus a new test that the window is applied locally after
   filtering, which the old fake made vacuous.

4. The staging lifecycle scripts identified production by hardcoded pids, so a
   correct teardown FATAL'd: production moved into containers and pids
   3631953/3374264 died with a container restart days ago. A pinned pid rots
   into the worst of both worlds -- a kill-refusal gate that no longer names
   any real production process, and a liveness gate that fails a healthy
   teardown. New shared `scripts/openfut_production.py` resolves production
   pids AND published ports from the container runtime at the moment they are
   needed, refuses to signal anything it cannot see, and proves production is
   the same processes serving the same ports before and after. Both lifecycle
   scripts use it, which also closed a real gap: port 8085 is published by
   openfut-fut-backend but was missing from the up script's forbidden list, so
   staging could have bound a production port.

Also fixes the economy differential, red because `complete_match` unlocks
achievements in the same transaction that pays the match reward -- a deliberate
Core feature the Python oracle has no counterpart for. `rust WIN +400` asserted
that progression did not exist; it now asserts the delta is the 400 match reward
plus exactly the achievements the match unlocked, read from Core's own report.
This commit is contained in:
funman300
2026-08-21 04:10:02 +00:00
parent db743ffd1f
commit 3442eac6f0
7 changed files with 568 additions and 107 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/openfut_production.py
+143
View File
@@ -0,0 +1,143 @@
"""Runtime identity of the PRODUCTION FIFA-17 stack, shared by the staging
lifecycle scripts.
This exists because both `sold-staging-up.py` and `sold-staging-down.py` need the
same answer to the same safety question -- "what is production right now, and is
it still healthy?" -- and two hand-maintained copies of a safety gate is two
chances to rot.
Production runs in containers, so its pids are NOT stable facts: every pid changes
when a container is restarted. A hardcoded pid list decays into the worst of both
worlds -- a kill-refusal gate that guards nothing (the real production pids are no
longer in it) and a liveness gate that fails a perfectly good teardown (the pids it
does list are long dead). So pids and published ports are both resolved from the
container runtime at the moment they are needed.
"""
from __future__ import annotations
import subprocess
# Production containers. Anything running inside one of these is production.
PROD_CONTAINERS = ("openfut-fut-backend", "openfut-bridge-1", "openfut-core-1")
# Reserved ports: staging may never bind one of these, whether or not it is
# currently published. 8199 (Python oracle) and 18080 (Core) belonged to the
# retired host-process deployment and are kept so an old port map cannot be
# silently reused by staging.
PROD_PORTS = frozenset(
{8080, 8081, 8085, 8094, 8099, 8199, 8443, 4216, 18080, 42127, 42130, 42131}
)
class ProductionError(RuntimeError):
"""Production could not be observed, or is not healthy."""
def _inspect(container: str, template: str) -> str:
"""One `docker inspect -f` field.
Any failure is fatal by design: a script that cannot see production must
refuse to signal anything rather than assume the best.
"""
try:
result = subprocess.run(
["docker", "inspect", "-f", template, container],
capture_output=True,
text=True,
timeout=30,
check=False,
)
except (OSError, subprocess.SubprocessError) as exc:
raise ProductionError(
f"cannot inspect production container {container!r}: {exc}"
) from exc
if result.returncode != 0:
detail = result.stderr.strip() or f"docker exited {result.returncode}"
raise ProductionError(
f"cannot inspect production container {container!r}: {detail}"
)
return result.stdout.strip()
def listening_ports() -> set[int]:
"""Ports in state LISTEN, from the kernel socket table.
A trial bind() would report EADDRINUSE for a stopped server's TIME_WAIT
sockets and so wrongly claim a port is still served.
"""
ports: set[int] = set()
for path in ("/proc/net/tcp", "/proc/net/tcp6"):
try:
with open(path) as fh:
next(fh, None) # header
for line in fh:
fields = line.split()
if len(fields) < 4 or fields[3] != "0A": # TCP_LISTEN
continue
ports.add(int(fields[1].rsplit(":", 1)[1], 16))
except OSError:
continue
return ports
class ProductionState:
"""A snapshot of production as the container runtime reports it."""
__slots__ = ("pids", "published")
def __init__(self, pids: dict[int, str], published: dict[int, str]) -> None:
self.pids = pids
self.published = published
def describe(self) -> list[str]:
return [f"{what} pid {pid}" for pid, what in sorted(self.pids.items())]
def assert_serving(self) -> None:
"""Every port production publishes must actually be listening."""
listening = listening_ports()
silent = sorted(port for port in self.published if port not in listening)
if silent:
raise ProductionError(
"production port(s) no longer listening: "
+ ", ".join(f"{port} ({self.published[port]})" for port in silent)
)
def assert_unchanged(self, before: "ProductionState") -> None:
"""Production must be the same processes serving the same ports."""
if self.pids != before.pids:
raise ProductionError(
f"production pids CHANGED: before={before.pids}, after={self.pids}"
)
if set(self.published) != set(before.published):
raise ProductionError(
"production published ports CHANGED: "
f"before={sorted(before.published)}, after={sorted(self.published)}"
)
def production_state() -> ProductionState:
"""Resolve production's current pids and published ports, proving every
production container is running."""
pids: dict[int, str] = {}
published: dict[int, str] = {}
for container in PROD_CONTAINERS:
status = _inspect(container, "{{.State.Status}}")
if status != "running":
raise ProductionError(
f"production container {container} is {status!r}, not running"
)
pid = int(_inspect(container, "{{.State.Pid}}") or 0)
if pid <= 0:
raise ProductionError(
f"production container {container} is running but reports no pid"
)
pids[pid] = f"prod {container}"
ports = _inspect(
container,
"{{range $port, $bindings := .NetworkSettings.Ports}}"
"{{range $bindings}}{{.HostPort}} {{end}}{{end}}",
)
for field in ports.split():
published[int(field)] = container
return ProductionState(pids, published)
scripts/sold-staging-down.py
+31 -46
View File
@@ -10,10 +10,12 @@ ones. So there is no pattern matching here at all:
* before any signal, /proc/<pid>/cmdline is read and MUST contain the staging
directory -- production's cmdline never can, because staging runs binaries
copied into that directory;
* the known production pids are refused explicitly, as a second gate;
* production's CURRENT pids, resolved from the container runtime, are refused
explicitly as a second gate;
* only the process GROUP the up script created (pgid == pid, via
start_new_session) is signalled, so a responder thread/child cannot be orphaned;
* afterwards every staging port is proven free and production is proven alive.
* afterwards every staging port is proven free, and production is proven to be
the same running containers serving the same ports as before.
python3 scripts/sold-staging-down.py
python3 scripts/sold-staging-down.py --purge # also delete the staging dir
@@ -29,20 +31,20 @@ import signal
import sys
import time
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from openfut_production import ( # noqa: E402
PROD_PORTS,
ProductionError,
listening_ports,
production_state,
)
DEFAULT_STAGING_DIR = "/home/alex/openfut-sold-staging"
FORBIDDEN_PATHS = ("/home/alex/openfut-promotion/state",)
# Production processes that MUST be alive before and after this script runs. These
# two are the ones the batch contract names, and they live in the host pid view.
PROD_PIDS = {3631953: "prod utas-host", 3374264: "prod Core"}
# Reported but not gated: container pids change when the operator restarts the
# container, and a stale entry here would turn a successful teardown into a FATAL.
PROD_PIDS_INFO = {2090886: "prod blaze", 2091170: "prod python oracle",
2090888: "prod pow"}
PROD_PORTS = (8099, 8199, 18080, 8443, 42127, 42130, 42131, 4216, 8080, 8081, 8094)
class Fatal(RuntimeError):
class Fatal(ProductionError):
pass
@@ -87,37 +89,17 @@ def cmdline_of(pid: int) -> str:
return ""
def listening_ports() -> set[int]:
"""Ports in state LISTEN, from the kernel socket table. A trial bind() would
report EADDRINUSE for a stopped server's TIME_WAIT sockets and wrongly claim the
teardown failed."""
ports: set[int] = set()
for path in ("/proc/net/tcp", "/proc/net/tcp6"):
try:
with open(path) as fh:
next(fh, None) # header
for line in fh:
fields = line.split()
if len(fields) < 4 or fields[3] != "0A": # TCP_LISTEN
continue
ports.add(int(fields[1].rsplit(":", 1)[1], 16))
except OSError:
continue
return ports
def port_free(port: int) -> bool:
return port not in listening_ports()
def stop_one(rec: dict, staging_dir: str) -> str:
def stop_one(rec: dict, staging_dir: str, prod_pids: dict[int, str]) -> str:
"""Stop exactly one recorded process. Returns a human-readable outcome."""
name, pid = rec["name"], int(rec["pid"])
known_prod = {**PROD_PIDS, **PROD_PIDS_INFO}
if pid in known_prod:
if pid in prod_pids:
raise Fatal(
f"manifest entry {name} names PRODUCTION pid {pid} ({known_prod[pid]}). "
f"manifest entry {name} names PRODUCTION pid {pid} ({prod_pids[pid]}). "
"REFUSING to signal anything from this manifest."
)
if not pid_alive(pid):
@@ -192,8 +174,14 @@ def main() -> int:
)
step(f"variant : {manifest.get('variant')}")
# Resolved BEFORE anything is signalled: the refusal gate below is only
# meaningful if it knows production's pids as they are right now.
before = production_state()
for line in before.describe():
step(f"production : {line}")
for rec in manifest.get("processes", []):
ok(stop_one(rec, staging_dir))
ok(stop_one(rec, staging_dir, before.pids))
banner("PROVE STAGING IS GONE")
ports = manifest.get("ports", {})
@@ -214,16 +202,13 @@ def main() -> int:
ok("no recorded staging process is alive")
banner("PROVE PRODUCTION IS STILL UP")
dead = [f"{what} pid {pid}" for pid, what in PROD_PIDS.items()
if not pid_alive(pid)]
for pid, what in PROD_PIDS.items():
if pid_alive(pid):
ok(f"{what} pid {pid} alive")
if dead:
raise Fatal("production process(es) NOT alive: " + ", ".join(dead))
for pid, what in PROD_PIDS_INFO.items():
state = "alive" if pid_alive(pid) else "not found (informational only)"
step(f"{what} pid {pid} {state}")
after = production_state()
for line in after.describe():
ok(f"{line} alive")
after.assert_unchanged(before)
after.assert_serving()
ok(f"all {len(after.published)} published production ports still listening: "
+ ", ".join(str(port) for port in sorted(after.published)))
if args.purge:
shutil.rmtree(safe_path(staging_dir), ignore_errors=True)
@@ -243,7 +228,7 @@ def main() -> int:
print()
print(" then RELAUNCH the FIFA 17 client. See docs/SOLD_STAGING_RUNBOOK.md.")
return 0
except Fatal as exc:
except ProductionError as exc:
print(f"\nFATAL: {exc}", file=sys.stderr)
return 1
scripts/sold-staging-up.py
+38 -38
View File
@@ -60,16 +60,24 @@ import subprocess
import sys
import time
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from openfut_production import ( # noqa: E402
PROD_PORTS,
ProductionError,
ProductionState,
listening_ports,
production_state,
)
# --- fixed facts -------------------------------------------------------------------
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
# Production. Never bind, never connect, never open.
FORBIDDEN_PORTS = frozenset(
{8099, 8199, 18080, 8443, 42127, 42130, 42131, 4216, 8080, 8081, 8094}
)
# Production. Never bind, never connect, never open. Production's pids are NOT
# listed here: they live in containers and change on every restart, so they are
# resolved from the container runtime by openfut_production.production_state().
FORBIDDEN_PORTS = PROD_PORTS
FORBIDDEN_PATHS = ("/home/alex/openfut-promotion/state",)
PROD_PIDS = {3631953: "prod utas-host", 3374264: "prod Core"}
# The staging port block. One obvious place; every one of these is asserted free.
# 42227 (the first choice for the redirector) is permanently occupied by
@@ -178,7 +186,7 @@ def ok(msg: str) -> None:
print(f" [ OK ] {msg}")
class Fatal(RuntimeError):
class Fatal(ProductionError):
"""Anything that must abort bring-up loudly rather than degrade."""
@@ -200,31 +208,12 @@ def check_port_allowed(port: int, what: str) -> None:
raise Fatal(f"REFUSING: {what} port {port} is a PRODUCTION port")
def listening_ports() -> set[int]:
"""Every TCP port in state LISTEN in this network namespace, read straight from
the kernel socket table.
A trial bind() is the wrong test: after a server exits, its accepted sockets sit
in TIME_WAIT holding the same local port, so bind() reports EADDRINUSE for a
minute even though nothing is serving -- and every server here sets SO_REUSEADDR
and would bind fine. This mirrors `ss -ltn` (and host-lifecycle.sh's
hl_port_listening), which is the question actually being asked."""
ports: set[int] = set()
for path in ("/proc/net/tcp", "/proc/net/tcp6"):
try:
with open(path) as fh:
next(fh, None) # header
for line in fh:
fields = line.split()
if len(fields) < 4 or fields[3] != "0A": # TCP_LISTEN
continue
ports.add(int(fields[1].rsplit(":", 1)[1], 16))
except OSError:
continue
return ports
def port_free(port: int) -> bool:
"""A trial bind() is the wrong test: after a server exits, its accepted sockets
sit in TIME_WAIT holding the same local port, so bind() reports EADDRINUSE for a
minute even though nothing is serving -- and every server here sets SO_REUSEADDR
and would bind fine. listening_ports() mirrors `ss -ltn` (and
host-lifecycle.sh's hl_port_listening), which is the question actually asked."""
return port not in listening_ports()
@@ -271,14 +260,25 @@ def cmdline_of(pid: int) -> str:
return ""
_PROD_BASELINE: ProductionState | None = None
def assert_prod_alive(where: str) -> None:
for pid, what in PROD_PIDS.items():
if not pid_alive(pid):
raise Fatal(f"{what} pid {pid} is NOT alive at {where} -- stop and investigate")
ok(
f"production untouched at {where}: "
+ ", ".join(f"{what} pid {pid} alive" for pid, what in PROD_PIDS.items())
)
"""Prove production is untouched.
The first call records the baseline; every later call must observe the SAME
container pids publishing the SAME ports, and every published port must still
be listening. Pids are read from the container runtime each time because a
restarted container gets a new one.
"""
global _PROD_BASELINE
state = production_state()
state.assert_serving()
if _PROD_BASELINE is None:
_PROD_BASELINE = state
else:
state.assert_unchanged(_PROD_BASELINE)
ok(f"production untouched at {where}: " + ", ".join(state.describe()))
# --- HTTP ---------------------------------------------------------------------------
@@ -1091,7 +1091,7 @@ def main() -> int:
print_summary(lay, args.variant, args.coins_processed, args.count_mode,
args.roster_host, records)
return 0
except Fatal as exc:
except ProductionError as exc:
print(f"\nFATAL: {exc}", file=sys.stderr)
if started:
print(" rolling back the partial bring-up...", file=sys.stderr)