funman300/OpenFUT
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(market): isolated two-identity SOLD-row A/B harness (staging only, not promoted)

Browse Source 
Static RE exhausted CardsDLL on the one open question: for a closed row
IS_GLOW = (bidState != none) and INBOX = (bidState in {highest, buyNow}), so
closed/highest and closed/buyNow are BIT-IDENTICAL natively. But bidState is
published to the movie verbatim as YOURBID, so the FUT ActionScript CAN separate
them. This builds the controlled experiment that asks the client which one it
treats as the seller's sale.

PRODUCTION SAFETY IS THE FIRST CONCERN
New module openfut-utas-host/src/sold_experiment.rs. Every knob is OFF unless its
env var is set, an unrecognised value is OFF rather than a default token (silently
picking one would fabricate the answer being measured), and the host logs a startup
banner naming the active variant so a staging capture can never be mistaken for a
production one. With no env set, /tradePile and /trade/status emit only real active
auctions (the Fix A invariant) and counts still report sold: 0. The entire existing
test suite now passes SoldExperiment::OFF explicitly, making it a regression guard.

  OPENFUT_FIFA17_SOLD_EXPERIMENT      = highest | buyNow   (else OFF)
  OPENFUT_FIFA17_SOLD_COINS_PROCESSED = 1                  (else 0)
  OPENFUT_FIFA17_SOLD_COUNT_MODE      = active_plus_sold    (else active)

WHAT THE EXPERIMENT PROJECTS
Uncleared sold listings appear in /tradePile and /trade/status as tradeState
"closed" with the token under test and currentBid = the sale price; counts report
the real sold tally. There is ONE record builder, so the A/B changes only what is
passed into it, and a test asserts that EXACTLY ONE field differs between the two
variants -- without that control the client's reaction is not attributable to the
token and the whole experiment is void. coinsProcessed (Flash COINS_AWARDED) varies
independently so the third pass cannot be confounded with the first.

CLEAR-SOLD, PE-PROVEN
New EconomyRoute::MarketClearSold for DELETE .../trade/sold, classified BEFORE the
generic trade cancel arm -- a `sold` tail carries no id, so the cancel handler would
have parsed nothing and acked while clearing nothing. Builder 0x1801647c0 emits
"/sold" when the tradeId field is zero and "/%lld" otherwise; the client calls it
RemoveAllSoldFromTradePile. New market-store column cleared_at records the seller's
acknowledgement SEPARATELY from the sale, so clearing can never be mistaken for
re-settling: it is presentation only, moves no coins and no ownership, and is
idempotent for client retries.

FOUND AND FIXED A LATENT STORE BUG
Adding a column via the additive ALTER path immediately after CREATE TABLE in the
same open() desynced sqlx's per-connection schema cache: a fresh store then read a
12-column row while metadata said 13, panicking a pool worker with an index
out-of-bounds and silently returning zero listings. Declaring cleared_at in
CREATE_LISTINGS fixes it; the ALTER now only serves pre-existing stores. This would
have bitten the next column too.

STAGING, WITHOUT TOUCHING PRODUCTION
The client learns the UTAS base from BLAZE (blaze_responder_v3b.py:646 hardcodes
:8099), and it dials that port directly, so redirecting UTAS means changing Blaze or
port 8099 -- both production. 10.10.0.121 is unreachable. The compliant path is a
parallel stack on spare ports plus a one-line change to the CLIENT's own config:
  * scripts/sold-staging-up.py / sold-staging-down.py -- staging Core 18081,
    utas-host 8299, Blaze 42327/42330/42331 advertising :8299, two seeded identities,
    own DBs under /home/alex/openfut-sold-staging/. Patches a COPY of the Blaze
    responder and asserts every substitution applied, so a silent no-op cannot leave
    it pointing at production. Kills only recorded pids whose cmdline contains the
    staging dir (openfut-utas-host matches BOTH, so pkill-by-pattern is banned).
  * docs/SOLD_STAGING_RUNBOOK.md -- the exact client change and its revert.
  * src/bin/staging_sell.rs -- the synthetic Buyer B, running the REAL settlement
    (CoreEconomy::settle_sale) then mark_sold. Settle-first ordering: a failure
    leaves the listing live with nothing moved. Refuses any path containing
    openfut-promotion or the production ports.
  * scripts/sold-wire-check.py -- proves the whole flow headless before any operator
    time is spent.

WIRE CHECK: 35/35 PASS on the canonical 150-coin sale. Seller 1,000 -> 1,143 (fee 7,
proceeds 143), buyer 20,000 -> 19,850, ownership transferred, exactly ONE
authoritative instance, economy shrank by exactly the fee. Sold row: closed,
currentBid 150, expires 0, twelve atoms, counts sold 1 / selling 0, /trade/status
agreeing. Variant B differs only in bidState and coinsProcessed. Clear: 200 {}, row
gone, counts.sold 0, no coins moved, buyer keeps the item, second clear a safe no-op.

Gates: 104 host lib tests (+9), all 7 host targets green, clippy clean, zero fmt
diffs in the new code. Settlement candidate unchanged. NOT PROMOTED.

Production untouched: prod-host pid 3631953 uptime 2h44m restarts=0, coins and
/tradePile unchanged, nothing under /home/alex/openfut-promotion/state/ opened.

The A/B itself is NOT yet run: it needs a real FIFA client, which is operator work.
This commit is contained in:
funman300
2026-08-18 02:14:18 +00:00
parent 571c5f9261
commit 468bc0fba9
13 changed files with 2880 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/SOLD_STAGING_RUNBOOK.md
+293
View File
@@ -0,0 +1,293 @@
# SOLD-row A/B experiment — isolated FIFA-17 staging runbook
Operator procedure for driving the seller-facing **SOLD row** experiment with a real
FIFA 17 client against a **completely separate** staging stack, while the frozen P1
production stack keeps running untouched.
The experiment asks one question the PE cannot answer: for a `closed` trade-pile row,
does the FUT ActionScript front end treat `bidState:"highest"` or `bidState:"buyNow"`
as *"you sold this"*? Both are bit-identical to every native CardsDLL consumer, but
`bidState` is published to the movie verbatim as `YOURBID`, so only the client can
say. See `openfut-utas-host/src/sold_experiment.rs`.
---
## 1. What the two scripts do
| Script | Purpose |
| --- | --- |
| `scripts/sold-staging-up.py` | Brings up the **whole** staging stack (Core + utas-host + Blaze), seeds two identities, prints the client config block. One entry point. |
| `scripts/sold-staging-down.py` | Stops **exactly** the processes the up script recorded, proves the staging ports are free, and proves production is still alive. |
```bash
cd /home/alex/OpenFUT
# variant A — the sold row is closed / highest
python3 scripts/sold-staging-up.py --variant highest
# variant B — same state, only bidState (and optionally coinsProcessed) differ
python3 scripts/sold-staging-up.py --variant buyNow --coins-processed 1 \
--count-mode active_plus_sold
# projection disabled: identical to production behaviour (control run)
python3 scripts/sold-staging-up.py --variant off
python3 scripts/sold-staging-down.py # stop, keep dbs/logs as evidence
python3 scripts/sold-staging-down.py --purge # stop and delete the staging dir
```
To flip A → B, run `sold-staging-down.py` then `sold-staging-up.py --variant buyNow`.
The staging databases are throwaway: a fresh `up` deletes the previous ones and
reseeds, so each variant run starts from a known state. Never edit a live stack's
environment in place — the banner in `logs/utas-host.log` is the only record of which
variant produced a capture, and it is written at startup.
---
## 2. Staging port block and state
Everything lives under `/home/alex/openfut-sold-staging/` (override with `--dir` or
`OPENFUT_SOLD_STAGING_DIR`).
| Service | Bind | Notes |
| --- | --- | --- |
| staging Core | `127.0.0.1:18081` | loopback only; the client never talks to Core |
| staging utas-host | `0.0.0.0:8299` | the UTAS the client reaches |
| staging Blaze redirector | `0.0.0.0:42327` | TLS; receives EA `:10041` and `:42230` |
| staging Blaze main | `0.0.0.0:42330` | receives EA `:42127` |
| staging Blaze nucleus | `0.0.0.0:42331` | local OAuth stub |
| dead Python upstream | `127.0.0.1:8399` | **must stay unbound** (see §5) |
The redirector is on `42327`, not the "obvious" `42227`: that port is permanently
held by `openfut-redirector-host` (pid 862419).
| State file | Path |
| --- | --- |
| Core sqlite | `staging-core.db` |
| market sqlite | `staging-market.db` |
| pile sqlite | `staging-pile.db` |
| identity store | `staging-identity.json` |
| clientdata blobs | `staging-clientdata.json` |
| content pack | `content/fifa17-production-cards.json` |
| identity catalog | `content/fifa17-production-catalog.json` |
| patched Blaze | `blaze/blaze_responder_staging.py` |
| process manifest | `manifest.json` |
| logs | `logs/{core,utas-host,blaze,blaze-responder}.log` |
Seeded identities (direct SQL against the schema Core migrates for itself):
* **Seller A** — profile `prof-seller-a-cage`, club `club-seller-a-cage`, username
`CAGE`, `game_id = fifa17`, 1 000 coins, 11 starters in a canonical squad, plus
**one disposable item** (`owned-a-disposable`, card `fifa17_232273`, Nelson Atiagli
LB 51) — that is the card to list and sell during the experiment.
Seller A carries `game_id = fifa17`, so the retail client (which sends
`X-OpenFUT-Game: fifa17`) resolves to this profile and can log in as persona
`33068179`.
* **Buyer B** — profile `prof-buyer-b`, club `club-buyer-b`, 20 000 coins, parked on
its own `game_id = fifa17-buyer-b`. Core is single-profile-per-game, so this keeps
Buyer B from ever shadowing Seller A as the active `fifa17` profile while still
being reachable by club id for a settlement.
---
## 3. The client change — the only thing the operator touches
The FIFA client learns its **UTAS** base URL from Blaze, not from `openfut.cfg`:
`blaze_responder_v3b.py` builds `UTAS_BASE = "http://%s:8099/" % _ADVERTISE` with the
port **hardcoded**. The up script therefore copies the responder into the staging dir
and rewrites that literal to `:8299`. **Pointing the client at staging Blaze is
sufficient to move UTAS too** — there is no UTAS line in `openfut.cfg`.
On the FIFA client machine **10.10.0.105**, file
`"/mnt/games/FIFA 17/openfut.cfg"`:
```bash
# back it up FIRST
cd "/mnt/games/FIFA 17"
cp openfut.cfg openfut.cfg.prod
```
Set these **three** lines:
```
host=10.10.0.120
blaze_redirector_port=42327
blaze_main_port=42330
```
Leave `https_port=8443` **unchanged** — that is Bridge, which holds no economy
state. The resulting file is:
```
host=10.10.0.120
https_port=8443
blaze_redirector_port=42327
blaze_main_port=42330
```
Then **relaunch the FIFA 17 client.** A client that is already running caches its
UTAS session (SID) in memory and will not re-auth against a different stack; the
symptom is the dialog *"An error occurred downloading the FUT Squad Update"* with
**zero** requests in the staging host log. "The operator is at the main menu" is not
the same as "the client disconnected".
### Revert to production
```
host=10.10.0.120
https_port=8443
blaze_redirector_port=42127
blaze_main_port=42130
```
or simply `cp openfut.cfg.prod openfut.cfg` — then **relaunch the client again**.
These are the values the file holds today; the staging scripts never write to
10.10.0.105, so this revert is the *only* client-side change to undo.
---
## 4. Running the experiment
1. `python3 scripts/sold-staging-up.py --variant highest`
2. Apply §3 to the client and relaunch it. Confirm it reaches the FUT hub;
`logs/utas-host.log` should show `route=auth status=200 sid_opened=true`.
3. List the disposable item on the transfer market from the client.
4. Complete the sale (Buyer B side) through the real settlement path.
5. Observe the seller's Transfer List: which bucket the row lands in, what the
counter says, and which request the client issues to clear it. Capture both the
screen and `logs/utas-host.log`.
6. `python3 scripts/sold-staging-down.py`
7. `python3 scripts/sold-staging-up.py --variant buyNow`, relaunch the client, and
repeat steps 3–5. The two runs differ **only** in `bidState` (and
`coinsProcessed`/`count` if those flags are used), which is what makes the client's
reaction attributable.
8. When done: `sold-staging-down.py` and revert the client per §3.
`DELETE /ut/delete/game/fifa17/trade/sold` clears the sold rows (logged as
`route=market-clear-sold cleared=N`) — that is the verb the client is expected to
issue from the *Clear Sold* affordance, and seeing whether it does is part of the
observation.
---
## 5. Isolation: what guarantees production is untouched
Production frozen P1 must keep running throughout. The scripts enforce, not assume:
* **Hard port deny-list.** `8099 8199 18080 8443 42127 42130 42131 4216 8080 8081
8094` are refused before every bind and before every HTTP request the scripts make.
Each staging port is also proven free first, and bring-up aborts before launching
anything if one is not.
* **Hard path deny-list.** Every filesystem path goes through `safe_path()`, which
refuses `/home/alex/openfut-promotion/state/` (the live `prod-core.db`,
`prod-market.db`, `prod-pile.db`, `prod-identity.json`). Nothing there is ever
opened — not even read. The FIFA17 content the staging stack loads is copied from
`/home/alex/openfut-post-p1/staging/emit/content/`, a build tree outside that
directory.
* **The Python fallback fails closed.** `OPENFUT_UTAS_PYTHON_URL` points at
`http://127.0.0.1:8399`, where **nothing listens**, so a fallback to the Python
oracle surfaces as a visible connection error instead of silently serving the
production oracle on `8199`.
* **Binaries are copied into the staging dir and run from there.** A later
`cargo build` cannot change what staging is running, and — more importantly — every
staging process's `/proc/<pid>/cmdline` provably contains the staging directory,
which production's never can.
* **Teardown kills by recorded pid only, never by pattern.** `openfut-utas-host` and
`openfut-core` each name *two* live processes on this machine. `sold-staging-down.py`
reads pids from `manifest.json`, re-reads `/proc/<pid>/cmdline` and **refuses** to
signal anything whose cmdline does not contain the staging dir; it refuses the known
production pids explicitly; and it signals only the process group the up script
created (`pgid == pid`, via a new session). There is no `pkill`/`pgrep` anywhere.
* **The Blaze patch cannot silently no-op.** Six substitutions are applied to the
copy, each anchored to a whole assignment line and each required to match **exactly
once**; the file is then re-read from disk and every value re-checked, including an
explicit assertion that `UTAS_BASE` no longer contains `:8099`. A responder that
changed shape aborts bring-up rather than running half-patched and pointing at
production. The four protocol values are `REDIR_PORT`, `BLAZE_PORT`,
`NUCLEUS_PORT`, `UTAS_BASE`; the two extra ones are the responder's hardcoded
`LOG = "/tmp/blaze_responder.log"` and `RXDIR = "/tmp/blaze_rx"`, redirected into
the staging dir so a capture can never be ambiguous about which stack wrote it.
* **Production liveness is asserted** at preflight, at the end of bring-up, and after
teardown (`prod utas-host` pid 3631953, `prod Core` pid 3374264).
* **Nothing on 10.10.0.105 is written.** The scripts only *print* the config block;
the operator edits it by hand.
### Production services deliberately shared, read-only
Staging Blaze advertises the **production** auxiliary endpoints, exactly the strings
production Blaze advertises:
| Advertised to client | Value | Why sharing is safe |
| --- | --- | --- |
| roster XML / roster URL | `10.10.0.120:8081` | static roster update XML; **no economy state**. Hardcoded as `ROSTER_HOST = "%s:8081" % _ADVERTISE` in the responder. |
| `FIFA_POW_CONTENT_SERVER_URL` | `10.10.0.120:8085` | static POW content; no economy state (`POW_CONTENT_HOST`) |
| `FIFA_POW_URL` / nucleus proxy | `10.10.0.120:8094` | POW API, only emitted when `FUT_POW=1`, which staging leaves unset (`POW_HOST`) |
| LSX layer-1 responder | `:4216` | persona/session handshake for the same persona `33068179`; no economy state |
The staging stack **never connects** to any of these itself — it only hands the client
the same strings production hands it. Every service that owns economy state (UTAS,
Core, market/pile/identity) is duplicated on staging ports against staging files.
Because those aux services are shared, **do not run a staging session and a
production session simultaneously**: one client at a time.
`heat2` (TDF codec) and `fut_account` (the single source of truth for persona
`33068179` / `CAGE`, shared with LSX and the oracle) are imported read-only from
`fifa17-recon/tools/` via `PYTHONPATH` rather than copied, so the staging Blaze
asserts a **byte-identical** identity to what LSX already asserts. Cross-layer
identity consistency is the constraint; a divergent copy would break login.
---
## 6. Verifying a live stack by hand
All read-only, and none of it touches a production port:
```bash
S=/home/alex/openfut-sold-staging
# which variant is live
grep sold-experiment $S/logs/utas-host.log
# staging UTAS answers, from staging state
python3 - <<'PY'
import http.client, json
c = http.client.HTTPConnection("127.0.0.1", 8299, timeout=5)
c.request("GET", "/ut/game/fifa17/tradePile/counts",
headers={"X-OpenFUT-Game": "fifa17"})
print(c.getresponse().read().decode())
PY
# the patched Blaze points at staging UTAS, not production
grep -nE '^(UTAS_BASE|REDIR_PORT|BLAZE_PORT|NUCLEUS_PORT) = ' \
$S/blaze/blaze_responder_staging.py
# exactly six lines differ from the original responder
diff fifa17-recon/tools/blaze_responder_v3b.py \
$S/blaze/blaze_responder_staging.py
# no staging process has a handle on production state
for p in $(python3 -c "import json;print(' '.join(str(x['pid']) for x in json.load(open('$S/manifest.json'))['processes']))"); do
echo "pid $p: $(sudo ls -l /proc/$p/fd | grep -c openfut-promotion/state) prod-state handles"
done
# production is still up
ps -o pid=,cmd= -p 3631953
```
Never `curl` port 8099/8199/18080 to "compare with production" — those ports are on
the deny-list. Independence is proven structurally (separate pids, separate ports,
separate files, zero shared handles), not by poking the live stack.
---
## 7. Troubleshooting
| Symptom | Cause / fix |
| --- | --- |
| `REFUSING to start -- these staging ports are not free` | something else holds a staging port. Nothing was launched. Free it, or edit the port block at the top of `sold-staging-up.py`. |
| `a previous staging stack is STILL UP` | run `sold-staging-down.py` first. The up script never steps on a live stack. |
| `blaze patch <NAME> applied 0 times` | `blaze_responder_v3b.py` changed shape. Fix the regex in `blaze_patches()`; do **not** disable the assertion. |
| `seed card ids are not resolvable` | the emitted content in `CONTENT_SRC` no longer contains a seeded `card_id`. Re-point `CONTENT_SRC` or update the seed ids — both memberships (content pack **and** identity catalog) are required. |
| client shows *"error occurred downloading the FUT Squad Update"*, staging host log shows **no request at all** | the failure is upstream of UTAS. Almost always a client that was not relaunched after the config change (§3). Check the *absence* of requests in `logs/utas-host.log` before touching any code. |
| `<name> pid N is alive but its cmdline does NOT contain <staging dir>` | pid reuse or a stale manifest. Teardown refuses rather than guessing — identify the process by hand and stop it by pid. |
| staging host log shows connection errors to `127.0.0.1:8399` | expected and intended: a UTAS route fell through to the Python fallback, which is deliberately dead. Fix the route; do **not** point the fallback at `8199`. |