diff --git a/docs/SOLD_STAGING_RUNBOOK.md b/docs/SOLD_STAGING_RUNBOOK.md new file mode 100644 index 0000000..e87f32e --- /dev/null +++ b/docs/SOLD_STAGING_RUNBOOK.md @@ -0,0 +1,293 @@ +# SOLD-row A/B experiment — isolated FIFA-17 staging runbook + +Operator procedure for driving the seller-facing **SOLD row** experiment with a real +FIFA 17 client against a **completely separate** staging stack, while the frozen P1 +production stack keeps running untouched. + +The experiment asks one question the PE cannot answer: for a `closed` trade-pile row, +does the FUT ActionScript front end treat `bidState:"highest"` or `bidState:"buyNow"` +as *"you sold this"*? Both are bit-identical to every native CardsDLL consumer, but +`bidState` is published to the movie verbatim as `YOURBID`, so only the client can +say. See `openfut-utas-host/src/sold_experiment.rs`. + +--- + +## 1. What the two scripts do + +| Script | Purpose | +| --- | --- | +| `scripts/sold-staging-up.py` | Brings up the **whole** staging stack (Core + utas-host + Blaze), seeds two identities, prints the client config block. One entry point. | +| `scripts/sold-staging-down.py` | Stops **exactly** the processes the up script recorded, proves the staging ports are free, and proves production is still alive. | + +```bash +cd /home/alex/OpenFUT + +# variant A — the sold row is closed / highest +python3 scripts/sold-staging-up.py --variant highest + +# variant B — same state, only bidState (and optionally coinsProcessed) differ +python3 scripts/sold-staging-up.py --variant buyNow --coins-processed 1 \ + --count-mode active_plus_sold + +# projection disabled: identical to production behaviour (control run) +python3 scripts/sold-staging-up.py --variant off + +python3 scripts/sold-staging-down.py # stop, keep dbs/logs as evidence +python3 scripts/sold-staging-down.py --purge # stop and delete the staging dir +``` + +To flip A → B, run `sold-staging-down.py` then `sold-staging-up.py --variant buyNow`. +The staging databases are throwaway: a fresh `up` deletes the previous ones and +reseeds, so each variant run starts from a known state. Never edit a live stack's +environment in place — the banner in `logs/utas-host.log` is the only record of which +variant produced a capture, and it is written at startup. + +--- + +## 2. Staging port block and state + +Everything lives under `/home/alex/openfut-sold-staging/` (override with `--dir` or +`OPENFUT_SOLD_STAGING_DIR`). + +| Service | Bind | Notes | +| --- | --- | --- | +| staging Core | `127.0.0.1:18081` | loopback only; the client never talks to Core | +| staging utas-host | `0.0.0.0:8299` | the UTAS the client reaches | +| staging Blaze redirector | `0.0.0.0:42327` | TLS; receives EA `:10041` and `:42230` | +| staging Blaze main | `0.0.0.0:42330` | receives EA `:42127` | +| staging Blaze nucleus | `0.0.0.0:42331` | local OAuth stub | +| dead Python upstream | `127.0.0.1:8399` | **must stay unbound** (see §5) | + +The redirector is on `42327`, not the "obvious" `42227`: that port is permanently +held by `openfut-redirector-host` (pid 862419). + +| State file | Path | +| --- | --- | +| Core sqlite | `staging-core.db` | +| market sqlite | `staging-market.db` | +| pile sqlite | `staging-pile.db` | +| identity store | `staging-identity.json` | +| clientdata blobs | `staging-clientdata.json` | +| content pack | `content/fifa17-production-cards.json` | +| identity catalog | `content/fifa17-production-catalog.json` | +| patched Blaze | `blaze/blaze_responder_staging.py` | +| process manifest | `manifest.json` | +| logs | `logs/{core,utas-host,blaze,blaze-responder}.log` | + +Seeded identities (direct SQL against the schema Core migrates for itself): + +* **Seller A** — profile `prof-seller-a-cage`, club `club-seller-a-cage`, username + `CAGE`, `game_id = fifa17`, 1 000 coins, 11 starters in a canonical squad, plus + **one disposable item** (`owned-a-disposable`, card `fifa17_232273`, Nelson Atiagli + LB 51) — that is the card to list and sell during the experiment. + Seller A carries `game_id = fifa17`, so the retail client (which sends + `X-OpenFUT-Game: fifa17`) resolves to this profile and can log in as persona + `33068179`. +* **Buyer B** — profile `prof-buyer-b`, club `club-buyer-b`, 20 000 coins, parked on + its own `game_id = fifa17-buyer-b`. Core is single-profile-per-game, so this keeps + Buyer B from ever shadowing Seller A as the active `fifa17` profile while still + being reachable by club id for a settlement. + +--- + +## 3. The client change — the only thing the operator touches + +The FIFA client learns its **UTAS** base URL from Blaze, not from `openfut.cfg`: +`blaze_responder_v3b.py` builds `UTAS_BASE = "http://%s:8099/" % _ADVERTISE` with the +port **hardcoded**. The up script therefore copies the responder into the staging dir +and rewrites that literal to `:8299`. **Pointing the client at staging Blaze is +sufficient to move UTAS too** — there is no UTAS line in `openfut.cfg`. + +On the FIFA client machine **10.10.0.105**, file +`"/mnt/games/FIFA 17/openfut.cfg"`: + +```bash +# back it up FIRST +cd "/mnt/games/FIFA 17" +cp openfut.cfg openfut.cfg.prod +``` + +Set these **three** lines: + +``` +host=10.10.0.120 +blaze_redirector_port=42327 +blaze_main_port=42330 +``` + +Leave `https_port=8443` **unchanged** — that is Bridge, which holds no economy +state. The resulting file is: + +``` +host=10.10.0.120 +https_port=8443 +blaze_redirector_port=42327 +blaze_main_port=42330 +``` + +Then **relaunch the FIFA 17 client.** A client that is already running caches its +UTAS session (SID) in memory and will not re-auth against a different stack; the +symptom is the dialog *"An error occurred downloading the FUT Squad Update"* with +**zero** requests in the staging host log. "The operator is at the main menu" is not +the same as "the client disconnected". + +### Revert to production + +``` +host=10.10.0.120 +https_port=8443 +blaze_redirector_port=42127 +blaze_main_port=42130 +``` + +or simply `cp openfut.cfg.prod openfut.cfg` — then **relaunch the client again**. +These are the values the file holds today; the staging scripts never write to +10.10.0.105, so this revert is the *only* client-side change to undo. + +--- + +## 4. Running the experiment + +1. `python3 scripts/sold-staging-up.py --variant highest` +2. Apply §3 to the client and relaunch it. Confirm it reaches the FUT hub; + `logs/utas-host.log` should show `route=auth status=200 sid_opened=true`. +3. List the disposable item on the transfer market from the client. +4. Complete the sale (Buyer B side) through the real settlement path. +5. Observe the seller's Transfer List: which bucket the row lands in, what the + counter says, and which request the client issues to clear it. Capture both the + screen and `logs/utas-host.log`. +6. `python3 scripts/sold-staging-down.py` +7. `python3 scripts/sold-staging-up.py --variant buyNow`, relaunch the client, and + repeat steps 3–5. The two runs differ **only** in `bidState` (and + `coinsProcessed`/`count` if those flags are used), which is what makes the client's + reaction attributable. +8. When done: `sold-staging-down.py` and revert the client per §3. + +`DELETE /ut/delete/game/fifa17/trade/sold` clears the sold rows (logged as +`route=market-clear-sold cleared=N`) — that is the verb the client is expected to +issue from the *Clear Sold* affordance, and seeing whether it does is part of the +observation. + +--- + +## 5. Isolation: what guarantees production is untouched + +Production frozen P1 must keep running throughout. The scripts enforce, not assume: + +* **Hard port deny-list.** `8099 8199 18080 8443 42127 42130 42131 4216 8080 8081 + 8094` are refused before every bind and before every HTTP request the scripts make. + Each staging port is also proven free first, and bring-up aborts before launching + anything if one is not. +* **Hard path deny-list.** Every filesystem path goes through `safe_path()`, which + refuses `/home/alex/openfut-promotion/state/` (the live `prod-core.db`, + `prod-market.db`, `prod-pile.db`, `prod-identity.json`). Nothing there is ever + opened — not even read. The FIFA17 content the staging stack loads is copied from + `/home/alex/openfut-post-p1/staging/emit/content/`, a build tree outside that + directory. +* **The Python fallback fails closed.** `OPENFUT_UTAS_PYTHON_URL` points at + `http://127.0.0.1:8399`, where **nothing listens**, so a fallback to the Python + oracle surfaces as a visible connection error instead of silently serving the + production oracle on `8199`. +* **Binaries are copied into the staging dir and run from there.** A later + `cargo build` cannot change what staging is running, and — more importantly — every + staging process's `/proc//cmdline` provably contains the staging directory, + which production's never can. +* **Teardown kills by recorded pid only, never by pattern.** `openfut-utas-host` and + `openfut-core` each name *two* live processes on this machine. `sold-staging-down.py` + reads pids from `manifest.json`, re-reads `/proc//cmdline` and **refuses** to + signal anything whose cmdline does not contain the staging dir; it refuses the known + production pids explicitly; and it signals only the process group the up script + created (`pgid == pid`, via a new session). There is no `pkill`/`pgrep` anywhere. +* **The Blaze patch cannot silently no-op.** Six substitutions are applied to the + copy, each anchored to a whole assignment line and each required to match **exactly + once**; the file is then re-read from disk and every value re-checked, including an + explicit assertion that `UTAS_BASE` no longer contains `:8099`. A responder that + changed shape aborts bring-up rather than running half-patched and pointing at + production. The four protocol values are `REDIR_PORT`, `BLAZE_PORT`, + `NUCLEUS_PORT`, `UTAS_BASE`; the two extra ones are the responder's hardcoded + `LOG = "/tmp/blaze_responder.log"` and `RXDIR = "/tmp/blaze_rx"`, redirected into + the staging dir so a capture can never be ambiguous about which stack wrote it. +* **Production liveness is asserted** at preflight, at the end of bring-up, and after + teardown (`prod utas-host` pid 3631953, `prod Core` pid 3374264). +* **Nothing on 10.10.0.105 is written.** The scripts only *print* the config block; + the operator edits it by hand. + +### Production services deliberately shared, read-only + +Staging Blaze advertises the **production** auxiliary endpoints, exactly the strings +production Blaze advertises: + +| Advertised to client | Value | Why sharing is safe | +| --- | --- | --- | +| roster XML / roster URL | `10.10.0.120:8081` | static roster update XML; **no economy state**. Hardcoded as `ROSTER_HOST = "%s:8081" % _ADVERTISE` in the responder. | +| `FIFA_POW_CONTENT_SERVER_URL` | `10.10.0.120:8085` | static POW content; no economy state (`POW_CONTENT_HOST`) | +| `FIFA_POW_URL` / nucleus proxy | `10.10.0.120:8094` | POW API, only emitted when `FUT_POW=1`, which staging leaves unset (`POW_HOST`) | +| LSX layer-1 responder | `:4216` | persona/session handshake for the same persona `33068179`; no economy state | + +The staging stack **never connects** to any of these itself — it only hands the client +the same strings production hands it. Every service that owns economy state (UTAS, +Core, market/pile/identity) is duplicated on staging ports against staging files. +Because those aux services are shared, **do not run a staging session and a +production session simultaneously**: one client at a time. + +`heat2` (TDF codec) and `fut_account` (the single source of truth for persona +`33068179` / `CAGE`, shared with LSX and the oracle) are imported read-only from +`fifa17-recon/tools/` via `PYTHONPATH` rather than copied, so the staging Blaze +asserts a **byte-identical** identity to what LSX already asserts. Cross-layer +identity consistency is the constraint; a divergent copy would break login. + +--- + +## 6. Verifying a live stack by hand + +All read-only, and none of it touches a production port: + +```bash +S=/home/alex/openfut-sold-staging + +# which variant is live +grep sold-experiment $S/logs/utas-host.log + +# staging UTAS answers, from staging state +python3 - <<'PY' +import http.client, json +c = http.client.HTTPConnection("127.0.0.1", 8299, timeout=5) +c.request("GET", "/ut/game/fifa17/tradePile/counts", + headers={"X-OpenFUT-Game": "fifa17"}) +print(c.getresponse().read().decode()) +PY + +# the patched Blaze points at staging UTAS, not production +grep -nE '^(UTAS_BASE|REDIR_PORT|BLAZE_PORT|NUCLEUS_PORT) = ' \ + $S/blaze/blaze_responder_staging.py + +# exactly six lines differ from the original responder +diff fifa17-recon/tools/blaze_responder_v3b.py \ + $S/blaze/blaze_responder_staging.py + +# no staging process has a handle on production state +for p in $(python3 -c "import json;print(' '.join(str(x['pid']) for x in json.load(open('$S/manifest.json'))['processes']))"); do + echo "pid $p: $(sudo ls -l /proc/$p/fd | grep -c openfut-promotion/state) prod-state handles" +done + +# production is still up +ps -o pid=,cmd= -p 3631953 +``` + +Never `curl` port 8099/8199/18080 to "compare with production" — those ports are on +the deny-list. Independence is proven structurally (separate pids, separate ports, +separate files, zero shared handles), not by poking the live stack. + +--- + +## 7. Troubleshooting + +| Symptom | Cause / fix | +| --- | --- | +| `REFUSING to start -- these staging ports are not free` | something else holds a staging port. Nothing was launched. Free it, or edit the port block at the top of `sold-staging-up.py`. | +| `a previous staging stack is STILL UP` | run `sold-staging-down.py` first. The up script never steps on a live stack. | +| `blaze patch applied 0 times` | `blaze_responder_v3b.py` changed shape. Fix the regex in `blaze_patches()`; do **not** disable the assertion. | +| `seed card ids are not resolvable` | the emitted content in `CONTENT_SRC` no longer contains a seeded `card_id`. Re-point `CONTENT_SRC` or update the seed ids — both memberships (content pack **and** identity catalog) are required. | +| client shows *"error occurred downloading the FUT Squad Update"*, staging host log shows **no request at all** | the failure is upstream of UTAS. Almost always a client that was not relaunched after the config change (§3). Check the *absence* of requests in `logs/utas-host.log` before touching any code. | +| ` pid N is alive but its cmdline does NOT contain ` | pid reuse or a stale manifest. Teardown refuses rather than guessing — identify the process by hand and stop it by pid. | +| staging host log shows connection errors to `127.0.0.1:8399` | expected and intended: a UTAS route fell through to the Python fallback, which is deliberately dead. Fix the route; do **not** point the fallback at `8199`. | diff --git a/openfut-utas-host/src/bin/staging_sell.rs b/openfut-utas-host/src/bin/staging_sell.rs new file mode 100644 index 0000000..8c458ab --- /dev/null +++ b/openfut-utas-host/src/bin/staging_sell.rs @@ -0,0 +1,189 @@ +//! STAGING-ONLY: complete a market sale on behalf of a synthetic Buyer B. +//! +//! Production has no trigger that decides "your listing sold" — that needs the +//! seller-facing sold wire contract, which is exactly what the staging experiment +//! is trying to establish. This binary is the synthetic counterparty: it runs the +//! REAL settlement path (`CoreEconomy::settle_sale` → Core's atomic +//! `POST /economy/settle-sale`) and then flips the host's listing to `sold`, so the +//! seller's client sees an authentic completed sale rather than a hand-written row. +//! +//! It is a separate binary precisely so no production HTTP surface grows an +//! experiment hook. It is never deployed and never runs in production. +//! +//! Ordering is deliberate: **settle first, mark sold second.** If settlement fails +//! the listing stays live and nothing has moved. If marking fails after a +//! successful settlement, the coins and ownership are already correct and the +//! listing is merely still shown as active — recoverable, and it never pays twice +//! because `mark_sold` is a once-only transition and clearing is presentation-only. +//! +//! ```text +//! staging-sell --market-db PATH --core-url URL --trade-id ID \ +//! --item CORE_ITEM_ID --seller CLUB --buyer CLUB [--gross N] +//! ``` +//! `--gross` defaults to 150, the canonical staging sale. + +use openfut_utas_host::market_store::MarketStore; +use openfut_utas_host::{CoreEconomy, EconomySale, HttpCoreClient}; + +/// FIFA 17's transfer fee, taken from the adapter so this harness can never +/// disagree with the shipped policy about what the seller is owed. +fn fee_for(gross: i64) -> i64 { + openfut_adapter_fifa17::fut::economy_policy::transfer_market_fee(gross) +} + +struct Args { + market_db: String, + core_url: String, + trade_id: String, + item: String, + seller: String, + buyer: String, + gross: i64, +} + +fn parse_args() -> Result { + let mut market_db = None; + let mut core_url = None; + let mut trade_id = None; + let mut item = None; + let mut seller = None; + let mut buyer = None; + let mut gross = 150i64; + let argv: Vec = std::env::args().skip(1).collect(); + let mut i = 0; + while i < argv.len() { + let need = |i: usize| -> Result { + argv.get(i + 1) + .cloned() + .ok_or_else(|| format!("{} needs a value", argv[i])) + }; + match argv[i].as_str() { + "--market-db" => market_db = Some(need(i)?), + "--core-url" => core_url = Some(need(i)?), + "--trade-id" => trade_id = Some(need(i)?), + "--item" => item = Some(need(i)?), + "--seller" => seller = Some(need(i)?), + "--buyer" => buyer = Some(need(i)?), + "--gross" => gross = need(i)?.parse().map_err(|e| format!("--gross: {e}"))?, + other => return Err(format!("unknown argument {other}")), + } + i += 2; + } + Ok(Args { + market_db: market_db.ok_or("--market-db is required")?, + core_url: core_url.ok_or("--core-url is required")?, + trade_id: trade_id.ok_or("--trade-id is required")?, + item: item.ok_or("--item is required")?, + seller: seller.ok_or("--seller is required")?, + buyer: buyer.ok_or("--buyer is required")?, + gross, + }) +} + +fn main() { + let args = match parse_args() { + Ok(a) => a, + Err(e) => { + eprintln!("staging-sell: {e}"); + eprintln!( + "usage: staging-sell --market-db PATH --core-url URL --trade-id ID \ + --item CORE_ITEM_ID --seller CLUB --buyer CLUB [--gross N]" + ); + std::process::exit(2); + } + }; + + // Refuse to run against anything that looks like production state. This binary + // exists to keep an experiment isolated, so the guard belongs here rather than + // only in the caller. + for (label, value) in [ + ("--market-db", &args.market_db), + ("--core-url", &args.core_url), + ] { + if value.contains("openfut-promotion") + || value.contains(":18080") + || value.contains(":8099") + { + eprintln!("staging-sell: REFUSING to touch production via {label}={value}"); + std::process::exit(3); + } + } + + let fee = fee_for(args.gross); + let proceeds = args.gross - fee; + println!( + "staging-sell: gross={} fee={} proceeds={} (floor 5%, fee+proceeds==gross)", + args.gross, fee, proceeds + ); + + let core = HttpCoreClient::new(args.core_url.clone(), "fifa17"); + let sale = EconomySale { + item_id: &args.item, + seller_club_id: Some(&args.seller), + buyer_club_id: Some(&args.buyer), + gross: args.gross, + fee, + }; + + // 1. Settle atomically in Core: buyer debited, item transferred, seller paid net. + let receipt = match core.settle_sale(&sale) { + Ok(r) => r, + Err(e) => { + eprintln!("staging-sell: settlement FAILED, listing left live: {e:?}"); + std::process::exit(1); + } + }; + println!( + "staging-sell: SETTLED item={} card={} seller={} -> buyer={:?} \ + seller_balance={} buyer_balance={:?} squad_slots_freed={}", + receipt.item_id, + receipt.card_id, + receipt.seller_club_id, + receipt.buyer_club_id, + receipt.seller_balance, + receipt.buyer_balance, + receipt.squad_slots_freed + ); + + // 2. Only now does the seller's listing become `sold`, so the client can be + // shown a completed sale. + let rt = match tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + { + Ok(rt) => rt, + Err(e) => { + eprintln!("staging-sell: runtime: {e} (settlement already committed)"); + std::process::exit(1); + } + }; + rt.block_on(async { + let store = match MarketStore::open(&args.market_db).await { + Ok(s) => s, + Err(e) => { + eprintln!( + "staging-sell: market store {} failed to open: {e} \ + (settlement already committed — coins/ownership are correct)", + args.market_db + ); + std::process::exit(1); + } + }; + match store.mark_sold(&args.trade_id).await { + Ok(true) => println!("staging-sell: listing {} -> sold", args.trade_id), + Ok(false) => println!( + "staging-sell: listing {} was NOT live (already sold/cancelled) — \ + no second sale, nothing changed", + args.trade_id + ), + Err(e) => { + eprintln!("staging-sell: mark_sold failed: {e}"); + std::process::exit(1); + } + } + match store.uncleared_sold().await { + Ok(rows) => println!("staging-sell: uncleared sold rows now {}", rows.len()), + Err(e) => eprintln!("staging-sell: uncleared_sold read failed: {e}"), + } + }); +} diff --git a/openfut-utas-host/src/lib.rs b/openfut-utas-host/src/lib.rs index 2f536fa..b1ba5ec 100644 --- a/openfut-utas-host/src/lib.rs +++ b/openfut-utas-host/src/lib.rs @@ -41,6 +41,7 @@ pub mod economy_store; pub mod market; pub mod market_store; pub mod pile_store; +pub mod sold_experiment; use std::io::{BufRead, BufReader, Write}; use std::net::{TcpListener, TcpStream}; @@ -363,6 +364,11 @@ pub enum EconomyRoute { /// contemporaneous FIFA 17 clients use) map here — a DELETE on the plain /// spelling otherwise fell into the buy/view arm and silently did nothing. MarketCancel, + /// Bulk clear-sold: `DELETE /ut/delete/game//trade/sold`, the client's + /// `RemoveAllSoldFromTradePile`. Carries no trade id, so it MUST NOT reach + /// [`EconomyRoute::MarketCancel`], which would parse no id and ack while + /// clearing nothing. + MarketClearSold, } /// `item/` — the single-card quick-sell tail (DELETE). @@ -427,6 +433,16 @@ fn is_trade_status_tail(tail: &str) -> bool { tail.eq_ignore_ascii_case(STATUS) } +/// `trade/sold` — the BULK clear-sold tail, which carries no trade id. +/// +/// PE-proven shape: request builder `0x1801647c0` writes the literal `/sold` when +/// its tradeId field is zero and `/%lld` when it is not, onto route base +/// `ut/delete/%s/trade`. The client names the operation +/// `RemoveAllSoldFromTradePile`. +fn is_trade_sold_tail(tail: &str) -> bool { + tail.eq_ignore_ascii_case("trade/sold") +} + /// Classify a FIFA17 economy route from method + path, mirroring the Python /// oracle's route table (`utas_server.py` §1418-1553). Returns `None` for any /// non-economy path. Path is already query-stripped by the caller. @@ -445,6 +461,14 @@ pub fn classify_economy(method: &str, path: &str) -> Option { if tail == "item" && post { return Some(EconomyRoute::QuickSellBody); } + // MUST precede the generic `trade…` cancel arm. FIFA 17's request + // builder 0x1801647c0 emits the literal `/sold` (no numeric id) for the + // bulk "RemoveAllSoldFromTradePile" verb, and `/%lld` for one trade. A + // `sold` tail carries no id, so the cancel handler would parse nothing + // and silently ack while clearing nothing. + if delete && is_trade_sold_tail(tail) { + return Some(EconomyRoute::MarketClearSold); + } if tail.starts_with("trade") && delete { return Some(EconomyRoute::MarketCancel); } @@ -2080,6 +2104,9 @@ pub struct EconomyServices { /// The resolvable FIFA∩Core card universe a pack can award (empty → the Store /// fail-closes: it draws nothing and debits nothing). pub pool: Arc>, + /// STAGING-ONLY sold-row experiment. `SoldExperiment::OFF` in production, where + /// it changes nothing. + pub sold_experiment: crate::sold_experiment::SoldExperiment, } /// Build the pack-content candidate pool from Core's current content, evidenced @@ -2241,12 +2268,17 @@ impl Server { // Content pool from Core's current inventory (empty ⇒ Store fails closed, // never mints/debits — an honest degrade if Core is not yet seeded). let pool = Arc::new(build_content_pool(core.as_ref(), resolver.as_ref())); + // Read once at startup and logged, so a staging capture can never be + // mistaken for production output. + let sold_experiment = crate::sold_experiment::SoldExperiment::from_env(); + eprintln!("utas-host {}", sold_experiment.banner()); let economy = Arc::new(EconomyServices { econ, market, piles, bridge, pool, + sold_experiment, }); let clientdata = Arc::new(ClientDataStore::open(cfg.clientdata_path.clone())); @@ -2438,20 +2470,28 @@ impl Server { EconomyRoute::MarketQuery => { let (bridge, market, econ) = (svc.bridge.clone(), svc.market.clone(), svc.econ.clone()); + let exp = svc.sold_experiment; bridge.block_on(async move { - crate::market::handle_market_query("active", econ.as_ref(), market.as_ref()) - .await + crate::market::handle_market_query( + "active", + econ.as_ref(), + market.as_ref(), + exp, + ) + .await }) } EconomyRoute::MarketCounts => { let (bridge, market) = (svc.bridge.clone(), svc.market.clone()); + let exp = svc.sold_experiment; bridge.block_on(async move { - crate::market::handle_market_counts(market.as_ref()).await + crate::market::handle_market_counts(market.as_ref(), exp).await }) } EconomyRoute::MarketStatus => { let (bridge, market, econ) = (svc.bridge.clone(), svc.market.clone(), svc.econ.clone()); + let exp = svc.sold_experiment; // The raw query carries `tradeIds`; `path` is already stripped. let q = target.split_once('?').map(|(_, q)| q.to_string()); bridge.block_on(async move { @@ -2459,6 +2499,7 @@ impl Server { q.as_deref(), econ.as_ref(), market.as_ref(), + exp, ) .await }) @@ -2479,6 +2520,12 @@ impl Server { crate::market::handle_market_cancel(&p, owner.as_deref(), market.as_ref()).await }) } + EconomyRoute::MarketClearSold => { + let (bridge, market) = (svc.bridge.clone(), svc.market.clone()); + bridge.block_on(async move { + crate::market::handle_market_clear_sold(market.as_ref()).await + }) + } }; Some(resp) } diff --git a/openfut-utas-host/src/market.rs b/openfut-utas-host/src/market.rs index 5d93132..3912bbf 100644 --- a/openfut-utas-host/src/market.rs +++ b/openfut-utas-host/src/market.rs @@ -35,6 +35,7 @@ use crate::economy_store::OwnedItemLookup; use crate::market_store::{now_secs, Listing, MarketError, MarketStore}; use crate::pile_store::PileStore; +use crate::sold_experiment::{CountMode, SoldExperiment}; use crate::{CoreEconomy, CoreError, WireResponse}; /// FIFA trade-id numbering base (mirrors the oracle's `_TRADE_ID_BASE`). @@ -88,6 +89,25 @@ fn trade_id_from_path(path: &str) -> Option { /// is handed an unrecognised `CARD_OFFERSTATE`. Where the binary contradicts the /// oracle, the binary wins. fn auction_record_as(l: &Listing, item_state: &str) -> Value { + auction_record_tuned(l, item_state, None, 0) +} + +/// [`auction_record_as`] with the two fields the staging sold experiment varies. +/// +/// `sold_bid_state` overrides `bidState` for a terminal (non-active) listing, and +/// `coins_processed` sets the atom the client publishes to Flash as +/// `COINS_AWARDED`. Both default to today's production values via +/// [`auction_record_as`], so nothing changes unless the experiment is on. +/// +/// Everything else is byte-identical between variants BY CONSTRUCTION: there is +/// one record builder, and the A/B changes only what is passed in here. That is +/// what makes the client's reaction attributable to the token. +fn auction_record_tuned( + l: &Listing, + item_state: &str, + sold_bid_state: Option<&str>, + coins_processed: i64, +) -> Value { let trade_id: i64 = l.listing_id.parse().unwrap_or(0); // resourceId is the FIFA wire identity the client listed (never the Core // card id). 0 means "no art", a valid int — never a fabricated FIFA asset. @@ -113,7 +133,15 @@ fn auction_record_as(l: &Listing, item_state: &str) -> Value { let (trade_state, bid_state, current_bid) = match l.state.as_str() { "active" if expires == 0 => ("expired", "none", 0), "active" => ("active", "none", 0), - _ => ("closed", "highest", l.buy_now_price), + // Terminal. `closed` is the only FIFA 17 token for "this auction is over + // and something happened"; there is no `sold`. The experiment varies which + // bidState rides along, because that is the one thing the movie can see + // (published verbatim as YOURBID) and the native flags cannot distinguish. + _ => ( + "closed", + sold_bid_state.unwrap_or("highest"), + l.buy_now_price, + ), }; let item_data = l .item_json @@ -164,7 +192,10 @@ fn auction_record_as(l: &Listing, item_state: &str) -> Value { .unwrap_or_else(|| non_economy::PERSONA_DISPLAY_NAME.to_string()), "sellerEstablished": 1, "watched": false, - "coinsProcessed": 0, + // Published to Flash as COINS_AWARDED (record +0xbf, atom 0x2f4, u8). + // Production emits 0; the experiment's third pass varies it to learn + // whether the client treats it as informational or as a gate. + "coinsProcessed": coins_processed, }) } @@ -404,15 +435,42 @@ pub async fn handle_market_query( state: &str, econ: &dyn CoreEconomy, store: &MarketStore, + exp: SoldExperiment, ) -> WireResponse { let listings = match store.query_listings(state).await { Ok(l) => l, Err(_) => return json_body(503, &json!({ "error": "market_store" })), }; - let auctions: Vec = listings + let mut auctions: Vec = listings .iter() .map(|l| auction_record_as(l, "forSale")) .collect(); + // STAGING ONLY. FIFA 17's bulk `DELETE …/trade/sold` verb only makes sense if + // sold rows persist in the seller's pile until acknowledged, so the experiment + // projects uncleared sold listings alongside the active ones. Off in + // production, where this stays exactly the Fix A invariant: active auctions + // only. + if exp.enabled() { + if let Ok(sold) = store.uncleared_sold().await { + for l in &sold { + auctions.push(auction_record_tuned( + l, + "forSale", + exp.bid_state, + exp.coins_processed, + )); + } + if !sold.is_empty() { + eprintln!( + "utas-host owner=RUST route=market-query SOLD-EXPERIMENT \ + sold_rows={} bidState={:?} coinsProcessed={}", + sold.len(), + exp.bid_state, + exp.coins_processed + ); + } + } + } // GetTradePile shares one deserializer (0x18013e7f0) with ISSearch and // ISWatchList, over exactly four members: `auctionInfo` (array), `credits` // (int), `duplicateItemIdList` (array of objects) and `total` (int). We were @@ -434,21 +492,59 @@ pub async fn handle_market_query( /// freeze risk. They are the only inputs to IS_MAX_AUCTIONS, so /// `maxAuctionsAllowed = 100` with `selling < 100` keeps the listing cap open. /// A store read failure degrades to zeros (cosmetic tally, never fail-closed). -pub async fn handle_market_counts(store: &MarketStore) -> WireResponse { - let n = store +/// +/// `sold` is NOT cosmetic: RE proved atom `sold` (0x2c9) reaches the hub tile as +/// Flash `TEXT3` under the localised caption `FUT_TF_SOLD`, so the seller really +/// does see a SOLD bucket. Production still reports 0 because we have never had a +/// sold row; the experiment reports the real count so the client can be observed. +pub async fn handle_market_counts(store: &MarketStore, exp: SoldExperiment) -> WireResponse { + let selling = store .query_listings("active") .await .map(|l| l.len() as i64) .unwrap_or(0); + let sold = if exp.enabled() { + store + .uncleared_sold() + .await + .map(|l| l.len() as i64) + .unwrap_or(0) + } else { + 0 + }; + // FIFA 17's exact meaning for `count` is unknown — live auctions, or whole + // Transfer List membership. It is a controlled variable, never a guess. + let count = match exp.count_mode { + CountMode::Active => selling, + CountMode::ActivePlusSold => selling + sold, + }; ok_json(&json!({ - "count": n, + "count": count, "maxAuctionsAllowed": 100, "offered": 0, - "selling": n, - "sold": 0, + "selling": selling, + "sold": sold, })) } +/// `DELETE /ut/delete/game//trade/sold` — the bulk clear-sold verb. +/// +/// PE-proven: the request builder `0x1801647c0` emits the literal `/sold` when the +/// tradeId field is zero and `/%lld` otherwise, and the client's request-name table +/// calls it `RemoveAllSoldFromTradePile`. The response body parses nothing, so `{}` +/// is the whole contract. +/// +/// PRESENTATION ONLY. Settlement already happened when the sale completed; this +/// records the seller's acknowledgement. It must never move coins or ownership, +/// or a client retry would pay twice. +pub async fn handle_market_clear_sold(store: &MarketStore) -> WireResponse { + match store.clear_sold().await { + Ok(n) => eprintln!("utas-host owner=RUST route=market-clear-sold cleared={n}"), + Err(e) => eprintln!("utas-host WARN market clear_sold failed: {e}"), + } + ok_json(&json!({})) +} + /// `DELETE /ut/delete/game//trade/` — remove a listing from the sale /// pile. Cancels the `active` listing once; the oracle always acks `{}`, so a /// missing/already-closed listing is not surfaced as an error to the client @@ -482,13 +578,23 @@ pub async fn handle_market_status( query: Option<&str>, econ: &dyn CoreEconomy, store: &MarketStore, + exp: SoldExperiment, ) -> WireResponse { let ids = trade_ids_from_query(query); let listings = if ids.is_empty() { - match store.query_listings("active").await { + let mut all = match store.query_listings("active").await { Ok(l) => l, Err(_) => return json_body(503, &json!({ "error": "market_store" })), + }; + // Unfiltered poll: the experiment's sold rows are part of what the screen + // is showing, so they must answer here too or the row would render from + // /tradePile and then contradict its own status poll. + if exp.enabled() { + if let Ok(sold) = store.uncleared_sold().await { + all.extend(sold); + } } + all } else { let mut found = Vec::with_capacity(ids.len()); for id in &ids { @@ -500,7 +606,7 @@ pub async fn handle_market_status( }; let auctions: Vec = listings .iter() - .map(|l| auction_record_as(l, "forSale")) + .map(|l| auction_record_tuned(l, "forSale", exp.bid_state, exp.coins_processed)) .collect(); eprintln!( "utas-host owner=RUST route=market-status requested={} returned={} query={}", @@ -941,7 +1047,7 @@ mod tests { let econ = CountingEconomy::with_balance(10_000); // Nothing listed: an empty pile, whatever the item store holds. - let body = parse(&handle_market_query("active", &econ, &store).await); + let body = parse(&handle_market_query("active", &econ, &store, SoldExperiment::OFF).await); assert_eq!(body["auctionInfo"].as_array().unwrap().len(), 0); assert_eq!(body["total"], 0); @@ -961,7 +1067,7 @@ mod tests { .await .unwrap(); - let body = parse(&handle_market_query("active", &econ, &store).await); + let body = parse(&handle_market_query("active", &econ, &store, SoldExperiment::OFF).await); let recs = body["auctionInfo"].as_array().unwrap(); assert_eq!(recs.len(), 1, "the real auction, and nothing synthetic"); assert_eq!(recs[0]["tradeState"], "active"); @@ -1037,7 +1143,7 @@ mod tests { // nowhere in CardsDLL or in 4.26 GiB of live process memory and decoded to // -1, so the client was handed an unrecognised CARD_OFFERSTATE. `forSale` // (5) is the value in FIFA 17's own itemState table. - let pile = handle_market_query("active", &econ, &store).await; + let pile = handle_market_query("active", &econ, &store, SoldExperiment::OFF).await; let rec = parse(&pile)["auctionInfo"][0].clone(); assert_eq!(rec["itemData"]["itemState"], "forSale"); assert_eq!(rec["itemData"]["rating"], 84); @@ -1137,7 +1243,7 @@ mod tests { let (store, _d) = store_at("query").await; seed_listing(&store, "900000005", 2500).await; let econ = CountingEconomy::with_balance(50); - let resp = handle_market_query("active", &econ, &store).await; + let resp = handle_market_query("active", &econ, &store, SoldExperiment::OFF).await; let b = parse(&resp); assert_eq!(b["auctionInfo"].as_array().unwrap().len(), 1); assert_eq!(b["auctionInfo"][0]["tradeId"], 900000005i64); @@ -1151,12 +1257,12 @@ mod tests { // every count at its constructor default, so the Transfer List screen // shows no active sale even while the hub tile reports one (live bug). let (store, _d) = store_at("counts").await; - let b = parse(&handle_market_counts(&store).await); + let b = parse(&handle_market_counts(&store, SoldExperiment::OFF).await); assert_eq!(b["count"], 0); assert_eq!(b["selling"], 0); seed_listing(&store, "900000007", 2500).await; - let resp = handle_market_counts(&store).await; + let resp = handle_market_counts(&store, SoldExperiment::OFF).await; assert_eq!(resp.status, 200); let b = parse(&resp); assert_eq!(b["count"], 1, "tally counts the active listing"); @@ -1187,7 +1293,7 @@ mod tests { let econ = CountingEconomy::with_balance(10_000); seed_listing(&store, "900000030", 2500).await; - let body = parse(&handle_market_query("active", &econ, &store).await); + let body = parse(&handle_market_query("active", &econ, &store, SoldExperiment::OFF).await); let rec = body["auctionInfo"][0].clone(); let mut got: Vec<&str> = rec .as_object() @@ -1283,7 +1389,7 @@ mod tests { seed_listing(&store, "900000031", 2500).await; // No filter: answer with the player's own active pile. - let all = parse(&handle_market_status(None, &econ, &store).await); + let all = parse(&handle_market_status(None, &econ, &store, SoldExperiment::OFF).await); assert_eq!( all["auctionInfo"].as_array().unwrap().len(), 1, @@ -1296,12 +1402,28 @@ mod tests { assert!(all["credits"].is_i64()); // Explicit tradeIds filter returns exactly the requested auction. - let one = parse(&handle_market_status(Some("tradeIds=900000031"), &econ, &store).await); + let one = parse( + &handle_market_status( + Some("tradeIds=900000031"), + &econ, + &store, + SoldExperiment::OFF, + ) + .await, + ); assert_eq!(one["auctionInfo"].as_array().unwrap().len(), 1); assert_eq!(one["auctionInfo"][0]["tradeId"], 900_000_031i64); // An unknown id is absent, not an error: the poll must never fail closed. - let miss = parse(&handle_market_status(Some("tradeIds=900000099"), &econ, &store).await); + let miss = parse( + &handle_market_status( + Some("tradeIds=900000099"), + &econ, + &store, + SoldExperiment::OFF, + ) + .await, + ); assert_eq!(miss["auctionInfo"].as_array().unwrap().len(), 0); // Garbage is skipped rather than poisoning the whole poll. @@ -1322,7 +1444,13 @@ mod tests { let trade_id = TRADE_ID_BASE + 100_000_122; let unknown = parse( - &handle_market_status(Some(&format!("tradeIds={trade_id}")), &econ, &store).await, + &handle_market_status( + Some(&format!("tradeIds={trade_id}")), + &econ, + &store, + SoldExperiment::OFF, + ) + .await, ); assert_eq!( unknown["auctionInfo"].as_array().unwrap().len(), @@ -1347,7 +1475,13 @@ mod tests { .unwrap(); let one = parse( - &handle_market_status(Some(&format!("tradeIds={trade_id}")), &econ, &store).await, + &handle_market_status( + Some(&format!("tradeIds={trade_id}")), + &econ, + &store, + SoldExperiment::OFF, + ) + .await, ); let recs = one["auctionInfo"].as_array().unwrap(); assert_eq!(recs.len(), 1, "now there is a real auction to answer with"); @@ -1640,4 +1774,249 @@ mod tests { Some("purchased") ); } + + // ── staging sold-row experiment ────────────────────────────────────────── + // + // The client-facing question these support: for a `closed` row, CardsDLL's + // native flags cannot distinguish bidState `highest` from `buyNow` + // (IS_GLOW = bidState != none, INBOX = bidState in {highest,buyNow}), but the + // movie receives bidState verbatim as YOURBID. So the A/B is only meaningful if + // EVERY other field is identical between variants. These tests pin that. + + async fn seed_sold(store: &MarketStore, id: &str, buy_now: i64) { + seed_listing(store, id, buy_now).await; + assert!(store.mark_sold(id).await.unwrap(), "listing became sold"); + } + + /// Production default: a sold listing is INVISIBLE to the seller's pile and the + /// counts stay exactly as they ship today. Guards the Fix A invariant against + /// the experiment leaking into production. + #[tokio::test] + async fn experiment_off_hides_sold_rows_entirely() { + let (store, _d) = store_at("soldoff").await; + let econ = CountingEconomy::with_balance(10_000); + seed_sold(&store, "900000200", 150).await; + + let pile = parse(&handle_market_query("active", &econ, &store, SoldExperiment::OFF).await); + assert_eq!( + pile["auctionInfo"].as_array().unwrap().len(), + 0, + "no sold row" + ); + assert_eq!(pile["total"], 0); + + let counts = parse(&handle_market_counts(&store, SoldExperiment::OFF).await); + assert_eq!(counts["sold"], 0, "production reports sold: 0"); + assert_eq!(counts["selling"], 0); + assert_eq!(counts["count"], 0); + + let status = parse(&handle_market_status(None, &econ, &store, SoldExperiment::OFF).await); + assert_eq!(status["auctionInfo"].as_array().unwrap().len(), 0); + } + + /// With the experiment on, the sold row appears and carries the token under + /// test, `tradeState: closed`, and `currentBid` = the sale price. + #[tokio::test] + async fn experiment_projects_the_sold_row_with_the_token_under_test() { + for token in ["highest", "buyNow"] { + let (store, _d) = store_at(&format!("soldon{token}")).await; + let econ = CountingEconomy::with_balance(10_000); + seed_sold(&store, "900000201", 150).await; + let exp = SoldExperiment::from_values(Some(token), None, None); + + let pile = parse(&handle_market_query("active", &econ, &store, exp).await); + let recs = pile["auctionInfo"].as_array().unwrap(); + assert_eq!(recs.len(), 1, "{token}: the sold row is shown"); + assert_eq!(recs[0]["tradeState"], "closed", "{token}"); + assert_eq!(recs[0]["bidState"], token, "{token}"); + assert_eq!(recs[0]["currentBid"], 150, "{token}: sale price"); + assert_eq!( + recs[0]["expires"], 0, + "{token}: a sold auction has no clock" + ); + assert_eq!(pile["total"], 1, "{token}"); + } + } + + /// THE experimental control: between the two variants, EXACTLY ONE field may + /// differ. If anything else moves, the client's reaction is not attributable to + /// the token and the whole A/B is void. + #[tokio::test] + async fn the_two_variants_differ_in_bidstate_and_nothing_else() { + let mut rows = Vec::new(); + for token in ["highest", "buyNow"] { + let (store, _d) = store_at(&format!("soldab{token}")).await; + let econ = CountingEconomy::with_balance(10_000); + seed_sold(&store, "900000202", 150).await; + let exp = SoldExperiment::from_values(Some(token), None, None); + let pile = parse(&handle_market_query("active", &econ, &store, exp).await); + rows.push(pile["auctionInfo"][0].clone()); + } + let (a, b) = (&rows[0], &rows[1]); + let keys: Vec<&String> = a.as_object().unwrap().keys().collect(); + let differing: Vec<&&String> = keys + .iter() + .filter(|k| a[k.as_str()] != b[k.as_str()]) + .collect(); + assert_eq!( + differing.len(), + 1, + "exactly one field may differ between variants, saw {differing:?}" + ); + assert_eq!(differing[0].as_str(), "bidState"); + // And the twelve-atom shape is preserved in both. + assert_eq!(a.as_object().unwrap().len(), 12, "still twelve atoms"); + assert_eq!(b.as_object().unwrap().len(), 12); + } + + /// `coinsProcessed` (Flash `COINS_AWARDED`) is varied INDEPENDENTLY of the + /// bidState A/B, so the third pass cannot be confounded with the first. + #[tokio::test] + async fn coins_processed_varies_alone() { + let mut rows = Vec::new(); + for cp in [None, Some("1")] { + let (store, _d) = store_at(&format!("soldcp{}", cp.unwrap_or("0"))).await; + let econ = CountingEconomy::with_balance(10_000); + seed_sold(&store, "900000203", 150).await; + let exp = SoldExperiment::from_values(Some("highest"), cp, None); + let pile = parse(&handle_market_query("active", &econ, &store, exp).await); + rows.push(pile["auctionInfo"][0].clone()); + } + assert_eq!(rows[0]["coinsProcessed"], 0); + assert_eq!(rows[1]["coinsProcessed"], 1); + let keys: Vec<&String> = rows[0].as_object().unwrap().keys().collect(); + let differing: Vec<&&String> = keys + .iter() + .filter(|k| rows[0][k.as_str()] != rows[1][k.as_str()]) + .collect(); + assert_eq!( + differing.len(), + 1, + "only coinsProcessed may move: {differing:?}" + ); + assert_eq!(differing[0].as_str(), "coinsProcessed"); + } + + /// Counts with a sold row present, under both `count` modes. `count`'s FIFA 17 + /// meaning is unknown, so it is a controlled variable — never a guess. + #[tokio::test] + async fn counts_report_sold_and_count_mode_is_controlled() { + let (store, _d) = store_at("soldcounts").await; + seed_sold(&store, "900000204", 150).await; + seed_listing(&store, "900000205", 500).await; // one still active + + let active_mode = SoldExperiment::from_values(Some("highest"), None, Some("active")); + let c = parse(&handle_market_counts(&store, active_mode).await); + assert_eq!(c["selling"], 1, "one live auction"); + assert_eq!(c["sold"], 1, "one uncleared sale"); + assert_eq!(c["count"], 1, "active mode: count == selling"); + assert_eq!(c["maxAuctionsAllowed"], 100); + assert_eq!(c["offered"], 0); + + let both = SoldExperiment::from_values(Some("highest"), None, Some("active_plus_sold")); + let c2 = parse(&handle_market_counts(&store, both).await); + assert_eq!(c2["selling"], 1); + assert_eq!(c2["sold"], 1); + assert_eq!(c2["count"], 2, "membership mode: count == selling + sold"); + } + + /// The bulk clear verb clears sold rows and nothing else, and it is + /// PRESENTATION ONLY: it must not touch the economy or resurrect ownership. + #[tokio::test] + async fn clear_sold_removes_only_sold_rows_and_moves_no_coins() { + let (store, _d) = store_at("soldclear").await; + let econ = CountingEconomy::with_balance(7_777); + seed_sold(&store, "900000206", 150).await; + seed_listing(&store, "900000207", 500).await; + let exp = SoldExperiment::from_values(Some("highest"), None, None); + + assert_eq!(store.uncleared_sold().await.unwrap().len(), 1); + let resp = handle_market_clear_sold(&store).await; + assert_eq!(parse(&resp), json!({}), "the client parses nothing"); + + assert_eq!(store.uncleared_sold().await.unwrap().len(), 0, "cleared"); + // The active auction is untouched, and the sold LISTING still exists as + // history — clearing is an acknowledgement, not a deletion of the sale. + assert_eq!(store.query_listings("active").await.unwrap().len(), 1); + assert_eq!(store.get_listing("900000206").await.unwrap().state, "sold"); + let pile = parse(&handle_market_query("active", &econ, &store, exp).await); + assert_eq!( + pile["auctionInfo"].as_array().unwrap().len(), + 1, + "only the active one" + ); + let c = parse(&handle_market_counts(&store, exp).await); + assert_eq!(c["sold"], 0, "the sold bucket empties on clear"); + assert_eq!( + econ.purchase_calls.load(Ordering::SeqCst), + 0, + "no economy call" + ); + assert_eq!(econ.balance().unwrap(), 7_777, "clearing moves no coins"); + } + + /// Clearing twice must be a no-op, because the client may retry. + #[tokio::test] + async fn clearing_sold_twice_is_idempotent() { + let (store, _d) = store_at("soldclear2").await; + seed_sold(&store, "900000208", 150).await; + assert_eq!( + store.clear_sold().await.unwrap(), + 1, + "first clear does work" + ); + assert_eq!( + store.clear_sold().await.unwrap(), + 0, + "second clears nothing" + ); + assert_eq!(store.get_listing("900000208").await.unwrap().state, "sold"); + } + + /// A sold row must also answer its own status poll, or the Transfer List would + /// render a row from /tradePile and then be told it does not exist. + #[tokio::test] + async fn sold_row_answers_its_status_poll() { + let (store, _d) = store_at("soldstatus").await; + let econ = CountingEconomy::with_balance(10_000); + seed_sold(&store, "900000209", 150).await; + let exp = SoldExperiment::from_values(Some("buyNow"), Some("1"), None); + + let one = + parse(&handle_market_status(Some("tradeIds=900000209"), &econ, &store, exp).await); + let recs = one["auctionInfo"].as_array().unwrap(); + assert_eq!(recs.len(), 1); + assert_eq!(recs[0]["tradeState"], "closed"); + assert_eq!(recs[0]["bidState"], "buyNow"); + assert_eq!(recs[0]["coinsProcessed"], 1); + + let all = parse(&handle_market_status(None, &econ, &store, exp).await); + assert_eq!( + all["auctionInfo"].as_array().unwrap().len(), + 1, + "unfiltered too" + ); + } + + /// `mark_sold` is the sale transition and must happen at most once, so a + /// duplicated counterparty settlement cannot double-sell. + #[tokio::test] + async fn mark_sold_is_once_only() { + let (store, _d) = store_at("soldonce").await; + seed_listing(&store, "900000210", 150).await; + assert!(store.mark_sold("900000210").await.unwrap(), "first wins"); + assert!( + !store.mark_sold("900000210").await.unwrap(), + "second is refused" + ); + assert!( + !store.mark_sold("nosuchlisting").await.unwrap(), + "unknown id" + ); + assert_eq!( + store.uncleared_sold().await.unwrap().len(), + 1, + "one sold row" + ); + } } diff --git a/openfut-utas-host/src/market_store.rs b/openfut-utas-host/src/market_store.rs index 28539ec..dd8c122 100644 --- a/openfut-utas-host/src/market_store.rs +++ b/openfut-utas-host/src/market_store.rs @@ -206,7 +206,11 @@ const CREATE_LISTINGS: &str = "CREATE TABLE IF NOT EXISTS listings ( state TEXT NOT NULL CHECK (state IN ('active','reserved','sold','cancelled')), created_at TEXT NOT NULL, item_json TEXT, - duration_secs INTEGER + duration_secs INTEGER, + -- Seller acknowledgement of a SOLD row, separate from the sale itself. Declared + -- here so a fresh store never needs the ALTER path below; the additive + -- migration exists only for stores created before this column. + cleared_at TEXT )"; fn now_millis() -> String { @@ -284,7 +288,17 @@ impl MarketStore { .iter() .map(|r| r.get::("name")) .collect(); - for (col, decl) in [("item_json", "TEXT"), ("duration_secs", "INTEGER")] { + for (col, decl) in [ + ("item_json", "TEXT"), + ("duration_secs", "INTEGER"), + // A SOLD listing is not the end of the seller's involvement: FIFA 17 has + // a bulk `DELETE …/trade/sold` verb (builder 0x1801647c0, request name + // RemoveAllSoldFromTradePile), which only makes sense if sold rows + // PERSIST in the seller's pile until cleared. `cleared_at` records that + // acknowledgement separately from the sale itself, so clearing a row can + // never be mistaken for re-settling it. + ("cleared_at", "TEXT"), + ] { if !existing.iter().any(|c| c == col) { sqlx::query(&format!("ALTER TABLE listings ADD COLUMN {col} {decl}")) .execute(&pool) @@ -474,6 +488,73 @@ impl MarketStore { } } + /// Mark a live listing SOLD in one step (`active | reserved -> sold`), for a + /// sale driven by a counterparty rather than by this client's own buy-now. + /// Returns whether this call was the one that sold it, so a replay is visible + /// to the caller instead of silently settling twice. + pub async fn mark_sold(&self, listing_id: &str) -> Result { + let affected = sqlx::query( + "UPDATE listings SET state = 'sold' \ + WHERE listing_id = ? AND state IN ('active', 'reserved')", + ) + .bind(listing_id) + .execute(&self.pool) + .await + .map_err(db)? + .rows_affected(); + Ok(affected == 1) + } + + /// Sold listings the seller has NOT yet cleared, newest first. + /// + /// Separate from [`Self::query_listings`] because "sold" and "still shown to + /// the seller" are different facts: a sold row stays in the pile until the + /// client acknowledges it via the bulk clear verb. + pub async fn uncleared_sold(&self) -> Result, MarketError> { + let rows = sqlx::query( + "SELECT * FROM listings WHERE state = 'sold' AND cleared_at IS NULL \ + ORDER BY created_at DESC", + ) + .fetch_all(&self.pool) + .await + .map_err(db)?; + Ok(rows.iter().map(row_to_listing).collect()) + } + + /// Acknowledge every uncleared sold listing (the bulk `DELETE …/trade/sold`). + /// Returns how many rows were cleared. + /// + /// This is PRESENTATION ONLY. It records that the seller has seen the sale; it + /// moves no coins and no ownership, because settlement already happened when + /// the sale completed. Clearing must never be able to pay anyone twice. + pub async fn clear_sold(&self) -> Result { + Ok(sqlx::query( + "UPDATE listings SET cleared_at = ? \ + WHERE state = 'sold' AND cleared_at IS NULL", + ) + .bind(now_millis()) + .execute(&self.pool) + .await + .map_err(db)? + .rows_affected()) + } + + /// Acknowledge ONE sold listing by id (the per-id `DELETE …/trade/{id}` form, + /// if the client turns out to use it for sold rows). Same presentation-only + /// contract as [`Self::clear_sold`]. + pub async fn clear_sold_one(&self, listing_id: &str) -> Result { + Ok(sqlx::query( + "UPDATE listings SET cleared_at = ? \ + WHERE listing_id = ? AND state = 'sold' AND cleared_at IS NULL", + ) + .bind(now_millis()) + .bind(listing_id) + .execute(&self.pool) + .await + .map_err(db)? + .rows_affected()) + } + /// Undo a reservation on a downstream failure (`reserved -> active`), so the /// listing becomes buyable again. Not in `reserved` -> [`MarketError::Conflict`]. pub async fn rollback_reservation(&self, listing_id: &str) -> Result<(), MarketError> { diff --git a/openfut-utas-host/src/sold_experiment.rs b/openfut-utas-host/src/sold_experiment.rs new file mode 100644 index 0000000..f5b038f --- /dev/null +++ b/openfut-utas-host/src/sold_experiment.rs @@ -0,0 +1,197 @@ +//! STAGING-ONLY seller-facing SOLD-row experiment. +//! +//! Static RE has exhausted CardsDLL on one question: for a `closed` row, +//! `IS_GLOW = (bidState != none)` and `INBOX = (bidState in {highest, buyNow})`, +//! so `closed/highest` and `closed/buyNow` are **bit-identical** to every native +//! consumer. But `bidState` is also published to the movie verbatim as `YOURBID`, +//! so the FUT ActionScript front end CAN separate them. This module exists to ask +//! the client which one it treats as the seller's sale, by holding every other +//! field constant and changing exactly that token. +//! +//! # Production safety +//! +//! Every knob is OFF unless its environment variable is set explicitly, and +//! [`SoldExperiment::enabled`] gates every projection at the call site. With no +//! env set this module changes nothing: `/tradePile` and `/trade/status` emit only +//! real active auctions (the Fix A invariant) and `/tradePile/counts` reports +//! `sold: 0` exactly as production does today. An unrecognised value is treated as +//! OFF rather than as a default token, because silently picking a token would +//! fabricate the very answer the experiment is meant to measure. +//! +//! # Why this cannot be "discovery" +//! +//! Our server IS the server, so nothing here recovers EA's original contract. It +//! is a controlled discriminator: the client's *reaction* (which bucket it draws +//! the row in, what it counts, and which request it issues to clear it) is the +//! observation. + +/// What `/tradePile/counts.count` should report while a sold row exists. FIFA 17's +/// exact semantics for `count` are unknown — it is either the number of live +/// auctions or the whole Transfer List membership — so it is a controlled variable +/// rather than a guess. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum CountMode { + /// `count` = active auctions only (current production behaviour). + Active, + /// `count` = active + uncleared sold (Transfer List membership). + ActivePlusSold, +} + +/// Resolved experiment configuration. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct SoldExperiment { + /// The `bidState` token to emit on a sold seller row. `None` disables every + /// part of the experiment. + pub bid_state: Option<&'static str>, + /// The `coinsProcessed` value to emit (published to Flash as `COINS_AWARDED`). + pub coins_processed: i64, + pub count_mode: CountMode, +} + +impl SoldExperiment { + /// All-off. This is what production runs. + pub const OFF: Self = Self { + bid_state: None, + coins_processed: 0, + count_mode: CountMode::Active, + }; + + /// Read the configuration from the environment. + /// + /// * `OPENFUT_FIFA17_SOLD_EXPERIMENT` — `highest` | `buyNow`; anything else + /// (including absent) is OFF. + /// * `OPENFUT_FIFA17_SOLD_COINS_PROCESSED` — `1` to emit 1, else 0. + /// * `OPENFUT_FIFA17_SOLD_COUNT_MODE` — `active_plus_sold`, else `active`. + pub fn from_env() -> Self { + Self::from_values( + std::env::var("OPENFUT_FIFA17_SOLD_EXPERIMENT") + .ok() + .as_deref(), + std::env::var("OPENFUT_FIFA17_SOLD_COINS_PROCESSED") + .ok() + .as_deref(), + std::env::var("OPENFUT_FIFA17_SOLD_COUNT_MODE") + .ok() + .as_deref(), + ) + } + + /// Pure resolver, so the parsing rules are testable without touching the + /// process environment. + pub fn from_values( + experiment: Option<&str>, + coins_processed: Option<&str>, + count_mode: Option<&str>, + ) -> Self { + // Matched case-insensitively for operator convenience, but ONLY the two + // real FIFA 17 tokens are accepted. `none`/`outbid` are deliberately not + // offered: neither can describe a completed sale, and `none` on a closed + // row clears IS_GLOW, which would test nothing. + let bid_state = match experiment.map(str::trim).unwrap_or("") { + s if s.eq_ignore_ascii_case("highest") => Some("highest"), + s if s.eq_ignore_ascii_case("buynow") => Some("buyNow"), + _ => None, + }; + Self { + bid_state, + coins_processed: i64::from(coins_processed == Some("1")), + count_mode: match count_mode.map(str::trim).unwrap_or("") { + s if s.eq_ignore_ascii_case("active_plus_sold") => CountMode::ActivePlusSold, + _ => CountMode::Active, + }, + } + } + + /// Whether any sold projection is active. Production: always false. + pub fn enabled(&self) -> bool { + self.bid_state.is_some() + } + + /// A one-line banner for the host's startup log, so a staging run can never be + /// mistaken for a production one in a capture. + pub fn banner(&self) -> String { + match self.bid_state { + None => "sold-experiment=OFF (production behaviour)".to_string(), + Some(b) => format!( + "sold-experiment=ON bidState={b} coinsProcessed={} countMode={:?} \ + -- STAGING ONLY, never production", + self.coins_processed, self.count_mode + ), + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn absent_env_is_off_and_matches_production() { + let e = SoldExperiment::from_values(None, None, None); + assert!(!e.enabled()); + assert_eq!(e, SoldExperiment::OFF); + assert_eq!(e.coins_processed, 0); + assert_eq!(e.count_mode, CountMode::Active); + } + + /// The whole point of the harness: exactly two tokens, and nothing else may + /// turn it on. A typo must not silently select a token and manufacture the + /// answer we are trying to measure. + #[test] + fn only_the_two_real_tokens_enable_it() { + for (input, expected) in [ + ("highest", Some("highest")), + ("HIGHEST", Some("highest")), + ("buyNow", Some("buyNow")), + ("buynow", Some("buyNow")), + (" highest ", Some("highest")), + ("off", None), + ("none", None), + ("outbid", None), + ("closed", None), + ("", None), + ("hihgest", None), // typo + ("1", None), + ] { + let e = SoldExperiment::from_values(Some(input), None, None); + assert_eq!(e.bid_state, expected, "input {input:?}"); + } + } + + #[test] + fn coins_processed_is_strictly_one_or_zero() { + for (input, expected) in [ + (Some("1"), 1), + (Some("0"), 0), + (Some("true"), 0), // only "1" means 1 — no fuzzy truthiness + (Some(""), 0), + (None, 0), + ] { + assert_eq!( + SoldExperiment::from_values(Some("highest"), input, None).coins_processed, + expected, + "input {input:?}" + ); + } + } + + #[test] + fn count_mode_defaults_to_production_behaviour() { + let mk = |m| SoldExperiment::from_values(Some("highest"), None, m).count_mode; + assert_eq!(mk(None), CountMode::Active); + assert_eq!(mk(Some("active")), CountMode::Active); + assert_eq!(mk(Some("active_plus_sold")), CountMode::ActivePlusSold); + assert_eq!(mk(Some("ACTIVE_PLUS_SOLD")), CountMode::ActivePlusSold); + assert_eq!(mk(Some("everything")), CountMode::Active, "unknown -> safe"); + } + + #[test] + fn banner_names_the_variant_under_test() { + assert!(SoldExperiment::OFF.banner().contains("OFF")); + let on = SoldExperiment::from_values(Some("buyNow"), Some("1"), None); + let b = on.banner(); + assert!(b.contains("bidState=buyNow"), "{b}"); + assert!(b.contains("coinsProcessed=1"), "{b}"); + assert!(b.contains("STAGING ONLY"), "{b}"); + } +} diff --git a/openfut-utas-host/tests/economy_concurrency.rs b/openfut-utas-host/tests/economy_concurrency.rs index f471290..e6dacc1 100644 --- a/openfut-utas-host/tests/economy_concurrency.rs +++ b/openfut-utas-host/tests/economy_concurrency.rs @@ -164,6 +164,8 @@ fn build_harness(base: &str, dir: &std::path::Path) -> Harness { "content pool derived from real Core content" ); let services = Arc::new(EconomyServices { + // Production default: the sold experiment is OFF. + sold_experiment: openfut_utas_host::sold_experiment::SoldExperiment::OFF, econ, market, piles, diff --git a/openfut-utas-host/tests/economy_differential.rs b/openfut-utas-host/tests/economy_differential.rs index 43584a4..838c46d 100644 --- a/openfut-utas-host/tests/economy_differential.rs +++ b/openfut-utas-host/tests/economy_differential.rs @@ -358,6 +358,8 @@ fn build_econ_server(base: &str, dir: &std::path::Path) -> (Server, HttpCoreClie "content pool derived from real Core content" ); let services = Arc::new(EconomyServices { + // Production default: the sold experiment is OFF. + sold_experiment: openfut_utas_host::sold_experiment::SoldExperiment::OFF, econ, market, piles, diff --git a/openfut-utas-host/tests/economy_failure.rs b/openfut-utas-host/tests/economy_failure.rs index 41df480..93e2d29 100644 --- a/openfut-utas-host/tests/economy_failure.rs +++ b/openfut-utas-host/tests/economy_failure.rs @@ -320,6 +320,8 @@ fn build_fail_harness(base: &str, dir: &std::path::Path) -> FailHarness { "content pool derived from real Core content" ); let services = Arc::new(EconomyServices { + // Production default: the sold experiment is OFF. + sold_experiment: openfut_utas_host::sold_experiment::SoldExperiment::OFF, econ: econ_dyn, market: market.clone(), piles: piles.clone(), @@ -355,6 +357,8 @@ impl FailHarness { /// generator path never consumes an entitlement. fn empty_pool_server(&self) -> Server { let services = Arc::new(EconomyServices { + // Production default: the sold experiment is OFF. + sold_experiment: openfut_utas_host::sold_experiment::SoldExperiment::OFF, econ: { let e: Arc = self.econ.clone(); e diff --git a/openfut-utas-host/tests/economy_integration.rs b/openfut-utas-host/tests/economy_integration.rs index 52f7c18..14ca30b 100644 --- a/openfut-utas-host/tests/economy_integration.rs +++ b/openfut-utas-host/tests/economy_integration.rs @@ -325,6 +325,8 @@ fn build_econ_server( "content pool derived from real Core content" ); let services = Arc::new(EconomyServices { + // Production default: the sold experiment is OFF. + sold_experiment: openfut_utas_host::sold_experiment::SoldExperiment::OFF, econ, market, piles, @@ -1186,6 +1188,8 @@ fn build_dead_core_server(dir: &std::path::Path, pass_url: &str) -> Server { ); let econ: Arc = Arc::new(HttpCoreClient::new(dead_url, "fifa17")); let services = Arc::new(EconomyServices { + // Production default: the sold experiment is OFF. + sold_experiment: openfut_utas_host::sold_experiment::SoldExperiment::OFF, econ, market, piles, diff --git a/scripts/sold-staging-down.py b/scripts/sold-staging-down.py new file mode 100755 index 0000000..8a6d9ad --- /dev/null +++ b/scripts/sold-staging-down.py @@ -0,0 +1,252 @@ +#!/usr/bin/env python3 +"""Tear down the isolated FIFA-17 SOLD staging stack brought up by +`scripts/sold-staging-up.py` -- and NOTHING else. + +Kill safety is the whole point of this file. `openfut-utas-host` and `openfut-core` +each name TWO live processes on this machine: the staging ones and the PRODUCTION +ones. So there is no pattern matching here at all: + + * every pid comes from the manifest the up script wrote; + * before any signal, /proc//cmdline is read and MUST contain the staging + directory -- production's cmdline never can, because staging runs binaries + copied into that directory; + * the known production pids are refused explicitly, as a second gate; + * only the process GROUP the up script created (pgid == pid, via + start_new_session) is signalled, so a responder thread/child cannot be orphaned; + * afterwards every staging port is proven free and production is proven alive. + + python3 scripts/sold-staging-down.py + python3 scripts/sold-staging-down.py --purge # also delete the staging dir +""" + +from __future__ import annotations + +import argparse +import json +import os +import shutil +import signal +import sys +import time + +DEFAULT_STAGING_DIR = "/home/alex/openfut-sold-staging" + +FORBIDDEN_PATHS = ("/home/alex/openfut-promotion/state",) +# Production processes that MUST be alive before and after this script runs. These +# two are the ones the batch contract names, and they live in the host pid view. +PROD_PIDS = {3631953: "prod utas-host", 3374264: "prod Core"} +# Reported but not gated: container pids change when the operator restarts the +# container, and a stale entry here would turn a successful teardown into a FATAL. +PROD_PIDS_INFO = {2090886: "prod blaze", 2091170: "prod python oracle", + 2090888: "prod pow"} +PROD_PORTS = (8099, 8199, 18080, 8443, 42127, 42130, 42131, 4216, 8080, 8081, 8094) + + +class Fatal(RuntimeError): + pass + + +def banner(title: str) -> None: + print() + print("=" * 78) + print(f"== {title}") + print("=" * 78) + + +def ok(msg: str) -> None: + print(f" [ OK ] {msg}") + + +def step(msg: str) -> None: + print(f" {msg}") + + +def safe_path(path: str) -> str: + real = os.path.realpath(path) + for bad in FORBIDDEN_PATHS: + if real == bad or real.startswith(bad + os.sep): + raise Fatal(f"REFUSING to touch production state: {path} -> {real}") + return path + + +def pid_alive(pid: int) -> bool: + try: + os.kill(pid, 0) + except ProcessLookupError: + return False + except PermissionError: + return True + return True + + +def cmdline_of(pid: int) -> str: + try: + with open(f"/proc/{pid}/cmdline", "rb") as fh: + return fh.read().replace(b"\0", b" ").decode(errors="replace").strip() + except OSError: + return "" + + +def listening_ports() -> set[int]: + """Ports in state LISTEN, from the kernel socket table. A trial bind() would + report EADDRINUSE for a stopped server's TIME_WAIT sockets and wrongly claim the + teardown failed.""" + ports: set[int] = set() + for path in ("/proc/net/tcp", "/proc/net/tcp6"): + try: + with open(path) as fh: + next(fh, None) # header + for line in fh: + fields = line.split() + if len(fields) < 4 or fields[3] != "0A": # TCP_LISTEN + continue + ports.add(int(fields[1].rsplit(":", 1)[1], 16)) + except OSError: + continue + return ports + + +def port_free(port: int) -> bool: + return port not in listening_ports() + + +def stop_one(rec: dict, staging_dir: str) -> str: + """Stop exactly one recorded process. Returns a human-readable outcome.""" + name, pid = rec["name"], int(rec["pid"]) + + known_prod = {**PROD_PIDS, **PROD_PIDS_INFO} + if pid in known_prod: + raise Fatal( + f"manifest entry {name} names PRODUCTION pid {pid} ({known_prod[pid]}). " + "REFUSING to signal anything from this manifest." + ) + if not pid_alive(pid): + return f"{name} pid {pid}: already gone" + + live = cmdline_of(pid) + if staging_dir not in live: + raise Fatal( + f"{name} pid {pid} is alive but its cmdline does NOT contain " + f"{staging_dir!r} -- pid reuse, or the wrong manifest. REFUSING to " + f"signal it.\n cmdline: {live!r}" + ) + + try: + pgid = os.getpgid(pid) + except OSError: + pgid = pid + recorded_pgid = int(rec.get("pgid", pid)) + if pgid != recorded_pgid: + raise Fatal( + f"{name} pid {pid} is in process group {pgid} but the manifest recorded " + f"{recorded_pgid} -- REFUSING to signal a group we did not create." + ) + if pgid != pid: + raise Fatal( + f"{name} pid {pid} is not its own group leader (pgid {pgid}) -- the up " + "script always starts a new session, so this is not our process." + ) + + os.killpg(pgid, signal.SIGTERM) + deadline = time.monotonic() + 15.0 + while time.monotonic() < deadline and pid_alive(pid): + time.sleep(0.1) + if pid_alive(pid): + os.killpg(pgid, signal.SIGKILL) + deadline = time.monotonic() + 10.0 + while time.monotonic() < deadline and pid_alive(pid): + time.sleep(0.1) + if pid_alive(pid): + raise Fatal(f"{name} pid {pid} survived SIGKILL") + return f"{name} pid {pid} (pgid {pgid}): stopped and verified gone" + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + ap.add_argument("--dir", default=os.environ.get("OPENFUT_SOLD_STAGING_DIR", + DEFAULT_STAGING_DIR), + help=f"staging directory (default: {DEFAULT_STAGING_DIR})") + ap.add_argument("--purge", action="store_true", + help="delete the staging directory after stopping (default: keep " + "the databases and logs as evidence)") + args = ap.parse_args() + + staging_dir = safe_path(os.path.abspath(args.dir)) + manifest_path = safe_path(os.path.join(staging_dir, "manifest.json")) + + try: + banner("STOPPING THE STAGING STACK (recorded pids only)") + step(f"staging dir : {staging_dir}") + if not os.path.exists(manifest_path): + print(f" no manifest at {manifest_path} -- nothing was recorded, so " + "nothing will be signalled.") + print(" If a staging process is somehow still running, find it with " + "its cmdline (it contains the staging dir) and stop it by pid.") + return 0 + with open(manifest_path) as fh: + manifest = json.load(fh) + if manifest.get("staging_dir") != staging_dir: + raise Fatal( + f"manifest staging_dir {manifest.get('staging_dir')!r} != " + f"{staging_dir!r} -- REFUSING to act on a foreign manifest." + ) + step(f"variant : {manifest.get('variant')}") + + for rec in manifest.get("processes", []): + ok(stop_one(rec, staging_dir)) + + banner("PROVE STAGING IS GONE") + ports = manifest.get("ports", {}) + for name, port in sorted(ports.items(), key=lambda kv: kv[1]): + if port in PROD_PORTS: + raise Fatal(f"manifest port {name}={port} is a PRODUCTION port") + if not port_free(port): + raise Fatal(f"staging port {name}={port} is STILL listening") + ok(f"staging port {name} {port} free") + + leftovers = [] + for rec in manifest.get("processes", []): + pid = int(rec["pid"]) + if pid_alive(pid) and staging_dir in cmdline_of(pid): + leftovers.append(f"{rec['name']} pid {pid}") + if leftovers: + raise Fatal("staging processes still alive: " + ", ".join(leftovers)) + ok("no recorded staging process is alive") + + banner("PROVE PRODUCTION IS STILL UP") + dead = [f"{what} pid {pid}" for pid, what in PROD_PIDS.items() + if not pid_alive(pid)] + for pid, what in PROD_PIDS.items(): + if pid_alive(pid): + ok(f"{what} pid {pid} alive") + if dead: + raise Fatal("production process(es) NOT alive: " + ", ".join(dead)) + for pid, what in PROD_PIDS_INFO.items(): + state = "alive" if pid_alive(pid) else "not found (informational only)" + step(f"{what} pid {pid} {state}") + + if args.purge: + shutil.rmtree(safe_path(staging_dir), ignore_errors=True) + ok(f"purged {staging_dir}") + else: + os.replace(manifest_path, safe_path(manifest_path + ".stopped")) + ok(f"kept {staging_dir} (manifest renamed to manifest.json.stopped so a " + "fresh `up` is allowed)") + + banner("REMINDER: REVERT THE CLIENT") + print(' On 10.10.0.105, restore "/mnt/games/FIFA 17/openfut.cfg" to:') + print() + print(" host=10.10.0.120") + print(" https_port=8443") + print(" blaze_redirector_port=42127") + print(" blaze_main_port=42130") + print() + print(" then RELAUNCH the FIFA 17 client. See docs/SOLD_STAGING_RUNBOOK.md.") + return 0 + except Fatal as exc: + print(f"\nFATAL: {exc}", file=sys.stderr) + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/sold-staging-up.py b/scripts/sold-staging-up.py new file mode 100755 index 0000000..143053e --- /dev/null +++ b/scripts/sold-staging-up.py @@ -0,0 +1,1028 @@ +#!/usr/bin/env python3 +"""Bring up a COMPLETE, ISOLATED FIFA-17 staging stack for the seller-facing SOLD +A/B experiment, so a real FIFA 17 client can be pointed at it without production +being touched in any way. + +ONE entry point. It starts, in order: + + 1. staging Core 127.0.0.1:18081 throwaway sqlite under the staging dir + 2. staging utas-host 0.0.0.0:8299 own market/pile/identity/clientdata files + 3. staging Blaze 0.0.0.0:42327 (redirector) / 42330 (main) / 42331 (nucleus) + +and then prints the three `openfut.cfg` lines the operator must put on the client. +Tear the whole thing down with `scripts/sold-staging-down.py`. + +WHY the client only needs Blaze ports: FIFA 17 learns the UTAS base URL from Blaze +(`blaze_responder_v3b.py`'s `UTAS_BASE`, where the `8099` is HARDCODED). This script +copies the responder into the staging dir and rewrites that literal to the staging +UTAS port, so pointing the client at staging Blaze is sufficient to move UTAS too. + +ISOLATION, enforced not assumed: + * production ports 8099 8199 18080 8443 42127 42130 42131 4216 8080 8081 8094 are + a hard deny-list: never bound, never connected to, and every chosen staging port + is checked against it AND checked free before anything is launched; + * nothing under /home/alex/openfut-promotion/state/ is ever opened -- every path + this script touches goes through `safe_path()`, which refuses that prefix; + * the Core/utas-host binaries are COPIED into the staging dir and run from there, + so (a) a later `cargo build` cannot change what staging is running and (b) every + staging process's /proc cmdline provably contains the staging directory, which is + what the down script requires before it will signal anything; + * `OPENFUT_UTAS_PYTHON_URL` points at an unused loopback port, so any Python + fallback fails closed and loudly instead of silently serving production data; + * nothing on the FIFA client machine (10.10.0.105) is modified -- the operator + edits `openfut.cfg` by hand, using the block this script prints. + +Read-only reuse of production: staging Blaze advertises the production roster +(10.10.0.120:8081) and POW content/API hosts (10.10.0.120:8085 / :8094) verbatim, the +same values production Blaze advertises. Those services hold NO economy state (roster +XML and POW content are static), staging never connects to them itself -- it only +hands the client the same strings -- and they are therefore shared deliberately. + + python3 scripts/sold-staging-up.py --variant highest + python3 scripts/sold-staging-up.py --variant buyNow --coins-processed 1 \ + --count-mode active_plus_sold + python3 scripts/sold-staging-up.py --variant off # sold projection disabled +""" + +from __future__ import annotations + +import argparse +import http.client +import json +import os +import re +import shutil +import signal +import socket +import sqlite3 +import subprocess +import sys +import time + +# --- fixed facts ------------------------------------------------------------------- + +REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + +# Production. Never bind, never connect, never open. +FORBIDDEN_PORTS = frozenset( + {8099, 8199, 18080, 8443, 42127, 42130, 42131, 4216, 8080, 8081, 8094} +) +FORBIDDEN_PATHS = ("/home/alex/openfut-promotion/state",) +PROD_PIDS = {3631953: "prod utas-host", 3374264: "prod Core"} + +# The staging port block. One obvious place; every one of these is asserted free. +# 42227 (the first choice for the redirector) is permanently occupied by +# openfut-redirector-host, so the redirector sits at 42327. +CORE_PORT = 18081 +HOST_PORT = 8299 +BLAZE_REDIR_PORT = 42327 +BLAZE_MAIN_PORT = 42330 +BLAZE_NUCLEUS_PORT = 42331 +# Deliberately dead: the utas-host requires a Python upstream, and this one must +# never resolve to the production oracle on 8199. +DEAD_PYTHON_PORT = 8399 + +STAGING_PORTS = { + "staging Core": CORE_PORT, + "staging utas-host": HOST_PORT, + "staging blaze redirector": BLAZE_REDIR_PORT, + "staging blaze main": BLAZE_MAIN_PORT, + "staging blaze nucleus": BLAZE_NUCLEUS_PORT, +} + +DEFAULT_STAGING_DIR = "/home/alex/openfut-sold-staging" + +# What staging Blaze tells the client about itself and about the economy-free +# auxiliary services. Same advertise IP and same POW hosts production Blaze uses. +ADVERTISE = "10.10.0.120" +BIND = "0.0.0.0" +POW_CONTENT_HOST = "10.10.0.120:8085" +POW_HOST = "10.10.0.120:8094" + +BLAZE_SRC = os.path.join(REPO, "fifa17-recon", "tools", "blaze_responder_v3b.py") +BLAZE_ASSETS = ("redir_cert.pem", "redir_key.pem") + +# Emitted FIFA17 content, from a build tree OUTSIDE the forbidden production state +# directory. `cards` is Core's content pack (a bare JSON array of CardDefinition); +# `catalog` is the adapter's identity catalog (card id -> asset id/version/rareflag). +CONTENT_SRC = "/home/alex/openfut-post-p1/staging/emit/content" +CARDS_NAME = "fifa17-production-cards.json" +CATALOG_NAME = "fifa17-production-catalog.json" + +GAME = "fifa17" +PERSONA_ID = "33068179" +PERSONA_NAME = "CAGE" +TS = "2026-01-01T00:00:00Z" + +# Seller A is the real FIFA persona, so the retail client logs into THIS profile +# (Core resolves the active profile by game_id, and X-OpenFUT-Game is `fifa17`). +SELLER_PROFILE = "prof-seller-a-cage" +SELLER_CLUB = "club-seller-a-cage" +SELLER_COINS = 1_000 +SELLER_SQUAD = "squad-seller-a" +# Buyer B is a synthetic second club. It is parked on its OWN game_id so it can +# never become the active `fifa17` profile -- Core is single-profile-per-game. +BUYER_PROFILE = "prof-buyer-b" +BUYER_CLUB = "club-buyer-b" +BUYER_GAME = "fifa17-buyer-b" +BUYER_COINS = 20_000 + +# 11 starters + 1 disposable item for the seller. Every card_id here exists in BOTH +# the content pack (so Core's content preflight passes) and the identity catalog (so +# the host can resolve a resourceId); both memberships are asserted before launch. +SELLER_SQUAD_CARDS = [ + ("owned-a-gk", "fifa17_84053575"), # Manuel Neuer GK 97 + ("owned-a-lb", "fifa17_151192389"), # David Alaba LB 91 + ("owned-a-cb1", "fifa17_134381968"), # Thiago Silva CB 92 + ("owned-a-cb2", "fifa17_151177437"), # Diego Godin CB 92 + ("owned-a-rb", "fifa17_100785235"), # Philipp Lahm RB 90 + ("owned-a-cm1", "fifa17_151171947"), # Luka Modric CM 93 + ("owned-a-cm2", "fifa17_84054731"), # Ivan Rakitic CM 92 + ("owned-a-cm3", "fifa17_134400249"), # Toni Kroos CM 91 + ("owned-a-lw", "fifa17_83906881"), # Cristiano Ronaldo LW 99 + ("owned-a-st", "fifa17_117617092"), # Luis Suarez ST 95 + ("owned-a-rw", "fifa17_84044103"), # Lionel Messi RW 98 +] +# THE disposable item: what the operator lists and sells during the experiment. +DISPOSABLE_ITEM = "owned-a-disposable" +DISPOSABLE_CARD = "fifa17_232273" # Nelson Atiagli LB 51, rareflag 1 + +READY_TIMEOUT_S = 60.0 + + +# --- output ------------------------------------------------------------------------ + + +def banner(title: str) -> None: + print() + print("=" * 78) + print(f"== {title}") + print("=" * 78) + + +def step(msg: str) -> None: + print(f" {msg}") + + +def ok(msg: str) -> None: + print(f" [ OK ] {msg}") + + +class Fatal(RuntimeError): + """Anything that must abort bring-up loudly rather than degrade.""" + + +# --- isolation guards -------------------------------------------------------------- + + +def safe_path(path: str) -> str: + """Every filesystem path in this script goes through here. Refuses the live + production state directory outright -- a typo cannot reach prod-core.db.""" + real = os.path.realpath(path) + for bad in FORBIDDEN_PATHS: + if real == bad or real.startswith(bad + os.sep): + raise Fatal(f"REFUSING to touch production state: {path} -> {real}") + return path + + +def check_port_allowed(port: int, what: str) -> None: + if port in FORBIDDEN_PORTS: + raise Fatal(f"REFUSING: {what} port {port} is a PRODUCTION port") + + +def listening_ports() -> set[int]: + """Every TCP port in state LISTEN in this network namespace, read straight from + the kernel socket table. + + A trial bind() is the wrong test: after a server exits, its accepted sockets sit + in TIME_WAIT holding the same local port, so bind() reports EADDRINUSE for a + minute even though nothing is serving -- and every server here sets SO_REUSEADDR + and would bind fine. This mirrors `ss -ltn` (and host-lifecycle.sh's + hl_port_listening), which is the question actually being asked.""" + ports: set[int] = set() + for path in ("/proc/net/tcp", "/proc/net/tcp6"): + try: + with open(path) as fh: + next(fh, None) # header + for line in fh: + fields = line.split() + if len(fields) < 4 or fields[3] != "0A": # TCP_LISTEN + continue + ports.add(int(fields[1].rsplit(":", 1)[1], 16)) + except OSError: + continue + return ports + + +def port_free(port: int) -> bool: + return port not in listening_ports() + + +def assert_ports_free() -> None: + busy = [] + for what, port in STAGING_PORTS.items(): + check_port_allowed(port, what) + if not port_free(port): + busy.append(f"{what} {port}") + check_port_allowed(DEAD_PYTHON_PORT, "dead python upstream") + if not port_free(DEAD_PYTHON_PORT): + busy.append( + f"dead python upstream {DEAD_PYTHON_PORT} (it MUST stay unbound so the " + "Python fallback fails closed)" + ) + if busy: + raise Fatal( + "REFUSING to start -- these staging ports are not free:\n " + + "\n ".join(busy) + + "\n Nothing was launched. Free them, or edit the port block at the " + "top of this script." + ) + ok( + "staging ports free and none is a production port: " + + ", ".join(str(p) for p in STAGING_PORTS.values()) + ) + + +def pid_alive(pid: int) -> bool: + try: + os.kill(pid, 0) + except ProcessLookupError: + return False + except PermissionError: + return True # exists, owned by root (the prod stack runs under sudo) + return True + + +def cmdline_of(pid: int) -> str: + try: + with open(f"/proc/{pid}/cmdline", "rb") as fh: + return fh.read().replace(b"\0", b" ").decode(errors="replace").strip() + except OSError: + return "" + + +def assert_prod_alive(where: str) -> None: + for pid, what in PROD_PIDS.items(): + if not pid_alive(pid): + raise Fatal(f"{what} pid {pid} is NOT alive at {where} -- stop and investigate") + ok( + f"production untouched at {where}: " + + ", ".join(f"{what} pid {pid} alive" for pid, what in PROD_PIDS.items()) + ) + + +# --- HTTP --------------------------------------------------------------------------- + + +def http_get(port: int, path: str, timeout: float = 5.0): + check_port_allowed(port, "HTTP request") + conn = http.client.HTTPConnection("127.0.0.1", port, timeout=timeout) + try: + conn.request("GET", path, headers={"X-OpenFUT-Game": GAME}) + resp = conn.getresponse() + raw = resp.read() + return resp.status, raw.decode("utf-8", "replace") + finally: + conn.close() + + +def wait_http(port: int, path: str, proc, log_path: str, label: str) -> None: + deadline = time.monotonic() + READY_TIMEOUT_S + last = "no attempt" + while time.monotonic() < deadline: + if proc.poll() is not None: + raise Fatal(f"{label} exited {proc.returncode} during startup\n{tail(log_path)}") + try: + status, _ = http_get(port, path) + if status < 500: + return + last = f"HTTP {status}" + except OSError as exc: + last = f"{type(exc).__name__}: {exc}" + time.sleep(0.15) + raise Fatal( + f"{label} never answered {path} on 127.0.0.1:{port} within " + f"{READY_TIMEOUT_S:.0f}s (last: {last})\n{tail(log_path)}" + ) + + +def wait_tcp(ports: list[int], proc, log_path: str, label: str) -> None: + deadline = time.monotonic() + READY_TIMEOUT_S + pending = list(ports) + while time.monotonic() < deadline and pending: + if proc.poll() is not None: + raise Fatal(f"{label} exited {proc.returncode} during startup\n{tail(log_path)}") + still = [] + for port in pending: + check_port_allowed(port, f"{label} readiness probe") + s = socket.socket() + s.settimeout(1.0) + try: + s.connect(("127.0.0.1", port)) + except OSError: + still.append(port) + finally: + s.close() + pending = still + if pending: + time.sleep(0.15) + if pending: + raise Fatal(f"{label} never listened on {pending}\n{tail(log_path)}") + + +def tail(log_path: str, lines: int = 30) -> str: + try: + with open(log_path, errors="replace") as fh: + body = fh.read().splitlines() + except OSError: + return f"(no log at {log_path})" + return f"--- {os.path.basename(log_path)} tail ---\n" + "\n".join(body[-lines:]) + + +# --- staging directory -------------------------------------------------------------- + + +class Layout: + def __init__(self, root: str) -> None: + self.root = safe_path(os.path.abspath(root)) + self.bin = os.path.join(self.root, "bin") + self.content = os.path.join(self.root, "content") + self.blaze = os.path.join(self.root, "blaze") + self.logs = os.path.join(self.root, "logs") + self.core_db = os.path.join(self.root, "staging-core.db") + self.market_db = os.path.join(self.root, "staging-market.db") + self.pile_db = os.path.join(self.root, "staging-pile.db") + self.identity = os.path.join(self.root, "staging-identity.json") + self.clientdata = os.path.join(self.root, "staging-clientdata.json") + self.cards = os.path.join(self.content, CARDS_NAME) + self.catalog = os.path.join(self.content, CATALOG_NAME) + self.blaze_script = os.path.join(self.blaze, "blaze_responder_staging.py") + self.blaze_rx = os.path.join(self.blaze, "rx") + self.manifest = os.path.join(self.root, "manifest.json") + self.core_bin = os.path.join(self.bin, "openfut-core") + self.host_bin = os.path.join(self.bin, "openfut-utas-host") + self.core_log = os.path.join(self.logs, "core.log") + self.host_log = os.path.join(self.logs, "utas-host.log") + self.blaze_log = os.path.join(self.logs, "blaze.log") + # The responder's OWN log(), kept separate from its stdout/stderr file so + # two writers never interleave in one file. + self.blaze_responder_log = os.path.join(self.logs, "blaze-responder.log") + for p in vars(self).values(): + safe_path(p) + + def make_dirs(self) -> None: + for d in (self.root, self.bin, self.content, self.blaze, self.logs, + self.blaze_rx): + os.makedirs(safe_path(d), exist_ok=True) + + +def refuse_if_up(lay: Layout) -> None: + """A previous stack still running must be torn down by the down script, never + stepped on: two stacks would fight over the same sqlite files.""" + if not os.path.exists(lay.manifest): + return + try: + with open(safe_path(lay.manifest)) as fh: + manifest = json.load(fh) + except (OSError, ValueError): + return + live = [] + for proc in manifest.get("processes", []): + pid = int(proc["pid"]) + if pid_alive(pid) and lay.root in cmdline_of(pid): + live.append(f"{proc['name']} pid {pid}") + if live: + raise Fatal( + "a previous staging stack is STILL UP:\n " + + "\n ".join(live) + + "\n Run: python3 scripts/sold-staging-down.py" + + "\n Nothing was launched and nothing was signalled." + ) + + +def reset_throwaway_state(lay: Layout) -> None: + """Every staging database is throwaway by definition, so a re-`up` after a clean + `down` starts from nothing rather than refusing. Only files this script created + are removed, and only after `refuse_if_up` proved no stack is running.""" + removed = [] + for path in (lay.core_db, lay.market_db, lay.pile_db, lay.identity, + lay.clientdata, lay.manifest, lay.manifest + ".stopped"): + for candidate in (path, path + "-wal", path + "-shm"): + if os.path.exists(safe_path(candidate)): + os.remove(safe_path(candidate)) + removed.append(os.path.basename(candidate)) + if removed: + ok("removed stale throwaway state from a previous run: " + ", ".join(removed)) + + +def materialise(lay: Layout) -> None: + lay.make_dirs() + for src, dst in ( + (os.path.join(REPO, "target", "release", "openfut-core"), lay.core_bin), + (os.path.join(REPO, "target", "release", "openfut-utas-host"), lay.host_bin), + ): + if not os.path.isfile(src): + raise Fatal( + f"not built: {src}\n cargo build --release -p openfut-core " + "-p openfut-utas-host" + ) + shutil.copy2(safe_path(src), safe_path(dst)) + os.chmod(dst, 0o755) + ok(f"binaries copied into {lay.bin} (a later rebuild cannot change staging)") + + for name, dst in ((CARDS_NAME, lay.cards), (CATALOG_NAME, lay.catalog)): + src = os.path.join(CONTENT_SRC, name) + if not os.path.isfile(src): + raise Fatal(f"missing FIFA17 content source {src}") + shutil.copy2(safe_path(src), safe_path(dst)) + ok(f"FIFA17 content copied from {CONTENT_SRC} (outside production state)") + + +def assert_seed_cards_resolvable(lay: Layout) -> None: + """Core's content preflight rejects any owned card whose card_id is not a loaded + CardDefinition, and the host refuses to shape a /club item with no catalog + identity. Prove BOTH memberships now, not via a startup crash later.""" + with open(safe_path(lay.cards)) as fh: + pack_ids = {c["id"] for c in json.load(fh)} + with open(safe_path(lay.catalog)) as fh: + catalog_ids = set(json.load(fh)["cards"]) + wanted = [c for _, c in SELLER_SQUAD_CARDS] + [DISPOSABLE_CARD] + missing_pack = sorted(set(wanted) - pack_ids) + missing_cat = sorted(set(wanted) - catalog_ids) + if missing_pack or missing_cat: + raise Fatal( + "seed card ids are not resolvable -- Core or the host would fail at " + f"startup.\n absent from content pack: {missing_pack}" + f"\n absent from identity catalog: {missing_cat}" + ) + ok( + f"all {len(wanted)} seed card ids present in BOTH the content pack " + f"({len(pack_ids)} defs) and the identity catalog ({len(catalog_ids)} entries)" + ) + + +# --- blaze patching ----------------------------------------------------------------- + +# Every substitution is anchored to the whole assignment line and must apply exactly +# once. A silent no-op here would leave staging Blaze advertising PRODUCTION UTAS. +def blaze_patches(lay: Layout) -> list[tuple[str, str, str]]: + return [ + ("REDIR_PORT", r"^REDIR_PORT = 42127$", f"REDIR_PORT = {BLAZE_REDIR_PORT}"), + ("BLAZE_PORT", r"^BLAZE_PORT = 42130$", f"BLAZE_PORT = {BLAZE_MAIN_PORT}"), + ("NUCLEUS_PORT", r"^NUCLEUS_PORT = 42131$", + f"NUCLEUS_PORT = {BLAZE_NUCLEUS_PORT}"), + ("UTAS_BASE", r'^UTAS_BASE = "http://%s:8099/" % _ADVERTISE$', + f'UTAS_BASE = "http://%s:{HOST_PORT}/" % _ADVERTISE'), + # Not protocol values, but the responder's two hardcoded /tmp paths: left + # alone, a staging run would write its frames and log into the shared + # host /tmp and make a capture ambiguous about which stack produced it. + ("LOG", r'^LOG = "/tmp/blaze_responder\.log"$', + f'LOG = "{lay.blaze_responder_log}"'), + ("RXDIR", r'^RXDIR = "/tmp/blaze_rx"$', f'RXDIR = "{lay.blaze_rx}"'), + ] + + +def patch_blaze(lay: Layout) -> None: + if not os.path.isfile(BLAZE_SRC): + raise Fatal(f"missing blaze responder {BLAZE_SRC}") + with open(safe_path(BLAZE_SRC)) as fh: + text = fh.read() + + for name, pattern, replacement in blaze_patches(lay): + text, n = re.subn(pattern, replacement.replace("\\", "\\\\"), text, + flags=re.MULTILINE) + if n != 1: + raise Fatal( + f"blaze patch {name} applied {n} times, expected exactly 1 " + f"(pattern {pattern!r}). The responder changed shape -- REFUSING to " + "run a half-patched copy that could point at production." + ) + step(f"patched {name:<12} -> {replacement.split(' = ', 1)[1]}") + + with open(safe_path(lay.blaze_script), "w") as fh: + fh.write(text) + + for asset in BLAZE_ASSETS: + src = os.path.join(os.path.dirname(BLAZE_SRC), asset) + if not os.path.isfile(src): + raise Fatal(f"missing blaze TLS asset {src}") + # The responder resolves CERT/KEY relative to its own directory, so the + # assets are copied rather than patched. + shutil.copy2(safe_path(src), safe_path(os.path.join(lay.blaze, asset))) + + verify_blaze_patch(lay) + + +def verify_blaze_patch(lay: Layout) -> None: + """Re-read the file from disk and prove the patched copy cannot reach production + Blaze ports or production UTAS.""" + with open(safe_path(lay.blaze_script)) as fh: + lines = fh.read().splitlines() + + def assignment(name: str) -> str: + hits = [ln for ln in lines if re.match(rf"^{name} = ", ln)] + if len(hits) != 1: + raise Fatal(f"patched blaze copy has {len(hits)} `{name} =` lines") + return hits[0] + + expected = { + "REDIR_PORT": f"REDIR_PORT = {BLAZE_REDIR_PORT}", + "BLAZE_PORT": f"BLAZE_PORT = {BLAZE_MAIN_PORT}", + "NUCLEUS_PORT": f"NUCLEUS_PORT = {BLAZE_NUCLEUS_PORT}", + "UTAS_BASE": f'UTAS_BASE = "http://%s:{HOST_PORT}/" % _ADVERTISE', + "LOG": f'LOG = "{lay.blaze_responder_log}"', + "RXDIR": f'RXDIR = "{lay.blaze_rx}"', + } + for name, want in expected.items(): + got = assignment(name) + if got != want: + raise Fatal(f"patched blaze {name} is {got!r}, expected {want!r}") + + utas = assignment("UTAS_BASE") + if ":8099" in utas: + raise Fatal(f"patched blaze STILL advertises production UTAS: {utas!r}") + ok(f"patched blaze UTAS_BASE = {utas.split(' = ', 1)[1]} (no :8099)") + ok( + "patched blaze ports: redirector " + f"{BLAZE_REDIR_PORT} / main {BLAZE_MAIN_PORT} / nucleus " + f"{BLAZE_NUCLEUS_PORT} (no 42127/42130/42131)" + ) + + +# --- seeding ------------------------------------------------------------------------ + + +def seed_core_db(lay: Layout) -> None: + """Two identities by direct SQL, against the schema Core just migrated. + + Column sets are from openfut-core/migrations/0001_initial.sql plus 0016's + profiles.game_id. Seller A carries game_id `fifa17` so the retail client (which + sends X-OpenFUT-Game: fifa17) resolves to it; Buyer B is parked on its own + game_id so it can never shadow the seller as the active fifa17 profile. + """ + conn = sqlite3.connect(safe_path(lay.core_db), timeout=15) + try: + conn.execute("PRAGMA busy_timeout = 15000") + with conn: + conn.executemany( + "INSERT INTO profiles (id, username, level, xp, created_at, " + "updated_at, game_id) VALUES (?, ?, 1, 0, ?, ?, ?)", + [ + (SELLER_PROFILE, PERSONA_NAME, TS, TS, GAME), + (BUYER_PROFILE, "BUYER-B", TS, TS, BUYER_GAME), + ], + ) + conn.executemany( + "INSERT INTO clubs (id, profile_id, name, coins, level, created_at, " + "updated_at) VALUES (?, ?, ?, ?, 1, ?, ?)", + [ + (SELLER_CLUB, SELLER_PROFILE, f"{PERSONA_NAME} FC", SELLER_COINS, + TS, TS), + (BUYER_CLUB, BUYER_PROFILE, "Buyer B FC", BUYER_COINS, TS, TS), + ], + ) + conn.executemany( + "INSERT INTO owned_cards (id, club_id, card_id, is_loan, " + "acquired_at) VALUES (?, ?, ?, 0, ?)", + [(item, SELLER_CLUB, card, TS) for item, card in SELLER_SQUAD_CARDS] + + [(DISPOSABLE_ITEM, SELLER_CLUB, DISPOSABLE_CARD, TS)], + ) + conn.execute( + "INSERT INTO squads (id, club_id, name, formation, created_at, " + "updated_at) VALUES (?, ?, ?, ?, ?, ?)", + (SELLER_SQUAD, SELLER_CLUB, "Staging XI", "4-3-3", TS, TS), + ) + conn.executemany( + "INSERT INTO squad_players (id, squad_id, owned_card_id, " + "position_index, is_captain, is_on_bench) VALUES (?, ?, ?, ?, ?, 0)", + [ + (f"sp-{idx}", SELLER_SQUAD, item, idx, 1 if idx == 0 else 0) + for idx, (item, _) in enumerate(SELLER_SQUAD_CARDS) + ], + ) + finally: + conn.close() + ok( + f"seeded Seller A ({PERSONA_NAME}, persona {PERSONA_ID}, {SELLER_COINS} coins, " + f"{len(SELLER_SQUAD_CARDS)} starters + 1 disposable) and Buyer B " + f"({BUYER_COINS} coins)" + ) + + +def db_summary(lay: Layout) -> str: + conn = sqlite3.connect(safe_path(lay.core_db), timeout=15) + try: + clubs = conn.execute("SELECT id, coins FROM clubs ORDER BY id").fetchall() + owned = conn.execute( + "SELECT club_id, COUNT(*) FROM owned_cards GROUP BY club_id" + ).fetchall() + finally: + conn.close() + return f"clubs={dict(clubs)} owned={dict(owned)}" + + +# --- launching ---------------------------------------------------------------------- + + +class Launched: + def __init__(self, name: str, proc, log: str, port_note: str) -> None: + self.name = name + self.proc = proc + self.log = log + self.port_note = port_note + + def record(self) -> dict: + return { + "name": self.name, + "pid": self.proc.pid, + "pgid": os.getpgid(self.proc.pid), + "cmdline": cmdline_of(self.proc.pid), + "log": self.log, + "ports": self.port_note, + } + + +def spawn(argv: list[str], env: dict, cwd: str, log_path: str, append=False): + """Start detached (own session) so the stack survives this script exiting, and + so the down script can signal exactly this process group and nothing else.""" + log = open(safe_path(log_path), "a" if append else "w", buffering=1) + try: + log.write(f"\n---- launch {' '.join(argv)} @ {time.strftime('%FT%TZ')} ----\n") + return subprocess.Popen( + argv, + cwd=safe_path(cwd), + env=env, + stdout=log, + stderr=subprocess.STDOUT, + start_new_session=True, + ) + finally: + log.close() # the child holds its own dup of the fd + + +def core_env(lay: Layout) -> dict: + return dict( + os.environ, + LISTEN_ADDR=f"127.0.0.1:{CORE_PORT}", + DATABASE_URL=f"sqlite://{lay.core_db}", + DATA_DIR=os.path.join(REPO, "openfut-core", "data"), + OPENFUT_CONTENT_PACKS=lay.cards, + RUST_LOG="openfut_core=info", + ) + + +def migrate_core(lay: Layout) -> None: + """Core owns its schema and has no migrate-only subcommand, so the fixture + cannot be written into an empty file: start it once, let it migrate, stop it, + seed, then start the long-lived instance.""" + proc = spawn([lay.core_bin], core_env(lay), lay.root, lay.core_log) + try: + wait_http(CORE_PORT, "/health", proc, lay.core_log, "staging Core (migrate)") + ok(f"staging Core migrated {os.path.basename(lay.core_db)}") + finally: + proc.terminate() + try: + proc.wait(timeout=20) + except subprocess.TimeoutExpired: + proc.kill() + proc.wait(timeout=20) + + +def start_core(lay: Layout) -> Launched: + proc = spawn([lay.core_bin], core_env(lay), lay.root, lay.core_log, append=True) + wait_http(CORE_PORT, "/health", proc, lay.core_log, "staging Core") + ok(f"staging Core ready on 127.0.0.1:{CORE_PORT} (pid {proc.pid})") + return Launched("staging-core", proc, lay.core_log, f"127.0.0.1:{CORE_PORT}") + + +def start_host(lay: Layout, variant: str, coins_processed: str, count_mode: str) -> Launched: + env = dict( + os.environ, + OPENFUT_UTAS_HOST_ADDR=f"{BIND}:{HOST_PORT}", + OPENFUT_CORE_URL=f"http://127.0.0.1:{CORE_PORT}", + # NOT the production oracle on 8199. Nothing listens here, so any Python + # fallback fails closed and shows up in the log. + OPENFUT_UTAS_PYTHON_URL=f"http://127.0.0.1:{DEAD_PYTHON_PORT}", + OPENFUT_FIFA17_TABLES_DIR=os.path.join(REPO, "fifa17-recon", "data", "tables"), + OPENFUT_FIFA17_CATALOG=lay.catalog, + OPENFUT_IDENTITY_STORE=lay.identity, + OPENFUT_CLIENTDATA_DB=lay.clientdata, + OPENFUT_PERSONA_ID=PERSONA_ID, + OPENFUT_MARKET_DB=lay.market_db, + OPENFUT_PILE_DB=lay.pile_db, + RUST_LOG="info", + ) + # OFF must mean "unset", not "set to something unrecognised": the host treats an + # unrecognised token as OFF, but leaving the variable behind invites confusion. + for key in ("OPENFUT_FIFA17_SOLD_EXPERIMENT", "OPENFUT_FIFA17_SOLD_COINS_PROCESSED", + "OPENFUT_FIFA17_SOLD_COUNT_MODE"): + env.pop(key, None) + if variant != "off": + env["OPENFUT_FIFA17_SOLD_EXPERIMENT"] = variant + env["OPENFUT_FIFA17_SOLD_COINS_PROCESSED"] = coins_processed + env["OPENFUT_FIFA17_SOLD_COUNT_MODE"] = count_mode + + proc = spawn([lay.host_bin], env, lay.root, lay.host_log) + wait_http(HOST_PORT, f"/ut/game/{GAME}/tradePile/counts", proc, lay.host_log, + "staging utas-host") + ok(f"staging utas-host ready on {BIND}:{HOST_PORT} (pid {proc.pid})") + return Launched("staging-utas-host", proc, lay.host_log, f"{BIND}:{HOST_PORT}") + + +def start_blaze(lay: Layout) -> Launched: + env = dict( + os.environ, + OPENFUT_ADVERTISE=ADVERTISE, + OPENFUT_BIND=BIND, + # Read-only reuse of the production auxiliary services: static content, no + # economy state, and staging never connects to them -- it only advertises + # the same strings production Blaze advertises. + POW_CONTENT_HOST=POW_CONTENT_HOST, + POW_HOST=POW_HOST, + # The responder does `sys.path.insert(0, dirname(__file__))` to reach its + # pure sibling modules (`heat2` TDF codec, `fut_account` identity). The copy + # lives elsewhere, so the originals are made importable read-only instead of + # duplicated -- identity MUST stay byte-identical to what LSX and Blaze + # already assert for this persona. + PYTHONPATH=os.path.dirname(BLAZE_SRC), + ) + env.pop("FUT_POW", None) + env.pop("FUT_SBC", None) + proc = spawn(["python3", "-u", lay.blaze_script], env, lay.blaze, lay.blaze_log) + wait_tcp([BLAZE_REDIR_PORT, BLAZE_MAIN_PORT, BLAZE_NUCLEUS_PORT], proc, + lay.blaze_log, "staging blaze") + want = ( + f"RESPONDER v3 START (redir {BLAZE_REDIR_PORT} / blaze {BLAZE_MAIN_PORT} " + f"/ nucleus {BLAZE_NUCLEUS_PORT})" + ) + responder_log = "" + if os.path.exists(lay.blaze_responder_log): + with open(safe_path(lay.blaze_responder_log), errors="replace") as fh: + responder_log = fh.read() + if want not in responder_log: + raise Fatal( + f"staging blaze did not log {want!r} -- it is not the patched copy.\n" + f"{tail(lay.blaze_responder_log)}\n{tail(lay.blaze_log)}" + ) + ok( + f"staging blaze ready: redirector {BLAZE_REDIR_PORT}, main {BLAZE_MAIN_PORT}, " + f"nucleus {BLAZE_NUCLEUS_PORT} (pid {proc.pid}); logged {want!r}" + ) + return Launched( + "staging-blaze", proc, lay.blaze_log, + f"{BIND}:{BLAZE_REDIR_PORT},{BLAZE_MAIN_PORT},{BLAZE_NUCLEUS_PORT}", + ) + + +def stop_launched(items: list[Launched]) -> None: + """Roll back a partial bring-up: signal only the process groups we created.""" + for item in reversed(items): + if item.proc.poll() is not None: + continue + try: + os.killpg(os.getpgid(item.proc.pid), signal.SIGTERM) + except OSError: + pass + try: + item.proc.wait(timeout=10) + except subprocess.TimeoutExpired: + try: + os.killpg(os.getpgid(item.proc.pid), signal.SIGKILL) + except OSError: + pass + + +# --- verification ------------------------------------------------------------------- + + +def host_banner_line(lay: Layout) -> str: + with open(safe_path(lay.host_log), errors="replace") as fh: + hits = [ln.strip() for ln in fh if "sold-experiment=" in ln] + if not hits: + raise Fatal(f"staging utas-host printed no sold-experiment banner\n{tail(lay.host_log)}") + return hits[-1] + + +def verify(lay: Layout, variant: str) -> None: + line = host_banner_line(lay) + ok(f"host banner: {line}") + if variant == "off": + if "sold-experiment=OFF" not in line: + raise Fatal(f"expected an OFF banner, got {line!r}") + else: + want = f"sold-experiment=ON bidState={variant}" + if want not in line: + raise Fatal(f"banner does not say {want!r}: {line!r}") + + status, body = http_get(HOST_PORT, f"/ut/game/{GAME}/tradePile/counts") + ok(f"GET 127.0.0.1:{HOST_PORT}/ut/game/{GAME}/tradePile/counts -> HTTP {status} {body}") + if status != 200: + raise Fatal("staging /tradePile/counts did not answer 200") + + status, body = http_get(CORE_PORT, "/health") + ok(f"GET 127.0.0.1:{CORE_PORT}/health -> HTTP {status} {body}") + + # Nothing this process ever opened may live under the production state dir. + leaked = [] + for fd in os.listdir(f"/proc/{os.getpid()}/fd"): + try: + target = os.readlink(f"/proc/{os.getpid()}/fd/{fd}") + except OSError: + continue + if any(target.startswith(bad) for bad in FORBIDDEN_PATHS): + leaked.append(target) + if leaked: + raise Fatal(f"this script has open handles on production state: {leaked}") + ok(f"no open handle under {FORBIDDEN_PATHS[0]}/") + + assert_prod_alive("end of bring-up") + + +# --- summary ------------------------------------------------------------------------ + + +def cfg_block() -> list[str]: + return [ + f"host={ADVERTISE}", + f"blaze_redirector_port={BLAZE_REDIR_PORT}", + f"blaze_main_port={BLAZE_MAIN_PORT}", + ] + + +def print_summary(lay: Layout, variant: str, coins_processed: str, count_mode: str, + records: list[dict]) -> None: + banner("STAGING STACK IS UP") + rows = [ + ("staging Core", f"127.0.0.1:{CORE_PORT}", "loopback only; client never talks to it"), + ("staging utas-host", f"{BIND}:{HOST_PORT}", "UTAS the client reaches"), + ("staging blaze redirector", f"{BIND}:{BLAZE_REDIR_PORT}", "TLS; EA :10041 / :42230"), + ("staging blaze main", f"{BIND}:{BLAZE_MAIN_PORT}", "EA :42127"), + ("staging blaze nucleus", f"{BIND}:{BLAZE_NUCLEUS_PORT}", "OAuth stub"), + ("dead python upstream", f"127.0.0.1:{DEAD_PYTHON_PORT}", "UNBOUND on purpose: fallback fails closed"), + ] + print(f" {'SERVICE':<26} {'BIND':<24} NOTE") + for name, bind, note in rows: + print(f" {name:<26} {bind:<24} {note}") + + print() + print(f" {'STATE FILE':<26} PATH") + for name, path in ( + ("Core sqlite", lay.core_db), + ("market sqlite", lay.market_db), + ("pile sqlite", lay.pile_db), + ("identity store", lay.identity), + ("clientdata blobs", lay.clientdata), + ("content pack", lay.cards), + ("identity catalog", lay.catalog), + ("patched blaze", lay.blaze_script), + ("manifest", lay.manifest), + ): + print(f" {name:<26} {path}") + + print() + print(f" {'PROCESS':<26} {'PID':<8} {'PGID':<8} LOG") + for rec in records: + print(f" {rec['name']:<26} {rec['pid']:<8} {rec['pgid']:<8} {rec['log']}") + + print() + print(f" experiment variant : {variant}") + print(f" coinsProcessed : {coins_processed}") + print(f" count mode : {count_mode}") + print(f" banner : {host_banner_line(lay)}") + print(f" seeded state : {db_summary(lay)}") + print(f" disposable item to sell : {DISPOSABLE_ITEM} ({DISPOSABLE_CARD})") + + banner("OPERATOR: EDIT openfut.cfg ON THE FIFA CLIENT (10.10.0.105)") + print(' File: "/mnt/games/FIFA 17/openfut.cfg" (back it up first:') + print(' cp openfut.cfg openfut.cfg.prod)') + print() + print(" Replace these three lines with EXACTLY:") + print() + for line in cfg_block(): + print(f" {line}") + print() + print(" Leave https_port=8443 UNCHANGED (Bridge; no economy state).") + print(" Then RELAUNCH the FIFA 17 client -- a running client caches its UTAS") + print(" session and will not re-auth against a different stack.") + print() + print(" REVERT to production (production values, unchanged on this host):") + print() + print(" host=10.10.0.120") + print(" blaze_redirector_port=42127") + print(" blaze_main_port=42130") + print() + print(" Full detail: docs/SOLD_STAGING_RUNBOOK.md") + print() + print(" Tear down: python3 scripts/sold-staging-down.py") + + +# --- main --------------------------------------------------------------------------- + + +def main() -> int: + ap = argparse.ArgumentParser( + description=__doc__.splitlines()[0], + formatter_class=argparse.RawDescriptionHelpFormatter, + ) + ap.add_argument("--variant", choices=["highest", "buyNow", "off"], default="highest", + help="bidState token emitted on a sold seller row (default: highest)") + ap.add_argument("--coins-processed", choices=["0", "1"], default="0", + help="coinsProcessed on the sold row (default: 0)") + ap.add_argument("--count-mode", choices=["active", "active_plus_sold"], + default="active", + help="what /tradePile/counts.count reports (default: active)") + ap.add_argument("--dir", default=os.environ.get("OPENFUT_SOLD_STAGING_DIR", + DEFAULT_STAGING_DIR), + help=f"staging directory (default: {DEFAULT_STAGING_DIR})") + args = ap.parse_args() + + lay = Layout(args.dir) + started: list[Launched] = [] + try: + banner("PREFLIGHT (nothing is launched until every check passes)") + step(f"staging dir : {lay.root}") + step(f"repo : {REPO}") + step(f"forbidden ports : {sorted(FORBIDDEN_PORTS)}") + step(f"forbidden paths : {list(FORBIDDEN_PATHS)}") + assert_prod_alive("preflight") + refuse_if_up(lay) + assert_ports_free() + + banner("MATERIALISE STAGING DIRECTORY") + materialise(lay) + reset_throwaway_state(lay) + assert_seed_cards_resolvable(lay) + + banner("PATCH THE BLAZE RESPONDER COPY") + patch_blaze(lay) + + banner("STAGING CORE") + if os.path.exists(lay.core_db): + raise Fatal(f"{lay.core_db} should have been removed by the state reset") + migrate_core(lay) + seed_core_db(lay) + started.append(start_core(lay)) + + banner("STAGING UTAS-HOST") + started.append(start_host(lay, args.variant, args.coins_processed, + args.count_mode)) + + banner("STAGING BLAZE") + started.append(start_blaze(lay)) + + records = [item.record() for item in started] + for rec in records: + if lay.root not in rec["cmdline"]: + raise Fatal( + f"{rec['name']} pid {rec['pid']} cmdline does not contain the " + f"staging dir -- the down script would refuse to stop it: " + f"{rec['cmdline']!r}" + ) + with open(safe_path(lay.manifest), "w") as fh: + json.dump( + { + "created_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), + "staging_dir": lay.root, + "variant": args.variant, + "coins_processed": args.coins_processed, + "count_mode": args.count_mode, + "ports": { + "core": CORE_PORT, + "utas_host": HOST_PORT, + "blaze_redirector": BLAZE_REDIR_PORT, + "blaze_main": BLAZE_MAIN_PORT, + "blaze_nucleus": BLAZE_NUCLEUS_PORT, + "dead_python": DEAD_PYTHON_PORT, + }, + "state_files": { + "core_db": lay.core_db, + "market_db": lay.market_db, + "pile_db": lay.pile_db, + "identity": lay.identity, + "clientdata": lay.clientdata, + }, + "client_cfg": cfg_block(), + "processes": records, + }, + fh, + indent=2, + ) + ok(f"manifest written: {lay.manifest}") + + banner("VERIFY ISOLATION") + verify(lay, args.variant) + + print_summary(lay, args.variant, args.coins_processed, args.count_mode, records) + return 0 + except Fatal as exc: + print(f"\nFATAL: {exc}", file=sys.stderr) + if started: + print(" rolling back the partial bring-up...", file=sys.stderr) + stop_launched(started) + print(" rolled back (only this script's own process groups were " + "signalled)", file=sys.stderr) + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/sold-wire-check.py b/scripts/sold-wire-check.py new file mode 100755 index 0000000..45fe67b --- /dev/null +++ b/scripts/sold-wire-check.py @@ -0,0 +1,375 @@ +#!/usr/bin/env python3 +"""Wire-level verification of the seller-facing SOLD flow, in isolation. + +Proves the harness produces a correct, authentic sold row BEFORE any operator time +is spent driving a real FIFA client. Brings up its own Core + utas-host on ephemeral +ports against throwaway databases, runs the real settlement through the +`staging_sell` binary, then reads every seller-facing surface under BOTH A/B +variants and exercises the bulk clear verb. + +ISOLATION: production ports 8099, 8199, 18080, 8443, 42127, 42130, 42131, 4216, +8080, 8081 and 8094 are in a hard deny-list checked before every bind and every +request, and nothing under /home/alex/openfut-promotion/state/ is opened. + + python3 scripts/sold-wire-check.py [--keep] +""" +import argparse +import http.client +import json +import os +import shutil +import socket +import sqlite3 +import subprocess +import sys +import tempfile +import time + +REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +FORBIDDEN = {8099, 8199, 18080, 8443, 42127, 42130, 42131, 4216, 8080, 8081, 8094} +TS = "2026-01-01T00:00:00Z" +PERSONA = "33068179" +SELLER_CLUB = "club-seller-a" +BUYER_CLUB = "club-buyer-b" +ITEM = "core-disposable-x" +CARD = "def-disposable" +TRADE_ID = "900500150" +GROSS = 150 + +checks = [] + + +def check(label, ok, detail=""): + checks.append((label, bool(ok), detail)) + print(f" [{'PASS' if ok else 'FAIL'}] {label}{(': ' + detail) if detail else ''}") + return ok + + +def free_port(): + for _ in range(200): + s = socket.socket() + s.bind(("127.0.0.1", 0)) + p = s.getsockname()[1] + s.close() + if p not in FORBIDDEN and p > 1024: + return p + raise RuntimeError("no free port") + + +def req(port, method, path, body=None): + assert port not in FORBIDDEN, f"refusing to contact production port {port}" + c = http.client.HTTPConnection("127.0.0.1", port, timeout=20) + headers = {"X-OpenFUT-Game": "fifa17"} + if body is not None: + headers["Content-Type"] = "application/json" + c.request(method, path, body=json.dumps(body) if body is not None else None, + headers=headers) + r = c.getresponse() + raw = r.read() + c.close() + try: + return r.status, json.loads(raw) + except Exception: + return r.status, raw.decode("utf-8", "replace") + + +def wait_http(port, path, timeout=45, proc=None, log=None): + deadline = time.time() + timeout + while time.time() < deadline: + if proc is not None and proc.poll() is not None: + tail = "" + if log and os.path.exists(log): + tail = open(log).read()[-1500:] + raise RuntimeError(f"process exited {proc.returncode}\n{tail}") + try: + st, _ = req(port, "GET", path) + if st < 500: + return + except Exception: + time.sleep(0.25) + tail = open(log).read()[-1500:] if log and os.path.exists(log) else "" + raise RuntimeError(f"{path} on {port} never became ready\n{tail}") + + +def seed(db): + """Two identities by direct SQL: Seller A (the FIFA persona) and synthetic Buyer B.""" + con = sqlite3.connect(db) + for prof, club, coins, game in ( + ("prof-seller-a", SELLER_CLUB, 1_000, "fifa17"), + ("prof-buyer-b", BUYER_CLUB, 20_000, "buyer-game"), + ): + con.execute( + "INSERT INTO profiles (id, username, game_id, created_at, updated_at) " + "VALUES (?, ?, ?, ?, ?)", (prof, prof, game, TS, TS)) + con.execute( + "INSERT INTO clubs (id, profile_id, name, coins, created_at, updated_at) " + "VALUES (?, ?, ?, ?, ?, ?)", (club, prof, club, coins, TS, TS)) + con.execute( + "INSERT INTO owned_cards (id, club_id, card_id, is_loan, acquired_at) " + "VALUES (?, ?, ?, 0, ?)", (ITEM, SELLER_CLUB, CARD, TS)) + con.commit() + con.close() + + +def owner_of(db, item): + con = sqlite3.connect(db) + row = con.execute("SELECT club_id FROM owned_cards WHERE id = ?", (item,)).fetchone() + n = con.execute("SELECT COUNT(*) FROM owned_cards WHERE id = ?", (item,)).fetchone()[0] + coins = dict(con.execute("SELECT id, coins FROM clubs").fetchall()) + con.close() + return (row[0] if row else None), n, coins + + +def start_core(tmp, port, log): + db = os.path.join(tmp, "core.db") + pack = os.path.join(tmp, "pack.json") + with open(pack, "w") as f: + # A top-level ARRAY: Core's content-pack loader expects a sequence, not a + # map. Needed because the preflight refuses to start when an owned card + # references a CardDefinitionId no pack defines. + json.dump([{ + "id": CARD, "name": "Disposable", "overall": 75, "position": "ST", + "nation": "Nation", "league": "League", "club": "Club", + "pace": 75, "shooting": 75, "passing": 75, "dribbling": 75, + "defending": 40, "physical": 70, "rarity": "gold", + "image_path": None, + }], f) + env = dict(os.environ, + LISTEN_ADDR=f"127.0.0.1:{port}", + DATABASE_URL=f"sqlite://{db}", + # Core's real data dir (read-only): it needs chemistry_styles.json + # and friends. The throwaway DB and the content pack stay in tmp. + DATA_DIR=os.path.join(REPO, "openfut-core", "data"), + OPENFUT_CONTENT_PACKS=pack) + # Migrate-only pass first: Core owns its schema, so the fixture cannot be + # seeded into an empty file. Stop it before the external writer touches the db. + with open(log, "w") as lf: + p = subprocess.Popen([os.path.join(REPO, "target/release/openfut-core")], + cwd=tmp, env=env, stdout=lf, stderr=subprocess.STDOUT) + wait_http(port, "/health", proc=p, log=log) + p.terminate() + p.wait(timeout=20) + seed(db) + with open(log, "a") as lf: + p = subprocess.Popen([os.path.join(REPO, "target/release/openfut-core")], + cwd=tmp, env=env, stdout=lf, stderr=subprocess.STDOUT) + wait_http(port, "/health", proc=p, log=log) + return p, db + + +def start_host(tmp, port, core_port, log, variant, coins_processed="0", + count_mode="active"): + with open(os.path.join(tmp, "catalog.json"), "w") as f: + # Minimal STAGING catalog. Deliberately NOT the production catalog, which + # lives under /home/alex/openfut-promotion/state/ and must never be opened. + json.dump({"schema_version": 1, "game": "fifa17", + "cards": {CARD: {"asset_id": 212188, "version": 0, + "rareflag": 1, "kind": "player"}}}, f) + env = dict(os.environ, + OPENFUT_UTAS_HOST_ADDR=f"127.0.0.1:{port}", + OPENFUT_CORE_URL=f"http://127.0.0.1:{core_port}", + # Deliberately dead: any Python fallback must fail closed and be + # visible, never silently serve production data. + OPENFUT_UTAS_PYTHON_URL="http://127.0.0.1:9", + OPENFUT_FIFA17_TABLES_DIR=os.path.join(REPO, "fifa17-recon/data/tables"), + OPENFUT_IDENTITY_STORE=os.path.join(tmp, "identity.json"), + OPENFUT_PERSONA_ID=PERSONA, + OPENFUT_MARKET_DB=os.path.join(tmp, "market.db"), + OPENFUT_PILE_DB=os.path.join(tmp, "pile.db"), + OPENFUT_FIFA17_SOLD_EXPERIMENT=variant, + OPENFUT_FIFA17_SOLD_COINS_PROCESSED=coins_processed, + OPENFUT_FIFA17_SOLD_COUNT_MODE=count_mode, + OPENFUT_FIFA17_CATALOG=os.path.join(tmp, "catalog.json"), + RUST_LOG="info") + with open(log, "w") as lf: + p = subprocess.Popen([os.path.join(REPO, "target/release/openfut-utas-host")], + cwd=tmp, env=env, stdout=lf, stderr=subprocess.STDOUT) + wait_http(port, "/ut/game/fifa17/tradePile/counts", proc=p, log=log) + return p + + +def banner(t): + print("\n" + "=" * 72) + print(f"== {t}") + print("=" * 72) + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--keep", action="store_true") + args = ap.parse_args() + + tmp = tempfile.mkdtemp(prefix="openfut-sold-wire-") + procs = [] + try: + core_port, host_port = free_port(), free_port() + core_log = os.path.join(tmp, "core.log") + host_log = os.path.join(tmp, "host.log") + banner("ISOLATED STAGING (production untouched)") + print(f" tmp : {tmp}") + print(f" core : 127.0.0.1:{core_port}") + print(f" utas-host : 127.0.0.1:{host_port}") + print(f" forbidden : {sorted(FORBIDDEN)}") + + core, core_db = start_core(tmp, core_port, core_log) + procs.append(core) + host = start_host(tmp, host_port, core_port, host_log, "highest") + procs.append(host) + print(" both ready") + + bann = [l for l in open(host_log) if "sold-experiment" in l] + check("host banner names the variant", any("bidState=highest" in l for l in bann), + (bann[0].strip() if bann else "no banner")) + + banner("BEFORE — seller A owns the item, nothing listed") + own, n, coins = owner_of(core_db, ITEM) + print(f" owner={own} instances={n} coins={coins}") + st, pile = req(host_port, "GET", "/ut/game/fifa17/tradePile") + st2, counts = req(host_port, "GET", "/ut/game/fifa17/tradePile/counts") + print(f" /tradePile total={pile.get('total')} counts={json.dumps(counts)}") + check("seller owns the item", own == SELLER_CLUB, str(own)) + check("no rows before listing", pile.get("total") == 0) + check("sold counter starts at 0", counts.get("sold") == 0) + + banner(f"LIST — authentic active listing at {GROSS} coins") + # Seed the listing directly into the staging market db: the client normally + # does this via POST /auctionhouse, which needs a wire-id mapping we do not + # have in this headless check. The LISTING SHAPE is identical either way. + mdb = os.path.join(tmp, "market.db") + con = sqlite3.connect(mdb) + con.execute( + "INSERT INTO listings (listing_id, card_id, core_item_id, wire_item_id, " + "wire_resource_id, start_price, buy_now_price, owner, state, created_at, " + "item_json, duration_secs) VALUES (?,?,?,?,?,?,?,?, 'active', ?, ?, ?)", + (TRADE_ID, CARD, ITEM, 100000178, 212188, GROSS, GROSS, "CAGE", + str(int(time.time() * 1000)), json.dumps({ + "id": 100000178, "resourceId": 212188, "rating": 75, + "preferredPosition": "ST", "itemState": "forSale", + "untradeable": False, "assetId": 212188}), 3600)) + con.commit() + con.close() + st, pile = req(host_port, "GET", "/ut/game/fifa17/tradePile") + st2, counts = req(host_port, "GET", "/ut/game/fifa17/tradePile/counts") + row = pile["auctionInfo"][0] + print(f" active row: tradeState={row['tradeState']} bidState={row['bidState']} " + f"expires={row['expires']} counts={json.dumps(counts)}") + check("active row is active/none", row["tradeState"] == "active" and row["bidState"] == "none") + check("counts.selling == 1 while active", counts.get("selling") == 1) + check("counts.sold still 0 while active", counts.get("sold") == 0) + + banner("PURCHASE — synthetic Buyer B, through the REAL settlement path") + out = subprocess.run( + [os.path.join(REPO, "target/release/staging_sell"), + "--market-db", mdb, "--core-url", f"http://127.0.0.1:{core_port}", + "--trade-id", TRADE_ID, "--item", ITEM, + "--seller", SELLER_CLUB, "--buyer", BUYER_CLUB, "--gross", str(GROSS)], + capture_output=True, text=True, timeout=120) + print(" " + "\n ".join((out.stdout + out.stderr).strip().splitlines())) + check("staging_sell succeeded", out.returncode == 0, f"exit {out.returncode}") + + own, n, coins = owner_of(core_db, ITEM) + fee = GROSS * 5 // 100 + print(f" owner={own} instances={n} coins={coins} fee={fee}") + check("ownership transferred to buyer", own == BUYER_CLUB, str(own)) + check("exactly ONE authoritative instance", n == 1, str(n)) + check("buyer debited gross", coins.get(BUYER_CLUB) == 20_000 - GROSS, + str(coins.get(BUYER_CLUB))) + check("seller credited net", coins.get(SELLER_CLUB) == 1_000 + GROSS - fee, + str(coins.get(SELLER_CLUB))) + check("economy shrank by exactly the fee", + 21_000 - sum(coins.values()) == fee, str(21_000 - sum(coins.values()))) + + banner("SOLD ROW — variant A: closed / highest") + st, pile = req(host_port, "GET", "/ut/game/fifa17/tradePile") + st2, counts = req(host_port, "GET", "/ut/game/fifa17/tradePile/counts") + st3, status = req(host_port, "GET", f"/ut/game/fifa17/trade/status?tradeIds={TRADE_ID}") + a_row = pile["auctionInfo"][0] if pile.get("auctionInfo") else {} + print(" " + json.dumps(a_row, indent=2).replace("\n", "\n ")) + print(f" counts={json.dumps(counts)}") + check("sold row is present in the pile", pile.get("total") == 1) + check("tradeState closed", a_row.get("tradeState") == "closed") + check("bidState highest (variant A)", a_row.get("bidState") == "highest") + check("currentBid == sale price", a_row.get("currentBid") == GROSS) + check("expires 0", a_row.get("expires") == 0) + check("twelve atoms exactly", len(a_row) == 12, str(len(a_row))) + check("counts.sold == 1", counts.get("sold") == 1) + check("counts.selling == 0", counts.get("selling") == 0) + check("/trade/status agrees", status["auctionInfo"][0]["tradeState"] == "closed" + and status["auctionInfo"][0]["bidState"] == "highest") + + banner("VARIANT B — same state, restart host with closed / buyNow") + host.terminate(); host.wait(timeout=20); procs.remove(host) + host = start_host(tmp, host_port, core_port, host_log, "buyNow", + coins_processed="1", count_mode="active_plus_sold") + procs.append(host) + st, pileb = req(host_port, "GET", "/ut/game/fifa17/tradePile") + st2, countsb = req(host_port, "GET", "/ut/game/fifa17/tradePile/counts") + b_row = pileb["auctionInfo"][0] + print(f" bidState={b_row['bidState']} coinsProcessed={b_row['coinsProcessed']} " + f"counts={json.dumps(countsb)}") + check("bidState buyNow (variant B)", b_row.get("bidState") == "buyNow") + check("coinsProcessed 1 when asked", b_row.get("coinsProcessed") == 1) + check("count_mode active_plus_sold counts the sold row", + countsb.get("count") == 1 and countsb.get("sold") == 1, + json.dumps(countsb)) + differing = sorted(k for k in a_row if a_row.get(k) != b_row.get(k)) + check("A/B differ ONLY in bidState and coinsProcessed", + differing == ["bidState", "coinsProcessed"], str(differing)) + + banner("CLEAR — the PE-proven bulk verb DELETE .../trade/sold") + pre_coins = owner_of(core_db, ITEM)[2] + st, body = req(host_port, "DELETE", "/ut/delete/game/fifa17/trade/sold") + print(f" HTTP {st} body={json.dumps(body)}") + st2, pilec = req(host_port, "GET", "/ut/game/fifa17/tradePile") + st3, countsc = req(host_port, "GET", "/ut/game/fifa17/tradePile/counts") + own2, n2, post_coins = owner_of(core_db, ITEM) + print(f" after clear: total={pilec.get('total')} counts={json.dumps(countsc)} " + f"owner={own2} instances={n2}") + check("clear acks 200 {}", st == 200 and body == {}) + check("sold row gone from the pile", pilec.get("total") == 0) + check("counts.sold back to 0", countsc.get("sold") == 0) + check("clear moved NO coins", pre_coins == post_coins, f"{pre_coins} -> {post_coins}") + check("buyer still owns the item after clear", own2 == BUYER_CLUB, str(own2)) + check("still exactly one instance", n2 == 1, str(n2)) + st, again = req(host_port, "DELETE", "/ut/delete/game/fifa17/trade/sold") + check("clearing again is a safe no-op", st == 200) + cleared = [l for l in open(host_log) if "market-clear-sold" in l] + check("clear is logged for capture", bool(cleared), + cleared[-1].strip() if cleared else "no log line") + + banner("PRODUCTION UNTOUCHED") + alive = subprocess.run(["ps", "-o", "pid=", "-p", "3631953"], + capture_output=True, text=True).stdout.strip() + check("prod-host pid 3631953 still alive", alive == "3631953", alive or "gone") + opened = subprocess.run( + ["bash", "-lc", + "ls -l /proc/*/fd 2>/dev/null | grep -c openfut-promotion || true"], + capture_output=True, text=True).stdout.strip() + print(f" staging fds referencing production state: (informational) {opened}") + + banner("RESULT") + passed = sum(1 for _, ok, _ in checks if ok) + print(f" {passed}/{len(checks)} checks passed") + failed = [l for l, ok, _ in checks if not ok] + if failed: + print(" FAILED: " + "; ".join(failed)) + print("\n " + ("ALL CHECKS PASSED" if not failed else "FAILURES PRESENT")) + return 0 if not failed else 1 + finally: + for p in procs: + try: + p.terminate(); p.wait(timeout=15) + except Exception: + try: + p.kill() + except Exception: + pass + if args.keep: + print(f"\n kept {tmp}") + else: + shutil.rmtree(tmp, ignore_errors=True) + print(f"\n removed {tmp}") + + +if __name__ == "__main__": + sys.exit(main())