750d6c2e180951d0174705a89d7513dee57b1bc8
25 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 funman300
|
16771b0b33 |
test(scripts): recover the client's error text, and diff hub shapes against production
Three diagnostics from chasing a FUT error that four server-side fixes failed to resolve, kept because the technique generalises. client-error-string.py recovers FIFA's on-screen message from /proc/<pid>/mem, read-only, scanning ASCII and UTF-16LE (FIFA UI strings are wide). This ended the guessing: the dialog reads "An error occurred downloading the FUT Squad Update. Please try again." -- a CONTENT DOWNLOAD failure, not the player's lineup. Every squad fix before it was aimed at the wrong subsystem, because "squad update" in FIFA means the roster update, and the server-side symptom (a squad the client would not accept) was consistent with both readings. When the server says 200 and the client says no, the client's own words are the cheapest evidence available and should have been the FIRST thing recovered, not the fifth. hub-dump.py + hub-diff-prod-staging.py diff every hub route between production (known-good, same client accepts it) and staging, comparing key presence and JSON types rather than values, since values legitimately differ. Result: 0 structural differences across 14 routes, which retired the whole "a missing field breaks bootstrap" line of investigation in one run instead of one restart at a time. Also ruled out with evidence: cert gates ARE patched (autopatch logs "pid 56298: PATCHED cert gates", and the gate bytes read back as the patched patterns); the roster server serves the FUT Squad Update fine (TLS1.2 AES256-GCM-SHA384, HTTP/1.0 200, application/xml) once probed with ALL:@SECLEVEL=0 -- a default modern context gets SSLV3_ALERT_HANDSHAKE_FAILURE and would have been a false alarm; production and staging Blaze advertise identical roster/POW hosts; the Blaze session is healthy and answering PINGs; and the squad round-trips exactly through PUT/GET. |
||
|
funman300
|
022634704a |
fix(scripts): staging squad now matches production's known-good shape exactly
Adds the manager reference, the last remaining difference from the squad the same client demonstrably accepts. Staging's squad shape is now identical to production's: zero missing keys, zero type differences, zero empty-vs-populated mismatches. The manager looked unfixable. Production points at instance 100000427 while the staging club holds 11 players and zero staff, so there was apparently nothing to reference, and inventing an id would have pointed at a non-existent item. Checking production properly dissolved the problem: 100000427 is absent from production's OWN club listing too. /club/staff returns 1975 items spanning ids 100000001..100004826 and 100000427 is not among them, and the type=staff/type=manager filters are ignored (200 players either way). Production's manager reference is dangling and the client accepts that squad anyway, which proves the client does not validate the manager id against the club -- only a populated array matters. So the reference is mirrored verbatim, dangling id included. That replicates the known-good state exactly and is better than pointing the manager slot at a player, which would have been a guess dressed up as a fix. Method note: every step here came from diffing against production rather than reading the client. The host reported squad-active 200 outcome=ok throughout, and 200 with the right players was never evidence the client accepted the body. |
||
|
funman300
|
96ca7c0484 |
fix(scripts): the seeded squad was structurally valid but the client still refused it
First seed sent only squadName/formation/captain/players. The host logged squad-active 200 outcome=ok and /squad/0 showed 11 occupied slots, yet the client still threw a FUT squad update error -- a 200 with the right players is not proof the client accepts the body. Diffed against production's known-good squad, which the same client accepts, comparing key presence and JSON TYPES rather than values. Staging returned null for exactly the five fields the PUT never carried, because the extension stored nothing for them: squadType (a string enum), chemistry, rating, starRating (ints) and custom (the opaque 33-int tactics array the client definitely parses). kicktakers was empty where production carries five. Now sends all of them: squadType REGULAR_SQUAD, chemistry, rating/starRating derived from the XI's mean rating, production's custom array verbatim (opaque server-side, only its shape matters), and five kicktakers. Re-diff leaves exactly one difference -- manager, which production points at owned staff instance 100000427 while the staging club holds 11 players and zero staff. Left empty rather than inventing an id that references a non-existent item; recorded in the code as the one known remaining gap. Also fixes a KeyError from the rewrite dropping the players key. |
||
|
funman300
|
202366611e |
test(scripts): front-load the hub preconditions instead of finding them one restart at a time
The sold A/B stalled twice on preconditions no headless check exercised: the staging identity had no squad (the hub refuses to open, showing a squad update error), and any route without a Rust owner falls through to a deliberately dead Python upstream and answers 502. Each cost a full operator cycle. Sweeps the routes the client is observed to request and separates three failure classes that need different fixes: 502/PYTHON_FALLBACK (no Rust owner), missing_integrity (200 but the underlying state is absent -- exactly 'no extension stored' before the squad was seeded), and 200-but-unusable (a squad with zero occupied slots). A 200 is not proof the client is satisfied, so squad responses are judged on occupied slots. Also reads the host's own classification for the requests just made, since the host is the authority on ownership and integrity rather than the response body. Every path is verified against what the client actually sends. A first pass flagged five 'fatal' routes that were my own guesses -- /accountinfo (client uses /user/accountinfo), bare /squad (uses /squad/active), and /watchlist (camelCase watchList). Crying wolf about the stack is worse than not checking, so the list now carries only observed paths and that trap is written down in the comment. Current result: 14 ok, 0 integrity warnings, 0 fatal. |
||
|
funman300
|
ea92057e53 |
test(scripts): seed the staging seller an XI so the FUT hub will open
The A/B identity had owned items but no squad, because the sold-row work only ever
needed the tradePile wire. Every headless check passed -- none of them asks for a
squad -- but the real client refuses to enter the FUT hub with an empty one and shows
a squad update error. A squad is a hub precondition, not a Transfer-List detail.
Seeds via the real PUT /ut/game/fifa17/squad/0, the same request the client sends, so
parse_squad_put/build_squad_write produce exactly what a genuine save would. Writing
Core rows by hand could yield a shape the live path never emits, which is the kind of
divergence that quietly invalidates an experiment.
Picks one owned player per 4-3-3 slot, best rating first, without reusing an instance;
fills the fixed 23-slot array with 0..=10 as the pitch and empty slots as
itemData.id == 0; refuses to write a partial XI and reports any out-of-position
substitution loudly rather than silently reproducing the broken state. Verified
0 -> 11 occupied with no substitutions and a {"id":0} ack.
|
||
|
funman300
|
c71593b286 |
test(scripts): enable the hook on the deployed launcher without a rebuild
Bridges openfut-launcher c542415 onto the already-built launcher on .105 by writing WINEDLLOVERRIDES=version=n,b into game_profile.env, which that build does apply. Forward-compatible: the fixed launcher defers to a profile that already pins version=, so this value simply wins. Records the prior value -- including its absence, as the literal <absent> -- to a sidecar before mutating, so revert restores the real previous state instead of assuming the key was missing. Refuses to edit while the launcher runs, since it holds its config in memory and would write the stale value back. |
||
|
funman300
|
e0e46d8a57 |
fix(scripts): the port switcher was editing a derived file, so the A/B ran on production
openfut.cfg is not the source of truth for the client's Blaze ports -- the launcher
is. It reconciles openfut.cfg from ~/.config/openfut-launcher/config.json,
fail-closed, immediately before every launch. So `staging` set the ports, verified
them, and the next launch silently reverted them.
Caught only because the capture harness cross-checks instead of trusting the screen.
The operator reported "the Transfers tile does not show Sold" -- which looked like a
clean negative result about the sold counter, and was in fact a reading of their
PRODUCTION club, where sold:0 is correct. Evidence chain:
* route-log delta contained 5 lines, all of them the harness's own GETs; the
client issued nothing to staging at all;
* `ss -tnp` on the client showed FIFA17.exe pid 39482 ESTAB to 10.10.0.120:42130
(production Blaze) plus TIME-WAIT to :8099 (production UTAS);
* the hook logged `blaze_redir=42127 blaze_main=42130`;
* openfut.cfg mtime was 2s before process start, sha back to the production value.
Had the harness reported the tile at face value, the sold counter recovered from
CardsDLL would now be recorded as refuted by a run that never reached the code.
Fixes: own the launcher config (source) before openfut.cfg (derived), with the same
record-before-mutate sidecar discipline on both; refuse to edit while the launcher is
running, since it holds config in memory and would write the stale values back;
report both files and both guards in `show`. launcher_running() matches the
kernel-truncated comm "openfut-launche" -- the full name exceeds 15 chars, which has
bitten this project before.
No production change; staging stack and its variant-A sold row untouched.
|
||
|
funman300
|
fbe29da05b |
fix(scripts): sold-client-ports guard self-matched its own shell, wedging it ON
`pgrep -f FIFA17.exe` matched the remote shell executing it -- the SSH command line contains the literal pattern -- so fifa_running() always returned True and the port switcher could never edit openfut.cfg. It refused with "REFUSING to edit ... while a FIFA client is running" moments after FIFA had actually exited. Fail-closed, so nothing unsafe happened, but the guard was permanently stuck and blocked the A/B entirely. Now matches /proc/<pid>/comm exactly, which is the executable name: the invoking shell reads as zsh and cannot self-match, while a genuine FIFA process still does. Validated both directions with the same loop -- it found pid 36958 while FIFA was up, and reports gone once it exited. Still fail-closed on read errors. The lesson generalises: a pattern-matching process guard checked over a transport that carries the pattern in its own argv is self-satisfying, and a guard that can only ever say "yes" is not a guard. |
||
|
funman300
|
9ffbd651b1 |
test(market): one-command live capture for the sold A/B, with in-run validation
Turns the operator's job into "navigate, say go" and removes any chance of a half-recorded variant. One command captures and labels: the staging wire surfaces, the client's OWN auction record decoded read-only from /proc/<pid>/mem (STATE, YOURBID, COINS_AWARDED, MIN_CREDITS, IS_GLOW, INBOX, CARD_OFFERSTATE), and the staging host route-log DELTA since the last capture -- which is how a client-issued DELETE .../trade/sold gets OBSERVED rather than assumed. The part that matters is the wire-vs-memory cross-check. It validates the observation mechanism against a known-positive in the SAME run: if the wire says bidState "highest" and the client's memory decodes 2(highest), the probe is demonstrably reading the right struct this time. It also recomputes the native IS_GLOW/INBOX formulas from the wire and compares them to what the client stored. Proven honest on first run: with the client attached to PRODUCTION and not on the Transfer List, it reported the staging sold row on the wire, 0 client records, and INSTRUMENTATION NOT VALIDATED -- refusing to draw a conclusion from an empty read. Two earlier sessions were misled by exactly that (a sampler bug printing "countdown NO", and auction containers read while the screen was unbound), so an empty container is explicitly not treated as an empty pile. Probe base-address discovery was separately confirmed against the live client (pid 36958, FNV control=MATCH, model resolved, containers read cleanly), and production's wire independently agreed at total=0. No production change. Client config untouched (still production Blaze ports). |
||
|
funman300
|
aa5fb2cc40 |
test(market): make the sold A/B one-field attributable, add classified differential
The brief's gate: if the harness varies bidState AND coinsProcessed together, the client's reaction is attributable to neither. The env knobs were already orthogonal (--variant and --coins-processed are independent, cp defaults to 0), but sold-wire-check.py was flipping BOTH for variant B as a convenience, which is exactly the contaminated A/B the brief forbids. Fixed: the primary pair now holds coinsProcessed at 0 and asserts the differing-field set is exactly ['bidState']. New scripts/sold-ab-differential.py is the pre-live gate. It settles ONE synthetic sale, then re-reads every seller-facing surface under each variant by restarting only the host (same Core, same DBs, same sale), and diffs with explicit classification -- MISSING / EXTRA / TYPE_MISMATCH / VALUE_MISMATCH -- rather than a boolean "equal?". Two orthogonal pairs: PRIMARY bidState highest vs buyNow, coinsProcessed held at 0 ORTHOGONAL coinsProcessed 0 vs 1, bidState held at highest Result, 36/36: the ONLY finding on /tradePile is VALUE_MISMATCH auctionInfo[0].bidState A='highest' B='buyNow'; /trade/status differs in exactly the same one path; counts are byte-identical. The orthogonal pair's only finding is auctionInfo[0].coinsProcessed. C_cp0's sha256 equals A_highest's, so the capture is reproducible rather than merely consistent. Counts states the live run has to interpret, measured not guessed: S1 0 active + 1 sold -> count 0, selling 0, sold 1 S2 1 active + 1 sold -> count 1 (active mode) vs 2 (membership mode) That divergence IS the open question for the client; production is unchanged. scripts/sold-client-ports.py switches ONLY the two client Blaze port lines, and is built so restoration cannot depend on memory: it records the production values to a sidecar on the client BEFORE the first edit and restore reads that sidecar, refusing if it is absent. It rewrites only known keys (a missing key is an error, never a silent append), re-reads and verifies afterwards, and REFUSES to edit while a FIFA client is running because the hook reads the file at connect time. Phase 0 evidence under docs/evidence/sold-ab-2026-08-18/ with a sha256 per surface, one file per variant so A can never overwrite B. Live client A/B NOT run: a production FIFA session is currently live on 10.10.0.105 (pid 32188), and live-session mutual exclusion applies. The client config was NOT touched -- the switcher's guard refused, as designed. Production untouched: prod-host pid 3631953, coins 29,843,976, /tradePile 0, counts.sold 0, club 1966; nothing under /home/alex/openfut-promotion/state/ opened. |
||
|
funman300
|
468bc0fba9 |
feat(market): isolated two-identity SOLD-row A/B harness (staging only, not promoted)
Static RE exhausted CardsDLL on the one open question: for a closed row
IS_GLOW = (bidState != none) and INBOX = (bidState in {highest, buyNow}), so
closed/highest and closed/buyNow are BIT-IDENTICAL natively. But bidState is
published to the movie verbatim as YOURBID, so the FUT ActionScript CAN separate
them. This builds the controlled experiment that asks the client which one it
treats as the seller's sale.
PRODUCTION SAFETY IS THE FIRST CONCERN
New module openfut-utas-host/src/sold_experiment.rs. Every knob is OFF unless its
env var is set, an unrecognised value is OFF rather than a default token (silently
picking one would fabricate the answer being measured), and the host logs a startup
banner naming the active variant so a staging capture can never be mistaken for a
production one. With no env set, /tradePile and /trade/status emit only real active
auctions (the Fix A invariant) and counts still report sold: 0. The entire existing
test suite now passes SoldExperiment::OFF explicitly, making it a regression guard.
OPENFUT_FIFA17_SOLD_EXPERIMENT = highest | buyNow (else OFF)
OPENFUT_FIFA17_SOLD_COINS_PROCESSED = 1 (else 0)
OPENFUT_FIFA17_SOLD_COUNT_MODE = active_plus_sold (else active)
WHAT THE EXPERIMENT PROJECTS
Uncleared sold listings appear in /tradePile and /trade/status as tradeState
"closed" with the token under test and currentBid = the sale price; counts report
the real sold tally. There is ONE record builder, so the A/B changes only what is
passed into it, and a test asserts that EXACTLY ONE field differs between the two
variants -- without that control the client's reaction is not attributable to the
token and the whole experiment is void. coinsProcessed (Flash COINS_AWARDED) varies
independently so the third pass cannot be confounded with the first.
CLEAR-SOLD, PE-PROVEN
New EconomyRoute::MarketClearSold for DELETE .../trade/sold, classified BEFORE the
generic trade cancel arm -- a `sold` tail carries no id, so the cancel handler would
have parsed nothing and acked while clearing nothing. Builder 0x1801647c0 emits
"/sold" when the tradeId field is zero and "/%lld" otherwise; the client calls it
RemoveAllSoldFromTradePile. New market-store column cleared_at records the seller's
acknowledgement SEPARATELY from the sale, so clearing can never be mistaken for
re-settling: it is presentation only, moves no coins and no ownership, and is
idempotent for client retries.
FOUND AND FIXED A LATENT STORE BUG
Adding a column via the additive ALTER path immediately after CREATE TABLE in the
same open() desynced sqlx's per-connection schema cache: a fresh store then read a
12-column row while metadata said 13, panicking a pool worker with an index
out-of-bounds and silently returning zero listings. Declaring cleared_at in
CREATE_LISTINGS fixes it; the ALTER now only serves pre-existing stores. This would
have bitten the next column too.
STAGING, WITHOUT TOUCHING PRODUCTION
The client learns the UTAS base from BLAZE (blaze_responder_v3b.py:646 hardcodes
:8099), and it dials that port directly, so redirecting UTAS means changing Blaze or
port 8099 -- both production. 10.10.0.121 is unreachable. The compliant path is a
parallel stack on spare ports plus a one-line change to the CLIENT's own config:
* scripts/sold-staging-up.py / sold-staging-down.py -- staging Core 18081,
utas-host 8299, Blaze 42327/42330/42331 advertising :8299, two seeded identities,
own DBs under /home/alex/openfut-sold-staging/. Patches a COPY of the Blaze
responder and asserts every substitution applied, so a silent no-op cannot leave
it pointing at production. Kills only recorded pids whose cmdline contains the
staging dir (openfut-utas-host matches BOTH, so pkill-by-pattern is banned).
* docs/SOLD_STAGING_RUNBOOK.md -- the exact client change and its revert.
* src/bin/staging_sell.rs -- the synthetic Buyer B, running the REAL settlement
(CoreEconomy::settle_sale) then mark_sold. Settle-first ordering: a failure
leaves the listing live with nothing moved. Refuses any path containing
openfut-promotion or the production ports.
* scripts/sold-wire-check.py -- proves the whole flow headless before any operator
time is spent.
WIRE CHECK: 35/35 PASS on the canonical 150-coin sale. Seller 1,000 -> 1,143 (fee 7,
proceeds 143), buyer 20,000 -> 19,850, ownership transferred, exactly ONE
authoritative instance, economy shrank by exactly the fee. Sold row: closed,
currentBid 150, expires 0, twelve atoms, counts sold 1 / selling 0, /trade/status
agreeing. Variant B differs only in bidState and coinsProcessed. Clear: 200 {}, row
gone, counts.sold 0, no coins moved, buyer keeps the item, second clear a safe no-op.
Gates: 104 host lib tests (+9), all 7 host targets green, clippy clean, zero fmt
diffs in the new code. Settlement candidate unchanged. NOT PROMOTED.
Production untouched: prod-host pid 3631953 uptime 2h44m restarts=0, coins and
/tradePile unchanged, nothing under /home/alex/openfut-promotion/state/ opened.
The A/B itself is NOT yet run: it needs a real FIFA client, which is operator work.
|
||
|
funman300
|
f6606accb3 |
feat(market): FIFA 5% transfer fee policy, host settle_sale capability, isolated staging harness
Core gains the generic settlement (gitlink 31ab4a6); the FIFA-specific parts live here. FEE (openfut-adapter-fifa17/src/fut/economy_policy.rs), beside pack_price and match_reward_total because 5% is a game policy constant and Core must stay game-neutral — Core only validates 0 <= fee <= gross and never computes a rate: TRANSFER_MARKET_FEE_PERCENT = 5 transfer_market_fee(gross) = floor(gross * 5 / 100), i128 intermediate seller_proceeds(gross) = gross - fee Integer only. Floating point is never used for coin settlement: 0.05 is not representable in binary and a f64 round trip can create or destroy a coin at large prices. Widening to i128 makes overflow unreachable for any i64 price, so no price ceiling has to be assumed. ROUNDING IS A CHOICE AND IT IS NOT CONFIRMED. The fee is floored, so the seller keeps the fractional coin, chosen because it makes fee + proceeds == gross hold exactly at every input — the property the accounting invariant rests on. The discriminating case against flooring the seller's 95% instead is a gross of 150: this rule pays 143, the alternative 142. Nothing in the corpus or the client binary settles which the real server did (the client is only ever told the gross; no tax/netPrice/sellerProceeds wire field exists). Pinned at 0/1/19/20/21/39/40/100/ 150/200/1_000/15_000/15_000_000/i64::MAX plus a fee+proceeds==gross sweep. HOST: CoreEconomy gains settle_sale + EconomySale/EconomySaleReceipt, implemented on HttpCoreClient as POST /economy/settle-sale. Request field names were checked against Core's actual SettleSaleRequest/SaleReceipt rather than assumed. Absent club ids are OMITTED from the body (not null), which is what Core's Outside/active-club defaults depend on, so a unit test pins that body shape. handle_market_buy is deliberately untouched: the synthetic buy path has no counterparty, so minting there is correct. HARNESS: scripts/settlement-staging.py, stdlib only, drives a REAL Core over real HTTP on an ephemeral port against a throwaway DB (production 8099/8199/18080 in a hard deny-list checked in three places), seeds the canonical two-party fixture, prints BEFORE/PURCHASE/AFTER with PASS-FAIL lines, cleans up in a finally. 31/31 pass. It found the rejection-precedence bug fixed in Core, and that Core's content preflight aborts startup on an owned card whose CardDefinitionId no pack defines. Gates: Core 194, adapter 217, host 127, harness 31/31, clippy clean, new code fmt-clean. Nothing deployed; no production process, port or database was touched. |
||
|
funman300
|
71fcf5e251 |
feat(fifa17): own club/stats/{year,consumables} in Rust (Core-accurate)
Migrate the MY CLUB stat set from the Python proxy to a Rust handler computing Core-accurate counts: player tiers + rare from the collection, staff/consumable families from catalog kind+subtype, per-nation buckets via the reverse entity resolver. Faithful port of fut_club_stats.py (VOCAB + global_counts + context_rows). Unlike the oracle (stale profile + synthetic consumable shelf), this reflects the real imported content (incl. the content-gap consumables/staff). Fail-closed 503 on Core error. club/stats/country|league|team sub-screens remain Python (documented). Adds adapter club_stats module (5 tests), host handler + classify arm + resolver subtype_of/rareflag_of, ownership + integration tests; reachability tool splits club/stats global(migrated) vs context(residual). |
||
|
funman300
|
70eb3fc13f |
feat(fifa17): own FUT hub tile counts in Rust
Migrate GET /hub from the Python proxy to a Rust handler deriving counts from authoritative state: clubPlayers = owned PLAYER cards in Core (consumables/staff excluded via catalog kind; may be lower than Python's profile count by the deferred Legend instances = DIFFERENT-BY-DESIGN), auction/tradePile counts from the durable market store via the async bridge. Fail-closed 503 on Core error; market read failure degrades cosmetic counts to 0. Adds classify arm, handler, ownership + integration tests; reachability tool marks hub migrated. |
||
|
funman300
|
6eec3b9ec7 |
test(fifa17): add UTAS route-reachability reporter (Python-hit gate)
Parses the host owner= dispatch log into per-owner + per-domain counts and gates on the post-P1 invariants: economy Python hits == 0 (P1 regression), migrated non-economy routes (accountinfo/settings/leaderboards/match-reset/phishing) == 0, and residual Python domains == documented set. Read-only; staging-preflight and Phase 40 live-ownership use. |
||
|
funman300
|
88da16a11e | feat(fifa17): curated dev content pack generator + Core dev seed (submodule 36abd4b) | ||
|
funman300
|
36fe1caa3f |
feat(fifa17): deterministic base-card seed generator + full identity catalog
scripts/seed_fifa17_cards.py: deterministic pipeline from committed FIFA17 data (pool.json + roster.json + leagues/nations/teams tables) -> the card-definition identity catalog. CardDefinitionId is opaque + deterministic (fifa17_<asset>), version 0 (base cards only; resource_id == asset_id). --check mode diffs against committed output (drift-detection mutation-proven). Provenance embedded. Generated openfut-adapter-fifa17/data/fifa17-card-identities.json: all 17,563 base assets. Semantic definition coverage (to /tmp, not committed here): 17,547 resolvable; 16 skipped for missing roster name (reported, never fabricated). Adapter loads the committed catalog (test: 17,563 entries, Ronaldo fifa17_20801 -> asset 20801 v0). Phase commit 3/5. NOT owned inventory: this is 'which cards exist', not 'which the user owns'. Core content seeding + dev-owned set next. |
||
|
funman300
|
04c5043aba |
utas: My Squad filter corpus — every filter identified, root cause measured
Controlled retail capture, one criterion at a time, cleared between each. 47 transactions. Every filter the My Squad picker sends is now known from the wire rather than guessed. Route: GET /ut/game/fifa17/club -- the picker hits UTAS and reuses the general club-inventory route. level=any|gold quality lowercase, ALWAYS present rare=SP "Special" uppercase, OMITTED when off position=ST position uppercase, omitted when off nation=52 entity id numeric league=13 entity id numeric team=5 entity id numeric, NESTED under league sort=desc client constant; the UI has no sort control start=/count=11 pagination Two encoding families: short string enums, and numeric FIFA ids. The ids must never reach Core. Filters compose as plain ANDs in one query -- string and id filters alike -- so each maps independently. THE ROOT CAUSE IS SELF-AMPLIFYING. club_route honours type, team and league; it never reads start, count, level, sort or year. Because start is ignored, every page returns the same full set, so the client concludes the page was full and asks for the next one. One scroll produced 22 requests and 6.2 MB, stopping at start=200 only because the client gave up -- against a filtered set of 32 items that should have been three pages. That also explains why the bug reads as erratic rather than broken: league=13&position=ST returns every Premier League player instead of Premier League strikers. Plausible, wrongly sized, hard to notice. Measured filtered sets, from the real cluttered club -- these are the acceptance test for the fix: unfiltered 1962 league=13 350 league=13&team=5 32 Two client behaviours worth carrying forward: the picker fires a query per highlighted entry, not per selection (two requests for one club pick), and parameter ORDER is not stable, so parsing must be key-value. FIXTURE SIZE: bodies over 4 KB are truncated in the committed fixture, with body_full_len and body_full_sha256 retained, because the same 1.1 MB club response repeats ~25 times and its hash already proves identity. 13.3 MB -> 247 KB. The raw .ofcap keeps every byte, privately and gitignored. Truncation is recorded per transaction so a trimmed fixture is never mistaken for a whole response. Audited across all three identifier surfaces -- headers, JSON bodies, query strings -- before and after the size change: no leaks. 6/6 sanitiser mutations still killed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
funman300
|
0b66662525 |
utas: first real corpus, and two sanitiser gaps the audit caught
24 transactions across 11 connections from a retail session: login, hub, one pack open, two squad saves, a quick-sell, with before/after state manifests. Raw .ofcap stays gitignored at 0600; the sanitized corpus is committed as adapter fixtures. TWO GAPS FOUND BY AUDITING THE OUTPUT, NOT BY TRUSTING THE SANITISER. 1. `POST /ut/auth` carries `macAddress` and `deviceId`. Session tokens were being redacted correctly and these were not. A committed fixture is a published fixture. 2. Then, with those fixed, the audit fired AGAIN on the file about to be committed: `GET .../phishing/trusteddevice?deviceId=...` puts the id in the QUERY STRING. Three input surfaces carry identifiers -- headers, JSON bodies, and query strings -- and the sanitiser knew about two. Both fixed in the tool rather than by editing the file, with a regression test and a mutation for the query path. AND A THIRD ARTEFACT MIX-UP, in the mutation harness itself. It reported the query-redaction mutation as SURVIVED while a hand-run of the same mutation killed it. Cause: the harness pointed at a stale scratchpad copy of the test that pre-dated the query assertion, so it was faithfully testing the mutated tool against a test that could not detect the mutation. That is the same class as the build guard checking the wrong binary and cargo reusing a binary compiled from mutated source -- the third instance today of measuring the wrong artifact. The harness now resolves ROOT from its own location and runs the COMMITTED test; the stale copy is deleted. Harness committed as scripts/mutate-utas-observe.py so this is repeatable rather than a thing that happened once in a scratch directory. 6/6 killed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
funman300
|
cdea85e214 |
utas: standalone recording proxy that tees rather than rebuilds
UTAS needs a real request/response corpus before any Rust is written: it is where protocol shape and FUT state start being coupled, so guessing is worse here than it was for Blaze. The oracle truncates logged bodies at ~200 chars, and raising that cap would mean editing the behavioural specification to make it easier to copy -- backwards. A proxy gets the same evidence and leaves the oracle untouched. THE DESIGN RULE: TEE, DO NOT REBUILD. UTAS is plaintext HTTP/1.1 on ThreadingHTTPServer, so keep-alive, pipelining and chunked transfer are all live. A proxy that parses a request and re-emits it can corrupt the traffic it exists to observe -- and that corruption would present as a UTAS bug, pointing the investigation in exactly the wrong direction. So bytes are copied verbatim in both directions and a second copy goes to disk; transactions are reconstructed later, offline, from that copy. A parser bug therefore spoils the record and never the session. Standalone, NOT in the container, so the same tool can later sit in front of a Rust UTAS host and replay an identical captured request against both. Two layers, as with the Blaze captures: raw/*.ofcap is exact bytes at mode 0600 and gitignored; sanitized/transactions.jsonl is the committed artefact. Bodies are preserved EXACTLY and sanitised second -- only known-secret headers and JSON keys are replaced, structure is never reshaped, and every redaction is recorded in the transaction so a reader knows what was touched. Captured per transaction: connection id, sequence, relative and wall time, elapsed ms, method, path, query, HTTP version, headers IN RECEIVED ORDER as pairs (a dict would drop duplicates and ordering), raw body and length for both directions, status, and observed keep-alive. Verified as two independent properties, because they fail differently: transparency (bytes through the proxy identical to bytes direct, Date masked, with the mask asserted to have fired) and fidelity (parsed transactions match what was sent, including a dechunked response and a 300-byte POST body). 5/5 mutations killed, including "record but do not forward", "drop the last byte of every chunk" and "stop redacting". scripts/test-utas-observe.py is committed alongside it: a capture tool nobody can re-verify is not evidence. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
funman300
|
8f3b659c33 |
lifecycle: one host-lifecycle helper; roster.sh; ban pkill -f
redirector.sh and the coming roster.sh needed the same five rules, each
of which cost something to learn:
* resolve /proc/PID/exe; never match a command line. `pkill -f` /
`pgrep -f` match any shell whose ARGUMENTS mention the name, including
the shell running the command. That has killed this session's own
shell twice, and is now banned in migration tooling -- the helper
contains no `-f` matching and the header says why.
* `readlink`, not `readlink -f`. After a rebuild the link reads
"<path> (deleted)" and -f resolves it to nothing, so the orphan check
goes blind to exactly the long-lived processes it exists to find. Two
orphans hid there, one serving the wrong certificate.
* stop PROVES the process is gone and the port free.
* an ambiguous binary is an error for start/verify but NOT for
stop/status: rollback must never be blocked by a question about the
build tree.
* verify the RUNNING process's commit, not the artifact on disk, which
a rebuild can silently advance past.
Copying those into a second script would have been the same mistake as
copying the TLS setup. Instead scripts/host-lifecycle.sh owns them and a
service supplies four facts: name, crate, executable, port variable.
redirector.sh goes from 178 lines to 26 and roster.sh is 24, with no
behaviour change -- the refactored redirector.sh still sees the live
armed process (pid 830736, port 42227) and still refuses correctly
because HEAD has moved past it.
Paths are unchanged (rundir, pidfile, portfile, commit stamp, log), so
the currently running redirector stays manageable across this refactor.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
funman300
|
fc411bb6f1 |
scripts: require one certificate across the whole FIFA-facing TLS stack
Two live gates were lost to a second variable I had been asked to eliminate. The Rust redirector was pointed at the repo's fifa17-recon/tools/redir_cert.pem (fingerprint F9:16:1A...), while the running container serves a different cert baked into its image (E7:F9:46...) which the Python redirector, roster and the rest of the stack all share. So the A/B compared TLS implementation AND certificate identity at once. FIFA 17's ProtoSSL caches the server certificate for a backend. The redirector is the first TLS connection of a session, so its cert becomes the one the client expects; the next service presenting a different cert fails its handshake. That is why the redirect itself always succeeded and the failure surfaced later, on the roster fetch -- "An error occurred downloading the FUT Squad Update". It stayed invisible because Python's socketserver swallows it: a handshake failure at accept() raises ssl.SSLError, which subclasses OSError and is discarded by _handle_request_noblock. No request log, no stderr. Every server looked healthy while the client could not talk to any of them. Confirmed on the wire: tls-observe in front of the roster server captured four ClientHellos from the client, correct SNI and the same 8 static-RSA suites it offers the redirector, none of which produced a request. The check is mutation-tested against the real bug: with a redirector started on the stale repo cert it exits 1 and names the mismatch. |
||
|
funman300
|
5bc39e902d |
tooling: observe client connection ATTEMPTS; make the build guard reject bad args
openfut-observe.sh answers the one question no server log can: when a gate fails and a service logged nothing, did the client try and fail, or never try? Both look like silence. Two redirector gates were lost to that ambiguity -- "roster server logged nothing" was equally consistent with a broken roster service, a wrong roster URL, and a client that never asked. Built on iptables packet counters because this box has no tcpdump, no conntrack, and no readable kernel log. That last one is verified rather than assumed: an initial LOG-based version installed correctly and its rules matched (counters proved it), but the output went nowhere -- journalctl -k has no entries and dmesg is empty. Counters are also lower volume and record only SYNs, so no payload can be captured even in principle. Validated against the live client, not a loopback stand-in: an initial self-test using this host's own address counted almost nothing, because locally-generated packets never traverse PREROUTING. Against the real remote client it counts 8081 at ~4/min, matching the roster server's own log. Known gap, recorded rather than hidden: the catch-all TOTAL runs well above the sum of the named ports, so the client makes steady background attempts to ports not tracked here. It is present during a working session, so it is not the failure signature, and it is not chased further here. verify-build-identity.sh now rejects an argument that is not a commit hash. Passing the binary path instead of its stamp previously produced a plausible "REFUSING: binary was built from ./target/release/... but HEAD is <sha>", which reads as a real stale-build finding rather than a caller mistake -- and a safeguard that cries wolf is one people learn to route around. Usage error is now exit 2, distinct from a genuine stale build (1) and success (0). |
||
|
funman300
|
c03702707b |
redirector: commit stamp + shared build-identity verifier that REFUSES
The binary records only the commit it was built from -- no dirty-tree flag. Cargo will not re-run a build script because another crate's source changed, so a compiled-in 'clean' claim can be stale and is not a safeguard; that was verified on the Blaze host. scripts/verify-build-identity.sh establishes both facts at LAUNCH, where they cannot go stale: the stamped commit equals HEAD, and the migration crates are clean. It REFUSES rather than warns, because for a migration gate a warning on stderr is something to scroll past. --identity prints the stamp without valid configuration. The launcher must be able to establish which commit a binary came from BEFORE deciding whether to run it; requiring a correct environment first would invert the check. redirector.sh mirrors sidecar.sh: refuses to start with an orphan present or the port busy, matches the resolved executable rather than the command line (pgrep -f matches any shell mentioning the name), and stop PROVES the process is gone and the port free. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
funman300
|
8f5f54833f |
ci: tripwire against lab addresses creeping back into tracked source
Cheap insurance, explicitly not the real check -- the semantic tests in
deployment_config.rs are what prove propagation, using two TEST-NET addresses
and bind != advertise. This grep only stops the lab subnet reappearing months
from now when the reasoning has been forgotten.
Deployment config legitimately contains real addresses and lives in gitignored
files, so it is never scanned. The frozen baseline doc is allowlisted BY PATH:
it records what a past deployment actually was, and rewriting it would falsify
the record.
Also swapped the lab IP for a TEST-NET placeholder in the usage examples and
error messages of compose/entrypoint/client_arm. Those were already correct
architecture -- every one requires the address via ${VAR:?} -- but using the
real lab IP as the example is the same 'happens to match our lab' smell, and
placeholders keep the tripwire allowlist near-empty.
Mutation-tested: adding a lab address to a source file makes it exit 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|