fifa17: carry the staff rating the client re-rates to, verified live

Staff quick-sell could not be priced correctly: for cardtypes 2/3/4/5/10 the
client overwrites the rating and rare flag we send with values from its own card
database, and a staff wire record carries no rating, no rareflag and no
discardValue at all. The server had no way to know the displayed price from what
it sent, so pricing declined for staff and fell back to the placeholder ladder.

The missing input was read straight out of the running client (pid 6580), no UI
interaction required:

  * tools/coach_probe.py grades all four resident staff records HIT, which by
    construction requires record +0xb4 == the table's `value` and +0x58 == its
    `rare`. That settles `value`-is-the-rating, which was previously an inference
    and was deliberately not shipped on that basis.
  * tools/discard_probe.py (new) reads both discard slots -- +0x38, the value we
    sent, and +0x3c, the value the client computed for itself:

      1000509  sub 4  ct 2  rat 88  rare 1   sent 0   calc 282   predicted 282
      9000081  sub 6  ct 10 rat 66  rare 0   sent 0   calc  36   predicted  36
      3000083  sub 8  ct 4  rat 66  rare 0   sent 0   calc  36   predicted  36

    4 of 4 agree, 0 disagree. 36 on the value-66 GK coach was the exact falsifier
    written for this last commit.

Entities::enrich_staff now fills rating from `value` and rareflag from `rare` for
the five staff families, and the catalog emits the real rareflag instead of a
hardcoded 0 (it is not cosmetic -- it selects the discard price column, which is
why the rare-1 manager prices at 282 and a rare-0 coach at 36). Players and
consumables are untouched; their wire values are authoritative.

Verified on staging: a GK coach quick-sells for 36, not the 150 floor. The
catalog diff is exactly the two coach entries gaining rating 66; 1710 entries in
and out, nothing else changed.

The same probe shows what production does to PLAYERS today: every resident player
carries sent+38 = 1500, which suppresses the client's own computation, against a
real 688..752 for a gold rare and 72,800 / 74,400 for the two legends.

Still open, and not a discard problem: manager fifa17_1000509 is owned in Core
but has no catalog entry or definition (it reaches the client through the opaque
squad extension), so it declines to the ladder. That is definition coverage.

Importer 41 tests, fmt and clippy clean.

openfut-import-fifa17/src/lib.rs

@@ -87,16 +87,37 @@ pub fn load_roster(path: impl AsRef<Path>) -> Result<Roster> {
 
 // --------------------------------------------------------------- entities
 
-/// Forward numeric-id -> name resolution for the committed FIFA 17 tables
-/// (`leagues.json`/`nations.json`/`teams.json`, `{schema, rows:[…]}` dump).
-/// Mirrors `scripts/seed_fifa17_cards.py` (`_table_map`).
+/// Forward lookups over the committed FIFA 17 tables: numeric id -> name for
+/// `leagues.json`/`nations.json`/`teams.json` (`{schema, rows:[…]}` dump, mirrors
+/// `scripts/seed_fifa17_cards.py`'s `_table_map`), plus EA's authored rating and
+/// rare flag for the five STAFF families.
 #[derive(Default)]
 pub struct Entities {
     leagues: BTreeMap<i64, String>,
     nations: BTreeMap<i64, String>,
     teams: BTreeMap<i64, String>,
+    /// `(cardsubtypeid, carddbid)` -> `(value, rare)` for the five staff tables.
+    ///
+    /// `value` is the RATING the client re-rates a staff card to, and `rare`
+    /// selects its discard price column. Both are LIVE-VERIFIED: reading the
+    /// running client's own card records back out of memory
+    /// (`tools/coach_probe.py`, 4/4 HIT) shows record `+0xb4` == this `value`
+    /// and `+0x58` == this `rare`, and the discard value the client computes at
+    /// record `+0x3c` (`tools/discard_probe.py`) matches the price those two
+    /// select — 36 for the `value`-66 GK coach, 282 for the `rare`-1,
+    /// `value`-88 manager.
+    staff: BTreeMap<(i64, i64), (i64, i64)>,
 }
 
+/// `cardsubtypeid` -> staff table basename (`FUN_1800d8330`'s family selector).
+const STAFF_TABLES: [(i64, &str); 5] = [
+    (4, "managercards"),
+    (5, "headcoachcards"),
+    (6, "gkcoachcards"),
+    (7, "physiocards"),
+    (8, "fitnesscoachcards"),
+];
+
 fn load_table(path: &Path, id_key: &str, name_key: &str) -> Result<BTreeMap<i64, String>> {
     let raw = std::fs::read_to_string(path)
         .with_context(|| format!("reading table {}", path.display()))?;
@@ -118,17 +139,48 @@ fn load_table(path: &Path, id_key: &str, name_key: &str) -> Result<BTreeMap<i64,
     Ok(map)
 }
 
+/// `carddbid -> (value, rare)` from one staff table dump.
+///
+/// `value` is EA's authored rating for the card and `rare` its rare flag; both
+/// are read verbatim, never defaulted, and a row missing either is skipped so a
+/// gap stays a gap.
+fn load_staff_table(path: &Path) -> Result<BTreeMap<i64, (i64, i64)>> {
+    let raw = std::fs::read_to_string(path)
+        .with_context(|| format!("reading staff table {}", path.display()))?;
+    let doc: serde_json::Value = serde_json::from_str(&raw)
+        .with_context(|| format!("parsing staff table {}", path.display()))?;
+    let rows = doc
+        .get("rows")
+        .and_then(|r| r.as_array())
+        .with_context(|| format!("staff table {} has no rows[]", path.display()))?;
+    let mut map = BTreeMap::new();
+    for row in rows {
+        let get = |k: &str| row.get(k).and_then(serde_json::Value::as_i64);
+        if let (Some(id), Some(value), Some(rare)) = (get("carddbid"), get("value"), get("rare")) {
+            map.insert(id, (value, rare));
+        }
+    }
+    Ok(map)
+}
+
 impl Entities {
     pub fn from_tables_dir(dir: impl AsRef<Path>) -> Result<Self> {
         let dir = dir.as_ref();
+        let mut staff = BTreeMap::new();
+        for (subtype, table) in STAFF_TABLES {
+            for (carddbid, row) in load_staff_table(&dir.join(format!("{table}.json")))? {
+                staff.insert((subtype, carddbid), row);
+            }
+        }
         Ok(Entities {
             leagues: load_table(&dir.join("leagues.json"), "leagueid", "leaguename")?,
             nations: load_table(&dir.join("nations.json"), "nationid", "nationname")?,
             teams: load_table(&dir.join("teams.json"), "teamid", "teamname")?,
+            staff,
         })
     }
 
-    /// Build directly from id->name maps (tests).
+    /// Build directly from id->name maps (tests). Carries no staff table.
     pub fn from_maps(
         leagues: BTreeMap<i64, String>,
         nations: BTreeMap<i64, String>,
@@ -138,6 +190,32 @@ impl Entities {
             leagues,
             nations,
             teams,
+            staff: BTreeMap::new(),
+        }
+    }
+
+    /// EA's `(rating, rare)` for one staff definition, or `None` when the id is
+    /// absent from its family's table (the client would draw its own miss-fill).
+    pub fn staff_stats(&self, subtype: i64, carddbid: i64) -> Option<(i64, i64)> {
+        self.staff.get(&(subtype, carddbid)).copied()
+    }
+
+    /// Fill in the rating and rare flag the client re-rates STAFF cards to.
+    ///
+    /// The wire never sends either (measured: a staff record arrives with no
+    /// `rating`, no `rareflag` and no `discardValue`), so without this a staff
+    /// definition reaches the catalog with `rating: None` and the server cannot
+    /// price a quick sell to match what the client displays. Players and
+    /// consumables are untouched — their wire values are authoritative.
+    pub fn enrich_staff(&self, plan: &mut NonPlayerPlan) {
+        for def in &mut plan.supported {
+            if def.kind != ContentKind::Staff && def.kind != ContentKind::Manager {
+                continue;
+            }
+            if let Some((value, rare)) = self.staff_stats(def.subtype, def.resource_id) {
+                def.rating.get_or_insert(value);
+                def.rareflag.get_or_insert(rare);
+            }
         }
     }
 
@@ -604,6 +682,11 @@ pub struct NonPlayerDefinition {
     pub contract: Option<i64>,
     /// Card rating, when the source carries one (matches the `fcc_*` row).
     pub rating: Option<i64>,
+    /// EA's authored `rare` flag for a STAFF definition, from the same table row
+    /// as [`Self::rating`]. The wire never carries it (the client re-rates staff
+    /// from its own DB), and it is NOT cosmetic: it selects the discard price
+    /// column, so a `rare`-1 manager and a `rare`-0 coach price differently.
+    pub rareflag: Option<i64>,
     /// Honest functional label (e.g. "Player Contract", "GK Coach", "Kit").
     pub name: String,
     /// Wire ids of every owned copy of this resourceId (preserved).
@@ -789,6 +872,7 @@ pub fn plan_non_player_definitions(profile: &Profile) -> NonPlayerPlan {
             amount,
             contract,
             rating,
+            rareflag: None,
             name: label.to_string(),
             wire_ids,
         });
@@ -1013,7 +1097,10 @@ pub fn analyze(
     let identity = plan_identity(profile, &supported_rids);
     let supported_wire: BTreeSet<i64> = identity.import_wire_ids.iter().copied().collect();
     let squad = plan_squad(profile, &supported_wire);
-    let non_player = plan_non_player_definitions(profile);
+    let mut non_player = plan_non_player_definitions(profile);
+    // Staff carry no rating or rare flag on the wire; fill them from the tables
+    // the client itself re-rates from, so a quick sell can be priced to match.
+    entities.enrich_staff(&mut non_player);
     Report {
         game: "fifa17".to_string(),
         persona_id: profile.persona_id,
@@ -1270,7 +1357,7 @@ pub fn emit_content(
     }
     for d in &report.non_player.supported {
         // asset_id falls back to resource_id (staff carry no assetId); version 0,
-        // rareflag 0 — a consumable/staff never renders as a special card.
+        // because a consumable/staff never renders as a versioned special card.
         cards.insert(
             d.card_id.clone(),
             serde_json::json!({
@@ -1286,7 +1373,10 @@ pub fn emit_content(
                 "rating": d.rating,
                 "asset_id": d.asset_id.unwrap_or(d.resource_id),
                 "version": 0,
-                "rareflag": 0,
+                // NOT cosmetic: `rareflag` selects the discard price column, and
+                // a staff card's is EA's own `rare`, filled from its family table
+                // by `Entities::enrich_staff`. 0 for everything else, as before.
+                "rareflag": d.rareflag.unwrap_or(0),
                 "kind": d.kind.as_str(),
                 "subtype": d.subtype,
             }),

openfut-import-fifa17/src/tests.rs

@@ -1068,3 +1068,76 @@ fn deferred_non_player_instances_gate_a_production_apply() {
     assert!(gate_staging(&plan, false).is_err(), "production blocks");
     assert!(gate_staging(&plan, true).unwrap(), "staging opt-in allows");
 }
+
+/// The staff rating and rare flag come from the client's OWN tables, and these
+/// exact values were read back out of the RUNNING client's memory:
+/// `tools/coach_probe.py` graded all four resident staff records HIT (record
+/// `+0xb4` == `value`, `+0x58` == `rare`), and `tools/discard_probe.py` read the
+/// discard value the client computed for itself at record `+0x3c` — 36 for both
+/// `value`-66 coaches and 282 for the `rare`-1, `value`-88 manager.
+///
+/// So this is not a table-parsing test. It pins the importer to numbers the live
+/// client demonstrably uses.
+#[test]
+fn staff_stats_are_the_values_the_live_client_re_rates_to() {
+    let dir = concat!(env!("CARGO_MANIFEST_DIR"), "/../fifa17-recon/data/tables");
+    let ent = Entities::from_tables_dir(dir).expect("committed tables load");
+
+    // (subtype, carddbid) -> (value, rare), verified live.
+    assert_eq!(ent.staff_stats(6, 9000081), Some((66, 0)), "GK coach");
+    assert_eq!(ent.staff_stats(8, 3000083), Some((66, 0)), "fitness coach");
+    assert_eq!(ent.staff_stats(4, 1000509), Some((88, 1)), "manager");
+
+    // Keyed per family: a coach id must not resolve through another's table.
+    assert_eq!(
+        ent.staff_stats(4, 9000081),
+        None,
+        "gkcoach id is not a manager"
+    );
+    assert_eq!(ent.staff_stats(6, 12345678), None, "absent id stays absent");
+}
+
+/// `enrich_staff` fills ONLY staff, and only where the wire left a gap.
+#[test]
+fn enrich_staff_fills_staff_and_leaves_everything_else_alone() {
+    let dir = concat!(env!("CARGO_MANIFEST_DIR"), "/../fifa17-recon/data/tables");
+    let ent = Entities::from_tables_dir(dir).expect("committed tables load");
+
+    let items = vec![
+        // A GK coach (subtype 6) and a contract consumable, which carries its own
+        // rating on the wire and must not be touched.
+        staff(100000280, 9000081, 6),
+        consumable(100000300, 5001004, 201),
+    ];
+    let mut plan = plan_non_player_definitions(&profile(&items, "[]", 100000500));
+    let before: Vec<Option<i64>> = plan.supported.iter().map(|d| d.rating).collect();
+    ent.enrich_staff(&mut plan);
+
+    let coach = plan
+        .supported
+        .iter()
+        .find(|d| d.resource_id == 9000081)
+        .expect("coach planned");
+    assert_eq!(
+        coach.rating,
+        Some(66),
+        "rating filled from gkcoachcards.value"
+    );
+    assert_eq!(coach.rareflag, Some(0), "rare filled from the same row");
+
+    let cons = plan
+        .supported
+        .iter()
+        .find(|d| d.resource_id == 5001004)
+        .expect("consumable planned");
+    assert_eq!(cons.rareflag, None, "a consumable gets no staff rare flag");
+    let cons_before = before[plan
+        .supported
+        .iter()
+        .position(|d| d.resource_id == 5001004)
+        .unwrap()];
+    assert_eq!(
+        cons.rating, cons_before,
+        "the wire rating is left untouched"
+    );
+}