feat(import-fifa17): --apply — recoverable two-store real-profile import

FIFA17-specific orchestration that installs the real profile across BOTH durable stores (openfut-identity + Core SQLite) recoverably and idempotently, handing Core only a GENERIC request (all FIFA17 semantics stay in this adapter layer). apply module: - owned_item_id(persona, wire) = deterministic UUIDv5 from a private namespace; identical in BOTH stores (identity core_id AND Core owned_cards.id), so the running host's wire->owned reverse lookup resolves exactly what Core stored. - plan_apply(report, raw_profile, fp): pure translation to a GenericImportRequest (card_id = fifa17_<resourceId>) + the preserved (owned_item_id <-> source wire) mappings + watermark. Canonical squad + Fifa17SquadExtensionV1 are built by the SAME adapter code (parse_squad_put + build_squad_write) the retail-validated live squad path uses. Refuses if the report has blockers. - Two-store protocol (apply): staging gate -> local Core-preflight mirror -> identity dry-preflight -> idempotent seed (insert_existing_mapping per instance + set_watermark) -> ONE generic Core import transaction (spawned binary) -> cross-store post-validation -> completion record. A crash after identity seeding re-converges on re-run (idempotent mappings + Core already_imported): no cleanup, no reminting. - gate_staging: deferred players are ABSENT from an import; allowed only for a staged run behind --allow-deferred-players-for-staging (never a silent default; prints an INCOMPLETE banner). Production requires zero deferred instances. CLI: --apply (with --emit-content, --core-bin, --core-db, --core-data, --identity-store, --allow-deferred-players-for-staging). Depends on openfut-adapter-fifa17 + openfut-identity + uuid(v5). Proven end-to-end on the real 33068179/CAGE profile (staged): first apply imports 1949 supported instances (293 base + versioned), 11/11 f433 squad + opaque extension, coins 28,112,944, fingerprint 8dc5582d2414af28; re-run is an idempotent no-op (already_imported, DB unchanged); staging-not-default refuses before any write; every OwnedItemId is an opaque UUID; identity wire-id set == source supported set exactly (0 minted, 0 dropped); 13 deferred instances leak 0. 10 new apply tests (determinism, request/mapping/squad translation, blocker refusal, staging gate both ways, local preflight, identity seed/dry/postvalidate/ idempotency, conflict detection, graceful spawn failure). clippy -D warnings clean; crate suite 23 tests green.
2026-08-12 20:36:34 +00:00
parent c71c2a8d33
commit 1631d3b1a2
6 changed files with 811 additions and 4 deletions
@@ -0,0 +1,494 @@
+//! `--apply`: the FIFA17-specific orchestration that installs a real profile
+//! across the TWO durable stores — the `openfut-identity` external-id store and
+//! OpenFUT Core's SQLite DB — recoverably and idempotently.
+//!
+//! Boundary: everything FIFA17-specific stays here. This module chooses every
+//! `CardDefinitionId` (`fifa17_<resourceId>`, from the report) and every
+//! deterministic opaque Core `OwnedItemId`, preserves each owned instance's
+//! existing Python wire id, builds the canonical squad + `Fifa17SquadExtensionV1`
+//! with the SAME adapter code the retail-validated live squad path uses, then
+//! hands Core a GENERIC [`GenericImportRequest`]. Core never sees a resourceId,
+//! a wire id, or the extension's meaning.
+//!
+//! Recoverable two-store protocol (see [`apply`]):
+//!   validate → local Core-preflight mirror → identity dry-preflight →
+//!   idempotently install existing wire-id mappings + watermark →
+//!   ONE generic Core import transaction → cross-store post-validation →
+//!   completion record.
+//! If Core fails after identity seeding, rerunning the same manifest re-installs
+//! identical identity mappings (idempotent) and Core converges (same
+//! `source_fingerprint` → `already_imported`). No cleanup, no reminting.
+
+use std::collections::{BTreeSet, HashMap};
+use std::path::{Path, PathBuf};
+use std::process::Command;
+
+use anyhow::{bail, Context, Result};
+use serde::Serialize;
+use uuid::Uuid;
+
+use openfut_adapter_fifa17::fut::catalog::Fifa17WireItemIdPolicy;
+use openfut_adapter_fifa17::fut::squad::{parse_squad_put, SquadWireResolver};
+use openfut_adapter_fifa17::fut::squad_ext::{
+    build_squad_write, EXT_NAMESPACE, EXT_SCHEMA_VERSION,
+};
+use openfut_identity::{ExternalIdentityStore, JsonIdentityStore};
+
+use crate::Report;
+
+/// Private, project-specific UUIDv5 namespace for deriving Core OwnedItemIds from
+/// a source account + source owned-item identity. Fixed forever so a re-import of
+/// the same account yields the SAME opaque ids (idempotent across both stores).
+/// Core never parses why these ids are stable — they are opaque primary keys.
+const OWNED_ITEM_NAMESPACE: Uuid = Uuid::from_bytes([
+    0x0f, 0x17, 0x0f, 0x17, 0x9a, 0x11, 0x5e, 0xd0, 0xb0, 0x0f, 0x0e, 0x11, 0x0c, 0x0a, 0x11, 0x17,
+]);
+/// Deterministic opaque Core OwnedItemId for one source owned instance. Stable
+/// across runs and identical in BOTH stores (identity mapping core_id AND Core
+/// `owned_cards.id`), so the running host's wire→owned reverse lookup resolves
+/// exactly what Core stored.
+pub fn owned_item_id(persona_id: i64, source_wire_id: i64) -> String {
+    let name = format!("fifa17:{persona_id}:{source_wire_id}");
+    Uuid::new_v5(&OWNED_ITEM_NAMESPACE, name.as_bytes()).to_string()
+}
+
+// ----------------------------------------------------- generic Core request
+
+#[derive(Debug, Serialize)]
+pub struct GenericProfile {
+    pub username: String,
+    pub game_id: String,
+}
+
+#[derive(Debug, Serialize)]
+pub struct GenericClub {
+    pub name: String,
+    pub coins: i64,
+}
+
+#[derive(Debug, Serialize)]
+pub struct GenericOwned {
+    pub owned_item_id: String,
+    pub card_id: String,
+}
+
+#[derive(Debug, Serialize)]
+pub struct GenericSlot {
+    pub owned_item_id: String,
+    pub position_index: i64,
+    pub is_captain: bool,
+    pub is_on_bench: bool,
+}
+
+#[derive(Debug, Serialize)]
+pub struct GenericExtension {
+    pub namespace: String,
+    pub schema_version: i64,
+    pub payload: String,
+}
+
+#[derive(Debug, Serialize)]
+pub struct GenericSquad {
+    pub formation: String,
+    pub name: String,
+    pub slots: Vec<GenericSlot>,
+    pub extension: GenericExtension,
+}
+
+/// The game-agnostic request Core consumes. Field names match
+/// `openfut_core::services::import::ProfileImportRequest` exactly.
+#[derive(Debug, Serialize)]
+pub struct GenericImportRequest {
+    pub source_fingerprint: String,
+    pub profile: GenericProfile,
+    pub club: GenericClub,
+    pub owned: Vec<GenericOwned>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub squad: Option<GenericSquad>,
+}
+
+/// One preserved (opaque OwnedItemId ↔ source wire id) mapping to install into
+/// the identity store under `(fifa17, owned-item)`.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct IdentityMapping {
+    pub core_id: String,
+    pub wire_id: i64,
+}
+
+/// The complete, side-effect-free plan for an apply: the generic Core request,
+/// the identity mappings to install, and the watermark to set.
+#[derive(Debug)]
+pub struct ApplyPlan {
+    pub request: GenericImportRequest,
+    pub mappings: Vec<IdentityMapping>,
+    pub watermark: i64,
+    pub supported_instances: usize,
+    pub deferred_instances: usize,
+    pub source_fingerprint: String,
+}
+
+struct MapResolver<'a>(&'a HashMap<i64, String>);
+impl SquadWireResolver for MapResolver<'_> {
+    fn owned_id_for_wire(&self, wire: i64) -> Option<String> {
+        self.0.get(&wire).cloned()
+    }
+}
+
+/// Build the full apply plan from an already-analysed [`Report`] + the raw
+/// profile JSON (needed verbatim for the squad wire: `custom`, `kicktakers`,
+/// `kitNumber`, manager, squadType). Pure: no store or Core writes.
+///
+/// Refuses if the report has blockers — never plans an unsafe or lossy import.
+pub fn plan_apply(
+    report: &Report,
+    raw_profile: &serde_json::Value,
+    snapshot_fingerprint: &str,
+) -> Result<ApplyPlan> {
+    if report.has_blockers() {
+        bail!(
+            "refusing to plan apply while blockers are present: {}",
+            report.blockers().join("; ")
+        );
+    }
+    let persona = report.persona_id;
+
+    // Owned cards + identity mappings: one per SUPPORTED owned instance. Each
+    // instance keeps its existing Python wire id; its Core OwnedItemId is derived
+    // deterministically from (persona, wire).
+    let mut owned = Vec::with_capacity(report.identity.import_wire_ids.len());
+    let mut mappings = Vec::with_capacity(report.identity.import_wire_ids.len());
+    let mut wire_to_owned: HashMap<i64, String> = HashMap::new();
+    for def in &report.definitions.supported {
+        for &wire in &def.wire_ids {
+            let core_id = owned_item_id(persona, wire);
+            wire_to_owned.insert(wire, core_id.clone());
+            owned.push(GenericOwned {
+                owned_item_id: core_id.clone(),
+                card_id: def.card_id.clone(),
+            });
+            mappings.push(IdentityMapping {
+                core_id,
+                wire_id: wire,
+            });
+        }
+    }
+
+    // Canonical squad + opaque extension, built by the SAME adapter code the live
+    // squad-write path uses, over the raw source squad. The resolver maps every
+    // occupied wire id to its deterministic OwnedItemId.
+    let squad = if report.squad.present {
+        let raw_squad = raw_profile
+            .get("squads")
+            .and_then(|s| s.get(0))
+            .context("report says a squad is present but profile has no squads[0]")?;
+        let body = serde_json::to_vec(raw_squad).context("re-serialize source squad")?;
+        let put = parse_squad_put(&body).map_err(|e| anyhow::anyhow!("parse source squad: {e}"))?;
+        let build = build_squad_write(&put, &MapResolver(&wire_to_owned))
+            .map_err(|e| anyhow::anyhow!("build squad write: {e}"))?;
+        let formation = build
+            .canonical
+            .formation
+            .clone()
+            .context("source squad has no formation token")?;
+        let slots = build
+            .canonical
+            .slots
+            .iter()
+            .map(|s| GenericSlot {
+                owned_item_id: s.owned_card_id.clone(),
+                position_index: s.index,
+                is_captain: s.is_captain,
+                is_on_bench: s.is_on_bench,
+            })
+            .collect();
+        Some(GenericSquad {
+            formation,
+            name: build
+                .canonical
+                .name
+                .clone()
+                .unwrap_or_else(|| report.club_name.clone()),
+            slots,
+            extension: GenericExtension {
+                namespace: EXT_NAMESPACE.to_string(),
+                schema_version: EXT_SCHEMA_VERSION,
+                payload: build.extension.to_payload(),
+            },
+        })
+    } else {
+        None
+    };
+
+    let request = GenericImportRequest {
+        source_fingerprint: snapshot_fingerprint.to_string(),
+        profile: GenericProfile {
+            username: report.persona_name.clone(),
+            game_id: report.game.clone(),
+        },
+        club: GenericClub {
+            name: report.club_name.clone(),
+            coins: report.coins,
+        },
+        owned,
+        squad,
+    };
+
+    Ok(ApplyPlan {
+        request,
+        mappings,
+        watermark: report.identity.source_watermark,
+        supported_instances: report.identity.import_wire_ids.len(),
+        deferred_instances: report.deferred_instances(),
+        source_fingerprint: snapshot_fingerprint.to_string(),
+    })
+}
+
+// -------------------------------------------------------------- side effects
+
+/// Where the apply reads/writes the two stores and Core.
+pub struct ApplyPaths {
+    /// The `openfut-core` binary to invoke for the generic import transaction.
+    pub core_bin: PathBuf,
+    /// `DATABASE_URL` passed to that Core invocation (the target profile DB).
+    pub core_db_url: String,
+    /// The emitted production content pack (`OPENFUT_CONTENT_PACKS`).
+    pub content_pack: PathBuf,
+    /// `DATA_DIR` for the Core invocation (base catalog directory).
+    pub data_dir: String,
+    /// The identity store JSON file.
+    pub identity_store: PathBuf,
+    /// Where to write the generic request JSON handed to Core.
+    pub request_out: PathBuf,
+    /// Where to write the completion record.
+    pub completion_out: PathBuf,
+}
+
+#[derive(Debug, Serialize)]
+pub struct ApplyOutcome {
+    pub core_outcome: String,
+    pub mappings_installed: usize,
+    pub watermark: i64,
+    pub supported_instances: usize,
+    pub deferred_instances: usize,
+    pub staging: bool,
+    pub source_fingerprint: String,
+}
+
+/// The staging/production gate. Deferred players are absent from an import; that
+/// is acceptable ONLY for a staged run, and ONLY with the explicit opt-in — never
+/// a silent default. Production requires zero deferred instances.
+pub fn gate_staging(plan: &ApplyPlan, allow_deferred_staging: bool) -> Result<bool> {
+    if plan.deferred_instances == 0 {
+        return Ok(false); // production-complete
+    }
+    if !allow_deferred_staging {
+        bail!(
+            "{} player instance(s) are DEFERRED (unsupported/conflicted) and would be ABSENT from \
+             this import. Production import requires zero deferred instances. To import anyway for \
+             a STAGED test, pass --allow-deferred-players-for-staging (this is never the default).",
+            plan.deferred_instances
+        );
+    }
+    eprintln!(
+        "\n================= STAGING IMPORT (INCOMPLETE) =================\n\
+         {} player instance(s) are DEFERRED and WILL BE ABSENT from this import.\n\
+         This profile is NOT production-complete. Resolve the deferred definitions\n\
+         before a production cutover.\n\
+         ==============================================================\n",
+        plan.deferred_instances
+    );
+    Ok(true)
+}
+
+/// Local mirror of Core's pre-transaction preflight, run BEFORE touching either
+/// store so a doomed import never seeds identity. `content_card_ids` is the set
+/// of CardDefinitionIds in the emitted production pack.
+pub fn local_core_preflight(plan: &ApplyPlan, content_card_ids: &BTreeSet<String>) -> Result<()> {
+    let mut owned_ids: BTreeSet<&str> = BTreeSet::new();
+    for o in &plan.request.owned {
+        if !content_card_ids.contains(&o.card_id) {
+            bail!(
+                "local preflight: owned card references CardDefinitionId {} absent from the \
+                 production content pack",
+                o.card_id
+            );
+        }
+        if !owned_ids.insert(o.owned_item_id.as_str()) {
+            bail!("local preflight: duplicate OwnedItemId {}", o.owned_item_id);
+        }
+    }
+    if let Some(sq) = &plan.request.squad {
+        for slot in &sq.slots {
+            if !owned_ids.contains(slot.owned_item_id.as_str()) {
+                bail!(
+                    "local preflight: squad slot OwnedItemId {} not in imported ownership set",
+                    slot.owned_item_id
+                );
+            }
+        }
+    }
+    Ok(())
+}
+
+/// Dry-check that installing the identity mappings would not conflict with any
+/// mapping already present (a re-run's identical mappings are fine). No writes.
+pub fn identity_dry_preflight(store: &JsonIdentityStore, plan: &ApplyPlan) -> Result<()> {
+    let (g, k) = (
+        Fifa17WireItemIdPolicy::GAME,
+        Fifa17WireItemIdPolicy::OWNED_ITEM_KIND,
+    );
+    let mut conflicts = Vec::new();
+    for m in &plan.mappings {
+        if let Some(existing) = store.external_for(g, k, &m.core_id)? {
+            if existing != m.wire_id {
+                conflicts.push(format!(
+                    "core {} already maps to wire {existing} (want {})",
+                    m.core_id, m.wire_id
+                ));
+            }
+        }
+        if let Some(existing) = store.core_for(g, k, m.wire_id)? {
+            if existing != m.core_id {
+                conflicts.push(format!(
+                    "wire {} already maps to core {existing} (want {})",
+                    m.wire_id, m.core_id
+                ));
+            }
+        }
+    }
+    if !conflicts.is_empty() {
+        bail!(
+            "identity dry-preflight found {} conflict(s): {}",
+            conflicts.len(),
+            conflicts.join("; ")
+        );
+    }
+    Ok(())
+}
+
+/// Idempotently install every preserved wire-id mapping + the allocation
+/// watermark. Re-running re-inserts identical mappings (no-op) and re-sets the
+/// same watermark — safe after a mid-apply crash.
+pub fn seed_identity(store: &JsonIdentityStore, plan: &ApplyPlan) -> Result<()> {
+    let (g, k) = (
+        Fifa17WireItemIdPolicy::GAME,
+        Fifa17WireItemIdPolicy::OWNED_ITEM_KIND,
+    );
+    for m in &plan.mappings {
+        store
+            .insert_existing_mapping(g, k, &m.core_id, m.wire_id)
+            .with_context(|| format!("install mapping {} -> {}", m.core_id, m.wire_id))?;
+    }
+    store
+        .set_watermark(g, k, plan.watermark)
+        .context("set allocation watermark")?;
+    Ok(())
+}
+
+/// After a successful Core import, re-open the identity store and assert every
+/// preserved wire id resolves exactly, and the watermark is set. Core's own
+/// return (`imported`/`already_imported`) is the Core-side proof.
+pub fn post_validate_identity(store: &JsonIdentityStore, plan: &ApplyPlan) -> Result<()> {
+    let (g, k) = (
+        Fifa17WireItemIdPolicy::GAME,
+        Fifa17WireItemIdPolicy::OWNED_ITEM_KIND,
+    );
+    for m in &plan.mappings {
+        let got = store.external_for(g, k, &m.core_id)?;
+        if got != Some(m.wire_id) {
+            bail!(
+                "post-validate: core {} resolves to {:?}, expected wire {}",
+                m.core_id,
+                got,
+                m.wire_id
+            );
+        }
+    }
+    match store.watermark_for(g, k) {
+        Some(w) if w == plan.watermark => Ok(()),
+        other => bail!(
+            "post-validate: watermark is {:?}, expected {}",
+            other,
+            plan.watermark
+        ),
+    }
+}
+
+/// Load the CardDefinitionIds from an emitted production content pack.
+pub fn content_card_ids(pack: &Path) -> Result<BTreeSet<String>> {
+    let raw =
+        std::fs::read(pack).with_context(|| format!("read content pack {}", pack.display()))?;
+    let defs: Vec<serde_json::Value> =
+        serde_json::from_slice(&raw).context("parse content pack as CardDefinition[]")?;
+    Ok(defs
+        .iter()
+        .filter_map(|d| d.get("id").and_then(|v| v.as_str()).map(String::from))
+        .collect())
+}
+
+/// Full recoverable apply across both stores. See module docs for the protocol.
+pub fn apply(
+    plan: &ApplyPlan,
+    paths: &ApplyPaths,
+    allow_deferred_staging: bool,
+) -> Result<ApplyOutcome> {
+    // 1. staging/production gate (explicit, never silent).
+    let staging = gate_staging(plan, allow_deferred_staging)?;
+
+    // 2. local Core-preflight mirror BEFORE any store write.
+    let card_ids = content_card_ids(&paths.content_pack)?;
+    local_core_preflight(plan, &card_ids)?;
+
+    // 3. identity dry-preflight, then idempotent seed + watermark.
+    let store = JsonIdentityStore::open(&paths.identity_store)
+        .with_context(|| format!("open identity store {}", paths.identity_store.display()))?;
+    identity_dry_preflight(&store, plan)?;
+    seed_identity(&store, plan)?;
+
+    // 4. one generic Core import transaction (Core owns atomicity).
+    let req_json = serde_json::to_vec_pretty(&plan.request)?;
+    std::fs::write(&paths.request_out, &req_json)
+        .with_context(|| format!("write request {}", paths.request_out.display()))?;
+    let out = Command::new(&paths.core_bin)
+        .arg("import")
+        .arg(&paths.request_out)
+        .env("DATABASE_URL", &paths.core_db_url)
+        .env("DATA_DIR", &paths.data_dir)
+        .env("OPENFUT_CONTENT_PACKS", paths.content_pack.as_os_str())
+        .output()
+        .with_context(|| format!("spawn core import: {}", paths.core_bin.display()))?;
+    if !out.status.success() {
+        bail!(
+            "core import failed ({}): {}\n(identity already seeded idempotently; re-run the same \
+             manifest to converge — no cleanup needed)",
+            out.status,
+            String::from_utf8_lossy(&out.stderr)
+        );
+    }
+    let stdout = String::from_utf8_lossy(&out.stdout);
+    let core_result: serde_json::Value =
+        serde_json::from_str(stdout.trim()).context("parse core import outcome JSON")?;
+    let core_outcome = core_result
+        .get("outcome")
+        .and_then(|v| v.as_str())
+        .unwrap_or("unknown")
+        .to_string();
+
+    // 5. cross-store post-validation.
+    post_validate_identity(&store, plan)?;
+
+    // 6. completion record.
+    let outcome = ApplyOutcome {
+        core_outcome,
+        mappings_installed: plan.mappings.len(),
+        watermark: plan.watermark,
+        supported_instances: plan.supported_instances,
+        deferred_instances: plan.deferred_instances,
+        staging,
+        source_fingerprint: plan.source_fingerprint.clone(),
+    };
+    let rec = serde_json::to_vec_pretty(&outcome)?;
+    std::fs::write(&paths.completion_out, &rec)
+        .with_context(|| format!("write completion {}", paths.completion_out.display()))?;
+    Ok(outcome)
+}