From 159774f811ec4e9316a9ebab95c8fa33b45a27ef Mon Sep 17 00:00:00 2001
From: funman300 <funman300@gmail.com>
Date: Mon, 8 Jun 2026 19:09:22 -0700
Subject: [PATCH] docs: add analytics validation runbook

---
 CHANGELOG.md                            |  2 +
 SESSION_HANDOFF.md                      | 11 ++--
 docs/analytics-validation.md            | 67 +++++++++++++++++++++++++
 solitaire_engine/src/settings_plugin.rs |  3 +-
 4 files changed, 73 insertions(+), 10 deletions(-)
 create mode 100644 docs/analytics-validation.md

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 740d0bd..8d462cb 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,6 +8,8 @@ project follows [Semantic Versioning](https://semver.org/).
 
 ### Added
 
+- **Analytics validation runbook.** Documented native Matomo live validation,
+  expected event payloads, and the current web/WASM analytics split.
 - **Browser Bevy canvas route and automation support.** Added the `solitaire_web`
   Bevy WASM build, wired `/play` to the Bevy canvas, added a
   `window.__FERROUS_DEBUG__` bridge, and introduced Playwright coverage for the
diff --git a/SESSION_HANDOFF.md b/SESSION_HANDOFF.md
index 598892b..3fe38fc 100644
--- a/SESSION_HANDOFF.md
+++ b/SESSION_HANDOFF.md
@@ -119,14 +119,9 @@ touch events, so physical-device smoke testing is the only gate.
 
 `Settings` has `analytics_enabled`, `matomo_url`, and `matomo_site_id`; the engine
 consumes them via `AnalyticsPlugin` on non-wasm targets. Remaining work is live
-validation against the deployed Matomo instance:
-- Configure `matomo_url` and opt in through Settings.
-- Play a short session that starts a game, wins or forfeits, and unlocks or
-  verifies an achievement event path if practical.
-- Confirm Matomo receives `Game / Start`, `Game / Won` or `Game / Forfeit`, and
-  any achievement events.
-- Decide whether the web/WASM route should eventually use browser-side tracking,
-  since the native `AnalyticsPlugin` is intentionally gated out on wasm32.
+validation against the deployed Matomo instance. Use
+`docs/analytics-validation.md` for the native validation checklist and the
+current web/WASM decision notes.
 
 ---
 
diff --git a/docs/analytics-validation.md b/docs/analytics-validation.md
new file mode 100644
index 0000000..1f82850
--- /dev/null
+++ b/docs/analytics-validation.md
@@ -0,0 +1,67 @@
+# Analytics Validation Runbook
+
+Ferrous Solitaire currently has two analytics paths:
+
+- Native desktop/Android gameplay events use `solitaire_engine::AnalyticsPlugin`
+  and `solitaire_data::MatomoClient`.
+- Hosted web pages include Matomo page-view snippets in
+  `solitaire_server/web/*.html`.
+
+The Bevy `/play` WASM canvas does not emit the native gameplay events because
+`AnalyticsPlugin` is intentionally gated out on `wasm32`; it depends on the
+native Tokio/reqwest stack.
+
+## Native Matomo Validation
+
+Use this when a deployed Matomo instance and a native build are available.
+
+1. Configure `settings.json` with a Matomo URL and site ID:
+
+   ```json
+   {
+     "analytics_enabled": true,
+     "matomo_url": "https://analytics.example.com",
+     "matomo_site_id": 1
+   }
+   ```
+
+2. Launch the native app and open Settings.
+3. Confirm the Privacy section appears and "Share usage data" is `ON`.
+4. Start a new confirmed game.
+5. Win or forfeit the game.
+6. Unlock an achievement if practical, or use an existing achievement path that
+   is easy to trigger in a test profile.
+7. Wait at least 60 seconds, or close after the win/forfeit path has fired its
+   immediate flush.
+8. In Matomo, confirm the following custom events arrived:
+
+   | Category | Action | Name |
+   | --- | --- | --- |
+   | `Game` | `Start` | `classic`, `zen`, `challenge`, `time_attack`, or `difficulty` |
+   | `Game` | `Won` | empty |
+   | `Game` | `Forfeit` | empty |
+   | `Achievement` | `Unlocked` | achievement id |
+
+## Web/WASM Decision
+
+Keep the current split unless the project explicitly needs in-canvas gameplay
+events for `/play`.
+
+Current behavior:
+
+- `/`, `/play-classic`, `/account`, `/leaderboard`, and `/replays` emit Matomo
+  page views through the hosted HTML snippets.
+- `/play` hosts the Bevy canvas but does not emit gameplay events from the
+  engine.
+- The browser Content-Security-Policy already allows the deployed Matomo host
+  for scripts, images, and connections.
+
+If gameplay events are needed on `/play`, add a small `wasm32`-only analytics
+bridge instead of trying to compile the native plugin:
+
+- keep the same event contract as native (`Game / Start`, `Game / Won`,
+  `Game / Forfeit`, `Achievement / Unlocked`);
+- read `Settings::analytics_enabled`, `matomo_url`, and `matomo_site_id`;
+- send through browser APIs or the existing `_paq` queue;
+- keep the Settings opt-in behavior identical to native;
+- add Playwright coverage that stubs Matomo and verifies emitted payloads.
diff --git a/solitaire_engine/src/settings_plugin.rs b/solitaire_engine/src/settings_plugin.rs
index 3e2453e..09d5fb0 100644
--- a/solitaire_engine/src/settings_plugin.rs
+++ b/solitaire_engine/src/settings_plugin.rs
@@ -252,8 +252,7 @@ enum SettingsButton {
     /// player's last window size always wins.
     ToggleSmartDefaultSize,
     /// Toggle [`Settings::analytics_enabled`]. Only rendered when a
-    /// sync server is configured — there is no server to send to in
-    /// local-only mode.
+    /// Matomo URL is configured.
     ToggleAnalytics,
     /// Scan `user_theme_dir()` for new `.zip` files and import each one.
     #[cfg(not(target_arch = "wasm32"))]