docs: add analytics validation runbook

2026-06-08 19:09:22 -07:00
parent b3c4d08dfc
commit 159774f811
4 changed files with 73 additions and 10 deletions
@@ -0,0 +1,67 @@
+# Analytics Validation Runbook
+
+Ferrous Solitaire currently has two analytics paths:
+
+- Native desktop/Android gameplay events use `solitaire_engine::AnalyticsPlugin`
+  and `solitaire_data::MatomoClient`.
+- Hosted web pages include Matomo page-view snippets in
+  `solitaire_server/web/*.html`.
+
+The Bevy `/play` WASM canvas does not emit the native gameplay events because
+`AnalyticsPlugin` is intentionally gated out on `wasm32`; it depends on the
+native Tokio/reqwest stack.
+
+## Native Matomo Validation
+
+Use this when a deployed Matomo instance and a native build are available.
+
+1. Configure `settings.json` with a Matomo URL and site ID:
+
+   ```json
+   {
+     "analytics_enabled": true,
+     "matomo_url": "https://analytics.example.com",
+     "matomo_site_id": 1
+   }
+   ```
+
+2. Launch the native app and open Settings.
+3. Confirm the Privacy section appears and "Share usage data" is `ON`.
+4. Start a new confirmed game.
+5. Win or forfeit the game.
+6. Unlock an achievement if practical, or use an existing achievement path that
+   is easy to trigger in a test profile.
+7. Wait at least 60 seconds, or close after the win/forfeit path has fired its
+   immediate flush.
+8. In Matomo, confirm the following custom events arrived:
+
+   | Category | Action | Name |
+   | --- | --- | --- |
+   | `Game` | `Start` | `classic`, `zen`, `challenge`, `time_attack`, or `difficulty` |
+   | `Game` | `Won` | empty |
+   | `Game` | `Forfeit` | empty |
+   | `Achievement` | `Unlocked` | achievement id |
+
+## Web/WASM Decision
+
+Keep the current split unless the project explicitly needs in-canvas gameplay
+events for `/play`.
+
+Current behavior:
+
+- `/`, `/play-classic`, `/account`, `/leaderboard`, and `/replays` emit Matomo
+  page views through the hosted HTML snippets.
+- `/play` hosts the Bevy canvas but does not emit gameplay events from the
+  engine.
+- The browser Content-Security-Policy already allows the deployed Matomo host
+  for scripts, images, and connections.
+
+If gameplay events are needed on `/play`, add a small `wasm32`-only analytics
+bridge instead of trying to compile the native plugin:
+
+- keep the same event contract as native (`Game / Start`, `Game / Won`,
+  `Game / Forfeit`, `Achievement / Unlocked`);
+- read `Settings::analytics_enabled`, `matomo_url`, and `matomo_site_id`;
+- send through browser APIs or the existing `_paq` queue;
+- keep the Settings opt-in behavior identical to native;
+- add Playwright coverage that stubs Matomo and verifies emitted payloads.