From 5b83d9f510a1b4a410af578c9a42a3fc46910767 Mon Sep 17 00:00:00 2001 From: FuelBoard Contributor Date: Sat, 15 Aug 2026 09:24:33 +0100 Subject: [PATCH] Backlog: P0 mirror push side DONE (repo live, deploy key, launchd daily job); size math corrected to full-dump 2.58MB/day @180d retention (Option 2) --- BACKLOG.md | 45 +++++++++++++++++++++++++-------------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/BACKLOG.md b/BACKLOG.md index 3bc0e7d..1c73f12 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -11,31 +11,36 @@ Status: TODO / IN PROGRESS / DONE / BLOCKED. ## P0 — Next build -- [ ] **GitHub price-history mirror (start now — warm the data for Trends)** — the - relay does the GOV.UK heavy lifting (OAuth token stays in relay `.env`) and - pushes a daily snapshot to a PUBLIC repo; the app reads from GitHub, so this - doubles as the production data source (App Review can't reach the LAN relay) AND - the history archive the Trends graph (P1) reads. Shape: relay writes - `latest.json` (tiny pointer: date, station_count, data_updated, available range) - + `history/YYYY-MM-DD.json` (full ~200 KB dump) and pushes once/day; the app - fetches `raw.githubusercontent.com/aptonline/fuelboard-data/main/latest.json`, - then only the missing day files for starred (station, fuel) pairs (deterministic - URLs — 404 = no data that day, leave a graph gap, never error). LAN relay stays - the fallback source. Telemetry implication: raw GitHub fetches are invisible to - repo analytics (page views/clones only) — keep the widget-diag beacon firing +- [ ] **GitHub price-history mirror (warm the data for Trends)** — the relay does + the GOV.UK heavy lifting (OAuth token stays in relay `.env`) and pushes a daily + snapshot to a PUBLIC repo; the app reads from GitHub, so this doubles as the + production data source (App Review can't reach the LAN relay) AND the history + archive the Trends graph (P1) reads. Shape: relay writes `latest.json` (tiny + pointer: date, station_count, data_updated, available range) + + `history/YYYY-MM-DD.json` (FULL dump — 2.58 MB/day, decision Option 2) and + pushes once/day; the app fetches + `raw.githubusercontent.com/aptonline/fuelboard-data/main/latest.json`, then only + the missing day files for starred (station, fuel) pairs (deterministic URLs — + 404 = no data that day, leave a graph gap, never error). LAN relay stays the + fallback source. Telemetry implication: raw GitHub fetches are invisible to repo + analytics (page views/clones only) — keep the widget-diag beacon firing regardless of data source (build/stale-build checks survive) and add an opportunistic fire-and-forget X-Client app beacon on fetch (LAN = attribution + cadence; off-LAN skip is itself a reachability datum). `/stats` keeps the intrusion canary and gains a new signal: app-hit spikes = GitHub path failing → - fallback active. Setup: create the PUBLIC repo `aptonline/fuelboard-data` - (gh installed); GitHub PAT (aptonline) lives in relay `.env` only — never in the - repo or IPA. App Review: public repo required (reviewer fetches the data + fallback active. App Review: public repo required (reviewer fetches the data themselves); HTTPS/ATS fine; plain JSON is not "remote code" (2.5.2-safe); bundle - a small fallback dump so the app never demos empty. Growth ~200 KB/day (~73 MB/yr - — no pruning for years); git history doubles as an audit trail. Optional - first-weeks tweak: hourly pushes to build history faster, then settle to daily. - Accumulating TODAY means the Trends graph launches with weeks of real history - instead of an empty chart. + a small fallback dump so the app never demos empty. Retention: 180 days (~470 MB, + pruning enforced by the push script — full dump, NOT the earlier ~200 KB + price-only estimate). *PUSH SIDE DONE 2026-08-15 — repo live + (https://github.com/aptonline/fuelboard-data); auth = SSH deploy key + ~/.ssh/fuelboard_deploy via alias github-fuelboard-data (no token anywhere); + clone ~/workspace/fuelboard-data; launchd com.apt.fuelboard-mirror daily 10:00 + (script ~/.hermes/scripts/mirror_push.py, log + ~/Library/Logs/fuelboard-mirror.log); manual trigger `launchctl start + com.apt.fuelboard-mirror`; first snapshot 2026-08-15 landed + verified raw 200. + REMAINING: app-side read (provider chain GitHub → relay → bundled dump) + the + telemetry beacon.* ## P1 — Soon