Msurface: Real-time Reporting Platform Development

# Msurface — Developer Handoff Brief

*Field-to-office self-reporting platform for microsurfacing road construction.*

## 1. What we are building, and why

Work on a road site stops the moment a resource runs out — **Material, Manpower, or Machine (the "3 Ms")** — or any other hindrance hits. Today that bad news travels slowly, and a crew loses a shift before the office knows. Msurface makes it travel in real time, captures it cleanly, and resolves or escalates it automatically.

Two field-facing capture surfaces feed one shared office layer:

1. **A Telegram bot** field staff use all day to log problems and progress as they happen — and it tries to *resolve or escalate* in the moment, not just record.
2. **A web/mobile end-of-day (EOD) form** for the consolidated daily record: loads laid, materials in/out, closing stock, manpower, machinery.

Both write into the **same database with the same semantics**, so the office sees one coherent picture, not two parallel reports. On top sit role-based dashboards for GM, Coordinator, and Director, covering operations, approvals, masters/HR, and external interactions (vendors/purchase).

**Three guiding principles for the whole build:**
1. **Prefer dynamic data over hard-coded standards.** If a value lives in the DB (tolerances, mix targets, lead times, rates), read it from there — and give a UI to update it at the right user level.
2. **Things must tie together.** The same fact captured twice (bot + web, or two pages) reconciles to one record, not two independent impacts.
3. **Build for compliance through easy UI.** Correct reporting should be the path of least resistance.

## 2. Microsurfacing context (enough for a developer)

A paver lays a thin cold bituminous slurry over a road at a designed mix ratio. Key model concepts:
- **Loads** — work is batched into loads (always "loads," never "sections"); a load can split along its run.
- **Chainage** — position in metres from a survey origin, written `18+745`. Above ~500 m, warn-and-confirm rather than hard-reject.
- **JMF (Job Mix Formula)** — the approved recipe (aggregate/bitumen/cement/water/additives per m²) per site/treatment. The dynamic standard driving expected consumption and stock cover — a versioned DB record, never hard-coded.
- **Treatments** — Type 2/3/5, each with its own billing basis, rate, and JMF.

## 3. Architecture

- **Backend:** Python/Flask on Railway — one service that is the REST API, the Telegram bot host, and the scheduler for time-based nudges/escalations.
- **Database:** Supabase (PostgreSQL) via PostgREST through a custom `/api/data` endpoint. Flask proxies everything; **no client talks to Supabase directly, no keys in the frontend.**
- **Frontend:** self-contained static HTML on Netlify (one file per page), inlined CSS + a single `<script>` block; shared `tokens.css` for design tokens.
- **Bots:** **M. Rosef Singh** (`@MRosefSingh_bot`, field ops) and **Panam Mistry-ji** (maintenance only).
- **AI:** Claude API powers bot reasoning, proxied via `/api/ai`.

The bot's `save_to_supabase` family is the **canonical save logic**. Any web save must mirror the bot exactly — fields, defaults, derived values. Read the bot first.

## 4. Roles & access

- **Site Monitor** (bot + mobile EOD web): daily capture; sees challan numbers only, never invoices.
- **Coordinator** (bot + web): validation, corrections, 10-day plans, stock ledger, vendor/master proposals, disputes; owns invoices.
- **GM** (web): cross-site oversight, approvals (vendors, manpower, JMF, calibrations, POs), masters/HR.
- **Director**: uses the GM workspace in v1; countersigns JMF. Dedicated view deferred.
- **Proxy** (Telegram only): **blocked from web by design.**
- **Admin**: system admin; only role allowed to `DELETE` via `/api/data`.

**Auth:** 8-digit login = first 4 of Telegram ID + 4-digit PIN (SHA-256). First-timers get an inline PIN-confirm step (last-4 preview) before `set_pin`. Sessions are **Supabase-backed** (survive Railway restarts), 7-day expiry, daily cleanup. Token in `localStorage` as `msurface_token`, sent via `X-Session-Token` header (**not** `Authorization: Bearer`). Every page: auth check at boot, validate via `/api/me` before redirect, per-page role gate.

## 5. Vocabulary that must be correct

- **Chainage** stored as **integer metres** (`18745`), displayed `KM.MMM` / `18+745`.
- **TAT step labels:** `CO / SI / LS / LE / SO / CI`.
- **All times IST** — every display and schedule. Showing UTC is a bug. Use `todayIST()` and `combineTimeToISO(hhmm)` (builds a `+05:30` timestamptz).
- **Stock rollup** (bot writes, web matches): `opening = prior closing`; in/out = sum of today's IO rows for that material; `consumed = opening + inward − outward − closing`.
- **Vendor resolution:** usable only if `active=true AND approval_status='APPROVED'`. Unmatched name is **blocked** with "ask Coordinator to add to master" — never silently dropped.
- **Discretion is a hard requirement:** field users never see upward-reporting or escalation-chain language; the bot addresses the person it's speaking to, not whoever messaged last.
## 6. Current state

Schema is deployed and verified (materials and calibration types seeded, RLS in place). **Mockups are design truth and LOCKED** — "wiring" means keep the visual structure, swap static content for dynamic containers (`id="…"`), add one self-contained `<script>` with the standard `msApi`/`msToast`/`msSignOut` helpers, render live data. Never reflow layouts.

- **Site Monitor (mobile, 380px) — wired:** login, EOD home (5-tile checklist), DPR loads, inward/outward (photo intent), closing stock (JMF-driven), manpower roll-call, machinery + fuel events, vendor picker.
- **GM workspace — wired:** Overview, Masters, Approvals, site-detail tabs (Today, Stock, Resources, Diary).
- **Coordinator — mockups present, wiring pending:** 10-day plan, stock ledger, stock summary/reorder, IO validation, DPR row-states.
- **Shared:** `index.html`, `app.js`, `tokens.css`, `_redirects`, a pending placeholder.
- **Bots + full schema dumps** (columns/keys/constraints/indexes/triggers/functions/RLS) **+ paver manuals** in repo.

## 7. Scope to build

- **A. Finish Coordinator surface (wire existing mockups):** 10-day plan editor; stock ledger + summary/reorder; IO validation incl. dispute reconciliation (`vws_qty_kg ≠ aws_qty_kg`); pendency list; correction trail (`corrected_by_coordinator=true`).
- **B. Complete GM Overview wiring** incl. `events_view` signal feed and `approvals_queue_v` queue.
- **C. Build the Admin panel** (only role with DELETE).
- **D. Cross-cutting (missing everywhere):** photo/file upload (Supabase Storage; mockups already reserve slots); polling/auto-refresh (today a reload is needed to see bot writes); offline storage/sync queue (field disconnect currently = failed save); notification badges.
- **E. Reporting (later, once 2–4 weeks of data):** search, decision-history, exports, Reports tab (TAT trends, calibration drift, productivity, cross-site pace, vendor analytics, JMF cost).
- **F. Bot/scheduler:** wire real acknowledgement tracking for adaptive nudge thresholds (gated until ~10k observations).
- **G. Deferred:** dedicated Director dashboard (when multiple GMs exist).

## 8. Non-negotiable principles

- **Bot/web write parity** — every web save mirrors the bot's save for that record type. Bot is original; web mirrors. Don't change bot semantics for web's convenience.
- **Mockups locked** — wire only, never reflow.
- **One self-contained file per page**, helpers replicated, **zero credentials in HTML**, all DB/AI proxied through Railway.
- **Upsert by natural key for capture** (`on_conflict=<natural_key>`); **PATCH-by-id for in-place edits** (never a new row). `on_conflict` needs a real UNIQUE constraint or errors `42P10`. Unique: `ten_day_plan`, `daily_stock`, `attendance`. `fuel_log` has **no** unique constraint (multiple fuelings/day are valid).
- **Single bulk fetch on load** (`Promise.all`), render from in-memory state.
- **IST everywhere.**
- **Toasts must reflect reality** — if no notification is sent, the toast must not claim one. Fabricated feedback is a correctness bug.
- **Trace bugs to root cause** — no symptom-patching.

**Schema realities:** no `dpr_subloads` table (each sub-load is a `dpr` row keyed `(load_number, section_number)`; UI groups by `(load_number, asset_id)`). No generic `approvals` table — `approvals_queue_v` is a UNION view over 5 sources; query sources directly. `/api/data` DELETE is Admin-only (JMF drafts are commit-once; Site Monitor manpower adds need Coordinator mediation). Triggers can't see Flask session (`SET LOCAL app.current_user_id` is invisible via PostgREST) — identity-dependent logic runs in Flask. Don't let the server broaden a specific `site_id=eq.N` filter to `in.(allowed)` — it silently breaks single-site views. Audit columns/statuses differ per approvable table (e.g. JMF has no REJECTED) — check dumps.

**Quality gate before delivering any HTML:** (1) extract inline `<script>` blocks, run Node `--check`; (2) tag-balance count for `html/head/body/script/style/div/button`. Every time.

## 9. Locked decisions (don't reopen for v1)

Director uses GM workspace. JMF v1: GM submits + approves with Director countersign (Coordinator JMF → v2). Tolerances use schema defaults. Edits PATCH in place. Mockups locked. Fuel Y→N blocked if events exist. Machinery start-hours are user-logged each shift (equipment runs overnight). Fitters = Mechanic (regex `operator|driver|mechanic`). Site Monitors see challan only; invoices are Coordinator's.

## 10. First steps for the incoming developer

(1) Read the field bot end to end; map its save functions to record types. (2) Read the schema dumps for tables you'll touch. (3) Wire one Coordinator page (mirroring its bot save) before the cross-cutting work (photos, polling, offline) that touches every page.