A score drift on a chart is a question: what caused it? A score drift on a chart with a vertical deploy marker is already an answer: the prompt version that went live last Tuesday at 14:23. The difference is one vertical dashed line, and that vertical dashed line is the self-learning loop's first step.
What we mark
Every prompt deployment event (activation of a new prompt_version for a tenant) generates a row in the eng_prompt_deploy_markers table: tenant_id, prompt_slug, version_before, version_after, deployed_at, deployed_by, change_summary. change_summary is a one-liner the deploying engineer enters — for example "Added a GDPR disclaimer to the output" or "Reduced hallucination tendency with an 'only answer from the given context' directive".
The marker is NOT the full prompt diff. Another table (eng_prompt_versions) and another UI (PromptDiffViewer) carry that. The marker is the temporal anchor — on charts that is enough for the human eye to link drift to deploy.
The chart overlay
The MiniChart component (shared across every score-trend panel in the cockpit) accepts a markers prop. Each marker is a vertical line at the deployed_at timestamp; hover tooltip shows: "Tenant-X / checkout-summary / v17→v18 / 'GDPR disclaimer added' / [email protected]". Line colour encodes deploy type: green (tenant-level, normal), yellow (platform-level, broad impact), red (revert).
Why it works for the human eye
Score drift sometimes starts seven days after the change — the new behaviour of the deployed prompt only manifests gradually through usage pattern erosion. An engineer watching the metric without knowing what shipped has two choices: search the email archive, or assume. Neither scales. The marker overlay gives the answer right inside the panel. The cognitive cost of linking "cause" to "effect" drops close to zero.
The self-learning loop's first step
Why call this the loop's first step? Because a human can now see whether the new prompt version improved or regressed. Score-drift direction + deploy marker = decision. If the score dropped within 30 minutes of deploy, the next step is revert (which is itself a marker, in red). If it rose, the next step is observation and meta-learning: what did I change in the prompt that improved the 7-dim score?
The next iteration (Phase 6) makes this more automatic: the system itself flags a 30-minute post-deploy score drop. We call that closing the deploy-driven self-learning loop. For the first iteration it is enough that the marker is there and a human is watching.
Implementation note
Marker introduction was part of Phase 5δ (BE PR #887 + FE PR #559). Storage was simple (one table); UI was less so (the Recharts Brush + ReferenceLine combo blew the stack at a 90-day window with 200+ markers). Fix: cluster markers via a bucket aggregator when density exceeds one marker per pixel. The aggregate marker's tooltip lists every deploy in the cluster.
A last-week example
On the nortinia-site tenant, cart-recommendation's prompt moved to v22 at Tuesday 11:00. The semantic_accuracy score fell from 4.1 to 3.7 by Wednesday afternoon. The drill-down took 12 minutes to show that the new prompt added a VAT-calculation directive that confused the recommendation copy. Revert at Tuesday 18:30 (red marker). Score climbed back to 4.1 by Thursday morning. Without the marker overlay this debug would have taken 3-5 days; with markers it took three chart views.