A dashboard is not a report you commission once a quarter — it is the instrument panel that tells you, in near real time, whether the version of your organisation that AI assistants keep repeating still matches reality. Most teams that come to us already have the audit habit. What they are missing is the mechanism that turns isolated findings into a trend worth watching.
Why a dashboard, not a one-off audit
A single audit is a photograph, and photographs age. Query ten models today and you might get a fair, well-sourced picture of your organisation. Six weeks later, a stale press mention, a scraped forum thread, or a competitor's press release has quietly entered the retrieval mix, and nobody notices until a client repeats the error back to you in a meeting.
Nothing malicious needs to happen for this drift to occur. Model versions change, retrieval indexes refresh, and the web content models draw on keeps shifting under everyone's feet. A dashboard exists to catch that drift while it is still small — before it becomes the line a journalist, an investor or a regulator quotes back at you as fact.
The core KPIs
A dashboard earns its keep by tracking a short list of indicators well, not a long list poorly. Six metrics do most of the work, and each answers a distinct question a one-off audit cannot:
- Presence rate — the share of relevant prompts where your organisation surfaces without being named directly, a proxy for how "top of mind" you are inside a model's answers.
- Accuracy rate — the share of factual claims about you that check out against a primary source, scored claim by claim rather than answer by answer.
- Sentiment lean — whether the unprompted tone around your name skews neutral, favourable or unfavourable, independent of whether the underlying facts are correct.
- Source diversity — whether answers draw on your own official content, or lean on unverified secondary sites, forums and aggregators.
- Volatility — how much a given answer shifts week to week with no underlying change in the facts, a proxy for source instability.
- Correction latency — once an error is flagged and a sourced fix is published, how many measurement cycles pass before it stops appearing in fresh answers.
Anatomy of a single metric
Take accuracy rate as a worked example, because it is the one boards ask about first. It is not a single yes/no judgment on a model's answer — that would hide too much useful detail. Instead, each answer is broken into atomic claims: a headcount figure, a founding date, a certification, a leadership title, a description of a product line. Each claim is checked independently against a dated, primary source and scored on a simple scale — verified, partially verified, unverifiable, or contradicted.
The percentage you see on the dashboard is the aggregate of those claim-level scores across your full query set, not an average of subjective impressions about whether an answer "felt" fair. That distinction is what separates a KPI you can defend in a board meeting from a vibe you cannot. Our methodology documents the scoring rules in full, because a metric nobody can audit is not really a metric — it is an opinion with a number attached to it.
Don't know what AI says about you today?
A SetRecord audit queries supported AI models about your organisation and measures the gap with verifiable facts — within five business days.
Cadence and alert thresholds
A weekly refresh is the right baseline for most organisations — frequent enough to catch drift while it is still small, infrequent enough to stay affordable to run and to review. Sensitive windows change that math: a fundraising round, an executive transition, a regulatory filing or a product recall all justify daily checks for as long as the exposure lasts.
Thresholds are what turn a dashboard from a chart you glance at into a system that actually taps you on the shoulder. A workable starting set: flag any accuracy-rate drop of more than ten points week over week, any new claim appearing in two or more models simultaneously, and any volatility spike on a query that had previously been stable for a month or more. None of these thresholds are universal constants — calibrate them against your own baseline noise for a few cycles before you let them trigger real alerts.
Building it in practice
The mechanics are straightforward even when the discipline to sustain them is not.
1. Fix the query set
Lock in the fifteen to twenty-five prompts that actually matter — not just "who is [company]", but the adjacent questions a buyer, journalist or investor would realistically ask. A dashboard built on a query set that keeps changing measures nothing at all, because there is no stable baseline to compare against.
2. Capture on a schedule
Run the same queries against the same panel of at least five models, on the same cadence, and store the raw answers verbatim, timestamped. Without a stored record you cannot prove what changed between two periods — you can only assert it, which is not the same thing in a dispute.
3. Score against your fact base
Maintain a living reference document of verifiable facts — figures, dates, titles, certifications, product names — and score each new answer's claims against that document, never against last week's answer. A platform built for this automates the comparison at scale; a well-kept spreadsheet can do it for a handful of queries.
4. Visualise and route
Put the six KPIs on one screen, trended over time, with the underlying claim-level evidence one click away for anyone who doubts a number. An alert that cannot be traced back to the specific claim behind it will be ignored the second time it fires — and rightly so.
Reading the dashboard without overreacting
The most common failure mode is not under-reacting — it is over-reacting to a single session's noise. Models are non-deterministic by design; the same prompt run twice, minutes apart, can produce two meaningfully different answers even though nothing in the world has actually changed. Treat any single dip as a data point worth logging, not as a verdict worth escalating.
The rule we recommend to every team we work with: a metric only counts as a real signal once it holds across two consecutive measurement periods, not one. That single rule filters out most of the noise a dashboard generates without slowing down your response to anything that genuinely matters.
From dashboard to action
A dashboard that only displays numbers is an expensive wall decoration. Its value comes from being wired into a workflow: an accuracy dip routes to whoever owns the fact base, a newly detected phantom claim routes to whoever can trace its likely source, and every open item stays tracked until correction latency closes it out for good.
SetRecord builds and runs exactly this instrument for organisations that would rather not maintain it in-house — the same six KPIs, the same claim-level scoring, wired directly into a sourced-correction workflow rather than left to sit in a spreadsheet nobody opens. Ask for a walkthrough against your own query set before committing to anything.