You cannot measure it by sampling answers for correctness. A correct answer tells you nothing about where it came from, and an AI system that guesses well will pass that test until the day it does not.
What you can measure is provenance: for a given answer, which definitions and which facts did the system resolve, and were those the agreed ones. That is a different question from accuracy and it is the one that predicts whether accuracy holds.
This matters more than it sounds. In our own work with enterprise customers, roughly 80 to 85% of the accuracy gain comes from getting ground truth right, ahead of anything done to the model or the retrieval layer. If that is where the gain lives, it is also where the measurement should point.
Four signals, and only the first requires any new plumbing.
| Signal | What it tells you | How to get it |
|---|---|---|
| Resolution rate | What share of governed terms in a query were resolved against the authority | Log every call the system makes to the context layer |
| Local-copy count | How many consuming systems keep their own definitions | Count the systems that never call |
| Version pinning | Whether an answer can name which agreed version it used | Return a version with every resolution |
| Unresolved terms | Terms the system needed and could not resolve | Log the misses, which is where the next governance work is |
The fourth is the most useful and the most often discarded. A log of what the system asked for and could not find is a prioritised list of what to govern next, generated by actual demand instead of by a modelling exercise.
Because it measures the output of a process whose inputs you are not watching.
A system can produce a correct answer from an ungoverned source, a stale definition or a lucky retrieval. It passes. Three months later the underlying document changes, the same question produces a different answer, and nothing in the sampling regime predicted it.
Worse, sampling creates a false sense of safety that scales badly. A 95% pass rate on a hundred sampled answers tells you very little about the ten thousand you did not sample. The failures cluster on exactly the contested terms a sample is unlikely to hit.
Sampling and provenance measure different things, and a programme that runs only one of them has a blind spot it cannot see from the inside.
Pick ten questions the business already argues about, and answer them three ways.
Ask the AI system. Ask the governed source, if one exists. Ask two people in different teams. Record all three. The gap between the second and third answers tells you whether the governance is real. The gap between the first and second tells you whether the system is using it.
Most organizations running this for the first time find that the three answers disagree, and that the system is confident in all cases. That is the baseline, and it is more useful than any accuracy percentage, because it names the specific terms where the work needs to happen.
Resolution rate is the one worth a target, and the honest answer is that it depends on how much has been governed.
Measuring resolution against the full universe of terms a system encounters produces a demoralising number and a useless one, because most of those terms were never contested. Measure against the governed set. Of the terms that have an agreed definition, what share did the system resolve instead of inferring? That number should approach total, and anything well below it means consuming systems are holding local copies.
Track unresolved terms as a separate count that is allowed to grow, because growth there means the system is encountering more of the business, which is progress. Keeping context from going stale covers the freshness side, and data governance for AI the wider programme.
Smaller than a measurement programme. One system, one log line, ten questions.
None of this needs a platform, a dashboard or a budget line. It needs somebody to add a log line and read the output for a week, and it will tell you more about whether your AI is grounded than any accuracy benchmark.
The common objection is that the log is incomplete because only some systems call the authority. That is the finding, not an obstacle to it.
Who actually needs this measurement?Three audiences, and they want different cuts of it.
The third is the one that forces the discipline. If a regulator asks why a system answered as it did in March, you need the definition that applied in March, attached to that answer. Nothing reconstructs it afterwards.
Measure provenance as well as correctness. Log every resolution, count the systems keeping local copies, pin a version to every answer, and treat unresolved terms as the backlog instead of a failure. The instrumentation sits on top of the work described in how to use a knowledge graph for AI context, and the commercial counterpart is knowledge graph ROI and what to measure.
TopQuadrant builds for this. Our knowledge graph platform resolves governed terms at query time and returns the version and the approval with them, so an answer can say which agreed definition it used. The 80 to 85% figure above is our own field measurement. It is no kind of industry statistic. Treat it as a reason to instrument your own and find out.