AI & context
7 min

Knowledge Graph ROI: What to Measure

Key takeaways

  • The savings land in other teams' budgets under other headings, which is why they go uncounted.
  • Baseline disagreement cost, time to answer, local copies and audit effort before choosing a platform.
  • Operational signals move within a quarter. Financial effects arrive in year two and arrive lumpy.
  • There is no credible published ROI benchmark for this. A percentage without its methodology is worth nothing.
In this article
Share

Why is knowledge graph ROI so hard to state?

Because knowledge graph ROI lands in other people's budgets, and nobody is counting it there.

A governed knowledge graph reduces the cost of disagreement. The savings show up in four places. A reconciliation meeting that stops happening. A search project that is no longer needed. A migration that does not become a re-authoring exercise. An audit response assembled in hours instead of weeks. Each of those is booked to a different team under a different heading, and none of them appear on the line item that funded the graph.

So the honest position is that this is measurable and the measurement has to be designed before the work starts. Retrofitting it afterwards means reconstructing a baseline nobody recorded.

What should you baseline before you start?

Four things, all of them cheap to capture and impossible to recover later.

BaselineHow to capture itWhy it matters
Disagreement costCount meetings in the last quarter that existed to reconcile a definitionThe clearest before-and-after, and finance understands meeting time
Time to answerPick five definitional questions, time how long an answer takes todayMeasurable, repeatable, and obviously improved or not
Local copiesCount systems holding their own version of a contested termEach one is a future divergence with a cost attached
Audit effortAsk how long the last definitional audit response tookInfrequent, expensive, and very visible when it improves

Capture these in a week, before any platform is chosen. They cost almost nothing and they are the difference between a defensible claim in year two and an anecdote.

What should you track once it is running?

Operational signals first, because they move early and predict whether the financial case will ever materialise.

Signals that move in the first six months

  • Resolution rate. What share of governed terms are resolved against the authority by consuming systems. Low means local copies are still winning.
  • Time to decide. How long a proposed term waits for its approver. Rising latency is the earliest warning of all.
  • Unresolved terms. What systems needed and could not find. This should grow, because growth means broader use.
  • Repeat disagreements. How often a governed term comes back into dispute. A governed term should go quiet.

The last one is the closest thing to a direct benefit measure. A term that stops generating arguments has stopped costing what it used to cost, and that is attributable in a way most of the financial story is not.

How do you turn that into a financial case?

By attaching costs to the baselines you captured, and by being conservative enough that nobody has to defend the arithmetic.

Meeting time is the easiest and the most credible. Reconciliation meetings have attendees, durations and salaries, and the before-and-after is countable. Avoided projects are larger and softer; a search procurement that gets cancelled because the structure was the real problem is a real saving and it needs a sponsor willing to say so.

Resist building a model that extrapolates a per-term saving across the whole vocabulary. It produces a large number that nobody believes, and it invites an argument about the multiplier instead of about the work.

What should you not claim?

Three things, and the first one is where most vendor material in this category loses its audience.

An industry benchmark figure. There is no credible published ROI number for governed knowledge graphs, and a percentage presented without its methodology is worth nothing. If somebody quotes one, ask for the sample, the period and the counterfactual.

Attribution you cannot defend. AI answer quality improves for several reasons at once. Claiming the governance caused all of it will be challenged by the first person who changed a prompt that quarter.

Savings that were never real. Avoided headcount that was never budgeted is not a saving. Be precise about what would otherwise have been spent.

We have one number from our own work and we state it as that: in our engagements, roughly 80 to 85% of the accuracy gain comes from getting ground truth right. It is our field measurement and not research. How to measure whether your AI answers from governed data covers the instrumentation behind it.

Who is the ROI case actually for?

Three audiences, and writing one case for all three is how it fails.

What each one needs

  • The sponsor needs the operational signals, because those tell them within a quarter whether the thing is working and whether to keep funding it.
  • Finance needs countable avoided cost with a defensible method. Meeting time works. Extrapolated per-term savings do not survive a second conversation.
  • The board or the executive committee needs one sentence about risk, usually audit exposure, and no model at all.

The common mistake is building the finance version and presenting it to all three. The sponsor cannot act on it, and the executive discounts it because every programme arrives with a spreadsheet.

Build the operational dashboard first. It is the only one that changes a decision inside the first year, and it is the evidence base the other two are eventually assembled from.

When should you expect to see anything?

Operational signals within a quarter. Financial effects in the second year, and lumpy when they arrive.

The operational numbers move quickly because they measure behaviour. Resolution rate and time to decide respond within weeks of the first term going live.

The financial effects arrive late and unevenly, because the big ones are avoided events. A migration that stays a migration, an audit answered from the data, a project cancelled because the structure was fixed instead. None of those happen on a schedule, and a business case promising smooth quarterly returns will be wrong in a way that damages the next one.

So what should I take away?

Baseline four things in the week before you start. Track operational signals from month one. Build the financial case from countable meeting time and be conservative about everything else.

TopQuadrant will not hand you an ROI percentage, because we do not have one we could defend. What we can tell you is which numbers moved for the enterprises we work with, and that the operational signals move first. Our knowledge graph platform is instrumented so resolution rate and unresolved terms are available from day one. Capture the baseline first. It is the part that cannot be recovered. How to sponsor a knowledge graph initiative covers the rest of the setup.

Give your AI the context it's been missing

See how the TQ Data Foundation turns your enterprise knowledge into trusted, Al-ready context.