Why Your AI Gives Wrong Answers About Your Own Company
- Guessing and inferring is what the system does natively. It is a feature everywhere except on your own facts.
- Context fails four ways: too little, too much, contaminated, contradictory. They look identical from outside and need different fixes.
- Most of the questions it gets wrong, nobody in your organization could answer either. That is why better prompting never closes the gap.
- If two people in your company would answer a question differently, the AI has nothing to be right about.
What does an AI system actually do when you ask it something?
It guesses, and that is not a criticism.
An AI system works in natural language, infers what you probably meant, and produces the most plausible continuation given what it has in front of it. That behaviour is the entire reason it is useful. It handles a question phrased six different ways, fills small gaps sensibly, and does not need you to know the schema before you can ask.
On general knowledge this works because the material is abundant, redundant and argued out in public over decades. Ask about a tax treaty and the answer has been written down a thousand times by people who checked each other.
Ask how many customers you have and none of that applies.
Why does it get your own facts wrong?
Because the context it needs is not there in the form it needs, and there are four distinct ways that happens. They look identical from the outside, and they need different fixes.
| Failure | What happened | What it looks like in your business |
|---|---|---|
| Context gap | Too little. What was needed was never supplied, so the system produced something plausible instead. | Asked which accounts are at renewal risk, it answers from support tickets because nobody connected it to the entitlement data. |
| Context overload | Too much. The relevant fact is present and buried among material that merely resembles it. | Ninety pages of pricing policy retrieved for one discount threshold, and the threshold that applies is on page sixty. |
| Context poisoning | Bad input. Something wrong, superseded or half-migrated got in, and nothing marks it as untrustworthy. | A deprecated product hierarchy still in the warehouse, quietly shaping every revenue rollup that touches it. |
| Context clashes | Two truths. Both versions are real, both are defensible, and nothing decides between them. | Finance counts customers by billing entity and sales counts by logo, so any answer is wrong for one of them. |
Poisoning is the one that produces the most memorable failures, because the system is behaving perfectly. Someone demonstrated this on Hacker News under the title your RAG still thinks the old CFO is the CFO. The retrieval worked. It found a true-sounding sentence in a real document that had simply stopped being true.
The uncomfortable part
Here is what usually goes unsaid in these conversations. For most of the questions your AI is getting wrong, nobody in your organization could answer them either.
Not because people are not clever or do not care. Because the answer was never settled. What counts as an active customer lives partly in a finance policy, partly in a sales convention, partly in the head of someone who has run the renewals process for nine years and knows which exceptions are real. The definition through to the linkages through to the edge cases, spread across a dozen people, none of whom has ever been in a room together to reconcile it.
That is not a data problem. It is tribal knowledge, and every organization past a certain size runs on it without noticing, because humans route around it constantly and invisibly. Somebody always knows who to ask.
An AI system cannot ask anyone. It gets one shot at the material in front of it.
So are these questions answerable at all?
Yes, and this is the part worth holding onto.
The answers do exist. They are distributed across the company's collective knowledge rather than missing from it: in the policies, the models, the conventions, the corrections people make in meetings. Nothing has to be invented. It has to be gathered, reconciled and written down somewhere a system can reach.
The reason this feels impossible is that aggregating it has always been more expensive than working around it. A person who needs the answer asks two colleagues and moves on. That workaround does not scale to a system answering a few thousand questions a day, and it never had to until now.
What would actually fix it?
A company brain. Something that holds what the organization has collectively settled, and serves it to every system that asks.
Concretely, four jobs:
| What it has to do | Why the wrong answers stop |
|---|---|
| Reconcile definitions so they align | Context clashes end, because one version has been chosen and the others are marked as superseded rather than left in circulation. |
| Unify the view of the data landscape | Context gaps close, because what exists and how it connects is known rather than rediscovered per project. |
| Make retrieval machine-readable | Overload stops, because a system resolves the fact it needs instead of being handed everything that resembles it. |
| Carry and share what has been decided | Poisoning gets caught, because there is a current version to compare against and an owner who notices when it changes. |
None of that is a model upgrade, and none of it is prompt engineering. It is the unglamorous work of deciding what your organization actually means by the terms it runs on, and then making those decisions reachable.
Which is why the fix starts somewhere smaller than it sounds. Pick one definition two teams argue about. Get the people who hold the judgement in a room and settle it. Record it where a system can resolve it. The set of terms that needs this is far shorter than the list of things your AI gets wrong, because most of those wrong answers trace back to the same handful of unsettled facts.
Context management is the discipline for doing that deliberately, runtime context covers why an estimate can be excellent and still not settle anything, and keeping context from going stale is the method for the poisoning case specifically.
