A finance director at an insurer asks the assistant for loss ratio by product line. One version writes SQL against the warehouse and returns a number at once. The other routes the question through a semantic layer and returns an error, because nobody modelled product line on loss ratio. The semantic layer vs text-to-SQL question is which of those two answers belongs in front of an auditor.
Text-to-SQL turns a question into a query against the schema and runs it. A semantic layer routes the question through definitions written in advance and refuses anything outside them. The first answers everything, some of it wrongly. The second answers what was modelled and says so when it cannot. The refusal is what a semantic layer sells. This article is about what it is worth.
A refusal is worth exactly as much as the definition behind it. A semantic layer that declines a question outside its scope protects the board deck. It protects nothing if the definition it answered with was never agreed, and "in scope" is only as wide as the concepts someone modelled.
The difference is what sits between the question and the warehouse. In our post What Is a Semantic Layer? we set out the four things a layer holds: concepts, metrics, mappings, and entitlements. Text-to-SQL holds a schema and a model's guess about what it means.
| Question | Text-to-SQL | Semantic layer |
|---|---|---|
| What does the model receive? | The schema, or as much of it as fits in context | Named metrics, dimensions, and the joins between them |
| What happens outside the modelled scope? | A query is written anyway | The request is refused |
| How does a wrong answer look? | A plausible number | An error message |
| Who can check it? | Whoever can read the generated SQL | Anyone, against the shared definition |
| What does it cost up front? | Nothing beyond the schema | The modelling |
The measured evidence comes from dbt Labs, which sells a semantic layer: "Semantic Layer vs. Text-to-SQL: 2026 Benchmark Update" (Jason Ganz and Benoit Perigaud, dbt Developer Blog, 2026-04-07). The test is the ACME Insurance benchmark by Juan Sequeda and colleagues at data.world. In their words, "11 questions, each run 20 times, across multiple LLMs" (dbt Developer Blog). The page states its own caveat: "we loaded the entire schema as context, which isn't practical for larger datasets."
Our post What Is a Semantic Layer? quotes the in-scope and out-of-scope pairs for Sonnet 4.6. Here is the rest.
| Run | Text-to-SQL | dbt Semantic Layer |
|---|---|---|
| 2023, all questions | 32.7% | 60.5% |
| 2026, all questions, either model | 64.5% | 72.7% |
| 2026, GPT-5.3 Codex, in scope | 51.2% | 100.0% |
| 2026, GPT-5.3 Codex, out of scope | 100.0% | 0.0% |
Text-to-SQL "nearly doubled, from 32.7% to 64.5% on the full question set". The layer gained less, because its ceiling was set by scope. "The trade-off is coverage: the Semantic Layer can only answer questions that fall within the scope of what's been modeled" (dbt Developer Blog). Out of scope, one model answered every question correctly through text-to-SQL and none through the layer.
dbt's sentence on the difference is the one every page on this query quotes: "With text-to-SQL, failure looks like a plausible but incorrect answer. With the Semantic Layer, failure looks like an error message" (dbt Developer Blog). The error message has a value and a price.
A catalog vendor's page on the self-serve decision (published 2026-08-19, updated 2026-09-18) lands in the same place. It calls a semantic layer "governed and accurate on the questions it was built to answer, capped at those questions". Its failure-mode row reads "Silent gap; question falls outside the model, no answer". Both pages treat the refusal as the end of the analysis. This one starts there.
A refusal is a claim with two halves: this question is outside the definitions I hold, and the definitions I hold are right. The benchmark tests the first half. Each answer is scored against a fixed key. Nothing in the method tests whether the layer's definition of "customer" was the one the business agreed, who agreed it, or whether it has expired.
| Assumption | What the benchmark checks | What an enterprise has to check |
|---|---|---|
| The definitions inside the scope are correct | Against a fixed answer key | Against a named owner's approval, with a date on it |
| The scope covers the questions that matter | Eleven questions chosen in advance | Whatever the board, the regulator, and the agent ask next |
| The join paths are the right ones | The modelled joins returned the key's number | The modelled joins are the only ones the business permits |
| The refusal reaches someone who can act | An error message in a test harness | An owner, a route to widen the scope, and a record of the decision |
Our post Ground Truth: SME-Approved vs Inferred Metadata draws the line between a definition a person approved and one a system inferred. A refusal from an inferred definition is a guess declining to guess further. Who gets to approve one when the consumer is a model is the subject of our post What Is Data Governance for AI?.
On dbt's page the scope was widened by a model. "We prompted an LLM to create as few dbt models as possible to answer all 11 questions via the Semantic Layer. It created 3 new models, joining a few tables together and creating the corresponding semantic models, without us writing any code" (dbt Developer Blog). Accuracy rose to 98.2% for Sonnet 4.6 and 100.0% for GPT-5.3 Codex. The page describes what the LLM created and what the scores became. Any review of the three models sits outside the write-up.
That is the whole question for an enterprise. A layer's scope grows when someone adds a model. If that someone is an LLM and nobody confirms the join it chose, the scope was widened by inference. Every answer inside the new scope rests on a guess. Inferred context is a guess with good grammar, and a confident refusal from a guessed scope is the same guess in a uniform.
| Step | Generated and applied | Proposed and confirmed |
|---|---|---|
| A question falls outside scope | The agent refuses | The agent refuses |
| Someone adds the missing model | An LLM writes the join and the semantic model | AI proposes the concept, the join, and the mapping, with a score |
| The addition becomes authoritative | On merge | When the named owner approves it, with a version and a date |
| The next refusal means | Outside what a model wrote | Outside what the business agreed |
The catalog vendor's page says the right thing here and stops: "Governance questions need an owner before they need a tool." Its own table answers "Who owns 'the truth'" with a team name. An owner is a person. dbt's benchmark describes its layer as "a structured ontology (metrics, dimensions, entities, and the relationships between them)". We would add one item to that list: an owner per definition. Our post Semantic Layer vs Ontology: Which One Says What a Customer Is covers holding a concept that way.
dbt's conclusion is "both, for different things" (dbt Developer Blog). The semantic layer takes anything that reaches a board, an auditor, or a KPI. Text-to-SQL takes exploration, with "as much schema context as possible". Sound advice for a metrics layer. For an agent the better arrangement is one layer, with the order fixed.
| Pattern | What the model gets | What it can still get wrong |
|---|---|---|
| Raw text-to-SQL | The schema | The concept, the join, and the formula |
| Through a metrics layer | Metrics, dimensions, and declared joins | The concept. A customer is a join key, so which records count is inferred |
| Through a concept model | What a customer is, the relationships it may traverse, then the metric logic | Only what the owner got wrong, and that has a name on it |
In the third pattern the agent resolves what a term means before it composes a query. Then it generates, inside the join paths the model permits. The class of failure where a model invents a plausible join is unavailable, because the join was never offered. Our agent-ready semantic layer page calls this constrained query construction. It is the refusal rebuilt as a boundary the agent cannot see past. Our post A Semantic Layer for AI Agents: What the Agent Needs Before It Asks for Revenue lists what the agent needs, in order. Our post Semantic Layer and MCP: What the Agent Actually Receives shows what travels with each answer.
TQ Data Foundation is the layer that makes the refusal worth having. Every metric carries its business definition, its owner, its calculation, and its mapping to physical columns, on one governed asset. Definitions come before generation: the agent resolves what a term means before composing a query. Agents receive concepts, permitted relationships, and metric logic over API, SPARQL, and MCP. The join paths come from the ontology, so an invented join is unavailable. AI proposes the models and the mappings; your experts confirm them, so the scope widens only when a named person says so. Every definition is validated with SHACL, the W3C Shapes Constraint Language, which TopQuadrant helped author, and the standards are open. We maintain 300 reusable standards-based models. Your catalog stays, and it syncs. Each answer keeps a record: question, resolution path, versions, and result.
A refusal from a metrics layer tells you where the modelling stopped. A refusal from TQ Data Foundation tells you where the agreed meaning stops, and who to ask to extend it. The first is an error message. The second is governance.