If both numbers come from real data, why don’t they match?
Two numbers built from the same real data disagree for three mechanical reasons: they cover different shares of the business, they use different definitions of the same word, and the connection behind one of them can be frozen at a version that no longer matches the live system.
The instinct is to assume one of them is wrong, go hunting for the bug, and stop once you find it. Most of the time there is no bug to find. The two numbers were built to answer two slightly different questions, and they disagree because of that, not because either system made a mistake. Treat a mismatch like this as a symptom of coverage and definition, not as evidence that something broke.
This matters because the fix people usually reach for (rebuilding the connector or picking whichever number “feels” more official) doesn’t actually address either mechanism.
Why does an assistant see less of the business than a dashboard does?
A dashboard is often built directly on one system’s own data, a CRM’s native pipeline report, for instance. An assistant answering the same question out loud might be doing something structurally different: joining that CRM to an ad platform or a field-service system over a join (matching records in one system to records in another) that isn’t complete. When only part of the underlying records can actually be traced end to end, the assistant’s answer reflects that part. It will not match a dashboard number built inside one tool where nothing needed to be joined at all.
On one account we worked on, the ad platform reported 303 conversions and $38,227 of tracked value against $37,947 of spend in a single month. In that same month, only 46 of those 303 could actually be traced through to completed work. Neither number is fabricated, and nobody entered anything wrong.
The ad platform counted a conversion the instant the phone rang, with no mechanism to know whether a real job ever came out the other end of that call. A number resting on a partial join, presented with the same confident tone as a complete one, is worse than having no number at all. Nothing in how it’s displayed signals what share of the business it’s actually looking at.
This isn’t two CRMs disagreeing about the same field. That’s a sync problem with a known fix (see how to get one honest number when your CRMs disagree), where two systems of record exist and need reconciling. This is different: there’s often only one true system of record, and the gap is in how much of it any single answer actually reaches before it gets phrased as a sentence.
Why do two systems disagree about what counts as revenue?
Even inside one business, a word like “revenue” doesn’t have one meaning. One system might count it the moment a job gets booked. Another counts it only once the job is invoiced.
On the account above, roughly a third of completed jobs were never invoiced at the moment they closed out. Pull revenue from the invoicing system for a given month and it will read lower than the same period pulled from the booking system, and both numbers are correct, each against its own definition. Neither system is lying to you.
An assistant answering from whichever system it happens to be pointed at inherits that system’s definition silently, with total confidence and zero disclosure. It doesn’t know a second, equally valid definition of the same word exists three systems over, because nobody told it there was a disagreement to flag. This is a quieter failure than a partial join, because it produces answers that are individually defensible and collectively contradictory. There’s no error message anywhere to alert you that a conflict even exists.
Can a connection go stale without anyone noticing?
Yes. Once an admin approves a custom connector, ChatGPT works from a frozen snapshot of that connector’s tools and the inputs they accept, and changes the developer makes later are not applied until an admin reviews and publishes an update. A custom connector is a bridge you set up so the assistant can plug into another piece of software, like a CRM or ad platform, using MCP, the standard most of these connections run on.
OpenAI’s help center states plainly that “admins are not proactively notified when an app needs review” (verified August 28, 2026). The mechanics differ by plan: on Business, an app cannot be updated after publishing at all, so any change means recreating and republishing it. On Enterprise and Edu, an admin can hit Refresh to pull the latest set of actions, but newly added actions arrive switched off by default. On Pro, custom connectors run in developer mode with read and fetch permissions, with no admin publish cycle involved.
If the live server has since moved on from that frozen snapshot, tool calls can start erroring outright, and those error messages don’t include a prompt telling anyone to update. (Custom connections with read and fetch permissions are available on Pro as well as the workspace plans; they work on the web only, not the mobile app, and only apply to remote servers, so the exposure here is specific rather than universal, but it’s real wherever it applies.)
A frozen snapshot does not corrupt the number directly. It means the assistant may be calling an older version of the tool (a newly added optional filter the snapshot never learned to send, or a field that has since been renamed or dropped; the renamed-field case errors rather than answers), so the query it actually runs can be scoped differently from the question you asked. When the mismatch is total you get an error. When it is partial you get an answer scoped to something you did not ask for.
How do you know which number to trust?
Don’t guess. Ask what share of the volume each source can actually see, which definition of the metric it is using, and when the connection was last checked against the live system.
- What share of the underlying volume can this source actually see? This means what fraction of calls, leads, or jobs it traces end to end, rather than assumes.
- Which definition of the metric is it using (booked or invoiced, opened or closed), and does that match the definition the person reading the number has in their head?
- When was the connection behind this number last verified against the live system? Not against whatever snapshot got frozen the day someone clicked approve.
For a fuller version of this check, run through the AI readiness assessment checklist before you connect anything else.
A number that can’t answer those three questions should be treated as provisional, no matter which tool produced it and no matter how confidently it was phrased. The fix isn’t switching tools, and it isn’t picking a permanent winner between the dashboard and the assistant. It’s stating coverage and definition next to every number, as a standing habit rather than a one-time cleanup you do after the board asks an awkward question.
What does an Answer Audit check before anything gets connected?
This exact mismatch, between what a number implies by how it’s phrased and what it actually covers underneath, is what an Answer Audit is built to catch before it ever reaches a dashboard or a chat window. We check every real question a business wants to ask against what the underlying data can actually see, and we state the coverage next to each answer rather than letting confident phrasing stand in for completeness. One week, from $1,900, and the document is yours to keep whether or not any further work follows.
For the limits a vendor’s own free connector hits before any of this even becomes relevant, see where a vendor’s own connector stops. Or skip ahead to the Answer Audit directly.