Your AI Project Does Not Need More Data, It Needs a Schema

Why does "we have no data" usually mean something narrower?

When an operations director tells me they have no data for AI, the sentence is almost never true as stated. It is shorthand for one of three different problems, and only one of them is about missing data.

The first is that nobody has labelled examples of the task: no record of which past invoice was coded correctly, no outcome column on the ticket table. The second is that the process genuinely is not instrumented, which happens on paper, in phone calls, and in a spreadsheet that lives on one person's laptop. The third is that the data exists but nobody has agreed who may read it. The first is a schema problem. The third is a governance decision with a legal owner. Only the second is an absence.

I sat in a meeting where a logistics team described itself as having no data, while four years of scan events, pick times, and exception codes sat in a warehouse management system nobody had ever exported. In that company, "data" had quietly come to mean "dashboard". There was no dashboard, so the conclusion followed. That is a category error with a capital budget attached to it.

Almost every organisation already holds the data its AI project needs; what it lacks is the schema that turns an operational byproduct into a queryable asset, and building that schema is a one-time cost rather than a recurring one.

If that is right, it changes the decision in front of you. Buying a dataset, hiring a data team, or waiting a year for the warehouse project are all responses to the wrong problem.

What is already sitting in the systems you pay for?

Every system below is already licensed, already backed up, and already used by people who care whether its contents are correct. That last property matters more than volume: data that someone's job depends on is data that gets corrected.

System already in place	What it holds that is usable	What is missing before it counts as an asset
ERP or finance system	Every invoice, approval, cost centre, and posting date	Line-level outcome on where the coding was later corrected
Ticketing or ITSM	Ticket text, category, assignee, timestamps, reopen events	Whether the fix held; reopen logic is recorded but rarely modelled
CRM	Account history, deal stages, meeting notes, close reasons	Entity identity when the same account appears under three spellings
Email and document stores	Contracts, amendments, exception approvals, supplier correspondence	Structure: the clause or field you actually need is inside prose
Call recordings and meeting transcripts	How experienced people describe decisions and workarounds	Speaker attribution and a reason code per call outcome
Individual spreadsheets	The reconciliation logic and local rules that make the process work	Ownership, version control, and any route off one laptop

Read the third column again. Not one row says "more data". Every row asks for a decision about identity, outcome, or structure — which is schema work, and it is done once.

What actually separates a byproduct from an asset?

Six properties do the work, and you can audit any candidate dataset against them in an afternoon.

Stable identity: one key per real-world entity, with the messy aliases resolved. Event time in a single timezone, not a mix of local timestamps and export dates. State transitions preserved, because "how it got here" is usually more valuable than "where it ended up". An outcome field, even a coarse one. Provenance, so a reviewer can trace any record back to the system it came from. And an access boundary, decided before the first embedding is written rather than after.

The identity point is where most projects stall, and it is unglamorous. One mid-sized supplier in a set of books I reviewed appeared as eleven vendor strings: three with a legal suffix, two misspellings, one trading name, and five variants with trailing whitespace. Any model that answers "how much did we spend with this supplier" is answering a false question until that is resolved. Resolving it took an afternoon with the accounts payable owner and a rules file, plus a manual list of 40 exceptions. The result is inherited by every downstream question anyone ever asks.

That is the shape of the work. It is domain knowledge applied once, not model capability applied repeatedly.

What does the schema cost, and why is it one-time?

Separate the two cost structures honestly, because they behave differently.

Extraction and embedding are marginal and small. Published API list prices put a competitive small embedding model at roughly $0.02 per million tokens at the time of writing, so three million tokens of historical documents — a realistic corpus for a mid-sized back office — costs about six cents to embed. Object storage for the resulting records runs in the region of $0.02 per gigabyte per month. Anyone quoting a large figure for the compute side is pricing the wrong thing.

The schema is fixed and human. On the last three of these I scoped, the pattern settled between eight and fifteen working days of one engineer plus intermittent time from the process owner who knows the exceptions. At a fully loaded cost of $900 a day for the engineer, that is $7,200 to $13,500, once. Compare that against the alternative most teams reach for first: a data platform programme with a steering committee, quoted in quarters.

Cost line	Behaviour	Typical size for one process
Entity resolution and schema definition	One-time, human	8-15 days, engineer plus process owner
Extraction pipeline build	One-time, engineering	3-5 days, then runs unattended
Historical backfill and embedding	One-time, marginal	Cents to low tens of dollars in API cost
Refresh and monitoring	Recurring, small	Hours per month, grows with schema changes
Re-labelling as the business changes	Recurring, occasional	The honest reason this is not literally free forever

The last row is the tradeoff I will not hide. A schema is one-time in the sense that it does not scale with volume and is not re-purchased per query. It is not one-time in the sense that it never changes. When a new product line, a new entity, or a new approval rule appears, the schema is extended. What you are buying is a stable base that absorbs those changes cheaply, rather than a rebuild. If you want the per-unit arithmetic that a controller will eventually ask for — retries, failed runs, and exception handling priced into one number — it is worked through in a per-token cost model that prices caching and failures into a single figure.

Which data should you deliberately leave out?

Scope discipline is what makes the one-time cost stay one-time. Three exclusions I now propose before writing any extraction code.

Anything whose access boundary cannot be agreed in writing is out of scope for the first build. It can be added later, and adding it later is cheaper than litigating it during a pilot. Personal data that is not needed for the decision should be dropped at extraction rather than masked downstream, because masking that runs after storage is a promise, not a control. And any table whose owner cannot tell you what a row means should not be modelled at all, because a schema built on a field nobody understands produces confident wrong answers at scale, which is the worst possible failure mode for a system a manager might act on.

The uncomfortable one: data that is only useful if it is complete, but is only 60 per cent captured, is usually worse than no data. Detection-and-response on partial coverage is a different system design from reporting on partial coverage.

When is the honest answer that you do need new data?

Three cases, and I say so plainly when I meet them. The process runs on verbal decisions with no system of record, which is common in field service and in some clinical and legal workflows. The outcome you are trying to predict was never written down anywhere, so there is nothing to learn from retrospectively. Or the decision depends on information held by third parties who will not share it, which is a commercial problem before it is a technical one.

In all three, the first deliverable is instrumentation, not a model: capture the decision and its outcome for ninety days, in the smallest format the people doing the work will actually maintain. That gives you a schema and a dataset at the same time, and it costs a fraction of a pilot built on assumptions.

The question to ask before you approve anything

Ask what the schema is, who owns it, and what it will cost to build. If nobody in the room can answer, you are being sold a model on top of data nobody has structured, and the failure will arrive looking like a model failure when it was an identity and outcome problem from the start. The reason this matters commercially is that the schema asset is durable: it outlives whichever model you happen to be using this quarter, and it makes the second workflow in the same function materially cheaper than the first. Budget for the boring part, audit those six properties against one process, and decide with your own numbers rather than a vendor's: most organisations that believe they are data-poor are in fact schema-poor, which is the better problem to have.