What private AI actually means (and what it doesn't)

“Private AI” turns up on just about every vendor slide now, and it means something different on every one of them. Before you sign anything, it pays to work out which version you’re actually being sold, because the gap between them is enormous.

The reason the phrase got so slippery is that it’s doing sales work, not technical work. “Private” is what the buyer wants to hear, so it gets attached to everything from a standard API with a tick-box promise not to train on your data, right through to hardware humming in your own server room. Those two things share a marketing word and almost nothing else. The vendor isn’t necessarily lying to you. They’re just using a word that lets you hear whatever you were hoping for, and the burden of pinning down what it actually covers lands on you.

It matters because the stakes aren’t abstract. If you’re a law firm feeding client matters into a model, an accounting practice summarising financials, a health provider triaging referrals, or any business bound by privacy law, professional privilege or a client NDA, then “where does this data physically go and who can access it” is not a detail. It’s the whole question. Get it wrong and you’re not dealing with a disappointing product; you’re dealing with a breach notification, a professional-conduct issue, or a client finding out their information went somewhere they never agreed to.

Four models, four trade-offs

The Public API model is the one most people meet first. You send your prompts off to a third-party model and get answers back. It’s easy and it’s capable, but your data has left your boundary to get there. That’s fine for non-sensitive work and a real problem for everything else. Worth knowing: the paid business tiers of the big providers usually come with better terms than the free tiers, no training on your data, shorter retention, and that difference alone rules the free tiers out for anything commercial. But even on the best contractual terms, the data still travels, still lands on infrastructure you’ll never see, often offshore, and your protection is a clause, not a control.

Private cloud tenancy is a step up. You get a hosted model with contractual guarantees about how your data is handled: a dedicated tenancy, sometimes a nominated region, promises about isolation and retention. Better, sure, but notice that “private” here is a promise written into a contract, not a wall you can see. You’re trusting the vendor’s implementation of their own promise, and your ability to verify it is roughly zero. For a lot of businesses this is a perfectly reasonable landing point. Just call it what it is: managed trust, not control.

Self-hosted in your own cloud is where it starts to feel private in a way you can verify. The model runs in infrastructure you control, an Australian region of your own cloud account, inside your network boundary, and your data stays inside your environment. You own the configuration and the logs rather than taking someone’s word for them, and when someone asks “can the vendor see our prompts?”, the answer is that there is no vendor in the loop to see anything. The trade-off is that you’ve taken on operational work: someone has to run, patch and monitor the thing, and capable open-weight models, while now solid, still trail the frontier APIs on the hardest tasks. For most business workloads, summarising, extracting, drafting, answering questions over your documents, that gap doesn’t matter.

On-prem or air-gapped is the far end. The model runs on your own hardware, with no internet connection at all if that’s what the work demands. You get maximum control for a higher upfront cost, and for the most sensitive workloads it’s the right answer. The hardware bill is real but smaller than people assume these days; a surprisingly capable model runs on a single well-specced box. What you’re really paying for is the operational discipline around it. This is the model for the work where “we’d rather it be impossible than promised”: defence-adjacent, privileged legal matters, anything where a regulator or client contract says the data doesn’t leave the building.

Matching the model to the work

No business needs one answer across the board, and the expensive mistake is buying the far end for everything or, worse, the near end for everything. The practical move is to sort your workloads by sensitivity. Marketing copy and public-facing drafting can live happily on a public API. Internal documents and operational data might sit in a decent cloud tenancy or self-hosted setup. Client-privileged and regulated material gets the verifiable options only. That triage takes an afternoon and instantly clarifies every vendor conversation, because now you’re asking “which of our three tiers does this tool qualify for?” instead of “is it private?”

The triage also future-proofs the decisions. Models, prices and providers will keep moving, but your data’s sensitivity tiers won’t, and a tool change becomes a simple re-check against the same three questions rather than a fresh philosophical debate. Businesses that skip the sorting renegotiate everything from scratch every time a vendor knocks.

It also pairs with a short written policy on what goes where, which we’ve covered in the minimum viable AI policy. The deployment model and the policy are two halves of one control: the policy says what’s allowed, the deployment makes the important cases enforceable.

Costs move along the same line, and it’s worth knowing the shape before a vendor frames it for you. The public API end is almost free to start and priced per use; the private tenancy adds a platform premium; self-hosting trades usage fees for infrastructure and someone’s time; and on-prem trades again for hardware plus operations. None of these is universally cheaper. Heavy steady workloads eventually favour owning; light or spiky ones favour renting; and the crossover point is a calculation your usage numbers can settle in an afternoon. What matters is that the cost decision and the control decision are made together, deliberately, rather than the control level being whatever fell out of the pricing page.

Ask the question that cuts through

Forget the label on the slide. There’s one question that sorts the real thing from the marketing: can our data leave our control, and if so, under exactly what conditions? Make the vendor answer that in plain terms. A vague answer is a vague guarantee.

Then follow it with the supporting three, and write the answers down. Where is the model physically hosted, which country and whose account? What’s retained after each request, for how long, and who can read it, including for “service improvement” and “abuse monitoring”, the two clauses where data access usually hides? And what happens when we leave: what gets deleted, what’s already been absorbed, and can we take our fine-tuning or our document indexes with us? A vendor selling something honestly answers all four without flinching. A vendor who answers with the word “enterprise-grade” has answered you too, just not the way they intended.

One last habit worth forming: get the answers into the contract, not just the meeting notes. A verbal assurance about data handling from a sales engineer binds nobody, and eighteen months later, when the product has been acquired and the terms have been “updated”, the paragraph you insisted on is the only version of “private” that still exists.

Private AI comes down to where your data lives and who can reach it, full stop. If you’re weighing up options, or you’ve been handed a “private AI” proposal and want it translated into which of the four models it actually is, send us the workload and the sensitivity level and we’ll give you a straight answer, including when the boring public API on a business tier is honestly all you need.