Private AI: the four ways to run it, and an API that keeps no prompts

What is private AI?

Private AI is AI where the person or company using it decides who sees the prompts, what gets stored and for how long. The enterprise pages on Google’s first page for this query on October 27, 2026 (VMware, Cloudera, Teradata, HPE) define it as models, data and compute kept inside one organisation’s control. The consumer result at the top, a chat app, defines it as conversations nobody else keeps. Both are right. They differ on where the model runs.

Three questions decide how private any setup is:

• Who sees the text? Whatever runs the model has to read the prompt to answer it. That is your own machine, your company’s cluster or someone else’s.
• What is written down? Prompt and reply bodies, usage records, account details, logs, and for how long.
• Who controls the weights? A closed model can only ever run where its owner runs it. An open-weight model can move to your hardware the day you need it to.

Why it matters

Prompts carry the material people are careful with everywhere else: contract drafts, source code, customer records, health and legal questions. A stored prompt can leak in a breach, be read by staff, or be produced under a legal order. A prompt that was never stored cannot.

For a developer there is a second reason. An app that sends user text to a model inherits that service’s retention as its own. If your privacy policy says you keep nothing, the model API behind it has to keep nothing too, and you need it in writing.

Regulated work adds a third: HIPAA, GDPR and similar rules ask where the data went and who processed it. This page makes no compliance claim for any option. It lays out the facts each option gives you to answer that question.

Four kinds of private AI

Local is the most private and the most limited. An 8B model runs on a recent laptop; the large open models need far more GPU memory than a consumer card has. The Ollama uncensored models guide covers the local route.

Enterprise private cloud is the definition the vendor pages rank with. It fits a company with a security team, a procurement process and a budget for both.

Consumer apps are the quickest start and the hardest to check: you read their policy and trust it. The app side of the question, including a dated comparison of three of them, is private AI chatbot.

A no-retention API sits between local and enterprise. You get large models with no hardware, and the provider states exactly what it keeps. The rest of this page is that option, line by line.

What this API keeps, and what it does not

Quoted from the privacy page, last updated 2026-10-27 and read for this post on October 27, 2026:

Prompt and reply bodies

Not stored

They pass through the gateway and are not written anywhere, unless you turn on replay for a key, which keeps that key’s request bodies for 7 days.

Per request

One usage record

Request id, model, token counts by class, the source of those counts, and the charge.

Fingerprints

SHA-256, never text

Of each request and response, so a signed receipt can show what was billed; and of large system prompts, kept 7 days, to cache prompts many accounts send.

Account

Email, balance, ledger

The sign-in email, the amount paid as checkout reports it, and every balance change. The card number never reaches this site.

Keys

A salted hash

The key is shown once and not stored; its hash, name, status and limits are.

First visit

Referrer and landing page

The site that sent you, the link’s campaign tags and the page you landed on, saved with the account when your first sign-in creates it.

Who sees the text

The model host

Every model is served by third-party hosting providers under contract with the operator; they receive the request and response for the calls they serve.

The last row is the honest limit of every hosted option. The text reaches the machine that runs the model; nothing else can answer it. What the gateway adds is that it keeps no copy on the way through, and that the record it does keep holds counts and hashes you can audit, not words.

The privacy page makes no statement about model training, and this post does not add one. Chat requests are not filtered for topic; the acceptable use policy lists what is never allowed.

Open weights you can take with you

The reason to pick an open-weight build for private work is the exit. Each build below has its own weights published on Hugging Face, so the model you call today is a repository you can download and run on your own server when the work calls for it. Card facts are from the model cards; prices are read from the live catalog as this page renders.

The job: 10,000 documents classified, 2,000 input and 300 output tokens each, at the prices below.

Not every build works this way. DeepSeek V4.1 Flash Abliterated is served here with its refusal removed, and the published weights are the stock base, deepseek-ai/DeepSeek-V4.1-Flash (MIT, 552B backbone, 8B active per token at prefill and 16B at decode). Moving that job in-house means the base model, not this edit.

The daily check at /proof sent its fixed prompt to every listed model on November 9, 2026; 161 of 170 came back stable. That check is how you know the id you called this week is the same model you would download: it flags any build whose answer changed under its name. The proof page has the verdict per model.

How to set up private AI as a developer

1. Sort the data. Anything that must never leave your hardware goes to a local model. Everything else can go to an API whose storage you have read.
2. Pick an open-weight build from the table above or the models page, so the exit stays open.
3. Create a key in the console with a monthly spend limit. Leave replay off; it is off unless you turn it on.
4. Point your code at the base URL. The OpenAI SDK works with the base URL changed. Ollama serves the same chat-completions shape locally (its OpenAI compatibility notes), so moving the job is a one-line change.
5. Keep your own logs clean. The model API keeping no bodies means nothing if your app writes every prompt to its own log.

The commented line is the move. Pull the same weights into your local server, swap the base URL, and the code runs unchanged; the model name becomes whatever your server calls the download.

Common mistakes

• Reading “private” as “nobody sees it”. Every hosted model is read by whatever runs it. The useful question is what is kept afterwards.
• Turning on replay to debug and leaving it on. Replay keeps that key’s request bodies for 7 days. Use a separate key for debugging and revoke it after.
• Pasting secrets. API keys, passwords and other people’s personal data do not belong in a prompt on any service. Strip them before the call.
• Trusting a slogan over a list. “We respect your privacy” is not a retention statement. A field-by-field list of what is stored is.
• Building on a closed model. If the weights are not published, local is never an option later, however sensitive the work becomes.
• Sharing one key across apps. One key per app keeps spend limits and revocation separate, and the usage rows tell you which app sent what.

Questions