The Model Is an Engine, Not the Database

Every enterprise already knows how to run a shared engine that sits between many people and valuable data. It is the database, and the working habits are old. Nobody connects without being identified. Results are limited to what the asker may see. Privileged queries leave a trail. Versions are pinned and retired on a schedule. The administrator looks after the engine but does not own what is in it. Each habit was learned the hard way.

Then a language model arrived, and in a surprising number of estates it was handed the database’s job instead of the engine’s.

What it looks like when the model is the database

A fictional freight company builds an HR assistant. To make it useful, the team fine-tunes a model on years of case notes, connects its retrieval to the HR file share using a service account, and writes a line in the prompt: only share information the person asking is entitled to see. A team lead then asks, has anyone on my team raised a grievance?

Illustrative scenario: a fictional composite written to make the point concrete. It does not describe a real organisation, and its figures are not reported outcomes.

What the assistant says now depends on two things that have nothing to do with the team lead: what the weights remember, and what the service account can read. Each failure has its own cause, and none of them is the model being stupid.

Knowledge in the weights cannot be filtered for the person asking. A model that has absorbed the case notes will draw on them for anyone. There is also no dependable, verifiable way that I know of to remove one person’s record from the weights short of retraining, which is a poor answer to an erasure request.

Retrieval under the agent’s own identity makes every user the most privileged reader. In database terms this is an application that connects as the superuser and then filters results in its own code. It works until the day the filter has a gap, and then it fails for everyone at once.

A rule in the prompt is a request. It is more text in a context that also holds whatever the model has just read. In June 2025 researchers at Aim Security disclosed a flaw in Microsoft 365 Copilot, tracked as CVE-2025-32711 and known as EchoLeak, in which a crafted email could lead the assistant to disclose data from its context without the user clicking anything. Microsoft scored it 9.3 out of 10; the US National Vulnerability Database’s own score was 7.5. Microsoft’s classifiers for this class of attack were reportedly bypassed, Microsoft fixed it on its own servers, and no malicious exploitation was reported. The lesson is not about one vendor. Defences that live in the same context as the attack are defences the attack can talk to.

The database habits, mapped

Each habit has a counterpart for a model, and each counterpart has a test you can run this week.

Two further rows are organisational. The administrator becomes the model operator, who runs the serving stack and the gateway but does not own the knowledge. And the record of what was shown should be kept by someone other than the operator.

Where the analogy breaks, and why that gives the rule

A database keeps data in rows you can address and enforces permissions inside itself. A model keeps nothing you can address, and enforces nothing. So permissions cannot live in the model, and whatever needs enforcing has to be enforced around it.

In practice that means a short, ordered path. The request arrives carrying the end user and the task. An authorisation system, not the model, decides what that person may read for that task. Only those documents are retrieved, with their provenance. The model receives that slice and nothing else: no handle to the store, no token of its own with wider reach. Its answer comes back across the boundary as untrusted, and is checked for shape and for anything the person should not hold before it is released. If it proposes an action, the action still crosses a deterministic shell before it becomes real.

Two things follow. The policy decision no longer depends on which model is in use, so changing the model becomes a test rather than a migration. And a wrong or manipulated model produces a refusal instead of a leak, because the worst it can do is misuse a slice it was already allowed to hold. Owning the weights does not change any of this, which is the argument of Owning the weights is not owning the system.

What a slice does not fix

Be clear about the limits, because this pattern is easy to oversell.

A slice limits what an injected instruction can reach. It does not stop the injection. A permitted document can still carry hostile text, which is why the output check and the trust boundaries around the engine still matter.

Permissions mirrored into a search index are as stale as the mirror. A revocation that takes an hour to arrive is, for that hour, a leak. Treat the delay as you would treat replica lag, and say how long it is allowed to be.

Source permissions can themselves be too generous. If years of broad sharing exist in the file share, an engine that faithfully respects them will faithfully surface them. Filtering by the end user is necessary, not sufficient, and tidying the source is separate work.

And fine-tuning is not banned. Tuning a model for tone, format or an organisation’s vocabulary is a reasonable use. The rule is about knowledge that has an owner and an audience. Keep that out of the weights.

I have not costed this. The expensive part is usually not the code but the inventory of where the data actually sits, and I would not plan from a number in an article.

What to do on Monday

1. Find the identity. For one assistant you run, establish whose identity its retrieval uses. If the answer is a service account, you have found the superuser connection.
2. List what is inside. Write down everything held in any fine-tune or provider-side memory, and who is entitled to each item. Anything with an audience that is not everyone is a candidate to move out.
3. Run the two-person test. Take one real question, ask it as two people with different access, and compare the documents retrieved. If the sets are the same, find out why.

Key takeaways

• A model is a shared engine in front of valuable data, so it should be run with the discipline databases already have: attributed calls, filtering by the end user, audit, pinned versions and a named operator.
• Unlike a database, a model holds nothing addressable and enforces nothing, so knowledge, memory and permission state must live outside it.
• Entitlements are decided before context is built, as the end user, and the model sees only the resulting slice.
• A rule in the prompt is a request, and a defence inside the context an attacker writes to can be talked past.
• A slice limits what an attack can reach. It does not prevent injection, mirror staleness or over-generous sources.

Sources and further reading

• The Hacker News, “Zero-Click AI Vulnerability Exposes Microsoft 365 Copilot Data Without User Interaction”, June 2025. https://thehackernews.com/2025/06/zero-click-ai-vulnerability-exposes.html
• Hack The Box, “CVE-2025-32711 (EchoLeak): Copilot vulnerability”, 2025. https://www.hackthebox.com/blog/cve-2025-32711-echoleak-copilot-vulnerability
• National Vulnerability Database, CVE-2025-32711 record, published 11 June 2025. https://nvd.nist.gov/vuln/detail/CVE-2025-32711
• Microsoft Security Response Center, CVE-2025-32711 advisory. https://msrc.microsoft.com/update-guide/vulnerability/CVE-2025-32711
• Saltzer, J. H. and Schroeder, M. D., “The Protection of Information in Computer Systems”, Proceedings of the IEEE 63(9), 1975. https://doi.org/10.1109/PROC.1975.9939

Where the book takes this further

The Contextual Secure Agentic Enterprise, a forthcoming book in the series, works through this boundary in detail: the model contract, a small adapter that holds the line in code, how entitlement-aware retrieval is built and where it fails, and how a model swap is tested. See the books page for its status.

This article is part of CAE Field Notes, a series accompanying the book, The Contextual Agentic Enterprise.