How AI Architects Keep Company Data Out of Model Training

10 July 2026 · Bedrock AI

Company data ends up in model training when a vendor’s default terms permit it and nobody checks. An AI architect keeps it out through three layers: a contractual zero-retention or no-training commitment with every model vendor, a routing architecture that separates sensitive data from any endpoint that lacks that commitment, and a classification system that tells the routing layer what counts as sensitive in the first place. Toggling an account setting is not a control. It is one input to a system that needs all three.

What Actually Puts Company Data Into Model Training?

Data enters training through exactly two doors: the vendor’s default retention policy on API and product traffic, and any fine-tuning job the business itself runs on top of a foundation model.

Most consumer-facing AI products retain prompts and outputs by default for abuse monitoring and product improvement, and that retained data is a candidate for future training runs unless the account sits under a no-training agreement. Enterprise API tiers from the major model providers generally exclude API traffic from training by default, but “generally” is doing a lot of work in that sentence, because the exclusion is a contractual term, not a law of physics. It is negotiated, it is scoped to specific products, and it does not automatically extend to every surface the business touches — a team using the same vendor’s consumer chat interface for a quick task sits under a completely different policy than the same vendor’s enterprise API, even inside one organisation.

The second door is self-inflicted. When a business fine-tunes a model on its own support tickets, contracts, or CRM exports to improve accuracy, that data becomes part of the model’s weights. If the resulting model is ever shared, hosted by a third party, or used across business units without contractual walls, the fine-tuning data has left the building in a form that cannot be deleted or redacted after the fact. This is the risk architects worry about most, because it has no data retention setting to switch off. The data is baked in.

What Is the Difference Between No-Training and Zero Data Retention?

These two commitments get conflated constantly, and the gap between them is where most exposure hides.

No-training means the vendor will not use submitted prompts or outputs to train or fine-tune its models. It says nothing about how long that data is stored or who inside the vendor’s organisation can access it for other purposes, like abuse review or debugging.

Zero data retention (ZDR) means the vendor does not persist inputs or outputs beyond the immediate request. Data flows in, a response is generated, and the record is deleted, typically within seconds. ZDR implies no-training as a side effect, because data that no longer exists cannot be used for anything. But no-training does not imply ZDR: a vendor can commit to never training on your data while still holding thirty days of logs for security review.

An architect’s job is knowing which tier every workflow in the business actually sits on, not which tier the vendor’s marketing page implies.

Why Isn’t a Vendor Setting Enough on Its Own?

A no-training toggle protects exactly the traffic that passes through the account it was set on. It does nothing about the traffic that never should have reached that vendor in the first place.

The failure mode architects see most often is not a misconfigured setting. It is an employee pasting a customer contract into a general-purpose assistant that the business never onboarded, because the enterprise-tier tool was slower or required an extra login step. The no-training agreement the business negotiated is real, and it is irrelevant, because the data went somewhere else entirely. Shadow AI usage — tools adopted informally, outside procurement and IT visibility — is the single largest source of company data reaching untracked training pipelines, and it exists precisely because the sanctioned tools weren’t the path of least resistance.

This is why the control has to sit above the vendor relationship, in the routing architecture, rather than inside any single vendor’s account settings.

How Do Architects Design the Routing Layer?

The pattern that works is a gateway: one internal chokepoint that every application and every employee-facing AI tool must route through, rather than calling model APIs directly.

A model gateway does three things a scattered set of direct API integrations cannot. First, it enforces the no-training and retention policy centrally, so a new application can’t accidentally launch against an unvetted endpoint — the gateway simply won’t route to one. Second, it logs and classifies traffic in one place, which makes an audit answerable in an afternoon instead of a week of chasing down every team’s integration. Third, it gives the architect a single place to insert data controls: masking, redaction, or blocking specific data categories before they ever leave the network, regardless of which model sits behind the gateway this quarter.

Retrieval-augmented generation reinforces the same principle at the data layer. Instead of fine-tuning a model on proprietary records, RAG keeps those records in a secured vector store or database and retrieves only the relevant fragment at query time, injecting it into context rather than into weights. The model never trains on the data, because the data never enters a training run at all — it’s read, used once, and discarded from the model’s perspective. This is why the architect’s default answer to “how do we make the assistant know about our customers” is a retrieval pipeline, not a fine-tuning job, whenever the alternative is viable.

What Does Data Classification Have to Do With It?

Routing rules are only as good as the classification feeding them. Without a schema for what counts as sensitive, the gateway can’t decide what to block, mask, or redirect.

A workable classification scheme for this purpose is short: public (marketing copy, published documentation), internal (process docs, non-sensitive analytics), confidential (contracts, financials, strategy), and regulated (health data, payment data, anything under GDPR special category rules or equivalent). Each tier maps to a routing rule — regulated data might only ever reach a self-hosted model inside the company’s own infrastructure, while public data can go to any vendor with a standard no-training agreement. Architects who skip this step end up with gateways that are technically correct but practically useless, because every request gets treated identically and the business either blocks too much (killing adoption) or blocks too little (recreating the original risk).

What Should Be in the Vendor Contract?

Technical controls only hold if the paper underneath them does. Before any model vendor is approved for use with company data, the contract review should confirm four things: an explicit no-training clause covering the specific product tier in use (not the vendor’s general policy page), a defined data retention period with a stated deletion mechanism, a data processing agreement that names subprocessors and their jurisdictions, and audit rights or attestations — SOC 2, ISO 27001, or equivalent — that let the business verify the commitment rather than take it on faith. Vendors that won’t put a no-training clause in writing, even when their marketing describes exactly that behaviour, should be treated as unwilling to guarantee it.

FAQ

Does using the API instead of the consumer app automatically protect our data? Not automatically. Most enterprise API tiers exclude data from training by default, but the exclusion is a contractual term specific to that product and tier. Confirm it in the agreement rather than assuming it from general vendor reputation.

Is RAG always safer than fine-tuning for using proprietary data? For most business use cases, yes, because RAG keeps sensitive records outside the model’s weights entirely. Fine-tuning is still appropriate when the goal is teaching a model a style, format, or reasoning pattern rather than giving it facts to recall — but proprietary facts belong in retrieval, not training data.

Can we stop employees from pasting sensitive data into unapproved AI tools? Policy alone rarely works. The combination that holds is a sanctioned tool that’s genuinely as fast and convenient as the alternative, a gateway that can detect and block traffic to unapproved endpoints on the corporate network, and a data classification scheme employees actually understand.

How often should the vendor contract review happen? At minimum, every time a vendor changes its terms of service or product tier, and at renewal. AI vendor terms change faster than most other software categories, and a no-training clause negotiated eighteen months ago may not automatically carry over to a new product the vendor has since bundled in.

Does zero data retention cost more than a standard agreement? Usually, yes. ZDR is typically a negotiated enterprise add-on rather than a default on pay-as-you-go plans, and it’s worth reserving for the data categories that genuinely need it — regulated and confidential tiers — rather than applying it uniformly, which mainly adds cost without a corresponding drop in risk elsewhere.

Bedrock AI maps your systems, team and workflows to show where AI actually pays, before you spend a pound building. Book a strategy call.