AI Supply-Chain Security: Models, Data, Tools, and Permissions

Your application code can be perfect and your AI system can still be compromised before it ever runs. The model weights, the fine-tuning dataset, the MCP server your agent calls, and the credential that server holds are all part of your attack surface now — and most of them were never reviewed by anyone on your team.

The Trust Boundary Moved, and Most Teams Did Not Move With It

Traditional dependency review rests on a quiet assumption: someone can read the code. You pull a library, you inspect it, you pin a version, you scan it for known vulnerabilities. The review step exists because the artifact is inspectable.

A model weight file is not inspectable. It is a large binary blob of floating-point numbers. There is no line to read, no function to audit, no obvious place where a backdoor would sit. You can run evaluations, probe behavior, and compare outputs against a reference, but you cannot read the model the way you read a dependency. The review step that used to catch problems does not exist in the same form.

Data makes this worse. In a code-only supply chain, the build inputs are source files. In an AI system, the build inputs include training data, fine-tuning data, and grounding data — and dataset version control is far less mature than code version control. A dataset can change without a commit, without a diff, and without anyone noticing. Poisoning is the dramatic version of this risk. Untracked version drift is the operational one: a dataset gets updated, the update is not recorded, and now you cannot reproduce the model you shipped last quarter.

Then there is the lifecycle itself. AI development is often a series of ad hoc incremental steps — a fine-tune here, a prompt tweak there, a swapped embedding model — that never land in a central configuration. The build is not reproducible in the way a compiled binary is reproducible, which means the audit trail you would normally rely on does not fully exist.

The weak mental model is treating a model or dataset like a library dependency. The stronger model is this: the unit of trust is not the artifact. It is the artifact plus everything it can reach at runtime.

That reframe matters because it changes what you measure. Provenance tells you where a component came from. Permissions and isolation determine how much damage it can do when it turns hostile. A model with clean provenance and a credential that can write to your production database is a bigger problem than a model with murky provenance and no network access.

Four Attack Surfaces: Model, Data, Tooling, and Permission

A durable map beats a long risk list. Four surfaces cover most of what actually goes wrong, and each one has a mechanism, a failure mode, and an observable signal.

Model surface

This includes pretrained weights, fine-tunes, and adapters — the small trained layers that modify a base model's behavior. The compounding property is the important part: if a base model is later found to have a vulnerability or a backdoor, every derivative that incorporated it becomes a candidate for investigation. Risk propagates down the lineage, and the further you are from the base, the harder it is to trace. Whether a given derivative is actually exploitable depends on how it uses or transforms the base, so inherited risk is a reason to investigate, not an automatic compromise.

The observable signal is indirect. You are looking for behavioral drift against a reference, unexpected outputs on known inputs, or a derivative model that fails evaluations its base passed.

Data surface

Training, fine-tuning, and grounding data. Poisoning gets the headlines, but untracked dataset changes are the more common operational failure. A dataset gets updated, the update is not versioned, and now you cannot reproduce the model you shipped last quarter.

The observable signal is a change in model behavior that correlates with a dataset change you did not record. If you cannot correlate the two, you cannot diagnose the cause.

Tooling surface

Plugins, skills, MCP servers, and add-ons that let agents reach the internet and internal systems. MCP, the Model Context Protocol, is a standard way for models to call external tools and data sources. This is the newest and least-governed part of the chain, and it loads executable behavior without a broadly adopted, cross-platform signing convention comparable in maturity and reach to established software artifact signing.

The driver analogy is useful here, as long as you keep its boundary in view. In the early 2000s, installing a driver did not require a signature. Today every driver carries a signature identifying who signed it, because the driver loads code into the kernel. Agent skills and plugins load executable behavior into a system with access to your data, and they lack an equivalent convention at the ecosystem level. The trust problem rhymes. The execution models are not identical, and the analogy stops being exact once you start comparing kernel privilege boundaries to agent tool permissions.

The observable signal is an unexpected tool invocation — a call to a tool the agent has never used before, or a call pattern that does not match the task.

Permission surface

Agent identities, scopes, and long-lived credentials. This is where a single compromised component becomes a lateral-movement path. An over-permissioned agent that can read three databases and write to two of them turns one poisoned input into a multi-system incident.

The observable signal is anomalous data access: reads or writes outside the agent's normal pattern, or access to resources the agent's task does not require.

The cross-cutting failure

The boundary between data and instructions is blurred. When an agent consumes untrusted content — a web page, a document, an email — that content can contain instructions that hijack tool calls. This is not a model producing a wrong answer. It is a model being steered into taking an action. The failure mode is an action, not an output, which is why output filtering alone does not catch it.

For each surface, the useful question is the same: what can this component reach, and what signal would tell you it reached somewhere it should not have?

Why Provenance Is Necessary but Not Sufficient

Provenance is a tamper-evident record of an artifact's origins and modifications: where it came from, who authored or trained it, what datasets and source code produced it. It is the foundational control, and it is the only practical way to answer lineage questions when a base model or dataset is later found to be compromised. Without it, you cannot identify which of your derivatives are affected.

The practical starting point is not full attestation. Attestation, in this context, is a signed statement that a build or artifact met specific conditions — a stronger, more formal claim than a metadata record. The first move is capturing enough metadata to answer basic lineage questions: where did this artifact come from, who trained or changed it, what datasets were used, what source code generated it. Partial provenance beats an aspirational framework that never ships. Google's published guidance on AI supply-chain security makes the same point — start with lineage metadata, then build toward formal attestation.

Cryptographic integrity checks matter more for datasets than for code, precisely because dataset version control is less mature. If you cannot trust that the dataset you trained on is the dataset you think you trained on, the model's provenance is built on sand.

Here is the boundary. Provenance is largely a post-production verification mechanism. It tells you what an artifact is, not what it will do at runtime. It does not stop a malicious component that was correctly signed by a compromised or careless producer. A signed artifact from a producer whose build pipeline was breached is still a signed artifact.

I treat provenance as the receipt, not the lock. Receipts matter for forensics and accountability. When something goes wrong, the receipt tells you what you bought and who sold it. Locks matter for containment. When something goes wrong, the lock determines whether the damage stays in one room or spreads through the building. You need both, and confusing one for the other is how teams end up with a clean attestation chain and a breached production database.

Permissions Are the Real Blast Radius

Scanning finds problems. Permissions decide how bad the problems get. This is the section where I would push back hardest on the instinct to buy a scanner and call it done.

Every agent, tool, and pipeline component should carry a unique, auditable identity. Without one, you cannot make an authorization decision, and you cannot trace which component did what. Shared credentials collapse the audit trail into a single indistinguishable actor.

Least privilege applies to agent identities the same way it applies to human ones, with one adjustment: grant access for the current task, not for the agent's imagined future usefulness. The temptation with agents is to grant broad access because you do not know what the agent will need. That is exactly backwards. You do not know what it will need, so you grant the minimum and expand when a task actually requires it.

Agent sprawl is a supply-chain problem in disguise. Every unmanaged, over-permissioned agent is another component that can be compromised, and another component nobody is watching. The count grows faster than the oversight.

Lifecycle governance is the control that keeps sprawl bounded: registration, approval, expiration, and decommissioning. An agent nobody remembers owning is an unmonitored entry point. If it has a credential and no owner, it is a door with no lock and no one checking it.

Deterministic safeguards deserve their own line. Prohibited actions should be blocked by policy that does not depend on model behavior. A model that has been manipulated will not reliably refuse. If your only defense against an agent deleting a production table is the model's judgment, you do not have a defense. You have a hope.

My decision rule: if you cannot name the owner, the scope, and the expiry of a credential, you cannot reason about the blast radius of the component holding it. That is not a compliance checkbox. It is the minimum information required to make any authorization decision at all.

Isolation and Monitoring: Designing for the Assumption of Compromise

Assume individual components will fail. Design compensating controls rather than trusting any single layer. This is the shift from prevention to containment and detection, and it is where the operational tradeoffs get real.

Isolation reduces blast radius and prevents cascading failures. A component that cannot reach the database cannot exfiltrate the database, regardless of what it was told to do. The tradeoff is friction: isolated components are harder to wire together, and every boundary you add is a boundary you have to maintain. The question is not whether isolation is free. It is whether the cost of the boundary is lower than the cost of the incident it prevents.

Runtime enforcement catches what build-time checks cannot. Build-time scanning cannot see an attempt to extract system instructions, access sensitive data, misuse an internal-only tool, or route information to an untrusted destination. Those are runtime behaviors.

The control point matters more than the detection quality. Blocking before tool invocation is materially different from alerting after exfiltration. If the block happens before the tool runs, no data moves. If the alert happens after, the data is already gone and you are doing forensics. Microsoft's published approach to agent runtime protection makes this explicit: block the risky action before invocation, then generate an alert that names the agent, user, and tool involved so the SOC can act.

Sensitive data leakage risk rises when agents aggregate across multiple sources or retain long-lived context. The more an agent remembers and the more it can see, the more it can leak. Limit memory persistence and enforce deterministic rules for retention and output. This is a design decision, not a monitoring one.

The honest limitation: runtime monitoring is a detection and containment layer, not a proof of safety. It reduces dwell time — the window between compromise and detection — and damage. It does not make an untrusted component trustworthy. If you find yourself treating your monitoring as a reason to skip provenance or permissions, you have inverted the stack.

What the Ecosystem Still Does Not Solve

Separating confirmed capability from open problems matters here, because the temptation to over-invest in immature controls is strong and the vendor claims are loud.

Tool and plugin ecosystems lack a broadly adopted, cross-platform signing and vetting convention comparable in maturity and reach to signed drivers. Provenance for agent tooling remains immature at the ecosystem level. Individual platforms are building their own approaches, but there is no shared standard yet.

Dataset version control and data provenance tooling remain less mature than code equivalents. This limits how far lineage tracking can go today. You can build good practice on top of immature tooling, but you should not assume the tooling will catch what your practice misses.

Vendor tooling in this space is early and consolidating. A number of security vendors now offer discovery, access control, and runtime monitoring for agents and MCP servers, and there is significant venture funding behind the category. Treat product claims about coverage as claims, not verified coverage of your specific stack. The category is real; the coverage is unproven until you test it against your own environment.

Research directions such as proactive, real-time supply-chain defense are research signals, not deployed mainstream capability. Current provenance and attestation mechanisms are largely post-production verification tools with limited real-time or proactive prevention. That is a description of where the research is, not a product you can buy today.

The open question worth tracking: whether agent tool marketplaces converge on a signing standard, or whether each platform invents an incompatible one. The answer determines whether cross-platform provenance is a solvable problem or a permanent gap.

A Practical Starting Sequence for Your Own Stack

Here is the order I would actually execute, and why each step comes where it does.

Step one: inventory. List every model, dataset, tool, plugin, and data source in the system. Treat each as a security dependency inside the boundary. This is the cheapest artifact that makes every later control possible, and it is the one most teams skip because it feels like paperwork. It is not paperwork. It is the map you will use when something breaks.

A list alone is too weak to act on. Capture a minimum schema per component: origin and version or digest, owner, identity, reachable resources, update path, isolation boundary, and monitoring point. Those fields are what let you rank components by reach and blast radius instead of producing documentation nobody reads. The map drives a go/no-go decision on each component: high privilege plus broad reach plus weak provenance plus no owner is an escalation, not a line item.

Step two: lineage metadata. For each artifact, capture origin, author, training data, and source code. Do this before attempting formal attestation. Partial provenance beats an aspirational framework that never ships.

Step three: identities and scopes. Assign unique identities and least-privilege scopes to every agent and pipeline component, with owners and expiry dates. If a credential has no owner and no expiry, it is not governed.

Step four: isolation and runtime enforcement. Isolate components, add enforcement at the tool-invocation boundary, and make alerts name the agent, user, and tool. Block before invocation where you can.

Step five: monitoring and rehearsal. Monitor for dependency compromise and data poisoning. Then rehearse the response for a compromised base model or dataset. A response plan you have never walked through is a document, not a capability.

When this is overkill: a single-team internal tool with no third-party models, no external tool access, and no sensitive data behind it does not need full attestation. It still needs an inventory and a permission review. Those two steps are cheap and they scale.

The learning path that sustains this: provenance and attestation concepts, identity and least-privilege design for non-human actors, runtime policy enforcement, and dataset versioning practice. Build the inventory script first. It forces you to confront the components you did not know you were trusting.

The discipline is continuous, not a checkpoint. Provenance tells you what you have. Permissions decide what it can do. Isolation decides how much it costs you when it turns. Use the map to prioritize: components with high privilege, broad reach, weak provenance, or no owner get attention first. That is the decision the inventory exists to enable.