Securing Inherited AI: Models, Runtimes, and Tools Inside Vendor Software

A research report from Wiz revealed that 68% of organizations running self-hosted AI models actually ingest those models through third-party software. This means that two out of three companies with AI models running on their own infrastructure did not intentionally run those models. The security team often did not sign off on these models because they were quietly shipped inside vendor products, such as knowledge tools, code assistants, or support routers, without their knowledge.

This situation highlights the challenges of supply chain security in the age of AI, where new software is embedded faster than the tooling that can monitor it.

The scenario described is not a typical supply chain problem, as it involves a structural blind spot in current tools and processes. When an organization inherits a vulnerable log4j vulnerability, they can read the code and understand the failure. However, when an LLM is inherited through a bundled product, the artifact is a pile of weights that cannot be read or statically analyzed. The attack surface includes text and data, not just code, making it a fluid and unpredictable risk.

The implications of this issue are significant. If a bundled product indexes an internal corpus for retrieval, a well-crafted prompt can quietly pull data across trust boundaries that were not intended by the original vendor. This data exposure through embedded retrieval is a major concern, and it can lead to compliance issues, such as those outlined in the EU AI Act.

Additionally, if a product exposes MCP (Model Control Plane) servers that overprivilege internal APIs, lateral movement becomes possible through these servers, which traditional network segmentation cannot detect.