Bad news: your AI application isn't that special

The AI landscape is currently dominated by two main types of companies: those focused on AI infrastructure and those offering surface applications. AI infrastructure encompasses tools like agent frameworks, model routing, evaluation and monitoring tools, guardrails, and controls to ensure safe AI usage at scale. On the other hand, surface applications are the tools that actual people use to perform work, such as underwriting assistants, contract reviewers, and sales copilots.

In the market, there is a blending of these two types of companies. Some sell architecture, others sell tools, and many attempt to sell both. This blending is driven by the belief that owning the entire stack is the safest position. However, skepticism exists regarding the profitability of most of these firms, as history has shown that a similar pattern has played out in previous tech bubbles.

The dot-com era serves as a prime example, with researchers estimating that around 50,000 AI-related startups were founded between 1998 and 2002. Of these, approximately 1,700 internet companies went public, with only about 14% being profitable. By late 2002, most internet stocks had lost more than three-quarters of their value, and roughly $1.7 trillion had been wiped out. The number of enduring, large-scale winners, such as Amazon, eBay, Priceline, and Expedia, can be counted on two hands.

The cloud computing market provides a similar case study. Initially, there were hundreds of cloud providers, but today, three companies control roughly two-thirds of the market. The middleware layer, which once operated independently, was absorbed by these hyperscalers. Meanwhile, the application layer on top of the consolidated infrastructure saw thousands of SaaS companies thrive without owning a single server.

This shift saw the bottom of the stack concentrated in a few hands, while the top produced thousands of winners.

The modern data stack, the foundation for AI systems, has already consolidated. The last decade witnessed the rise of hundreds of startups selling pipelines, catalogs, transformation tools, and warehouses. Today, only two companies at scale, Snowflake and Databricks, hold around five billion dollars in annual revenue each, with the hyperscalers' native offerings holding most of the rest of the market.

This trend mirrors the consolidation seen in cloud computing, with large enterprises consolidating their data tools to a single source of truth.

These patterns of consolidation in both AI infrastructure and the modern data stack suggest that the AI orchestration layer will likely follow a similar path. Large enterprises are increasingly making strategic platform decisions to consolidate the AI ecosystem, favoring a few dominant players. As a result, expect the AI orchestration layer to end up consolidated similarly to cloud computing, with a few major players controlling the majority of the market, while independent companies struggle to survive.