Silicon Valley 101: Unpacking the Wild Growth of the AI‑Data Industry | 硅谷101：深度解析AI数据行业的野蛮生长

Title: Silicon Valley 101: Unpacking the Rapid Growth of the AI-Data Industry

The opening segment of the podcast discusses the upcoming hackathon event called STORY101 LAB AI video creation challenge, part of the Alignment 2026 annual event. This event caters to both experienced teams using AI for video production and newcomers interested in experimenting with AI video creation. Winners of the competition can secure prizes and have their work showcased publicly during the ten-day event in San Jose's Plug and Play technology center.

The 2026 Alignment conference also opens its doors to attendees from renowned companies like OpenAI and Nvidia.

Investors are currently focusing on the AI data sector, with many startups experiencing rapid growth due to the increasing complexity and specialization of training data requirements for large model enterprises. A prime example is AfterQuery, which saw its valuation skyrocket from $300 million to $3 billion in less than half a year.

Established data companies, such as Meta's investment in Scale AI, valued at over $29 billion, and Mercor with a $10 billion financing, also demonstrate the immense value investors place on data providers.

While the high valuations of data companies may seem puzzling, the core question remains: what exactly are these companies selling, and why are large model enterprises willing to pay exorbitant sums for their services? Traditional data annotation, which primarily involved labeling images and scoring model outputs, has evolved into a new breed of data products, such as scoring rules and reinforcement learning environments, developed by industry experts.

Current AI evaluation types are expanding to include vertical sectors like finance and law, as well as comprehensive workflow testing for intelligent systems. However, the correlation between high-ranking scores and actual model capabilities remains a point of confusion, as the elevation of rankings may not always equate to genuine improvements in model performance.

Two interviewees, Scale AI's Head of Post-Training and Evaluation Research, He Yunming, and Berkeley Ph.D. candidate and Intelligent Test Baseline Project researcher, Sun Yixin, discuss the major categories of AI data service providers. Some firms originate from recruitment businesses, like Scale AI, while others focus on synthetic data generation.

Additionally, some previously model-focused companies have ventured into data sales. These various players exhibit distinct characteristics, ranging from comprehensive enterprises targeting all needs to specialized research-oriented startups and niche vertical companies that handle data as intermediaries.

The discussion on industry demands transitions from crowd-sourced annotation data to intelligent body data, highlighting the evolving priorities within the sector. He Yunming recalls the rise in salaries and talent competition, observing the swift transition from score rules to reinforcement learning environments. Over the past year, there has been a shift from evaluating scoring rules to focusing on reinforcement learning environments, including evaluation procedures, scoring rules, and real-world task data.

The limitations of pre-training and model-based reinforcement learning, particularly in domains like medicine, finance, and general text, necessitate the development of scoring rules and environments that can handle subjective outputs.

The construction of evaluation benchmarks faces challenges in accommodating diverse correct outputs and quantifying subjective metrics. While code-based evaluation benefits from a wealth of verifiable benchmarks, other industries struggle to achieve the same level of standardization. The solution lies in a hybrid approach, combining specialized reward models with hard-scored evaluation rules tailored to specific tasks.

The second part of the podcast delves into the difficulties of creating evaluation benchmarks, emphasizing the need for a balance between scientific research and commercial data acquisition. Both approaches have their advantages and challenges, with academic projects relying on volunteer expertise and commercial data collection requiring significant financial investments to secure top-tier expertise.