The Content Relevance: The Candidate Set Is Destiny - Part 2

Retrieval serves as the foundation for what a ranking system can present to users. When a top editor is asked to curate the ten most important books for a reader, even with flawless selection, the recommendations will still be limited. This illustrates the inherent constraint of large-scale ranking systems. While discussions often center around the final score, the ranker only sees a small portion of the vast available content.

The process begins with retrieval, which determines which items are considered based on limited computational resources. At the scale of social or entertainment platforms, evaluating every post, video, song, creator, or show with the most advanced models is unfeasible. Thus, a funnel approach is employed, with eligibility filtering out items that cannot be displayed due to various constraints, retrieval narrowing down millions or billions of items to thousands, pre-ranking using efficient models to eliminate weaker candidates, heavy ranking leveraging richer features to predict multiple outcomes, reranking constructing the final selection while considering diversity and other constraints, and finally presentation deciding layout, position, explanation, and interaction design, which influence subsequent user feedback.

The classic 2016 YouTube paper outlined a two-stage process of candidate generation followed by ranking, while Instagram has publicly detailed a four-stage Explore funnel including retrieval, first-stage ranking, second-stage ranking, and final reranking. The fundamental principle remains consistent: the theoretical upper limit of a ranker's performance is limited by the recall and composition of its candidate set.

Retrieval can be viewed as a portfolio, with multiple candidate generators each employing a distinct strategy. For social or entertainment surfaces, potential sources may include content from followed accounts, similar items to recent high-signal interactions, long-term interest clusters, fresh content from established creators, new creators, trending content, collaborative signals from similar behavior, semantic matches based on text, audio, or visuals, continuation items like the next episode of a show, exploration pools for uncertainty, and various other sources aimed at providing both a sense of familiarity and a taste of the unknown.

The allocation of candidates across these sources is a crucial product decision, as an overemphasis on similarity to recent consumption without sufficient diversity may result in a lack of breadth, while neglecting new creators or sources may hinder the system's ability to adapt to evolving user interests. Measuring eligibility recall—the percentage of valuable items correctly included—source recall—the percentage of retrieved items that were actually included in the final pool—merge health—the percentage preserved after elimination due to quotas or deduplication—pre-rank recall—the percentage maintained by the efficient model—rank quality—the appropriateness of the final order—slate quality—the cohesion of the assembled final selection—are essential diagnostic metrics.

Ignoring these stages could lead teams to invest heavily in optimizing a late-stage objective while overlooking bottlenecks in the retrieval phase. To quantify retrieval performance, measuring oracle recall at each stage—comparing the items suggested by a more advanced downstream model or later user behavior against those that survived each funnel step—provides insight into where the system may be losing potential value.

Retrieval also carries its own biases. Traditional two-tower models revolutionized large-scale learned retrieval by encoding users and items into separate embedding spaces and applying approximate nearest-neighbor search for efficient similarity matching. While these models offer practicality and efficiency, they impose certain constraints.

Since the user and item towers are computed independently, they cannot leverage the most nuanced pairwise interaction features during retrieval. Additionally, similarity-based methods may overemphasize items with abundant historical interactions, stable representations, and clear connections to user preferences, leading to three common blind spots: freshness blindness where new items receive little consideration due to limited behavioral history; semantic blindness where collaborative signals fail to capture why a new item might be relevant; and possibility blindness where nearest neighbors excel at extending known tastes but struggle to discover entirely new interests.

To overcome these limitations, a diversified approach to retrieval methods, coupled with intentional uncertainty preservation, is essential. Recent advancements, such as Meta's SilverTorch system, demonstrate the shift towards integrating multiple components into a single GPU-native model, enabling wider candidate evaluation, richer relevance logic, and real-time processing—all while maintaining a sub-100-millisecond retrieval budget.

This architectural evolution signifies more than just improved efficiency; it represents a fundamental shift in how retrieval is perceived and utilized within the product development process.