100,000 Videos in Your Pocket: How Douyin's Recommender Reads Your Whole Life in O(1)

Douyin, the Chinese short video platform owned by ByteDance, has developed a technology called SequenceO1 that allows them to handle 100,000 interactions from each user in an efficient manner. Traditionally, processing such a large amount of data would be too costly in terms of both training time and serving (real-time recommendation) time. SequenceO1 solves this problem by compressing a user's entire history into a smaller "sketch" of a few hundred to a few thousand summary vectors.

The key innovation is the use of attention mechanisms that allow the system to read from this compact sketch instead of the full history. There are two time scales involved: recent interactions are handled by attending directly to the last 10,000 videos, while long-term preferences are captured by the sketch. When a user makes a recommendation request, the system reads the cached sketch in constant time (O(1)), rather than scanning through 100,000 items.

The SequenceO1 model is much cheaper to train and serve than the previous approaches, achieving up to 63.9 times lower inference costs. At the same time, it retains 83% of the ranking quality compared to naive scaling to 100K history items. In practice, Douyin has deployed SequenceO1 at full traffic and it helps the platform read users' tastes from their long interaction histories while still being able to finish video recommendations quickly, even with 100,000 interactions per user.