HLD: Instagram Feed

Instagram aims to create a photo-sharing platform where users can easily upload and share photos and videos. The application will feature a feed showcasing photos from users a person follows, displayed in a chronological order with the most recent posts appearing first. Users will be able to like and comment on photos, and each user's profile will display a grid of their photos.

Functional requirements include the ability to upload photos and videos, follow and unfollow users, view a home feed, like and comment on photos, and view a user's profile. Non-functional requirements involve ensuring high availability and eventual consistency for the feed, with a low latency of around 200 milliseconds for feed loading.

The system must handle a significant amount of traffic, processing 100 million photos daily, equating to approximately 1,160 uploads per second and 58,000 read queries per second. To support this massive scale, the platform will employ a tiered storage system with Amazon S3 for hot data, S3 Infrequent Access for warm data, and S3 Glacier for cold data.

To serve photos, the platform will leverage a CDN, never serving images directly from origin servers. The feed generation process employs a hybrid fan-out approach, combining push updates for regular users and read fan-out for celebrities, merged at read time using a Redis cache. The database schema consists of tables for storing photo metadata, the social graph (followers and following), and other essential data structures.

Key challenges include handling the scale of data storage, ensuring efficient upload patterns, and implementing a hybrid feed generation strategy for both regular and celebrity users. By employing a combination of Amazon S3, CDN, Redis, and Cassandra, Instagram aims to provide a seamless and scalable photo-sharing experience for millions of users worldwide.