Design YouTube

You're asked to design a video sharing platform. Creators upload files ranging from a phone clip to a multi-gigabyte 4K master, and within a few minutes those videos are watchable worldwide on a phone over cellular, a laptop on wifi, and a television on fiber. Viewers expect playback to start in under a second and never buffer, and each video shows a view count.

The uploaded file is not the file viewers receive. We need to turn one source into a dozen encoded variants within minutes and deliver them from an edge near each viewer. Let's separate upload, transcoding, and playback into distinct paths.

Clarify the requirements

• How long after upload must a video be watchable? Minutes, not hours, and that expectation is what forces parallel transcoding rather than a simple job per file.
• What playback quality range? From 144p on poor cellular to 4K, which is what makes adaptive bitrate necessary rather than optional.
• Are view counts exact? Almost never. Confirming that approximate counts are acceptable saves you from designing a strongly consistent counter for a number nobody verifies.
• Is recommendation in scope? Usually it is not. Confirm that we can focus on upload and delivery rather than ranking.
• Is live streaming in scope? Treat it as a separate problem covered in Design Twitch and exclude it explicitly.

Assume: minutes to availability, 144p through 4K, approximate view counts, on-demand video only.

Back-of-envelope numbers

• Uploads: $500\text{ hours/minute} \approx 8.3\text{ hours of video per second}$
• Ingest bytes at roughly $1\text{ GB/hour}$ for a source file: $500 \times 1\text{ GB} = 500\text{ GB/minute} \approx 8\text{ GB/sec}$
• Encoded output at roughly $3\times$ the source, once every quality variant is produced: $\approx 25\text{ GB/sec}$, or $2\text{ PB/day}$
• Transcode compute: real-time encoding of $8.3$ hours of video per second means thousands of cores continuously, which is why chunked parallelism matters
• Playback: $1\text{B hours/day}$ watched at an average $2\text{ Mbps}$ is $\approx 1\text{ Pbps}$ of egress at peak, which no origin serves, and that number is why the CDN exists

A datacenter cannot serve a petabit per second of playback traffic. Let's put encoded segments close to viewers before they request them.

High-level architecture

Components

1. Raw storage. The original upload, written directly by the client with a presigned URL.
2. Upload API. Creates the video record and enqueues the transcode job; bytes never pass through it.
3. Job store. Per-video transcode state, including each chunk and each rendition.
4. Transcode queue. Work distribution across the encoding fleet.
5. Transcode workers. Encode chunks in parallel into every rendition in the ladder.
6. Encoded storage. The segments and manifests that are actually served.
7. CDN edge. Where essentially all playback traffic is served.
8. Metadata store. Titles, descriptions, ownership, and view counts.

Deep dive 1: the transcoding pipeline

A one-hour 4K upload encoded serially into eight renditions takes many hours on one machine. The creator won't wait, so the work has to be parallelized, and the way to do it is to stop treating the video as a unit.

Split the source into chunks at keyframe boundaries, a few seconds each, and treat every (chunk, rendition) pair as an independent job. A one-hour video becomes roughly a thousand chunks, and with eight renditions that's eight thousand jobs that can run on eight thousand cores simultaneously. Wall-clock time drops from hours to minutes, bounded by the slowest chunk rather than the sum of all of them.

Splitting at keyframes is what makes this valid: each chunk is independently decodable, so encoding it needs no information from its neighbours. Splitting anywhere else produces chunks that can't be encoded in isolation or that don't stitch back together cleanly.

Several useful properties follow:

• Retries are cheap. A failed chunk re-encodes one chunk, not the whole video.
• Progress is real. Chunks completed over chunks total is an accurate percentage to show the creator.
• Renditions can be prioritized. Publish 480p first so the video is watchable in a minute, and let 4K finish later. Most viewers never request the top rendition anyway.
• Cost scales with demand. A video nobody watches doesn't need its highest renditions encoded eagerly, and encoding them lazily on first request is a real optimization at this scale.

Assembly is a manifest write, not a copy. Once a rendition's chunks are done, publishing it means writing a manifest listing the segments in order. No bytes move.

Deep dive 2: adaptive bitrate delivery

A single file can't serve a phone on a weak cellular connection and a television on fibre. Adaptive bitrate solves it by making the client choose, continuously.

Each video is encoded into a ladder of renditions, which are resolution and bitrate pairs from 144p up to 4K, and every rendition is cut into aligned segments of a few seconds. A manifest, a small index file in a format such as HLS or DASH, lists every rendition and the segments that make it up, so the player knows what's available and where to fetch it.

The player downloads the manifest, measures its own throughput, and requests segments from whichever rendition it can sustain. When the network degrades mid-video it steps down at the next segment boundary; when it recovers it steps back up. Nothing on the server side is involved in that decision, which is what makes it scale, since the server is just serving static files.

Call out two details:

• Segment alignment across renditions is required. Switching mid-stream only works if segment boundaries line up, which means all renditions share the same keyframe positions. This is why the chunking in the previous section is defined once from the source rather than per rendition.
• Start at a low rendition deliberately. Beginning with a small segment gets the first frame on screen fast, and stepping up after a second is invisible to the viewer. Starting at the highest sustainable quality means a longer wait before anything appears, and startup time correlates with abandonment far more strongly than quality does.

Deep dive 3: delivering a petabit per second

No origin serves this traffic. The CDN isn't an optimization here; it's the only way the system exists.

Segments are ideal CDN objects. They're immutable, they're addressed by a stable URL, and they're requested by many viewers, so a segment fetched once at an edge serves everyone in that region. Cache hit ratios in the high nineties are normal.

The interesting decisions are about what's cached where, since the full catalog is far larger than any edge:

• Popularity is extremely skewed. A small fraction of videos accounts for most of the watch time, so a modest edge cache covers the large majority of requests.
• Push, don't pull, for predictable hits. A major creator's release is going to be requested by millions within minutes. Pre-positioning it at edges before publication turns a synchronised origin stampede into a scheduled transfer.
• Popularity is regional. What trends in Brazil isn't what trends in Korea, and caching decisions made per region rather than globally use edge capacity far better.
• Tier the origin. Regional caches sit between edges and origin, so a cold edge fetches from a nearby regional cache rather than crossing an ocean.

Storage tiering matters at petabyte scale too. Most videos are watched almost entirely in their first weeks and then approach zero. Moving the long tail to colder, cheaper storage, and deleting the highest renditions of videos nobody watches, is where the storage bill is actually controlled.

Deep dive 4: view counts

View counts need a different consistency model from video metadata.

A row increment per view fails immediately: a popular video is a single hot row taking thousands of writes a second, which is distributed storage's hot key problem in its purest form.

The mechanism that works is aggregation rather than incrementing. View events go to a stream, a job aggregates counts per video over short windows, and the total is updated periodically. The displayed count is seconds to minutes stale, which no viewer can detect and no product requirement forbids.

Two refinements:

• Approximate counters for very large numbers. Once a video is past a million views, nobody is checking the exact figure, and probabilistic counting or coarse rounding is entirely adequate.
• A view is a product definition, not every open event. We can require a minimum watch duration, deduplicate by user within a window, and filter obvious automation so the count remains meaningful.

Common pitfalls

• Routing upload bytes through the application tier. Clients should write directly to object storage with a presigned URL.
• Transcoding each video as one serial job. Encoding time then scales with video length and creators wait hours.
• Splitting chunks at arbitrary boundaries. Chunks must start at keyframes to be independently encodable.
• Serving playback from the origin. The bandwidth required doesn't exist in any datacenter.
• Incrementing a view counter per view. A popular video becomes a single hot row.

Leveling signals

Related lessons

Design NetflixHard

Stream a fixed catalog with per-title encoding and content pre-positioned inside ISP networks.

Design TwitchHard

Stream live video to a large audience, where nothing can be encoded ahead of time.