Design Slack
You're asked to design a workplace chat product like Slack or Teams. It has workspaces, channels ranging from three members to one hundred thousand, threads, mentions, and unread badges. Let's start with the standard real-time chat architecture, then examine what breaks as channels and traffic grow. We'll make sure the design still holds if usage explodes.
Clarify the requirements
- What is the scale distribution? Assume workspaces with up to ~500k users and channels ranging from 2 to 100k+ members. Establish the channel-size range early because it determines the fan-out strategy.
- Features in scope? Channels, DMs, threads, mentions, unread counts, presence, history with pagination. Search and file sharing: acknowledge and defer (search is its own index; files are object storage plus links).
- Delivery contract? No message lost after the server acks, per-channel ordering, sync across a user's devices, offline catch-up.
- Latency? Sub-second delivery to online members in the same region; seconds tolerable cross-region.
Back-of-envelope numbers
- Connections: , and at ~200k connections per gateway that's plus headroom
- Messages: average, with a 5× peak of roughly
- Fan-out at the average:
- Fan-out for one large channel: a single message to a 100k-member channel is on its own
- History: before media
One large-channel message creates almost as many pushes as one sixth of a second of global traffic. Let's use separate fan-out strategies for small and large channels.
High-level architecture
- Gateway fleet. Holds the WebSockets, authenticates on connect, heartbeats, stays dumb. Least-connections balancing, graceful draining.
- Connection registry.
user_id → gateway(s), TTL-cleaned by heartbeat, multi-device aware. - Messaging service. Stateless business logic: validate membership, persist, assign the channel sequence number, trigger fan-out.
- Message store. Partitioned by
channel_id, ordered within a channel by sequence number, so "load this channel's recent messages" is one partition read. - Channel pub/sub. The large-channel path: gateways subscribe to channel topics instead of the sender pushing per member.
The message path follows real-time and collaborative systems exactly: persist first, assign the per-channel sequence number, ack the sender, then push. Offline members get nothing pushed; their catch-up is the cursor sync below.
Deep dive 1: fan-out, and the threshold that saves you
For a 50-member channel, fan-out per member is fine: look up each member's gateways in the registry, push. For the 100k-member #general, per-member fan-out means 100k registry lookups and pushes for every message, and a busy minute in that channel overwhelms the messaging tier.
Use the inversion above a configurable membership threshold, such as 1k. Publish once to the channel topic, then let each gateway deliver locally to its connected members. Cost now scales with roughly 50 gateways instead of 100k members. The threshold lets us keep direct fan-out for small channels without letting large ones overwhelm the messaging tier.
Deep dive 2: unread counts and mentions, the sleeper hard problem
Unread badges can become the largest write amplifier in the system. If we increment one counter per member, a message in a 100k-member channel creates 100k writes. We also cannot scan full history whenever a user checks a badge.
Store one tiny record per (user, channel): last_read_seq, updated when the user reads. The unread count is then channel_latest_seq - last_read_seq, one subtraction over two cheap reads, with no per-message per-member write at all. Mentions break the symmetry because a badge with your name on it must be reliable: write an explicit mention row per mentioned user (bounded, since a message mentions a handful of people), and treat @channel in huge rooms as a product question, since 100k mention rows for one message is a policy decision (rate-limit it, require permission) before it's an engineering one. Total-unread-across-workspace badges are then a small cached aggregate per user, refreshed lazily. The lesson to say out loud: derive counts from sequence numbers instead of materializing per-member state per message; it's the difference between O(members) writes and O(1).
Deep dive 3: multi-device sync and offline catch-up
One user, three devices, each with its own socket on possibly different gateways. The registry maps a user to a set of connections, and delivery pushes to all of them. Read state syncs through the same channel: mark a channel read on the laptop, and a small read_state_updated event pushes to the phone so the badge clears everywhere.
Reconnect is the cursor sync with Slack's shape: the client holds last_seen_seq per channel, and on reconnect asks for everything after its cursors, ordered by channel activity so the visible channel fills first. A client offline for a week doesn't replay a week of #general; past a gap threshold, send the last N messages plus an unread summary and lazy-load the rest on scroll. And the reconnect stampede after a gateway restart gets the standard treatment: jittered client backoff plus server-side drain, without which a deploy becomes an outage.
Deep dive 4: what actually breaks at 100x
Let's walk through the bottlenecks in order. At 100x, the registry becomes a hot spot, so shard by user, cache at gateways, and skip member lookups for large channels through subscriptions. Presence updates grow rapidly, so use TTLs, debounce updates, and switch large rooms to pull-only presence if needed. Hot channels concentrate writes on one message partition, so put a sequence-assigning cache in front. Cross-region latency appears, so place gateways near users while keeping each channel's sequencer in one home region to preserve ordering.
Leveling signals
Related lessons
Picks up where chat's per-message model ends and concurrent editing begins.
The same real-time delivery problem with a per-user inbox instead of a channel fan-out.