Designing the Cache: Building a Distributed Cache, Part 1

This is the first part of a series where I build a small distributed cache from scratch — not because the world needs another one, but because building one is the fastest way to actually understand the trade-offs that every off-the-shelf cache has already made for you.

What we’re building

A key-value store that:

• Shards keys across multiple nodes
• Replicates each shard for fault tolerance
• Resolves conflicting writes with a simple, explainable rule
• Exposes a tiny client library so the rest of the series can focus on behavior, not plumbing

We are explicitly not building something production-ready. We’re building something legible — every design decision should be traceable to a sentence you could say out loud.

Consistent hashing, briefly

The first decision that shapes everything downstream is how keys map to nodes. A naive hash(key) % nodeCount works right up until nodeCount changes, at which point almost every key remaps to a different node and your cache hit rate falls off a cliff.

Consistent hashing fixes this by placing both nodes and keys on a ring:

function nodeForKey(key: string, ring: RingEntry[]): string {
  const hash = hashToRingPosition(key);
  const entry = ring.find((e) => e.position >= hash) ?? ring[0];
  return entry.nodeId;
}

When a node joins or leaves, only the keys between it and its neighbor on the ring need to move. Everything else stays put.

Replication, and the rule we’re choosing

Each shard is replicated to N nodes. For this series, N = 3, and we resolve conflicting concurrent writes with last-write-wins by timestamp — the simplest rule that’s still correct enough to reason about, at the cost of silently dropping a legitimate concurrent write now and then.

That trade-off is worth sitting with:

We start with last-write-wins on purpose. Part 3 revisits this once we’ve felt the cost of it firsthand.

What’s next

In Part 2, we implement the ring and the client library, and write the first test that proves a node can leave the cluster without losing data.