Learn / System design

CAP, applied

Lesson 11 of 37 · 8 min read ·

What it actually says

In the presence of a network Partition, a distributed system must choose between Consistency and Availability.

The triangle diagram misleads people into thinking you pick two of three. You do not. Partitions are not a design choice — networks fail, cables get cut, a switch reboots, a cloud AZ becomes unreachable. So P is mandatory, and the only real choice is what you do during a partition:

  • CP — refuse to serve rather than serve possibly-wrong data. The system returns errors until the partition heals.
  • AP — keep serving, accept that different nodes may disagree, reconcile afterwards.

"CA" would mean a system that gives up entirely when the network fails. That is a single node, not a distributed system.

During a partition there is no third option — you either refuse the write or accept divergence.During a partition there is no third option — you either refuse the write or accept divergence.

Why it matters in practice

The important consequence is that this is a per-feature decision, not a per-company one. The same product wants different answers in different places:

FeatureChoiceWhy
Payment authorisationCPDouble-spending is worse than an error message
Inventory at checkoutCPOverselling costs money and trust
Like counts, view countsAPNobody is harmed by a stale number
Feed / timelineAPSlightly stale content beats an empty screen
Chat message deliveryAPDeliver now, order and dedupe later
Username registrationCPTwo people cannot own one name

A senior answer to "CP or AP?" is never one word. It is "CP for the ledger, AP for the activity feed, and here is what the user sees in each case when it degrades."

PACELC: the half nobody quotes

CAP only describes behaviour during a partition. PACELC adds the rest: if Partition, then A or C; Else, Latency or Consistency.

That second clause is the one you live with daily. Even with a perfectly healthy network, strong consistency costs latency — a synchronous replica acknowledgement adds a round trip to every write. Most systems that call themselves "eventually consistent" are not defending against partitions at all; they are buying latency.

Eventual consistency, concretely

"Eventual" means the replicas converge if writes stop. It does not promise when, and it does not stop a user from seeing something surprising in the meantime — most often reading their own write from a lagging replica and concluding your app lost their data.

The standard mitigations:

  • Read-your-writes — route a user to the primary for a short window after they write.
  • Monotonic reads — pin a session to one replica so time never appears to go backwards.
  • Version vectors / last-write-wins — decide up front how conflicts resolve, because they will occur.

Replication lag is a number you should be able to look up right now for your own database. If you cannot, you do not know how stale your reads are.

Gotchas

  • A single Postgres with one synchronous replica is a distributed system. CAP applies to you.
  • "Strongly consistent" claims usually carry conditions — a specific read mode, a specific consistency level. Read them.
  • Microservices make this worse: a write spanning three services has no transaction. You need sagas and compensating actions.
  • Users tolerate stale data far better than they tolerate errors or lost work. That asymmetry should drive the choice.

Prove you know it

List the five most important operations in something you have built. For each, write CP or AP, and one sentence on what the user sees when that side degrades. Any operation you cannot answer for is one you have not designed yet.

Go deeper