Distributed Systems: Consistency Tradeoffs

CAP

A partition-tolerant distributed database system continues to work in the face of network partition. Modern distributed database systems are partition-tolerant.

In the face of network partition, the system can tradeoff between consistency and availability. When network becomes partitioned, a CP system chooses consistency over availability and an AP system chooses availability over consistency.

How does the system know the network is partitioned so it can smartly choose availability over consistency or vice versa?

The system does not explicitly know. With a CP system, a node in the minority partition will not get response from enough (3 above) nodes so it will stop accepting writes and reads and will become unavailable. But the majority partition of a CP system can still elect leader and continue serving both reads and writes. An AP system (even minority partition) on the other hand, will continue serving reads and writes and the partitions will diverge — causing split brain or multiple source-of-truth. Once the network partition heals, the system has to reconcile the conflicts to restore consistency.

CAP is moot about a system’s baseline behavior: how does the system behave when there is no network partition which is the majority of the time?

PACELC

Pronounced “pass-elk”, rectifies CAP’s limitation.

In a distributed database system the possibility of failure makes replication required. More consistent we want to keep the views of the replicas, more work we need to do. So, with replication the tradeoff between consistency and latency is inevitable and the tradeoff is always present: with or without network partition.

With PACELC, we have:

  1. If network is partitioned: tradeoff between consistency and availability.
  2. Else: tradeoff between consistency and latency.

Leave a comment