Consensus Based vs Best Effort Lock Services: Latency and Safety Trade-offs

Two fundamentally different architectures dominate production distributed lock implementations, each with distinct latency, safety, and availability characteristics. Consensus based lock services built on Paxos or Raft (like Google Chubby or etcd) replicate lock state across 3 to 5 nodes and require quorum agreement for acquisitions. Every lock acquisition involves at least one or two quorum round trips plus durable writes, resulting in p50 latencies of 2 to 5 milliseconds and p99 of 10 to 50 milliseconds within a region. The benefit is strong safety: even during network partitions within the quorum, at most one client holds the lock as observed by any majority. Leader failover takes several seconds (often 5 to 15 seconds) causing temporary unavailability, but recovery is predictable and correct. Best effort locks use eventually consistent stores or in memory caches (like Redis or DynamoDB) with conditional writes and expirations. Acquisition is a single atomic operation against one or a few nodes, delivering p50 latencies under 1 millisecond and p99 of 1 to 3 milliseconds within an availability zone, handling hundreds of thousands to millions of operations per day. However, safety degrades during network partitions or node failures: two clients can simultaneously believe they hold the lock if nodes disagree or if failover races occur. Production teams mitigate this by always pairing best effort locks with fencing tokens and idempotent operations, treating the lock as an advisory hint rather than an absolute guarantee. Choosing between them depends on your latency budget and blast radius. Google systems place only control plane operations like master election on Chubby, never high queries per second (QPS) data path operations, because single digit to tens of milliseconds overhead is acceptable for infrequent coordination but would cripple request serving. For low risk critical sections like deduplicating scheduled jobs or rate limiting, teams often use Redis backed locks with sub millisecond latency, accepting that occasional duplicate work may occur but ensuring conflicting writes are fenced at the resource. The key insight is that no lock service alone provides perfect safety; you must always implement fencing and idempotency at the protected resource regardless of which service you choose.

Kubernetes etcd clusters with 3 to 5 members: median lease write 2 to 5 milliseconds, p99 10 to 50 milliseconds, 10,000 to 20,000 ops/second sustained throughput for leader election

Redis based application locks achieving p50 under 1 millisecond within availability zone, used for deduplicating scheduled jobs with idempotency to handle rare overlaps

Google Chubby: control plane only usage for Bigtable master election and Google File System (GFS) master coordination, avoiding high QPS data path operations that cannot tolerate multi millisecond overhead