Replication Topology and Cross Region Cost Trade-offs

Replication topology determines how changes propagate between leaders and directly impacts availability, latency tail behavior, and operational cost. The three primary patterns are all to all (each leader replicates directly to every other leader), hub and spoke (leaders send changes through central relay nodes), and circular or ring (each leader forwards to the next in a chain). All to all topology minimizes replication latency and eliminates single points of failure, making it the preferred choice for small numbers of Regions (typically 2 to 5). Each leader maintains per origin monotonic sequence numbers so peers can apply operations in order per origin while allowing concurrent operations from different origins. However, connection count grows quadratically: 5 Regions require 20 bidirectional replication streams, and 10 Regions require 90 streams.

Hub and spoke reduces connection overhead by routing through dedicated relay nodes, but introduces single points of failure and an extra network hop adding 20ms to 50ms to replication latency. Production systems using hubs provision multiple hub nodes and implement automatic failover with anti entropy repair to detect and fill gaps when a hub fails. Circular or ring topology is simple and uses minimal connections but is fragile: a single failed link in the chain can stall propagation to downstream leaders. If Tokyo replicates to London, London to Frankfurt, and Frankfurt back to Tokyo, a Tokyo to London link failure blocks Frankfurt from receiving updates. Production multi leader systems avoid pure ring topologies and instead use all to all for small cluster sizes or hub and spoke with redundancy.

Cost scales roughly linearly with the number of Regions due to write amplification and cross Region data transfer charges. A concrete example: 10,000 writes per second of 1 KB items replicated across 3 Regions produces 10,000 × 1 KB × 2 peer Regions = 20 MB per second of egress per Region, totaling approximately 1.7 TB per day in inter Region transfer. AWS charges roughly $0.02 per GB for cross Region transfer within the same continent and up to $0.09 per GB across continents, translating to $34 to $153 per day just for replication traffic. Secondary indexes and change data capture streams multiply this cost. DynamoDB Global Tables also charge write capacity in every Region, so 10,000 write request units per second across 3 Regions costs 3 times the single Region price.

💡 Key Takeaways

•All to all topology provides lowest replication latency and no single point of failure, preferred for 2 to 5 Regions; connection count grows as N × (N minus 1) requiring 20 streams for 5 Regions and 90 streams for 10 Regions

•Cross Region data transfer costs dominate at scale: 10,000 writes per second of 1 KB across 3 Regions generates 1.7 TB per day costing $34 to $153 per day depending on Region pairs, before accounting for secondary indexes or change streams

•Write amplification is multiplicative: writes consume capacity in every Region, so DynamoDB Global Tables with 3 Regions at 10,000 write request units per second costs 3 times single Region pricing plus replication transfer fees

•Hub and spoke reduces connection overhead but adds 20ms to 50ms latency per hop and requires redundant hubs with anti entropy repair to avoid single points of failure in production deployments

•Circular or ring topology uses minimal connections but is fragile; a single failed link blocks propagation to downstream nodes, making it unsuitable for production multi leader systems without bypass paths

•Provision WAN bandwidth with headroom for p99 bursts and retransmissions; throttle replication streams to prevent head of line blocking of foreground user traffic during catch up after outages

📌 Examples

AWS DynamoDB Global Tables all to all replication: 3 Regions (us-east-1, eu-west-1, ap-southeast-1) with 5,000 writes per second of 2 KB items produce 5,000 × 2 KB × 2 = 20 MB per second egress per Region; cross Atlantic transfer at $0.02 per GB plus cross Pacific at $0.09 per GB yields approximately $100 per day in transfer costs alone

Amazon Dynamo internal retail: N=3 replication factor with preference list routing; coordinator node for a key forwards write to N minus 1 peer nodes; typical configuration routes within same availability zone when possible to reduce cross AZ transfer charges

Multi Region hub design: central relay cluster in us-east-1 receives changes from us-west-2, eu-west-1, ap-southeast-1 and fans out; relay cluster sized at 2x write throughput to handle bidirectional traffic and provision automatic failover to secondary hub on primary failure

← Back to Multi-Leader Replication Overview