Block Storage Deep Dive: IOPS, Latency, and Consistency Guarantees

IOPS: The Fundamental Block Storage Metric
IOPS (Input Output Operations Per Second) measures how many discrete read or write operations the storage can handle. A database performing 4KB random reads cares about IOPS more than throughput. Reading 10,000 separate 4KB blocks requires 10,000 IOPS regardless of total data volume. Modern NVMe SSDs deliver 100,000-500,000 IOPS while spinning disks max out at 150-200 IOPS due to physical seek time.
Cloud block storage often provisions IOPS separately from capacity. A 500GB volume might support 3,000 IOPS baseline with burst to 10,000. Database workloads that exhaust burst credits see latency spike from 1ms to 50ms+ until credits regenerate.
Latency Breakdown: Where Milliseconds Go
Block storage latency has three components: network transit, storage processing, and media access. Local NVMe drives deliver 50-100 microseconds because there is no network hop. Network attached block storage adds 200-500 microseconds for round trip. Cross availability zone replication adds another 1-2 milliseconds for synchronous writes.
Tail latency matters more than average. A database performing 1,000 queries per second experiences p99 latency on 10 queries every second. If p99 is 10ms versus 1ms average, those 10 queries dominate user perceived performance. Block storage SLAs specify p99 or p999 latency, not just average.
Consistency Guarantees: What Writes Mean
Block storage provides strong consistency by default. A write that returns success is durable and visible to subsequent reads. This differs from some distributed systems with eventual consistency. The guarantee comes from write ahead logging: data hits stable storage before acknowledgment.
Synchronous replication strengthens durability at latency cost. Writing to two availability zones means waiting for both to acknowledge. Asynchronous replication reduces latency but risks data loss if the primary fails before replication completes. The choice is recovery point objective (how much data can be lost) versus recovery time objective (how fast to resume).
Block Size Selection and Alignment
Modern storage operates on 4KB physical sectors. Writing 1 byte still reads the 4KB block, modifies it, writes it back. Misaligned writes spanning two blocks require reading and writing both. Databases align their page sizes to block boundaries: a 16KB database page aligns to four 4KB blocks with no wasted IO.
💡 Key Insight: IOPS limits hit before throughput limits for random workloads. A volume with 3,000 IOPS and 250 MB/s throughput caps at 12 MB/s for random 4KB reads (3,000 IOPS times 4KB).

💡 Key Takeaways

✓IOPS measures discrete operations per second: NVMe delivers 100,000-500,000 IOPS while spinning disks max at 150-200 IOPS

✓Block storage latency: local NVMe at 50-100 microseconds, network attached adds 200-500 microseconds, cross zone sync adds 1-2ms

✓p99 latency matters more than average for databases: 10 queries per second see tail latency at 1,000 QPS

✓Synchronous replication guarantees durability across zones but doubles write latency versus async replication

✓4KB block alignment is critical: misaligned writes cause read modify write cycles doubling IO cost

📌 Interview Tips

1When sizing block storage, calculate IOPS first. A database with 8KB pages doing 5,000 random reads per second needs at least 5,000 IOPS, not just sufficient GB capacity.

2Explain burst credits proactively: baseline IOPS sustain forever, burst depletes. A sudden traffic spike exhausting burst credits explains why the database got slow for 30 minutes.

3If asked about consistency, clarify that block storage is strongly consistent by default. The complexity of eventual consistency appears in distributed databases built on top of block storage, not in the storage layer itself.

← Back to Block vs Object vs File Storage Overview