Three Universal Idempotency Patterns and When to Use Each

There are three fundamental patterns for achieving idempotency in distributed systems, each with distinct trade-offs in complexity, storage cost, and applicability. First, natural idempotence via resource semantics leverages HTTP PUT and DELETE operations that overwrite to a target state. When you issue PUT /users/123 with a complete user object, repeating the request converges to the same outcome regardless of how many times it executes. This pattern is simple and requires no additional infrastructure, but only applies to operations that can be expressed as setting or deleting a known resource to a specific state. It does not work for operations that create new resources with server generated identifiers or operations with cumulative effects like incrementing counters. Second, request level idempotency tokens use a unique key per logical operation, where the server deduplicates by key and returns the original result for duplicates. This is the most flexible pattern, applicable to any operation including resource creation and complex multi step transactions. The client generates a unique token (often a UUID Version 4) and attaches it to the request. The server maintains a deduplication store keyed by token, atomically reserves the token on first sighting, executes the operation, records the result, and returns it. On duplicate requests with the same token, the server returns the previously recorded outcome without re-executing. Stripe uses this pattern for payment operations, storing the idempotency key with request parameters and returning the original charge object for duplicate keys. The trade-off is storage cost and lifecycle management: at 1,000 requests per second with a 24 hour deduplication window, you store up to approximately 86.4 million keys, requiring around 17 gigabytes per day at 200 bytes per record plus index overhead. Third, state based idempotence detects and rejects duplicates using versioning for optimistic concurrency, deduplication tables, or event identity in the write path, often within a single atomic transaction. For example, you maintain a version number on each aggregate and require clients to specify the expected version. On mutation, you check whether the current version matches the expected version; if not, you reject the update as stale or duplicate. Alternatively, you maintain a processed requests table with a uniqueness constraint on request identifier, wrapping both the business update and the deduplication check in a transaction. This pattern provides the strongest consistency guarantees and keeps deduplication logic within domain boundaries, but increases schema complexity, indexing requirements, and transaction scope.

Natural idempotence: PUT /profiles/user123 {"name": "Alice", "email": "[email protected]"} sets the profile to exactly this state; repeating the request 10 times results in the same profile state.

Request token at Stripe: Client sends POST /v1/charges with Idempotency-Key: req_7a4b2c3d. Stripe stores the key, creates the charge, and records charge ID ch_xyz. A retry with the same key returns ch_xyz without creating a second charge.

State based versioning: An order aggregate at version 5 receives an update with expected_version: 5. The service increments to version 6 and commits. A duplicate request with expected_version: 5 is rejected because current version is now 6.

Uber streaming deduplication: Consumer writes event ID event_12345 to a processed_events table with uniqueness constraint and updates trip state, both in one transaction. Replay of event_12345 fails the uniqueness check and skips the update.