Message Queues & Streaming • Delivery Guarantees (At-least-once, Exactly-once)Hard⏱️ ~3 min
Transactional Patterns: Outbox and Inbox
The outbox and inbox patterns achieve exactly once processing by confining coordination to local database transactions rather than distributed coordination across message brokers and sinks. The outbox pattern solves the producer side problem of atomically updating state and publishing an event. In the same transaction that updates business state, the producer appends an outbox record representing the event to publish. A background relay process reads committed outbox rows and publishes them to the message bus, marking them as sent. This achieves atomic state change plus publish without distributed transactions, with typical intra database commit latencies of single digit milliseconds and relay polling adding 10 to 100 milliseconds depending on interval.
The inbox pattern solves the consumer side problem. On message consumption, the consumer writes the message identifier to an inbox table atomically with applying side effects in a single database transaction. If the identifier already exists, the consumer skips side effects, achieving exactly once processing at the consumer plus sink boundary even with at least once delivery. Combined with TTL or compaction, this prevents unbounded storage growth.
These patterns trade local transaction overhead for elimination of distributed coordination. A single database transaction is typically sub 10 milliseconds, while distributed two phase commit across message brokers and sinks can add tens to hundreds of milliseconds and reduce throughput by 10 to 40 percent. The patterns require that the business state and the outbox or inbox table share the same transactional boundary, which means they work best when state is colocated in a single database or shard.
💡 Key Takeaways
•Outbox pattern achieves atomic state update plus event publish by writing both in a single local transaction (single digit milliseconds), then relaying outbox rows to the message bus asynchronously (adds 10 to 100 milliseconds).
•Inbox pattern writes message identifiers atomically with side effects in a single transaction. If the identifier exists, side effects are skipped, achieving exactly once processing at the consumer plus sink boundary.
•These patterns avoid distributed two phase commit, which typically adds tens to hundreds of milliseconds latency and reduces throughput by 10 to 40 percent due to coordination and write amplification.
•Patterns require business state and outbox or inbox table to share the same transactional boundary, working best when state is colocated in a single database or shard.
•Relay processes must handle failures idempotently. If relay crashes after publishing but before marking the outbox row as sent, the message will be published again, requiring downstream consumers to implement idempotency.
📌 Examples
An e commerce order service receives a create order request. It begins a transaction, inserts the order into an orders table, inserts an order_created event into an outbox table, and commits. A relay process polls the outbox every 50 milliseconds, publishes events to Kafka, and marks them as sent.
A payment processor consuming charge requests writes the message identifier to an inbox table in the same PostgreSQL transaction that inserts the charge record. If a duplicate message arrives (due to at least once delivery), the unique constraint on the inbox table causes the transaction to fail harmlessly, preventing double charging.
A multi tenant SaaS application uses one database per tenant. Each tenant database has an outbox table. When a user updates their profile, the transaction writes to the users table and the outbox table atomically. A relay publishes profile_updated events to a shared Kafka topic partitioned by tenant_id.