Message Queues & Streaming • Delivery Guarantees (At-least-once, Exactly-once)Hard⏱️ ~3 min
Transactional Patterns: Outbox and Inbox
The Outbox Pattern:
The outbox pattern solves the producer-side problem of atomically updating state and publishing an event without distributed transactions.
The Inbox Pattern:
The inbox pattern solves the consumer-side problem. On message consumption, the consumer writes the message identifier to an inbox table atomically with applying side effects in a single database transaction. If the identifier already exists, the consumer skips side effects, achieving exactly-once processing at the consumer-plus-sink boundary.
Trade-off:
These patterns trade local transaction overhead for elimination of distributed coordination. A single database transaction is typically sub-10ms, while distributed two-phase commit can add tens to hundreds of milliseconds and reduce throughput by 10-40%. The patterns require that business state and outbox/inbox table share the same transactional boundary.
1
Atomic write: In the same transaction that updates business state, append an outbox record representing the event to publish.
2
Relay process: A background relay reads committed outbox rows and publishes them to the message bus, marking them as sent.
3
Result: Atomic state change plus publish without distributed transactions (single-digit ms commit, 10-100ms relay polling).
💡 Key Takeaways
✓Outbox pattern achieves atomic state update plus event publish by writing both in a single local transaction (single digit milliseconds), then relaying outbox rows to the message bus asynchronously (adds 10 to 100 milliseconds).
✓Inbox pattern writes message identifiers atomically with side effects in a single transaction. If the identifier exists, side effects are skipped, achieving exactly once processing at the consumer plus sink boundary.
✓These patterns avoid distributed two phase commit, which typically adds tens to hundreds of milliseconds latency and reduces throughput by 10 to 40 percent due to coordination and write amplification.
✓Patterns require business state and outbox or inbox table to share the same transactional boundary, working best when state is colocated in a single database or shard.
✓Relay processes must handle failures idempotently. If relay crashes after publishing but before marking the outbox row as sent, the message will be published again, requiring downstream consumers to implement idempotency.
📌 Interview Tips
1An e commerce order service receives a create order request. It begins a transaction, inserts the order into an orders table, inserts an order_created event into an outbox table, and commits. A relay process polls the outbox every 50 milliseconds, publishes events to Kafka, and marks them as sent.
2A payment processor consuming charge requests writes the message identifier to an inbox table in the same PostgreSQL transaction that inserts the charge record. If a duplicate message arrives (due to at least once delivery), the unique constraint on the inbox table causes the transaction to fail harmlessly, preventing double charging.
3A multi tenant SaaS application uses one database per tenant. Each tenant database has an outbox table. When a user updates their profile, the transaction writes to the users table and the outbox table atomically. A relay publishes profile_updated events to a shared Kafka topic partitioned by tenant_id.