The Transactional Outbox Pattern: Reliable Events Without Dual Writes

A common piece of code:

await db.orders.insert(order)                 // 1. commit to the database
await broker.publish("order.created", order)  // 2. notify the world

If the process crashes, the network blips, or the broker is down between 1 and 2, the order exists but nobody hears about it: no confirmation email, no inventory reservation, no analytics. Swap the order and you can publish an event for an order that was never committed. Wrapping both in try/catch doesn't help; there's no transaction spanning your database and a broker (or an email API, or a webhook). This is the dual-write problem, and it produces rare, silent inconsistencies that are miserable to debug.

The transactional outbox removes the dual write by making "publish" a database write.

The pattern

In the same database transaction as the business change, insert a row into an outbox table describing the event.
A separate relay reads committed outbox rows and delivers them to the broker, queue, or external API.
The relay marks rows delivered (or they're deleted), and retries until delivery succeeds.

Because step 1 is a single local transaction, the event exists if and only if the business change committed.

CREATE TABLE outbox (
  id             bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  aggregate_type text        NOT NULL,        -- 'order'
  aggregate_id   text        NOT NULL,        -- '8f1c…'
  event_type     text        NOT NULL,        -- 'order.created'
  payload        jsonb       NOT NULL,
  headers        jsonb       NOT NULL DEFAULT '{}',
  created_at     timestamptz NOT NULL DEFAULT now(),
  published_at   timestamptz,
  attempts       int         NOT NULL DEFAULT 0,
  last_error     text
);
CREATE INDEX outbox_unpublished_idx ON outbox (id) WHERE published_at IS NULL;

await db.transaction(async (tx) => {
  const order = await tx.orders.insert(input)
  await tx.outbox.insert({
    aggregateType: "order",
    aggregateId: order.id,
    eventType: "order.created",
    payload: { orderId: order.id, total: order.totalCents, customerId: order.customerId },
  })
})

Design the payload as a versioned public contract (schema_version in headers), not a dump of your internal row — consumers will depend on it.

Relay option 1: polling

A worker repeatedly claims unpublished rows:

WITH batch AS (
  SELECT id FROM outbox
  WHERE published_at IS NULL
  ORDER BY id
  LIMIT 100
  FOR UPDATE SKIP LOCKED
)
SELECT o.* FROM outbox o JOIN batch USING (id) ORDER BY o.id;
-- publish each, then:
-- UPDATE outbox SET published_at = now() WHERE id = ANY($1);

Simple, uses only your database, easy to reason about.
SKIP LOCKED lets several relay instances run without double-claiming — but see ordering below. (Postgres as a Job Queue.)
Latency is the poll interval; LISTEN/NOTIFY on insert can wake the relay sooner.
Adds query load; keep the partial index small by deleting or archiving published rows.

A subtle issue with ID-ordered polling: identity values are assigned at insert time, but transactions commit in a different order. A relay that tracks "last published ID" can skip a row with a lower ID that commits later. Claiming by published_at IS NULL (as above) rather than a high-water mark avoids this.

Relay option 2: log-based change data capture

Read the outbox inserts from the database's write-ahead log via logical decoding — Debezium's outbox event router is the common implementation, publishing to Kafka or other sinks.

Lower latency and no polling load; events come out in commit order.
More infrastructure: a CDC connector, a replication slot to monitor (an abandoned slot retains WAL and can fill the disk), and schema handling.
With CDC you can insert and immediately delete outbox rows in the same transaction — the insert still appears in the log — keeping the table empty.

Choose polling until latency or throughput demands CDC.

Delivery semantics: at-least-once

The relay can publish successfully and crash before marking the row. On restart it publishes again. The outbox guarantees at-least-once, never exactly-once. Consumers must be idempotent:

Include a unique event ID (the outbox id, or a UUID) in every message.
Consumers record processed IDs in an inbox table in the same transaction as their effect:

CREATE TABLE inbox (
  consumer   text   NOT NULL,
  event_id   text   NOT NULL,
  processed_at timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (consumer, event_id)
);
-- in the consumer's transaction:
INSERT INTO inbox (consumer, event_id) VALUES ('billing', $1) ON CONFLICT DO NOTHING;
-- if 0 rows inserted → duplicate, skip; else apply the effect, then commit

(Handling Webhooks Reliably applies the same technique to inbound webhooks, and Database Transactions Explained covers why the inbox insert and the effect must share a transaction.)

Ordering

Global ordering is expensive and rarely needed. What usually matters is per-aggregate ordering: order.created before order.paid before order.shipped for the same order.

With a single relay processing in commit order, you get it for free — at a throughput ceiling.
With parallel relays, partition by aggregate_id: a given aggregate is always handled by the same relay worker (e.g. hash of the ID modulo workers, or advisory locks per aggregate), and published with the aggregate ID as the message key so brokers like Kafka keep per-key order.
A failing message blocks its aggregate's later events if you preserve order strictly. Decide: retry with backoff and alert, or dead-letter and let consumers handle gaps (consumers can also check versions and re-fetch state).

Operational concerns

Cleanup. Delete or archive published rows on a schedule; partition by time for high volume. An unbounded outbox bloats and slows claiming.
Poison messages. After N attempts, move to a dead-letter state with the error and alert — don't retry forever silently.
Monitoring. Oldest unpublished event age (the key SLO), backlog size, publish error rate, and for CDC, replication slot lag. (How to Know When Your App Is Down.)
Payload size. Keep events small; put large data in storage and reference it.
Privacy. Events copy data into other systems and logs; include only what consumers need. (GDPR Basics for App Builders.)

When you don't need it

If the "event" is just a background job in the same database, transactional enqueue into a Postgres-backed job queue already gives you the atomicity, with no relay. The outbox earns its keep when events leave your database: a broker, another service, a third-party API. (Monolith vs Microservices.)

EasySpawn runs your app, a relay worker, and managed Postgres in one persistent workspace — so an outbox needs no extra infrastructure, and Claude Code can test crash-and-retry behaviour against the real database. See how it works or join the waitlist.