The outbox row belongs in the business transaction

An API marks an invoice as paid, commits the row, and then publishes an event for the rest of the system.

The database write succeeds. The process dies before the broker accepts the message.

The invoice is paid, but fulfilment, email, analytics, and every other consumer still believe it is open. Retrying the HTTP request might publish the event, or it might charge or mutate something twice. Reading application logs might explain the gap, but logs do not close it.

This is the small, brutal failure that the transactional outbox solves. Write the business change and the promise to publish its event in the same database transaction. A separate worker can deliver the event later. The system may publish more than once, but it no longer has to guess whether a committed business fact was ever scheduled for publication.

Start with the failure window #

The naive sequence usually looks reasonable:

begin transaction
update invoice set status = paid
commit transaction
publish invoice.paid to broker

There is a gap between commit and publish. No amount of careful ordering removes it. Publishing before commit creates the opposite failure: consumers can observe an event for a transaction that later rolls back.

Distributed transactions can coordinate some database and broker combinations, but they add operational coupling and are unavailable or undesirable in many systems. The outbox takes a simpler position. The database is authoritative for both the business fact and the intent to communicate it.

Inside one transaction, the application writes two rows:

begin;

update invoices
set status = 'paid', paid_at = now()
where id = $1 and status = 'open';

insert into outbox_events (
  event_id,
  aggregate_type,
  aggregate_id,
  event_type,
  payload,
  created_at
) values (
  $2,
  'invoice',
  $1,
  'invoice.paid',
  $3,
  now()
);

commit;

Either both writes commit or neither does. A crash after commit leaves a durable outbox row for a worker to find. A crash before commit leaves no paid invoice and no event promise. The dangerous half-state disappears.

Treat the outbox as application state #

Teams sometimes implement an outbox as a debug table beside the real system. That weakens the design. The outbox is part of the write contract. Its schema, retention, access pattern, and failure behavior deserve the same care as the business row.

Give every event a stable identifier at creation time. Include the aggregate identity, event type, creation time, and a payload version. Store enough information for publication without asking the worker to reconstruct the past from rows that may have changed since the transaction committed.

That last point matters. Suppose the worker loads the current invoice when it publishes an event created ten minutes ago. The invoice may now be refunded. The resulting invoice.paid payload describes present state while claiming to represent an earlier transition. The outbox should preserve the event facts that were true when the business transaction happened.

Do not put secrets, unnecessary personal data, or a complete database snapshot into the payload. The event needs the minimum durable facts required by its contract. Consumers that need current detail can fetch it through an authorised path, while accepting that current state may have moved beyond the event.

Claim rows without creating a second outage #

The publisher worker needs a safe way to find pending rows. Several workers may run at once, one may crash after claiming work, and a broker acknowledgement may arrive after the local lease expires.

A practical relational pattern uses a short claim inside a transaction, often with row locking that skips work already held by another worker. The claim records a worker identity and lease expiry. Publication happens outside the database transaction so a slow broker does not hold row locks for the whole network call.

The worker then records the result with a conditional update that still matches its claim. If the worker dies, the lease eventually makes the row eligible again. If an older worker returns late after another worker has reclaimed the row, it cannot overwrite the newer claim casually.

This design still permits duplicate publication. A worker can publish successfully and die before recording completion. The next worker sees the row as pending and publishes it again. That is not a defect hidden by the outbox. It is the delivery contract the outbox makes explicit: durable at-least-once publication.

Consumers still need idempotency #

The outbox closes the missing-event window. It does not provide exactly-once effects across every consumer.

Each consumer should record the event identifier or enforce a business idempotency boundary before applying its side effect. A fulfilment service might insert the event ID into a processed-events table in the same transaction that creates the shipment. A projection can upsert a versioned aggregate. An email service can use a deterministic delivery key so the same event does not create two messages.

The useful distinction is between duplicate delivery and duplicate effect. Brokers, workers, and networks may produce duplicate delivery. Consumer design decides whether that becomes a duplicate shipment, charge, notification, or row.

Do not generate a new event ID on every publish attempt. The outbox row represents one logical event. Attempts need their own operational identity and timestamps, but the event identity must remain stable so consumers can recognise repeated delivery.

Observe age, not only queue size #

An outbox can look healthy while an important row is stuck. Queue depth may stay low because newer events publish quickly around one poison payload. A worker can also keep retrying the oldest row without making useful progress.

Measure the age of the oldest unpublished event, publish latency by event type, attempt count, last error class, and the number of rows whose next attempt is overdue. Those signals answer whether committed business facts are still waiting to become visible elsewhere.

Add a dead-letter or blocked state only when it preserves the event and its history. Moving a row out of the normal scan must not make it disappear from operational responsibility. The record should retain the stable event ID, payload version, business aggregate, last broker response, retry history, and the exact condition for another attempt.

Alerts should follow business consequence. A delayed analytics event and a delayed fulfilment event may have different urgency even when they share the same publisher. Event type, age, and destination give the operator a better decision than one total pending count.

Make repair append to history #

Manual replay should not reset the outbox row until it looks new. Preserve the failed attempts and add another attempt with the operator, reason, and evidence that made retry appropriate. If the payload itself is invalid, create a corrected event under a new identity and link it to the superseded one. Editing a committed event in place destroys the record consumers and operators use to explain what happened.

Retention needs the same honesty. Published rows do not have to remain forever, but deletion should follow a policy that accounts for consumer deduplication windows, audit needs, incident investigation, and storage cost. Removing the producer record before consumers can safely forget the event ID creates a mismatch in the system's memory.

The transactional outbox is not glamorous infrastructure. It is a precise answer to one failure window. Commit the business change and its promise to notify together. Publish outside the transaction. Expect duplicate delivery. Make consumer effects idempotent. Watch the oldest unpublished promise, and keep failed attempts legible.

Stack Dispatch treats publication as durable workflow rather than a hopeful network call. The same principle applies whether the event announces an invoice, a shipment, a document, or a new post. If the business fact matters after the process dies, the outbox row belongs in the transaction that created it.