All posts
message orderingqueuesdelivery contractsdistributed systems

Name the boundary before you promise ordered delivery

An ordering guarantee is useful only when the producer and consumer agree on which messages belong to the same sequence

Ordered delivery sounds like a complete promise.

Message A was accepted before message B, so the consumer will see A first.

Then the questions begin.

Across every customer or only within one customer? Across retries? Across two producers? Across a partition move? Does completion stay ordered, or only receipt? What happens when A cannot be delivered but B is healthy?

If the contract cannot answer those questions, ordered is not a guarantee. It is a comforting adjective.

Name the boundary before you promise ordered delivery.

One sequence needs one explicit key

Most systems do not need every message in one global order. They need related messages to remain ordered where changing the sequence would alter meaning.

An account-created event may need to precede account-updated events for the same account. A document revision may need to arrive after the revision it replaces. Two commands affecting one customer balance may need a stable sequence. None of those requirements implies that an unrelated account, document, or customer should wait behind them.

The delivery contract should therefore name the ordering key. It may be an account ID, aggregate ID, conversation ID, or another stable identity that both producer and consumer understand.

ordering-key: account_742
sequence: 18
message-id: msg_91f
ordering-scope: accepted messages for this account
completion-order: outside this contract
gap-policy: hold for lookup, then dead-letter after deadline

This is stronger than setting an internal partition key and assuming the public promise is now obvious. The partition is an implementation choice. The ordering key is part of the contract.

That distinction matters when the implementation changes. A queue can add partitions, move tenants, or replace a broker while preserving the same per-account promise. It can also keep the same broker and accidentally break the promise by changing how keys are derived. Consumers should not have to reverse-engineer infrastructure to understand which messages belong to one sequence.

Receipt order and completion order are different

A dispatcher can deliver A before B and still observe B finishing first.

The consumer may process messages concurrently. A may call a slower dependency. B may take a fast local path. The first request reaches the consumer first, but the second effect becomes visible sooner.

If the business rule requires ordered completion, transport order alone is insufficient. The consumer needs serial processing for the ordering key, version checks, or another guard that prevents B from committing before A reaches the required state.

The contract should say which boundary it covers.

Accepted in order means the dispatcher presents messages in sequence. Started in order means the consumer begins A before B. Completed in order means later effects cannot become durable before earlier effects for the same key. These promises have different cost and failure behaviour.

Many integrations need only the first promise plus a version field. The consumer can reject or defer a message whose expected predecessor is missing. Other workflows genuinely need serial completion. The right answer follows the effect, not the queue feature list.

There is also an acknowledgement boundary. If the consumer accepts A and returns success before its effect is durable, the dispatcher may safely move to B while the application still has not completed A. A precise contract names whether acknowledgement means received, persisted, or fully applied.

Gaps are where the promise becomes real

Ordering is easy while every message succeeds on the first attempt.

The guarantee becomes expensive when A fails and B is ready.

One policy blocks the sequence. B waits while A retries. This preserves order but allows one poison message to stop every later message for the same key.

Another policy moves A to a dead-letter path and releases B. This restores movement but creates a gap the consumer must be able to recognise. If B assumes A completed, the system has traded queue delay for a quieter consistency failure.

A third policy asks the producer or consumer to resolve the missing state through lookup. B carries sequence 18, the consumer has only applied 16, and sequence 17 can be fetched or reconstructed before B proceeds. This can be robust, but only when lookup is a real supported part of the delivery contract.

The important thing is not that every product chooses the same policy. The important thing is that the policy is visible before failure.

The delivery record should show the ordering key, expected sequence, last acknowledged sequence, current gap, and the action that can retire it. An operator should be able to tell whether retrying A preserves the same message identity, whether releasing B weakens the guarantee, and whether the consumer has another source of truth.

Retries must remain inside the same sequence position. Creating a new sequence number for each attempt turns transport recovery into a new logical message. Reusing the position while changing the payload creates a different problem: the consumer cannot know whether a duplicate is truly the same command. Stable message identity and immutable sequence content belong together.

Parallelism should be visible in the promise

Teams often discover the real ordering boundary when throughput becomes uncomfortable.

A global sequence is simple to describe and expensive to scale. Per-key ordering allows unrelated work to move concurrently, but a hot key can still become its own bottleneck. Wider keys reduce contention and weaken the guarantee. Narrower keys increase concurrency and ask consumers to handle more cross-key coordination.

Those are product decisions, not merely broker tuning.

If the public contract says events for one account are ordered, changing the key to account plus event type may improve throughput while breaking consumers that rely on a single account sequence. If the contract already says ordering is per document, two documents can move independently without surprise.

Monitoring should follow the same boundary. A global queue depth can look healthy while one ordering key is blocked behind a failed message. Useful operations data includes the age of the oldest blocked key, the number of later messages waiting behind a gap, and the reason the gap has not retired. Those facts show where the promise is costing time.

At Stack Dispatch, we care about delivery contracts that stay legible when retries, concurrency, and failure enter the path. Ordered delivery is valuable when it names the sequence identity, the completion boundary, and the gap policy that preserves or deliberately relaxes the promise.

Name the boundary first. Then the producer knows which messages it is sequencing, the consumer knows what it may safely assume, and operations knows exactly what has stopped when one message cannot move.

0 comments

Join the conversation

Get the next dispatch

New writing on software architecture, AI systems, and shipping production software, sent by email. Unsubscribe anytime.