All posts
idempotencyAPI designretriesdistributed systems

Return the first outcome when an idempotency key repeats

Duplicate suppression is only half the contract; callers also need a stable account of what the original request produced

A client submits a command with an idempotency key. The server completes the work, commits the result, and loses the response on the way back.

The client retries with the same key. The server correctly refuses to perform the work twice. Then it returns a generic duplicate error.

The side effect is safe, but the caller still does not know what happened.

An idempotency contract should preserve more than the fact that a key was seen. It should preserve enough of the first outcome to answer later attempts consistently. When the same valid request returns, give the caller the result it missed, or a durable reference that can retrieve that result.

Suppression without an outcome leaves the request unresolved

Duplicate suppression protects the server from repeating a business effect. It does not automatically repair the caller's uncertainty.

Consider a create operation that returns an identifier, status, and location. If the first response disappears, a later 409 Duplicate tells the caller not to retry again. It does not tell the caller which resource exists, whether the operation fully completed, or how to continue the workflow.

That ambiguity often creates a second recovery path. The caller searches by approximate attributes, asks an operator to inspect logs, or invents another endpoint to discover the object. Those paths are slower and less reliable than returning the result already associated with the idempotency record.

The useful contract is simple: same key, same validated request, same observable outcome.

Store a response record, not every transient detail

Returning the first outcome does not require keeping an unlimited byte-for-byte HTTP response forever. Store the durable parts of the result that define the business outcome.

For a creation command, that may be the resource identifier, terminal status, response code, and a stable representation version. For an asynchronous command, it may be the operation identifier and the endpoint the caller uses to inspect progress. For a rejected command, it may be the stable rejection class when repeating the request cannot change the answer.

Avoid replaying transient headers, trace identifiers, or temporary server metadata as though they still describe the current attempt. The second response can have a new request trace while carrying the original business result.

If the current representation may evolve, returning a durable resource reference is often cleaner than freezing a large response body. The idempotency record proves which resource the command created. The resource endpoint provides its current state.

Reject the same key with different input

Outcome replay is safe only when the repeated request represents the same command.

Bind the idempotency record to a canonical fingerprint of the meaningful request fields, the authenticated caller, and the operation scope. If the same key arrives with a different amount, destination, payload, or caller, reject it as a key conflict. Do not return the first result as if the new input matched.

The error should explain the boundary without exposing sensitive data. State that the key is already attached to a different request and require a new key for a new command. Keep enough server-side evidence for an operator to determine which fields differed when investigation is authorised.

This check closes a dangerous gap. A system that looks up only the key can accidentally turn a changed request into a successful-looking replay of unrelated work.

Preserve failure outcomes deliberately

Not every first attempt deserves permanent replay.

A validation failure before any business work begins may be safe to evaluate again because the request has not claimed the key. A terminal business rejection may deserve a stable recorded outcome. An internal timeout with unknown side effects should not be flattened into a clean failure or success.

Define when the idempotency record becomes authoritative. A common boundary is the point where the server accepts responsibility for the command and begins a durable transaction or operation. From there, later attempts should retrieve the stored status rather than start again.

Unknown outcomes need their own state. Return an operation reference or a response that tells the caller the command is still being reconciled. Do not invite a fresh key merely because the first attempt did not produce a tidy final response.

Let expiry preserve a recovery path

Idempotency records cannot always live forever, but expiry should not make the original result undiscoverable while its business effect still matters.

Before removing a full response record, consider retaining a compact mapping to the durable resource or operation. If the risk window outlives the response cache, the service can still tell a late retry that the key belongs to an earlier command and direct the caller to the surviving outcome.

Document the retention boundary for clients. A caller that may retry after days needs a different contract from a client whose retry budget ends after minutes. The server should not advertise idempotency as durable safety while silently forgetting the only result the caller can use.

The retry response is part of the API

Idempotency is often implemented as middleware and reviewed as storage. Callers experience it as an API contract.

Test the lost-response path explicitly. Complete the first request, discard its response, then repeat the same request with the same key. Verify that the business effect happens once and that the caller receives a usable outcome. Repeat with changed input and verify that the conflict is clear. Repeat while the first command is still running and verify that the response points to the same operation rather than starting another.

Those tests prove more than duplicate suppression. They prove that uncertainty can converge.

When an idempotency key repeats, the server already knows the caller may have missed the first answer. Return the original business outcome, a stable reference to it, or an honest in-progress state. Preventing the second side effect is necessary. Helping the caller recover the first result is what completes the contract.

0 comments

Join the conversation

Get the next dispatch

New writing on software architecture, AI systems, and shipping production software, sent by email. Unsubscribe anytime.