A retry budget should name its final answer

A client times out while calling an API. It tries again. The second attempt also times out. The configured retry budget is now exhausted. What should the client tell the person or service that asked it to act?

Too many retry policies answer only how often to send another request. The final response matters just as much. When a budget closes, the caller needs to know whether the operation failed, succeeded, or remains unresolved. A generic retry limit exceeded error describes local effort. It does not describe the remote outcome.

Count attempts, then classify evidence #

A retry budget may limit attempts, elapsed time, or both. Those limits protect latency and remote capacity. They do not turn an uncertain command into a known failure.

Imagine a create request that reaches the server, commits, and loses its response. The client repeats with the same idempotency key, but the second response is also lost. When the budget ends, the client knows it sent two attempts. It does not know whether the object exists. Reporting operation failed invites a fresh command and a possible duplicate. Reporting success invents evidence.

The honest final state is unresolved, with the idempotency key or operation reference needed to reconcile it. The next action can be a lookup after a delay, an operator review, or a later retry under the same key if the contract permits it. The important point is that the client preserves the identity of the original command.

Separate local exhaustion from remote rejection #

A remote service may return an explicit terminal rejection before the budget ends. A validation error, for example, should usually stop attempts immediately. The result is not retry exhausted. It is rejected, with the bounded reason the service supplied.

A temporary response can mean something else. A rate limit may carry a retry time. A service unavailable response may justify another attempt within the budget. A transport timeout may leave the remote effect unknown. These conditions should not be collapsed into one counter, because the final answer and recovery path differ.

The budget implementation can keep a compact decision record: attempt count, last confirmed response class, last timeout, elapsed time, operation identity, and whether any attempt could have crossed the side effect boundary. That record is useful to the caller and to an operator. A raw list of HTTP codes without outcome classification is harder to act on.

Make the last attempt worth taking #

A final attempt should have enough time to complete and leave a response. If the total deadline has almost expired, starting a request that cannot finish before the caller gives up may add load and uncertainty without a realistic chance of resolving either.

Backoff and jitter spread attempts over time, but they need to fit inside the caller's deadline. A retry policy that spends most of its budget sleeping and then launches an impossible last attempt may obey the attempt count while doing little useful work. Before each attempt, compare the remaining time with the request timeout and any minimum useful response window.

Retries also need a safety rule. A read is often safe to repeat. A command with an idempotency key can be safe to repeat when the server binds the key to the same validated request and preserves the first outcome. An unkeyed money movement or irreversible action needs a different recovery path. A generic retry decorator should not decide that boundary by itself.

Return a state the caller can use #

An exhausted retry budget can end in several legitimate states. Confirmed success includes the remote result or a stable reference. Confirmed rejection includes the remote reason and should not invite another identical attempt. Unresolved means the client has no final remote answer and must preserve the operation identity. Local cancellation means the caller stopped before sending the command or knows the remote boundary was not crossed.

Those states should appear in the public API contract or client library, not only in internal logs. A consumer deciding whether to show a receipt, offer a retry button, or ask for support needs the distinction. If the library returns only an exception called MaxRetriesExceeded, each consumer will invent its own guess about whether another click is safe.

Test the closing boundary directly. Drop the response after the remote side effect. Exhaust the budget. Verify that the client returns unresolved with the original key. Then make the reconciliation lookup succeed and verify that the first outcome is recovered without a new command. Also test terminal rejection and cancellation before send. Those cases expose whether the retry policy describes reality or merely counts loops.

Stack Dispatch deals in work that crosses service boundaries, where an absent response can leave the caller guessing. The retry budget should end with an answer about evidence, even when the answer is uncertainty. Count attempts to protect systems. Classify the final outcome so people can act safely after the counting stops.