Manual recovery should write down what it refused to assume
A recovery path is easier to trust when the record states which tempting assumptions were left open instead of silently acting as though uncertainty had already narrowed
Manual recovery is often where a system stops being merely automatic and starts exposing its real operational judgment.
Something timed out.
A provider accepted slowly.
A callback never arrived.
Support needs an answer before the logs feel emotionally satisfying.
At that point the team usually has to choose what not to assume.
Did the remote side probably reject the request.
Did the local side probably fail before commit.
Did the missing receipt probably mean there was no side effect.
Those guesses can feel efficient.
They are also how duplicate sends, inconsistent customer answers, and weak postmortems quietly get created.
Recovery records should preserve the uncertainty that survived the incident
We think a manual recovery record should say what it refused to assume.
Not only what action the operator took.
Not only that the issue was investigated carefully.
The record should preserve the still-open boundaries that shaped the decision.
Suppose a provider timeout leaves no success response but does leave a provider reference.
One operator may decide not to replay because the reference keeps open the possibility that the remote side already accepted the request.
Another operator may choose lookup first because the missing answer is about status, not about identity.
A third may decide compensation is safer than resend because local side effects already committed.
Those decisions are different because the surviving uncertainty is different.
If the record only says manual recovery completed, the next person inherits the action without the reasoning.
That is a thin handoff.
It tells support what happened next while hiding what remained unknown at the moment the system chose that next step.
The uncertainty is often the most valuable part of the whole case.
It explains why the team did not resend yet.
It explains why lookup mattered more than logs.
It explains why a compensation path was opened before a retry path.
The refusal matters because teams are judged by what they did not guess past
A recovery system earns trust partly by the actions it enables and partly by the assumptions it blocks.
When operators say we did not assume the timeout meant failure, they are describing a safety boundary.
When they say we did not assume the callback absence meant no side effect, they are describing another one.
Those boundaries should survive the live moment because later readers still need them.
Support needs them for customer language.
Engineering needs them for design review.
Operations needs them for the next edge case that looks similar but is not identical.
A compact format is enough:
- refused to assume remote rejection because provider reference was already issued
- refused to assume no local write because the idempotency row reached pending commit
- refused to assume replay was safe because a prior manual resend remained unresolved
That list is not ceremony.
It is a way of making recovery judgment inspectable.
The next operator can see which uncertainty classes remained open and whether those open questions still justify the same boundary now.
If the lookup later clears one uncertainty, the team can narrow the restriction with confidence.
If the incident review decides one refusal was too cautious, it can improve the rule precisely instead of debating the whole recovery mood after the fact.
A reliable handoff keeps the missing certainty visible
The most fragile recovery records are the ones that sound confident while skipping the unresolved part.
Investigated and handled.
No resend needed.
Resolved after manual checks.
Those phrases make the case look tidy.
They also erase the exact uncertainty that forced a human to intervene in the first place.
We would rather keep the missing certainty visible.
That makes support updates sharper because the customer can be told what the team is still verifying.
It makes retries safer because a later operator can see whether the system was protecting against duplicate side effects, missing identity, or unresolved remote acceptance.
It makes product design better because the record itself exposes where automation still needed a human to carry doubt responsibly.
At Stack Dispatch, we care about this because delivery systems are trusted on their worst ambiguous days, not only on their clean successful ones.
Manual recovery should write down what it refused to assume because a good system does not only preserve the action it took.
It preserves the uncertainty it respected.
That is what lets the next decision stay careful without starting the whole incident from scratch.
0 comments