Archive / Topic
#reliability
30 posts, newest first
The outbox row belongs in the business transaction
Reliable publication starts by committing the business change and its promise to notify as one durable fact
A compensation record should preserve the uncertainty it closed
Reversing a side effect is safer when the case still explains what could not be proved about the original delivery
A customer timeline should separate facts from inference
Support trust improves when the case history clearly distinguishes durable events from the team's later interpretation of those events
Three facts a replay screen must show before resend
A replay control gets safer when the operator sees identity, surviving evidence, and duplicate risk before motion begins
The last lookup window should stay on the case
A recovery handoff gets weaker when the record preserves a receipt but drops the deadline after which that receipt stops being the safest next source of truth
Replay blocks should name the boundary
Recovery gets slower and riskier when the record says replay is unavailable but hides the exact condition that makes another send unsafe
Keep the provider reference on the failed row
A failed delivery is easier to explain and recover when the queue keeps the remote reference beside the failure state instead of hiding it in logs or side lookups
When retries stop, keep the last remote evidence
A failed delivery is easier to recover, explain, and replay safely when the dead-letter handoff preserves the last thing the remote system actually told us
Cancel still needs an outcome record
Stopping a stuck delivery is safer when the system preserves what the cancel action was trying to prevent, what the original attempt might still do, and how later evidence should be interpreted
Ask for the lookup path before you ask for a retry
A delivery system becomes easier to trust when manual recovery starts with evidence lookup, not with another send button
Start duplicate disputes with the first receipt
Support and recovery both get weaker when a duplicate report begins with the second visible event instead of the first durable proof the system ever recorded
Support should inherit the receipt trail
Delivery disputes get slower and riskier when support and operations read different evidence for the same uncertain request
The recovery window should start with the oldest safe proof
When a delivery system recovers from uncertainty, the first usable record is often the oldest piece of proof that still survives intact
When retries stop, leave a handoff worth reading
When automatic delivery gives up, the useful system preserves what was tried, what stayed uncertain, and what the next operator needs before touching replay
Rate limit headers should describe the next safe attempt
A 429 response is more useful when it tells the caller how recovery actually works, not only that a threshold was crossed
A manual replay needs the reason that justified it
An operator-triggered replay is safer when the system preserves why the original uncertainty was judged replayable at all
A 202 response needs a receipt trail
Accepted is a queueing answer, not proof that the later work finished, so the system still needs a trackable chain of receipts after the first response
Receipts should outlive the retry budget
A delivery record stays useful only if the acknowledgement evidence survives long enough for replay, support, and audit decisions
Timeouts should keep their doubt
A delivery timeout should preserve uncertainty clearly enough that the next operator does not confuse missing evidence with a confirmed failure
Dead letter queues should keep the reason
A failed delivery is more reviewable when the queue preserves why the handoff stopped, not only the payload that did not move
Replay starts with the original request ID
A safe replay path keeps the first delivery visible, refuses blind retries, and records the second attempt as its own event
Webhook retries need receipts
Reliable delivery starts by recording what the receiver accepted, not by sending the same payload louder
Idempotency comes before retries
Reliable dispatch systems make repeated delivery safe before they make repeated delivery fast
Why AI agents fail in production: state drift, not prompt drift
A practical state-convergence playbook for project-scoped agent systems
Credential leak response needs a prepared path
Credential leak response must be pre-designed. Improvisation guarantees longer exposure windows.
Edge reliability starts before application code
Why most production outages in small API platforms happen at the edge layer, not in business logic
Clever deployment pipelines hide the hard parts
Reliable delivery pipelines optimize for repeatability and recoverability, not cleverness
Investigation starts with the records you keep
Audit and observability are data models first, dashboards second
Start rate limits where the system can explain them
Build simple, visible, enforceable limits before you build complex ones
Background jobs need a cleanup contract
Why lifecycle integrity in stateful systems depends on explicit, observable maintenance jobs
Often tagged alongside reliability