Archive
74 posts published
Keep read progress separate from dispatch acknowledgement
A reader position helps someone resume a log. An acknowledgement records who has accepted the work
Give a changing list a stable reading point
An API cursor should say what ordering and visibility it preserves while new records arrive
Put the operation key in the timeout response
A timeout can leave a remote command unresolved. Preserve the original operation identity so the caller can reconcile safely
A retry budget should name its final answer
A caller needs a useful outcome when the retry window closes, especially when the remote result is still uncertain
Return the first outcome when an idempotency key repeats
Duplicate suppression is only half the contract; callers also need a stable account of what the original request produced
An idempotency key can expire before its risk does
Retry safety depends on the lifetime of the original side effect, not only the convenience of a short deduplication window
The outbox row belongs in the business transaction
Reliable publication starts by committing the business change and its promise to notify as one durable fact
Separate request identity from attempt identity
Retries stay auditable when the customer's command keeps one identity and every execution receives another
A dead letter needs an owner before it needs a replay button
Replay is an operational decision about unresolved side effects, not a convenience action for clearing a queue
Timeout budgets should leave room for the caller
A downstream deadline that consumes the entire request window leaves the upstream service no time to classify, record, or recover from the result
Queue depth hides the oldest promise
A backlog count can look healthy while one customer's message has been waiting far beyond the delivery contract
Name the boundary before you promise ordered delivery
An ordering guarantee is useful only when the producer and consumer agree on which messages belong to the same sequence
Reconciliation needs its own idempotency boundary
A recovery action can be safe to retry while still being unsafe to repeat after new evidence arrives
When a late receipt arrives after compensation, reopen the case
A delayed fact can change residual risk even when the customer's immediate problem was already resolved
A compensation record should preserve the uncertainty it closed
Reversing a side effect is safer when the case still explains what could not be proved about the original delivery
The case should keep the last honest unknown
Recovery records get safer when they preserve the one unresolved fact the team still cannot collapse into confidence
Before you replay, ask what the customer can already prove
Recovery decisions get narrower when the case starts with the evidence already visible to the customer
A customer timeline should separate facts from inference
Support trust improves when the case history clearly distinguishes durable events from the team's later interpretation of those events
Three facts a replay screen must show before resend
A replay control gets safer when the operator sees identity, surviving evidence, and duplicate risk before motion begins
The last lookup window should stay on the case
A recovery handoff gets weaker when the record preserves a receipt but drops the deadline after which that receipt stops being the safest next source of truth
Manual recovery should write down what it refused to assume
A recovery path is easier to trust when the record states which tempting assumptions were left open instead of silently acting as though uncertainty had already narrowed
Replay blocks should name the boundary
Recovery gets slower and riskier when the record says replay is unavailable but hides the exact condition that makes another send unsafe
A support escalation should keep the last durable write
A delivery dispute becomes harder to resolve when support inherits the latest error but not the last write the system knows actually landed
Support should inherit the lookup deadline
A failed delivery becomes harder to recover when support receives the receipt and error message but not the deadline after which the remote lookup path stops being the safest next move
If lookup still works, do not replay yet
A retry becomes much less trustworthy when the system still has a live remote lookup path but chooses to spend certainty later instead of using it now
A manual fix should keep the old receipt attached
When humans step in to repair a broken delivery path, the correction record is stronger if it stays linked to the original proof trail instead of starting a clean and misleading second history
Replay the oldest uncertain receipt first
When a backlog mixes accepted, unknown, and lookup-ready failures, queue order is a weak way to decide what deserves human attention first
The last status query belongs in the dead-letter record
When delivery retries stop, the handoff should preserve the best lookup-ready proof the sender already collected
ffmpeg-kit-react-native is dead. Here is a native iOS alternative.
The archived binaries return 404, WASM does not run on Hermes, and server-side encoding leaks your users' data. Apple already ships a hardware H.264 encoder in every iPhone. Here is a lean Expo module that uses it.
Keep the provider reference on the failed row
A failed delivery is easier to explain and recover when the queue keeps the remote reference beside the failure state instead of hiding it in logs or side lookups
When retries stop, keep the last remote evidence
A failed delivery is easier to recover, explain, and replay safely when the dead-letter handoff preserves the last thing the remote system actually told us
Cancel still needs an outcome record
Stopping a stuck delivery is safer when the system preserves what the cancel action was trying to prevent, what the original attempt might still do, and how later evidence should be interpreted
Ask for the lookup path before you ask for a retry
A delivery system becomes easier to trust when manual recovery starts with evidence lookup, not with another send button
Start duplicate disputes with the first receipt
Support and recovery both get weaker when a duplicate report begins with the second visible event instead of the first durable proof the system ever recorded
Separate shared health from fleet detail
Operational trust improves when broad service health stays visible to everyone while sensitive fleet detail fails closed to the right role
Running reproducible bioinformatics pipelines from your AI assistant
AI can already look up biological data and plan an analysis. What it could not do was run a pipeline and prove the result. FlowProof is an open MCP server that closes that gap with verifiable provenance.
Support should inherit the receipt trail
Delivery disputes get slower and riskier when support and operations read different evidence for the same uncertain request
After a provider timeout, protect the acceptance boundary
Recovery gets more dangerous when a timeout erases whether the remote side might already have accepted the request
A healthy registry can still point behind production
A healthy pull command can still move a service backward when the registry is no longer the system's most current source of truth
Flat error messages are weak replay evidence
A delivery system gets recovery wrong when a flattened failure message quietly steers the next operator toward duplicate side effects
Admin detail should fail closed
Broad operational visibility can be useful to many users, but infrastructure detail should disappear by default the moment privilege is unclear
The recovery window should start with the oldest safe proof
When a delivery system recovers from uncertainty, the first usable record is often the oldest piece of proof that still survives intact
A routine pull can revive stale production code
A convenient image tag becomes dangerous when a routine refresh command quietly acts like a rollback
When retries stop, leave a handoff worth reading
When automatic delivery gives up, the useful system preserves what was tried, what stayed uncertain, and what the next operator needs before touching replay
Rate limit headers should describe the next safe attempt
A 429 response is more useful when it tells the caller how recovery actually works, not only that a threshold was crossed
Idempotency needs a published lifetime
The retention window is part of the contract, because replay safety ends the moment the original decision record disappears
A manual replay needs the reason that justified it
An operator-triggered replay is safer when the system preserves why the original uncertainty was judged replayable at all
A 202 response needs a receipt trail
Accepted is a queueing answer, not proof that the later work finished, so the system still needs a trackable chain of receipts after the first response
When the error message lies about the failure
A production incident gets slower and riskier when the recorded failure names the wrong boundary
Receipts should outlive the retry budget
A delivery record stays useful only if the acknowledgement evidence survives long enough for replay, support, and audit decisions
Timeouts should keep their doubt
A delivery timeout should preserve uncertainty clearly enough that the next operator does not confuse missing evidence with a confirmed failure
Dead letter queues should keep the reason
A failed delivery is more reviewable when the queue preserves why the handoff stopped, not only the payload that did not move
Replay starts with the original request ID
A safe replay path keeps the first delivery visible, refuses blind retries, and records the second attempt as its own event
Webhook retries need receipts
Reliable delivery starts by recording what the receiver accepted, not by sending the same payload louder
Idempotency comes before retries
Reliable dispatch systems make repeated delivery safe before they make repeated delivery fast
Why AI agents fail in production: state drift, not prompt drift
A practical state-convergence playbook for project-scoped agent systems
Security friction needs a job
Security improvements that stick are the ones that reduce cognitive load for normal users while narrowing attack surface
Credential leak response needs a prepared path
Credential leak response must be pre-designed. Improvisation guarantees longer exposure windows.
Edge reliability starts before application code
Why most production outages in small API platforms happen at the edge layer, not in business logic
Clever deployment pipelines hide the hard parts
Reliable delivery pipelines optimize for repeatability and recoverability, not cleverness
Investigation starts with the records you keep
Audit and observability are data models first, dashboards second
MCP tools need API-grade contracts
Why MCP servers should be thin, reliable protocol adapters - not application logic hosts
SDKs earn trust by keeping semantics aligned
SDK quality is contract fidelity plus ergonomic defaults, not HTTP calls with types
Contract drift is an API bug
Why contract discipline is the fastest path to scalable integrations and lower support overhead
Run pgvector like production database infrastructure
What it actually takes to run Postgres with pgvector reliably in production
Duplicate storage quietly becomes infrastructure debt
Content-addressable deduplication and integrity checks for API platforms using object storage
Start rate limits where the system can explain them
Build simple, visible, enforceable limits before you build complex ones
Background jobs need a cleanup contract
Why lifecycle integrity in stateful systems depends on explicit, observable maintenance jobs
Retrieval output still needs a context contract
Search output is not prompt-ready context. Assembly is a first-class subsystem.
Retrieval works when vector search has backup
Why reliable retrieval needs layered fallback chains, not vector-only optimism
Reject secrets before they become records
Build secret detection into the write path before credentials spread through your data and logs
Schema discipline decides retrieval reliability
Schema quality determines long-term correctness more than any algorithm choice
A leaked API key should fail small
A production-grade model for key format, storage, scope, and rotation
Authorization is where multitenant systems earn trust
Why multi-tenant systems fail at authorization boundaries, and how to fix it