← Back to Daily Builds

Daily build note · October 6, 2026

Backoff Lab

Watch a crowd of clients retry against a rate-limited API, tune exponential backoff and jitter until the thundering herd calms down, then copy the retry policy.

AI Ops Interactive demo live Static HTML/CSS/JS

What shipped

Backoff Lab is a self-contained simulator for the moment an API hiccups and every client retries at once: LLM calls hitting 429s, webhooks redelivering after an outage, a fleet of phones reconnecting. Set how many clients are retrying, how spread out they arrive, how long the outage lasts, and the rate limit once the API comes back. Then tune the retry policy: base delay, multiplier, cap, max retries, and one of four jitter strategies (None, Full, Equal, Decorrelated). A seeded simulation replays every attempt, draws the load on the API against the dashed rate-limit line, and plots each client's attempts as a row of dots so you can literally see the herd move in lockstep. A verdict badge (CALM / BUMPY / HERD) sums up success rate, p95 time to success, peak load versus the limit, and requests per client. A side table reruns the same storm with every jitter strategy, a worst-case schedule shows how long a client could wait, and the policy is one click away as JS or Python code or a markdown receipt. Settings save in localStorage under backoff-lab-v1.

How to try

  • Open the live demo (iframe on the right, or the full-page link below). The LLM 429 storm preset loads first: 200 clients, no jitter, and a red HERD badge with most clients giving up.
  • Click Full jitter and watch the red spikes on the load chart melt into a band near the limit.
  • Try Mobile reconnect, then raise Max retries or the cap until the badge stops saying HERD.
  • Flip the code panel to Python and copy the policy, or copy the markdown receipt into a design doc.

Scope trim

No real network calls, no Retry-After header handling, no circuit breaker or token bucket on the client side, and no per-error-type policies. One storm, one policy, a deterministic simulation, and code you can paste. The point is to make the case for jitter visible in ten seconds.

Limitations

  • The server is modeled as fully down during the outage, then a strict limit per 100 ms slice; real APIs degrade more gradually.
  • Request latency is treated as zero, so attempts land exactly when the delay ends.
  • Decorrelated jitter uses the classic 3× growth and ignores the multiplier slider.
  • The swarm chart samples up to 90 clients so rows stay readable.

Suggested next steps

  1. Honor a Retry-After header and show how much it helps versus blind backoff.
  2. Add a client-side retry budget or circuit breaker and compare total wasted requests.
  3. Import a real 429 log (timestamps) and replay it against different policies.