Exponential backoff for social API retries
How to implement exponential backoff correctly for retrying social platform API calls, and why naive fixed-interval retries make outages worse.
By André LugExponential backoff doubles the wait between retries — delay = base × 2^attempt, capped (e.g. 500ms, 1s, 2s, 4s… up to 30s) — with full jitter, a random delay between zero and the computed value, to break the thundering herd. Retry only errors marked retryable, cap the attempt count, and reuse the same idempotency key on every attempt.
"Just retry it" is the easiest fix to reach for and the easiest one to get wrong. A retry loop that fires on a fixed interval — every second, forever, until it works — behaves fine against a single flaky request and terribly against an actual outage: hundreds of clients all retrying on the same cadence turn a brief platform hiccup into a sustained wall of traffic that keeps the rate limiter (or the outage itself) from ever clearing. Exponential backoff with jitter fixes that, but only if you also know when to stop retrying at all — a retry that can never succeed is just wasted latency with extra steps.
PostGate handles OAuth, publishing and delivery evidence for every network through one API. Start free or read the docs.
Why does fixed-interval retry make outages worse?
Picture a platform having a brief availability blip. Every client hitting it gets an error at roughly the same moment and, if they all retry after exactly one second, they all hit it again at roughly the same moment too — a synchronized wave that repeats every second until the platform recovers, at a request rate that never actually drops. This is the same thundering-herd problem that makes retrying too aggressively against a 429 counterproductive, and it applies to any transient failure, not just rate limits.
What is the exponential backoff formula?
Each retry waits roughly twice as long as the previous one: delay = base × 2^attempt, usually capped at some maximum so attempt 10 doesn't wait an hour. A typical schedule with a 500ms base and a 30-second cap looks like 500ms, 1s, 2s, 4s, 8s, 16s, 30s, 30s… — spreading load out over time instead of hammering the same window repeatedly.
Why do you need jitter?
Exponential backoff alone doesn't solve synchronization — if every client started failing at the same moment, a pure exponential schedule still has them all retrying together at every step, just further apart. Full jitter — picking a random delay between 0 and the computed exponential value on every attempt — spreads retries across the whole window instead of clustering them at its edge, which is what actually breaks the thundering herd.
async function callWithBackoff(fn, { maxRetries = 5, baseMs = 500, maxMs = 30_000 } = {}) {
let attempt = 0;
while (true) {
try {
return await fn();
} catch (err) {
// isRetryable comes from your error classification — see below
if (!isRetryable(err) || attempt >= maxRetries) throw err;
const capped = Math.min(maxMs, baseMs * 2 ** attempt);
const jittered = Math.random() * capped; // full jitter: [0, capped)
await new Promise((resolve) => setTimeout(resolve, jittered));
attempt += 1;
}
}
}When should you NOT retry?
Backoff scheduling only matters for errors worth retrying at all. PostGate's error envelope carries a retryable boolean specifically so your client doesn't have to guess from the HTTP status code or parse a message string — see Errors & retries. A code like media_invalid or platform_content_rejected means the request itself needs to change; retrying the identical payload will fail the same way every time and just adds latency. A code like profile_disconnected needs the customer to reconnect their account — no retry schedule fixes that either. Only rate_limited and platform_transient are marked retryable: true, and even a reused idempotent request should stop retrying on a 409 conflict, which is never retryable by definition.
function isRetryable(err) {
// PostGate's structured error envelope
return err?.error?.retryable === true;
}
// Example: a non-retryable error PostGate returns as-is
{
"error": {
"code": "media_invalid",
"message": "The provided image could not be processed.",
"platform": "instagram",
"retryable": false,
"request_id": "req_01JXYZ"
}
}How do you retry safely, not just slowly?
Backoff controls when you retry; it says nothing about whether a retry is safe. For any create-post request, reuse the same Idempotency-Key across every attempt in the loop above. PostGate honors that key for 72 hours, so a retried request after a timeout returns the original result instead of creating a second post — the backoff schedule and the idempotency key are meant to be used together, not as alternatives to each other.
One more distinction worth keeping straight: PostGate already retries provider delivery internally on your behalf for transient platform failures. Client-side retries with the backoff above are for API transport failures and request-acceptance errors — a timeout before you got a response, or a retryable: true rejection at the API layer — not a second, competing retry loop stacked on top of PostGate's own.
Frequently asked questions
Why is retrying on a fixed interval a bad idea?
Because it synchronizes clients into a thundering herd. When a platform has a brief availability blip, every client gets an error at roughly the same moment — and if they all retry after exactly one second, they all hit it again together, in a wave that repeats until the platform recovers, at a request rate that never actually drops. That sustained wall of traffic keeps the rate limiter or the outage itself from clearing.
What is jitter and why does backoff need it?
Exponential backoff alone does not solve synchronization — if every client started failing at the same moment, a pure exponential schedule still has them all retrying together at every step, just further apart. Full jitter picks a random delay between zero and the computed exponential value on every attempt, spreading retries across the whole window instead of clustering them at its edge. That randomness is what actually breaks the thundering herd.
Which social API errors should never be retried?
Errors where the request itself must change: media_invalid or platform_content_rejected will fail identically on every attempt. Errors needing human action: profile_disconnected requires the customer to reconnect their account. And a 409 conflict is never retryable by definition. In PostGate’s error envelope only rate_limited and platform_transient carry retryable: true — check that boolean instead of guessing from the HTTP status or parsing message strings.
Should my client retry if the API already retries internally?
Yes, but for a different layer. PostGate already retries provider delivery internally for transient platform failures; client-side backoff is for API transport failures and request-acceptance errors — a timeout before you got a response, or a retryable: true rejection at the API layer — not a second, competing retry loop stacked on top. And reuse the same Idempotency-Key across every attempt: it is honored for 72 hours, so a retried request returns the original result instead of creating a second post.
