Rate limiting and idempotency

Two problems, one root cause

Clients retry. Sometimes because you asked them to, sometimes because a timeout fired even though the request succeeded, sometimes because someone wrote a while(true).

Rate limiting protects you from too many requests. Idempotency protects the client from the same request being processed twice. You need both, and they are usually built together.

Rate limiting

Token bucket is the right default. A bucket holds N tokens and refills at R per second; each request consumes one; empty bucket means 429. It permits bursts up to N while bounding the sustained rate to R, which matches how real clients behave — bursty, then idle. A sliding-window counter is the common alternative: cheaper, approximate, no burst allowance.

A token bucket with capacity 20 refilling at 2/s: bursts are absorbed until the allowance is gone.

Choose what to limit by, in order of preference: authenticated user or API key (best — accurate and fair), then IP (blunt, and shared NATs mean an office building is one IP), then endpoint (expensive endpoints deserve tighter limits than cheap ones). Most real APIs combine them.

It must be shared state. Per-instance counters multiply your limit by your instance count. Redis with an atomic INCR, or a Lua script when the logic needs several steps — otherwise you have the read-modify-write race again.

Tell the client what happened. Return 429 with:

Retry-After: 30
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 1755859200

Without these, a well-intentioned client retries immediately and makes things worse. With them, it backs off correctly.

Decide your failure mode. When Redis is unreachable: fail open (serve everything, risk overload) or fail closed (reject everything, cause an outage)? Most APIs fail open with a conservative in-process fallback. Decide deliberately rather than discovering it during an incident.

Idempotency

An idempotent operation produces the same result whether applied once or five times. DELETE /orders/42 is naturally idempotent. POST /payments is emphatically not — and that is the one that matters.

The scenario: a client posts a payment, your server charges the card, the response is lost to a network blip, the client retries, the customer is charged twice.

The idempotency key pattern:

1. The client generates a UUID per logical operation and sends it as Idempotency-Key.
2. On receipt, you attempt to insert that key into a table with a unique constraint, in the same transaction as the work.
3. Insert succeeds → this is new. Do the work, store the response body and status against the key, commit.
4. Insert violates the constraint → you have seen it. Return the stored response. Do not do the work again.
5. Key seen but still in progress → return 409 Conflict and let the client retry shortly.

Two details people miss:

• Store the response, not just the key. A retry should get the original result, including the resource id — otherwise the client still cannot reconcile.
• Hash the request body against the key. If the same key arrives with different parameters, that is a client bug: return 422, do not silently serve the old response.

Expire keys after 24 hours; that is longer than any legitimate retry window.

Gotchas

• Idempotency keys must be scoped per API key, or one tenant can probe another's keys.
• Never make the key derivable from the payload alone — two genuinely separate identical payments must be allowed.
• Rate-limit the auth endpoints hardest. Credential stuffing is the most common attack you will actually see.
• Consumers of a queue need this too: at-least-once delivery means duplicates. See queues and async.
• Do not 429 your own health checks or internal traffic. Exempt them explicitly.

Prove you know it

Add Idempotency-Key support to one write endpoint, then send the same request twice with the same key and confirm exactly one record is created and both responses are identical. Then send the same key with a changed body and confirm it is rejected.