Idempotency Keys

🔴 P0 — Stripe’s public API is built on this; the implementation of idempotency for APIs

Problem #

A client sends POST /v1/charges and gets a timeout. Did the charge succeed? The client must retry, but retrying might create a duplicate charge. Idempotency keys solve this.

Instinct #

The idempotency key is the client’s contract with the server: “this is the same logical operation.” The server’s job is to guarantee that the operation’s side effects happen exactly once regardless of how many times the key is presented.

In the context of HTTP, it’s primarily for the POST endpoint. The header is ignored for the others that are supposed to be idempotent by default.

Mechanism #

Here’s an example of using idempotency keys in the context of an API like Stripe’s model.

Client → POST /v1/charges
         Header: Idempotency-Key: "key_abc123"
         Body: {amount: 5000, currency: "usd", customer: "cus_xyz"}

Server flow:
  1. BEGIN TRANSACTION
  2. SELECT * FROM idempotency_keys WHERE key = "key_abc123"
  3. If found AND completed → return stored response (200 + original body)
  4. If found AND in-progress → return 409 Conflict (concurrent request)
  5. If not found:
     a. INSERT INTO idempotency_keys (key, request_hash, status='started')
     b. Execute the charge
     c. UPDATE idempotency_keys SET status='completed', response=...
  6. COMMIT
  7. Return response

Here, the Idempotency-Key HTTP-header is used, it’s typically ignored for GET/PUT/DELETE (since they’re already expected to be idempotent by default) – the use of this Header is part of an IETF draft as of 2025, but it seems to be a non-standard for browsers (ref MDN docs)

Design Decisions #

Following the HTTP verb-table is more of a rule of thumb, simple heuristic.

A key framing that applies in more situations for this is that idempotency keys are more about the atomic phases and foreign state mutations within our system (less about HTTP-verbs). Any operation that mutates state outside the local ACID boundary (e.g. a Stripe charge call, push notification, an email send…) is a place where there can be ambiguity on “did this actually happen?” in face of retries, regardless of the HTTP verb.

Data-modelling #

It’s typically like a state-tracking table, the meta contents is business-context specific.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

CREATE TABLE idempotency_keys (
  id               BIGSERIAL PRIMARY KEY,
  idempotency_key  TEXT NOT NULL CHECK (char_length(idempotency_key) <= 100),
  created_at       TIMESTAMPTZ NOT NULL DEFAULT now(),
  last_run_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
  locked_at        TIMESTAMPTZ,                    -- concurrency guard

  -- fingerprint of the incoming request
  request_method   TEXT NOT NULL,
  request_path     TEXT NOT NULL,
  request_params   JSONB NOT NULL,

  -- checkpoint for resumability
  recovery_point   TEXT NOT NULL DEFAULT 'started', -- e.g. 'started' → 'ride_created' → 'charge_created' → 'finished'

  -- cached terminal response
  response_code    INT,
  response_body    JSONB,

  UNIQUE (idempotency_key)   -- or (account_id, idempotency_key) — see below
);

Some interesting pointers:

1. Better to NOT FK-link it to the actual domain-graph, better to keep it as a side-table that sites outside of the entities and relations of the core-schema – so the linking happens implicitly via the contents of the response_body and recovery_points

   this allows the idempotency key to exist and be queryable before the actual entity/relationship exists

2. The recovery point requires that a logical operation is deconstructed to its atomic phases, which are separated by foreign state mutations.

   Honestly, this is just classic state-machine management. Each transition in this SM is transaction-wrapped – so recovery_point allows a retried request to skip straight past phases already committed instead of needing to rerun the entire lifecycle

      started
        → (local txn: create ride, in state "pending")           → recovery_point = 'ride_created'
        → (foreign call: POST to Stripe, create the charge)        [not atomic — can fail/hang]
        → (local txn: attach charge_id to ride, mark ride "paid")  → recovery_point = 'charge_created'
        → (foreign call: send receipt email)                       [not atomic]
        → (local txn: mark ride "completed")                       → recovery_point = 'finished'

   Code Snippet 2: example of phases in the context of a ride-hailing platform

3. For idempotency table in multi-tenancy situations, the uniqueness constraints needs to use a composite of the key for idempotency AND for the tenant e.g. UNIQUE (tenant_id, idempotency_key), global unique keys may collide otherwise

4. At a large enough scale (e.g. Stripes scale), table cleanups should be batched, e.g. via a CTE like so :

      WITH deleted AS (
          DELETE FROM idempotency_keys WHERE id IN (
            SELECT id FROM idempotency_keys WHERE created_at < @horizon LIMIT @batch_size
          ) RETURNING *
        ) SELECT count(*) FROM deleted;

   Code Snippet 3: example of batched cleanup @ large scale using CTEs

Fingerprint-checking and Request-body Hashing #

There’s a need to guard against client-side bugs/misuse of idempotency keys e.g. usage of a single idempotency key across multiple requests where the request body content is not the same (for whichever reason, adversarial or just misuse of API). This is guarded using fingerprint = hash(req_method, req_path, req_params).

References #

• Stripe: Idempotent Requests
• Implementing Stripe-like Idempotency Keys in Postgres — Brandur Leach
• Designing Robust and Predictable APIs with Idempotency — Stripe Engineering