Idempotent Tool Calls: How to Let AI Agents Retry Without Doing It Twice

Your agent calls the refund API. The request goes through, the response gets lost, the client times out, and the retry logic does what retry logic does. The customer is refunded twice. Idempotent tool calls are the fix: make the second attempt a no-op that returns the first result.

Below is a small fake refund API with a dropped reply, four ways of calling it, and what each one did. It is standard-library Python I ran on 2026-10-31. The API is invented for the demo; the Stripe behaviour I cite is from Stripe's own docs, read on the same date.

Why do agents retry the same call?

Three separate reasons, and you need to handle all of them:

1. Transport retries. Your HTTP wrapper retries on timeouts and 5xx. See retries and backoff for LLM APIs for the policy side.
2. Model retries. The model sees an error or an empty result and calls the tool again, sometimes with slightly different wording in the arguments.
3. Resume retries. The process died mid-call and a restarted run re-executes the step; that case is built in agent checkpointing.

Only the first is under your wrapper's control. A key in the tool's own code covers all three.

What does a duplicate look like?

The fake API creates a refund, then raises LostResponse once to simulate a timeout after the work was done. The client retries up to three times.

def scenario(title, key_for_attempt):
    api = RefundAPI(drop_first_reply=True)
    resp = call_with_retry(
        lambda i: api.create_refund("order_1042", 499, key=key_for_attempt("run7", 4, i))
    )
    print(f"  agent saw: {resp['refund_id']}   refunds on server: {len(api.refunds)}")

Output:

--- no key
  attempt 1: timed out waiting for response
  agent saw: re_2   refunds on server: 2
--- random key per attempt (common bug)
  attempt 1: timed out waiting for response
  agent saw: re_2   refunds on server: 2
--- key = run id + step number
  attempt 1: timed out waiting for response
  agent saw: re_1   refunds on server: 1

The agent saw re_2 in the first two cases and never learned that re_1 also exists. Its own logs say one refund. The payment system says two.

What is the random-key bug?

The second scenario is the one worth staring at. The code did use idempotency keys, so it looks careful in review:

key = str(uuid.uuid4())   # inside the function that makes the request

If that line runs on every attempt, each retry gets a new key, and the server rightly treats it as a new request. The key must be created once per intended action and reused across attempts. Put it outside the retry loop, or derive it from stable inputs:

key = f"{run_id}:{step}"

Stripe's docs suggest V4 UUIDs or another random string with enough entropy, which is correct for a client that generates the key once and stores it with the request. In an agent, the safest place to generate it is where the intent is recorded (the step number in your log), so a restarted process arrives at the same key. Stripe also says to avoid sensitive data such as email addresses in keys.

What should the server do with a repeated key?

If you build the API side, or you want to know what to expect from one, the rules are short. Stripe documents them for API v1, per its idempotent requests page as read on 2026-10-31:

• It saves the status code and body of the first request made with a key, after execution begins, and returns that saved response on retries, including 500 errors.
• It compares incoming parameters with the original and errors if they differ.
• Keys can be removed automatically once they are at least 24 hours old; after that a reused key creates a new request.
• Keys are up to 255 characters.
• Validation failures and requests that conflict with a concurrently executing one are not saved, so you can retry those.
• All POST requests accept keys; sending them on GET and DELETE has no effect.

The demo API follows the same shape. Reusing a key with different parameters:

--- same key, different amount
  Conflict: key 'run7:4' was used with different parameters

That conflict is a feature. If your key scheme ever maps two different actions to one key, you want a loud error, not a silently returned old result. In an agent, it also catches the case where the model retries with changed arguments: it should be a new action with a new step, or a bug you want to see.

Two consequences for you as the caller:

• The 24 hour window (for that vendor) means a run that resumes two days later cannot rely on the key. Log what you did and look it up instead.
• A saved 500 means a retry with the same key can return the same failure. If you want a genuinely fresh attempt, that is a new action and a new key, a decision your code should make deliberately.

Other vendors differ. Check your provider's page for window, scope and failure handling before relying on it. I only verified Stripe's.

What about two duplicates at the same time?

Retries are not always sequential. A model that issues parallel tool calls, or two workers resuming one run, can send the same key twice at once. The server must make check-then-act atomic. In the demo that is a lock; in a database it is a unique constraint on the key, inserted before the work starts.

--- two concurrent duplicates, same key
  responses: ['re_1', 're_1']   refunds on server: 1

Both callers got the same refund id and only one refund exists. If you write your own tool backend, the pattern is: insert the key row (unique), do the work, store the response on the row. A second insert fails, and the second caller either waits or gets "in progress, retry shortly". The wording of that failure matters to an agent, so make it say the action is already running and not to repeat it. See designing tools for agents for error messages models handle well.

The same principle protects inbound webhooks; Razorpay webhooks with idempotency in FastAPI has the database version with a real unique constraint.

What if the API has no idempotency keys?

Many do not. Then the retry question becomes: did the first attempt work? Ask before acting again.

The rule is look before you retry, and it only works if you put your own reference on the thing you create (an order id, a metadata field, a client-supplied id):

try:
    api.create_refund(order, amount)
except LostResponse:
    existing = api.find_by_order(order)
    if existing:
        print(f"found {existing[0]['refund_id']} already; not retrying")
    else:
        api.create_refund(order, amount)

Output from the demo:

--- API with no key support: look before you retry
  first attempt: timed out waiting for response
  found re_1 already; not retrying
  refunds on server: 1

Limits of this approach, stated plainly. There is a race between the lookup and the retry if something else creates the same thing. Lookup is only reliable if the system is consistent right after a write; some search APIs are eventually consistent, and a record may not appear for seconds. And if you have no reference to search by, there is nothing to look up. In that case treat the timeout as unknown and escalate, as in the approval gate.

Which tools are naturally idempotent?

Design for it where you can, so you need fewer keys:

Treating "already deleted" or "already exists with identical contents" as success, not an error, saves a surprising number of agent loops where the model keeps retrying a failure that is really a success.

Where does the key go in the tool?

Keep it out of the model's hands. The model should not invent the key, because it may produce a different one on each attempt. Your tool wrapper adds it from the run context:

def issue_refund(args, ctx):
    key = f"{ctx.run_id}:{ctx.step}"          # stable, not model-chosen
    return payments.create_refund(order_id=args["order_id"],
                                  amount=args["amount"], idempotency_key=key)

If a single step can legitimately make two different refunds (two orders), include the discriminating fields in the key or the step index of each call. Your key should name the intended action, no more.

What I did not test

• A live payment API. The refund service is a fake I wrote; Stripe's behaviour is quoted from its docs, not run.
• Real models retrying. There is no model in these demos. Whether a given model re-issues a call after a timeout depends on the model and your prompt; log tool calls and look.
• Multi-node servers. The lock stands in for a database unique constraint, which I did not run here.
• Eventually consistent lookups. The lookup demo reads local memory. Real search endpoints can lag.

Checklist

1. List every tool that changes something outside your process.
2. For each, mark the idempotency story: natural, vendor key, own reference plus lookup, or none.
3. Generate the key once per intended action, from the run and step, outside the retry loop.
4. Never let the model choose the key.
5. Treat "already done" as success where the end state is what you wanted.
6. For tools with no story, require human approval and log the ambiguity.
7. Test with a server that does the work and then drops the reply. That single fault finds most of the bugs above.