In an earlier post about idempotency and safety of HTTP methods we learned that idempotency is a positive API feature. It helps making an API more fault-tolerant as a client can safely retry a request in case of connection problems.
The HTTP specification defines GET, HEAD, OPTIONS, TRACE, PUT and DELETE methods as idempotent. From these methods GET, PUT and DELETE are the ones that are usually used in REST APIs. Implementing GET, PUT and DELETE in an idempotent way is typically not a big problem.
POST and PATCH are a bit different, neither of them is specified as idempotent. However, both can be implemented with regard of idempotency making it easier for clients in case of problems. In this post we will explore different options to make POST and PATCH requests idempotent.
Using a unique business constraint
The simplest approach to provide idempotency when creating a new resource (usually expressed via POST) is a unique business constraint.
For example, consider we want to create a user resource which requires a unique email address:
POST /users
{
"name": "John Doe",
"email": "john@doe.com"
}
If this request is accidentally sent twice by the client, the second request returns an error because a user with the given email address already exists. In this case, usually HTTP 400 (bad request) or HTTP 409 (conflict) is returned as status code.
Note that the constraint used to provide idempotency does not have to be part of the request body. URI parts and relationship can also help forming a unique constraint.
A good example for this is a resource that relates to a parent resource in a one-to-one relation. For example, assume we want to pay an order with a given order-id.
The payment request might look like this:
POST /order/<order-id>/payment
{
... (payment details)
}
An order can only be paid once so /payment is in a one-to-one relation to its parent resource /order/<order-id>. If there is already a payment present for the given order, the server can reject any further payment attempts.
Using ETags
Entity tags (ETags) are a good approach to make update requests idempotent. ETags are generated by the server based on the current resource representation. The ETag is returned within the ETag header value. For example:
Request
GET /users/123
Response
HTTP/1.1 200 Ok
ETag: "a915ecb02a9136f8cfc0c2c5b2129c4b"
{
"name": "John Doe",
"email": "john@doe.com"
}
Now assume we want to use a JSON Merge Patch request to update the users name:
PATCH /users/123
If-Match: "a915ecb02a9136f8cfc0c2c5b2129c4b"
{
"name": "John Smith"
}
We use the If-Match condition to tell the server only to execute the request if the ETag matches. Updating the resource leads to an updated ETag on the server side. So, if the request is accidentally sent twice, the server rejects the second request because the ETag no longer matches. Usually HTTP 412 (precondition failed) should be returned in this case.
I explained ETags a bit more detailed in my post about avoiding issues with concurrent updates.
Obviously ETags can only be used if the resource already exists. So this solution cannot be used to ensure idempotency when a resource is created. On the good side this is a standardized and very well understood way.
Using a separate idempotency key
Yet another approach is to use a separate client generated key to provide idempotency. In this way the client generates a key and adds it to the request using a custom header (e.g. Idempotency-Key).
For example, a request to create a new user might look like this:
POST /users
Idempotency-Key: 1063ef6e-267b-48fc-b874-dcf1e861a49d
{
"name": "John Doe",
"email": "john@doe.com"
}
Now the server can persist the idempotency key and reject any further requests using the same key.
There are two questions to think about with this approach:
- How to deal with requests that have not been completed successfully (e.g. by returning HTTP 4xx or 5xx status codes)? Should the idempotency key be saved by the server in these cases? If so, clients always need to use a new idempotency key if they want to retry requests.
- What to return if the server retrieves a request with an already known idempotency key.
Personally I tend to save the idempotency key only if the request finished sucessfully. In the second case I would return HTTP 409 (conflict) to indicate that a request with the given idempotency key has already been executed.
However, opinions can be different here. For example, the Stripe API makes use of an Idempotency-Key header. Stripe saves the idempotency key and the returned response in all cases. If a provided idempotency key is already present, the stored response gets returned without executing the operation again.
The later can confuse the client in my opinion. On the other hand, it gives the client the option retrieve the response of a previously executed request again.
Summary
A simple unique business key can be used to provide idempotency for operations that create resources.
For non-creating operations we can use server generated ETags combined with the If-Match header. This approach has the advantage of being standardized and widely known.
As an alternative we can use a client generated idempotency key provided in a custom request header. The server saves those idempotency keys and rejects requests that contain an already used idempotency key. This approach can be used for all types of requests. However, it is not standardized and has some points to think about.
Leave a reply