Relevance of idempotency in REST APIs

Idempotency ensures that an API request completes no more than one time. Idempotency important in rest api calls as duplicates of the same logical request can be created by retries.

• User app to server retries
  • the user double accidently double clicks on submit button.
  • the user retries an action because of latency 
  • the user accidently retries a completed action
• server to server retries: on connection timeout a server would not know if the api call has been successful or has failed. server may typicall retry the action. (alternatively get status can be done with the idempotency key )

note that event streaming platforms with at least once delivery guarantee can deliver the same message twice to subscriber.

Solution

Client associates a logical request with a unique idempotency key so that the request can be uniquely identified/tracked and duplicate requests can be removed , thus making the api calls idempotent. In other words An idempotency key is a unique value generated to recognize subsequent retries of the same logical request.Idempotency-Key request header should be included in api requests which should be handled by an Idempotent Receiver. Idempotent reciever extracts the Idempotency-Key and uses it to ignore duplicate requests made by the client on retries. 

Idempotency key can be a 

• uuid 
• A logical id like in case of a bill payment userid+bill_generation_date can be idempotency key.
• hash of (userid + data + ts) to uniquely identify the logical request.

Idempotent receiver can typically use database or redis for deduplication . 

Often for scalability idempotency-key are purged after 10 to 30 mins .

Idempotency key can be generated by 

• server side components: eg a client app instead of directly generating idempotency key may get from server so that the server can track all transactions intiated by client
• client apps :  eg a chatting app assigining hash of (userid+ data+ts) as idempotence key to each user message/request.
• api gateway : (attaching uuid to each request coming to apigateway)

Examples

• User's can be sent payment link for payments. The idempotency key is part of the request parameter.The idempotency key can be uuid or logica id like userid+bill_generation_date.
• Idemptoency key is a hidden field in form post.
• Native app/webapp chat button
  • each submit will lead to request which can be associated with idemptonecy key = hash(userid+data+ts)
  • note that the key is generated on client, server would be too slow and pointless.
• A payments application could generate the idempotency key = hash (usrid + toaccount + amount + ts) on screen load to make requests uniquely identifiable and thus making deduplication by idempotency layer/ middleware possible. This will protect against accidental double click by user also. Note that in this approach user retrying the transaction because of latency or accidentally will not be handled by idempotency layer. It has to handled by business logic. (Eg the server has to check if a similar trans has happened in last 5 mins)