- Different “Receipt Handle” BUT Same “Message ID”
Each message insert into Simple Queue Service (SQS) will be assigned with a unique message ID by the system. You will get the message ID if you are calling sendMessage with AWS SDK as the response. Uniqueness of the ID is important so the system can identify each message.
Usually SQS messages will trigger lambda to handle the process, like send SMS, push notification, etc. We have to ensure lambda done the job before that message is deleted from the queue. Default behaviour of Lambda will automatically delete the message after it processes complete without error, so throwing error is a way to stop Lambda from deleting the message from SQS.
Since we prevent default for delete message, will the message stay in the queue forever? The answer is NO. Because there is message retention period for SQS. Even though we don’t have to worry about that, message will still retrying for quite a number of time during the period staying in queue. So we have to use AWS SDK to deleteMessage from SQS. Take a guess, are we using the message ID or Receipt Handle to delete the message?
3, 2, 1~ The answer is Receipt Handle.
Well, when SQS trigger Lambda, and pass it the message, it generate a new Receipt handle for each message. If the message is not handle, after visibility timeout, SQS will trigger another Lambda to handle the same message, but this time the Receipt handle will be different from the first one. May refer to the diagram below.
In simple, Receipt handle is used by Lambda. While message ID is used by SQS.
2. Only the latest receipt handle works
From the first point, we know that each message trigger Lambda will have new Receipt Handle. So once another Receipt Handle is generated. All old one will ineffective immediately. It means you have to delete with its latest Receipt Handle.
3. SQS visibility timeout should not shorter than Lambda timeout
Extend tips from above, SQS visibility timeout is important and crucial to the whole process, from SQS receive message until trigger Lambda to handle. SQS default visibility timeout should not shorter than Lambda timeout. Visibility timeout control how long a message will be visible to other handler again.
If the visibility timeout is 1 minutes and Lambda timeout is 15 minutes, while Lambda take average 3 minutes to process. Under this scenario, terrible things like same message will be executed twice or even more, because when Lambda 1 still processing the message, Lambda 2 is trigger and pass with the same message (as visibility timeout and it is visible to other handler). When Lambda 1 finish process and try to delete message, its old Receipt Handle unable to do so, hence it was executed twice.
Conclusion : Above tips is what I experienced and learned from my job. I hope it is helpful. Always exploring and happy coding~
