You are AMAZING!!! I've not had a single video explain anything as clearly as you do. Please keep doing what you're doing and MORE of it, because you are a natural teacher of the best kind.
You really don't understand how this video just saved me stress. Thank you. Love the visual explanations and the details such as common use cases, etc... keep it up
PURE GOLD. Thanks a loooooot Be A Better Dev is my shepherd; I shall not want. He maketh me to lie down in green pastures: he leadeth me beside the still waters. He restoreth my soul: he leadeth me in the paths of righteousness for his name's sake.
Men! I can't appreciate all your videos enough. They had guided me on from zero to "hero" on my serverless journey in my new job (I started 6 months ago). I have a whole notification system for AWS Backup using SNS / SQS / Lambda running right now!
Thanks a TON for such a tremendous , easy to digest video. Quiet explanatory at the same time. LOVED your pace of teaching and explaining the concepts. Stay Blessed Brother !
I was always scared of the SQS/SNS, and could never really understood them😅. Thank to your videos I do now! Very grateful for your great content and very neat and clear explanation, thank you a lot!☺
Hey Better dev, Just wanted to thank you for the tremendous time you spend on putting all these materials together and then compiling them for the great and educative videos you produce for all. You are a true leader, a role model, and a trendsetter for many. Keep up the good work because you are impacting lives worldwide, and I want to thank you for your selfless sacrifice to the human race. Jah, Rastafari!!!
Hi @BeABetterDev , 1). Can SQS accept an Event Published from a System (example : Salesforce - Events) ? 2). What is the count of messages that an SQS can accept from an external System (i.e, Published Message Count from System -> Amazon_SQS ) ? 3). Can SQS accept Bulk Messages / Events ? Please let me know. Great video, Clarified in a single go.
Indeed the video is very informative ,Thanks for sharing . I have a small doubt ,similar to query parameters in API how we incrementally poll messages from SQS queue. I would need to poll messages based on a particular timestamp.
Great video. What are the downsides/tradeoffs of using SQS vs. SNS? In my mind SNS can do everything SQS does and more (i.e. supports multiple subscribers). Also you mention that the consumer sets the consumption rate. How would that work with something like having a Lambda as the consumer? I'd imagine Lambda would spin up as many instances are required to process as many SQS messages as quickly as possible.
Lovely AWS tutorials. Kudos to you and I hope you will keep it going! Could you please better explain the beginning? 00:00:42.160 in the past when two services had to 00:00:44.399 communicate with one another one service 00:00:46.559 would perform an api call to inform the 00:00:48.960 other service that something changed or 00:00:51.039 some kind of event occurred on its side 00:00:53.680 now using sqs this contract has changed 00:00:57.039 instead the producing service of the 00:00:59.359 event or the notification can 00:01:01.359 asynchronously notify the consumer that 00:01:04.080 something changed in its system I do not understand what you were trying to imply.. by how I interpret it & how the above text is phrased; you say - in the past: one service had to inform the other something changed on its end through an API call - now: the contract changed, id: the service in which something changed (same as above till now) can asynchronously notify the consumer that something changed in its system... how do the above differ? did you mean API calls in the past were always "blocking" instead of "async"? is this what you meant? I do not follow, sorry..
Nice. My question is: what are the drawbacks of this continual "polling" being done by Service B in your minute 17:30 example? Suppose your service A does not send messages to Service B for months on end, is this "polling" costing anything (i.e., computational cost in terms of site performance, AWS fees)?
Should have finished the video before asking... it seems like Long Polling avoids the need to continually poll for messages. Still curious about the drawbacks of this, though.
Great Explanation. Have a quick question. I have multiple queues where the poller should be fetching the messages from all of them based on some priority. And the priority would be like this - for example 1st queue having only 2 messages, 2nd queue having 100 messages and 3rd queue having 1000 messages then I want the poller to initially poll the queue which has less number of messages i.e here the poller should initially poll 1st queue then the 2nd and then the 3rd. How can this be achieved.
What happens in a FIFO if a consumer could not correctly handle a message and it is requeued? Does it block other messages being processed, since it's a FIFO?
Hi B R, great question. I couldn't find any direct answer to this on the AWS documentation but here is something insightful. From here, under "FIFO queues": docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html "FIFO queues provide exactly-once processing by consuming messages in sequence from a message group. Thus, although the consumer can continue to retrieve ordered messages from another message group, the first message group remains unavailable until the message blocking the queue is processed successfully." I feel like based on this it probably blocks the queue from processing messages in the same message group. However to know for sure its probably worth a quick experiment.
@@BeABetterDev I played around with it and FIFO-s are blocking. Inserted 3 messages into a FIFO with send-message, then the first receive-message returned the first inserted one, but then calling receive-message again didn't return anything. Had to do a delete-message with the receipthadle of the first message to get the second one inserted. You'd better handle your exceptions in the code that processes the messages.
Thanks for this great overview! Quick question, as part of the "at least once delivery" that standard SQS provide, can I assume that duplicates will have the same messageID? Also, even though it is "at least once delivery", I can assume that there won't be two versions of the same message in a queue at the same time, thus respecting the visiblity timeout you mentioned at 11:20, right?
Great video. Quick question: when you mention cross account patterns, do they also support cross partition data transfer? For example if I have an AWS account in China and another in USE1, can I publish from my SNS queue in China into my SQS queue in USE1?
Hi Bijan, Cross region sns to sqs works for all cases except involving the china region. I found this post on the topic if you're interested: stackoverflow.com/questions/69061869/is-it-possible-to-publish-sns-messages-from-global-aws-account-to-sqs-queues-in Hope this helps
@@BeABetterDev LOL, thanks heaps for checking it out. Yeah, unfortunately it seems it is not possible. I am investigating if there is a way to make that happen; IAM seems to be out of the picture; I am thinking of using temporary security credentials with SQS: docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-using-temporary-security-credentials.html ; will keep you posted if I could somehow pull it off
Thanks for all these videos. Do you have a video on local dev environment with aws - I couldn't find one? How do you approach this? Do you test against another dev aws environment or do you spin up everything locally with mocked/faked services like localstack?
Hi Potato Man! Check out this video on my setup: ua-cam.com/video/CGCn0b4FOfs/v-deo.html I haven't made one for IDE / plugins but if you think it would be useful I can make one. Cheers
Your AWS videos are far better than paid courses. Very clear explanation. Thank You!
"Be A Better Dev" is supercool and comprehensive.
You are AMAZING!!! I've not had a single video explain anything as clearly as you do. Please keep doing what you're doing and MORE of it, because you are a natural teacher of the best kind.
You really don't understand how this video just saved me stress. Thank you. Love the visual explanations and the details such as common use cases, etc... keep it up
Thanks so much Jude and glad you found it helpful!
PURE GOLD. Thanks a loooooot
Be A Better Dev is my shepherd; I shall not want. He maketh me to lie down in green pastures: he leadeth me beside the still waters. He restoreth my soul: he leadeth me in the paths of righteousness for his name's sake.
Men! I can't appreciate all your videos enough. They had guided me on from zero to "hero" on my serverless journey in my new job (I started 6 months ago). I have a whole notification system for AWS Backup using SNS / SQS / Lambda running right now!
Elegant way of explaining - simple and crisp - no fancy show business. Great job!
one of the best explanation out there and it's free. Good job man
Glad you liked it!
This is the best tutorial video ive seen for SQS
Thanks a TON for such a tremendous , easy to digest video. Quiet explanatory at the same time. LOVED your pace of teaching and explaining the concepts.
Stay Blessed Brother !
Glad the video helped and thank you so much for the kind words!
Thanks brother your videos are so simple even a school going children can understand.
Thanks
Thank you so much for your generosity!!!
This video has been extremely helpful with my studies for SAA, just wanted to thank you so much for this
Glad it was helpful!
Thanks!
I think this is the best channel to get a great overview of AWS.
I was always scared of the SQS/SNS, and could never really understood them😅. Thank to your videos I do now! Very grateful for your great content and very neat and clear explanation, thank you a lot!☺
Same same, it all sounded to me like some sort of witchcraft, but now i get it 😂
Thanks for sharing comprehensive understanding of SNS. Appreciate your efforts!!
Hey Better dev,
Just wanted to thank you for the tremendous time you spend on putting all these materials together and then compiling them for the great and educative videos you produce for all. You are a true leader, a role model, and a trendsetter for many. Keep up the good work because you are impacting lives worldwide, and I want to thank you for your selfless sacrifice to the human race. Jah, Rastafari!!!
Excellent introduction to SQS!
Thanks so much Mark!
Hi @BeABetterDev ,
1). Can SQS accept an Event Published from a System (example : Salesforce - Events) ?
2). What is the count of messages that an SQS can accept from an external System (i.e, Published Message Count from System -> Amazon_SQS ) ?
3). Can SQS accept Bulk Messages / Events ?
Please let me know.
Great video, Clarified in a single go.
Another clear, straightforward, awesome video!! Thanks, man!!
You're very welcome Clerton!
Excellent introduction to SQS! Thank you!
25:24 I guess Clodudwatch is for monitoring failures to launch (duds)
Awesome content…saved me hours of google and reading through documentation 👌🏽
Glad it helped!
Thoroughly appreciate this video. You have no idea
Great illustration, thanks for your efforts. Much appreciated!
thanks for putting all this information short for us, lazy ones
You're a great teacher - thank you for the video
Thanks so much Andrew! Glad you enjoyed :)
Excellent video as always!
Indeed the video is very informative ,Thanks for sharing . I have a small doubt ,similar to query parameters in API how we incrementally poll messages from SQS queue. I would need to poll messages based on a particular timestamp.
great, thanks for explaining it in a really simple way
Thanks a lot for this video, great explanation!
Great video. What are the downsides/tradeoffs of using SQS vs. SNS? In my mind SNS can do everything SQS does and more (i.e. supports multiple subscribers).
Also you mention that the consumer sets the consumption rate. How would that work with something like having a Lambda as the consumer? I'd imagine Lambda would spin up as many instances are required to process as many SQS messages as quickly as possible.
Hi Jean, I have a video that compares the two check out the link below. ua-cam.com/video/mXk0MNjlO7A/v-deo.html
Lovely AWS tutorials. Kudos to you and I hope you will keep it going!
Could you please better explain the beginning?
00:00:42.160 in the past when two services had to
00:00:44.399 communicate with one another one service
00:00:46.559 would perform an api call to inform the
00:00:48.960 other service that something changed or
00:00:51.039 some kind of event occurred on its side
00:00:53.680 now using sqs this contract has changed
00:00:57.039 instead the producing service of the
00:00:59.359 event or the notification can
00:01:01.359 asynchronously notify the consumer that
00:01:04.080 something changed in its system
I do not understand what you were trying to imply.. by how I interpret it & how the above text is phrased; you say
- in the past: one service had to inform the other something changed on its end through an API call
- now: the contract changed, id: the service in which something changed (same as above till now) can asynchronously notify the consumer that something changed in its system...
how do the above differ? did you mean API calls in the past were always "blocking" instead of "async"? is this what you meant? I do not follow, sorry..
Nice. My question is: what are the drawbacks of this continual "polling" being done by Service B in your minute 17:30 example? Suppose your service A does not send messages to Service B for months on end, is this "polling" costing anything (i.e., computational cost in terms of site performance, AWS fees)?
Should have finished the video before asking... it seems like Long Polling avoids the need to continually poll for messages. Still curious about the drawbacks of this, though.
I hit like on these videos before even watching 30 secs
Simply explained, thank you !
Keep up ;)
You're welcome!
Great Explanation. Have a quick question. I have multiple queues where the poller should be fetching the messages from all of them based on some priority. And the priority would be like this - for example 1st queue having only 2 messages, 2nd queue having 100 messages and 3rd queue having 1000 messages then I want the poller to initially poll the queue which has less number of messages i.e here the poller should initially poll 1st queue then the 2nd and then the 3rd. How can this be achieved.
What happens in a FIFO if a consumer could not correctly handle a message and it is requeued? Does it block other messages being processed, since it's a FIFO?
Hi B R, great question. I couldn't find any direct answer to this on the AWS documentation but here is something insightful.
From here, under "FIFO queues":
docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html
"FIFO queues provide exactly-once processing by consuming messages in sequence from a message group. Thus, although the consumer can continue to retrieve ordered messages from another message group, the first message group remains unavailable until the message blocking the queue is processed successfully."
I feel like based on this it probably blocks the queue from processing messages in the same message group. However to know for sure its probably worth a quick experiment.
@@BeABetterDev I played around with it and FIFO-s are blocking. Inserted 3 messages into a FIFO with send-message, then the first receive-message returned the first inserted one, but then calling receive-message again didn't return anything. Had to do a delete-message with the receipthadle of the first message to get the second one inserted.
You'd better handle your exceptions in the code that processes the messages.
Thanks bro, nice video. Keep it up.
You're very welcome and thank you!
Top notch content man. Thanks!
Very Clear Explanation!
very well explained. Thanks!
Amazing tutorial loved it
You convinced me. Thanks!
Thanks for this video!! Keep it up!!
Would this be ideal for returning a dictionary from a lambda as a message to be consumed by other lambdas?
Very Good explanation, i like it👌
Thanks for this great overview! Quick question, as part of the "at least once delivery" that standard SQS provide, can I assume that duplicates will have the same messageID?
Also, even though it is "at least once delivery", I can assume that there won't be two versions of the same message in a queue at the same time, thus respecting the visiblity timeout you mentioned at 11:20, right?
Thank you for this fancy explanation
Great explanation thanks!
Great video. Quick question: when you mention cross account patterns, do they also support cross partition data transfer? For example if I have an AWS account in China and another in USE1, can I publish from my SNS queue in China into my SQS queue in USE1?
Hi Bijan,
Cross region sns to sqs works for all cases except involving the china region. I found this post on the topic if you're interested: stackoverflow.com/questions/69061869/is-it-possible-to-publish-sns-messages-from-global-aws-account-to-sqs-queues-in
Hope this helps
@@BeABetterDev LOL, thanks heaps for checking it out. Yeah, unfortunately it seems it is not possible. I am investigating if there is a way to make that happen; IAM seems to be out of the picture; I am thinking of using temporary security credentials with SQS: docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-using-temporary-security-credentials.html ; will keep you posted if I could somehow pull it off
U R A WONDER
17:55 Standard Vs FIFO (Good)
Great video !
Glad you enjoyed it!
thank you for this video!
Thank you
You're welcome!
Thank you very much
thank you so much for this
Amazing video
Why sending messages to Queue is an anti-pattern?
Sqs has a limit of 10 messages and 256kb message size. That sounds something very obsolete technology not really modern high performance
You are misunderstanding. You can add or pull up to 10 messages per request. You can make as many requests as you need as fast as you need.
Perfect explanation
Glad it was helpful!
thanks for the video, liked :)
Thanks for all these videos. Do you have a video on local dev environment with aws - I couldn't find one? How do you approach this? Do you test against another dev aws environment or do you spin up everything locally with mocked/faked services like localstack?
Hi Potato Man! Check out this video on my setup: ua-cam.com/video/CGCn0b4FOfs/v-deo.html
I haven't made one for IDE / plugins but if you think it would be useful I can make one.
Cheers
Legend !!
man ! i wish i work under you as my senior !
One day ! :)
@@BeABetterDev thx
Gratitude
Can we do an interview, need you input for my university project
Please brother
Virgil Common
Arch Estate
Nona Forge
Little Forest
👍
4979 O'Hara Plains
Ortiz Brook
fucking legend.
Rohan Hill
Rolfson Rest
What if the Queue goes down lol
Good, you mispronounce the word 'process' in all of your videos. It's a little distracting but the content is excellent👍
Dewitt Loaf
VonRueden Club
Excellent introduction to SQS! Thank You!
Thank you so much