Event Driven Architectures vs Workflows (with AWS Services!)
Вставка
- Опубліковано 2 жов 2022
- Event based applications and workflows are two ways to coordinate actions across a distributed system. Both have pros and cons and are appropriate in different circumstances. In this video, learn about Event Based architecture vs Workflows and how they could be built using AWS services.
Looking to get hands on experience building on AWS with a REAL project? Check out my course - The AWS Learning Accelerator! courses.beabetterdev.com/cour...
📚 My Courses 📚
AWS Lambda - A Practical Guide - www.udemy.com/course/aws-lamb...
🎉SUPPORT BE A BETTER DEV🎉
Become a Patron: / beabetterdev
📚 MY RECOMMENDED READING LIST FOR SOFTWARE DEVELOPERS📚
Clean Code - amzn.to/37T7xdP
Clean Architecture - amzn.to/3sCEGCe
Head First Design Patterns - amzn.to/37WXAMy
Domain Driven Design - amzn.to/3aWSW2W
Code Complete - amzn.to/3ksQDrB
The Pragmatic Programmer - amzn.to/3uH4kaQ
Algorithms - amzn.to/3syvyP5
Working Effectively with Legacy Code - amzn.to/3kvMza7
Refactoring - amzn.to/3r6FQ8U
🎙 MY RECORDING EQUIPMENT 🎙
Shure SM58 Microphone - amzn.to/3r5Hrf9
Behringer UM2 Audio Interface - amzn.to/2MuEllM
XLR Cable - amzn.to/3uGyZFx
Acoustic Sound Absorbing Foam Panels - amzn.to/3ktIrY6
Desk Microphone Mount - amzn.to/3qXMVIO
Logitech C920s Webcam - amzn.to/303zGu9
Fujilm XS10 Camera - amzn.to/3uGa30E
Fujifilm XF 35mm F2 Lens - amzn.to/3rentPe
Neewer 2 Piece Studio Lights - amzn.to/3uyoa8p
💻 MY DESKTOP EQUIPMENT 💻
Dell 34 inch Ultrawide Monitor - amzn.to/2NJwph6
Autonomous ErgoChair 2 - bit.ly/2YzomEm
Autonomous SmartDesk 2 Standing Desk - bit.ly/2YzomEm
MX Master 3 Productivity Mouse - amzn.to/3aYwKVZ
Das Keyboard Prime 13 MX Brown Mechanical- amzn.to/3uH6VBF
Veikk A15 Drawing Tablet - amzn.to/3uBRWsN
🌎 Find me here:
Twitter - / beabetterdevv
Instagram - / beabetterdevv
Patreon - Donations help fund additional content - / beabetterdev
#aws
#SoftwareDeveloper
There's a very good article about it called "Choreography vs Orchestration in the land of serverless", TL;DR: Use both in the same project, using orchestrators inside a bounded context and choreography between bounded contexts.
Good article!
Did the author said how could define the boundary? AKA when to use orchestrator and when to use choreography
I was wondering if he would end on that note. Seems like together they would counter each other's cons.
As always, this video covers everything in a comprehensive manner and makes it easy to apply practically
Fantastic video! Extremely clear and thorough on both patterns
Love the way you explain things. Very clear and concise. Great work!
Be nice if channel owner just mentioned a quick appreciation note for your generosity …
Thank you very much for your explanation, which is nice and easy to understand. 👍
Great explanation!
Thanks for sharing this information, useful 👍
so glad I found this channel, much appreciate your effort. I am just learning AWS and this mthod of teaching find is so much easier to understand. Whereas others will just mention Services in silos and then start drilling down into all the granual info on each Service (without even explaining how it relates/connects to other services (making it difficult to get your head around it all).
You're very welcome and welcome!
Great job man. Thanks
This is a great video, and can be greater when adding the monolith client retrying approach because tons of people are still stuck in there.
Thank you so much for the explanation.. pleas can you do a video more on Aws IOT service and also the more on database
great video. it reminds me of the difference between OOP (mapped to EDA) vs non-OOP (mapped to workflow). Solving the same problem but modeling the problem differently which leads to various tradeoffs
Wow. This is what I just said.
Thank you for a great presentation
Glad you enjoyed it!
Very helpful!
With event driven you could use traces for observability to visualize what step you’re at.
Thanks for the video and example, learned new kinds of stuff on EDA and workflow.
I was thinking, in the case of the EDA approach again for monitoring, maybe we should log or publish the state and the success/failure of every operation. Triggering a lambda on failed operation/step could also help us in unwinding initial steps and marking all of them as invalid. Does it make sense?
very helpful video
Awesome video
This really brings back memories of when I used to use OOP in C++ to create an event driven application. Thanks! Now, let me implement this in OOP Python first.
Informative! What s/w have you used to show the workflow?
another great video as always!
Glad you enjoyed!
you are a super hero. thanks a looot
We heavily use Step Functions at work. We don't require a rollback in the event a problem happens but we do need to know that a failure happened. For that we use Cloudwatch integration with slack that monitors each lambda in the workflow. Note that in each step function task that executes a lambda, if mandated you could essentially add the retries, exponential backoff and trigger details as well to reprocess the lambda and perhaps move it into a DLQ for redriving it at a later time.
BTW, thank you for all the AWS content and easy explanation that you provide. I've benefited a lot from it.
There is another option if you are using Standard step functions. You could create an Event Bus rule that reacts to Step Function transition to Failed and as an action it can publish the message to SNS and from SNS you can send it everywhere. But this is an option only if you use Standard. For Express I usually use the Async invocation and let the caller deal with the potential failure of the Step function.
@@cristiannechita2032 Is there a benefit to doing that using event bus instead of having the step function routing the error message and input payload to say another step for SNS in the step function itself? I've tested this and it works well when a lambda fails and the retries are exhausted since step functions are async operations.
@@arkster00 The benefit that I see is that you keep your step function cleaner (without putting those transition from various nodes towards a common error handling task). Also, there are Tasks without Lambdas (e.g. Pass) that can apply transformation and those can fail as well.
This should not replace the retry operations of your lambda step (it makes sense to keep it to deal with any transient errors).
@@cristiannechita2032 Thanks. Something to think about.
Thanks
@Be A Better Dev can't you use Distributed Tracing to see status of orders in SOA?
Great lecture!! Which tool do you use to draw a visual representation of architecture ?
Thanks Eduard! I used Google slides for the first part, and the Step Functions Studio Editor for the second half. Cheers.
Is Prefect a good tool for production ready , asynchronous microservices based workflow orchestrion?
what will be the best template in lucid chart to replicate the step function workflow part?
There are so many more sub-events in the logic and execution in handling the transaction steps in a distributed system just for placing the order based on the user's shopping cart that you gloss over. That would be enough for a single video alone. The credit card service would be inside the order placement, just for starters! Before doing any subsequent fulfillment steps let alone packaging, a hold would need to be placed for the required funds on the user's account with the card issuer 😂
You need something like Kafka/Kinesis as an event log to distribute the events to appropriate consumers, etc etc.
I do agree that the Workflow approach has significant benefits, and you are right about the costs and tradeoffs of orchestration with this approach.
I just think that you need to go into a bit more detail with the event-based architecture side, because your diagram is extremely oversimplified. And I don't think IoT is an essential aspect of the system at all, these sorts of systems functioned before such devices even existed.
Hi what do you think about using SQS + Dead letter queue in event-driven architecture to help monitor
Step functions are great, but I’ve found monitoring to be a weak spot. The state display on the diagrams is very rudimentary, and you’ve got to use other tools and work with the logs to actually get a real monitoring dashboard.
I am not sure what is happening between the two lambda functions that can determine the success/timeout/default of the first Choice node. An iot device? Like a sensor? Is it forwarding data? What reads that data?
wouldn't you have to revert changes made in a step in case of failure when using step function approach, or you would simply correct the failing part i.e ask customer for another credit/debit card and retry? In my experience only durable functions on azure have the functionality to re-submit the request, i am sure there would be something similar on AWS end.
Well it depends on where you redirect the customer. If you want your customers to pay with another channel then you can build your logic in triggering another lambda and at the same time route the user on the frontend of payments channels list, so that user can select the payment channel of his choice. Auto resubmission will make user stuck in the loop.
can you do workflow withut step functions?
Looks like absolute overkill architecture for 99,9999 % of all apps and businesses 🤔. But, good explanation and overview of AWS madness. 😊
Great content, I've been watching you for a few months.
I wanted to give feedback on the timing at the end of your videos with suggestions. Within a few seconds of suggesting other videos and overlaying them to be clicked, the video ends and your suggestions are no longer available. It would be nice to have a bit more time to react: read the video titles, determine the relevancy to what I am trying to learn, and then clicking.
Hi, Thanks for this video.
I would like to know why we cannot maintain microservice architecture that are handled by individual small team while using step functions?
Still the step functions can be created or maintained by the architect and we can have different teams to maintain the services right?
Thanks.
Look at the step-function as being part of a micro-service. That team owns everything regarding that workflow (from it's definition in IaC till deployment).
Christian's point is spot-on.
So basically if my application has 3 services working independently, i can assume that those services can be monitored separately using step functions right ?
Its to say, micro monitoring sortof a way, eh ?
Got it. Thanks Cristian.
Nice
What are you doing step-function?
good video
Glad you enjoyed!
👍💯
What do you use to draw the illustration? 9:20
I'm almost sure this is the editor for StateMachine (a.k.a StepFunctions) in AWS.
Workflow is Event-driven with global state.
Good video. Reality is fancy bla bla architects see the first diagram on event driven choreography as a short running stateful work flow just because there are arrows in diagram & then bar of argumentation starts with integration SMEs 😂😂
all workеd
Not totally true😊
In the Eda design you are using SNS and it will generate chatty conversation between teams
And if we focus on microservice only following the essence of DDD
Each step in Workflow could be a service so just handling service with well defined bounded context will fit team responsibilities very well
Customer Charging happens first and rest other events follow
Not always. Cards are usually pre-authorized for a charge and charged only upon shipping.
While I love this vid, I think @sarc007 has a point. This is almost never the case in reality. Companies want your money ASAP and will confirm pre-auths in a nightly batch, or up to 3 days later. If your shipping flow gets stalled, you don't want your pre-auths timing out. It feels like you used artistic license to switch them around so you could make your point about observability.@@BeABetterDev
well, good video but seems this is not event-driven.
Thanks