Message Driven Architecture to DECOUPLE a Monolith
Вставка
- Опубліковано 24 лип 2024
- Message driven doesn't need to apply just to Microservices. You can apply a Message Driven Architecture to a Monolith to decouple concerns. Consuming Commands and Events to handle concerns into their isolated units. The asynchronous and isolation will also enable your application to be more reliable and resilient.
🔔 Subscribe: / @codeopinion
💥 Join this channel to get access to source code & demos!
/ @codeopinion
🔥 Don't have the JOIN button? Support me on Patreon!
/ codeopinion
📝 Blog: codeopinion.com
👋 Twitter: / codeopinion
✨ LinkedIn: / dcomartin
0:00 Intro
1:29 Events & Commands
3:16 Code Sample
4:55 Publish/Subscribe
8:30 Command/Queue - Наука та технологія
I love the lack of mumbo jumbo on this channel. it gets straight to the point. Thank you, sir.
Thanks 😊
I've seen 20min videos that has a fraction of valuable information than this one, Derek really respects devs time!
Thank you.
I've been following you for almost a year now, and almost all of the concepts have finally clicked for me.
It was really hard as I was involved in a lot of "Turd Pile" codebases that were primarily data-centric, but after shifting gears towards behavior-centric thinking and design, it aaaaall makes sense now.
The Actor model is such a powerful, yet severely underrated concept in software design.
I hope you reach an even broader audience, because the software market is overflown with hot mess maestros trying to shoe-horn CRUD into every single type of problem domain.
Thanks. I appreciate you taking the time to comment. Glad it clicked. Keep progressing!
Great video. I've been watching a lot of your videos lately and have learnt a lot of new stuff from you. You show a really high level of overall software engineering knowledge. Really enjoying that you cover latest trends and give concrete examples, especially in the context of enterprise-level applications. I just became a developer level subscriber and encourage others to do the same 🤓
Appreciate the support and glad the videos help!
when order is placed, another use case I was wondering about is how to handle updating inventory and quantity on-hand amounts so the shopping cart system can show items that are in stock or out of stock
please keep making great vids like this!
Great job, keep up the good work!
Great video! On top of technical benefits like retry, Pub/Sub is just such a great and simple tool to foster adhering to the Single Responsibility Principle, and it really helps adaping the mindset around it. It's the best way of the Hollywood Principle - Don't call us, we'll call you. You're done with your responsibility, and now it's up to whoever feels like working on top of it, it's not your duty to tell explicitly tell people.
Excellent content! Insofar as they can be, your opinions are correct.
A great video once again! I disagree that messaging _must_ be asynchronous. If you're starting with a big monolith (as I am) and working toward a CQRS architecture, you can use Paramore's Brighter to add a message broker (command processor) and send commands in-process synchronously. You still must obey the command pattern (you don't get to return updated values) but you get that first step of separation without having to code in asynchronicity from the start. Once you have turned service calls into commands then you can find those commands that are slow and send them off to the out of process queue.
Your example is great and has been good for validation, thank you!
Ya absolutely you can use in process for commands. That's a good point. Events on the other hand I do not recommend consuming the the same process as the publisher.
Great stuff.
Great video! But would you recommend to also use the outbox pattern including idempotent consumers?
I have videos on both. Here's the one about the outbox pattern ua-cam.com/video/u8fOnxAxKHk/v-deo.html
Hey,
Thanks for knowledge. Do you think it's important to use third party tool like rabbitMQ to emit events ? Or for example EventEmitter in Javascript can be used to achieve the same.
The issue is with not having durable storage if you're in process only (memory). RabbitMQ is durable. Meaning if you restart, your messages are still in the queue. If you restart your node process, you'll lose events you've emitted that weren't processed/run.
If for some reason you can't use message broker and you want this same functionality you can "simulate" with a job and error table (in the case of not sending email)
Meaning using your db as a queue? If you were to do that, I'd recommend using a library that abstracts that for you. It sounds "simple" but can up in you creating a lot of infrastructure code.
Great video! I like the Command/Queue part. Even if all the business logic is complex and time consuming, the server can respond very quickly. The downside would be that the client would not get any meaningful response. For example the order might fail because the product is out of stock. I guess real time (websocket) connection with the client could solve the problem. Wonder if this is the best way to go because scaling websocket is also quite a challenge
Yes, I created a video on using websockets and events to push down events to the client. Check out ua-cam.com/video/Tu1GEIhkIqU/v-deo.html
If you're in the .NET Space, SignalR with a Redis backplane works well. If you're using Azure, their SignalR service is also an option.
@@CodeOpinion Nice! Im in Node.js space, but it still helps a lot and the code is quite similar especially with Typescript
Are Commands also available in CAP or MassTransit ? Can you give some advice which to choose for Messaging when you can not use NServiceBus ? Btw thanks for all this information about the latest trends which are not covered much at university.
CAP is specifically events in the sense of being able to have multiple consumers. That doesn't mean you can't use it for commands, just that a single consumer isn't forced. I can't speak to MassTransit I'll text you test it.
@@CodeOpinion Thanks, I will try out CAP. One thing I was wondering about is, when using IHttpContextAccessor to get for example the Identity Id of a user. (Saving Entities with a userId Property) I can use the IHttpContextAccesor in other Services than the Controller because it is within the same Request Scope. After publishing an Event does my Service that subscribes to it also belong to the same Request scope and can access the IHttpContextAccesor or do I have to post the UserId in the Event?
@@christianresdezch1303 After sending command or publishing events you handle them in different thread or even different process, so you lose context of http request. You can pass userId along as event property.
MassTransit can send or publish messages. You can send command to one endpoint or even publish it, but then you need to remember to have only one subscriber, which I don't recommend.
Great video!
How would you deal with an event that has multiple subscribers and one of them fails?
If the message gets picked up from the queue again then both notification handlers would get invoked again, even though one or them already handled the message.
Consumers operate in isolation. If one consumer fails to process a event, it does affect the others.
@@CodeOpinion Maybe I'm missing some knowledge here. Does it matter which queue you're running off?
Say I have a monolith Web API that posts an event to AWS SQS, I have a console app with the same code base that is reading the queue, deserializing the event and shooting it into mediator as INotification.
In this case all the listeners are in the same process and if one handler breaks then it would get retried again. How do I store state here on whether a notification was processed or not?
Otherwise I would need to implement a console app for each notification handler and do a one-to-many publish action with SNS->SQS.
MediatR is fanning out in process calling many handlerd. Not really what you want likely. You'd be better served using NSeviceBus, MassTransit, etc
Is anyone successfully applied this pattern into the monolith codebase, essencially in the write-heavy system? because when the monolith is both publisher and consumer, It has to process x2 (or even more) work load, Is It still effective?
I usually break it down into multiple processes. Usually a message consumer/processor will be deployed separately from say a web/http API. Both will share the same underlying code but they represent different inputs into the system (web & message broker)
Where does the Handle method live? Is it "just another request"? Is it a background thread like a hosted service?
If a user clicks the button, it
1) sends an http request to one endpoint
2) does work + publish a message
3) send back a response
... and then? my first request is done, where is the subscribe code being executed?
NServiceBus is being used as the messaging library that's using rabbitmq. It's invoking the handler.
Be aware that the message schema can be a (hidden) form of coupling. You may need anti-corruption layers on any consumer.
That's indeed a topic in my list as well as versioning.
I loved what McDonald's did with the schema registry. That stuff is genius
👍👍
Coupling the root of all evil 😁
Almost! Without it there is nothing. Untamed it's a nightmare.
Sounds like you've replaced highly reliable and simple intra-process messages with less reliable and additionally complex inter-process messages. Congrats, you've got a distributed monolith. Imagine thinking installing a message bus and creating all the additional infrastructure is better than a 'sent' flag in your data source.
While intra-process communication is reliable in terms of not having to make an network call to a broker, you lose all the fault tolerance (or have to build it in-process). I would not call using a broker a disturbed monolith at all, I'd call it a loosely coupled monolith. What I'd call a distributed monolith is making RPC where it was an in-process call.
Bad idea, unnecessary complexity. Difficult to debug or trace bugs. Simple class, method calls is sufficient and perfect in a monolith.