Don’t Build a Distributed Monolith - Jonathan "J." Tower - NDC London 2023
Вставка
- Опубліковано 4 тра 2023
- Don’t Build a Distributed Monolith: How to Avoid Doing Microservices Completely Wrong - Jonathan "J." Tower
As a consultant, I get to see many systems built by many different developers. Recently, I’ve seen an uptick in the number of systems built with a microservice architecture in mind, but those systems often include a lot of the same mistakes that keep them from working well.
In this session you’ll learn from my experiences and get pointers on what to avoid in your microservices implementations so that you don’t accidentally build something which has all the worst aspects of a monolithic application and the worst aspects of microservices These monsters are what I call “distributed monoliths”, and I can help you avoid building one accidentally.
Check out our new channel:
NDC Clips:
@ndcclips
Check out more of our featured speakers and talks at
ndcconferences.com/
ndclondon.com/ - Наука та технологія
0:00 Intro
2:31 About Me
3:53 Monolith Definition
4:56 Monolith Advantages
6:45 Monolith Disadvantages
8:07 Monolith Architecture
8:37 Scaling Monoliths - Increase memory, load balancing, data sharding
11:13 Microservice Definition
13:58 Microservice Architecture
14:48 Event Bus
16:00 Distributed Monolith - Bad guy
17:25 Distributed Monolith Architecture 1 - Not using event bus to communicate between different services
18:39 Distributed Monolith Architecture 2 - Having one shared database among all services
20:54 Good vs Bad. Good: modular monolith, microserice. Bad: ball of mud monolith, distributed monolith
24:01 10 Common Mistakes of Distributed Microservices
24:24 Problem 1: Assuming microservices are always better
25:27 Reasons not to use microservices: monoliths are easier and not unscalable, microservices are hard to implement
26:54 Good reasons to use microservices: more scalability options, independent deployability, isolate surface area of failure
27:32 Microservice vs Monolith Tradeoff: microservices have high availability and eventual consistency, monoliths have low availability and immediate consistency
28:44 Problem 2: Using shared data store in your microservices
32:20 Problem 3: Microservices that are too big
36:14 Problem 4: Microservices that are too small
37:36 Problem 5: Creating microservice from scratch
41:07 Migrating from monolith to microservices: Big bang, Evolution, Strangler Fig
44:23 Problem 6: Coupling microservices through cost-cutting, i.e. shared modules
46:50 Problem 7: Synchronous communication between microservices
50:31 Problem 8: Breaking changes to event contracts
52:49 Problem 9: Not automating build and release
54:12 Problem 10: Changes to microservice affect other modules (Iceberg anaology)
55:49 Bonus Problem: Mismatched teams
59:13 Summary
59:56 Further reading
1:00:24 Recap/Q&A
1:00:57: Q1: How to migrate to microservice at the database level?
1:02:21: Q2: Adding fields to event signatures?
1:03:32: Q3: Is there a use case for distributed monolith?
The hero we need
I've used loadbalanced monolith with a single database for a pretty busy site. It is the simplest thing to build, and it works perfectly. Architecture astronauts is also a thing.
Yes, monoliths are perfect for many/most projects, and microservices is hard enough to do right, that using them when they aren't necessary can be a bit dangerous. Thanks for watching!
True. There's no need to pretend you are Netflix unless it's true.
I mean, this should be obvious, right? Is this non-obvious to people?
@@JeremyAndersonBoise Unfortunately, yes. You must be above average in your software architecture skills. If you read the abstract for this talk, I explain that in my consulting role, I’ve encountered so many companies trying to do microservices for no good reasons. I can only guess this is because it’s such a hot buzzword right now.
back in 2008 I worked for a company that divided the webapp in multiple subapps, each of those subapps was a monolith, and at the front there was another subapp that redirected people to the other subapps, all with subdomains, IIS, using 3 stacks, ASP+Visual Basic, .net 1.1, bea beehive (java mvc web framework built on top of struts)
This is the best introduction to microservices I've found so far: It explains what microservices are, are not, and when to and not to use them.
@jonathantower for problem #7 and slide on 47:00 how'd you do it without sync call if you need order service to fail when customer isn't found? Fire and forget will cause you to introduce some intermediate state on Orders side and then process the reply from the Customers, that might be breaking for the user experience.
Thanks very much J.! I started a big microservices project about 4 years ago, and spent a lot of time reading articles and watching presentations; I wish your presentation had existed then as you've summarized the most important parts of all that I learned the long way in just 50 minutes. As always, there is so much more, and I'd love to see a followup presentation with more details, but I'll definitely be passing this link on to anyone who says to me "I'm thinking about using microservices".
Thanks. This is so affirming of the reasons I wanted to create this talk.
Hello same story as yours with my startup but we did start as distributed monolith and got lots of customers so slowly started going to microservices. Would you be open to discussion on how you we are both doing event buses, and data migration part and handling duplication. I work in telecom (Microservices and ddd shine in this domain due to complexity and scale) , so i am using rpc to do the event bus to request some data because it'd be duplicated too much, imagine every calls having your call settings for the number on the calls table -> i felt like this would be too much info and duplicated and decided to request via rpc. but i feel like this is maybe stressing settings table too much. I would be very interested to know the domain and scale you work at solutions you are employing for problems like this.
@@sortof3337 I was working in ecommerce, with about 8 million registered users - ie mid-size, and was deeply involved in moving from a monolith to services including implementing the first services using this distributed data approach. I'm no longer with that company, but am happy to share what we experienced.
The issue of what was reasonable to replicate was a concern for us too. The disadvantages of synchronous inter-service calls are clear, and we managed to partition domain responsibilities (so far) to avoid having to replicate these larger datasets. The largest dataset replicated was the core user data (8 million datasets) - ie services which needed it could build local caches - and that was ok. If usecases were to be discovered that required a service to access larger "external" datasets than that, then replication might not be feasible and synchronous calls might be needed. However it was still our feeling that appropriate domain boundaries could avoid that.
UA-cam filters out comments with links, but googling "vonos distributed read model" should find the articles I've published about what we did - and my email should be findable from there; I'd be happy to talk.
How'd one go about solving the 7th example (order service getting customer data from the customers service)? In my head even if the data is duplicated into the orders service, inserting a lot of customers would lead to the orders service also needing to duplicate them (by listening to some CustomerCreated event), meaning it'd still be impacted by this kind of load on the customers service?
Very good speech. Thank you!
A big cause for "distributed monoliths" is testing. People distribute everything, but then they try to test everything together, often in a "classic" acceptance test phase in a "release train".
It then becomes a pain in the neck, when you try to combine them all in the train. It is a mess
if you made a distributed monolith just because you wanted convenient tests, then your idea of microservices was flawed from the beginning.
This is a good distillation of a lot of wisdom around building microservices (or not!) without a lot of fluff. Some great diagrams in there too that do well to focus on the key points. I watched it at 1.5x which was about right for me given that it's not a new subject for me.
Thanks for watching, Russ. I'm glad it was useful content. I'll try to talk 50% faster next time ;)
Hello @@jonathantower. Very good talk. Thank you so much for taking time for this. My team started a project about 4 years ago. Started as a monolith, distrubted monolith and now combination of microservies and legacy portions being distrubuted monolith slowly being migrate to microservies. I work in telecom (Microservices and ddd i think can shine in this domain due to complexity and scale) , so i am using rpc to do the event bus to request some data because it'd be duplicated too much, imagine every calls having your call settings for the number on the calls table -> i felt like this would be too much info and duplicated and decided to request via rpc. Typical scale we work at depends on the day from few thousand to few millions calls per day. Is that 30 minute consultion still available? I have some questions and need some advice.
Superb!!
Great talk J. Thank you.
The thing about adding fields in event contracts, making sure the code expects unknown fields and forwards them is a good contender for some gnarly CI tests. Push out an event with a random new field, make sure it gets forwarded all the way to the end of all paths and that nothing crashes.
Great talk! Thanks
I did one team many services before, it worked fine and did not at all push us to monolith. We choose that architecture for a reason after all. What I see as main issue, so many people do microservices just because they are cool but do not reflect on it properly (we all learn, after all, but you need to allow for that) and people cut microservcies not by functions, but by data. Sometimes they are treated just like extended database tables.
Amazing summary!
I’ve worked on both Greenfield and Brownfield products… The Brownfield products were always more difficult. I completely believe that Brownfield product would be easier if Software Engineering were a more robust field where Clean Code and BDD were understood and the default method of development but I can’t keep up with 1000 line methods in 150 different classes.
This is great! I wish I was able to convey these concepts to teams where I have been part of. I was never able to convince team members to take a decoupled approach.
💯
For small teams decoupling in microservices its just nuts, you end up dedicating most of the time to infrastructure and tooling instead of actually coding in requirements. Also you will need an extreme tracing strategy if you want to be able to track down bugs, which comes with a HUGE price, because for otel the amount of data you need to save and the throughput you need is usually way higher than what you need for yous bussiness.
You can definitelly make a monolith that scales-out where it has to, and still gives you the unmatched productivity of a monolith vs microservices, this is a BAD talk in my opinion, extremelly dogmatic.
Sometimes I think microservices is all about politics / management. Everyone wants their own team, their own turf. You can operate independently and choose your tools, data structure, etc. But managing all the cross-talk on the data via the event bus seems like a nightmare. I'm sure I don't know enough about it but it sure seems complicated. Great talk.
If you cut the services or better the "domains" right, it should be not that much.
Its more like the foreign key constraints of your old monolyth datastructure that point out of the domain, need to be broadcasted. But if your modules are loosely coupled, only few of them should exist for each domain.
Publishing messages is typically a no brainer
Indeed. In my experience, the push for K8s / microservices was entirely "the new shiny". They didn't understand the first thing about it, but it's the current buzz, so "we" have to do it. The flagship product of this company was already a grand canyon of kitchen sinks (every new shiny for over a decade got bolted into it) - what he's calling a "ball of mud". It was already 300G - it already had a custom kernel to support _A_ container before linux natively supported containers; after 3 years of working on it, it wasn't even half done and had already grown to over a TB! It would've taken an entire cluster of machines just to run this crap. And because they were wasting all their time on k8s, they weren't doing a d***ed thing to get off CentOS 6 (yes! 6!) with unending customer complaints because it's full of shit their security scanners redline - things that will never, ever be fixed. (I had to answer one customer's over 180 issue list. Only 6 were things we _could_ address, and only 4 were actually fixed.)
For the Orders and Customers example at 46:57 what is the better approach? Should Customer data be stored in the Orders microservice?
"if done correctly" -- and what big company does this?
I thought data throughput is the MB/s you see in metrics, what's mentioned here I thought was client-to-server latency. (5:15)
3:54 Monolith. Advantages: easier to build, better data throughput (don't confuse with performance), can do transactions, scalable (add more memory, processor power, cores, hardware, virtualization, load balancing, data sharding, multi-tenant), fewer cross-cutting concerns, immediate consistency. Disadvantages: requires full deployment, promotes tight coupling, usage of the same technology stack, not as quate as Agile, doesn't allow to have small focused teams
11:15 Microservices - software is composed with well-defined loosely coupled, independently deployed services that communicate with well-defined messages on the backend, maintained by small, well contained teams. Allows to do more precision scalability, more Agile team processes
13:58 Typical microservices architecture: Clients; API Gateway consolidating application, security, cross-cutting concerns, brokering requests to individual microservices that are separate applications with its own database; Event Bus supporting pub-sub model
16:05 Distributed Monolith. DRY, SOLID principles need tweaks, high coupling. Performs and maintained worse. In addition to Event bus added exceptions, single database owned by db team with data normalization.
20:53 Good vs Bad Monoliths. XY: Logically Monolithic to Modular, Physically Monolithic to Distributed. 00:Ball of Mud; 01: Distributed Monolith; 10: Modular Monolith; 11: True Microservices
24:00 10 Common mistakes
24:23 #1. Assuming Microservices are always better. Sam Newman: "Have a really Good Reason". First rule of m-services: do not use them: Monoliths are not inherently bad or unscalable, Microservices are hard to do well, Wrong reasons create distributed monoliths.
26:55 Good reasons: More scalability options, independent deployability, isolate surface area of failure. James Lewis: :Microservices architectures buy you options"
27:33 The Big Trade Off. Microservices - high availability, Monoliths - immediate consistency
28:45 #2. Shared data Store or Models
34:20 #3. Microservices that are too big. DDD: Domain, subdomain, bounded context. Simple Rule: Smallest possible microservices without chatty communication between them
37:37 #5. Starting from scratch. Brownfield project - development of new in presence of legacy. Advantages: Code and relationships to examine; People to talk who knows the system; System which already works; Baseline to compare to the refactored
41:00 Browfield migration approaches: big bang, evolution, "strangler fig" pattern.
44:00 #6. Coupling through cross-cutting concerns. Example: logging with "seq"
46:50 #7. Use if synchronous communications
50:30 #8. Breaking changes to Event Contracts. Rules: 1. No new required fields, only optional with documented default values; 2. Unrecognized fields are ignored but forwarded; 3. Consumers of optional fields use default values when missing; 4. Otherwise create new event type (version), both version should be continued publishing until retired
52:48 #9. Not automating Build and Release. Otherwise time consuming, prone to human error. Do it first.
54:12 #10. Unencapsulated Services. Well-defined service is like Iceberg (10% surface area above waterline): encapsulate significant business logic, small and stable API, large implementation.
55:50 Mismatched Teams. Conway's law: "Any organization that designs a system will produce a design whose structure is a copy of the organization's communication structure"
Looking at the hotel room example I am instantly reminded why microservices can be a serious pain. Looking at this example I would expect that a maintenance and housekeeping service would also like to know if the room is occupied. At which point I am copying state of a room between a bunch of services that might or might not update when I ask them to (eventual consistency after all) adding a few more rooms means having to update many different services. Other reasons why this is not ideal, each service must have a way to add a room and potentially remove a room.
I would suggest to create a separate microservice that deals with the rooms and their states, each other microservice can still have their own details relevant to their service about each room but the shared state of the room would be held by the new microservice. This ensures that every service is aware of the shared state of the rooms without running the risk of all sorts of issues with data inconsistency as a result of duplication of data.
I totally disagree with the idea of duplicated data for the simple reason that after many, many years of supporting systems in various industries I can say with reasonable authority that duplicated data will always diverge over time. As a result your service becomes kind of useless and you end up spending inordinate amounts of time trying to keep things in sync at an ever increasing cost as usage of your solutions goes up.
I would suggest approaching this more from a object oriented point of view where you would not create 20 classes that all are basically operating on the same data as that becomes a nightmare in a very short order. You would also create a class that contains the shared data so that this is what everyone operates on and each instance of that class contains all shared for that instance.
You should in my humble opinion do the same in the microservices world. Of course that service should only deal with the shared data in order for everyone to have the most up to date information when it is needed. Now in the case of a hotel room this is not very rapidly changing data so it is not all that critical but I am sure we can all think of may other situations where data does change rapidly and having outdated information in your microservice because you have not processed all the events yet can lead to a disaster.
Staunch adherence to any architecture ideology is unhealthy and one should always consider the implications of such decisions. Again the hotel room example is probably not going to cause any serious issues, but that is not because it is a perfect solution it is just because a very slow rate of change and very small data volumes make even suboptimal solutions seem quite good.
In the video the only duplicated data is RoomNumber (int), right? Besides this, all services own their data and don't really duplicate it. They could ofc replicate the data they own, but that's just whole another story.
Totally agree with you on data consistency. On the other hand what is the name of a service that keeps data consistent and responds to other services queries... Hmm it seems like its called a database :) Of course database can do much more than that, but in the end you are suggesting a thin wrapper around a monolithic database. I don't know the solution to this problem just pointing out that this wrapper service is a monolith in disguise.
Totally agree if it just holds the data it is just a database. But I'd assume that there is more to the room or collection of rooms that the service could provide.
Like I said I see the reasons why this is not great, but I also see way more issues being caused by having the data of a single physical object stored in many services.
Also when you look at the maintenance service it is in essence nothing more than a wrapper around a data store of some description. As that is pretty much the essence of micro services, they are just wrappers around a database table with some logic to allow for additional utility.
In all cases one of the hardest things to do when building anything micro services based is correctly defining your services and knowing what should or should not be its own service or can be split across several services etc...
I think this is a very common viewpoint from people not yet familiar with microservices. Unfortunately IMO the suggestions above are wrong in a microservices system - and is exactly why many microservices projects have scalability, reliability, and testability issues. It does take time to get your head around the reasons why microservices are beneficial, and feels quite unnatural for a long time. It's also the reason why it's generally better to start any project as a monolith - and to stay with the monolith until forced by some specific reason(s) to refactor. Yes, microservices _can_ be a serious pain - and so should be considered only when the pain of not using them is worse. Of course this hotel example is far too trivial to be a justified microservice architecture..
The whole point of microservice is to decouple code and data, making each project independently maintainable and scalable. Shared code reduces that. Synchronous dependencies between services reduces that. Duplicating code and duplicating data have their costs, but in _some_ cases those costs are less than a centralised system - and that's when microservices should be used.
Each piece of data should have an owner - a "single source of truth". I've not had problems with replicated data getting "out of sync" but if it ever happens, there is always a "source of truth" from which that data can be re-synced. Is there a cost? Yes. But it needs to be balanced against the cost of a system which is monolithic (limited development and runtime scalability) or based on synchronous inter-service dependencies (limited performance and reliability). Everything is a tradeoff - and sometimes that balance says microservices with replicated data is the right solution for a specific problem. Sometimes not.
Ok let's change the hotel room to a book in a library...
How would I know how many books I have and what state they have, in house, lend, returned but not on the shelves, on the shelves or in need of repair.
What happens when I add another 1500 books to the library and decide that another 200 need replacing. The system would need to create a book for each of the services that deals with a state of the books as each is a different department with different needs and different relevant attributes. I'd have at least 5 data sources that all know a bit about each book that is a lot of replicated information and data drift is a thing that does happen the fact that you might not personally have encountered it does not mean it's not a thing. After some 15 years of supporting all manner of systems in all sorts of enterprises large and small I can assure you regardless of how well you architect a system things happen data gets messy over time.
Anyway back to my library, the system would need to ensure that each of these data sources is in sync and we would need to run reports that would query each data source individually or worse yet query every micro service making for a very very chatty solution which has many drawbacks.
The alternative is a book that has a state, the book service knows the state of all books it can at any point in time be enriched with more knowledge of the individual books for instance if we have an audiobook version and if so what formats it comes in etc...
The problem with micro services is the bloat on the maintenance side is endless as every service has its own data source and there is a cost for that. Since no service can tell me everything about a book or hotel room the services are inherently chatty which comes at a significant cost as well. Due to the chance that data drifts or simply disagrees, a hotel room that is rented and undergoing renovations at the same time is very well possible because the individual services do not have knowledge of each other's state, there is a need to constantly be on top of this reconciliation comes at a cost which certainly as the system grows will become quite costly on its own.
I'm very familiar with micro services have worked in very successful startups that where build on top of them and have worked in very successful organizations that had a mix of everything from mainframes to the latest micro services containerized solutions and everything in between.
There is not a single solution that is right but certainly for larger systems micro services can be a true disaster simply because the cost of operating them becomes way larger than it needs to be.
Micro services certainly have a place and they are a great design pattern to be aware of but just like ITIL (project management framework populair in Europe) it can be well overkill for many situations.
My approach is always to look at the needs that the solution is to solve now and in the reasonable future. If it can be done without the massive cost and complexity overhead that micro services bring to the table or by maybe not following the doctrine to the letter and allowing for a room or a book service to exist then it might very well be worth while going down that route and simplifying the design.
Because if there is one thing important it is that the service can be operated at the scale that it is designed for. And the less moving parts a system has the easier it is to operate and in the long run that is going to make a serious financial difference.
Keep in mind that no hotel would be scaled up to tens of thousands or millions of rooms. This means that a design that would scale to those sizes is total nonsense for a hotel system or even a library system.
One of the most disadvantages of microservices that I don't see it mentioned enough when talking about about microservices is the hosting COST, its much much expensive to host microservices. To me thats a huge disadvantage that should be mentioned
Where do you see the extra costs? I'm truly interested, as I don't see a significant difference.
The number of requests handled by a monolithic vs microservice architecture is the same, ie overall CPU usage should be similar; whether a server hosts one large application or 5 containers each with a microservice shouldn't make much difference as far as I can see.
Having 5 databases instead of one could be more costly if each is a separate instance. However it's also possible to have 5 databases within an instance, or even 5 schemas within a database - as long as different user-accounts are used by each service, with access rules set up appropriately to make cross-schema access impossible (ie truly isolate data).
If self-managing something like Kubernetes, there is some cost in the staff needed to do that - but in the cloud, managed k8s environments are basically charged according to the number of underlying VMs - and see first point about how that shouldn't be all that different.
Yes, a message-broker will be needed, and centralized logging systems, etc. But won't any company large enough to need microservices already have such things?
@@simonk1844 the point of keeping separate db was to prevent single point of failure, combining them in anyway would be turning the microservices into monoliths.
Keeping separate DB instances isn't gonna be free of cost right?!!🤷♂️😅...
@@mahee96 Well, if you're taking a system that has a single point of failure (one database) and moving to a system which doesn't, then you can expect extra costs, no matter how you do it, right? Set up a hot-standby database, or shard the data, or split the system into two services which each have their own database, it's all going to cost more - because you're providing something that wasn't there before. I don't see that as being a specific criticism of microservices..
@@simonk1844 sure, The point was about the OP emphasizing that there are actual costs due to this increased complexity both HW resource and system design hence indirectly increasing overall cost of doing business and that is a TRUE statement.
Moreover that is where the line is drawn b/w microservices and monoliths.
Not everything needs to become micro services and not all monolith design are unfit for use.
However people's claim is that monolith can achieve the same with huge reduction in cost compared to microservices.
I have worked in both old (IBM CICS transaction server) and new Microservices arch in my career till now. I am surprised by how much we are adapting the closed sw arch with different new names and calling them as modern ex: microservices.
CICS transaction server processes millions/billions of transaction around the world and it uses all the Distributed program link features providing ACID features for any transaction. It was developed 60+ years ago btw 😃.
Blindly following any dogma and advocating that should be the way to go forwards, isn't much useful considering how limited one's knowledge can be!
@@mahee96and what is your build and deploy frequency? Not to mention the difference between hardware - microservices, like all modern tech in the past several decades, run on commodity servers.
If the only issue of monolith (by author words) is deployment issues I take monolith approach 24/7. It's pretty solved class of problems by modern infra.
21:25 Ball of Mud monolith. You fix one thing, it breaks five other things. Fuck, that's like supporting Lotus Notes. We just called it The Domino Effect.
I don't understand how someone might change table name, yet they can't change message or channel on which they send update. Event bus doesn't solve problems with lack of policy and guards in a team. I would never write a piece of code that changes db structure at runtime. There is no immediate consistency within a monolith in that example either.
Event bus is one of tools that we have, but it's just a tool not holly grail.
What he calls distributed monolith also can work well. The problem is that someone needs to figure out how it works and set rules. The other choice is to take policies and rules from Kafka. First option can be much faster because can be optimized using domain knowledge. Kafka is resource hungry option that gives the safety.
monolith could also scale horizontally by dividing it into "macro" services: api nodes, shared state manager, and background worker
I find this is often the right answer. It strikes a balance between maintainability, scalability, and resiliency.
This got me thinking. Let's say you have a website and an app that communicates with your server. Isn't that at least 3 different "micro" services? The backend, the front-end, and the app? Should communicate through an event bus?
Hi
Is Ocelot gateway with Rabbit MQ a ggod solution ?
First rule about the microservice club: don't. Our hardware ain't designed for microservices, is designed for multithreaded monoliths.
34:22 with problem number 2 with the example of creating a room in an admin module and service subscribe to that and create their own room record… when a new service is added, what is the corresponding way of populating the existing rooms for that service?
The way I see it you can either:
A. Admin module creates dedicated migration events for each room, only new service consumes them. Basically a re-publishing of all room events.
B. DB migration scripts
@@GeorgianGrec could send a message to request the full list, I guess.
Or the service could request the room details at the time it needs it
I dunno…. Sounds like it all belongs together
The approach I used in a recent project is to use Kafka compacted topics. In this example it means the admin module (once) publishes the _current state_ of every room to a Kafka compacted topic (keyed by some room-id), and then whenever a room is created/updated/deleted, the new _current state_ of that room is published to the same topic. A new service just reads the entire topic and stores whatever subset of data it needs. Listening on the same topic keeps every service up-to-date with changes. Kafka regularly "compacts" the topic, keeping only the latest message for each key (room). This approach has been working very well. Data storage use is reasonable; avro-formatted messages (particularly with compression enabled) are compact, ie the storage needed for the topic is always less than the storage needed for the master copy of the data in a relational database.
If the owning component (the admin module in this case) uses event-sourcing then those events, which are stored forever, can also be used by any new service to build its representation of the needed data. Typically an event-sourcing events are stored in an "event store" which provides an API to either "get all events" or "send me all events over a stream". Obviously if the data-owning component already exists and is not build around event-sourcing then refactoring it to do so would be more like a re-write. I'm also concerned about the coupling that results when one service consumes event-sourcing events defined by another service; these events are quite fine-grained and the consumer must implement _all of them_ in order to rebuild state properly. Publishing the full entity state avoids those complications.
The alternative would be some kind of custom data-export/data-import process for each new service created. Not impossible, but it requires cooperation between the teams implementing the data-owning service and the new service - ie meetings about project priority, etc. Having the kafka topic available makes that unnecessary.
This was a pretty good talk. I wish it were simpler to say we should really think about how data and functions should logically be encapsulated in one thing vs another thing. Unfortunately we often use physical infrastructure and side effects of that as a proxy for this concept, because its easier to talk about. This is pretty typical, and there are lots of good nuggets of truth in here, but you kind of have to know them already
What do you mean, wouldn't scaling still be problem even if we separted data and effects?
I believe the best route is modular monolithic. Starting with microservices is a disaster waiting to happen, especially in a production.
Monoliths are awesome, some people just don't know how to cook it. Micro services are usually piles of junk, doing actually nothing important. The only case when you can loose one wheel of your car and nothing would happen - is when your car is actually not moving.
1. you can loose a car mat and drive just fine with (almost negligible reduction of comfort)
2. you can loose a side mirror and still drive being aware of reduced vision from one side
@@bfg5244 any metaphor has it's own boundaries.
I would argue that every non-trivial IT system is a kind of microservice 🙂
Look at any large company even in the 1980s and you'll see multiple code-bases - a large insurance company or a large bank is never going to implement all of its business requirements in a single application. Most of these applications have independent deployment schedules, have their own database and do some kind of data exchange - so as long as you zoom out far enough, won't a diagram of this look a lot like a microservice design?
It seems to me the difference is primarily that microservices turns up the dial on granularity. And also that the data integration between services is typically near-real-time (eventual consistency on the order of a few seconds usually) rather than overnight batches (eventual consistency on the order of 24 hours).
@@simonk1844 agree, zoom matters. In one company I saw an authentication segment itself being split in 4 different microservices. Of course, if one would die - the whole authentication would die. I wrote more about that kind of useless microservices "design".
@@astroganov 😂😂 Thanks for that, I was trying to explain something similar to one of my friend that you can't break down a component and call it multiple components because here the component itself is **unit of work** ex: auth and that would be meaningless.
We can design it split or unified or any way we want but does it actually provide any sense is the real question!!
It is like saying "I am creating a system(ex: multiple services making up the auth component) which helps my actual system(ex: authentication) to function" which is basically increasing redundancy at granular level but creating more complexity from a zoomed out perspective.
Shout out to @ididathing’s desert monolith shown @5:00.
We need to sell more cloud services... let's turn "monolith" into a dirty word!!
It's not a dirty word if you're intentional about being a monolith. It's dirty if you end up with it when you thought you doing MSs. I think that's the basis of this presentation
how does one solves problem no7, do you keep copy of the customer data in order service?
Oh damn i'm strugling with these. Can you come to speak our internal conference about these issues @jonathantower ? :)
Those migration patterns like evolution or strangler fig look cool on slides. In real life, in most applications the database is so tightly entagled that you can neither pull much out separately nor can you use a "facade" because the integration happens in the back thorough the database (and assuming instant consistency and locking/concurrency resolution via DB, of course). Now I do not want to advocate green field, I would perfer an evolutionary approach but it is very hard and I have seen it get stuck half way many times as focus shifts and buget runs out, resulting in an application more complex than before.
My tech lead is rejecting to regret have built a distributed monolith, what should I do?
Not sure how team topology as "Team per Service" might be solution as most of them (each) will be seen as a "Monolith" Service which will have to sync. with other "service team" as shown in distributed Monolith problems. (in the vision of Product Group having multiple Product Teams composed of multiple teams as Eduardo da Silva vision)
Did I miss something?
In the scenario of the Orders > Customers sync call... what would be an acceptable solution? Copying the customer data into the Orders service would make a lot of duplicated data, and in the case of a order for a new customer, you should first call the customer service to add the customer, then call the order service to create the order, giving this way responsability to the front end of part of the business logic.
Which would be the best solution for this? It's a very common thing that a service needs data from another.
I was wondering this too... based on the earlier examples, I'm assuming each module should be able to handle a customer update event that the customer module sends out. you probably need to accept that your orders module will at worst get the latest customer data eventually
I think the solution here is to draw appropriate service boundaries. If you take the "micro" part of microservices too seriously then you do indeed get these kinds of issues where data duplication becomes excessive - and IMO the answer is just "don't do that". Services are useful to (a) limit a code-base to a comprehensible and manageable size, (b) make development teams independent of each other, and (c) allow functions which have different non-functional requirements such as scalability or implementation language to be dealt with separately. As long as none of these apply, there's no reason to separate stuff. Here, orders and customers probably should just live in one codebase and one database. If a codebase does need to be split into services, I would recommend considering the data-dependencies carefully and choosing a split that minimises the need to synchronize data - particularly large datasets.
Having said that, the company I recently worked for does indeed separate orders and customers into different services and databases, with a minimal set of customer data replicated/synchronized. The reason was that in our case customer management and order management were complicated enough to each require a dedicated developer team, and separating the services removed a lot of coordination work related to release schedules, shared library updates, etc. as well as making compile and automated-test duration reasonable, making the codebases less intimidatingly large, etc.
@@simonk1844 I agree with this answer, drawing appropriate service boundaries is probably the hardest part which will rarely be perfect.
Would just add that there's a (d) as well which is another intrinsic benefit of physical modularity. It is that services encourage cohesive modules which are easier to move between teams if needed. As long as you're not building the distributed monolith!
@@simonk1844 So now ANY data fetching is not allowed amongst services? That’s insane. It’s a fair question and this still provides no real answer.
@@dave4148 It is of course possible to build a system of multiple processes interacting via synchronous network calls. Netflix, Spotify, and others do at least some of this according to various youtube presentations. However there are a lot of negative consequences - such calls introduce extra points of failure, possibilities of cascading failures, security issues, performance problems, difficult debugging, and more. That can be dealt with via tools such as distributed tracing, chaos monkey, automated API compatibility checkers,etc. But when components make important data available by broadcasting it, then the need for *many* synchronous calls just goes away - and so do those problems.
Hey, if my client wants me to build them a distributed monolith, I'm gonna build them a distributed monolith. Because when I tell them that what they want to build is a bad idea, they'll just refuse to listen, and they're just gonna get some other consultant to build them a distributed monolith. At least when I build it for them, I can ensure that their distributed monolith doesn't completely suck ash.
😂😂👏
Well, sounds like a pretty big waste of programmer potential. Just tell that client to f-off and get to work on a more sensible project.
Without execption every monotlih I've worked on the last two decades has been a ball of mud monolith. It requires skill and disipline in application architecture to keep a monolith in good shape over many years. In my experience most developers don't have that. This is where micro services really shine. When the application get small then application architecture matters less (but system architecture matters more).
I've faced many ball-of-mud monoliths too. There are of course tools to help with that - eg in the java world ArchUnit and jqassistant which allow rules to be enforced on the codebase. And proper modularity with good compile-time dependency organisation make it hard for developers to make too much of a mess without changing those dependencies - which reduces the problem of architecture oversight from looking at every commit to just ensuring dependency changes are reviewed by appropriately experienced people.
If a company doesn't have enough IT skill to apply those kinds of processes, I'm not sure that they will have any great success at building adequate microservices; they are IMO even harder to get right.
That is like saying, people have become dumb so lets create an architecture that is even dumber but reliable.
I understand that this one of the driver as it is inevitable to get around what you have currently (Gen Z) but then by fixing the human cost factor we now have increased the cost of doing business in terms of infrastructure setup and indirectly attributing them on the customer with minimal benefits (zero downtime with scaling)
@@mahee96 that's exactly correct however. A system that is more foolproof is always better, bar none. Developers and engineers in general aren't geniuses, our IQ is not that far off normal population. We don't all have the correct amount of experience and wisdom to always make the right calls. Anything that makes this process easier is welcome.
It is much much harder to build a clean distributed system architecture than it was to not build a ball of mud. The same developers will not suddenly turn into architectural geniuses because you made them do microservices.
Great talk. But I think he is misusing the term 'throughput'. Immediate consistency should be called low data latency (latency has to do with the time between request and a response or delay between an update request and the actual update happening everywhere). Throughput is how many updates you can do per second.
Thanks for the feedback. I'll try to remember to correctly use the word latency instead of throughput.
47:00 for orders not call the customer service during a sale seems off to me. How else would you know that they haven't recently updated their credit information or invalidated it. The eventual consistency would then require the customer to act after the fact. Which would be a terrible interaction for the customer."I have updated this already wtf why am I having issues".
How can you truly not rely on the performance of other services in critical paths where eventual data is not good enough. If anything you rely on the busses performance which is already slower than a http call straight to the service.
To me it feels like a bad example, because in my mind you need fresh customer data to get the sale.
Eventual consistency is measured in milliseconds.
I live in a community very similar to the picture you picked to criticize the distributed monolith. 😂 no hard feelings even though i like to live in here, its not as beautiful and pleasant as the houses of microservices.
It is very expensive to start with distributed app with cost as you develop the prototype
If you only have one team working on your application you should build a monolith.
So you agree that human resource is the actual driver here.
Then microservices should be under HDD (human-resources driven development)😉
Agreed. And that means that very often a microservice architecture should begin with a modular monolith.
Irrelevant..
Smart contract Virtual Machines are a modular monolith.
What's the proposed solution for problem #7?
The solution to synchronous communications is asynchronous communications. In other words, be very wary of creating these direct calls. If you want to write to another service, instead queue some information into a service bus that will eventually lead to data being synchronized. If you were trying to read from another service, you might need to create a duplicate of that information in the service that needs it.
@@jonathantower There are a few companies who, as far as I know, have managed to build workable microservice solutions based upon extensive synchronous calls - eg Netflix and Monzo. At least all the documentation I've see implies that synchronous calls play a big part in their systems. It feels crazy to me; I'm also in the async-where-possible camp - but it seems to work for them. Does their solution work just because of huge investments (brute force), or do they have some tricks that you are aware of?
@@jonathantower And if the orders service doesn’t have the data for that particular customer for whatever reason, then what? Or they update their shipping address, then order before the update is duplicated, or a bug prevented it from updating (but they don’t know because its async) now their product is sent to the wrong place. Seems insane just to avoid a bit of IO, which is not a performance impact with asyncio/green thread systems
I’d still have a sync call but cache customer info in the orders service and invalidate that cache on customer update events.
@@dave4148 Correctly maintaining the cache means there needs to be a reliable stream of events saying "data for customer X has been updated". So why not just include the new state for that customer in the same event, making the following synch. call unnecessary?
It's a very strange dichotomy HA vs IC in the context of microservises/monolith. It's completely orthogonal. You can make HA monolith and IC microservice perfectly fine without any hurdles.
The scale of your application can largely dictate if your claim holds. There is no way a service like Netflix can achieve IC, even for a few minutes, without grinding the entire thing to a halt due to latency issues that quickly build up. None!
@@smanqele again it depends from a scale, not from an architecture, isn't it?
don't build (preferentially) a microservice! 😂👍
Not having a previous application running for comparison doesn't mean you can only judge the new project by your feelings. There's also deductive reasoning that is used in science all over the place, including math. Sure, actually confirming old and new tech in an experiment is even better, but don't dumb it down to mere feelings either.
If microservices are supposed to be 'micro' in nature, why do we need separate teams. Isn't a teams supposed to manage a product (made of multiple services) and project (made of multiple releases)?
I often miss the argument that microservices are much easier to replace.
if you have trouble with MSA youll probably still have trouble with monolith.
ie, it has less to do with the pattern and more to do with the org.
Left out the easiest trap for falling into a distributed monolith. Required core/platform in-house libraries and required client libraries.
Thanks for the suggestion, Sean. I used to have more about this in the talk, but cut it out for time. I'll consider how to include it again.
Can you elaborate
@@strana As an example, it's common for the developers of some component A to provide a "client library" that others must use to communicate with their service. Then they announce that on a specific date they will release a new version of their component to production on date X which has a new internal API - but that's "fine" in their view because there is also a new version of their client library which matches it. However what this means in practice is that every other component which talks to A must be ready to also release on date X a new version of their code which depends on A's new client library - ie a company-wide synchronous release, ie a monolithic process.
Similarly, teams providing "platform" teams such as centralized logging sometimes declare "flag days" on which every software component must switch from some existing process to another. And sometimes business logic is encoded into "common libs", with business insisting that a new business rule in that library must be applied company-wide from a specific date - again meaning every team must release a new version of their component(s) linked against this new library version at the same time.
There are many variations of this kind of coupling.
@@simonk1844 This actually has a fairly standard solution: Multiple concurrent versions. Kubernetes is the exemplar for this. You have an "internal" API, and then different external "versions" which all translate to the internal API. You can add new endpoints, types, and optional fields without affecting anyone, but if you add a new required field or change a field, you must add a new API version with that change. You then deploy each api at its own prefix, and deprecate the previous version, support it for a pre-agreed upon time period before its removed. Basically the same idea as the new event type that was discussed in the talk, but for synchronous APIs.
In reference to 20:35 we are currently building our microservices using this structure and it works quite well. As long as you enforce the rule of no accessing schemas the service doesnt own then theres no problems. Its just that people always want to break this rule.
You then don't build it the way he addresses the anti-pattern in 20:35. In 20:35 the big database is one schema. You are building it with one physical database but multiple schemas. Nobody want's to break this rule.
The fact the rule gets broken by people in the real world is the reason we don't share the DB.
I got fired from a job for arguing with the DBA who wanted to have a single database for all the microservices xD
Skip the first 11 minutes. Here: 11:00
Do microservices but if you fail then Monolo
Iths can be good too. So i am safe front and back at the sane time.
20:46 This is not necessarily true. Even if multiple services use the same database you can always create different DB users for each service and use permissions to control which tables they can read or write. I agree that multiple services using the same database is bad. But the reason given here is not the main reason because you can easily prevent that on database level itself.
well there is a use for certain types of distributed monoliths when architected in certain ways. but non of them are professional ones. only for personal usage
only use case for distributed monolith is saving on performance/money with having single big database and coupling with single DB access layer but its cost saving measure at best so its only useful for personal applications(stuff you make for yourself)
So I just left a startup that purposely went with a monolith architecture to save time and money. The reason this is a rational choice is because in a startups often have features on their todo list that can 10X or more revenue. Not doing these features can mean death due to lack of funds. Monoliths are very much cheaper which means the money and time can be spent adding those 10X features and getting the business to a point where scaling becomes a problem. What is hilarious is that modern dbs and machines are fast enough that you can get quite far before worrying about scaling. AWS tools can push that out even more. We managed to punt the problem for a decade and were working on our series B when I left. Honestly we built the system to be easy to refactor into micro services so we could scale later so I feel no guilt about leaving any sort of problem for future maintainers. They shall be paid with the revenue we brought in to fix a problem we foresaw and planned for.
All architectural decisions are about tradeoffs. There is a cost to making different codebases independently deployable, and a cost to being limited to "all at once" deploys. Which cost is lower depends upon the number of codebases and the frequency of deployments. I can imagine a reasonable number of scenarios where deploying a system as a set of tightly coupled independent components makes sense. What is important is that the architecture is a deliberate decision with the costs estimated and documented; it's only a problem when this occurs "without planning".
Yeah, this is hitting a little too close to home 😅
I hope it was helpful. Thanks for watching!
@@jonathantower it was very helpful and our team have been going through it the last week to see what we could do to improve our chatty system's "low hanging fruit". Thank you for making a great and informative talk :-)
@@frankhaugen Glad it was helpful!
I've worked in investment banking for 20+ years on trading and risk systems. This is such a frustratingly over simplified talk! You *can* have high consistency *and* scalable high availability. In the JVM space, technologies like Coherence achieved that. In the open source world, similar architectures can be built with ZooKeeper and Kafka, where service nodes compete for partitions of a partitioned dataset and automatically rebalance as processes join and leave the cluster. Believe me, after a decade of building and debugging "true micro service" architectures that were plagued by data consistency issues, they only work in very narrow use cases. As soon as you know that there is a data consistency issue for a critical piece of operational reference data that's needed by several services (calculating sensitive things like real-time fees) your ivory tower micro service design is pretty much junk. 🤷🏻♂️
you are correct, because you work in a field where microservices cause more headache than anywhere else.
Yep, same experience we had.
Also I agree with you.
Best is to be flexible and build what is appropriate to the requirements and flexibilty.
MS for all uscases is not the best way to do it.
reminds me of the statment: "if you only have a hammer, everything is a nail"
Jonathan was very clear in his talk about checking first whether you have a problem that microservices provides an answer to. Your usecase doesn't appear to be something that microservices can solve, and that's fine.
I've been working in an environment with very complex business rules, leading to millions of lines of code being worked on by about 50 developers, with updates being released on average daily - and on some days, multiple features get released. Microservices are primarily used by us to scale the development process - and having decoupled codebases and data-schemas are important for that. However we also do have some performance needs, and appreciate that separate services can also be scaled separately, as can their databases. And I don' think this usecase is particularly unusual. On the other hand, we don't have the same scalability or reliability requirements that you appear to - so to each their own solution.
Microservices is built on eventual consistency. If this won't work for you, don't use them.
gRPC?
If I present 80 of those "Free 30 minute consultations" brochures do I get a full 40 hour week of consultations for FREE?
Guy, you've found a loophole. ;)
Yes, but you have to repeat the same question.
45:40 good joke!
Monoliths are not our nemesis. Microservices could be enemies of the enemies.
And the real enemies are the stake holders and YOU yourselves.
BTW you should consider that you don’t need Microsservices at all. Remeber the reasons why they are used.
Agreed. I always say you should have a "really good reason" (quoting Sam Newman) before creating microservices.
A typical software consultant...
- Your application is a distributed monolith! Here are these, these, and these problems!
- Okay, how do we solve the problems that led to these solutions?
- You have these and these problems!
- Okay, we understand. How do we address these specific business requirements?
- You have a distributed monolith, goodbye! (That'll be $100,500 from you)
that doesn't make him wrong, there are lots of vids that talks how to build/evolve to a modular monolith if that what's you've asked
Everything in engineering is a tradeoff. If microservices is a good fit for a client's needs, we'll try to educate them on the tradeoffs so they know what will be harder and what will be easier based on their choice. Same thing is true if we think a modular monolith is better.
Can you please put "microservices" into title to make it easier to find this video
hmm. Public interfaces are final except for very limited circumstances. Yes. Really.
This speech is good for beginners but very empty for seniors in services. Very important topics are not really covered.
Octavian, I'd love to hear more what you were hoping to hear about and see if it fits. There's only 50 minutes to get through a lot of information, so I can't include everything, but would love to make this as useful as possible. My main goal was to help people avoid choosing microservices for projects where they aren't necessary or a good fit.
@@jonathantowerI think touching a little bit more on what ‘event’ are (e.g you used in past tense, which is very important point to show that event are immutable by construction as the ‘happened’) as it could also having some people think of them as RPC or request/response protocols using this misleading ‘bus’ concept.
Also touching a little bit more about non happy paths a provide the importance of constructs (dead letter, circuit breaker, reconciliation) as complexity to tackle with (not entering into details, but as a indication of what always need to be addressed). Not too scary but a hint about good reason not to follow the micro-services approach if not equipped or prepared for those type of chores)
A great talk anyway (especially as a tool to communicate to management and their false perceptions of last new ‘things to do’ instead of what they really would like to have or dream of :-))
DBAs who are religiously chasing normalization are as bad as developers who religiously chase DRY by normalizing things that have no commonality semantically.
Truth. My experience is that DBAs are the most likely source of resistance for a well-designed microservices architecture, probably because it goes directly against everything they've always been taught.
@@jonathantower they are a cause of a lot of development dysfunction because they prioritize for operations not delivery.
You have to remember though that traditional DBA activities have their basis in the monolith world. With MSs the recommendation is one service one team, replete with its own independent DB resource. If you follow that logic you are unlikely to run into such conflicts.
As a DBA and a developer, I completely agree. It's a balancing act, really; it's usually worth the effort to deduplicate, but making it your ultimate goal is only going to cause headaches for everyone. Adding a field for OrderTotal to avoid a join/group/sum every time you just want to pull an order history might be warranted as a performance improvement. Adding Name2/Address2/Phone2/Email2 for an extra account contact means it's probably time to make a new table for contact info, plus you get the flexibility to add as many contacts as necessary without extra schema changes.
On the other hand, I've dealt with databases so badly denormalized (case in point, an inherited post-Access database, with nearly 100 fields on one table named things like "BoardDate3" and "Status8"), the only thing preventing me from attempting normalization was that safely reorganizing the years of existing data and updating the application to match would be a bigger headache than simply maintaining it as-is. I still kind of eventually want to, if only for the challenge and to keep my successor from the same quandary.
Normalization Druids and DRY Cultists are actually modern day's pagans. 😉
The same speakers 20 years ago that praised the ejb and weblogic are now praising architectures and methods that are ultimately harmless. DON'T TRUST THIS Guys. Do your own research.
1) Personally I didn't understand why a common database suddenly causes coupling between modules. Technically it is possible but if you do a clear module separation and code reviews, you won't be able to "accidentally" use a table from another module. Simply don't access tables that don't belong to your module and you will be just fine.
And what means "get away with it" [29:30]? If you have smart asses in your team, shared tables won't be your biggest problem 😉
2) Brownfield projects: I don't know what was author's previous life experience but I remember times when there was no documentation, people left the company, people didn't remember why certain code was written, people didn't want to explain how the code works because they were in a different department already and were not payed for consultancy, etc. So, theoretically you can have virtually any help you need but practically you suffer a lot 🙃But from another point of view you have something to debug and sooner or later you will find out how everything works.
yeah, dude introduced a misjudgment there, if someone writing microservices, a change in db table should involve testing for any microservice using it. its not a paradigm problem, its code organization problem
A shared database creates coupling because it makes it difficult for independent teams to deploy updates to different services totally independently, one of the goals of microservices, and Google's actual definition of them. Now they all have a dependency on the same database, which means they have to be versioned together with any database changes.
Brownfield projects are complicated and messy, BUT unlike starting from scratch, you have something that works while you're developing and a benchmark for performance to know if you're making it better or worse. Thanks for watching!
@@jonathantower "makes it difficult for independent teams to deploy updates to different services totally independently" - okay, maybe you explain us why? Let's assume separate teams (i.e., microservices) use a single shared database but each of them use different schemas and there are no cross-references between these schemas. How would a team "A" know that a team "B" changed one of their tables?
"goals of microservices, and Google's actual definition of them" - there are very few Google-level projects with similar scale, resources and problems, so there is no point in discussing what Google decided to implement😉
"which means they have to be versioned together with any database changes" - no, they don't. Each logical part (i.e., schema) can be easily versioned separately.
"a benchmark for performance to know if you're making it better or worse" - very questionable statement because (1) these performance targets are set on the beginning of the project, so usually this is your reference, and also (2) a new project will have different architecture, API, etc, so it can't be directly compared with an existing application.
But if it is a migration from techstack XY to XZ with fully compatible APIs, then it makes sense.
@@volodymyrusarskyy6987 In your scenario, the databases a separate schemas that just happen to be stored in the same database. This is virtually separate databases. This is mostly better than sharing a database.
However, if one of these databases has a slow, long-running query, it now has the potential to slow down all the other virtual databases since they share resources.
This, unintentional coupling.
@@jonathantower "This is mostly better than sharing a database." - sharing a database is sharing a database, even if you use different schemas. It is not better, it is still a shared database. What you are referring to is a "shared schema" which IMHO is a huge fail for any decent project.
"However, if one of these databases has a slow, long-running query, it now has the potential to slow down all the other virtual databases since they share resources." - first of all, this is true even if you have separate databases that share same hardware 😉Try to copy a 10 GB table in one of the database on a single server and start measuring query performance in another database 😉 Your DB admins will call you for a talk same day. Secondly, this is not always strictly true and depends on the query and a physical layout: do your tables reside on the same physical disk? do your queries intensively use TempDb? how TempDb is physically structured and where is it located? were data pages cached in memory already? where you execute those queries (master/replica nodes)? etc.
"This, unintentional coupling." - this is not "coupling" that we are talking about as modules/services are completely independent. The fact that they share same hardware resources in the background doesn't force them to have dependencies on each other. If your statement was true, all the cloud apps would be "coupled" automatically out of the box.
Micro services aren't allowed to call other services? Only communication via event bus allowed? This is a very bold claim.
Agree. this is not good advice. MS can call other MS. It all depends on business domain and whether that business transaction must have immediate consistency or has the tolerance for eventual consistency.
Yes, a bold claim. However I'd agree that it is a worthwhile goal.
Synchronous calls have a whole lot of disadvantages which just go away if you can avoid those calls. For any user request, the latency is the sum of the latencies of each service it passes through, plus the latencies of the network connections. Similarly the reliability (uptime) of any endpoint is the product of the reliability of every service it passes through and the network connections between them.
Testing a service also becomes much easier when synchronous calls to other services are reduced/removed.
Any business which existed before computers must support eventual consistency. Previously business workflows were implemented via paper forms that were carried from department to department - and those businesses worked. Banks used to support international transactions in the days of sailing ships.
Most important of all: ask any business expert whether they prefer their IT services to remain mostly-up during outage of a specific service, or whether they prefer immediate consistency. They will always want both, but in most cases will settle for being mostly up - and asynchronous communications are a big help there.
Easiest way to avoid doing microservices wrong: don't do it until it absolutely needs to be done. Debugging distributed calls in real time during prod outages is no joke.
One of the points raised in the talk was that if you have distributed calls in your hot path - you are doing it wrong.
@@Bokto1I only ever see people saying things are wrong about various approaches. But the mystical "right" way never seems to appear. The software industry really loves the no true Scotsman fallacy doesn't it?
@@seandavies5130 you seem to have selection bias. SOLID, DRY, loose coupling, no premature optimization - tons of practices are loudly stated to be the "right" way.
@@Bokto1 interesting, both examples of microservices I've seen in the wild don't possess those qualities, but let me guess... they just didn't do it right. Also, pay no attention to the fact that one of them was lead by a highly experienced architect. No no, it's only a problem with the people, not the idea. You know, in science when a "successful" experiment cannot be replicated it is regarded with suspicion, not adulated with religious fervour. The trajectory of this idea will likely follow this path: more and more people will point out it didn't work for their team/problem at which point the offending "thought leaders" will turn around and say "no, we never meant that it was a general solution to all problems only a restricted methodology for a small set of problems". And the whole emperor's New clothes style charade will continue
@@seandavies5130 you seem to like science, do tell how many articles nowdays describe fundamental laws of the universe in one swoop, and how many look for special-case-under-normal-pressure? Ofc the right ways will have asterisks. Ofc there are more wrong ways than right ones. The industry is complex and generally it is expected from "an engineer" to think about his exact case. But if you, say, have reasons to violate DRY, it is not the same if you just never thought about it.
Except theory nothing is there and I hardly understand a single thing
Why do we call it a brownfield project. Could we not have chosen a different colour. rofl.
'don't work in silos' - 'don't work in monoliths' which is it nerds?
Don’t forget to mention that you the owner of Trail Head and you like some business.
(Edit: I stopped watching the talk ~10 minutes in. I just skipped through it and a bit later there's some praise for monoliths. The way the talk started put me off from watching the rest)
This is a pretty poor talk. Monoliths are described here as they were built 20+ years ago. I would recommend reading about the idea of "majestic monolith", even though the idea of good monoliths is a lot older. An example: in the talk is mentioned that "scaling a monolith is a matter of adding more CPU or memory", this is not true. Well built monoliths scale horizontally. Facebook, Basecamp, github and many other orgs have architected their main applications as monoliths.
I’m in the first ten minutes and he’s mentioned load balancing and sharding twice. Both of those were common approaches 20 years ago whereas vertical scaling was less so because large cloud instances weren’t a thing.
Don’t know how the talk will progress but so far he’s IMO well in line with “modern” approaches to monoliths, to the extent that those even differ from the practice of 20 years ago. (The term “majestic monolith” signifies nothing to me but an attempt at branding)
@@andsnpl hello! This is just my impressions and as such will be wrong :) - I think monoliths are misrepresented in the talk as something bad and I really disagree with that. I believe that if an org cannot build a good monolith, there is a very high chance it will fail to build good microservices. A well built monolith is easy to split into microservices: it will have bounded contexts, and communication between them needs to happen between well defined interfaces/adapters.
I agree with your point about 'majestic monolith', that's just a name DHH came up with to counter the false belief that microservices are the best solution to any problem.
Sadly, old software still exists, and these talks usually point to them. A large portion of those softwares are kind of obselete that you wish to recreate it but stakes are too high, and you, as an engineer, are usually stuck in between cutting edge and too old school software.
FourTet, I encourage you to watch the whole thing. My point was to redirect developers back to monoliths for most projects, and only to use microservices when they are a great fit for their needs and they can live with the tradeoffs that come with them.
Pretty close to the beginning, I talk about modular monoliths and how those are a great way to build an application, and make the distinction between them and ball-of-mud monoliths.
These talks are not getting any better, its the same old "Introduction into XXX". Very basic and superficial...
Msp 12, I'd love to hear more what you were hoping to hear about and see if it fits. There's only 50 minutes to get through a lot of information, so I can't include everything, but would love to make this as useful as possible. My main goal was to help people avoid choosing microservices for projects where they aren't necessary or a good fit.
What a narcissistic guy! He's pretty much sure, he's the only one who knows stuff and tries to be funny to hide that. Fails on both fronts.
if you're going to have cross platform logging, you're not going to write a damn library... it's a non issue because people ain't dumb enough to do it...
Ivica, you'd be surprised! I also mention not to create your own service bus, too. Both of these are mentioned because I've seen them done multiple times.
@@jonathantower fair enough. it's a bit concerning such people are still in the industry though...