This was one of the best system design videos I have seen. You didn't brush over important concepts, went into good low level details and first principles and compelled me to take out my notebook and take notes! I really wish you make more of these!
You explained flight booking flow in quite detailed manner while focusing on all important topics. Which i really liked. I have seen so many videos on system design, usually people don't dive deep into these concepts. You did everything in balanced way, which makes this presentation perfect.
your explanations made me understand how to ask questions with the interviewer and how to discuss the details of the system overall fantastic experience 💯
Great video. I have some feedbacks. 1. Talk about Orchaterator vs Choreographer pattern. Booking Service is acting as a Orchasterator . We can also use decentralized system like Kafka. 2. 2PC is generally not practical to achieve distributed transaction. A better approach would be SAGA pattern and having compensating events. 3. Also, talk about creating some derived data. For example, is a flight is booked completely . Keep this info in cache and keep cache in sync with DB so that cancellation would mark this booked flight to available.
Great explanation. Appreciate that you took into account the scale part and actually discussed the numbers. Thats an important part and some interviewers like to ask it for mainly senior roles. Thank you!
40:15, You may not need to rollback after payment booking. What you can do is have a SQS queue in place and a scheduler will update the Booking Info let's say after 10 mins.
great video and thank you for sharing. I have a few comments. 1) about calculating TPS, it should not be considered using a daily estimated traffic, because traffic normally is not distributed evenly and it fluctuates. It is important to think about TPS as exactly "how many transactions does your system need to support per sec in the worst case scenario at any given time of the day". 2) avoid using distributed locking, 2-phase commit is complex and should be avoided. use some sort of recovery mechanism instead. 3) not sure about locking seats using DB locks, you can use DB to lock rows, tables or even columns for certain DBs but what you need here is some locking mechanism that you will need to design on top of DB locks to achieve atomicity or simply an in-memory cache can also do the job. 4) optimistic locking is designed for high concurrency and pessimistic locking kills concurrency.
You said to "avoid using distributed locking." Well then how do you handle/create a ticketing system (ticket master), with millions of users but only 100K seats event for every popular concerts/ sporting events? In big events, there will be high demand in a short period (hours) trying to buy the few tickets.
One thing you could do for the payments is simply process them later. At the moment what you need to fetch from the user is a payment method and this is something you can setup with Stripe. So if you manage to get a payment method, temporarily store in the database along with your booking and the seats, and bundle that in the same unit of work, you could then use a transactional outbox pattern to process the payment later. If something’s wrong with the users payment method you remove the seats and make the booking available again, this would free you from dealing with the refund service altogether and turns the system into a more event and domain driven system.
In the real world, I don't think the business side of the company is going to like this approach. The company wants to lock in the sale! The customers would assume their airline tickets are booked and go to the bed at night then straight to work in the morning. They may not see your notifications for hours or even days after the failure. What do you do with the seats in the meantime? Hold them? What happens if the customer changes their mind? Then you hold on for seats for 24 hours for nothing! If you don't hold the seats, then I miss my flight. In my opinion, it's better to fail fast! The customer would pull out another credit card and buy the ticket! The money goes into the company bank account!
I think in the reality, when customers are searching "available" flights, they are expected to get flights with availble seats, not the flights without seats. This means in the search path, we also need to visit some kind of inventory table to query available seats. Also, the flight table does not consider the situation that with one flight like "AA101", the flight will re-fly for the same routes iteratively, it is not one row can suffice that in the flght table.
Hey, I have a question on the flight information and flight seat information table. I am trying to understand how you handled concurrency in the system. Are you implementing concurrency control using materialized conflicts? Are you going to have a record present for each flight-seat in the table even before a booking happens and then use locks to handle concurrent requests for the same seat?
In scaling estimates for flight seat info table,100B*100*100*365 = 365,000,000B ~ 365 MB... i see its 3.65GB. should that be 365 MB? please explain.Thanks!
Suggestions - for flight information table, we have primary key as flight id but most relational dbs create unique index on primary key across partitions so we shall better than flight id and departure date as composite primary key. Though I like the idea of departure date as partition key. Here, we are assuming that if same plane does multiple round trips between two destination, each will have a separate flight id on given day. Also, instead of booking open, we can call that column as booking status - instead of just open and close, there could be more status like flight cancelled.
@@ethanz4928 of course flight Id is unique. The concern here is not about uniqueness but accessing data faster for most common queries. As far as possible query shouldn't require search across partitions.
Thank you very much sir for informative video. This is really very helpful. It would be great if you could create video for coding for this airline system
Date should also be part of seat information as booking status for a seat would be specific to date, same seat can be booked on one day and not on another.
Great explanation! Thank you so much for the video. One question though, you mentioned towards the end that databases should support linearization/consistency, can you please tell how to actually achieve consistency in such a distributed storage system with multiple read/write nodes?
The flight id in the Flight Seat Info is technically a Foreign Key, right? Seeing two PKs throws me off since I'm pretty sure you can't have two in a table
how does the movie booking companies handle the locking part, does this decision depends on the criticality of the usecase, for example we are okay with the user to retry here, we can take an optimistic locking and in other ways we choose pessimistic locking.
one small nitpick, the partition key on departure date doesn't make lots of sense. It can easily introduce scatter and gather problem if user searches a range of date. srce-destination is a good choice based on the access pattern.
Departure date partition key makes perfect sense. Why would anyone search for flights using a date range? Nobody provides such a feature either. Flight search is always done for a single date.
some src destination can be hotter than others, so it wouldnt make a good partition key. partition on flight id with departure date as sort key is probably a better option imo
Why can't we say that the search service should also be highly consistent? What if it is not consistent? If it is not. It can show seats as available which are actually not available
Scenario : Lets say while booking a seat, a seat is locked and ttl is set. After payment and updating booking status, it should update the seat state to 'BOOKED'.But we know that the same row is already locked and the update would fail, which would make the entire booking fail.
You've completely ignored the complexities of search. How do you search for 1- stop and 2-stop international flights? Which db would you use? How would you distribute that kind of data? You'd need a graph db to sort this out. You don't even mention that
This was one of the best system design videos I have seen. You didn't brush over important concepts, went into good low level details and first principles and compelled me to take out my notebook and take notes!
I really wish you make more of these!
really underrated video. Deserves more likes
This is one of the most underrated video! It deserves more views.
You explained flight booking flow in quite detailed manner while focusing on all important topics. Which i really liked. I have seen so many videos on system design, usually people don't dive deep into these concepts. You did everything in balanced way, which makes this presentation perfect.
your explanations made me understand how to ask questions with the interviewer and how to discuss the details of the system
overall fantastic experience 💯
Great video. I have some feedbacks.
1. Talk about Orchaterator vs Choreographer pattern. Booking Service is acting as a Orchasterator . We can also use decentralized system like Kafka.
2. 2PC is generally not practical to achieve distributed transaction. A better approach would be SAGA pattern and having compensating events.
3. Also, talk about creating some derived data. For example, is a flight is booked completely . Keep this info in cache and keep cache in sync with DB so that cancellation would mark this booked flight to available.
Great video !! Clearly explained everything in a simple manner. Thanks so much.
Great explanation. Appreciate that you took into account the scale part and actually discussed the numbers. Thats an important part and some interviewers like to ask it for mainly senior roles. Thank you!
Such a great video, excellent content presented in a very well balanced way. Thanks!
Really nice and in-depth video.
40:15, You may not need to rollback after payment booking. What you can do is have a SQS queue in place and a scheduler will update the Booking Info let's say after 10 mins.
Concepts are clearly explained and Easy to understand. Thanks for making this video.
Very Nicely explained !!! Best video for this topic ..thanks for sharing
Such a detailed video. Thank you!
Very detailed video. Thanks a lot.
Beautifully explained! good one
Thanks for the quality content.
Is it possibly or advisable to switch between optimistic lock and pessimistic lock on the fly? For example, if the airline seats are
Very good explanation. Will aid me in my quest of learning system design.
Anna please don't stop uploading videos. We love you 💘 excellent teacher.
nice explanation and great work
Excellent analysis.
Thanks, this is very useful for me, really appreciate your efforts
great video and thank you for sharing. I have a few comments. 1) about calculating TPS, it should not be considered using a daily estimated traffic, because traffic normally is not distributed evenly and it fluctuates. It is important to think about TPS as exactly "how many transactions does your system need to support per sec in the worst case scenario at any given time of the day". 2) avoid using distributed locking, 2-phase commit is complex and should be avoided. use some sort of recovery mechanism instead. 3) not sure about locking seats using DB locks, you can use DB to lock rows, tables or even columns for certain DBs but what you need here is some locking mechanism that you will need to design on top of DB locks to achieve atomicity or simply an in-memory cache can also do the job. 4) optimistic locking is designed for high concurrency and pessimistic locking kills concurrency.
You said to "avoid using distributed locking." Well then how do you handle/create a ticketing system (ticket master), with millions of users but only 100K seats event for every popular concerts/ sporting events? In big events, there will be high demand in a short period (hours) trying to buy the few tickets.
Thanks a ton!!! Quite clear and amazing
Glad it helped!
this is so so helpful thank you!!!
truely awesome
The content is amazing
Very Nicely Explained.
One thing you could do for the payments is simply process them later. At the moment what you need to fetch from the user is a payment method and this is something you can setup with Stripe. So if you manage to get a payment method, temporarily store in the database along with your booking and the seats, and bundle that in the same unit of work, you could then use a transactional outbox pattern to process the payment later. If something’s wrong with the users payment method you remove the seats and make the booking available again, this would free you from dealing with the refund service altogether and turns the system into a more event and domain driven system.
In the real world, I don't think the business side of the company is going to like this approach. The company wants to lock in the sale! The customers would assume their airline tickets are booked and go to the bed at night then straight to work in the morning. They may not see your notifications for hours or even days after the failure. What do you do with the seats in the meantime? Hold them? What happens if the customer changes their mind? Then you hold on for seats for 24 hours for nothing! If you don't hold the seats, then I miss my flight. In my opinion, it's better to fail fast! The customer would pull out another credit card and buy the ticket! The money goes into the company bank account!
Thank you!
good job gentelmen!
Very well explained.
I think in the reality, when customers are searching "available" flights, they are expected to get flights with availble seats, not the flights without seats. This means in the search path, we also need to visit some kind of inventory table to query available seats.
Also, the flight table does not consider the situation that with one flight like "AA101", the flight will re-fly for the same routes iteratively, it is not one row can suffice that in the flght table.
i also think this key aspect got missed
Hey, I have a question on the flight information and flight seat information table.
I am trying to understand how you handled concurrency in the system. Are you implementing concurrency control using materialized conflicts? Are you going to have a record present for each flight-seat in the table even before a booking happens and then use locks to handle concurrent requests for the same seat?
In scaling estimates for flight seat info table,100B*100*100*365 = 365,000,000B ~ 365 MB... i see its 3.65GB. should that be 365 MB? please explain.Thanks!
Very well explained
Suggestions - for flight information table, we have primary key as flight id but most relational dbs create unique index on primary key across partitions so we shall better than flight id and departure date as composite primary key. Though I like the idea of departure date as partition key. Here, we are assuming that if same plane does multiple round trips between two destination, each will have a separate flight id on given day. Also, instead of booking open, we can call that column as booking status - instead of just open and close, there could be more status like flight cancelled.
I thought each flight is unique in this world isn’t it?
@@ethanz4928 of course flight Id is unique. The concern here is not about uniqueness but accessing data faster for most common queries. As far as possible query shouldn't require search across partitions.
Thank you very much sir for informative video. This is really very helpful. It would be great if you could create video for coding for this airline system
fantastic video, would be even better to show how the site architecture will interact with multiple GDSs, airlines, and NDCs in real life scenarios
Date should also be part of seat information as booking status for a seat would be specific to date, same seat can be booked on one day and not on another.
Very informative
Thanks man, could you tell me which tool do you use for the drawing ??
Great explanation! Thank you so much for the video. One question though, you mentioned towards the end that databases should support linearization/consistency, can you please tell how to actually achieve consistency in such a distributed storage system with multiple read/write nodes?
yes i agreee i did get same question in google and and I got offer.
Really helpful
The flight id in the Flight Seat Info is technically a Foreign Key, right? Seeing two PKs throws me off since I'm pretty sure you can't have two in a table
this is called content.
how does the movie booking companies handle the locking part, does this decision depends on the criticality of the usecase, for example we are okay with the user to retry here, we can take an optimistic locking and in other ways we choose pessimistic locking.
Can you use saga pattern instead of a 2 phase commit?
one small nitpick, the partition key on departure date doesn't make lots of sense. It can easily introduce scatter and gather problem if user searches a range of date. srce-destination is a good choice based on the access pattern.
Departure date partition key makes perfect sense. Why would anyone search for flights using a date range? Nobody provides such a feature either. Flight search is always done for a single date.
some src destination can be hotter than others, so it wouldnt make a good partition key. partition on flight id with departure date as sort key is probably a better option imo
No stones unturned !
i liked the the sound of water pouring
nice content !! appreciate if you can zoom in bit.
Is pessimistic and optimistic lock implemented as a feature of the database?
Search service can have caching to reduce db read queries as most data may be static
Your flight seat information estimate (25:07) is off by a factor of 10.
Why can't we say that the search service should also be highly consistent? What if it is not consistent? If it is not. It can show seats as available which are actually not available
you need row locking so the seat will not be available
Scenario :
Lets say while booking a seat, a seat is locked and ttl is set.
After payment and updating booking status, it should update the seat state to 'BOOKED'.But we know that the same row is already locked and the update would fail, which would make the entire booking fail.
Hello, we are dedicated IT service team providing dedicated IT service to your business, we are looking forward to hearing from your team
You've completely ignored the complexities of search. How do you search for 1- stop and 2-stop international flights? Which db would you use? How would you distribute that kind of data?
You'd need a graph db to sort this out. You don't even mention that
Really?How about you make a video about that?
🤣🤣