Zerodha Stock Broker System Design with
Вставка
- Опубліковано 5 лип 2024
- In this video, we discuss the high-level design of Zeroda, a stock broker app. We start by talking about the app's capabilities, including features like watchlists, market depth, stock performance analysis, buying and selling stocks, and portfolio management. We also mention the order book and trade book, which track orders and transactions.
We explain that Zeroda acts as a broker between clients and stock exchanges like the National Stock Exchange (NSE) and Bombay Stock Exchange (BSE). Zeroda takes orders from clients and places them with the exchanges. The market depth feature allows users to see the number of orders and their prices in the market. This information helps users make informed decisions about buying or selling stocks.
We discuss different types of orders, such as market orders and limit orders. Market orders involve buying or selling at the current market price, while limit orders allow users to set a specific price at which we want to buy or sell stocks. We also mention the importance of real-time stock price updates and the need for a fast and reliable system architecture.
Towards the end of the video, we briefly touch on the challenges of connecting to stock exchanges, including the cost of acquiring permissions and the need for reliable communication lines. We mention that exchanges strive for fairness by ensuring equal wire lengths for all brokers to minimize the latency of updates.
The video sets the stage for a discussion on designing the high-level system architecture of Zeroda, focusing on real-time data updates and communication with stock exchanges.
00:00 Intro
00:48 Stockbroker Requirements
12:11 Joke - Message queues aren't free?
12:44 Some trivia about stock machines
15:00 WebSockets for stock updates?
20:00 Capacity estimation of updates
23:30 Order Management
26:09 Some more trivia - War Story
28:30 Deciding on service boundaries
36:36 Message Queue - needed?
40:55 Cache - Global caching?
44:39 Executing orders
50:53 Real-life connection purchasing
52:52 Deep dive into watch lists
59:14 Eventual consistency for updates?
1:02:40 TCP vs. UDP for watch lists?
1:03:48 Validating the architecture
1:05:44 InterviewReady Discount!
System Design Course at InterviewReady: interviewready.io?_aff=KEERTI
Designing Data-Intensive Applications Book: amzn.to/3SyNAOy
Keerti is at:
UA-cam: @KeertiPurswani
LinkedIn: / keertipurswani
Instagram: / keerti.purswani
You can follow me on:
Github: github.com/InterviewReady/sys...
Instagram: / interviewready_
LinkedIn: / interview-ready
Twitter: / gkcs_
#SystemDesign #InterviewReady #Coding
![Lp. Последняя Реальность #97 ЧЁРНАЯ МАТЕРИЯ [Анти Скинт] • Майнкрафт](http://i.ytimg.com/vi/k4MjXCzKsYw/mqdefault.jpg)
Thank you for having me Gaurav! It’s always fun to design with you! 😇
Hi Keerti and Gaurav,
The way through which the data is communicated to the brokers via Exchange is way too much efficient and the assumptions that are taken are far off from the real values.
1. The number of stocks on NSE + BSE is around 7.5k, but apart from that there are derivatives of the stocks and indexes (NIFTY and BANKNIFTY) the most popular ones. These derivatives are the most traded ones on the exchanges. If you include these derivatives as well, the number goes beyond 1Lakh. There are times where the number of ticks (Liquid + Illiquid) goes beyond 1 lakh ticks (websocket ticks) per second. The market goes for around 6.5hours. That's humungous.
2. The exchanges opens up UDP port for multicasting. You have to listen to these multicast ports to get the information. Now that information is not a simple JSON payload or an array of numbers. It will be very inefficient and costly. It is in a compressed format that too using a very old compression algorithm as it was designed years ago and it is still being used.
I have left out way too many things and this is just for listening the feeds by the broker. The real challenge comes when you listen to these feeds and distribute it towards your customer base which is growing. So scaling comes next.
I would recommend going once through the documentation of packet -
archives.nseindia.com/content/press/Realtime_FAO_L1.pdf
This is just for educational purposes. :)
A genuine feedback, The broker in itself is a very complex design, each feature of it actually is complex. What you could have done is taken one feature of it and then end up explaining it, in this way you could have been able to do more research on it. Just my honest feedback, nothing else.
Kudos to both of you.
Thank you!
I dont think we need to keep track of all watchlist for each user on server-side.
- Rather, we can simply keep watchlist info on client side(like watchList1:{stock1, stock3, ...}).
- In backend, WatchlistService can just keep a mapping of user to stocks he is interested in(across all his watchlists) and send the user this group of stock entries every second(or whatever freq we decide).
- Then, the client can do the job of segregating these stocks into separate watchlists.
Hey Gaurav, you really need a trader in your discussion. There are many wrong assumptions in terms of watchlists and users and order types and whatnot in this discussion. There is quite a mismatch between the reality of order types limit, Market, and GTT and what you are understanding. A limit order is not trigger-based. It is placed immediately to the exchange but at a given price. Now if the exchange finds a counterparty who agrees to that price, your order is fulfilled. For market order no price is given to exchange, stock is bought or sold at any available price. GTT is actually a trigger based where the broker triggers the order at a given price when the price of a stock crosses a certain threshold. Let me know if you want to rerecord another session, would be happy to be part of it or give you pointers offline as well. I am a trader as well as a technical guy who understands and is interested in system design. As always, you are quite fantastic guys and a good effort.
Thanks RK, where can I reach you?
Nithin kamath entered the comments section.
It'll be an honor haha 😁
Sorry if I missed , here are you using SQL or NO SQL for translation data, apart from catching service.SQL might will give performances issue. Also for inter service communication is REST API used , might gRPC is give more performance. Just my thoughts. 😊
The exchanges usually send out feeds with prices and quantities. The broker needs to to be in multicast group where they receive this information.
Pull based service for limit orders not going to fly. AFAIk Brokers OMS push the pending orders at day start and exchange push event if there is any change in orders.
Signal R is configurable and has a default fall back mechanism to fall with the precedence like web sockets -> sse -> long polling, serves all the purposed at once
Her handwriting with a stylus is pretty great tbh, lol
10 K for stocks.. well there are options and futures at different strike prices and the symbol list goes in 100K nearly. and broker needs to send this to ~1 crore people at the same time.. along with past data once .. to plot the chart.
Estimation needs to be done that way.
Explained as simple as possible by @KeertiPurswani, also the discussion is informative. And the need SQL and No-sql. Listening to you guys is really upgrading the knowledge, getting used to system design 😀😀😁. Also the need of TCP ack, ans importance of
Validation services.
I think @Gaurav is correct. For Watchlist - Prices are eventual consistent like watching mactch on hotstar i.e. near real time. Great Discussion. I wish I would have view this video before one of my interview would have not making mess in interview.
I have used SSE for Stcok Exchange communication, Socket for Human User/Client to Zerodha communication.
I did FR,NFR,Capacity Estimation, API design, after 25 min I was clue less what to discuss.. Well done @Keerti for deep dive. Got idea how to deep dive into components.
Great video! Loved Keerti's presentation. She managed to keep things simple and visualize her thoughts making it easier to understand.
Lover Gaurav's inputs and facts too! 🙂 Thanks for this
Thank you so much! 😇
Thank you so much Gaurav and Keerti, great information.
Thank you!
This was beautiful!!!!
Thanks Gaurav and Kirti!
In my view if compared to other videos of Gaurav this one didn't have any structure and wasn't well curated it seems.
Hey, thanks for the feedback, could you share how this be more structured? Will help us improve next time! 😇
I actually enjoyed this discussion more than most cut and dried talks on system design.
This is closer to a real team discussion or interview. Structuring is more of a hindsight thing 😅
@@gkcs
Hey Gaurav, I also feel that most of the design videos on your channel lack structure. Agreed, that this is how a real system design discussion occurs. But in interviews, I feel having a structure/order becomes important.
Also, do you suggest going with an unstructured approach during interviews or should we follow some sort of an order like jotting down requirements first, then estimates, then data model and finally the meat of the services, etc.
Great video. Looks like lot of things to learn not only in system design, but in the stock market as well :) !!
gaurav can you do a mobile banking and credit card application please high level?
what is gkcs full form?
Why do we need cache for watchlist? Prices of the stocks which are in Watchlist will frequently be updated (every second) .
Is it correct or am I missing something
We can cache the stocks in the watchlist. Their prices will change but the stocks will change only once in a while.
Watchlists will be viewed frequently, so having the stocks cached makes sense.
I believe she means caching the list of stocks, not the underlying pricing data.
@@ITTIsiKhushi-keertipurswani
Oh, now I got it.
Ticker names could be cached and getting the name of the ticker( to fetch the prices from separate api) from cache will be faster compared to fetching the tickers name from DB.
Understood.
Thank You 🤗
Damn. I replied from the wrong account by mistake 😅
@@KeertiPurswani what exactly does once in a while mean? Lets say some traders update their watchlist every day, will it still be feasible to maintain watchlist in cache?
Great Content
I was asked the same question in Datbricks interview, hope I had landed here before.BTW I had given the polling based solution which the interviewer did'nt agree
👍
Before 22:00 when she was saying just to fetch data for wishlist, I got confused, because end user will have wishlist but she was explaining connection between broker and exchange. But then Gaurav you point it and got that cleared at 22:10
here is the service that is being provided by seperate server
1. You have to decide on the roles between Zerodha and NSE. Is GTT reposnibility of Zerodha or NSE?
2. NSE does not have to maintain 10M watchlists, Zerodha has to. Zerodha will get all stock ticks from NSE in 1 request and then serve it to their clients as per their watchlist. Zerodha will not make 10M calls to NSE every day.
Video could have been bit more edited in order to reduce the length. Too much of internal clarification is going on which could be edited.
every 50ms would be terrible. high frequency trading is in nano seconds :)
Zerodha does not provide HFT to end users (nano seconds is impossible on the poor connection speeds we all have, including 5G).
watch dogs 😂