How Hotstar Application Scaled 25 Million Concurrent Users | Performance Testing | Load Testing
Вставка
- Опубліковано 24 сер 2024
- Credit : @HasGeekTV
The video is uploaded here for educational purpose
👍 Like us on Facebook : / perfology
👍 Add us on Instagram : / perfologys
👍 Follow us on Linkedin: / perfology
Speaker bio
Gaurav is a Cloud Architect at Hotstar, India’s leading OTT provider that holds a global concurrency record for live streaming to 25Mn+ viewers. At Hotstar, he loves building cost-efficient infrastructure that can scale to millions in minutes. He is also passionate about chaos engineering and cloud security.
The talk will focus on why traditional autoscaling doesn’t work for Hotstar (Disney’s OTT streaming service) who recently created a global record for live streaming to 25.3 million concurrent viewers. It will talk about challenges in scaling infrastructure for millions and how to overcome it, how we run gamedays before actual games, how we use load testing monster called Hulk to prepare for peak traffic and also how we add chaos (engineering) in between all these epic scale madness.
1. Outline
2. About Hotstar
3. Why traditional autoscaling doesn’t work for Hotstar
4. Challenges in scaling
5. Gamedays and load testing for millions
6. How we prepare for peak traffic (50M)
7. Chaos engineering at high level
Slides
docs.google.co...
wow those numbers are insane. I would love to see a cost graph during the match as well
Here after 54 million in IND v NZ Semi-Final of 2023, crazy to think that the new record is DOUBLE than this!
54 Million
58 million IND VS AUS world cup 2023 final
@@akshaysingla951touched 59M
Thats 54 million
Yup, my bad@@lfe360-sagarsvlogs3
streaming services: oh no.. , spikes are nasty,,
cable: hold my beer...
Technically cable loads are predictable and it’s broadcasting .So, thing is a lot different
Cable technology is truly impressive...
He is showing spikes in the video... For cables it's just another Tuesday
Cable technology is a broadcast even without any acknowledgement from the receiver. That is exactly is what is preventing anyone from rewinding 5 minutes to see a part they missed.
For streaming services it is like every user is having their own channel.
@@nikhilgv9 so is it a possibility that the system will crash if half of the viewers suddenly decides to rewind?
@@nikhilgv9You are talking about the analogue era, nowadays due to digitisation cable has become IPTV.
And now jio hired all hotstart employee to build juocinema 😅 and now jio is also able to handle that load. Good video❤
and handled of about 500M views during CSK vs RCB playoff selection 2024
Jio cinema shows cumulative views not concurrent views
Assembled all AWS Cloud & DevOps Engineers 😂😂
Watching this after 58M concurrent users in IND vs AUS final.
pretty fascinating that if service is degraded they'll just allow anyone to watch the video without logging in or paying. Very nice of them and makes sense, for live sports matches, better to let everyone in than deal with all the headlines of "match went down 5 minutes before game ended"
Well done Hotstar Team👏
Good info for Perf test - enthusiasts , well explained
Beautifully explanaed Gaurav, Thanks
Amazing Man. Hats off Gourav and team!
After watching this video, I am like
"Asli kaam to ye log kar rahe hain, hum to bus apni... "
True 😂
😂😂 shi m bhai.... 200 server ka scale up 🤥
Today, Pak vs India match in NewYork was Streaming on Tamasha App and there were 69+ millions viewers 😅
This is real Engineering.
Fantastic talk, would love to learn more
People directly close the tab most of the time in such cases.
Great job hotstar team . Kudos from a Cleartax Architect (60million scale).
Amazing Work
37:12 "bypass the payment"
I selected this video very eagerly on how to learn, after knowing it is the same game that gave me traumatic vibes
Awesome.. what is underlined load testing tool used to orchestrate?
Jio cinema handled 2.2 cr (22 million) concurrent traffic for a normal ipl 2023 league match of CSK vs RR
Wonder what will be the live numbers in IPL 2023 final
Jio cinema shows total users not live users
@@user-pm6qt6ee7p during that time they used to show concurrent live only
Too helpful🎉
Amazing explanation
Thank you for this video!!!!
5:40 gave a great perspective here
Great talk 🗣️
why hotstar didn't use private cloud in aws? why use public cloud?
Project HULK !!
Just some thoughts,
For a minimum base scale i guess its best to use bare metal server without any other things like virtualizattion, logging etc to get raw performance and to use private cloud instead of public to have higher bandwidth. Never worked on this much high scale but just my thoughts.
Also one thing i have noticed till date is hotstars marketing notifications are way delayed then the actual event. Not sure if this is decision making delay or tech delay.
Nevertheless its insane engineering ❤
actually, this is the perfect use case for public cloud. because, this kinda peak is rare and doesn't make sense to acquire CPUs just for this one occasion and let it be idle for almost 99% of the year and even then you never know what will be the actual peak. To give a comparison this year csk vs rcb had 500 million active users at one point and it was almost 10x than previous record viewers in IPL. If you go to public cloud, you can use multiple regions and multiple availability zones and multiple instance types and even multiple clouds too. this is why even Netflix still now uses AWS.
wonder if this team of hotstar has moved to jio cimena
At least their CTO has moved to jio cinema 😅
Excellent engineering❤🎉🎉
how is it a big deal to spin off more nodes on cloud?
Just suggestion stream your data 15s later so you can take advantage of auto scaling 😛😛 because you know now there will be spike
bro is challanging straight to AWS... i mean you need to redefine & rethink everything with such a huge scale.
Superb
I hope you meant terabits per second. Not terabytes. The difference is huge.
There is simple, add hpa, set threshold, spin more instances, spend thousands of dollars
What about 49.9 cr (499 million) during CSK vs RCB in 2024 IPL on Jio chinema
Views and concurrent viewers me difference hota ha. He is talking about handling 20+ mill users at a time.
@perfology how push notifications get to know about upcoming information?
Thanks for sharing
What is C59 class machine ?
did you find the answer?
C59x. It’s an instance type of EC2.
EC2 is amazon web service’s virtual machine offering.
Very useful ❤
I was always wonder, how apolo 11 launch was watched by 650 million people.
Dhoni for a reason ❤
Kudos
Awesome
i am a beginner, but i have a doubt, if we build any system in micro service arc, how we set a value to start a new service to avoid the booting time gap?
super anna
Insane work Hotstar!
In IPL 2024 Jio Cinema recorded a huge 590 million viewership on opening day which is just mind boggling when compared here with 25 million.
Views and live views, both are different
That is not concurrent users. Its just views (eg. page load), which is meaningless.
Jio hired most of their engineers from hotstar only
This comment shows me why most indian grads are unemployable
How you guys handle the fanout problem of push notification to 120M users?
In this main player are cloud and devops engineer
Watching again after the Ind vs Nz world cup-23
How to payment system in the login in background in the concreting details
This world cup india newzealand match 43 million 🔥🔥
53 million india newzealand semi final 🔥🔥
I expect 70-100 Million Simultaneous Viewers at Peak for India-Australia 2023 Cricket World Cup Final. All The Best To The Entire Disney Hotstar Team. ✌🏽😊
its 40 to 50 million i think
RPS vs TPS?
RPS - Requests per second
TPS - Transactions per second
we want to know about jio cinemo Application Scaled 2000 Million Concurrent Users | Performance Testing | Load Testing
thala for a reason
What this cabinets?
I have one doubt. In this video they mentioned that they are not using auto scaling of AWS. But they wrote their own autoscaling tool to achieve autoscaling. Am i correct? and if it is correct understanding means what is the difference between both?
For autoscaling you can use default autoscaling feature provided by AWS but as he mentioned, it is not reliable hence they had to come up with their own autoscaling, Hope that makes sense
I want ops hotstar suppliers.
What this can correct.
Woah, 216TB RAM
In short you did just one thing scaling. I'm not sure why every IPL, people come up stating why this problem is super complex when it's actually not. You just need to scale correctly, that's it. THAT'S LITERALLY IT.
You need to keep some servers ready to serve the traffic all the time. You need some more servers during peak time(toss, ending overs, beginning overs) and the rest auto scaling can take care very easily. That's it.
Do it can handle my website which has 100 visitors per month?
Only if they can anticipate the traffic and have ec2 instances warmed up😂
plot twist: Hotstar and IPL francise giving alot of money to Dhoni to not retire. 😆
Thanks to Harkirat found this video.
but now next ipl to jio
👍
Which match was this?
2019 sf
@@satyasaineelapala570 ah ok, got it.
4:14
Do hostingraja server can handle hotstar
Depends™️
Bhai ke pass macbook why techies prefer why not gaming laptop rbg se aag laga dunga
Just asking
we either go with linux distro pr mac beacuse of their offering and platform performance. Most of the programming and coding work dont require GPU unless you work on graphics intensive applications. Another is battery performance mac offers. there are windows workstations which has same characteristics like mac
C00L
1. How great we are
2. How great we became by ignoring issues
3. I don't care about people feedback
4. You guys just watch theory in PPT which you can read from any books
5. Read from books
6. Tum logo se nahi hoga, only we can do
7. Please clap 👏 and share this video for marketing purposes.
Real... He doesn't even go into the technical details
Nothing technical...
Edureka how much in the days is a continue my coaching gal I will friend for you I send in for your so money timing you have in receiving is a never to explaining in the total videos so simple tips for you cheap to take up for you my education my systems is my mind to attending after again reject I can do for you lot of tenses of only demo section them of election today more section 211 to change if are you demo section in the head my head is coming your head is pal so many times also here coming up
jio cinema handling load of 250+ million concurrent users and not showing off.
I see it as more educational than a brag. I wish more companies were transparent and share their learnings.
ipl never had 250M concurrent users. They are total views or something. No way ipl gets that kind of traffic. Only world cups or ind vs pak gets it
Wrong
have you served 1 people let alone 15 million people?
jio cinema is too slow
Buffer fail in the for buffer using for introduction essay in storage information in buffer file and concrete information in the handling this video?edu
Yes
what accent is this
Just having ipl and cricket rights was the reason for these metrics.
Whatever bullshit this guy said is all irrelevant.
At the end of the day if you have the rights for cricket in India all other things are stupidly irrelevant because ur ad revenue is constant for the entire 2 innings.
I don’t understand why hotstar invested and burnt money in all these bullshit metrics.
Superb