How NETFLIX onboards new content: Video Processing at scale 🎥
Вставка
- Опубліковано 8 лип 2024
- Everyday, #Netflix handles billions of requests regarding movies, trailers and other video content. Delivering at such a large scale needs an #engineering marvel. This #video will talk about how Netflix is able to onboard new video content onto their platform. We go from video chunking to collating 4 second shots into scenes.
Amazon S3 is used to store the video chunks. Netflix also provides open connect servers to internet service providers, which acts like a cache of movies. Most requests to Netflix can be served by this cache, and the remaining are sent over the network. This reduces the bandwidth and time required for Netflix to operate at scale. Synergy at it's finest.
00:00 Problem Description
00:32 Video formats and resolutions
03:18 Chunk processing
05:52 Storage
06:19 OpenConnect for video caching
10:13 Summary
Looking to ace your next interview? Try this System Design video course! 🔥
interviewready.io
With video lectures, architecture diagrams, capacity planning, API contracts and evaluation tests. It's a complete package.
References:
/ high-quality-video-enc...
/ optimized-shot-based-e...
/ video
/ stream
/ keystone-real-time-str...
media.netflix.com/en/company-...
System Design Playlist: • System Design for Begi...
You can follow me on:
LinkedIn: / gaurav-sen-56b6a941
Instagram: / applepie404
Quora: www.quora.com/profile/Gaurav-...
Twitter: / gkcs_
External content used:
Fast and Furious 7 (Car chase scene)
Boone Loves Video: • 🌎Create 3D PATH ANIMAT...
Sacred Games (Available on Netflix)
Johnny English Reborn (What's it called?)
Netflix Logo
I believe there is a variation to Gaurav's understanding on open-connect caching mechanism. A new uploaded video DOES NOT goes directly to the caching server, instead they are resolved to a 'a small Netflix DB replica" in that region, which in-term populates the caching layer for new videos or most frequently accessed videos. The distinction is important because otherwise the design is assumed to directly populate OC CDNs, which is not the case. So for example, if a viewer in Australia wants to stream a non-cached US content:
Instead of directing all the Netflix traffic through under-sea cables, Netflix copies the video file (transcoded) from US-based storage to a storage location in Australia once during off-peak hours. Once the video has reached the continent, it’s copied to all the Open Connect servers present in the ISP networks.
This is not to point out a flaw in the video, just a clarification.
Thank you for this amazing clarification 😁
@@gkcs I am glad. Keep making fun content😄
Content apart , the new animations that you've added are really nice. Extra efforts are appreciated!
Keep em coming !
Man, you know how to teach with fun! I must say presentation and video editing is very well done in your videos.
holy shit! Just fall for these videos; Thank you from bottom of my heart! Please make more video on this topic!
Wow, excellent video. Love your unique style of breaking down the topics in chunks and explaining them neatly. Keep them coming.
Bro - Thanks for the great video! Learned a lot. Just subscribed. Keep the great content coming!
What a beast! Loved your reference to Amazon S3 out of nowhere! 😎✔️
It's amazing to see how Guarav improves the quality of his content, and I can tell you, guys, as a newbie tech UA-camr, it's a big deal. Keep it up, bro! ;-)
Thanks Victor!
The material you have in this playlist is gold! I did not expect to learn so much from it. Thanks a lot!
You're welcome!
Whoa!!! Thank you so much for making such unique and great content videos ;)
Showing Gupt in local popular movie would make Booby Deol really Proud. Savage AF Gaurav! :D
Love the energy. Your videos are great.
Very helpful and simple for anyone who is preparing for software design interview. Your way of explaining is interesting and funny 😊
Blown by the simplicity you explained such a complex topic!
Moral : Steal that Open Connect Box
You have the most unique content on UA-cam, man. No bs, pure knowledge. Keep going.
Thanks!
Awesome man! Simple and to the point. The best kind of online content.
Another informative video from you. Nice explanation. Thanks. Just wanted to add something on the caching part. Many websites take help of CDN providers like Akamai to do the caching on behalf of them. The CDN providers have the required infrastructure across the globe wherein they have placed their caching servers in most of the countries.
That's interesting, thanks!
To be honest learned something useful today, and willing to learn from you more like this kinda of topic!
1:23 AWESOME GAURAV, perfect example of what you are explaining, Huge respect bro
I was waiting for this video, awesome
Hi Gaurav, thanks for the video. Impeccable. Keep doing the great work!!
Don't be surprised if you hit 500K subscribers by the end of 2019. Your content quality is skyrocketing.
Thank you!
@@talesara74 It's system design and it's not for everyone to understand, what did you expect? People mainly use this channel for technical system design interviews.
You said that from your perspective. But the sad reality is People really don't like good contents😅😂. They can watch a 3hrs movie, but couldn't watch this type videos for 10mins😑
Gaurav I love your content and soon I guess will be joining your InterviewReady platform...Please offer me some special discount bhai 😍
It's almost 2023 and he still has 488k 🥲
I'm very impressed! Great explanation! I just subscribed! Keep going!
You’re awesome. Thanks 😁😍 this was so helpful
As i have some experience in this area, i would like to say that you delivered crisp content on the same. Good job!
Thank you!
Keywords = CDN , predictive content delivery , content pre-fetch and so on...
Yes Content Delivery Network plays a really important role if the geographic location of physical server is far and also I think what these OTTs would be doing is what we are getting in recommendation might be based on geo location rather than your watch history , this way if one guy from ur location has seen some movie it gets cached to a CDN and then when u request for the same content , since it comes in ur recommendation its extremely fast
Have been working in this arena for a while now, you got everything correct man, other stuff you post usually goes over my head because I haven't dabbled in a lot of those things but for once, felt nice to already know what you were gonna say. Haha.
Thanks! Your replies on the comments are excellent btw :D
Gaurav, I can't tell how interesting this video was ! Thanks a lot ! I totally enjoyed learning this ! Keep making more such videos ! :)
Thanks!
That bit at the beginning was hilarious 😂
Wow this is sweet! Super smart solution to the issues of video streaming, will be interesting to see where it is improved upon in the future too!
great efforts thanks a lot ! i remember my childhood days after seeing the GUPT poster! :-)
Thank You for sharing. Very nice details and very good description
Thanks for this wonderful insight on the engineering side of Netflix, looking forward to more system design videos.
Thanks!
Champion presentation. Just wanted to see a graphical representation of the adaptive bit rate ladder through Netflix's eyes.
Your videos are pure delight to watch !
Thank you!
Very detailed. Great work!
Awesome bro ..!! Simple and very informative.. loved it .. keep it up 👍
As always one another brilliant content
That was a nice one !
Thanks for the video.
I just started watching your videos. Your content is Amazing.
Learning a lot . Thank You gaurav for creating these masterpieces.
Thanks for your descriptions
Cheers!
Thank you for the explanation
that's pretty interesting concept... very well explained 👏👏.....
Paaji ek dum kamaal ka video hai ...even understoodable for non cs
Short, simple and full of interesting concepts. Thanks!
Thank you!
I just love your system design videos.
Thanks 😁
Great, Nicely explained !!!
Nice and depth knowledge bro!
Wonderfully explained. Everything is clear and simple. Love your content.
Awesome, thank you!
must say u made it really Interesting with editing :P and great content!
Thank you!
Very nice explained and made it interesting. Will be waiting for more good system design question.
That's just awesome
Keep it up
Brief and precise.. Nice.
This was a very informative video! Awesome job.
Excellent video, well explained!
this isp cache thing is just simply amazing,
Thank you Gaurav for the content . Hope to see many more useful content .
Thank you for your reading on this topic...this is enlightening (at least for me)
OMG!!!! the opening😂😂
Great content always!
Good job explaining!!
Please do some lectures on docker internals like Cgroups, namespaces, union capable file systems
Nice explaination, Thank you so much.
Outstanding 😨😱
real effort here, good job mate
:) Thanks this is amazing
Really helpful man!
Thanks Gaurav. I am not an engineer but always wanted to get some resource to learn the concepts. I would love to see more of your videos and on advance concepts.
Thanks Ankit!
You are God of System Design!!
ISP cache is really interesting, wondering how they have handled security on top of it across multiple geography??
Great video ! Thanks for sharing
So well explained 🤟
What about user authentication and authorisation when dealing with open connect??
Happy Teachers Day Sir! U taught me a lot! I really appreciate that!
Thank you!
Not gonna lie, you got me in first second, sorting in O(n) lol
Fantastic explanation
Awesome!!
Great breakdown!
Can you explain some of the advanced concepts of video streaming too? For eg., Bandersnatch? I wonder how Netflix is providing conditional streaming of video chunks based on user selection.
Thanks!!
Superb explanation bro
Hey Gaurav,
I was just wondering how the Open Connect or UA-cam Red boxes are any different from the long existing concept of CDN (Content Delivery Network). The only change is, being big companies they have the money and resources to create their own personal CDNs and brand them which is kind of building your own house instead of renting one. Is there something I am missing here ?
They are very similar, yes.
I was having the same question as how these Oppen connect boxes are different from CDNs.
Very informative!
The content of this video was very dense. The breather at 5:29 was very much wanted :-). You have very good audience engaging skills. Good Luck !!
I was the first one to watch this video after him!
The content had no views but a like!!
That's a improper system design for UA-cam, probably slow.
Thank you!
This is great work Gaurav. Thank you for this!
😁
Thats a really great explanation. Thanks
Thank you!
loved it
Quality content!
Great Sir,
Awesome dude..
Great presentation skills and content🔥🔥🔥
Nice work.
Great content.. keep up the good work..
Bro u are great am a 14+ software engineer really appreciate ur content
Video -> Fragmentation -> ABR -> DASH + HLS -> Packaging -> Cache On CDN -> Play
Too good... sir