How TikTok short content is delivered through HTTP - Devtooling TikTok
Вставка
- Опубліковано 15 чер 2024
- Welcome to another dev tools video, in this video I use dev tools to break down how Tiktok website works to pull short-form content. We will use therock as a subject. It is an interesting episode, enjoy
0:00 Intro
2:00 Overall tiktok connections
3:00 First Request
8:30 Static Content
9:19 First Video
13:00 Videos
16:50 Paging
DevTool them All Playlist
• Dev Tool them ALL!
Dont use offset • don’t use “offset” in ...
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
database.husseinnasser.com
Introduction to NGINX (link redirects to udemy with coupon)
nginx.husseinnasser.com
Python on the Backend (link redirects to udemy with coupon)
nginx.husseinnasser.com
🏭 Backend Engineering Videos in Order
backend.husseinnasser.com
💾 Database Engineering Videos
• Database Engineering
🎙️Listen to the Backend Engineering Podcast
husseinnasser.com/podcast
Gears and tools used on the Channel (affiliates)
🖼️ Slides and Thumbnail Design
Canva
partner.canva.com/c/2766475/6...
🎙️ Mic Gear
Shure SM7B Cardioid Dynamic Microphone
amzn.to/3o1NiBi
Cloudlifter
amzn.to/2RAeyLo
XLR cables
amzn.to/3tvMJRu
Focusrite Audio Interface
amzn.to/3f2vjGY
📷 Camera Gear
Canon M50 Mark II
amzn.to/3o2ed0c
Micro HDMI to HDMI
amzn.to/3uwCxK3
Video capture card
amzn.to/3f34pyD
AC Wall for constant power
amzn.to/3eueoxP
Stay Awesome,
Hussein - Наука та технологія
1:53 Right under selected 'Network' tab, right next to 'Disable cache' checkbox there is a 'No Throttling' drop down.
If you need to slow down your network you can try that next time.
I think the reason why they put authors in every single item is to re-use the design of items both in the home page and search page. Overall I really enjoyed this video, I have always been intrigued by Tiktok because how seamless their whole experience is.
Yeah, agree with you. It just a trade off for re-use
Also videos can we stitched. I wonder whether the author object differs in that case.
This made me think about using some sort of reverse proxy that changes the schema of APIs, in the sense of trimming down fields, and making API responses better suited to different clients that only need specific fields.
@@iagosoriano3734 that's graphql
@@Delphino__ lol that's true
Responses are compressed as GZIP so there's not much difference in the payload size. When fetching from DB, any sensible ORM will use the cached author record, so the only price to pay is object -> JSON serialization.
But it does make frontend developers' job a lot easier.
Anything you need to display a feed item is packed into a single object.
This deserves a heart
I agree, most cost would be at the serialisation level, but I feel reducing the compressed data too would make a difference on user experience.
And since browser anyway can cache the author details there’s not much point of fetching it with every request.
@@anjaliverma9304 what do yo Mean the browser can cache it? Save it in js? (Genuine question)
@@Potzakv20 local storage
I think the reason it fetched everything with the first request is because It uses server side rendering, running all the frontend javascript and logic in the backend and sending the final result as an html css
How did I miss this series while binging your channel is beyond me. Well I'm binging now..
each and every video is a gold mine !!!
2 reasons why you'll bring all the data for each video 1. the type of db used; if nosql then they don't want to waste resources on filters. 2. you'll need the info things like tracking and you don't want to go back to the db for it
Great video! Love seeing system design architectures of popular services like this (as much as I despise the idea and 'infectious' nature of TikTok, personally)
Yes, it's very helpful for novice to gain introspection into industry standards. It's also useful for interview prep, especially if you anticipate the interviews will have a system design component
When the frontend developers ask for more data each day, the devs tend to put the whole object so that they are not bothered afterward, maybe?
By this logic, I would include all relationships, tables and probably return back complete db so that I never have to face UI guys again 😂
I’m loving this series Hussein!
Thanks Hussein! Had to save for later since I made it halfway thru the video before realizing I wasn't caffeinated enough to follow what was going on 😂
Great analysis Hussein, very interesting to see how they use cursor placement to fetch new data etc.
It’s not cursor placement but cursor pagination. A whole different thing. Look it up
You literally hit The Rock Bottom there 🔥
awesome to review the details, thank you!
You made my morning!! Thank you
These videos are great!! Thanks a lot
First comment on this.
This is great backend-knowledge channel
Thanks Hussein, perfectly explained.
I was devtooling prime video the other day and they also stream the videos through https, but obviously the video sent via the request is encrypted/encoded(it contains some random colors in the video and the actual video is not visible).
So do they decrypt/decode the content of the video dynamically on the go, if so, shouldn’t this be an expensive operation?
Also just asking, why streaming a video through web sockets is not a good idea ?
Not too sure but if I had to guess.. probably bandwidth and performance. Keep in mind that those vids arent been fetched from storage direclt, they are probably in edge cache locations which are MUCH more easy to scale and they dont take a *shit* on resources because you dont need 2 way communication...
Side note: Currently working with IOT devices through webSockets and they are a *fucking pain* to scale... i would exhaust all htto related options before even comming close to websockets 😉
I assume that is to prevent people from capturing the original video, copyright stuff...
@@jaydeep-p thats what I was thinking. I know some streaming sites embed metadata in the actual video stream so when the video gets leaked to different sites they can track when and where it got leaked from.
Prime Video likely uses Encrypted Media Extensions, and it definitely uses Widevine in some capacity. So it's a DRM thing.
Nice video! Thank you Hussein!
Great video!
Loved it!
I love this series
This is great thank you!!
ooo i relay like this video, please make more of these 👍🔥
Best of UA-cam!
Have you done a video on paging. Cursor got me curious on how it works on the backend. Many databases provide a token on per query basis but in general how is that implemented at database level? It cannot be a row number that can be stored with each row in database table since pagination happens on a specific query which could have filters.
They are just using a timestamp. Like "Get videos before this date"
@@daviidon so every row probably has a timestamp? or its something databases store internally?
@@ameyjain3462 yes...you usually want to store the date when a post was published. Some db (eg: Mongodb) automatically stores the timestamp
yes the cursor is the unix time. mostly it's created_at field in the database. so query might be like first 9 videos (where video.created_at < cursor)
@Charan so in that case the results will always be somehow sorted or datbase is pulling the lowest timestamp first. For next page client will need the max timestamp seen in the result so that next query can do " select * from T where created_at > prev_max_time." Without sorting the timestamp its not possible so won't it create performance problem?
Can you explain / elaborate on the OPTIONs requests more, why is the OPTION method being used? you briefly mentioned some stuff, but didn't fully follow your train of thought
Oh nice ASMR video! 👋👋
Should they bind the webdomain to the API? To solve that options-issue
Hey! Really cool and informative video. May i ask, which vpn do you use?
Express
Video Request: How Cloudflare CDN works? (Specifically their BoT management system)
Awesome!! Please keep making videos i learn a lot from you.
i love your voice so much
thx for the video
I like the way Hussein talks. 😂
I couldn't find the 'msToken' request btw. What could go wrong! 😥
Do you think that first page is rendered server-side?
Wow thanks, always learning from your videos.
About not using Offset, I see how it can be avoided when paginating a whole table, but how to avoid it when there is filtering and sorting on different columns according to your experience ?
As long as you have a column to order on and filter on the last value you read you should be able to
I gave examples here ua-cam.com/video/WDJRRNCGIRs/v-deo.html
@@hnasr I meant that I may have multiple columns to filter by and multiple ordering scenarios (columns)
my god you know a lot about the web.
can we fool the client tracking and get all data of profile at once?
So why is the options request bad? If it would be the other way around the mobile page would need to query it with options. Having 2 API endpoints for the same thing is even worse.
compression made the repetition content in json less problematic.
I'm guessing they are using HLS or DASH encoded video. That's pretty typical for video streaming websites.
yesss! dev tooling video 🔥❤️
bro, do it for instagram web version please
Can you please do one of video for instagram?
me enjoying the tutorial even if i did not understand 90% of what he said
Do Google with JS disabled
Hey Hussein! Thanks for all these great videos! Can you please make a video about some good beginner backend projects... A lot of companies in india don't really lookout for dedicated backend Developers in the early stage of their career or in college for an internship.... But I'm highly interested in backend development
Not related but Nice ASMR rendition.
what about FACEBOOK?
you have really great content and I love them, but they are too long and half of it is just chit-chat. Sometimes I want to watch something quickly but nah, always putting them in `watch later`.
Tried to watch with 1.5x speed but then I missing some interesting stuff.
Next we want to see is whatsapp
: )
i know that tiktok uses next js
Dev tool whatsapp web!?
You need to speed up your commentary my dude.
RE including all author info on each video item, I'd guess they did the math and realised that the number of request for these author-specific pages is minuscule compared to requests to their general feed page, so it wasn't worth optimizing for - i.e. in 99% of cases, you WANT all that author data in there, because you don't yet know the author.
There's value to simplicity.