How do we scale web applications?
Вставка
- Опубліковано 8 лип 2024
- 🤖 SaaS I'm Building: www.icongeneratorai.com/
💬 Discord: / discord
🔔 Newsletter: newsletter.webdevcody.com/
📁 GitHub: github.com/webdevcody
📺 Twitch: / webdevcody
🤖 Website: webdevcody.com
🐦 Twitter: / webdevcody
I'd be delightful if you created a similar-format tutorial in which you cover more on the implementations side:
1) Api Gateways
2). VPCs
3) Public/Private Subnets
4) NATs
5) Load Balancers
6) Regions, AZs and clusters
7) etc...
This one was the most straightforward brids-eye-view tutorial on the topic I've ever seen!
You are a talented teacher. Keep it up!
That's a LOT of ground to cover! Pursuing a CompTIA Security+ and an intermediate AWS cert should cover your bases.
Happy that you reached the 100k mark. Well deserved. You produce quality content, and somehow 'down to earth'. It's cool to watch something actually useful, and not the classic 'todo app tutorials'. ✌
Thank you very much!
Would love to see a system design course. Because I like your teaching style, specially your derivation where you start from simple procedures as it was probably done in the past, you show the issues of the procedures with eventually the solution of the problem and more modern procedures.
THIS
From the ground up. You are surely a very good teacher.😊
Only just found your channel and loving the content, the speed, the rambles! really enjoyed this one, I've seen others explain this type of stuff in 5+ hours (which is not good)! Keep up the great work, it's so inspiring.
You choose each word you talk so wisely that your message is really straightforward. Congrats on your effort for put this knowledge online!
Congrats on the 100k well deserved mate - Keep them coming Cody! 🎉
Thanks! Will do!
Thanks Cody. Just subbed. I’ve been working web dev for over 10 years, but never done any DevOps or server managing, this was very helpful and informative
Thank you. That was informative. You explained bigger concepts than I have seen before, but you made it easy to understand.
Precisely the kind of overview I was trying to find. Perfect 👌
These videos are so helpful. They helped me have a look inside something that i thought was a complete black box for me. Thank you so much for putting this out here.
There's another layer that might live between the api and the database - Redis / Memcache. They can allow your apis/ other backend services to cache data they retrieved from the database to reduce db load and queries against them tend to be a lot faster than queries against a traditional SQL database. They're also a separate service, so they can be shared by all your serverless functions / edge functions / api servers
This was a great description! Thanks so much !
Your explanations are soo clear and straight to the point. I love it. and these are some rare content as well ❤
Such a good overview for scaling applications for anyone who does application development but does not understand what comes next when applications receive excessive network traffic. Thankyou for creating this video!
I'm a non technical founder and this is so easy to follow that I got excited while watching the video. Thanks for putting this video together
I loved the rambling part the most :). Good stuff. Keep it up.
I find this approach of visually building the concept super-helpful.
Beautiful work, brother!
I never miss any of your videos, great content keep it up 👏🎬
Excellent!! Thank you so much for sharing your insight
i want to thank you so much for this content. you do an amazing job of explaining and simplifying these concepts that aren't talked about enough. i would love to see more system design/the bigger picture of software engineering covered. thank you!🙏🙏
+1
Wow, I rarely find somebody who explains this clearly.
Love the visual explanation
Great video. a lot of value thanks!
Quality stuff! Cheers!
Man I benefit a lot from your video❤ Clarify the system arch in my mind. Also I love the drawing tool😮 Plz keep doing videos like this🎉
wow - thank you very much buddy :) Awesome 20 minutes
this was very insightful and interesting
I am a super noob in this field but u managed to give me bigger picture thank you. You have +1 sub
This is awesome 🙌
This video is extremely useful.
That's subscription worthy content right there boah. You've earned a sub.
Love your videos man
Best video yet
Congrats on 100K, Cody!
Thank you!
Can you make a tutorial of creating a video streaming app ? And thanks for sharing your knowledge with us .
Please do more overview stuff like this! Perfect for beginners like me
Gooood stuff !!!!! that was very informative !!!!!!!
Congratulation brother , 100k Happy for you .. soon gonna hit 1M
Thank you so much 😀
amazing content!
Congrats for 100k 🔥
Thanks 🔥
Great video
I'm interviewing with a company that needs to scale so this is great information
more stuff like this please!
thank you :)
This content is lit, sure you can make another python app video similar to a million videos but something like this is very usefull to a person who works in small companies or complete beginners where very few coding standards are followed when it comes to writing any application.
I'm a data engineer getting into web development. Often, integrating stuff like firebase/supabase etc has been difficult for me compared to setting up pgsq + express + frontend which seemed as the easiest most powerful solution. Since, you know, I'm used to setting up vms, connecting stuff together. Using db as API only and not being able to mess with it seemed un-intuitive.
Then I found about SSR/SSG, which blew my mind again haha.
This video is a GEM for me. I've been wondering what would be more performant - nosql with horizontal scaling vs psql with vertical scaling, SSG on serverless combined with SSR.. This really helped me get my thoughts together. Thanks mate!
What I'd say, though. When talking about DB performance, you have multiple factors which are very important - not only processing and memory + raw db performance (e.g. mongo vs pgsq). but also network outpu of given instance. without load balancer which allows you to increase your network throughput massively you're done. most VMs without autoscaling run on 1GB network. You're going to hit this super quickly, much quicker then TOUCHING, say, 4+ core VPS performance running either mongo/postgres. Could you maybe elaborate on the network side of thing? That could be really interesting video!
@jonasjonaitis8571 why do you feel the need to publicly demonstrate how much trouble you have understanding written text?
I wrote that I'm new to web development, trying to pick it up. obivously i'm going to find out about a lot of new technologies, no matter how old they are.
did this explanation help?
loved it.
Legend!
Great video! Could you take a deeper dive into lamda functions and edge networking?
great video
For any startup just starting. The the very few advices from a devops/systems engineer with 20 years in the bussines that will save you a lot of headaches and money in the future:
For your online service apps:
- Create 2 users for the DB connections, one readonly and another write only.
- In whatever language you use create a DB handler for reading and a DB handler for writing, each one using the appropiate usernames. Don't, allow the write handler to be used for reads, in any case.
- Avoid DB transactions. Any kind of external locking is prefered. (optimistic,etc...). Seriously, there are very few times you will really need them. (Counters, things like that).
- Do not use stored procedures. NEVER.
That's all. Thank me when you need to scale your app. Can't list all the benefits, but you'll see this decision will make everything a lot easier. From caching to locking and mostly MONEY.
Do more of these!
You are a G.O.A.T. tech youtuber
Nice talk!
great stuffs : D
Great video on the subject Thanks ! Would have been maybe easier to understand with arbitrary values to illustrate your examples. Was really clear nonetheless :)
What’s the best learning process for new concepts? Heavy practice for hours? Best comprehension techniques?
Depends on the concept. Sometimes I need to actually prototype something to understand it, other times I can just read about it an get a good enough picture of how it all works
Good job babe!!! FIRST!!!!! 👑
Great explanation, is this like an API Gateway? If not maybe can you make a video ❤
Thank you for the amazing explanation! Now I have a clear idea of where k8s & docker come into use and why do we need experts like DevOps engineers.
I do know some of the basic linux commands and how to deploy a web page or api inside a VM, but you are right, many developers may not know about this, and deploying an application can be another tedious task, as you might face another set of problems figuring out how to deploy it or accept traffic from the internet, etc.
I have a question, do people expect full stack web developers to know the basic dev operations like deploying and maintaining the app? (on production)
I would say to be a full stack, you need to know how to build, deploy, and maintain an application in production. So this means being able to build the front end, backend, integrate with a database, host a database, run migrations on your database, deploy your app, monitor your app, explore production logs, get alerts when stuff fails. A lot of services exist which make the deploying / monitoring part super easy, so at least being able to deploy something to a live environment these days is a must imo.
@@WebDevCody Got it! Thank you so much! ❤
This topic is actually good i use to watch a lot of codedam videos like this breaking down technologies and how the work
He did them on a board so this is really ok I appreciate the explanation
One question. As far as i know CDNs are implemented in the network by ISPs. How those CDNs that you mentioned are deployed? Are they as a service from a cloud provider or are deployed manually by e.g. redis?
I heard of a new tech recently called Vector databases, is it what might be the solution for edge databases?
Mate, this is very useful, thank you! As I'm watching this I am wondering how is this different for mobile applications ios and google play store. Basically your "UI" is stored on their servers and they take care of people downloading it and you just have to work work on scaling the backend api, right? I am talking about for example a React Native built mobile app. Let's say a giant social network app with millions of users. How would you built something like that?
Would a web hosting service like TMD take care of this if traffic volume increases fast? Is it the kind of things that would have to be done only if I would rent for example a dedicated server without services? Also what about the costs using say planetscale as db, Vercel for functions and another server for login/auth, wouldn't it be better to have a centralized hosting service with redundancy around the globe so DB, functions and everything is always close from one another? Sorry if it doesn't make sense, I'm a newbie, learning all of this :)
Idk much about tmd. It depends on the service you rent from them. If you just rent a vps, then no they won’t scale anything for you. Shared hosting would probably be worst since it’s a single server hosting everyone’s applications.
In regards to using third party database and auth services, usually you can pick regions for these services and have everything close enough together that the latency won’t make much of a difference. Yes, having everything in the same data center will obviously be the fastest approach, and if you are ok writing your own auth and securing your user info, that might save you money and be faster in regards to latency. If you rather just pay a company to manage your auth, that can help you move fast at the start of the project so you can focus on features. There is a lot that goes into good auth, password reset, forgot password, Mfa, properly storing passwords, setting up jwt and sessions, cookies, etc. if your team can do all that, more power to you.
Idk if anything one approach is better than any other approach, they all have trade offs
Isn't putting the db, functions and static files on the edge, basically putting a server there?
Yeah i mean a server has to host your stuff, edge servers are just located closer to your user “usually”
Can you please share end to end flow how to configure auto scaling my UI application running on. AWS
2:50 he drew an original PlayStation it looks like 😂
you are missing the db scaling, for example sharding or partition.
it would be a good idea if you can make a practical example of this
Explain more of what you mean?
@@WebDevCody you basically take a Todo app or what ever and you show how to scale it , I'm struggling with scaling a multi tenant app using mern stack of you could tell what are best practices of this kind of apps (you could combine multi tenant apps with scalability) thanks 🙏
"firestore from frontend/mobile app" should be considered as one of scaling application phase?
You should deploy everything yourself you'll be grateful for how much you can learn. Even if they're not perfect. Taking the habit of abstracting everything makes you a tool hostage
Can confirm, this is how the internet works at the current day. One thing that kinda irked me is your statement of "You don't really have to worry about this stuff, devops/systems/db engineers handle this"; If you throw your app over the wall with zero consideration about this stuff, it's gonna suck for every party involved (you, your devops guys, your customers and their users). I love that you shed light in to the cloud world as well.
I agree, everyone on the team needs to at least understand how all of this stuff works. I made that statement with the idea that most of my audience are beginners just trying to deploy their todo list next app, so I still don't think they need to worry about all of this... yet. They should just use a service like vercel, netlify, aws amplify to deploy their apps until they feel confident in diving into the deep end of the cloud world, because it can get pretty dang deep to be honest. What i've seen is certain individual enjoy doing this type of work, and others HATE it. It's also important to consider the size of the product and team in regards to when you need to start caring about this stuff more. On projects I work on, our contracts state we MUST use AWS to host everything, so my hands are tied to use AWS, but I sometimes would like to not waste a full week messing with terraform code.
Basic service would be a static web page running direct on server hardware, no javascript (the horror!) , no database. Just CSS. I know everyone is all about VM, and back end processing services, but not everyone needs that.
These are age old design concepts for very small starup running on monoliths… that also 10 years back…
these days microservices are industry standard… K8S is not at all meant for monolith horizontal scaling as u r mentioniing… use of message driven architecture is very easy thede days like kafka to put between backend and DB…
Your tutorials are good but please focus on industry wide used deisgns like microservices, event driven based etc. so that your audience will get actual enterprise level content…
Nothing to disrespect you… Just a suggestion… keep up the good work 👍🏻
great talk, well explained.
Great Video! Congrats on 100k subscribers :)
Thanks so much!
Congrats for 100k 🎉👏