Your videos literally amazing. I always wanted a techie guy like you to hear about new technology related updates and things. keep making videos like this. 👍♥😊
I think running out of spot nodes is a risk. I guess they might have a contingency plan for using on demand machines in case no spot is available. The actual risk is however, why will AWS itself not do this from their end, thus killing this company's entire purpose. Cloud providers providing spot nodes, are not doing it for better marketing or anything, they are just trying to make use of their idle resources as much as possible, and if the only way to do that is, making it low cost, so be it.
I mean, if you use LXD/Incus, migrating a whole VM *does* become very do-able at super short notice with enough bandwidth - as long as your application isn't 50Gb+ I guess!
The more interesting part would be, if there are plenty of spots instances then they might not take back, but rather it depends on the AWS business requirements how they want to handle this, Do they want to keep it as spots instance or will they intentionally take back and you have to now migrate to a primary instance.
If that is the case, then the cloud companies will soon find a way out of it. They cannot afford loosing out on customers because of some loose strings.
📌 system design for that comes in my mind as a junior dev: - Put a route/request handler instance in front of a large EC2 spot instance - ec2's state always sync with a reliable instance - user request -> route handler -> ec2 spot -> goes back to route handler -> resolved to user. - if route handler lost connection with ec2, then it's initiate another ec2 spot & send the unresolved requests again.
no, there is no point in keeping a reliable instance per spot, at that point just use the reliable instance as primary. But you make a good point. For the network stack woes one could route traffic through a reliable instance (like a proxy), so when the spot needs to be shifted to another, you allocate the prev inet address to this new copy and then copy the whole VM somehow. This also explains the need for nested Virtualization ig since they need the VM copy and VM to have exact same "hardware" config for the Memory and processor states to work AS IS on the copy. But still, no idea how it's "Zero" Downtime
@theSoberSobber no, the reliable instance is there just to keep the state in sync, nothing else. So, think about a $8/hour spot instance, does it require the same size instance to keep the state in sync? Of course not. We just need a very minimum reliable instance e.g. $30/month.
@theSoberSobber zero downtime means you're not gonna throw 503 status at any point. Imagine, a user browsing a page that made 5 api call. If in a situation 3 api call succeed but 2 failed then the user have to reload the page. Which is very painful. & this is not a zero downtime system. This is what typical spot instance will get you into a similar scenario. Even with my proposed system, zero downtime is possible (for a very simple app, i assuming). Think that scenario again. 3/5 requests served successfully. Route handler didn't received the rest 2 api response. Route handler is waiting for another instance. Another instance becomes ready with that same previous state data. Now, route handler gonna served the rest 2 api requests to the user. Notice that the system is not shown down to the user. But it takes extra time. e.g. if a typical api call got served in 100ms. Then in the bad scenario, it'll take 100ms+ waiting period for another instance ~500ms + 100ms. It's taking a long time, but the system is still reliable.
@@howhello354 you do need a instance of the same size otherwise you can't keep the state in sync. Defining state here is the key problem. If you beleive it is possible to reduce all params of the VM to a set of params that matter then sure it might be possible to keep one reliable instance for X amount of VMs given the set is small enough. But the main point is the reduction problem and what is important to actually "Copy" a VM.
Really interesting idea but yeah same question as you how to move everything. They should create some resources for this. But nevertheless I am gonna pitch them in my next team tech talk.
Get over blockchain and rather look at how p2p works Check out iroh or things like that I suppose I think blockchain is built on top of p2p principles which you can actually learn a lot from making p2p things instead of shitcoins
Please make a video on unikernel (like mirage os) and how this can be used with docker I guess (or maybe even spot instances) there is also criu which can allow for live migration , but I am not sure if it can migrate network packets / recontinue them , I am not sure
that doesn't make much sense considering they have created a talk (I am not sure how you can fake a talk) Also , it doesn't matter if they are giving their own compute instance , cheap is cheap. they would actually BURN a LOT of MONEY for these spot instance
@@codedamn It is 2024. The content needs to be engaging. Rather than scrolling through various windows and a long monologue in background, it should have been broken into parts with relevant graphics/ images in an engaging manner. It seems like a web version of PPT presentation. It must have a central topic, sections, chapters, conclusions , summary etc., to keep the viewers engaged. The logical progression of the topic is lost in the process. People with good attention capacity won’t find any problem with current setup, but that is only 1% or maybe even less of all the viewers. As a long time viewer of your content, I think that it should have been presented in a manner that this high quality content deserves. Many people won’t like what I said, but social media engagement algorithm will tell otherwise.
@@codedamn I watch Mehul videos only due to Topics he pick for videos and clear explanation of it Recently I watched video on DRM Which was mind blowing for me Please keep uploading amazing content
Such a quality contents these days!!!
Keep delivering them sir :)
MAD RESPECT
Your videos literally amazing. I always wanted a techie guy like you to hear about new technology related updates and things. keep making videos like this. 👍♥😊
This company cannot scale, if everyone starts using spot instances as their primary machine then who is going to provide the spot instances.
i think you missed the part where mehul mentioned about the downside of these spot instances
😂😂 right. But not everyone gonna use that for sure.
this tbh, if there is "Zero" Downtime, why even get primary instances lmao
I think running out of spot nodes is a risk. I guess they might have a contingency plan for using on demand machines in case no spot is available.
The actual risk is however, why will AWS itself not do this from their end, thus killing this company's entire purpose. Cloud providers providing spot nodes, are not doing it for better marketing or anything, they are just trying to make use of their idle resources as much as possible, and if the only way to do that is, making it low cost, so be it.
Top notch quality content Mehul. Please keep them coming.
Once it became popular and mainstream, Will the cloud providers patch this & make it harder to use spot instances?
very likely
or they can just change TOA & you are no longer compliant
I mean, if you use LXD/Incus, migrating a whole VM *does* become very do-able at super short notice with enough bandwidth - as long as your application isn't 50Gb+ I guess!
Great Video!
It would be better if the description links to the website or his twitter account.
The more interesting part would be, if there are plenty of spots instances then they might not take back, but rather it depends on the AWS business requirements how they want to handle this, Do they want to keep it as spots instance or will they intentionally take back and you have to now migrate to a primary instance.
Sounds intresting 😮🤔
Exactly this is the content I am paying internet bill for.
If that is the case, then the cloud companies will soon find a way out of it. They cannot afford loosing out on customers because of some loose strings.
sir, i just love your channel
need some info regarding cloudfare workers and efficiency
Superb Explanation Sir
📌 system design for that comes in my mind as a junior dev:
- Put a route/request handler instance in front of a large EC2 spot instance
- ec2's state always sync with a reliable instance
- user request -> route handler -> ec2 spot -> goes back to route handler -> resolved to user.
- if route handler lost connection with ec2, then it's initiate another ec2 spot & send the unresolved requests again.
no, there is no point in keeping a reliable instance per spot, at that point just use the reliable instance as primary.
But you make a good point. For the network stack woes one could route traffic through a reliable instance (like a proxy), so when the spot needs to be shifted to another, you allocate the prev inet address to this new copy and then copy the whole VM somehow.
This also explains the need for nested Virtualization ig since they need the VM copy and VM to have exact same "hardware" config for the Memory and processor states to work AS IS on the copy.
But still, no idea how it's "Zero" Downtime
@theSoberSobber no, the reliable instance is there just to keep the state in sync, nothing else.
So, think about a $8/hour spot instance, does it require the same size instance to keep the state in sync? Of course not. We just need a very minimum reliable instance e.g. $30/month.
@theSoberSobber zero downtime means you're not gonna throw 503 status at any point.
Imagine, a user browsing a page that made 5 api call. If in a situation 3 api call succeed but 2 failed then the user have to reload the page. Which is very painful. & this is not a zero downtime system. This is what typical spot instance will get you into a similar scenario.
Even with my proposed system, zero downtime is possible (for a very simple app, i assuming).
Think that scenario again.
3/5 requests served successfully.
Route handler didn't received the rest 2 api response. Route handler is waiting for another instance.
Another instance becomes ready with that same previous state data.
Now, route handler gonna served the rest 2 api requests to the user.
Notice that the system is not shown down to the user. But it takes extra time.
e.g. if a typical api call got served in 100ms.
Then in the bad scenario, it'll take 100ms+ waiting period for another instance ~500ms + 100ms.
It's taking a long time, but the system is still reliable.
@@howhello354 you do need a instance of the same size otherwise you can't keep the state in sync. Defining state here is the key problem. If you beleive it is possible to reduce all params of the VM to a set of params that matter then sure it might be possible to keep one reliable instance for X amount of VMs given the set is small enough. But the main point is the reduction problem and what is important to actually "Copy" a VM.
@@howhello354 that makes sense, the route handler is the same thing as the proxy server I was yapping about, so I agree.
Really interesting idea but yeah same question as you how to move everything. They should create some resources for this. But nevertheless I am gonna pitch them in my next team tech talk.
Interesting, lets see when you can use it for codedamn and how effective it becomes
Looks intresting thanks for the info
Refrance video links?
Is this actually cheaper than Hetzner? I don't think so.
Dang yo, thats wild
Bhai Make a Video on Blockchain
Get over blockchain and rather look at how p2p works
Check out iroh or things like that I suppose
I think blockchain is built on top of p2p principles which you can actually learn a lot from making p2p things instead of shitcoins
Which software do you use to draw ?
Tldraw
Please make a video on unikernel (like mirage os) and how this can be used with docker I guess (or maybe even spot instances)
there is also criu which can allow for live migration , but I am not sure if it can migrate network packets / recontinue them , I am not sure
Ye canada me baith ke pagal bna ra. Mein bta ra kya kar re ye, under the hood ye khud ke hi compute instance denge spot ke price pe.
that doesn't make much sense considering they have created a talk (I am not sure how you can fake a talk)
Also , it doesn't matter if they are giving their own compute instance , cheap is cheap.
they would actually BURN a LOT of MONEY for these spot instance
Yes you are right. how can they afford this? if they can why big giants like AWS cannot. @Serizon_
doesn't make sense, how can they manage to sell compute cheaper than hetzner/AWS at scale?
This video is probably 2 months old
how so?
Sponsored video
i would have mentioned it if it was sponsored
Content is good. Video editing isn’t. Need to hire a good editor. I don’t know why people with good content don’t care about good presentation.
what can be improved? i'll pass on the feedback
@@codedamn It is 2024. The content needs to be engaging. Rather than scrolling through various windows and a long monologue in background, it should have been broken into parts with relevant graphics/ images in an engaging manner. It seems like a web version of PPT presentation. It must have a central topic, sections, chapters, conclusions , summary etc., to keep the viewers engaged. The logical progression of the topic is lost in the process.
People with good attention capacity won’t find any problem with current setup, but that is only 1% or maybe even less of all the viewers. As a long time viewer of your content, I think that it should have been presented in a manner that this high quality content deserves.
Many people won’t like what I said, but social media engagement algorithm will tell otherwise.
There's no need to overdo anything, I think content/presentation is good and to the point.
Bro has short attention span, ignore him. I was engaged throughout the video.
@@codedamn
I watch Mehul videos only due to
Topics he pick for videos and clear explanation of it
Recently I watched video on DRM
Which was mind blowing for me
Please keep uploading amazing content
I don't see why I will base my business over spot instances :)
Maybe some toy project or a frugal one can be hosted on it.
the point they are making as I understand is that Spot == Allocated when using their service.
@@theSoberSobber I got that, I think this is useful mainly for not-so-critical part of business such as CI-CD, adhoc jobs etc...
you can also build things like your backend API server on spot instance fleet if something like this works.