Im the devops manager at a ~5B company. I have a huge 'on prem' colo deployment of systems. I can confirm that AWS or other hyperscalar clouds run me about 10-12x the cost. Now you have to deal with having data center ops and paying those salaries etc. But even with all that its way more expensive.
I've worked for one of the largest Telcos in Australia. It is cheaper to run internally, but the problem is always the internal politics of who controls what. It would literally take months to get certain changes done that would be a sinch with AWS. This is one of the reasons why departments go to the cloud because they are sick of the politics.
Develop single code, but ready for both cloud and your DC. Deploy in parallel both in the cloud and on your hardware. When your hardware cannot handle it(or DC burns to the ground), redirect traffic to cloud and it will autoscale. If big traffic becomes a new standard then increase hardware in your DC.
One of the things not mentioned here is that there are other resources on AWS that cost money as well. EC2 and EBS is not the only thing. You probably will have some kind of firewall, virtual networks, backups, log ingestion, etc.
back when netflix got popular it forced a lot of ISPs around the world to upgrade their stuff as big chunk of their customers started maxing out their series of pipes at the same time. never been their customer but I appreciate the positive growth pressure
I don’t know how they did it but in my country where monthly usage is capped at a few hundred gigabytes (quota), only Netflix does not count into your monthly usage (offered by select ISPs) which basically makes it better than any other service
What I love about cloud is that it lets me, an individual, make stuff and deploy it for cheap-to-free without having to worry about maintaining additional hardware. It's kinda the same deal when it comes to installing internal applications at smaller companies (or smaller teams/orgs within companies). When scale comes into play, the waters get muddied. There's a ton of tradeoffs to consider, and how much money you pour into either on-prem or cloud infrastructure is going to determine how big those tradeoffs are and in what direction. Not every company will be better off one way or the other, it's about analyzing your individual situation and picking the option that's best for you.
Here's a profitable company, managing their own servers for several years now, telling you that they would pay 11x for worse service in AWS, and would thus no longer be profitable. Chat: But what if a hard drive fails? And what about labor costs?
@@raptyaxa5771 this is why you use a 5 year ammortization schedule and you sign contracts with Dell and a Storage vendor (NetApp, Pure, etc.). I would never have hardware without service agreements, what happens if you 1) dont have the expertise available or 2) you have a hardware failure. Service contracts are a must if you're running an enterprise size business. Before movign to the cloud we were paying about 300k to account for growth / obsoletion
@@Darth_Bateman didnt watch the video, but wait, is having your own server a thing again? Not the worse idea, but if your shit goes viral it'll go offline since it doesn't have a cluster to scale to..
If you take a 3 year reserved instance plan payed upfront it reduces the cost by 62%. Still massively cheaper with your own hardware, and once your paying for cloud upfront that obviates the entire advantage of cloud which is that you have month to month flexibility.
So what you didn't calculate in is: * Pay for system admin team to manage all of this for 30 months * Pay for developer time for all of the tools and things you will need to develop now that you don't have them ready * No multi-az since you are only renting one space * Equipment breakdowns due to fauilt / overuse etc. * Backup strategies you will need to develop. You ARE planning on backups right? * Since you are renting, I am guessing you are renting the space and the connection so who is doing the infrastructure costs of the building like data center security, building upkeep, upgrades, etc? * If you ever need to go global what will do to keep the response times low? I am not saying AWS is cheap. What I am saying is that you have a LOT more to consider than just rent and electricity.
(disclaimer, I'm a dev at ahrefs) 1. the devops team is more or less the same size when using aws 2. This one is partially problematic sometimes, but for our problems it's more the other way around in general, AWS is lacking some tools. Nevertheless it's indeed can be a real limitation. And here usually the cost isn't so much the salary of the devs but the time required to build a solution. 3. yep that's correct, if you need multi AZ the computation would be different, but the computation for AWS would also be different, as you would need to duplicate the storage to different zones. So the AWS cost would also drastically increase. 4. equipment breakdown is part of the price of the hardware 5. backups are mentioned in the article 6. The people the rent is going to (this question is exactly the same for AWS, who is paying to keep the AWS building in working order?) 7. we are global, but this isn't really a relevant question here, see point 3 So overall there's not much additional costs so long as we don't depend on a product that is only available on AWS.
Also If you have any compliance reqs cloud is much easier. And all of the bespoke shit you write is more technical debt. Then five years go by and you have to re train ops people on your bespoke shit. Not fun.
@@louisroche9574 "1. the devops team is more or less the same size when using aws" So the devops team size didn't change taking on more critical responsibilities , but did everyones pay go up to match. Sounds like a big mgmt win.
Cloud makes sense for businesses that require elastic availability, 100% on prem might not make sense then. Say ticketmaster has a consistent load of 50000 users per day, but now Taylor Swift is selling a new tour, it would make sense for them to scale into the cloud for that period to offset the abnormal load. Cloud by default will always be more expensive, but amazing to scale only when needed.
only VPS in a cloud environment by default will always be more expensive, the fact I see only people that can't build a solution without a service VPS like, is the main problem...website of Taylor Swift to sell tickets can run and scale on AWS for few bucks per month, VPS not needed at all
I think one thing we have to keep in mind when looking at these example companies who're doing these migrations is, that they're pretty special cases - be it massive scale web scraping (ahrefs) or video streaming (amazon prime video). __This is not what most of us are doing/serving__. Most devs here, me included, probably do standard web development - maybe with some SSR. But this will not lead to comparable load profiles in respect to user count. Most of Ahrefs capacities most likely go to 24/7 max load scraping, not to serving customer requests.
With a beginning of russian war against Ukraine, out major banks moved to cloud in EU from on-premise in UA. One of them (monobank) moved to AWS and they also got significant increase of bill for infrastructure. This was mentioned by their CEO Oleg Gorokhovsky. He also was questioning himself why orgs tend to use clouds.
I'm not a cloud expert, but I am a little surprised that competition between cloud providers hasn't seemed to reduce costs much over the last five years or more. Probably one aspect of it is that the differences in usage specifics between cloud providers means that once a typical company has started investing in one cloud provider, there's an aspect of lock-in at play that prevents easy transitioning from one to the next.
This is due to venture capital. You're not dealing with profitable companies competing with each other. You're dealing with massive, MASSIVE amounts of venture debt from companies that have no road to profitability in other areas of their businesses. And since it's rather expensive to get started, it's very difficult to bootstrap a cloud company. If you don't have the venture capital, you can't afford the servers. If you get the venture capital, you'll never be profitable enough to lower your prices.
(I'm working on a cloud platform with some buddies and it will always be both competitive and profitable as long as we never pay ourselves salaries - it would require several *thousand* servers before the cumulative profit-per-server could pay us competitive wages)
I'm a cloud expert instead: VPS are expensive, the key to lower the costs in a cloud environment like AWS is to use their native cloud services, avoiding any service VPS like: EC2, ECS, EKS, etc. this guy was unable to do, posting that just showed his lack of knowledge, TCO can be easily lower than on prem, but you can't improvise, you have to be prepared
For anything I've ever stood up, it's always been substantially cheaper to do it on our own hardware in a co-lo than putting it in any cloud, especially if we want anywhere near the performance we have today on our own hardware. None of this is a surprise to me.
For those wondering about Reserved Instances: they used 3-year Reserved Instances to calculate the price. Bottom-line, and how I personally look at this: have an off-cloud strategy. The cloud is great at a very early and/or explosive period (either customer growth, or new infrastructure), but you should plan to staff up and have a cloud exit strategy while the cloud is still advantageous.
@@neociber24 Exactly! I actually pushed to move our workloads from on-prem to cloud last year. We have all our workloads on the cloud, and the reason was two-fold: speed up moving to a new paradigm of development and deployment more quickly (aka implement DevOps, as a strategy), and for geographical access (we have customers in 3 different continents). But we're already working on our new on-premises infrastructure, first to move our development and testing workloads, but ultimately, to become another DC in our global infrastructure. End game is having our pipelines generate cost analysis so we can choose where to deploy different workloads, aware of the benefits (and costs) of each option.
I disagree. You're planning to hire an entire staff to deal with ordering hardware, configuring VM management, mega-licensing costs from someone like VMWare... All of these things that have *nothing* to do with your business and you have to hire a whole team to do it. Even if you end up saving some money (factoring in staff costs makes me question that you actually would), the management overhead there is huge, NO THANKS! I'll gladly outsource it to cloud services
@@themartdog You're assuming that's not a consideration. You're also assuming we don't have the trained staff to do so. Both those considerations are not true, but YMMV. I was just sharing my experience, not inferring knowledge of everyone's infrastructure decisions! ;) PS: you're also assuming that every company's needs fit into something that is "workable" in a cloud setting. For instance, egress and overall bandwidth costs are not a consideration for most people. In our case, egress alone was 30% of our bill, and we already found better solutions, both more cost-effective as well as more manageable for our teams. o/
The prime values that a cloud provider delivers are :- 1. avoid upfront cost of infra and talent ownership. 2. provide flexibility to scale up / down as necessary. 3. allow distributing workloads across multiple regions / zones. 4. reduces complexity by providing an API over infra and services to allow composing your implementations rather than deal with the underlying implementations. In case of on-prem deployments, considering that they're multi-regions, it's usually still a single zone per region. You could have hardware level redundancies but how do you auto-swap? How do you deal with ISP failures, weather, sabotage, etc. The problem is most people don't know what worms does the on-prem can has. If the on-prem infra is the physical filesystem, a cloud provider is the logical filesystem over top of it. Techniques like IaC (Infrastructure as Code) basically allow you to programmatically / declaratively not only spawn infra but also allow you to observe it, scale it up / down and even protect you from major security threats. Does all of this come at a premium? Obviously, DUH! It's not a best effort basis deployment which most on-prem deployments are. There's usually an SLA and a very high one at that. Also, almost no one discusses the talent cost, local regulatory costs, technical debt that accrues and slows everything down gradually? And lastly, what do you do when there's a sudden spike, let's say 300% on Black Friday and then you don't receive such traffic again for the whole year. What do you do? Over provision 5x? What about the unutilised capacity available for the rest of the year? If a company doesn't need most of the features mentioned above and has extremely predictable and mostly static loads, sure, the premium for the cloud provider might not make sense. The most probable reason why Netflix sticks to AWS is for the ability to seriously scale up / down in seconds and for AWS's expertise is maintaining a behemoth such as itself. Finally, scraping the internet is going to be expensive. Should've thought about that before building a business around it. LOL!
@@thewhitefalcon8539 who said it needs to be EC2. Though, I'm sure if a company that serves 10 - 15% of the internet, Netflix, is fine with using EC2s then there's some value proposition that really works well.
SLA for EC2 is only 99.9 (3 nines) and 95.0 (1.5 nines) for 10 and 25% discounts respectively (most of AWS is similar, you don't get 5 nines from Amazon) If you can't accomplish 95.0 (18.25 days of downtime per year, more than 24 hours per month), you're doing something truly wacky In regards to staffing, someone from ahrefs stopped by the comments (and other people backed them up here), the staff cost for devops is basically the same either way I don't find the dismissive comment at the end helpful at all. They aren't complaining, they're sharing their information. It seems strange and derisive to punctuate with 'By the way, what did you think was gonna happen, boneheads? LOL' if the information is meant to be conveyed compellingly or convincingly
Exactly, their use case is completely different than EC2’s value proposition. I’m not saying that doing yourself isn’t possible, I’m saying if your business isn’t around managing infra, you shouldn’t be mostly doing it yourself unless it makes business sense to do so. Ahrefs business revolves around scraping the internet and greedily too. I’m not privy about their approach to scraping but from what I understand they’re simply putting in compute to get as much content as they can. Considering the rate at which data is being generated and made available online, this is only going to become more expensive or time consuming. This is a scale and optimisation problem and throwing compute at it is not going to make it easier.
I've been conditioned to think that buying vs renting/leasing is more cost effective over the long term. Think real estate and autos. I remember when AWS cloud was taking off and having this twinge of misgivings about the costs. Plus the fact that you are beholden to the owners of the cloud. I agree with everyone who says there needs to be a balance. The ability to be able to spin up hundred/thousands of servers at a moment's notice is a great convenience but once your infrastructure is more mature, the cost savings are too great for standalone.
That's always was the supposed argument of all arguments for cloud - "you only pay per hour for what you use, so you can't multiply that cost by whole month". It always seemed silly to me because, yeah, sure, if your server ran for just a couple of hours every day it might make sense. But in reality, most of the time that's not what happens. You need to have some instances that just run all the time, even simply because the demand won't drop below some level. And it's not like you can just nuke all your databases or delete all your data from the cloud every day because it's "not in use". Ahref's case is especially striking, because the cloud cost overhead is so high that even if 90% of their servers were never needed, they still are better off not paying for AWS.
I wonder why in the recent years the tech industry is making dumb choice after dumb choice, it's like engineering has gone completely out of the window.
I think the key advantage of IaaS is not flexibility in capabilities you invest in, but simply the much, much shorter timeframe of investment. You can double your infrastructure today, and cut it down to a quater tomorrow. If you did that with hardware, the decision you made today would mean paying extra 200% capabilities for the next 60 months, on top of the fact you’d be stuck with the original excessive 100% you would have even without todays fuck-up. The fact you couldn’t even do such a choice because the timeframe from decision to actual capability is so much longer. If you buy “off the shelf”, you might have a bare supervisor running in less than a month - provided you only bought few machines, your network configuration didn’t need reworking, your UPS had a vacant spot, the server had memory pre-installed and the storage was delivered before the server or in the same shipment. it’s like buying a car versus renting it. You can think about renting a car for ten years, and conclude that it’s ridiculously expensive. But at the flip side, you can’t buy a car for a weekend - that’s not how buying a car works to begin with.
Good luck trying to get over ~10,000 instances in a single region spun up within a small time frame. Even on a corporate account, you need to plead a case for a need for that many instances which goes through a ticketing process that can take days to weeks which is still dependent on if the resources are even available in the region at the time, which in the case they're not your request is simply denied. Then take Netflix who was able to host the entirety of the US eastern client base off a single rack and is able to spin up hundreds of thousands of instances whenever they want at a fixed rate.
@@callowaysutton About 10k instances, it's actually pretty edge case, probably that number even more than any cloud provider have in a day worldwide. And it's actually hard to find a task for this 10k instances, so yeah they want to know will they be paid for this, and for which task it needed, because it can create a shortage for other clients. About netflix served from single rack, depends on which timeframe you refer, because earlier they used Akamai, and now they use their OpenConnect rack for ISPs. So it's almost never was a single rack. Per each 1gbit they can serve approximately 400 active streams(average 1080p stream ~512KB/s with some fluctuations), if we assume that some pause video + they ok with some small almost unnoticable delays, then it can be approx. 1000 users of decent content provisioning. From requirements of OpenConnect they can connect it max to single 100gbit, So 1 rack can serve max 100k users with ideal conditions of course, and it's only content serve without site logic/analytics/etc.. UPD: even less than 100k, I forgot to include IN traffic, because it's one same NIC for IN/OUT.
@@Sergey92zp Here's the thing though, I could spin up 10,000 instances very easily on any medium sized $100-500k privately hosted cluster and not even be close to reaching the total capacity, so it being an edge case isn't and shouldn't be used as an excuse for an IaaS type of commodity. For reference, spinning up 10,000 of AWS' smallest EC2 instance in US East costs a whopping 100k+/year, and for that you get 10k shared 2 core, 512MB of RAM instances... or about 20,000 vCPUs and 512GB of RAM. Let's break this down further; it's a common industry practice to split each physical CPU thread 32 ways to each vCPU so that gives us 625 actual threads. The newest Epyc servers come in configurations of 512 threads per server (you could even have a 4 way multinode in 2U for a total of 2,048 threads in 2-4U worth of space) and 512GB of RAM is frankly nothing considering these types of servers go into the terabytes of RAM. If we discuss space and networking, a single 42-47U rack with good 100Gbps can be had for ~6000/m now, or ~72k/y, and considering AWS doesn't ever let a single customer go above 100Gbps per zone it's a good frame of reference. If we were to amortize over a 3 year period that gives us a 150k budget to compete with AWS and with that kind of money, you could easily afford multiple tens of terabytes of RAM, thousands of cores and much faster disks. For what you're saying about Netflix, you're just completely off. I'd recommend looking at their many presentations about their infrastructure and how they've saved money by using their own hardware/network.
To be fair, if your idea of using AWS is just paying for dedicated EC2 instances, you’re doing it wrong. You need to architect your applications intelligently to keep your costs down.
@@lhxperimental Not necessarily. There are many AWS and Serverless providers. There’s tons of ways to roll databases. There’s many ways to do compute and single sign on and such. Not everything has to be in AWS and really you can architect your applications to appropriately fit cloud services in some places and monoliths on dedicated hardware for others.
@@lhxperimentalthats why you have a exit startegy from the cloud, or use terraform if you're sticking to cloud stuff to move as easily as possible. You actually choose whether you get locked in AWS. AWS is locked in WITH ME >:)
@@lhxperimentaldon’t use your phone send us letters with a parrot so google doesn’t lock you in with UA-cam. You obviously didn’t touch a server in the past 10 years if you think a company that large can rely on ec2 instances
As someone that is supposedly a software architect, but always ends up having to do Ops on the side, this ignores the biggest cost point in all this: IT and developer time. We're not just talking about making sure the power stays on. We're talking about automatic backups, configured with a few clicks, autoscaling easily, and starting up and tearing down developer instances in seconds. Suddenly, your IT, or worse, your developers have to do all that. If you're already big enough to afford 40 million in servers, chances are you have a big enough IT department already. My personal experience though is that the IT department is where every single CFO tries to save money, and it fucks EVERYONE over.
With things like OpenStack, OpenShift, OpenNebula, VMWare, Canonical MAAS, heck even Proxmox that is quite literally not an issue. Almost all platforms have a way of autoscaling, backups, high availability, hardware issue alerts and more features that used to need a lot of specialized knowledge. Now, a single person, or a small three to four person team, could potentially manage the actual hardware of a rack or whole DC location and it would still not even make a dent in the difference between collocated and cloud costs assuming salaries in the low 6 figures. In Proxmox for example, you could take a physical server down for maintenance and have it live migrate to another node with no noticeable affect to the client. All platforms have at least this feature and more; you could even migrate across two different DCs/locations if you really wanted to, which you can't do in AWS, GCP or Azure yet. Canonical MAAS literally manages the hardware for you and with its hooks you could potentially even have it automatically auto order parts that failed tests, safely turn off the server, turn on the iBMC LED (to show remote hands which server is having issues), submit a ticket to the DC with the shipping number and have maintenance effectively automated. The IT landscape today is not the same as 2008 and the large cloud providers are definitely starting to show their age big time
@@Victoria-ij3cb I definitely think, if the company is big enough, on-prem is can be nice, but only if your feature set (and therefore requirements) are basically static. Otherwise, what would a single click in AWS to try out (e.g.) K8S turns into MONTHS of waiting, and then getting a faulty, buggy on-prem K8S. Another BIG disadvantage I've observed in pretty much every bigger company is that IT is understaffed, underfunded, overworked and overregulated. Things that would be a single click on the cloud can take months.
@@invinciblemode Not even remotely true, at least in my experience. For example, my personal, very painful experience with on-prem K8S: We didn't have an expert, so someone just had to google shit and hope something useful happened. Obviously, it took forever. Then at least 5 different features that are a click away in the cloud, even for the dumbest developer, were missing. Things like a basic load balancer, an API gateway, a replicated database, or a decent storage solution. And again, you'd need experts for all those things. With a cloud K8S solution, all that just magically happens, and is maintained, automatically. Even if you only use the cloud for as dumb VMs, you're saving a lot on people that need to setup, install, and maintain your racks, and the software (hypervisor, etc.) running on it. I guess I just have PTSD, because EVERY SINGLE IT department I've ever come into contact with has been underfunded and overworked, making them decidedly far worse than the cloud, at least from a developer perspective. Maybe in heaven they have a wonderfully running on-prem cloud.
I never noticed the little light up border on the like button that happened when you had your early video breakdown... It only does it during that outburst too
The reason I use AWS as a startup that has a huge storage commitment and moderate EC2 Usage is that I can scale out and in at will, so for example I'm on a some downtime right now and I scale down the system dramatically lowering my monthly commitment while keeping the Infrastructure. I spent a few years in a COLO and while it definitely was cheaper, when I needed to scale up I had to put out a ton of cash and worry what happens when the scale goes down. I guess the most important thing for any company moving to the "cloud" is to spend some quality time with Excel and figure out your options.
If you want a reliable, scaleable service with backups, I think this is the sort of thing you think about when your AWS costs go above the cost of, say, 4 skilled full-time engineers. If you're spending more than a million, you can bother thinking about it.
You don’t need 4 full-time engineers to keep up few servers. I’d argue cloud is more labour intensive, because poor resource usage costs so much. On the other hand if you have ridiculously oversized on-prem setup, it can survive multiple disk-failures, hw-failure and still work. Yes you want to fix it ASAP, but the staff you need to do so is not notably expensive compared to managing cloud. You will lose few nines of reliability compared to amazon. But just because amazon has five nines doesn’t mean your product on AWS has. In either case the likeliest failure point is the same: your own stupid ass or asses
Even if you use AWS; you still have to have to spend time and energy handling backups. I'm not sure why people use "backups" and "reliable" as key benefits for Cloud. Cloud isn't inherently more reliable than computers; since that's what it's based on and it's not abstracted *that* much.
Idk, the difference between the two scenarios is so insane that it could mean that allocating your own hardware will always be cheaper than any approach on AWS
The upfront costs are very high though so like stated it's not feasible until you have enough cashflow that it doesn't hurt the company to make those upfront investments. Pay as you go is pretty much always more expensive in the end but you don't have to have the money now. Like buying a house for example.
So that is even with reserved instances. 1 TB of RAM sounds ridiculous to me anyway. And I am an actual Java developer, so quite used to stuff eating up ram like no other.
And I've done self hosting in colo in the early 2000s. For some reason management never seemed account for replacing cost every 3/4/5 years and trying to wring lots of 'free' time.
A lot of ML stuff can use a lot of ram. BTW a lot of RAM is also very useful for storage caching. Idk how necessary it is when you can have a 16x NVMe RAID 10 tho.
The amount of vector logic in AI/ML nowadays will make you crazy. Literally Python code jerking off of each other and produce pretty image or cosplaying as a human will cost you more than just 1TB of RAM.
it blows my mind that they just brushed over capacity planning and autoscaling as if they need to pay for Ec2 instances that handle their full capacity 100% of the time
Fair point but when you're dealing with very large instances like they would be you can't rely on AWS having enough capacity when and where you want them.
Regarding Netflix traffic: I select how much bandwidth to hire for my home by the number of TVs I'm planning to have on simultaneously. The rest of the traffic is mostly irrelevant.
My company is going through the same type of thing except they've decided to go for cloud. However, there are more costs involved that they didn't take into account. You have to host those server and then maintain those servers with staff etc. You also need to provide redundancy backups etc etc where as the cloud provider provides those things for you. There are also some things to consider since we are a multinational company as we need to maintain different clouds for different regulations etc. We had multiple individual datacenters all over and had to support them with hosting, network interconnectivity, etc etc. So for our solution, a cloud provider (ironically multiple) is what was chosen. However, I can also see that being temporary until we get the final costs and then someone gets the idea that we'd save more money by setting up our own datacenters again. It's the typical IT cycle.
Always good to have mix and build system they don't rely on a specific infra so you can always move it depending on cost, only concerning thing I have seen is a lot of people only having public could skills and it usually just a particular one, You can take a infra person and they can move/build platforms whereever, but these people with those single public cloud skills sets you kind of don't have they underlining knowledge of how it all works.
Key Takeaways: The video discusses the cost of running infrastructure on the cloud versus using on-premise hardware. The company, AREFS, compared the costs of their own co-located data center with a similar installation on Amazon Web Services (AWS). The analysis showed that running the infrastructure on AWS would cost AREFS significantly more money compared to their own data center. Summary: The video explores the cost implications of running infrastructure on the cloud versus using on-premise hardware. AREFS, a company, compared the costs of their co-located data center with a similar setup on AWS. The analysis revealed that running infrastructure on AWS would cost AREFS significantly more money, even taking into account their revenue. The video highlights the importance of periodically re-evaluating cloud benefits versus actual cost and suggests considering a reverse migration from the cloud for more mature companies. The analysis also takes into account factors such as people skills, financial controls, and capacity planning, among others. Overall, AREFS found that managing their own infrastructure was more cost-effective and allowed them to invest in product improvements and development.
Statement that at certain scale having your own infrastructure ia chepaer is not a magic or new one. It is known for decades. The problem is that your own DC will never produce a fraction of easily consumable cloud services than AWS, Azure or Google.
@@oscarljimenez5717 Yes and no. You have to keep that in mind during development or at least during planning. For example set up your clusters to rely as little as possible on proprietary technology and use your own instead. For example configure your own load balancers inside your k8s clusters so you can just move the cluster to another location easily instead of relying on proprietary stuff (like AWS's and Azure's load balancers).
@ThePrimeTime I am surprise that they didn't take into account that when you plan to have a server on AWS for 1 or 3 years there is a HUGE discount for reserved instances which lowers the price -70% down... even more if you use a mix of reserved instances and spot instances (which are -90% discounted). They can save money going to the cloud and serverless (dynamodb) + lambda + s3 can also be considered for super fast retrieval and serve of data instead of using EBS volumes. Maybe they had some issues with AWS and pepper sprayed this article to them 🤣😂
What is the successful cloud start in my opinion? You need to be a "nomad digital angel"... What do I mean? You need to be able to jump from clouds to clouds easily like an angel. So you need to architect your infrastructure in a way that allows seamless cloud switching without any significant troubles. Use only the services that are based on popular open-source projects. Even if they are a bit harder to use. Every proprietary service will be yet another heavy anchor tied to your leg. So when you start your project cloud helps you to reduce the initial costs. When you are a bit bigger you can switch to the cheaper cloud and hire some infrastructure specialists. When you are big you can build your own cloud and move there. Success...
Sounded like Ahrefs was doing a lift and shift comparison with just infrastructure. It would be interesting to have digged into their architecture to see if a cloud native architecture could have fitted better.
@@meletisflevarakis40 I would not read too much in to the Prime Video malarkey, it did make good headlines 😉 For me, main takeaway was it reminded us there are good and bad architectures, and just because it's serverless does not mean it's good. I.e, if you are hammering Event Bridge (as was the case with Prime Video), then you will have a bad (expensive) architecture.
@@aaronhamburg4428 IT's a "future aaron" problem. You deal with it as you approach the issue. Until then, don't waste your time - focus on product and business development.
@@aaronhamburg4428it’s like people just lose their memories every five years and all knowledge of previous bad behavior from vendors who abuse lock-in tactics is wiped from the face of the earth
If cloud didn't lock you in so much, a hybrid approach would be much more effective. You could rely on "the cloud" for usage spikes, or for scalability. Say you suddenly needed 100 more servers, you could order them, and then use 100 cloud servers until your servers arrive and get set up
AWS recently has been pushing for more hybrid technologies. Services they have been promoting lately help facilitate that. Better to capture part of a business than none at all
The real issue with clouds and why most choose cloud is because eventually you will have to go through legal stuff/regulations etc which will force you to move to the cloud anyways. Like take SOC 2 Type 1 certification. That BS costs anywhere from 75,000$ to 150,000$ per year to get done. And if you are on cloud you can literally cut that cost in half (or even more) of the audit. If you don't want enterprise customers you can stay away from the cloud. The moment you want enterprise customers, you will be forced to move in to the cloud. On premise is another beast.
16x15 gigs = 250 gb = 1/4 petabyte per server. RAID mirroring gives you half of that, so 120GB usable space. Amazon doesn’t get free computers, only cheap ones. Then you have utilities + profit margin + Bezos yacht fee = the same or higher cost as buying a better server yourself.
I think there's a lot of things overlooked there, but at the end of the day the premise that the cloud is more expensive isn't even unclear... it's a well-known fact. It comes down to busines priorities. Woudl you rather employ people to work on improving your products, or just running them? Companies that lean heavily into cloud, or are even 100% cloud, are prioritizing working on the products rather than infrastructure. They outsource the infrastructure.
Bung nose? It’s ‘Monsanto Season’, the special time of year when North America douses itself in expensive mystery compounds using expensive diesel machines so bread and meat can be cheap, in Morocco.
I think they made some mistakes in their calculations. Normally when you host such a system on AWS you don't compare your Servers directly because your instances run in Auto-Scaling Groups and your system scales up and down. This in the calculation means that all the server are 100% loaded at 24 hours of the day. But normally when you have times of a day where no one is on your website the system will scale down and delete your instances. So i'm not sure if that calculation is really fair. And there are some other opinions like Spot Instances. But yes AWS is expensive. And what another user mentioned is that you need Engineers and Backup strategies and so on. But i think when they scale very intelligent on the load i think AWS would be much cheaper then the calculation mentioned.
they specifically choose to reserve instance, means they literally rent the logical space for the server in the DC. And also, considering the HW specs they buy for the on-prem servers, I don't think spot instances (or non-HA instances) would be adequate for their use case.
7:00 Clustered file systems yield at best 50% of the storage space because you have 2-3 copies of the data scattered across many disks servers. So if you buy 3x10 TB, you can utilize at most 15TB. In reality you'd only get about 10TB with 3 disks, but as you approach 20 disks you get closer to the optimal 50% capacity.
As a SysOps guy watching here to better communicate with my dev colleagues, I can say the point where migrating away from cloud makes sense is much lower than you think. Granted I'm in Germany so we might have other expenditures for people and other stuff. But I would argue, that one of the first additional talent a start-up, with say 2 devs that founded the company should look for, is a SysOps guy, even if you stay on cloud for a while. Sure I'm biased because I am a SysOps dude, but cloud doesn't make infrastructure trivial tho. Or rather on-prem /colo isn't that complicated for someone who knows their way around servers. Also what you need to consider ist where all you data is stored (especially when you are a Europe based company), are you thinking about backups, where are those backups stored, and so on. With all that cloud easily gets more expensive than on-prem or colo. With cloud it is the same thing as with everything, it has it's use cases but it's not the be all an end all. The tech sector tends to jump on a hype and must use it for anything without considering it's pros and cons
they didn't include the salary cost of keeping infrastructure engineers on to frig around swapping out hard drives... setting up server patching.. making them actually work. Alternatively you can buy service contracts to get the skills. the 10X would quickly scrub that out though. We have done a similar cost analysis and reached similar conclusions. cloud is orders of magnitude more expensive.
Big part of the cloud is using it effectively. You have the power of the largest and most feature rich cloud provider of the world and you are using the most simple service they have for almost everything. This is an easy comparison, but it is not the right or fair one.
I think this explains why Amazon is profitable and so many other companies, who use AWS aren't. Years and years ago, I remember being told to invest money with Amazon and Microsoft because their Web Services divisions were making bank. Well, I know I know why? (Assuming the article is to be believed)
Not an apples to apples comparison by any means. AWS is about services and resilience not servers and power costs. If you don't need the services or the resilience AWS provides (think auto-scaling multi-region services in minutes) then there's no point even thinking going with Cloud.
My company is all about the cloud and migrating everything on the cloud for certification sake. The problem is that they don't think about the cost and performance. I've noticed the things we are running are more expensive and slower than before... Hey, if they are fine with this...
They should calculate how many engineers they have now vs how many they would need for server less . Probably they would still save money but maintenance of so many servers is also expensive
Its not as much works as its made out to be. Most systems you rack up, and leave alone for 3-5 years. Maybe 10 hours hands on for the whole life of a server.
experience cloud engineers are costy (at least looking by offered salaries for cloud ops) and there are less of them, many are people who only did some course sysadmins eg linux are easier to find, the more open tech the more people that can work with it also adopting projects to their techs eg servicebus, leaving it then - is lot to rewrite, not to say initially coding for azure/aws specifically may be extra job for extra person who knows its apis setting a server is really not that much work except initial bringup, since you virtualize everything and can provision VMs on demand, the cost of running 20 vs 40 VMs will be similar because you already have that hardware, while every VM on azure has own cost
Add 30 engineers to just work out terraform quirks an constantly broken compatibility/APIs. Just working out cloud quirks and limitations is equal if not higher time-cost than self-hosted.
Why scale down the workforce and deal with payouts when you can just massively increase the workload of the existing team piling on the critical responsibilities without increasing anyones pay except for mgmts. bonuses.
There must be some crossover point between AWS and on-prem in terms of cost vs performance. Clearly these guys need performance beyond that threshold. I'm certain there are plenty of people using AWS that fall below that threshold of performance requirement. I guess that's why there are people that get paid a bunch to work out the cost effective strategy for business infrastructure like this.
16x15 is super easy math for 8 bit programmers from 80s.. because 16x16 is 256 of course, so 16x15 is 256 - 16 = 240. It's apparent Prime is lot younger and haven't been doing 8bit assembly... ts ts... :) ;)
love these article reading videos! please continue with them! also, how do you find out about such articles? would love to get exposed to such content by the daily
Cloud: its cheap to get into when you're small, but once you've ditched your technical expertise, can you rebuild a DC from scratch? Do you need "cloud scale" Do you even need "cloud scaling" or will HAProxy be fine? Get your IT provisioning done by cookie cutter.
"imagine if all your money went to aws" I can think of at least a few companies which this caused their downfall, although not always aws, but using other cloud providers too.
Let's say you decide to leave the cloud. Do you need to replicate the "services" that you had before. Is there an opensource equivalent of google cloud or aws that you can host yourself? I'm far from being a cloud engineer but I was trying firebase but decided to go with supabase. Is there something similar with aws?
AWS does give you the ability to run some of their services outside AWS, yes. They also have AWS Outposts so you can set up your own AWS "partition" in your own datacenter that can have EC2, S3, etc. all through the normal API.
@@lhxperimental That is exactly what I'm asking. How to develop without tying the code to an specific platform. I've heard of openstack and terraform but I know little of them, are they even related to this?
I mean in your example supabase is a "Foss" (I don't think their dashboard is FOSS) "alternative" (doesn't really have the same set of functionalities ) to firebase but their hosting platform isn't open source at all, that's what you actually pay for, the managed hosting they provide. Or maybe you mean "is it possible to self host dynamo db (or whatever)?" ?
While growing, AWS is probably the best thing, but there's a point you get to where you need to move to yours, same with serverless, at a point you reach, you will have to move to using regular instances
It doesn't even make sense when growing, unless your growth is absolutely insane. You could easily overprovision everything by a factor of 5, and still spend less with on-prem. Can you quintuple your users or order some more servers faster? My guess is it's the latter. Cloud only makes sense if you're small and the whole cloud bill is less than the cost of a few skilled sysadmins, or if you're worried about some kind of 10x weekend taking down your infrastructure and that the costs of that are worth paying 5-10x as much for hardware all the time.
@@OMGclueless that's just objectively wrong. Growing = not a lot of users. Renting resources for a reasonable price will always cost less than hiring even one engineer if your workload and users aren't a lot. A cloud service is probably the best way to go
@@festusyuma1901 "growing" doesn't say anything about big or small. TikTok is (or was) growing, are they small? If you're small and a sysadmin costs more than your entire IT bill then cloud makes sense.
What the f**k. At the begging of the video Prime is saying hello to youtube and at the same time encouraging to press the like button. My like button for this video glows UP to press it. Not gonna lie, I got a bit spooked
I'm 41, call me old school but I'm at an expert at supporting on prem customers. I work for 15 large jails across the country and these are County entities who never have any money to do anything so there's no way they would go to the cloud.
At a certain point it's cheaper to hire your own talent to manage your own servers. AWS fills the talent gap a lot of companies have. I've been running my own servers since I got broadband in the early 2000's. Most developers wouldn't know where to begin. So for companies trying to run lean, they would have to hire a dedicated DevOps person or use AWS and pay a contractor a few bucks to set it up or let someone who has a lot of other responsibilities do it since it's such a part time concern. I have a $900 desktop running my stuff on my 1GB fiber connection out of my house that would cost $900 per month on AWS. Cloud services are basically insurance. You're paying a massive premium to shove liability onto another company.
Hey Prime, have you realized that the like ratio is usually 10%? it's pretty steady across all channels content, a few go above 15% but I haven´t seen a higher ratio, there should be some anomalies anyone could share examples???
You yelling press the F***** like button made me press the like button. Who would have guessed what it took was a moustached man in a hoodie yelling at me?
Netflix transfers more data than porn: The reason is that Netflix serves generally higher bandwidth streams compared to porn sites that usually are of lower bitrates. Some porn sites even default at 240p or don't go higher than 480p.
Honestly me neither, Primeagen is just fun to watch. Then again you probably won't have any real knowledge about the cloud unless you work at a company that uses them or are just into knowing about cloud infrastructure
What about all the support costs for all your hardware and how many people it takes to support that hardware??? The people costs would be enormous when you factor in benefits. I don't believe it would be that big of a Delta.
@@fritzstauffacher6931 but you have to pay people to run them. those are _not_ cheap. And their processes aren't as mature. Your SRE team isn't born with runbooks. I can stand up an arbitrarily large database cluster with multiregion HA and point in time recovery in like 3 mins clicking around. And it actually works. The hardware isn't the point at all.
The problem for the banking industry, is nobody will do this kind of investigation as the "safest" approach is to do what other enterprises do, which is getting in bed with a consultancy firms that make money off of convincing execs to go invest in IaaS, ay-ey-ey-esss
it's obvious taht its way more expensive than doing it on your own, but thats the point. almost all companies who use AWS are not capable of doing this stuff on their own, so they pay the premium anyway. and those companies who could maybe do it on their own, they sometimes will still willingly pay the money just to scale faster. for example if youre a young startup that just wants to grow quickly and has no aim to be profitable within the next 3-5 years, then they'll just spend this money and worry about doing it on their own, once theyve grown to a state where they can manage building an infrastructure on their own. This is exactly what is happening at the company that I am working for right now. We spent all our man power on growing teams and getting everyone on the same page and stuff like infrastructure we just spent the premium to get it going. If we built this stuff on our own (don't forget this isn't the only thing you need to build when starting a company) from the start, then we'd never have grown so quickly and probably got stuck after a few months
It's called Snowmobile (yes, it's a true name). Btw. there's also Snowcone and Snowball xD I just love how they name their services. As for financial aspect - no idea, but it's most likely costly af.
a company like ahrefs should have vertically integrated infrastructure anyway... unless im missing something, totally not the use case for cloud services
I mean, for the numbers of Ahref, the amount of people managing the system is pretty irrelevant. Even if you paid 2 million each to 5 system engineers, meaning 20 million in two years, that is still 380 millions cheaper for them than AWS.
If Cloud then EC2. Not Serverless. The Hyper Scaler makes more profit if their connected services are slow in responding. Simply as you pay for waiting in a serverless function.
@@louisroche9574 You would have an ops team which does Monitoring, Software patching, periodic vulnerability scanning and much more. I would also argue that you have an lot more cost, for example you would probably also have to ISO certify. I still think, that you are cheaper on prem, than in the cloud. I would also argue that having Infrastructure know -how and having fewer dependecies on third Parties is a bonus.
@@JustVincentD devops team would probably be about the same size. Administrating services on AWS isn’t an easy job. We have very few people actually going to the DC physically. And the legal requirements are relatively light. If we were manipulating a lot of sensitive data (like health related) that would probably be a very different story indeed.
@@louisroche9574 yeah ur right about the Ops team at aws, but patching switches over the Air or Firewalls can sometimes be tricky/risky. We are having those conversations like every week. I think it depends on multiple factors and it is a case by case decisison. Some companies should go into the cloud some shouldnt, but I still feel that manageing your infra is cheaper than cloud.
Im the devops manager at a ~5B company. I have a huge 'on prem' colo deployment of systems. I can confirm that AWS or other hyperscalar clouds run me about 10-12x the cost. Now you have to deal with having data center ops and paying those salaries etc. But even with all that its way more expensive.
All calculations I have done on cloud costs for customers is around the same. It's 10x ish.
I've worked for one of the largest Telcos in Australia. It is cheaper to run internally, but the problem is always the internal politics of who controls what. It would literally take months to get certain changes done that would be a sinch with AWS. This is one of the reasons why departments go to the cloud because they are sick of the politics.
@@dexterplameras3249 This is actually super true.
Inflation will not help at all in these calculations, we all know that by now.
You are doing something very wrong.
Develop single code, but ready for both cloud and your DC. Deploy in parallel both in the cloud and on your hardware. When your hardware cannot handle it(or DC burns to the ground), redirect traffic to cloud and it will autoscale. If big traffic becomes a new standard then increase hardware in your DC.
One of the things not mentioned here is that there are other resources on AWS that cost money as well. EC2 and EBS is not the only thing. You probably will have some kind of firewall, virtual networks, backups, log ingestion, etc.
gateways prices are huge too
@@OtakuArtful Yup, those WAFs are a major cost point.
Nat, vpc, cloudwatch, waf it's not even close. Between public cloud and self host, always self host if you have the ability.
Wet-ass firewalls
back when netflix got popular it forced a lot of ISPs around the world to upgrade their stuff as big chunk of their customers started maxing out their series of pipes at the same time. never been their customer but I appreciate the positive growth pressure
And now those same ISPs are pushing for charging companies like Netflix for "traffic", nevermind who is actually requesting (customers).
I remember my ISP is blocking netflix for this reason lol
I don’t know how they did it but in my country where monthly usage is capped at a few hundred gigabytes (quota), only Netflix does not count into your monthly usage (offered by select ISPs) which basically makes it better than any other service
Just so you know he's a Netfix Engineer
series of TUBES
What I love about cloud is that it lets me, an individual, make stuff and deploy it for cheap-to-free without having to worry about maintaining additional hardware. It's kinda the same deal when it comes to installing internal applications at smaller companies (or smaller teams/orgs within companies). When scale comes into play, the waters get muddied. There's a ton of tradeoffs to consider, and how much money you pour into either on-prem or cloud infrastructure is going to determine how big those tradeoffs are and in what direction. Not every company will be better off one way or the other, it's about analyzing your individual situation and picking the option that's best for you.
Here's a profitable company, managing their own servers for several years now, telling you that they would pay 11x for worse service in AWS, and would thus no longer be profitable.
Chat: But what if a hard drive fails? And what about labor costs?
trust the chat to actually know stuff
you can outsource just the storage to the cloud (aurora, bigquery etc).
@@raptyaxa5771 this is why you use a 5 year ammortization schedule and you sign contracts with Dell and a Storage vendor (NetApp, Pure, etc.). I would never have hardware without service agreements, what happens if you 1) dont have the expertise available or 2) you have a hardware failure. Service contracts are a must if you're running an enterprise size business. Before movign to the cloud we were paying about 300k to account for growth / obsoletion
Not to mention being a hostage and not being able to get your data back ever again.
@@Darth_Bateman didnt watch the video, but wait, is having your own server a thing again? Not the worse idea, but if your shit goes viral it'll go offline since it doesn't have a cluster to scale to..
If you take a 3 year reserved instance plan payed upfront it reduces the cost by 62%. Still massively cheaper with your own hardware, and once your paying for cloud upfront that obviates the entire advantage of cloud which is that you have month to month flexibility.
So what you didn't calculate in is:
* Pay for system admin team to manage all of this for 30 months
* Pay for developer time for all of the tools and things you will need to develop now that you don't have them ready
* No multi-az since you are only renting one space
* Equipment breakdowns due to fauilt / overuse etc.
* Backup strategies you will need to develop. You ARE planning on backups right?
* Since you are renting, I am guessing you are renting the space and the connection so who is doing the infrastructure costs of the building like data center security, building upkeep, upgrades, etc?
* If you ever need to go global what will do to keep the response times low?
I am not saying AWS is cheap. What I am saying is that you have a LOT more to consider than just rent and electricity.
(disclaimer, I'm a dev at ahrefs)
1. the devops team is more or less the same size when using aws
2. This one is partially problematic sometimes, but for our problems it's more the other way around in general, AWS is lacking some tools. Nevertheless it's indeed can be a real limitation. And here usually the cost isn't so much the salary of the devs but the time required to build a solution.
3. yep that's correct, if you need multi AZ the computation would be different, but the computation for AWS would also be different, as you would need to duplicate the storage to different zones. So the AWS cost would also drastically increase.
4. equipment breakdown is part of the price of the hardware
5. backups are mentioned in the article
6. The people the rent is going to (this question is exactly the same for AWS, who is paying to keep the AWS building in working order?)
7. we are global, but this isn't really a relevant question here, see point 3
So overall there's not much additional costs so long as we don't depend on a product that is only available on AWS.
There is no big difference when comparing team sizes. You still need specialised people managing AWS.
Still cheaper
Also If you have any compliance reqs cloud is much easier. And all of the bespoke shit you write is more technical debt. Then five years go by and you have to re train ops people on your bespoke shit. Not fun.
@@louisroche9574 "1. the devops team is more or less the same size when using aws"
So the devops team size didn't change taking on more critical responsibilities , but did everyones pay go up to match.
Sounds like a big mgmt win.
Cloud makes sense for businesses that require elastic availability, 100% on prem might not make sense then. Say ticketmaster has a consistent load of 50000 users per day, but now Taylor Swift is selling a new tour, it would make sense for them to scale into the cloud for that period to offset the abnormal load.
Cloud by default will always be more expensive, but amazing to scale only when needed.
only VPS in a cloud environment by default will always be more expensive, the fact I see only people that can't build a solution without a service VPS like, is the main problem...website of Taylor Swift to sell tickets can run and scale on AWS for few bucks per month, VPS not needed at all
I think one thing we have to keep in mind when looking at these example companies who're doing these migrations is, that they're pretty special cases - be it massive scale web scraping (ahrefs) or video streaming (amazon prime video). __This is not what most of us are doing/serving__. Most devs here, me included, probably do standard web development - maybe with some SSR. But this will not lead to comparable load profiles in respect to user count. Most of Ahrefs capacities most likely go to 24/7 max load scraping, not to serving customer requests.
When you have too much money, you build your own cloud :)
just build your own cloud 5head
Build your own space company, change from cloud to space 5Head
I mean you could also build your own cloud with your computer and cloudflare tunnels.
With a beginning of russian war against Ukraine, out major banks moved to cloud in EU from on-premise in UA. One of them (monobank) moved to AWS and they also got significant increase of bill for infrastructure. This was mentioned by their CEO Oleg Gorokhovsky. He also was questioning himself why orgs tend to use clouds.
They should try Hetzner
I'm not a cloud expert, but I am a little surprised that competition between cloud providers hasn't seemed to reduce costs much over the last five years or more. Probably one aspect of it is that the differences in usage specifics between cloud providers means that once a typical company has started investing in one cloud provider, there's an aspect of lock-in at play that prevents easy transitioning from one to the next.
This is due to venture capital. You're not dealing with profitable companies competing with each other. You're dealing with massive, MASSIVE amounts of venture debt from companies that have no road to profitability in other areas of their businesses. And since it's rather expensive to get started, it's very difficult to bootstrap a cloud company. If you don't have the venture capital, you can't afford the servers. If you get the venture capital, you'll never be profitable enough to lower your prices.
(I'm working on a cloud platform with some buddies and it will always be both competitive and profitable as long as we never pay ourselves salaries - it would require several *thousand* servers before the cumulative profit-per-server could pay us competitive wages)
I'm a cloud expert instead: VPS are expensive, the key to lower the costs in a cloud environment like AWS is to use their native cloud services, avoiding any service VPS like: EC2, ECS, EKS, etc. this guy was unable to do, posting that just showed his lack of knowledge, TCO can be easily lower than on prem, but you can't improvise, you have to be prepared
Competition reducing prices is economic propaganda
For anything I've ever stood up, it's always been substantially cheaper to do it on our own hardware in a co-lo than putting it in any cloud, especially if we want anywhere near the performance we have today on our own hardware. None of this is a surprise to me.
The conclusion is: Start in the iAss, when you grow you will know where to put the infrastructure in ;)
you mean IaaS?
Nah, prety clear, iAss
I am what about you guys? 😂
Instructions not clear. I ended up in prison.
In singapore right? right?
@@chudchadanstud roflmao
For those wondering about Reserved Instances: they used 3-year Reserved Instances to calculate the price.
Bottom-line, and how I personally look at this: have an off-cloud strategy. The cloud is great at a very early and/or explosive period (either customer growth, or new infrastructure), but you should plan to staff up and have a cloud exit strategy while the cloud is still advantageous.
But when you need to deploy globally for example because streaming, on premise doesn't look that good
@@neociber24 Exactly! I actually pushed to move our workloads from on-prem to cloud last year. We have all our workloads on the cloud, and the reason was two-fold: speed up moving to a new paradigm of development and deployment more quickly (aka implement DevOps, as a strategy), and for geographical access (we have customers in 3 different continents). But we're already working on our new on-premises infrastructure, first to move our development and testing workloads, but ultimately, to become another DC in our global infrastructure. End game is having our pipelines generate cost analysis so we can choose where to deploy different workloads, aware of the benefits (and costs) of each option.
Hybrid
I disagree. You're planning to hire an entire staff to deal with ordering hardware, configuring VM management, mega-licensing costs from someone like VMWare... All of these things that have *nothing* to do with your business and you have to hire a whole team to do it. Even if you end up saving some money (factoring in staff costs makes me question that you actually would), the management overhead there is huge, NO THANKS! I'll gladly outsource it to cloud services
@@themartdog You're assuming that's not a consideration. You're also assuming we don't have the trained staff to do so. Both those considerations are not true, but YMMV. I was just sharing my experience, not inferring knowledge of everyone's infrastructure decisions! ;)
PS: you're also assuming that every company's needs fit into something that is "workable" in a cloud setting. For instance, egress and overall bandwidth costs are not a consideration for most people. In our case, egress alone was 30% of our bill, and we already found better solutions, both more cost-effective as well as more manageable for our teams. o/
The prime values that a cloud provider delivers are :-
1. avoid upfront cost of infra and talent ownership.
2. provide flexibility to scale up / down as necessary.
3. allow distributing workloads across multiple regions / zones.
4. reduces complexity by providing an API over infra and services to allow composing your implementations rather than deal with the underlying implementations.
In case of on-prem deployments, considering that they're multi-regions, it's usually still a single zone per region. You could have hardware level redundancies but how do you auto-swap? How do you deal with ISP failures, weather, sabotage, etc. The problem is most people don't know what worms does the on-prem can has.
If the on-prem infra is the physical filesystem, a cloud provider is the logical filesystem over top of it. Techniques like IaC (Infrastructure as Code) basically allow you to programmatically / declaratively not only spawn infra but also allow you to observe it, scale it up / down and even protect you from major security threats.
Does all of this come at a premium? Obviously, DUH! It's not a best effort basis deployment which most on-prem deployments are. There's usually an SLA and a very high one at that.
Also, almost no one discusses the talent cost, local regulatory costs, technical debt that accrues and slows everything down gradually? And lastly, what do you do when there's a sudden spike, let's say 300% on Black Friday and then you don't receive such traffic again for the whole year. What do you do? Over provision 5x? What about the unutilised capacity available for the rest of the year?
If a company doesn't need most of the features mentioned above and has extremely predictable and mostly static loads, sure, the premium for the cloud provider might not make sense. The most probable reason why Netflix sticks to AWS is for the ability to seriously scale up / down in seconds and for AWS's expertise is maintaining a behemoth such as itself.
Finally, scraping the internet is going to be expensive. Should've thought about that before building a business around it. LOL!
EC2 doesn't provide most of those benefits - it's just a server rental. There are plenty of cheaper places to rent servers.
@@thewhitefalcon8539 who said it needs to be EC2. Though, I'm sure if a company that serves 10 - 15% of the internet, Netflix, is fine with using EC2s then there's some value proposition that really works well.
SLA for EC2 is only 99.9 (3 nines) and 95.0 (1.5 nines) for 10 and 25% discounts respectively (most of AWS is similar, you don't get 5 nines from Amazon)
If you can't accomplish 95.0 (18.25 days of downtime per year, more than 24 hours per month), you're doing something truly wacky
In regards to staffing, someone from ahrefs stopped by the comments (and other people backed them up here), the staff cost for devops is basically the same either way
I don't find the dismissive comment at the end helpful at all. They aren't complaining, they're sharing their information. It seems strange and derisive to punctuate with 'By the way, what did you think was gonna happen, boneheads? LOL' if the information is meant to be conveyed compellingly or convincingly
Exactly, their use case is completely different than EC2’s value proposition.
I’m not saying that doing yourself isn’t possible, I’m saying if your business isn’t around managing infra, you shouldn’t be mostly doing it yourself unless it makes business sense to do so.
Ahrefs business revolves around scraping the internet and greedily too. I’m not privy about their approach to scraping but from what I understand they’re simply putting in compute to get as much content as they can.
Considering the rate at which data is being generated and made available online, this is only going to become more expensive or time consuming.
This is a scale and optimisation problem and throwing compute at it is not going to make it easier.
I've been conditioned to think that buying vs renting/leasing is more cost effective over the long term. Think real estate and autos. I remember when AWS cloud was taking off and having this twinge of misgivings about the costs. Plus the fact that you are beholden to the owners of the cloud. I agree with everyone who says there needs to be a balance. The ability to be able to spin up hundred/thousands of servers at a moment's notice is a great convenience but once your infrastructure is more mature, the cost savings are too great for standalone.
That's always was the supposed argument of all arguments for cloud - "you only pay per hour for what you use, so you can't multiply that cost by whole month". It always seemed silly to me because, yeah, sure, if your server ran for just a couple of hours every day it might make sense. But in reality, most of the time that's not what happens. You need to have some instances that just run all the time, even simply because the demand won't drop below some level. And it's not like you can just nuke all your databases or delete all your data from the cloud every day because it's "not in use". Ahref's case is especially striking, because the cloud cost overhead is so high that even if 90% of their servers were never needed, they still are better off not paying for AWS.
I wonder why in the recent years the tech industry is making dumb choice after dumb choice, it's like engineering has gone completely out of the window.
so much abstraction
IaaS (ah yes... The cloud)
DC = data center
The Cloud just means someone’s server. MS and AWS now retain the same hardware for up to 7 years now. You are being had!
i love the datacenter being an api.
If you have a large enough scale for it to be worth it, open stack or canonical MAAS can give you API driven self hosted hardware :)
@@KadenCartwrightI’ve only worked for companies with
I think the key advantage of IaaS is not flexibility in capabilities you invest in, but simply the much, much shorter timeframe of investment. You can double your infrastructure today, and cut it down to a quater tomorrow. If you did that with hardware, the decision you made today would mean paying extra 200% capabilities for the next 60 months, on top of the fact you’d be stuck with the original excessive 100% you would have even without todays fuck-up.
The fact you couldn’t even do such a choice because the timeframe from decision to actual capability is so much longer. If you buy “off the shelf”, you might have a bare supervisor running in less than a month - provided you only bought few machines, your network configuration didn’t need reworking, your UPS had a vacant spot, the server had memory pre-installed and the storage was delivered before the server or in the same shipment.
it’s like buying a car versus renting it. You can think about renting a car for ten years, and conclude that it’s ridiculously expensive. But at the flip side, you can’t buy a car for a weekend - that’s not how buying a car works to begin with.
Good luck trying to get over ~10,000 instances in a single region spun up within a small time frame. Even on a corporate account, you need to plead a case for a need for that many instances which goes through a ticketing process that can take days to weeks which is still dependent on if the resources are even available in the region at the time, which in the case they're not your request is simply denied.
Then take Netflix who was able to host the entirety of the US eastern client base off a single rack and is able to spin up hundreds of thousands of instances whenever they want at a fixed rate.
@@callowaysutton
About 10k instances, it's actually pretty edge case, probably that number even more than any cloud provider have in a day worldwide. And it's actually hard to find a task for this 10k instances, so yeah they want to know will they be paid for this, and for which task it needed, because it can create a shortage for other clients.
About netflix served from single rack, depends on which timeframe you refer, because earlier they used Akamai, and now they use their OpenConnect rack for ISPs. So it's almost never was a single rack.
Per each 1gbit they can serve approximately 400 active streams(average 1080p stream ~512KB/s with some fluctuations), if we assume that some pause video + they ok with some small almost unnoticable delays, then it can be approx. 1000 users of decent content provisioning. From requirements of OpenConnect they can connect it max to single 100gbit, So 1 rack can serve max 100k users with ideal conditions of course, and it's only content serve without site logic/analytics/etc..
UPD: even less than 100k, I forgot to include IN traffic, because it's one same NIC for IN/OUT.
Is it real that Netflix has a single rack for an entire region?
@@Sergey92zp Here's the thing though, I could spin up 10,000 instances very easily on any medium sized $100-500k privately hosted cluster and not even be close to reaching the total capacity, so it being an edge case isn't and shouldn't be used as an excuse for an IaaS type of commodity. For reference, spinning up 10,000 of AWS' smallest EC2 instance in US East costs a whopping 100k+/year, and for that you get 10k shared 2 core, 512MB of RAM instances... or about 20,000 vCPUs and 512GB of RAM. Let's break this down further; it's a common industry practice to split each physical CPU thread 32 ways to each vCPU so that gives us 625 actual threads. The newest Epyc servers come in configurations of 512 threads per server (you could even have a 4 way multinode in 2U for a total of 2,048 threads in 2-4U worth of space) and 512GB of RAM is frankly nothing considering these types of servers go into the terabytes of RAM. If we discuss space and networking, a single 42-47U rack with good 100Gbps can be had for ~6000/m now, or ~72k/y, and considering AWS doesn't ever let a single customer go above 100Gbps per zone it's a good frame of reference. If we were to amortize over a 3 year period that gives us a 150k budget to compete with AWS and with that kind of money, you could easily afford multiple tens of terabytes of RAM, thousands of cores and much faster disks.
For what you're saying about Netflix, you're just completely off. I'd recommend looking at their many presentations about their infrastructure and how they've saved money by using their own hardware/network.
thank you for your videos! I really did find joy in engineering again since discovering your channel🙂
To be fair, if your idea of using AWS is just paying for dedicated EC2 instances, you’re doing it wrong. You need to architect your applications intelligently to keep your costs down.
And get 100% locked in to AWS?
Yep, lambdas, kube and etc, but al that depends on the load you have and the codebase.
RUST BTW
@@lhxperimental Not necessarily. There are many AWS and Serverless providers. There’s tons of ways to roll databases. There’s many ways to do compute and single sign on and such.
Not everything has to be in AWS and really you can architect your applications to appropriately fit cloud services in some places and monoliths on dedicated hardware for others.
@@lhxperimentalthats why you have a exit startegy from the cloud, or use terraform if you're sticking to cloud stuff to move as easily as possible.
You actually choose whether you get locked in AWS.
AWS is locked in WITH ME >:)
@@lhxperimentaldon’t use your phone send us letters with a parrot so google doesn’t lock you in with UA-cam. You obviously didn’t touch a server in the past 10 years if you think a company that large can rely on ec2 instances
As someone that is supposedly a software architect, but always ends up having to do Ops on the side, this ignores the biggest cost point in all this: IT and developer time.
We're not just talking about making sure the power stays on. We're talking about automatic backups, configured with a few clicks, autoscaling easily, and starting up and tearing down developer instances in seconds. Suddenly, your IT, or worse, your developers have to do all that.
If you're already big enough to afford 40 million in servers, chances are you have a big enough IT department already. My personal experience though is that the IT department is where every single CFO tries to save money, and it fucks EVERYONE over.
With things like OpenStack, OpenShift, OpenNebula, VMWare, Canonical MAAS, heck even Proxmox that is quite literally not an issue. Almost all platforms have a way of autoscaling, backups, high availability, hardware issue alerts and more features that used to need a lot of specialized knowledge. Now, a single person, or a small three to four person team, could potentially manage the actual hardware of a rack or whole DC location and it would still not even make a dent in the difference between collocated and cloud costs assuming salaries in the low 6 figures.
In Proxmox for example, you could take a physical server down for maintenance and have it live migrate to another node with no noticeable affect to the client. All platforms have at least this feature and more; you could even migrate across two different DCs/locations if you really wanted to, which you can't do in AWS, GCP or Azure yet.
Canonical MAAS literally manages the hardware for you and with its hooks you could potentially even have it automatically auto order parts that failed tests, safely turn off the server, turn on the iBMC LED (to show remote hands which server is having issues), submit a ticket to the DC with the shipping number and have maintenance effectively automated.
The IT landscape today is not the same as 2008 and the large cloud providers are definitely starting to show their age big time
What do you think of companies where they have people dedicated to managing the infrastructure so it doesn't bleed into developer responsibilities?
You need those engineers with AWS or on-prem anyways. So the cost comparison is fair.
@@Victoria-ij3cb I definitely think, if the company is big enough, on-prem is can be nice, but only if your feature set (and therefore requirements) are basically static. Otherwise, what would a single click in AWS to try out (e.g.) K8S turns into MONTHS of waiting, and then getting a faulty, buggy on-prem K8S.
Another BIG disadvantage I've observed in pretty much every bigger company is that IT is understaffed, underfunded, overworked and overregulated. Things that would be a single click on the cloud can take months.
@@invinciblemode Not even remotely true, at least in my experience. For example, my personal, very painful experience with on-prem K8S: We didn't have an expert, so someone just had to google shit and hope something useful happened. Obviously, it took forever.
Then at least 5 different features that are a click away in the cloud, even for the dumbest developer, were missing. Things like a basic load balancer, an API gateway, a replicated database, or a decent storage solution. And again, you'd need experts for all those things.
With a cloud K8S solution, all that just magically happens, and is maintained, automatically.
Even if you only use the cloud for as dumb VMs, you're saving a lot on people that need to setup, install, and maintain your racks, and the software (hypervisor, etc.) running on it.
I guess I just have PTSD, because EVERY SINGLE IT department I've ever come into contact with has been underfunded and overworked, making them decidedly far worse than the cloud, at least from a developer perspective. Maybe in heaven they have a wonderfully running on-prem cloud.
I never noticed the little light up border on the like button that happened when you had your early video breakdown... It only does it during that outburst too
The reason I use AWS as a startup that has a huge storage commitment and moderate EC2 Usage is that I can scale out and in at will, so for example I'm on a some downtime right now and I scale down the system dramatically lowering my monthly commitment while keeping the Infrastructure. I spent a few years in a COLO and while it definitely was cheaper, when I needed to scale up I had to put out a ton of cash and worry what happens when the scale goes down.
I guess the most important thing for any company moving to the "cloud" is to spend some quality time with Excel and figure out your options.
If you want a reliable, scaleable service with backups, I think this is the sort of thing you think about when your AWS costs go above the cost of, say, 4 skilled full-time engineers. If you're spending more than a million, you can bother thinking about it.
You don’t need 4 full-time engineers to keep up few servers. I’d argue cloud is more labour intensive, because poor resource usage costs so much.
On the other hand if you have ridiculously oversized on-prem setup, it can survive multiple disk-failures, hw-failure and still work. Yes you want to fix it ASAP, but the staff you need to do so is not notably expensive compared to managing cloud.
You will lose few nines of reliability compared to amazon. But just because amazon has five nines doesn’t mean your product on AWS has. In either case the likeliest failure point is the same: your own stupid ass or asses
Even if you use AWS; you still have to have to spend time and energy handling backups. I'm not sure why people use "backups" and "reliable" as key benefits for Cloud. Cloud isn't inherently more reliable than computers; since that's what it's based on and it's not abstracted *that* much.
Plus you will have less access to the underline system if you need it. It’s all about balance and ownership etc
Did Parler have reliable, scaleable service with backups. when they got cut off, they could not use their own database because they trusted.
@@w300x how would you build multi region redundant backups on prem?
Idk, the difference between the two scenarios is so insane that it could mean that allocating your own hardware will always be cheaper than any approach on AWS
The upfront costs are very high though so like stated it's not feasible until you have enough cashflow that it doesn't hurt the company to make those upfront investments. Pay as you go is pretty much always more expensive in the end but you don't have to have the money now. Like buying a house for example.
This is why aws is Amazon's money printing venture
as someone who came to development from SEO, I love the way your pronounce A-H-Refs.
So that is even with reserved instances.
1 TB of RAM sounds ridiculous to me anyway. And I am an actual Java developer, so quite used to stuff eating up ram like no other.
And I've done self hosting in colo in the early 2000s. For some reason management never seemed account for replacing cost every 3/4/5 years and trying to wring lots of 'free' time.
A lot of ML stuff can use a lot of ram. BTW a lot of RAM is also very useful for storage caching. Idk how necessary it is when you can have a 16x NVMe RAID 10 tho.
The amount of vector logic in AI/ML nowadays will make you crazy. Literally Python code jerking off of each other and produce pretty image or cosplaying as a human will cost you more than just 1TB of RAM.
Been preaching this for years; loving lifting and shifting out of the cloud and saving companies millions.
exacly the problem, expecting lifting and shifting services as it is and expecting a better result. That's not how cloud works.
Cloud is good for temporary things. Slight surcharge of traffic during holidays, proof of concept, testing a new product.
it blows my mind that they just brushed over capacity planning and autoscaling as if they need to pay for Ec2 instances that handle their full capacity 100% of the time
That is very true actually, but i assume their load is reasonably high based on their on-prem capacity already.
AWS is expensive if you don't know how to use it correctly
@@matthiasg4843and nobody knows even Amazon
Great point but to be fair since it was 10x more expensive you'd need to run at
Fair point but when you're dealing with very large instances like they would be you can't rely on AWS having enough capacity when and where you want them.
Regarding Netflix traffic: I select how much bandwidth to hire for my home by the number of TVs I'm planning to have on simultaneously. The rest of the traffic is mostly irrelevant.
My company is going through the same type of thing except they've decided to go for cloud. However, there are more costs involved that they didn't take into account. You have to host those server and then maintain those servers with staff etc. You also need to provide redundancy backups etc etc where as the cloud provider provides those things for you. There are also some things to consider since we are a multinational company as we need to maintain different clouds for different regulations etc. We had multiple individual datacenters all over and had to support them with hosting, network interconnectivity, etc etc. So for our solution, a cloud provider (ironically multiple) is what was chosen. However, I can also see that being temporary until we get the final costs and then someone gets the idea that we'd save more money by setting up our own datacenters again. It's the typical IT cycle.
High core-count CPUs, 2TB of RAM and 2x 100GBPS bandwidth servers and 80 TB of storaege.
Finally something that can handle "npm install"
Such a nice and enriching video Prime, I love this video format
Always good to have mix and build system they don't rely on a specific infra so you can always move it depending on cost, only concerning thing I have seen is a lot of people only having public could skills and it usually just a particular one, You can take a infra person and they can move/build platforms whereever, but these people with those single public cloud skills sets you kind of don't have they underlining knowledge of how it all works.
Key Takeaways:
The video discusses the cost of running infrastructure on the cloud versus using on-premise hardware.
The company, AREFS, compared the costs of their own co-located data center with a similar installation on Amazon Web Services (AWS).
The analysis showed that running the infrastructure on AWS would cost AREFS significantly more money compared to their own data center.
Summary:
The video explores the cost implications of running infrastructure on the cloud versus using on-premise hardware. AREFS, a company, compared the costs of their co-located data center with a similar setup on AWS. The analysis revealed that running infrastructure on AWS would cost AREFS significantly more money, even taking into account their revenue. The video highlights the importance of periodically re-evaluating cloud benefits versus actual cost and suggests considering a reverse migration from the cloud for more mature companies. The analysis also takes into account factors such as people skills, financial controls, and capacity planning, among others. Overall, AREFS found that managing their own infrastructure was more cost-effective and allowed them to invest in product improvements and development.
Statement that at certain scale having your own infrastructure ia chepaer is not a magic or new one. It is known for decades. The problem is that your own DC will never produce a fraction of easily consumable cloud services than AWS, Azure or Google.
Exactly, when you start is better use a cloud service. When you pass a certain scale of course you can own your own infrastructure to reduce cost.
@@oscarljimenez5717 Yes and no. You have to keep that in mind during development or at least during planning. For example set up your clusters to rely as little as possible on proprietary technology and use your own instead. For example configure your own load balancers inside your k8s clusters so you can just move the cluster to another location easily instead of relying on proprietary stuff (like AWS's and Azure's load balancers).
0:38 - 0:43. You better press that like button.
@ThePrimeTime I am surprise that they didn't take into account that when you plan to have a server on AWS for 1 or 3 years there is a HUGE discount for reserved instances which lowers the price -70% down... even more if you use a mix of reserved instances and spot instances (which are -90% discounted). They can save money going to the cloud and serverless (dynamodb) + lambda + s3 can also be considered for super fast retrieval and serve of data instead of using EBS volumes.
Maybe they had some issues with AWS and pepper sprayed this article to them 🤣😂
They did factor longevity. per the article, they used the price of a 3yr reservation.
What is the successful cloud start in my opinion? You need to be a "nomad digital angel"... What do I mean? You need to be able to jump from clouds to clouds easily like an angel. So you need to architect your infrastructure in a way that allows seamless cloud switching without any significant troubles. Use only the services that are based on popular open-source projects. Even if they are a bit harder to use. Every proprietary service will be yet another heavy anchor tied to your leg. So when you start your project cloud helps you to reduce the initial costs. When you are a bit bigger you can switch to the cheaper cloud and hire some infrastructure specialists. When you are big you can build your own cloud and move there. Success...
aka containerization
Agree 100%. And yes, containerization is the key.
Sounded like Ahrefs was doing a lift and shift comparison with just infrastructure. It would be interesting to have digged into their architecture to see if a cloud native architecture could have fitted better.
And vendor lock them to AWS for ever? Or spend more than Prime Video which ditched their "cloud native" approach?
@@meletisflevarakis40 I would not read too much in to the Prime Video malarkey, it did make good headlines 😉 For me, main takeaway was it reminded us there are good and bad architectures, and just because it's serverless does not mean it's good. I.e, if you are hammering Event Bridge (as was the case with Prime Video), then you will have a bad (expensive) architecture.
@@kazdaman1 vendor locking is a huge problem that very few seem to even consider nowadays
@@aaronhamburg4428 IT's a "future aaron" problem. You deal with it as you approach the issue. Until then, don't waste your time - focus on product and business development.
@@aaronhamburg4428it’s like people just lose their memories every five years and all knowledge of previous bad behavior from vendors who abuse lock-in tactics is wiped from the face of the earth
If cloud didn't lock you in so much, a hybrid approach would be much more effective. You could rely on "the cloud" for usage spikes, or for scalability. Say you suddenly needed 100 more servers, you could order them, and then use 100 cloud servers until your servers arrive and get set up
That's not how bozzo gets a new yacht
HPE is a good example of the hybrid cloud approach
Nitro Unjs
AWS recently has been pushing for more hybrid technologies. Services they have been promoting lately help facilitate that. Better to capture part of a business than none at all
Spikeloads are literally the #1 best usecase for AWS.
The real issue with clouds and why most choose cloud is because eventually you will have to go through legal stuff/regulations etc which will force you to move to the cloud anyways. Like take SOC 2 Type 1 certification. That BS costs anywhere from 75,000$ to 150,000$ per year to get done. And if you are on cloud you can literally cut that cost in half (or even more) of the audit. If you don't want enterprise customers you can stay away from the cloud. The moment you want enterprise customers, you will be forced to move in to the cloud. On premise is another beast.
16x15 gigs = 250 gb = 1/4 petabyte per server. RAID mirroring gives you half of that, so 120GB usable space.
Amazon doesn’t get free computers, only cheap ones. Then you have utilities + profit margin + Bezos yacht fee = the same or higher cost as buying a better server yourself.
I think there's a lot of things overlooked there, but at the end of the day the premise that the cloud is more expensive isn't even unclear... it's a well-known fact. It comes down to busines priorities. Woudl you rather employ people to work on improving your products, or just running them? Companies that lean heavily into cloud, or are even 100% cloud, are prioritizing working on the products rather than infrastructure. They outsource the infrastructure.
Bung nose? It’s ‘Monsanto Season’, the special time of year when North America douses itself in expensive mystery compounds using expensive diesel machines so bread and meat can be cheap, in Morocco.
I think they made some mistakes in their calculations. Normally when you host such a system on AWS you don't compare your Servers directly because your instances run in Auto-Scaling Groups and your system scales up and down. This in the calculation means that all the server are 100% loaded at 24 hours of the day. But normally when you have times of a day where no one is on your website the system will scale down and delete your instances. So i'm not sure if that calculation is really fair. And there are some other opinions like Spot Instances. But yes AWS is expensive.
And what another user mentioned is that you need Engineers and Backup strategies and so on. But i think when they scale very intelligent on the load i think AWS would be much cheaper then the calculation mentioned.
they specifically choose to reserve instance, means they literally rent the logical space for the server in the DC. And also, considering the HW specs they buy for the on-prem servers, I don't think spot instances (or non-HA instances) would be adequate for their use case.
AWS has been caught selling the shovels
7:00 Clustered file systems yield at best 50% of the storage space because you have 2-3 copies of the data scattered across many disks servers. So if you buy 3x10 TB, you can utilize at most 15TB. In reality you'd only get about 10TB with 3 disks, but as you approach 20 disks you get closer to the optimal 50% capacity.
As a SysOps guy watching here to better communicate with my dev colleagues, I can say the point where migrating away from cloud makes sense is much lower than you think.
Granted I'm in Germany so we might have other expenditures for people and other stuff. But I would argue, that one of the first additional talent a start-up, with say 2 devs that founded the company should look for, is a SysOps guy, even if you stay on cloud for a while. Sure I'm biased because I am a SysOps dude, but cloud doesn't make infrastructure trivial tho. Or rather on-prem /colo isn't that complicated for someone who knows their way around servers. Also what you need to consider ist where all you data is stored (especially when you are a Europe based company), are you thinking about backups, where are those backups stored, and so on. With all that cloud easily gets more expensive than on-prem or colo.
With cloud it is the same thing as with everything, it has it's use cases but it's not the be all an end all. The tech sector tends to jump on a hype and must use it for anything without considering it's pros and cons
they didn't include the salary cost of keeping infrastructure engineers on to frig around swapping out hard drives... setting up server patching.. making them actually work. Alternatively you can buy service contracts to get the skills. the 10X would quickly scrub that out though. We have done a similar cost analysis and reached similar conclusions. cloud is orders of magnitude more expensive.
Big part of the cloud is using it effectively. You have the power of the largest and most feature rich cloud provider of the world and you are using the most simple service they have for almost everything. This is an easy comparison, but it is not the right or fair one.
I think this explains why Amazon is profitable and so many other companies, who use AWS aren't. Years and years ago, I remember being told to invest money with Amazon and Microsoft because their Web Services divisions were making bank. Well, I know I know why? (Assuming the article is to be believed)
i like how when prime goes insane by saying press the like button, youtube detects the like keyword and highlights the like button
Not an apples to apples comparison by any means. AWS is about services and resilience not servers and power costs. If you don't need the services or the resilience AWS provides (think auto-scaling multi-region services in minutes) then there's no point even thinking going with Cloud.
My company is all about the cloud and migrating everything on the cloud for certification sake. The problem is that they don't think about the cost and performance.
I've noticed the things we are running are more expensive and slower than before... Hey, if they are fine with this...
They should calculate how many engineers they have now vs how many they would need for server less .
Probably they would still save money but maintenance of so many servers is also expensive
Its not as much works as its made out to be. Most systems you rack up, and leave alone for 3-5 years. Maybe 10 hours hands on for the whole life of a server.
Also after the mass layoff, the cost of hiring software engineer will be a lot cheaper
experience cloud engineers are costy (at least looking by offered salaries for cloud ops) and there are less of them, many are people who only did some course
sysadmins eg linux are easier to find, the more open tech the more people that can work with it
also adopting projects to their techs eg servicebus, leaving it then - is lot to rewrite, not to say initially coding for azure/aws specifically may be extra job for extra person who knows its apis
setting a server is really not that much work except initial bringup, since you virtualize everything and can provision VMs on demand, the cost of running 20 vs 40 VMs will be similar because you already have that hardware, while every VM on azure has own cost
Add 30 engineers to just work out terraform quirks an constantly broken compatibility/APIs. Just working out cloud quirks and limitations is equal if not higher time-cost than self-hosted.
Why scale down the workforce and deal with payouts when you can just massively increase the workload of the existing team piling on the critical responsibilities without increasing anyones pay except for mgmts. bonuses.
There must be some crossover point between AWS and on-prem in terms of cost vs performance. Clearly these guys need performance beyond that threshold. I'm certain there are plenty of people using AWS that fall below that threshold of performance requirement.
I guess that's why there are people that get paid a bunch to work out the cost effective strategy for business infrastructure like this.
AWS has its own power plants I can not recall what regoin but they bought a towns hydro plant and took it off grid , just for youtube.
16x15 is super easy math for 8 bit programmers from 80s.. because 16x16 is 256 of course, so 16x15 is 256 - 16 = 240.
It's apparent Prime is lot younger and haven't been doing 8bit assembly... ts ts... :) ;)
Moving to the cloud is not just creating ec2 instances lmao, there's a lot of architecture you need to change
for the overwhelming majority of saas businesses it is. You can run a pretty profitable business just with a lambda function and an api gateway.
love these article reading videos! please continue with them!
also, how do you find out about such articles? would love to get exposed to such content by the daily
i have been trying to up the amount of content i am creating.
we will see if i can make more!
@@ThePrimeTimeagenThe real question is: how much we can handle? 💀 I'm already cherry-picking because of not enought time to watch all of them 😭
Cloud: its cheap to get into when you're small, but once you've ditched your technical expertise, can you rebuild a DC from scratch? Do you need "cloud scale" Do you even need "cloud scaling" or will HAProxy be fine? Get your IT provisioning done by cookie cutter.
"imagine if all your money went to aws" I can think of at least a few companies which this caused their downfall, although not always aws, but using other cloud providers too.
But they’re not considering the cost of personnel for setting up, maintaining and securing the on prim servers right?
ya
Let's say you decide to leave the cloud. Do you need to replicate the "services" that you had before. Is there an opensource equivalent of google cloud or aws that you can host yourself? I'm far from being a cloud engineer but I was trying firebase but decided to go with supabase. Is there something similar with aws?
AWS does give you the ability to run some of their services outside AWS, yes. They also have AWS Outposts so you can set up your own AWS "partition" in your own datacenter that can have EC2, S3, etc. all through the normal API.
If you are so dependent on AWS, your tech stack was built with lot of omissions. Pull up your socks and build on stack that you can own
@@lhxperimental That is exactly what I'm asking. How to develop without tying the code to an specific platform. I've heard of openstack and terraform but I know little of them, are they even related to this?
I mean in your example supabase is a "Foss" (I don't think their dashboard is FOSS) "alternative" (doesn't really have the same set of functionalities ) to firebase but their hosting platform isn't open source at all, that's what you actually pay for, the managed hosting they provide.
Or maybe you mean "is it possible to self host dynamo db (or whatever)?" ?
@@heroe1486 Yep, possible to self host
While growing, AWS is probably the best thing, but there's a point you get to where you need to move to yours, same with serverless, at a point you reach, you will have to move to using regular instances
At the sole distinction that you actually don't need "serverless" (which is a marketing word meaning 10 different things at that point) from the start
It doesn't even make sense when growing, unless your growth is absolutely insane. You could easily overprovision everything by a factor of 5, and still spend less with on-prem. Can you quintuple your users or order some more servers faster? My guess is it's the latter.
Cloud only makes sense if you're small and the whole cloud bill is less than the cost of a few skilled sysadmins, or if you're worried about some kind of 10x weekend taking down your infrastructure and that the costs of that are worth paying 5-10x as much for hardware all the time.
@@OMGclueless that's just objectively wrong. Growing = not a lot of users. Renting resources for a reasonable price will always cost less than hiring even one engineer if your workload and users aren't a lot. A cloud service is probably the best way to go
@@festusyuma1901 "growing" doesn't say anything about big or small. TikTok is (or was) growing, are they small? If you're small and a sysadmin costs more than your entire IT bill then cloud makes sense.
@@OMGclueless my bad, be growing, I meant small business/startups without a large enough audience
even smb wants local compute not cloud - this was a good look at the costs - you can depreciate your own servers also but overall it isn't even close
Pressed the like button just when it glowed. Feels like I won at a mini-game.
That's like saying that you saved 20 Billion by not buying a Nuclear Powered Aircraft carrier. Not buying something isn't a saving.
What the f**k. At the begging of the video Prime is saying hello to youtube and at the same time encouraging to press the like button. My like button for this video glows UP to press it. Not gonna lie, I got a bit spooked
I'm 41, call me old school but I'm at an expert at supporting on prem customers. I work for 15 large jails across the country and these are County entities who never have any money to do anything so there's no way they would go to the cloud.
At a certain point it's cheaper to hire your own talent to manage your own servers. AWS fills the talent gap a lot of companies have. I've been running my own servers since I got broadband in the early 2000's. Most developers wouldn't know where to begin. So for companies trying to run lean, they would have to hire a dedicated DevOps person or use AWS and pay a contractor a few bucks to set it up or let someone who has a lot of other responsibilities do it since it's such a part time concern.
I have a $900 desktop running my stuff on my 1GB fiber connection out of my house that would cost $900 per month on AWS.
Cloud services are basically insurance. You're paying a massive premium to shove liability onto another company.
Hey Prime, have you realized that the like ratio is usually 10%? it's pretty steady across all channels content, a few go above 15% but I haven´t seen a higher ratio, there should be some anomalies anyone could share examples???
You yelling press the F***** like button made me press the like button. Who would have guessed what it took was a moustached man in a hoodie yelling at me?
Netflix transfers more data than porn: The reason is that Netflix serves generally higher bandwidth streams compared to porn sites that usually are of lower bitrates. Some porn sites even default at 240p or don't go higher than 480p.
i like how prime randomly gets autism attack's and then just shruggs it off as: 'anyway' and continues on reading 😆
Dont know much about the cloud but the host is entertaining
Honestly me neither, Primeagen is just fun to watch. Then again you probably won't have any real knowledge about the cloud unless you work at a company that uses them or are just into knowing about cloud infrastructure
What about all the support costs for all your hardware and how many people it takes to support that hardware??? The people costs would be enormous when you factor in benefits. I don't believe it would be that big of a Delta.
The main problem is how much resources do you currently have vs the real usage ones in the cloud!
AWS has discounts up to 70%
Still 2x more expensive for worse hardware.
It's a trap!
@@Ravengenoyup buying dedicated servers is way way cheaper.
@@fritzstauffacher6931 but you have to pay people to run them. those are _not_ cheap. And their processes aren't as mature. Your SRE team isn't born with runbooks. I can stand up an arbitrarily large database cluster with multiregion HA and point in time recovery in like 3 mins clicking around. And it actually works. The hardware isn't the point at all.
still muy caro
The problem for the banking industry, is nobody will do this kind of investigation as the "safest" approach is to do what other enterprises do, which is getting in bed with a consultancy firms that make money off of convincing execs to go invest in IaaS, ay-ey-ey-esss
This is crazy. Economies of scale should make the cloud cheaper. Prices will ultimately be the death of this model.
it's obvious taht its way more expensive than doing it on your own, but thats the point. almost all companies who use AWS are not capable of doing this stuff on their own, so they pay the premium anyway. and those companies who could maybe do it on their own, they sometimes will still willingly pay the money just to scale faster. for example if youre a young startup that just wants to grow quickly and has no aim to be profitable within the next 3-5 years, then they'll just spend this money and worry about doing it on their own, once theyve grown to a state where they can manage building an infrastructure on their own. This is exactly what is happening at the company that I am working for right now. We spent all our man power on growing teams and getting everyone on the same page and stuff like infrastructure we just spent the premium to get it going. If we built this stuff on our own (don't forget this isn't the only thing you need to build when starting a company) from the start, then we'd never have grown so quickly and probably got stuck after a few months
Can somebody explain how come Walmart transferred their workloads literally by trucks to AWS? How does that compute financially?
AWS has a service that will send you physical hardware to upload your stuff and physically place them into data centers
It's called Snowmobile (yes, it's a true name). Btw. there's also Snowcone and Snowball xD I just love how they name their services. As for financial aspect - no idea, but it's most likely costly af.
Wait till shareholders begin to understand the cost of operating in the Cloud.
You thought you would save on infrastructure management costs, but you still end up hiring AWS IOPS experts.
a company like ahrefs should have vertically integrated infrastructure anyway... unless im missing something, totally not the use case for cloud services
Even just our App Insights bill is astronomical. imagine having to pay for logging.
I mean, for the numbers of Ahref, the amount of people managing the system is pretty irrelevant.
Even if you paid 2 million each to 5 system engineers, meaning 20 million in two years, that is still 380 millions cheaper for them than AWS.
Hybrid is the way. Some stuff makes sense in the cloud. Some doesn't.
If Cloud then EC2. Not Serverless.
The Hyper Scaler makes more profit if their connected services are slow in responding. Simply as you pay for waiting in a serverless function.
AWS is certainly the 800 lb gorilla but I would be interested to see similar cost breakdowns for Google Cloud or Azure.
😂😂😂😂 all true. I am from VMware working as a SRE in SDDC operation. Managing own data center is the cloud choas and very very very complex and costly.
Maintenance cost of the metal and so on is not included, this should also be a big item in your spending. So I don't know how accurate it is.
What maintenance costs? Switching disks once in a while? The rest is provisioned in the price of the hardware
@@louisroche9574 You would have an ops team which does Monitoring, Software patching, periodic vulnerability scanning and much more. I would also argue that you have an lot more cost, for example you would probably also have to ISO certify. I still think, that you are cheaper on prem, than in the cloud. I would also argue that having Infrastructure know -how and having fewer dependecies on third Parties is a bonus.
@@JustVincentD devops team would probably be about the same size. Administrating services on AWS isn’t an easy job. We have very few people actually going to the DC physically. And the legal requirements are relatively light. If we were manipulating a lot of sensitive data (like health related) that would probably be a very different story indeed.
@@louisroche9574 yeah ur right about the Ops team at aws, but patching switches over the Air or Firewalls can sometimes be tricky/risky.
We are having those conversations like every week. I think it depends on multiple factors and it is a case by case decisison. Some companies should go into the cloud some shouldnt, but I still feel that manageing your infra is cheaper than cloud.