AWS's own training materials from a few years ago (I'm not sure how old this course was) provide the following use case for Lambda: a function that will update product database when an employee uploads a spreadsheet, to be used once in a while. Rather far from running an entire application this way.
Since the relative growth rate formula requires dividing by the previous value and division by zero is undefined, the relative growth rate from 0 to 1 is undefined. However, I doubt VCs can do math, so the "infinite growth rate" strat may work.
Former Amazonian here. Within the company people just throw code into lambdas, even if each transaction takes multiple seconds (like 7+ seconds). "But it scales, bro" This doesn't even take into account the added complexity of having to add extra layers to support statically linked libraries that aren't included in the lambda stack.
Speaking as someone that grew up on a computer with 4K of RAM I can absolutely agree with this. Because everything - compilers, code development, pipelines, hardware all got faster everyone stopped caring about quality of code. It became focused on “lines of code” measurement instead of the efficiency. Very happy to see some still value it.
@@tadghhenrydepends what the software is doing. Sometimes it matters but many times it does not. For some apps having a dev spend a couple of weeks optimising it costs more than you will save. For others performance is critical.
I work with a system that is mostly serverless (aws lambda) right now, and my take is that it actually forces us to hyperoptimize every function because of how much your cost is directly related to performance. 50ms extra latency per request when using containers? Probably not a huge issue. But 50ms per request when you are literally paying for every extra ms of runtime? Its a huge cost implication if you are dealing with millions of requests per hour. Also, memory leaks do matter despite a lot of contrary info out there, since lambdas that stay "hot" will share memory. We actually had a slow memory leak take down one of our lambdas in production.
@@farrongoth6712 I agree. It definitely limits you on options to optimize too, since you have to treat every request as an individual execution. There is some fancy stuff you can do with layers/extensions, but its still limited. I don't think serverless functions are bad, in fact I think they're a great tool. But, like everything, I think too many people are treating it as a silver bullet. There are definitely use cases for them, but I'm currently living the pain of going all serverless.
what is the reason to run this as lambda and not containsers? Seems like the amount of time you invested into making your code run cheaper could have been spent on setting up a k8s cluster, even with autoscaling if your traffic is variable
@@Qrzychu92 one example where I work is an overnight batch processing job. We have 16 hours every day where we need 0 instances and latency/startup is not a huge factor as it is not a customer facing service. Lambda makes it easy to simply maintain the python script used to process the data. No need to manage a kubernetes cluster just to run a script. With lambda we get the whole runtime provided for us. And devops is basically nonexistent because it just works. Assuming containers are always a better option is the equivalent of assuming serverless is always a better option. They're just two tools with two different sets of tradeoffs.
Thing is: 80% of ppl in the industry works for small medium companies where perfomance is not that important due the low number of concurrent users. They only start to think about perfomance when its too late: too much users onboarded and things go south. Some companies will never get to this level btw.
My little company qualifies absolutely. We're so barebones and basic that the value of cloudy services that automate our workloads just completely stomps the cost of a few hundred lambda calls every other day.
We are literally 2 devs. One Backend and one Fullstack. We just used simply a docker swarm, costed him about two weeks to learn and setup. Runs perfect and can easily scale up and down.
I paid $900 for an i7 Dell recently. Tons of power. It would cost at least $900 per month to have the same power on AWS. And I pay $80 per month for a 1GB fiber connection. You'd think people would be more concerned with costs if they're paying for AWS. Companies love to throw away money on everything but salaries.
I couldn't agree more. I have been leading migration of applications from these expensive cloud providers and CEOs get surprised at the cost saved. It is the economic downturn that had led some companies to realize how they have been taken advantage of.
It really depends on what your infrastructure requirements are. If high availability, local and regional accessibility, multiple redundant services, etc. are a must, services like AWS are actually quite cheap and affordable. AWS, for example, claims to offer 11 9's worth of availability, which would be monstrously expensive for a smaller or intermediate company to achieve. If we're analysing your setup, it's a single server with a single point of failure on the network. You might be able to safely run some web servers and a database on that, which is completely acceptable if your tolerance for downtime is around 2-7 days a year and you have your backups in order. But imagine adding redundant internet gateways, redundant ISP providers, electrical redundancy, redundant distribution and access switches, multiple hypervisors configured for HA, etc. The costs would easily run hundreds of thousands, even millions, and because of warranty issues these costs would be recurring every 5-10 years. Cloud providers are clearly a poor choice for smaller business entities and single developers, but claiming that using them is 'throwing away money' is quite asinine.
@@connoisseurofcookies2047 All spot on... I love that you put numbers on some of the risks being accepted. I once worked for a company that sold 5 9s when the telco only promised 3 9s. Our customents management didn't know what it meant... our management didn't know what it meant. It have a few hours unscheduled downtime each month - no-one got sued. A lot of management get high on their own hype and sense of importance "2-7 days downtime? we have to be 5 9s" do you? do you really? do you know how much that would cost if we actually did that? Even Office365 being Office 363, no-one cares. BUT most s/w people don't understand that MTBF is "within the warranty pperiod" and believe their SSDs will be found fully working by archeologists - so let's not beat up on management too much. Paying for the whole buffet when you just want a bit of salad is foolish though!
@@connoisseurofcookies2047 It's throwing away money exactly because of what you just wrote: small businesses do not need everything you just listed. So many companies use Azure nowadays and ALL their service objects are on the same region and very little of what you listed is actively managed and encouraged. The WARNINGS section of configuration settings for each object screams with red and yet nobody cares. The costs keep racking up by 1k a month and nobody cares because perception is king. Having two i7 machines with two SSDs in raid each and a simple UPS would easily cover 90% of users on Azure and would nearly never go out. By contrast last year in East US Azure went out 8 times - wouldn't be a problem if you had failover; if.
Newsflash: most companies dont need to worry about webscale. I worked for startup that did $10 million a year, and our aws serverless bill was under $20 per month. We scaled to 3 states, health services industry. We could scale to the entire US and not break a few hundred $. That would have been close to 200 million ARR. That is unicorn territory valuation. on a few hundred in infra costs.
Elixir, as a language is nothing out of the ordinary, it is the erlang VM that is insane, people learning elixir should focus on understanding the VM, if you only learn the language's syntax you wont understand why elixir gets all the praises it gets
@@majorhumbert676 “The Soul of Erlang and Elixir” by Sasa Juric is a great primer on what there is to love ua-cam.com/video/JvBT4XBdoUE/v-deo.htmlsi=tIpPXVbKgaqEIQjJ
@majorhumbert676 well, ill try to explain but is too much for a UA-cam comment, so as a summary is like a kubernetes in a box, it is like an OS, it has something like redis inside, is has pubsub, it has service discovery and consensus mechanism. All that is part of the standard library and available to the developer through simple elegant abstractions. I use Go at work and achieving all the functionality that the erlang VM has out of the box would require a monumental effort... raft, consul, kafka, etc. Is too much to explain in a short UA-cam comment.
If you were talking about renting VMs with ECS I'd agree but we're talking about lambda here. That's more like deciding to live in a restaurant. Heyyy you want a cherry on that sundae, just click your fingers and order it. Missing sprinkles? Just keep ordering. That's a company card you're using right? Don't worry about the bill sir.
@@Sonsequence Spot on. I used to be a solutions architect for AWS and majority of our customers start their footprint with one service to solve some critical business problem and then end up having most of their infrastructure on the cloud utilizing services they don't even need. We encourage our customers to adopt more and more services, exactly like your resturant analogy. and then when spending becomes a problem but they still want to use more services we issue EDP credits, free Proof of concept, etc.
Python has OS-level threads, but most of the python implementations (iron python and jython excepted) only ever let one thread run at a time. They actually end up _slower_ on multicore machines the moment you introduce threading because of lock contention issues. That said, it does have first class support for green threads at this point, in the javascript async/await style. And it has had promise-like async networking since 1999 or there abouts. You are also absolutely correct that there's a heavy expectation that you just know the libraries to do stuff. "import antigravity"
...but that doesn't really matter as Python(cPython) doesn't at all support parrallelism....concurrency : yes.....parrallelism that would actually make that support for concurrency useful : no.
@@thecollector6746 That's just about as untrue as it meaningfully can be. First, the restriction on running threads only applies to threads holding the GIL. Numpy, OpenCV, and most libraries written on top of PyBind11 or similar don't hold the GIL while doing heavy computation. You can even run CUDA (or OpenCL) kernels fully async, without blocking the GIL. Second, python does support full parallelism as well as most javascript implementations do (WebWorkers), the restriction is data must be serializable and can only be shared by explicit message passing. It actually is ahead of javascript in that you can share sockets between peers, so if you want to handle orthogonal connections through the same listen port, it's trivial to do so. (Also, the subinterpreter system will let this resource sharing work within the same process, but that's a little ways off still). Third, concurrency is incredibly useful _without_ full parallelism. That's why python grew its concurrency model before multi-core CPUs existed outside enterprise environments. It lets you work on computations while waiting on data to load from spinning rust (or tapes). It lets you multiplex connections without needing to duplicate your stack per request. And it makes it easier for the programmer to reason about certain classes of problems.
@@yellingintothewind Except for the fact that I am 100% correct. Cry harder. You wrote this wall of bullshit just to say that I am right, but you didn't like what I said.
He completely misses the initial point - THE CLOUD PROVIDERS don't care about performance at all, because YOU get charged for performance, so THEY don't optimize anything. And it's in their interest that YOU ALSO don't care about performance so they get all the moneys. The worse their tech runs, the more you pay. The worse your tech runs, the more you pay. The more you fail to configure something in their preferred convoluted way, the more you pay. The more of their products you use, the more you pay. The harder it is to use somebody else's service with their service, the more services from them you use, the more you pay. Also, you don't need async anything to have concurrent events in any language or system. You need: procedures, stack frames and queues - that's it, you can build an event system with those. Ask game programmers.
@@Bokto1 Yeah, the only optimizations they benefit from is on hardware power consumption and cooling versus numbers on the dashboard. Since you're paying for CPU time and cores, they might as well move your code to 256 core arm chips running at 20w. Which is (or might be) good for the environment, but not for your pocket, when you need 50 cores to do the same job a quadcore system used to do.
He didn't miss it, it's just a child's understanding of the incentives involved. If you can't also list some incentives AWS has to care about performance you have no real understanding of their business.
This also applies for alot of PaaS offerings, such as Azure Logic Apps. You wanna pay 10x the money for every CPU-cycle? Look no further than Azure Logic Apps.
Love this take. @15:56 Easiest way to do this is build the entire API in a lambda docker container, then transition to ECS Fargate once the service has a somewhat consistent load. Req -> Res cycle is faster too because of no cold starts.
@@GackFinderCan you, or one of your upvoters, elaborate? I understood @WillDelish point about services running locally in containers makes local testing easier, but I’m not following how “pressing F5” solves the use case he’s referring to.
@@utubes720 troubleshooting lambdas can be a huge pain if you don’t know if its your code or infra. Some tools like SAM help, but can be slow. Being able to use docker to chain your stuff together & locally test = cool beans, now its only infra problems. Also if you’re wanting to use like rust or latest python etc, lambda might not have a runtime for you, but you can build a custom image and use whatever you want.
Honestly worrying about "infra" when you're a startup is an oxymoron because a $5 VPS can very easily handle like 10-100k users depending on what the app does so long as it's in a moderately performant language like Go, C#, Java. I think the "need" for this is caused by using languages that just simply shouldn't ever be used on the backend for apps that plan on growing massively like JS/TS.
And you don't need to hire a full time infra admin... You can work with a company and pay only for managing / deploying the infra... it's litteraly my job, and the company pay far less with paying me deploying baremetal, that paying the same service on aws... With services running far better, with a better control on their data and system... But a lot of company prefer to pay aws, because you know "the cloud is the future", "baremetal is so old school"...
Ah, but if you make a load of bad design decisions in pursuit of "being scalable", then the majority of dev time needs to be spent on infra, which then justifies itself due to the horrendous performance of the system even as a startup with well under 1k users.
@@gto433it's not any one particular thing, it's just the general way the GC functions, how easy it is to accidentally make an object long-lived, the general ideology of copying memory as much as possible (I.e people using spread syntax when they could just append to an existing array), and the single threaded nature of node limiting the max concurrent requests too much. Don't get me wrong, it's cool that we can use the same language for front-end and back-end, but most node backend apps could have been written in golang with the same velocity while gaining a bunch of performance for free.
Ruby has fibers (green threads) natively. It also has Ractors (actor-model parallelism for CPU-bound operations) but they're not very performant yet and they're working on it. Ruby has also recently improved massively in terms of speed, but I guess it's past its prime, and there are still faster languages.
Serverless is great, we needed to extract some data from a PDF file that the php library we normally use couldn't get so we made a tiny gcloud function in python just to use a completely different pdf library that we can call from a http request.
What boils my piss the most is the fact that thanks to AWS everyone assumes you're talking serverless when you mention lambdas, when in fact lambdas have been about anonymous functions for ages, and I have no idea why the fuck did Amazon think it was appropriate to steal that name for their bullshit. Thank god at least buckets don't remind most people about object storage & shit.
lambda is meant to be event based. and it allows for fine grained control over how many events get processed per call. and so you can process multipe events at a time. most people however use it with api gateway where its always 1 event at a time. and thats just not what it is built for
Given that it is event driven how can there be more than one event processed at a time. I believe that you can send a large payload to a lambda to be processed all at once like a kind of batch processing but it's still just the one event
@@brokensythe when you e.g. feed events in from SQS, you can configure the amount of events to be sent together. The only place where you cant do that is through api gateway. for obvious reasons
GCP made a good move with second generation of cloud functions (analogue of AWS Lambda). Every gen2 cloud function is now capable of serving multiple concurrent requests per instance. This way compute resources (vCPU and RAM) get utilised in more efficient way and service still abstracts the complexity of scaling and maintaining of server infra. In the end of the day you basically pay more money not to manage the servers which can be beneficial for some small to medium products in some situations.
Yeah and then you've got K8s as an option down the road, once your costs justify an infra team and tooling team, plus there is now a lot of great kubernetes focused tooling available.
Python’s threading is more complicated because of the global interpreter lock. Basically, Python’s threading is real, but only kind of. To run a line of Python, you have to acquire the global interpreter lock. If you call a compute-heavy C function, that function can release the GIL while it runs, but Python must then re-acquire it to go back up the call stack and continue running. If most of your compute is happening in Python-land, threading is an illusion, because all threads are fighting for the same lock. If most of your compute is happening in C, it has a chance of allowing threads to work. Most I/O calls take advantage of this and allow multiple threads for example, due to their speed
@@IvanRandomDude What they did in the last version was removing GIL from multiply instances of python inside one process. This is a baby step in this direction. Example when this happened before: Pytroch when you used the fork library under windows effectively both multi process and multi threading were an illusion (and still is because they have to update to the new version yet)
most I/O function reading files, reading network ... is totally done on C land, GIL is released on those, so thats perfeclty multithreaded, so you can def. process multiple requests.
Regarding Python/Django: Running it with the dev-tools will by default be a single process that can handle a single request at a time. For production you commonly use a wsgi middleware instead (uwsgi, gunicorn, etc.) which will allow for enabling multiple processes and threads etc. Celery has nothing to do with that. Celery is a background task queue and needs its own processes for that.
@@thekwoka4707 depends on how you mean "multiple instances". What I did not mention yet is that Django is now getting decent async support, so there it would handle multiple requests in a single instance I guess. Although it might depends where exactly how you define "instance". But aparr from that ersonally I think I would even consider uwsgi with multithreading enabled to be "multiple requests per instance", at least from a sysadmin perspective.
Yeah the python gripe is old news. It used to be that your only option to utilize you CPU fully was choosing the right size of thread pool for your workload and I/O latency. In comparison, an event loop can automatically make full use of your compute no matter how the workload varies. Comes with some downside. If you're being a "proper engineer" you're forced to write more complex async code and having lots of threads is good insurance against outlier heavy requests blocking your whole server. But nowadays you can use async with FastAPI or if you love Django there's gevent so you'll use some other libs to monkeypatch your I/O and database connection pool and then you can write an async web server in the simple style of a synchronous one. Sounds dodgy, turns out to be trouble free.
Ruby servers have been threaded for almost a decade (something like that) but no one really writes threaded code in an API call (obviously there are exceptions, but generally).
JS devs just can't get their heads round the concept of one thread per request and that your whole app runs for every single request. They're used to dealing with multiple events in their code and their code being resident in memory indefinitely, spinning the event queue (like in a webpage). Which is fine, except when you do scale to multiple threads with each thread having multiple events - then you're in for a headache.
@@pdougall1 Yeah, so cue the discussions about shared thread memory (with the security concerns) and thread synchronization, both definitely needed now that the long running program is handling a session that might reside on 3 different threads/processors/machines. This never used to be a problem (the server software just passed very basic data structures about the request to all program instances when invoking them). It used to be simple, understandable and effective, CGI was and still is a workhorse.
As a person that has only ever programmed video games and generally has no clue how real software works I do always find it interesting when people talk about handling events or requests that are only in the triple or quadruple digits. Yet at the same time these are for infrastructures that handle millions of active users daily. It really shows just how vast the realm of software development can be and how massively different project requirements are.
As a game developer, your programs are doing far more computing than most devs using AWS. They're doing stupid amounts of serialization and network transfer for multi-second HTTP requests, while you're computing physics or game logic on 10s to thousands of objects 30-60 times per second. Or maybe the game engine is, but my point is that your application is actually doing something that's compute intensive, not passing trivial requests with layers of extra shit in-between like most web apps. Learn the networking stack and HTTP and you too can start passing trivial HTTP requests through an excessive number of layers. Or you will see the value in other protocols, like web sockets, and then you can write browser games with multiplayer.
@@david0aloha I mean, potato / potahto. Most of the extra crap those truly massive companies have to deal with is just to help with problem of how big they are. On the other hand those tiny companies that want to pretend they are google could probably learn a thing or two from your comment! As for me - I won't lie. I'm a dogshit programmer. If I need speed then I find clever smoke n mirror tricks to simply hide the fact that I stop doing any work at all. If I can't hide it then I eschew all of the modern amenities of new programming languages just for the sake of cache coherency and avoiding heap allocations in a managed language. If I still need more I reach for the hammers that are my preferred threading and SIMD libraries. Or if I can get away with it, I dump it all on the gpu. Either way I've never solved a problem by actually writing a smarter algorithm. *Every* single algorithm I've ever written in twenty years has always been a for-loop or a nested for-loop.
His love for Node concurrency is weird. Node wasn't unique in its concurrency in any way. The small BEAM VM would scale pretty well on old hardware with a small footprint. I had an Fortune 500 enterprise wide app handling tens of thousands of requests a second running on an Erlang setup on a single Pentium 4 workstation 10+ years ago with almost zero effort put into scaling. Node has enabled concurrency on a midtier, more common language but it isn't great at it.
If you write your code in a way that's tightly coupled to a particular serverless architecture, then you need to be very careful to pay down that tech debt as soon as you experience significant traffic. I don't think using serverless is necessarily a bad idea if you keep in mind that at some point you might need to shift those request handlers into a new infrastructure.
I totally agree with you on most serverless platforms selling lies and I love servers as modern programming languages like Go, Rust and others are very good with async and can effectively handle 1000's of requests per second on a $5/month nodes. What's your take on Cloudflare workers and WASM. Although they got limits in their use cases but I think they might make a better serverless model to run either as a external platform or as an internal platform.
As a cloud software engineer(7 years exp + degree), I’ve seen large enterprise systems built in pure serverless using IAC. The bills are laughably, comically small. The trick is designing with event-driven processes in mind. It’s a juggling act, but a decent SA can design a system to substantially keep the bills low. Understand the services you are using and stay away from the shiny stuff. Api gateway, lambda, dynamodb are your legendary mains. Eventbridge, sqs, rds, are epics worth picking up. EC2’s absolutely do not pick up; you may get lucky every once in a while, but you’ll more likely get killed trying to look fancy. If infrastructure is built using something like terraform, you can --somewhat- migrate serverless from AWS to GCP, however TF truly shines if you deploy elements of both (plus others) in the same codebase.
@@mixmastermaverick3336 consider your use case and compare the workload of requests to serverless resources vs an ec2 setup as you scale. Keep it tight.
I mean if your business plan is to milk all the money out of their clients..they won't have any clients after a while... Unchecked capitalism will eat itself alive. Companies do not learn...because they aren't people. They are legal contracts to protect the actual individuals from actual liability from their reckless behavior in pursuit of all that glitters and shines
You can only trust a company so much once it achieves market dominance. If they're declining from that spot, as AWS is being pushed by Azure, then they have incentives to play nice and act innocent.
It's pretty much about (once again) people buying into hype without understanding numbers. Serverless/Cloud is amazing in two scenarios. You have really low load or you have 'near infinite' load. The former will cost you next to nothing, the later will be prohibitively expensive to build infrastructure for. Thus, both make sense. It's all the middle ground in between those that might not.
so, basically spikey traffic. but I agree with prime in that you have to write your code in such a way that it is easily transferrable to a docker container at least
It always hit me how different the IT space in(central) Europe is compare to the US . I mean we have a lot of small and medium sized companies while in the US the big ones are dominant . Which means a lot more languages are used but also companies make the webpages for other companies . I mean I know one and they host for their clients when they can on their own server. By the way I read today they looking for Golang Devs D. There are also less .NET fear because a lot of companies are already thanks to Microsoft Office depended on Windows so a lot of system integration and inter connectivity works .NET so far you aren't a bank.
@@shadamethyst1258 The problems with C# are only that you need to use Visual Studio for the UI creation and that is strongly typed the rest is just bias. I programming currently in C++ and C# is way better: Where I meant were it is used is in interfacing with ABB robots, controls system units but the same time with logistics and such stuff. Actually it good it isn't so hyped. When Java got hyped it got a Jenga Tower of Frameworks on top of each other. JS got React *hust*.
You still have to worry about efficiency with serverless / lambda. I've spent a lot of time reducing execution time of lambdas. that's not to say I always use it though, it's still pretty easy to build and deploy a containerized app on ECS in AWS too. Use spot instances and you pay so little.
Blaming the entire serverless paradigm just because people don't understand how to actually use it it's plain dumb imho. Should we say that object oriented programming is dumb just because people started abusing interfaces and inheritance? As everything in IT if you don't know how to do stuff you will probably end up making a mess. Lambdas are not the answer to every problem pros and cons as everything else in life folks. Btw at the end of the day AWS still wins because they also sell raw compute resources so to say that they are sabotaging an entire industry feels too much to be honest.
That's one thing that annoy me as well, when people are comparing Node to PHP and then say PHP can only do single threaded. Yes, but Nginx is multithreaded. You don't use PHP on its own.
@@evergreen- You just return data content then two new lines and there's an event sent to client, and you'd need to ob_flush, all encapsulated away, similar to every thing else in http. And streaming the constant response is the same as any other streaming e.g. files. You could even mount your whole app under its framework as a daemon/container listening independently and load balance separately for better performance as you'd need to with most other languages (you could use PHP mostly on its own in other words).
@@evergreen- I did write a chat app with SSE in PHP this year, it was a piece of cake. Very easy. Hard to find documentation, though. But you can also use a network socket with php, python or whatever. Socket server libraries have been around for decades.
I'm actually converting away from serverless because I simply designed my serverless infra badly and I can design a monoservice REST API easier and more sustainably. It's likely going to cost me more at my app's size, but I'd rather have something sustainable since I'm a solo dev.
I’ve heard good things about Clojure performance wise as alternative to popular dynamic languages on the server side. It’s not amazing but people say is quicker to write than Java or Go.
We used aws lambda for image transcode. Was nice since if we didn't have images to transcode it was a lot less than a server and we could handle bursts of work within seconds. Was nice to not keep that much compute around.
the Python runner for a production server is gunicorn which handles a thread pool for you. So, yes, django multithread requests if you deploy it correctly. It's just that if you run a dev server in production, it will not multithread by default. but that would be dumb. Nobody does that. It takes 30sec on internet to know you need gunicorn and it is very easy to set up
The commenter talking about Celery is kinda misleading. Celery is like the sidekiq equivalent in Ruby land, it's for async/background job processing. Celery workers typically live on a completely separate machine and receive tasks through some sort of broker like Rabbit-MQ / Redis. When you run a Python web service in prod with Flask/Django you typically achieve per-request concurrency by using something like Gunicorn or uWSGI and pointing these at your app. These are pre-fork worker HTTP servers, so each worker essentially takes your application code and runs its own version of Django/Flask within the process. I'm not a NodeJS person, but it seems similar to the way you'd deploy an express app with PM2
From the Microsoft side, net gets big performance increases every year, including AOT complication, which they show being used on Azure Functions, so that you don't have to pay the JIT cost. Of course they do this because they want more people to use Azure Functions, and serverless is more expensive than other forms of hosting, but they don't intentionally hold back performance just to increase compute time. You can also host Azure functions inside k8s now with keda to scale to 0, which is more cost effective, but then you have to know more about k8s, although there are Azure container apps for that now too. The bigger you get, the more it becomes worth it to spend dev time to reduce operational costs. Docker containers are easy enough to start with to make serverless pretty pointless anyway though, and there are a lot of different ways to host them easily.
Thats why PHP still 80% of internet , PHP is stateless so you send a request to the server , the response will be a process that will die after finish his job such as HTTP protocol thats why PHP is the secret king 👑 of APIs and SaaS
@@h.s3187I don't believe the process actually dies after the request in PHP-FPM or most PHP servers, but all your variables are garbaged at the end of the request and not shared with whatever's done for the next request. So it's as if the process died but without all the overhead of starting a new process.
As to your opinion on Python I have always referred to this as "Do you want to know, or do you want to learn?" People who want to know something identify a skill or ability that it would be convenient or fun to have. "I would like to know French. I would like to know how to play guitar." Wanting to learn is the act of saying to yourself "I am going to suck at this, and I am going to put time and effort in, and eventually I will suck less. The more time and effort, the less suckage."
in serverless performance doesn't matter? cold start time is influenced by bundle size, and you're getting charged by the ms... compare that to containers which provide similar (but more effortful) abstractions, and you can just as easily "let's just wastefully scale up", and confusing performance issues related with "ongoing" process as well as "per-request" processes... the unit of measure for perf gains there feels like "number of pods" edge compute isn't about user proximity... it's about more efficient v8 reducing some of that "wasted time" you attributed serverless to earlier.... who's measuring the wasted compute on a pod? if you can't scale as much i dunnno... this has vibes of "serverless is making money by making things more convenient for devs", even "managed databases" are more "serverless" aren't they? not lambda, but serverless in terms of pricing model and mental model that you "think about the server less"
I think word serverless is also often used to exclusive talk about FAAS services, while it is not limited to them. For example in AWS SQS is serverless, DynamoDB is serverless, ECS Fargate is serverless. For example we have recently migrated from having HTTP served by lambdas to server running in ECS with Fargate launch type. Of course it is more expensive than running it in EC2 (I think it can be like 2x difference when reserving instances), but still better than lambdas, both cost and performance wise. They are many kinds of serverless services out there and people really should start noticing more of them.
“Design by Twitter choices” is what I have witnessed so far in my career and I it’s so frustrating. Management doesn’t care to try out solutions and pick the best one. Never seen a flame graph outside a YT video.
I love how developers are now realizing this and saying oh boy yeah that's bad, we shouldn't use aws lambda/az functions... and then immediately turn around and happily use the pay-per-compute-second k8s clusters they host in AWS/AZ. Literally the same thing, except now you're paying for machine CPU/RAM/DISK as well... you know the thing we finally escaped when we went serverless with lambda/functions... and it's all obscured behind the k8s resources syntax...
Been hit with the max lambda limit before, theres a max amount of lambdas you can create and a different amount for how many can be called and yet another amount for how many can be called concurrently. These are restricted by aws and you have to ask them to bump them up for you ($$$). This happened to me when working for a health company, covid hit and we were swarmed with requests. The queued lambdas were causing all kinds of problems, events were being dropped db connections exceeded max number which caused even more issues. It completely broke our system. A system that is very carefully tested and maintained yet we could not even properly test a scenario like this :(
What were you doing in the Lambda's? If they're being an SQS queue then it's pretty trivial. If the Lambda's can't keep up then the queues will just accumulate but you won't lose anything.
I believe one of the most overlooked benefits of serverless technology is its ability to digitalize processes in non-technical industries. Many people think of AWS primarily as a tool for developers, but the reality is that most industries are not tech-centric, yet they still require technical solutions. In these companies, many processes are manually executed, entire databases are maintained by hand, and poor communication due to bureaucracy or other factors is common. This scenario makes monitoring, tracking, and deploying services almost impossible. Transforming company culture, training employees, and addressing other challenges requires significant time, money, and effort. Using AWS infrastructure simplifies explaining the benefits, monitoring costs, and collaborating with various teams. While many solutions may not be optimal from a technical perspective, especially in large, non-tech companies, serverless technology makes it much easier to create sustainable and flexible solutions.
At my last job I had a debate with a senior dev and the CTO about performance. They were adamant about the fact that today we focus on readability over performance and that modern computers are so powerful that our crappy node backend was just fine. I found it crazy from an engineering and craftsmanship point of view.
9:00: With sanic (another name you'll "love") I can take, in python, more requests per second than nodejs to do the same work as in nodejs, as long as there's 3 or more async/await in the javascript sequence.
The greatest trick Jeff Bezos ever pulled was tricking developers into thinking serialisation and network latency are trivial, and that memory and compute are expensive
K8s + knative. Local dev better than lambda having to constantly push to AWS. Multi cloud. Serverless. Sure more cost up front, but better imho for enterprise. Smaller dev shops, sure lambda helps.
@20:30 "Maybe Elixir is something to learn." Yes and no. You should learn actor based concurrency. And probably the best, generally available implementations of actor based concurrency is Erlang (and therefore Elixir). I really believe every dev should learn Erlang. Not because they should use it on all their projects but because the programming model is truly different and can open one's eyes to different paradigms. Similar to how all devs should do some serious functional development at some point. That said... learn Erlang over Elixir first. It is a simpler, more concise language and all the important things about Elixir are based or straight on top of the core Erlang and BEAM VM infra. Elixir may have some nice syntactical sugar but that isn't the important stuff and IMO can be a distraction.
ah, the pay by second actually reminds me of making a mistake when developing a C program for the mainframe back in 2010 or so - the time on the IBM Mainframe was paid by second & you had some wild memory restrained, so if your program ran for a few minutes instead of 20s than that was a problem. This just reminded me how wild I thought the restrains and the prices were back then & now we're again paying wild prices for cloud services.
3:44 Node doesn't rely on stop-the-world green threads. It actually utilizes multiple threads via the event loop. 8:00 YES! People don't understand the thread-thrashing it causes. You SHOULD NOT parallel async _within_ a request, generally speaking (promise.all is a no-no).
Default Python everybody uses is single-threaded. Django and Flask web apps achieve concurrent request processing via external tools like gunicorn to run basically N instances of app server. However, Python now has full async/await support baked into language. FastAPI app server leverages that, and you can write apps in Node-style, just be careful and don’t accidentally use sync APIs
The AWS lock-in is wild. You can end up in situations where you are so dependent on their services to have your product up and running and scaling somewhat okay, that you can never really leave without insanely high cost. Also warm tip: keep track of your own usage with a third party service and monitor that well.
Python isn't single threaded by its limited by a global lock. Its supported async await for ages now. and has plenty of ways to get things into a thread (Futures, Threads, gevent, celery, and so on.)
Async/await runs on a single core. Python threads run one at a time, not all at once. You need some hacky library or to spawn multiple processes that talk with each other over the network
Serverless isn't the issue. There are other serverless technologies that you can self-host that are awesome. It is the misuse and lack of understanding of the AWS pricing model which is a bit of a pain to deal with.
Using AWS lambda like a web server seems crazy to me. I’ve only used it for background processes that get triggered by a queue or like a daily/hourly schedule. And I use step functions map iterator to launch them concurrently instead of having loops and things inside a single lambda
Having Lambda serverless components in your startup codebase is like have a chandelier in your one bed apartment. - more of an expensive art installation than appropriate decor.
Python actually is single threaded because of GIL.. so basically concurrency exists, but not parallelization, which is why people end up with multi processing, so each process takes up one core in multii-core env and attain actual parallelization. 😊
Clearly, these guys have no idea how to build aws serverless infrastructure 😮. 1. 1 request does NOT mean 1 new lambda. lambda stays idle for some time and it can receive more request while its still alive. for example, if you have less than thousands of request per sec, you are just using 1 lambda. 2. you should use some combination of SQS, SNS with lambda calls to handle request that does not require to respond immediately. there are huge chunk of contexts that they are not aware of.
Just going to bomb in with a little info about how a typical Ruby/Rails app works these days. It's somewhat like Node. You run Puma, an app server which runs a set of threads and manages a global interpreter lock for those threads to have access to Ruby code. Only one thread can run your Ruby code at once, but most of the time a thread is outside of the Ruby, doing IO of some sort. Yeah, this is a hack around the fact that Ruby is 20 years old, and wasn't originally designed with parallelism in mind. But it achieves good enough performance for all sorts of normal workloads, it's a reasonable tool for many jobs. On a relatively small instance, AWS will give Puma a default setup of 2 processes with 32 threads each. If your task is IO bound you can get quite a lot of grunt out of that instance.
As an infrastucture-as-code engineer I am in shambles. Just kidding, I prefer to run EC2 Linux, S3 and cloudfront, most of the rest is trash. Aurora is fine, ECS and Fargate are okay.
1000 free Lambdas. I once paid like $5 for some Ukrainian/Canadian access attempt chain. My script added trailing slashes. The attacker tried every Wordpress vulnerability known to man. It was pretty massive. They tried several times. Sometimes from both locations at the same time.
Infra probably doesn't need to cost so much. I mean, you can setup the server in your local machine/server, use a bit of firewall, have a systemd service renew the letsencrypt certificates, or maybe use Route53, and then connect an ec2 instance and local machine via zerotier and socat .Maybe add some nginix and SNI to the mix. And that's mostly about it. The things that need taken care of would be, system upgrades, database management, local machine failing, network goes off. But does that really need this much amount of billing? People can do so much with just one fat machine. Not saying people should do it on their own, but it shouldn't cost this amount.
Going back and forth between what comes out of prime's mouth and what comes out of theo's mouth is giving me a real bamboozle of two completely different philosophies
Some facts, cause homeboy is lost on this one, you can write C extensions in Ruby, meaning there's potential of making it fast, Python does have async and await (he corrected after) but homie here is like ChatGPT today...
I worked with Lambda and with standard containers and I think lambda is the greatest piece of shit ever. From deployment headaches to cost per ms, it's all pretty weak in comparison to a nicely built microservice and the only scenario that actually makes sense is a service that gets little traffic and long waits between calls or maybe some asynchronous work in conjunction with SQS.
You can dip that nodejs into c also... `npm install -g node-gyp` I built a node app that calls windows apis to send keystrokes and mouse movements to run a flash game for me once. The image processing was too slow doing the screen capture and looking at the bitmap in code so I I installed CUDA and figured out how to offload that to the GPU.
it's a trade off between how fast do you want to ship the product vs how much it would cost to ship it that fast. If you don't want to spend that money on infra, then you optimize your infra.
because the cloud is the future, old vm, or even worse, baremetal, are old school and for loosers... they are cheaper, reliable, but you know, you have to use the "last tech", it's so much better...
It's not actually that hard to move stuff out of cloud. It's when large companies put stuff on cloud the costs start counting, but tbh - even then quite often it's not really that expensive compared to development cost. If your AWS bill is less than 1 million a year - probably it's not worth going off cloud just yet. There is absolutely a point to be made that at a certain scale - on prem can save money and even offer a better experience. This also goes for control - sometimes the service latency just becomes problematic if you're on cloud and there is a legitimate case for moving off the cloud. There is also an argument about compatibility - not everything works in the cloud. There are good reasons to be off the cloud, but the majority of startups will pay 0 or very little for AWS. Might as well use it. About serverless in particular though, it's debatable whether it's actually really properly supported and worth it. It's not as clear cut that it's "just better in every way".
4:43 usually no one run python server code in a single thread. 99% time you use WSGI or ASGI to handle the incoming traffic, just like you do with PHP (PHP-FPM, etc...)
20:16 He should admit he's wrong at this point 😂 Erlang (whose VM Elixir runs on) was made for concurrency with built-in fault tolerance. It was designed by a telecom company to process phone calls at scale. Of course it's performant 💧
AWS Lambda is actually perfect for small startups where software is basically a cost center. The costs are simply too low, and the scale is infinitesimal
More efficiency leads to less total computation, which leads to a lower bill. Maybe I just don't understand Lambda well enough to know why they're saying this doesn't motivate companies/programmers to write efficient software? AWS/Azure/GCP might WANT you to write inefficient software, but the software engineers who use those services are certainly motivated to write more efficient software. If anything, you could see the pay-for-compute model is a way of punishing companies that write inefficient software.
Serverless isn't just Lambdas, for some business ; not having to worry about servers (patching/security/updates/migrations/scale) is worth the price. Serverless is a great tech, but the result depends on how you use it. I can put a caching layer in front of a Lambda and compute once in a while without holding the server running, that's great. Lambda@Edge is also great, I can run my compute near the user, and don't need to have a server for each region. For "not investing in performance" bit, there are multiple cloud providers, competition is a drive for them to optimize perf.
yes, all depend on the use case... For every tech... You don't need a k8s cluster everywhere, and you don't need serverless everywhere... But, in some situation, those product are very very interesting and are a really good choice...
Node is asynchronous, but NOT concurrent! Concurrent and parallel are effectively the same principle. Both are related to tasks being executed simultaneously. Asynchronous methods aren't directly related to the previous two concepts, asynchrony is used to present the impression of concurrent or parallel tasking.
AWS's own training materials from a few years ago (I'm not sure how old this course was) provide the following use case for Lambda: a function that will update product database when an employee uploads a spreadsheet, to be used once in a while.
Rather far from running an entire application this way.
I do not understand why people are trying to replicate monolith applications in lambda.
@@disguysn SERVERLESS
(Business loves this word)
Lambdas should never be in that hot path …
I have ran startups with over 100 million $ valuiation on aws lambdas, and our aws bill was under $20.
@@-Jason-L need more words for this to add up
Technically, going from 0 users to 1 user is an infinite growth rate. VCs will cream their pants over infinite growth.
Since the relative growth rate formula requires dividing by the previous value and division by zero is undefined, the relative growth rate from 0 to 1 is undefined. However, I doubt VCs can do math, so the "infinite growth rate" strat may work.
@@lylyscuir unless you evaluate the limit instead of doing the division.
Based af
Just use js
node -e "console.log((1 - 0) /
0)"
@@MrTerribleLieExcept that limit doesn't exist
Former Amazonian here. Within the company people just throw code into lambdas, even if each transaction takes multiple seconds (like 7+ seconds). "But it scales, bro"
This doesn't even take into account the added complexity of having to add extra layers to support statically linked libraries that aren't included in the lambda stack.
Speaking as someone that grew up on a computer with 4K of RAM I can absolutely agree with this. Because everything - compilers, code development, pipelines, hardware all got faster everyone stopped caring about quality of code. It became focused on “lines of code” measurement instead of the efficiency. Very happy to see some still value it.
The business is expecting it to be done by Friday not it should have latency less than 200ms
@@ivonakis Those 200ms add up
@@tadghhenry technically inept management doesn't get this and devs just roll over and accept the status quo
4k ? Good times, I was on 1 mb Ram at the beginning, working the whole summer to get that beautiful Amiga 600 Ram edition
@@tadghhenrydepends what the software is doing. Sometimes it matters but many times it does not. For some apps having a dev spend a couple of weeks optimising it costs more than you will save. For others performance is critical.
I work with a system that is mostly serverless (aws lambda) right now, and my take is that it actually forces us to hyperoptimize every function because of how much your cost is directly related to performance. 50ms extra latency per request when using containers? Probably not a huge issue. But 50ms per request when you are literally paying for every extra ms of runtime? Its a huge cost implication if you are dealing with millions of requests per hour. Also, memory leaks do matter despite a lot of contrary info out there, since lambdas that stay "hot" will share memory. We actually had a slow memory leak take down one of our lambdas in production.
Also no matter how much you optimized there is a inherent perfomance hit that can't be optimized away by you and your paying for their inefficiencies.
@@farrongoth6712 I agree. It definitely limits you on options to optimize too, since you have to treat every request as an individual execution. There is some fancy stuff you can do with layers/extensions, but its still limited.
I don't think serverless functions are bad, in fact I think they're a great tool. But, like everything, I think too many people are treating it as a silver bullet. There are definitely use cases for them, but I'm currently living the pain of going all serverless.
what is the reason to run this as lambda and not containsers? Seems like the amount of time you invested into making your code run cheaper could have been spent on setting up a k8s cluster, even with autoscaling if your traffic is variable
@@Qrzychu92with containers occasionally nodes can go down but lambda is very rare to fail
@@Qrzychu92 one example where I work is an overnight batch processing job. We have 16 hours every day where we need 0 instances and latency/startup is not a huge factor as it is not a customer facing service. Lambda makes it easy to simply maintain the python script used to process the data. No need to manage a kubernetes cluster just to run a script. With lambda we get the whole runtime provided for us. And devops is basically nonexistent because it just works. Assuming containers are always a better option is the equivalent of assuming serverless is always a better option. They're just two tools with two different sets of tradeoffs.
Thing is: 80% of ppl in the industry works for small medium companies where perfomance is not that important due the low number of concurrent users. They only start to think about perfomance when its too late: too much users onboarded and things go south. Some companies will never get to this level btw.
Never optimise too early. And if it's too late, you already made it.
Yes but we do need microservices in everything because it's the future! Everything needs to be infinitely scalable even if you have 0 users.
Indeed. I started my career long before the cloud and no-one cared about performance even back then. It was long forgotten after we got 8MB RAM.
My little company qualifies absolutely. We're so barebones and basic that the value of cloudy services that automate our workloads just completely stomps the cost of a few hundred lambda calls every other day.
most. after a few years most companies won't need any infrastructure at all!
We are literally 2 devs. One Backend and one Fullstack. We just used simply a docker swarm, costed him about two weeks to learn and setup. Runs perfect and can easily scale up and down.
I paid $900 for an i7 Dell recently. Tons of power. It would cost at least $900 per month to have the same power on AWS. And I pay $80 per month for a 1GB fiber connection. You'd think people would be more concerned with costs if they're paying for AWS. Companies love to throw away money on everything but salaries.
I couldn't agree more. I have been leading migration of applications from these expensive cloud providers and CEOs get surprised at the cost saved. It is the economic downturn that had led some companies to realize how they have been taken advantage of.
It really depends on what your infrastructure requirements are. If high availability, local and regional accessibility, multiple redundant services, etc. are a must, services like AWS are actually quite cheap and affordable. AWS, for example, claims to offer 11 9's worth of availability, which would be monstrously expensive for a smaller or intermediate company to achieve.
If we're analysing your setup, it's a single server with a single point of failure on the network. You might be able to safely run some web servers and a database on that, which is completely acceptable if your tolerance for downtime is around 2-7 days a year and you have your backups in order.
But imagine adding redundant internet gateways, redundant ISP providers, electrical redundancy, redundant distribution and access switches, multiple hypervisors configured for HA, etc. The costs would easily run hundreds of thousands, even millions, and because of warranty issues these costs would be recurring every 5-10 years.
Cloud providers are clearly a poor choice for smaller business entities and single developers, but claiming that using them is 'throwing away money' is quite asinine.
You're not buying servers. You're buying SLAs.
@@connoisseurofcookies2047 All spot on... I love that you put numbers on some of the risks being accepted. I once worked for a company that sold 5 9s when the telco only promised 3 9s. Our customents management didn't know what it meant... our management didn't know what it meant. It have a few hours unscheduled downtime each month - no-one got sued.
A lot of management get high on their own hype and sense of importance "2-7 days downtime? we have to be 5 9s" do you? do you really? do you know how much that would cost if we actually did that?
Even Office365 being Office 363, no-one cares.
BUT most s/w people don't understand that MTBF is "within the warranty pperiod" and believe their SSDs will be found fully working by archeologists - so let's not beat up on management too much.
Paying for the whole buffet when you just want a bit of salad is foolish though!
@@connoisseurofcookies2047 It's throwing away money exactly because of what you just wrote: small businesses do not need everything you just listed.
So many companies use Azure nowadays and ALL their service objects are on the same region and very little of what you listed is actively managed and encouraged. The WARNINGS section of configuration settings for each object screams with red and yet nobody cares. The costs keep racking up by 1k a month and nobody cares because perception is king.
Having two i7 machines with two SSDs in raid each and a simple UPS would easily cover 90% of users on Azure and would nearly never go out. By contrast last year in East US Azure went out 8 times - wouldn't be a problem if you had failover; if.
Newsflash: most companies dont need to worry about webscale. I worked for startup that did $10 million a year, and our aws serverless bill was under $20 per month. We scaled to 3 states, health services industry. We could scale to the entire US and not break a few hundred $. That would have been close to 200 million ARR. That is unicorn territory valuation. on a few hundred in infra costs.
Elixir, as a language is nothing out of the ordinary, it is the erlang VM that is insane, people learning elixir should focus on understanding the VM, if you only learn the language's syntax you wont understand why elixir gets all the praises it gets
Straight, no chaser, this is why Elixir matters.
Elixir is probably one of the best languages ever made
What is special about the Erlang VM?
@@majorhumbert676 “The Soul of Erlang and Elixir” by Sasa Juric is a great primer on what there is to love ua-cam.com/video/JvBT4XBdoUE/v-deo.htmlsi=tIpPXVbKgaqEIQjJ
@majorhumbert676 well, ill try to explain but is too much for a UA-cam comment, so as a summary is like a kubernetes in a box, it is like an OS, it has something like redis inside, is has pubsub, it has service discovery and consensus mechanism. All that is part of the standard library and available to the developer through simple elegant abstractions. I use Go at work and achieving all the functionality that the erlang VM has out of the box would require a monumental effort... raft, consul, kafka, etc. Is too much to explain in a short UA-cam comment.
The push to AWS, Azure, etc, is basically the same as choosing to be a renter vs a home owner. There’s pros and cons to each.
If you were talking about renting VMs with ECS I'd agree but we're talking about lambda here. That's more like deciding to live in a restaurant. Heyyy you want a cherry on that sundae, just click your fingers and order it. Missing sprinkles? Just keep ordering. That's a company card you're using right? Don't worry about the bill sir.
@@Sonsequence That is why scale economy increases, then decreases... 🤣🤣🤣
@@Sonsequence Spot on. I used to be a solutions architect for AWS and majority of our customers start their footprint with one service to solve some critical business problem and then end up having most of their infrastructure on the cloud utilizing services they don't even need. We encourage our customers to adopt more and more services, exactly like your resturant analogy. and then when spending becomes a problem but they still want to use more services we issue EDP credits, free Proof of concept, etc.
@KiraIRL wow that's a gem bit of insider knowledge
Python has OS-level threads, but most of the python implementations (iron python and jython excepted) only ever let one thread run at a time. They actually end up _slower_ on multicore machines the moment you introduce threading because of lock contention issues. That said, it does have first class support for green threads at this point, in the javascript async/await style. And it has had promise-like async networking since 1999 or there abouts.
You are also absolutely correct that there's a heavy expectation that you just know the libraries to do stuff. "import antigravity"
Never expected to see the Python import meme to show up here.
Yup, I had to stop watching after the python bit, it's rare to here so many incorrect things at once. Also, celery is not $&#&$@ threading.
...but that doesn't really matter as Python(cPython) doesn't at all support parrallelism....concurrency : yes.....parrallelism that would actually make that support for concurrency useful : no.
@@thecollector6746 That's just about as untrue as it meaningfully can be.
First, the restriction on running threads only applies to threads holding the GIL. Numpy, OpenCV, and most libraries written on top of PyBind11 or similar don't hold the GIL while doing heavy computation. You can even run CUDA (or OpenCL) kernels fully async, without blocking the GIL.
Second, python does support full parallelism as well as most javascript implementations do (WebWorkers), the restriction is data must be serializable and can only be shared by explicit message passing. It actually is ahead of javascript in that you can share sockets between peers, so if you want to handle orthogonal connections through the same listen port, it's trivial to do so. (Also, the subinterpreter system will let this resource sharing work within the same process, but that's a little ways off still).
Third, concurrency is incredibly useful _without_ full parallelism. That's why python grew its concurrency model before multi-core CPUs existed outside enterprise environments. It lets you work on computations while waiting on data to load from spinning rust (or tapes). It lets you multiplex connections without needing to duplicate your stack per request. And it makes it easier for the programmer to reason about certain classes of problems.
@@yellingintothewind Except for the fact that I am 100% correct. Cry harder. You wrote this wall of bullshit just to say that I am right, but you didn't like what I said.
He completely misses the initial point - THE CLOUD PROVIDERS don't care about performance at all, because YOU get charged for performance, so THEY don't optimize anything. And it's in their interest that YOU ALSO don't care about performance so they get all the moneys.
The worse their tech runs, the more you pay. The worse your tech runs, the more you pay. The more you fail to configure something in their preferred convoluted way, the more you pay. The more of their products you use, the more you pay. The harder it is to use somebody else's service with their service, the more services from them you use, the more you pay.
Also, you don't need async anything to have concurrent events in any language or system. You need: procedures, stack frames and queues - that's it, you can build an event system with those. Ask game programmers.
Yeah, it was strange that Prime totally missed it. Also: cloud providers are incetified to run their hardware as underclocked as possible.
@@Bokto1 Yeah, the only optimizations they benefit from is on hardware power consumption and cooling versus numbers on the dashboard.
Since you're paying for CPU time and cores, they might as well move your code to 256 core arm chips running at 20w. Which is (or might be) good for the environment, but not for your pocket, when you need 50 cores to do the same job a quadcore system used to do.
He didn't miss it, it's just a child's understanding of the incentives involved. If you can't also list some incentives AWS has to care about performance you have no real understanding of their business.
This also applies for alot of PaaS offerings, such as Azure Logic Apps. You wanna pay 10x the money for every CPU-cycle? Look no further than Azure Logic Apps.
@@rapzid3536 Why don't you list two such incentives?
Love this take. @15:56 Easiest way to do this is build the entire API in a lambda docker container, then transition to ECS Fargate once the service has a somewhat consistent load. Req -> Res cycle is faster too because of no cold starts.
Yep, this is the way. Docker / containers also make local dev a lot easier to test/mock
@@WillDelish Easier compared to what though? Because last time I checked, pressing F5 in an IDE wasn't that hard.
@@GackFinderCan you, or one of your upvoters, elaborate? I understood @WillDelish point about services running locally in containers makes local testing easier, but I’m not following how “pressing F5” solves the use case he’s referring to.
Lambdalith it’s a thing
@@utubes720 troubleshooting lambdas can be a huge pain if you don’t know if its your code or infra. Some tools like SAM help, but can be slow. Being able to use docker to chain your stuff together & locally test = cool beans, now its only infra problems. Also if you’re wanting to use like rust or latest python etc, lambda might not have a runtime for you, but you can build a custom image and use whatever you want.
Honestly worrying about "infra" when you're a startup is an oxymoron because a $5 VPS can very easily handle like 10-100k users depending on what the app does so long as it's in a moderately performant language like Go, C#, Java. I think the "need" for this is caused by using languages that just simply shouldn't ever be used on the backend for apps that plan on growing massively like JS/TS.
What syntax code in js can cause a huge tech debt?
And you don't need to hire a full time infra admin...
You can work with a company and pay only for managing / deploying the infra...
it's litteraly my job, and the company pay far less with paying me deploying baremetal, that paying the same service on aws... With services running far better, with a better control on their data and system...
But a lot of company prefer to pay aws, because you know "the cloud is the future", "baremetal is so old school"...
Ah, but if you make a load of bad design decisions in pursuit of "being scalable", then the majority of dev time needs to be spent on infra, which then justifies itself due to the horrendous performance of the system even as a startup with well under 1k users.
@@gto433 The language's performance is the bottleneck, not the syntax.
@@gto433it's not any one particular thing, it's just the general way the GC functions, how easy it is to accidentally make an object long-lived, the general ideology of copying memory as much as possible (I.e people using spread syntax when they could just append to an existing array), and the single threaded nature of node limiting the max concurrent requests too much.
Don't get me wrong, it's cool that we can use the same language for front-end and back-end, but most node backend apps could have been written in golang with the same velocity while gaining a bunch of performance for free.
Ruby has fibers (green threads) natively. It also has Ractors (actor-model parallelism for CPU-bound operations) but they're not very performant yet and they're working on it. Ruby has also recently improved massively in terms of speed, but I guess it's past its prime, and there are still faster languages.
ruby does not perform one request per process (actually one request per two processes), and it can be customized with async/await like rust
Serverless is great, we needed to extract some data from a PDF file that the php library we normally use couldn't get so we made a tiny gcloud function in python just to use a completely different pdf library that we can call from a http request.
Triggered by timer or http call, simple to work with, easy to lock down the scaling. Perfect tool for many small bits of work I do.
What boils my piss the most is the fact that thanks to AWS everyone assumes you're talking serverless when you mention lambdas, when in fact lambdas have been about anonymous functions for ages, and I have no idea why the fuck did Amazon think it was appropriate to steal that name for their bullshit. Thank god at least buckets don't remind most people about object storage & shit.
I mean just look at what they’ve done with open source projects. Amazon doesn’t give a fuck because they have the money to do what they want I guess.
Think of the name Alexa lmao
@@kneelesh48 facking kew
Buckets of shit? 🪣💩
Maybe Bitbucket == Shitbucket?
I, um, I will now be using the phrase “boils my piss” as opposed to “grinds my gears” 😂
lambda is meant to be event based. and it allows for fine grained control over how many events get processed per call. and so you can process multipe events at a time.
most people however use it with api gateway where its always 1 event at a time. and thats just not what it is built for
Given that it is event driven how can there be more than one event processed at a time. I believe that you can send a large payload to a lambda to be processed all at once like a kind of batch processing but it's still just the one event
@@brokensythe when you e.g. feed events in from SQS, you can configure the amount of events to be sent together. The only place where you cant do that is through api gateway. for obvious reasons
GCP made a good move with second generation of cloud functions (analogue of AWS Lambda). Every gen2 cloud function is now capable of serving multiple concurrent requests per instance. This way compute resources (vCPU and RAM) get utilised in more efficient way and service still abstracts the complexity of scaling and maintaining of server infra. In the end of the day you basically pay more money not to manage the servers which can be beneficial for some small to medium products in some situations.
Yeah and then you've got K8s as an option down the road, once your costs justify an infra team and tooling team, plus there is now a lot of great kubernetes focused tooling available.
Python’s threading is more complicated because of the global interpreter lock. Basically, Python’s threading is real, but only kind of. To run a line of Python, you have to acquire the global interpreter lock. If you call a compute-heavy C function, that function can release the GIL while it runs, but Python must then re-acquire it to go back up the call stack and continue running. If most of your compute is happening in Python-land, threading is an illusion, because all threads are fighting for the same lock. If most of your compute is happening in C, it has a chance of allowing threads to work. Most I/O calls take advantage of this and allow multiple threads for example, due to their speed
Isn't GIL fixed in the latest version?
@@IvanRandomDude What they did in the last version was removing GIL from multiply instances of python inside one process. This is a baby step in this direction. Example when this happened before: Pytroch when you used the fork library under windows effectively both multi process and multi threading were an illusion (and still is because they have to update to the new version yet)
most I/O function reading files, reading network ... is totally done on C land, GIL is released on those, so thats perfeclty multithreaded, so you can def. process multiple requests.
@@jordixboy exactly this. threading can be real if you've got an app that's basically all I/O
@@supergusano2 yep, not to mention that you can build mission critical things easily using the C api where you can release the gil, not that hard..
Regarding Python/Django: Running it with the dev-tools will by default be a single process that can handle a single request at a time. For production you commonly use a wsgi middleware instead (uwsgi, gunicorn, etc.) which will allow for enabling multiple processes and threads etc. Celery has nothing to do with that. Celery is a background task queue and needs its own processes for that.
But that's mostly just running multiple instances of your application, which is a bit different than the application itself running multiple requests.
@@thekwoka4707 depends on how you mean "multiple instances". What I did not mention yet is that Django is now getting decent async support, so there it would handle multiple requests in a single instance I guess. Although it might depends where exactly how you define "instance".
But aparr from that ersonally I think I would even consider uwsgi with multithreading enabled to be "multiple requests per instance", at least from a sysadmin perspective.
Yeah the python gripe is old news. It used to be that your only option to utilize you CPU fully was choosing the right size of thread pool for your workload and I/O latency. In comparison, an event loop can automatically make full use of your compute no matter how the workload varies. Comes with some downside. If you're being a "proper engineer" you're forced to write more complex async code and having lots of threads is good insurance against outlier heavy requests blocking your whole server.
But nowadays you can use async with FastAPI or if you love Django there's gevent so you'll use some other libs to monkeypatch your I/O and database connection pool and then you can write an async web server in the simple style of a synchronous one.
Sounds dodgy, turns out to be trouble free.
Ruby servers have been threaded for almost a decade (something like that) but no one really writes threaded code in an API call (obviously there are exceptions, but generally).
JS devs just can't get their heads round the concept of one thread per request and that your whole app runs for every single request.
They're used to dealing with multiple events in their code and their code being resident in memory indefinitely, spinning the event queue (like in a webpage). Which is fine, except when you do scale to multiple threads with each thread having multiple events - then you're in for a headache.
Thinking about multiple layers of async (theads AND event loop) seems like it might lead to... productivity loss @@PaulSpades 😂
@@pdougall1 Yeah, so cue the discussions about shared thread memory (with the security concerns) and thread synchronization, both definitely needed now that the long running program is handling a session that might reside on 3 different threads/processors/machines.
This never used to be a problem (the server software just passed very basic data structures about the request to all program instances when invoking them).
It used to be simple, understandable and effective, CGI was and still is a workhorse.
I believe Ruby MRI still has a GIL though. You'd have to go with JRuby to get around that.
As a person that has only ever programmed video games and generally has no clue how real software works I do always find it interesting when people talk about handling events or requests that are only in the triple or quadruple digits. Yet at the same time these are for infrastructures that handle millions of active users daily. It really shows just how vast the realm of software development can be and how massively different project requirements are.
As a game developer, your programs are doing far more computing than most devs using AWS. They're doing stupid amounts of serialization and network transfer for multi-second HTTP requests, while you're computing physics or game logic on 10s to thousands of objects 30-60 times per second. Or maybe the game engine is, but my point is that your application is actually doing something that's compute intensive, not passing trivial requests with layers of extra shit in-between like most web apps.
Learn the networking stack and HTTP and you too can start passing trivial HTTP requests through an excessive number of layers. Or you will see the value in other protocols, like web sockets, and then you can write browser games with multiplayer.
@@david0aloha I mean, potato / potahto. Most of the extra crap those truly massive companies have to deal with is just to help with problem of how big they are. On the other hand those tiny companies that want to pretend they are google could probably learn a thing or two from your comment! As for me - I won't lie. I'm a dogshit programmer. If I need speed then I find clever smoke n mirror tricks to simply hide the fact that I stop doing any work at all. If I can't hide it then I eschew all of the modern amenities of new programming languages just for the sake of cache coherency and avoiding heap allocations in a managed language. If I still need more I reach for the hammers that are my preferred threading and SIMD libraries. Or if I can get away with it, I dump it all on the gpu. Either way I've never solved a problem by actually writing a smarter algorithm. *Every* single algorithm I've ever written in twenty years has always been a for-loop or a nested for-loop.
His love for Node concurrency is weird. Node wasn't unique in its concurrency in any way. The small BEAM VM would scale pretty well on old hardware with a small footprint. I had an Fortune 500 enterprise wide app handling tens of thousands of requests a second running on an Erlang setup on a single Pentium 4 workstation 10+ years ago with almost zero effort put into scaling. Node has enabled concurrency on a midtier, more common language but it isn't great at it.
You can't have a discussion about concurrency without someone mentioning Erlang or Elixir - and for good reason, too.
reminds me of how the entirety of Battle Net for the Diablo (can't remember if 1 or 2) was run on a single random PC they had to spare.
If you write your code in a way that's tightly coupled to a particular serverless architecture, then you need to be very careful to pay down that tech debt as soon as you experience significant traffic. I don't think using serverless is necessarily a bad idea if you keep in mind that at some point you might need to shift those request handlers into a new infrastructure.
I totally agree with you on most serverless platforms selling lies and I love servers as modern programming languages like Go, Rust and others are very good with async and can effectively handle 1000's of requests per second on a $5/month nodes.
What's your take on Cloudflare workers and WASM. Although they got limits in their use cases but I think they might make a better serverless model to run either as a external platform or as an internal platform.
As a cloud software engineer(7 years exp + degree), I’ve seen large enterprise systems built in pure serverless using IAC. The bills are laughably, comically small. The trick is designing with event-driven processes in mind. It’s a juggling act, but a decent SA can design a system to substantially keep the bills low. Understand the services you are using and stay away from the shiny stuff. Api gateway, lambda, dynamodb are your legendary mains. Eventbridge, sqs, rds, are epics worth picking up. EC2’s absolutely do not pick up; you may get lucky every once in a while, but you’ll more likely get killed trying to look fancy.
If infrastructure is built using something like terraform, you can --somewhat- migrate serverless from AWS to GCP, however TF truly shines if you deploy elements of both (plus others) in the same codebase.
Laughably small bills with AWS lambda can only come from laughably small active users.
@@Sonsequence : No, it comes from laughably small concurrent active users, which is very true for Enterprise internal applications.
@@vikramkrishnan6414☺️👍
Unclear how serverless could possibly be cheaper than EC2 for any non-trivial workload.
@@mixmastermaverick3336 consider your use case and compare the workload of requests to serverless resources vs an ec2 setup as you scale. Keep it tight.
Wow what a strange thing! Can’t believe an AWS business would be doing a total business thing, and not acts of philanthropy. What an evil Bezoman!
I mean if your business plan is to milk all the money out of their clients..they won't have any clients after a while... Unchecked capitalism will eat itself alive. Companies do not learn...because they aren't people. They are legal contracts to protect the actual individuals from actual liability from their reckless behavior in pursuit of all that glitters and shines
0:55 He was referring to _Amazon's_ disincentives to invest in performance, not the user investing in performance.
You can only trust a company so much once it achieves market dominance. If they're declining from that spot, as AWS is being pushed by Azure, then they have incentives to play nice and act innocent.
It's pretty much about (once again) people buying into hype without understanding numbers. Serverless/Cloud is amazing in two scenarios. You have really low load or you have 'near infinite' load. The former will cost you next to nothing, the later will be prohibitively expensive to build infrastructure for. Thus, both make sense. It's all the middle ground in between those that might not.
Even the middle can be cost effective if you plan it out properly. You have to consider the cost of maintaining all this yourself.
so, basically spikey traffic. but I agree with prime in that you have to write your code in such a way that it is easily transferrable to a docker container at least
Software will grow to match the hardware it’s provided
It always hit me how different the IT space in(central) Europe is compare to the US . I mean we have a lot of small and medium sized companies while in the US the big ones are dominant . Which means a lot more languages are used but also companies make the webpages for other companies . I mean I know one and they host for their clients when they can on their own server. By the way I read today they looking for Golang Devs D. There are also less .NET fear because a lot of companies are already thanks to Microsoft Office depended on Windows so a lot of system integration and inter connectivity works .NET so far you aren't a bank.
The latest versions of .NET are cross platform.
Everyone swearing by C# is quite a pain, I must say. Please just give me something like go or ocaml or rust
@@shadamethyst1258 The problems with C# are only that you need to use Visual Studio for the UI creation and that is strongly typed the rest is just bias. I programming currently in C++ and C# is way better: Where I meant were it is used is in interfacing with ABB robots, controls system units but the same time with logistics and such stuff. Actually it good it isn't so hyped. When Java got hyped it got a Jenga Tower of Frameworks on top of each other. JS got React *hust*.
You still have to worry about efficiency with serverless / lambda. I've spent a lot of time reducing execution time of lambdas. that's not to say I always use it though, it's still pretty easy to build and deploy a containerized app on ECS in AWS too. Use spot instances and you pay so little.
Blaming the entire serverless paradigm just because people don't understand how to actually use it it's plain dumb imho. Should we say that object oriented programming is dumb just because people started abusing interfaces and inheritance? As everything in IT if you don't know how to do stuff you will probably end up making a mess. Lambdas are not the answer to every problem pros and cons as everything else in life folks.
Btw at the end of the day AWS still wins because they also sell raw compute resources so to say that they are sabotaging an entire industry feels too much to be honest.
That's one thing that annoy me as well, when people are comparing Node to PHP and then say PHP can only do single threaded. Yes, but Nginx is multithreaded. You don't use PHP on its own.
Oh boy, don't even try to explain to people how CGI works, it might explode their brain.
How do you do SSE (server-sent events) in PHP? Not possible
@@evergreen- You just return data content then two new lines and there's an event sent to client, and you'd need to ob_flush, all encapsulated away, similar to every thing else in http. And streaming the constant response is the same as any other streaming e.g. files. You could even mount your whole app under its framework as a daemon/container listening independently and load balance separately for better performance as you'd need to with most other languages (you could use PHP mostly on its own in other words).
real threads or green threads/asyncio? not the same
@@evergreen- I did write a chat app with SSE in PHP this year, it was a piece of cake. Very easy. Hard to find documentation, though.
But you can also use a network socket with php, python or whatever. Socket server libraries have been around for decades.
I'm actually converting away from serverless because I simply designed my serverless infra badly and I can design a monoservice REST API easier and more sustainably. It's likely going to cost me more at my app's size, but I'd rather have something sustainable since I'm a solo dev.
I learned about pickle the other day. Totally agree that python packages are named stupidly
I pickle my pandas with celery!
I’ve heard good things about Clojure performance wise as alternative to popular dynamic languages on the server side. It’s not amazing but people say is quicker to write than Java or Go.
"You design by Twitter choices."
This is the reason why I'm not improving. I'm just stuck in this cycle of chasing the next big thing.
We used aws lambda for image transcode. Was nice since if we didn't have images to transcode it was a lot less than a server and we could handle bursts of work within seconds. Was nice to not keep that much compute around.
the Python runner for a production server is gunicorn which handles a thread pool for you. So, yes, django multithread requests if you deploy it correctly. It's just that if you run a dev server in production, it will not multithread by default. but that would be dumb. Nobody does that. It takes 30sec on internet to know you need gunicorn and it is very easy to set up
You can search a couple of very good papers detailing how building your own infrastructure is actually more profitable long run
any specific references you had in mind? I'm interested in reading about this
In the long run. Not in the beginning
It actually depends what your workload looks like, constant vs bursts, how high is peak vs average. Its not that straightforward.
Serverless is pretty profitable as long as you don't have absurdly high usage, from my experience.
What if your startup fails and you never exceed free tier?
The commenter talking about Celery is kinda misleading. Celery is like the sidekiq equivalent in Ruby land, it's for async/background job processing. Celery workers typically live on a completely separate machine and receive tasks through some sort of broker like Rabbit-MQ / Redis.
When you run a Python web service in prod with Flask/Django you typically achieve per-request concurrency by using something like Gunicorn or uWSGI and pointing these at your app.
These are pre-fork worker HTTP servers, so each worker essentially takes your application code and runs its own version of Django/Flask within the process.
I'm not a NodeJS person, but it seems similar to the way you'd deploy an express app with PM2
From the Microsoft side, net gets big performance increases every year, including AOT complication, which they show being used on Azure Functions, so that you don't have to pay the JIT cost.
Of course they do this because they want more people to use Azure Functions, and serverless is more expensive than other forms of hosting, but they don't intentionally hold back performance just to increase compute time.
You can also host Azure functions inside k8s now with keda to scale to 0, which is more cost effective, but then you have to know more about k8s, although there are Azure container apps for that now too.
The bigger you get, the more it becomes worth it to spend dev time to reduce operational costs.
Docker containers are easy enough to start with to make serverless pretty pointless anyway though, and there are a lot of different ways to host them easily.
In reality, the language that is most suitable for serverless is PHP, which has always used one process per request.
Thats why PHP still 80% of internet , PHP is stateless so you send a request to the server , the response will be a process that will die after finish his job such as HTTP protocol thats why PHP is the secret king 👑 of APIs and SaaS
@@h.s3187I don't believe the process actually dies after the request in PHP-FPM or most PHP servers, but all your variables are garbaged at the end of the request and not shared with whatever's done for the next request. So it's as if the process died but without all the overhead of starting a new process.
3:44 As someone building an app with Django and Celery, I felt that 😑
As to your opinion on Python I have always referred to this as "Do you want to know, or do you want to learn?"
People who want to know something identify a skill or ability that it would be convenient or fun to have. "I would like to know French. I would like to know how to play guitar."
Wanting to learn is the act of saying to yourself "I am going to suck at this, and I am going to put time and effort in, and eventually I will suck less. The more time and effort, the less suckage."
Great point.
in serverless performance doesn't matter? cold start time is influenced by bundle size, and you're getting charged by the ms... compare that to containers which provide similar (but more effortful) abstractions, and you can just as easily "let's just wastefully scale up", and confusing performance issues related with "ongoing" process as well as "per-request" processes... the unit of measure for perf gains there feels like "number of pods"
edge compute isn't about user proximity... it's about more efficient v8 reducing some of that "wasted time" you attributed serverless to earlier.... who's measuring the wasted compute on a pod? if you can't scale as much
i dunnno... this has vibes of "serverless is making money by making things more convenient for devs", even "managed databases" are more "serverless" aren't they? not lambda, but serverless in terms of pricing model and mental model that you "think about the server less"
I think word serverless is also often used to exclusive talk about FAAS services, while it is not limited to them.
For example in AWS SQS is serverless, DynamoDB is serverless, ECS Fargate is serverless.
For example we have recently migrated from having HTTP served by lambdas to server running in ECS with Fargate launch type. Of course it is more expensive than running it in EC2 (I think it can be like 2x difference when reserving instances), but still better than lambdas, both cost and performance wise.
They are many kinds of serverless services out there and people really should start noticing more of them.
“Design by Twitter choices” is what I have witnessed so far in my career and I it’s so frustrating. Management doesn’t care to try out solutions and pick the best one. Never seen a flame graph outside a YT video.
I love how developers are now realizing this and saying oh boy yeah that's bad, we shouldn't use aws lambda/az functions... and then immediately turn around and happily use the pay-per-compute-second k8s clusters they host in AWS/AZ. Literally the same thing, except now you're paying for machine CPU/RAM/DISK as well... you know the thing we finally escaped when we went serverless with lambda/functions... and it's all obscured behind the k8s resources syntax...
Been hit with the max lambda limit before, theres a max amount of lambdas you can create and a different amount for how many can be called and yet another amount for how many can be called concurrently. These are restricted by aws and you have to ask them to bump them up for you ($$$).
This happened to me when working for a health company, covid hit and we were swarmed with requests. The queued lambdas were causing all kinds of problems, events were being dropped db connections exceeded max number which caused even more issues. It completely broke our system. A system that is very carefully tested and maintained yet we could not even properly test a scenario like this :(
What were you doing in the Lambda's? If they're being an SQS queue then it's pretty trivial. If the Lambda's can't keep up then the queues will just accumulate but you won't lose anything.
I believe one of the most overlooked benefits of serverless technology is its ability to digitalize processes in non-technical industries.
Many people think of AWS primarily as a tool for developers, but the reality is that most industries are not tech-centric, yet they still require technical solutions. In these companies, many processes are manually executed, entire databases are maintained by hand, and poor communication due to bureaucracy or other factors is common.
This scenario makes monitoring, tracking, and deploying services almost impossible. Transforming company culture, training employees, and addressing other challenges requires significant time, money, and effort.
Using AWS infrastructure simplifies explaining the benefits, monitoring costs, and collaborating with various teams. While many solutions may not be optimal from a technical perspective, especially in large, non-tech companies, serverless technology makes it much easier to create sustainable and flexible solutions.
some of the stuff in the livechat is truly hilarious. this was great
At my last job I had a debate with a senior dev and the CTO about performance.
They were adamant about the fact that today we focus on readability over performance and that modern computers are so powerful that our crappy node backend was just fine.
I found it crazy from an engineering and craftsmanship point of view.
9:00: With sanic (another name you'll "love") I can take, in python, more requests per second than nodejs to do the same work as in nodejs, as long as there's 3 or more async/await in the javascript sequence.
The greatest trick Jeff Bezos ever pulled was tricking developers into thinking serialisation and network latency are trivial, and that memory and compute are expensive
Actually it was delivering AAA batteries to my house 3 hours after ordering them.
K8s + knative. Local dev better than lambda having to constantly push to AWS. Multi cloud. Serverless. Sure more cost up front, but better imho for enterprise. Smaller dev shops, sure lambda helps.
that's cool, TIL
@20:30 "Maybe Elixir is something to learn."
Yes and no. You should learn actor based concurrency. And probably the best, generally available implementations of actor based concurrency is Erlang (and therefore Elixir). I really believe every dev should learn Erlang. Not because they should use it on all their projects but because the programming model is truly different and can open one's eyes to different paradigms. Similar to how all devs should do some serious functional development at some point.
That said... learn Erlang over Elixir first. It is a simpler, more concise language and all the important things about Elixir are based or straight on top of the core Erlang and BEAM VM infra. Elixir may have some nice syntactical sugar but that isn't the important stuff and IMO can be a distraction.
ah, the pay by second actually reminds me of making a mistake when developing a C program for the mainframe back in 2010 or so - the time on the IBM Mainframe was paid by second & you had some wild memory restrained, so if your program ran for a few minutes instead of 20s than that was a problem. This just reminded me how wild I thought the restrains and the prices were back then & now we're again paying wild prices for cloud services.
3:44 Node doesn't rely on stop-the-world green threads. It actually utilizes multiple threads via the event loop.
8:00 YES! People don't understand the thread-thrashing it causes. You SHOULD NOT parallel async _within_ a request, generally speaking (promise.all is a no-no).
Default Python everybody uses is single-threaded. Django and Flask web apps achieve concurrent request processing via external tools like gunicorn to run basically N instances of app server.
However, Python now has full async/await support baked into language. FastAPI app server leverages that, and you can write apps in Node-style, just be careful and don’t accidentally use sync APIs
The AWS lock-in is wild. You can end up in situations where you are so dependent on their services to have your product up and running and scaling somewhat okay, that you can never really leave without insanely high cost. Also warm tip: keep track of your own usage with a third party service and monitor that well.
Python isn't single threaded by its limited by a global lock. Its supported async await for ages now. and has plenty of ways to get things into a thread (Futures, Threads, gevent, celery, and so on.)
Async/await runs on a single core. Python threads run one at a time, not all at once. You need some hacky library or to spawn multiple processes that talk with each other over the network
Serverless isn't the issue. There are other serverless technologies that you can self-host that are awesome. It is the misuse and lack of understanding of the AWS pricing model which is a bit of a pain to deal with.
Using AWS lambda like a web server seems crazy to me. I’ve only used it for background processes that get triggered by a queue or like a daily/hourly schedule. And I use step functions map iterator to launch them concurrently instead of having loops and things inside a single lambda
Having Lambda serverless components in your startup codebase is like have a chandelier in your one bed apartment. - more of an expensive art installation than appropriate decor.
There’s an aws blog post where they talk about big savings switching some aspect of prime video away from lambdas to ec2
Python actually is single threaded because of GIL.. so basically concurrency exists, but not parallelization, which is why people end up with multi processing, so each process takes up one core in multii-core env and attain actual parallelization. 😊
Clearly, these guys have no idea how to build aws serverless infrastructure 😮. 1. 1 request does NOT mean 1 new lambda. lambda stays idle for some time and it can receive more request while its still alive. for example, if you have less than thousands of request per sec, you are just using 1 lambda.
2. you should use some combination of SQS, SNS with lambda calls to handle request that does not require to respond immediately.
there are huge chunk of contexts that they are not aware of.
Python has threads, multiprocessing. FastAPI uses the ASGI uvicorn. We use FastAPI with multiple workers and threading all the time.
16:45 you can always look at moving to knative at that point, putting in on top of your k8s cluster(s)
In the java world, we use Micronaut that can create apps that can run as a CLI, Microservice (K8S) and AWS Lambda without recompiling (same jar).
Just going to bomb in with a little info about how a typical Ruby/Rails app works these days. It's somewhat like Node. You run Puma, an app server which runs a set of threads and manages a global interpreter lock for those threads to have access to Ruby code. Only one thread can run your Ruby code at once, but most of the time a thread is outside of the Ruby, doing IO of some sort.
Yeah, this is a hack around the fact that Ruby is 20 years old, and wasn't originally designed with parallelism in mind. But it achieves good enough performance for all sorts of normal workloads, it's a reasonable tool for many jobs.
On a relatively small instance, AWS will give Puma a default setup of 2 processes with 32 threads each. If your task is IO bound you can get quite a lot of grunt out of that instance.
As an infrastucture-as-code engineer I am in shambles. Just kidding, I prefer to run EC2 Linux, S3 and cloudfront, most of the rest is trash. Aurora is fine, ECS and Fargate are okay.
Async-await was adopted long time ago. Was it adopted from C#?
1000 free Lambdas. I once paid like $5 for some Ukrainian/Canadian access attempt chain. My script added trailing slashes. The attacker tried every Wordpress vulnerability known to man. It was pretty massive. They tried several times. Sometimes from both locations at the same time.
Infra probably doesn't need to cost so much. I mean, you can setup the server in your local machine/server, use a bit of firewall, have a systemd service renew the letsencrypt certificates, or maybe use Route53, and then connect an ec2 instance and local machine via zerotier and socat .Maybe add some nginix and SNI to the mix. And that's mostly about it.
The things that need taken care of would be, system upgrades, database management, local machine failing, network goes off. But does that really need this much amount of billing? People can do so much with just one fat machine.
Not saying people should do it on their own, but it shouldn't cost this amount.
I absolutely agree with the title of this video (with the content too…) 👍😊 …more people should have realized this a long time ago…
Going back and forth between what comes out of prime's mouth and what comes out of theo's mouth is giving me a real bamboozle of two completely different philosophies
i think the good part is that people have different ideas how to approach problems
Some facts, cause homeboy is lost on this one, you can write C extensions in Ruby, meaning there's potential of making it fast, Python does have async and await (he corrected after) but homie here is like ChatGPT today...
you can write C extensions in PHP
I worked with Lambda and with standard containers and I think lambda is the greatest piece of shit ever. From deployment headaches to cost per ms, it's all pretty weak in comparison to a nicely built microservice and the only scenario that actually makes sense is a service that gets little traffic and long waits between calls or maybe some asynchronous work in conjunction with SQS.
You can dip that nodejs into c also... `npm install -g node-gyp` I built a node app that calls windows apis to send keystrokes and mouse movements to run a flash game for me once. The image processing was too slow doing the screen capture and looking at the bitmap in code so I I installed CUDA and figured out how to offload that to the GPU.
it's a trade off between how fast do you want to ship the product vs how much it would cost to ship it that fast. If you don't want to spend that money on infra, then you optimize your infra.
Why would your startup with less than 100 users need much more than a VPS?
That's not 300k per annum, that's like...100 (no k) per annum.
because the cloud is the future, old vm, or even worse, baremetal, are old school and for loosers...
they are cheaper, reliable, but you know, you have to use the "last tech", it's so much better...
It's not actually that hard to move stuff out of cloud. It's when large companies put stuff on cloud the costs start counting, but tbh - even then quite often it's not really that expensive compared to development cost. If your AWS bill is less than 1 million a year - probably it's not worth going off cloud just yet. There is absolutely a point to be made that at a certain scale - on prem can save money and even offer a better experience. This also goes for control - sometimes the service latency just becomes problematic if you're on cloud and there is a legitimate case for moving off the cloud. There is also an argument about compatibility - not everything works in the cloud. There are good reasons to be off the cloud, but the majority of startups will pay 0 or very little for AWS. Might as well use it. About serverless in particular though, it's debatable whether it's actually really properly supported and worth it. It's not as clear cut that it's "just better in every way".
4:43 usually no one run python server code in a single thread. 99% time you use WSGI or ASGI to handle the incoming traffic, just like you do with PHP (PHP-FPM, etc...)
Default per account limit of lambda is 1000, you can request an increase though. My company has a limit of 10k for example.
20:16 He should admit he's wrong at this point 😂 Erlang (whose VM Elixir runs on) was made for concurrency with built-in fault tolerance. It was designed by a telecom company to process phone calls at scale. Of course it's performant 💧
Lots of services on handle a few requests per minute. There is a rate above which switching to containers makes sense.
AWS Lambda is actually perfect for small startups where software is basically a cost center. The costs are simply too low, and the scale is infinitesimal
Is that a Miku with bazooka on your wallpaper?
At 0:08
Node js is not single threaded... under the hood it decides in c++ what is getting multithreaded and what isnt.
People who blanket hate serverless have never actually done the math of how much it can save you for low-frequency functions/lambdas/handlers/etc.
More efficiency leads to less total computation, which leads to a lower bill. Maybe I just don't understand Lambda well enough to know why they're saying this doesn't motivate companies/programmers to write efficient software? AWS/Azure/GCP might WANT you to write inefficient software, but the software engineers who use those services are certainly motivated to write more efficient software. If anything, you could see the pay-for-compute model is a way of punishing companies that write inefficient software.
Serverless isn't just Lambdas, for some business ; not having to worry about servers (patching/security/updates/migrations/scale) is worth the price.
Serverless is a great tech, but the result depends on how you use it. I can put a caching layer in front of a Lambda and compute once in a while without holding the server running, that's great. Lambda@Edge is also great, I can run my compute near the user, and don't need to have a server for each region.
For "not investing in performance" bit, there are multiple cloud providers, competition is a drive for them to optimize perf.
yes, all depend on the use case... For every tech...
You don't need a k8s cluster everywhere, and you don't need serverless everywhere...
But, in some situation, those product are very very interesting and are a really good choice...
With lambda & microservices, who earn a lot of money? Cloud providers.
Node is asynchronous, but NOT concurrent! Concurrent and parallel are effectively the same principle. Both are related to tasks being executed simultaneously. Asynchronous methods aren't directly related to the previous two concepts, asynchrony is used to present the impression of concurrent or parallel tasking.