- 91
- 15 478 377
Fly․io
United States
Приєднався 3 сер 2022
Run web, AI, & app servers close to your users on CPUs and GPUs. Deploy and scale your app in minutes at fly.io/speedrun.
We built an orchestrator from scratch. Here's why.
Fly.io doesn't use a typical orchestrator - ours is *decentralized*, which is kind of weird! In this video I walk us through what you might typically need to build an orchestrator from scratch, why the mainstream ones like Kubernetes and Nomad didn't work for us, and what we did instead.
BLOG POST: Carving The Scheduler Out Of Our Orchestrator
fly.io/blog/carving-the-scheduler-out-of-our-orchestrator/
Video written and produced by Annie Sexton.
BLOG POST: Carving The Scheduler Out Of Our Orchestrator
fly.io/blog/carving-the-scheduler-out-of-our-orchestrator/
Video written and produced by Annie Sexton.
Переглядів: 1 827
Відео
Not your grandma's SQLite • feat. LiteFS + Litestream creator Ben Johnson
Переглядів 2,3 тис.14 годин тому
It's entirely possible to have strong disaster recover AND distributed replicas with SQLite when using Litestream and LiteFS. In this video I chat with Litestream and LiteFS creator Ben Johnson about how this can be done. Learn more about LiteFS: fly.io/docs/litefs/ Learn more about Litestream: litestream.io/ Interview conducted and editing by Annie Sexton.
Do you REALLY need Kubernetes?
Переглядів 17 тис.14 днів тому
If you're on the fence about using Kubernetes for your project, this video will help break things down. It's not a simple "yes" or "no" answer. Fly Kubernetes Service (FKS): fly.io/docs/kubernetes/ Video written and produced by Annie Sexton.
Is serverless actually better? - w/ Phoenix framework creator Chris McCord
Переглядів 8 тис.28 днів тому
Is using serverless functions actually easier and cheaper? In this video I chat with the creator of the Elixir framework Phoenix, Chris McCord, about why you might not need serverless functions to elastically scale. Interview conducted and editing by Annie Sexton.
We did something *weird* with our CLI.
Переглядів 11 тис.Місяць тому
Fly.io does some CRAZY STUFF to allow you to connect to your Machines, and you'd never know it. SSH-ing into an app instance seems simple enough, until you put that instance on a virtual private network. In this video, I share the weird steps we took to make SSH-ing into your app's Machine's a one-command ordeal. wireguard-go library git.zx2c4.com/wireguard-go gVisor's user space TCP/IP stack: ...
Trash talking the cloud w/ Fly.io CEO Kurt Mackey
Переглядів 203 тис.Місяць тому
Kurt Mackey and I chat about why public clouds generally suck. There's also a lot of (unwarranted?) hate on PaaS's built on top of these clouds. Kurt Mackey and I chat about why clouds aren't really built for developers. Interview conducted and edited by Annie Sexton.
We don't use containers, and here's why.
Переглядів 6 млн2 місяці тому
Fly.io runs your applications in light-weight micro-VMs called Fly Machines instead of containers, but why is this? The short answer: Isolation. In this video, we'll explore a few different categories of isolation and what kind of security they provide. An Intro to Fly Machines: fly.io/docs/machines/overview/ Machines API: fly.io/docs/machines/api/ Fly Kubernetes: fly.io/docs/kubernetes/ Video ...
Globally Distributed Postgres on Fly.io | Single leader replication + fly-replay
Переглядів 2,8 тис.2 місяці тому
Deploying your app to multiple regions is great - until you have to make long-distant calls to your ONE database that lives halfway across the world. In this video I'll show you a simple technique that makes it easy to deploy a globally distributed Postgres cluster using a really cool feature on Fly.io called "fly-replay". Fly-Replay Header: fly.io/docs/networking/dynamic-request-routing/ Bluep...
How Fly.io does routing at the edge | BGP + Anycast
Переглядів 1,6 тис.2 місяці тому
Learn how Fly.io serves your app close to your users by routing on the edge with BGP (border gateway protocol) and Anycast. Learn about Fly Domains: ua-cam.com/video/vU9xcRCX7-U/v-deo.html Learn about networking on Fly.io: fly.io/docs/networking/ Video written and produced by Annie Sexton.
Fresh Produce | July 2024
Переглядів 4582 місяці тому
A quick overview and demonstration of features introduced on fly.io in the month of July. Community Forum: community.fly.io flyctl Features - python scanner improvements: community.fly.io/t/even-more-python-support/20764 - meteor 3: community.fly.io/t/fly-launch-now-with-more-meteor/20837 - github actions: community.fly.io/t/automatic-github-push-to-deploy/20671 - into flag: github.com/superfly...
Learning AI Deployments
Переглядів 2672 місяці тому
I've been learning how to deploy and augment LLM models using Fly.io's GPUs. If you are building AI apps, here are five things I've learned as I get more familiar with everything: 1. Every LLM model should be its own application 2. Don't allow the public to access your models. 3. Don't redownload your base models on every run. 4. Don't wipe your database. 5. Stop idle GPUs. If you want to learn...
Deploying an LLM-Powered Django App | Ollama + Fly GPUs
Переглядів 1,1 тис.3 місяці тому
Learn how to run LLMs locally, integrate with your Python/Django apps, self-host Ollama with one file and finally deploy an LLM-Powered Django app using self-hosted Ollama running on Fly GPUs. Source code: github.com/katiayn/django-fridge Ollama demo app: github.com/katiayn/fly-katia-ollama Related videos: - Fly.io networking: ua-cam.com/video/vU9xcRCX7-U/v-deo.html Related links: - Ollama webs...
Supercharge your Laravel app with AI
Переглядів 1,1 тис.3 місяці тому
Supercharge your Laravel app with AI
Stream Responses from an LLM | Ollama, React, Astro on Fly.io
Переглядів 1 тис.3 місяці тому
Stream Responses from an LLM | Ollama, React, Astro on Fly.io
Deploying an App using a Self-Hosted LLM | Fly GPUs + Ollama + Remix
Переглядів 4543 місяці тому
Deploying an App using a Self-Hosted LLM | Fly GPUs Ollama Remix
Scaling a social network on a whiteboard by Xe Iaso | DevOpsDays MSP
Переглядів 2,2 тис.3 місяці тому
Scaling a social network on a whiteboard by Xe Iaso | DevOpsDays MSP
Fly Domains Explained | .fly.dev, .internal, .flycast
Переглядів 3,3 тис.3 місяці тому
Fly Domains Explained | .fly.dev, .internal, .flycast
Build your Personal Language Assistant with an LLM | Fly.io GPUs + Ollama
Переглядів 6073 місяці тому
Build your Personal Language Assistant with an LLM | Fly.io GPUs Ollama
How to Self-Host an LLM | Fly GPUs + Ollama
Переглядів 2,5 тис.3 місяці тому
How to Self-Host an LLM | Fly GPUs Ollama
Globally-distributed Object Storage (no CDN required) | Tigris Data + Fly.io
Переглядів 2,4 тис.4 місяці тому
Globally-distributed Object Storage (no CDN required) | Tigris Data Fly.io
Boost Your App with Self-Hosted LLMs on Fly.io - Step-by-Step Guide
Переглядів 3 тис.4 місяці тому
Boost Your App with Self-Hosted LLMs on Fly.io - Step-by-Step Guide
Auto configuring Node (and Bun!) apps for Tigris
Переглядів 5994 місяці тому
Auto configuring Node (and Bun!) apps for Tigris
How Fly.io uses Docker (without Docker)
Переглядів 40 тис.4 місяці тому
How Fly.io uses Docker (without Docker)
Bounce traffic between Apps and Machines with fly-replay HTTP header
Переглядів 7194 місяці тому
Bounce traffic between Apps and Machines with fly-replay HTTP header
Using AI to Boost Accessibility and SEO - Fly.io blog
Переглядів 1224 місяці тому
Using AI to Boost Accessibility and SEO - Fly.io blog
Shouldn't a decentralized orchestrator be called a choreographer?
So flyctl keep track of empty workers to kill or in loaded region to create more workers?
Very intresting use case. Isnt this basically the same as the inner engine of GitLab / GitHub / ADO when they run pipelines with a pull instead of a push. Its exactly how runners / agents listen to broadcasted events from the main "API" service.
nice - out of interest have you open-sourced flyd?
😂😂😂😂 I did not know if to cry or to laugh. U know that real system engineers will not believe you?
So you wanted to perform bin-packing, but treat the machines' resources (vcores, ram, etc.) as the "capacity" of a container? When you were talking about how it's not ideal to try and fit as many applications as possible on a single worker, did "fit" refer to the size of the fly machine images on disk, or the expected resource utilization of an application? (Unsure how you could even predict resource utilization without relying on probalistic methods anyway, so I'm assuming it was the former)
This is a really cool approach! The bidding system is really clever.
thanks for watching? thanks for explaning :)
I need to start playing with Firecracker. GitLab runners use Docker but the isolation isn't great. For instance running Docker within Docker isn't supported unless you expose your Docker socket from the host into the container (which isn't secure).
Nice speed run through the problem space! 🏁
Awesome video Annie! 🤩You are killing it!
Kubernetes is dying. Our DevOps Team Stopped Using Kubernetes and They are Is Happier Than Ever
Can we use any llama based model ? In the destination xan we use the llm we have downloaded ? Imean the custom llm based on llama ?
Fantastic - great coverage of these tools
Hi, thanks for these helpful videos to a seasoned programmer but Docker newer-ish. The video is about a year old, and I'm wondering if fly still doesn't support Silicon Mac use in build architecture, as I'd imagine you would have some support considering the GPUs available? Otherwise not sure how they're running with Intel.. do tell
Great video! thank you
Loved this chat! I've been using LiteFS on fly for years and it's been great!
Virtualization doesn’t prevent hardware vulnerabilities.
I was just wondering what rad means.
I am a frontend engineer. I am learning SQLITE I love it just because of its simplicity.
you can use libsql if you want sqlite but want local mode, server mode, distributed mode, replicas, embedded replicas, muti database schema management, etc
Excellent show. Thanks for taking the time to put this together.
Thanks for the tip, it helped a lot
The rails people aren't recommending running your own database really. Most of us push people to use Crunchy or similar. It's more for stateless app/job servers.
Watch out, the company I work for used their service, got scammed and it's about to go bankrupt. Lot of people are going to lose their jobs
k8s would be cool if it weren't so expensive once you want to host it somewhere.
Fly is great. But yes sometimes we need Kubernetes
Whoa 4.7 million views
Virtual machines are no more secure than containers and both technologies increase attack vectors by introducing unnecessary libraries and vulnerabilities either specific to the virtual machine employed or the containerized operating system. I know you won't do this, but just use Cosmopolitan C and it's clever way of employing PKZIP files which delivers all of the same benefits with less risk and greater portability across multiple architectures. If you feel you still need to run everything in a VM after exploring it, at least just compile your virtual machine in it. ;)
A note about the first statement, about serverless and FaaS: they aren't synonims, and they can't be really interchanged. There are approaches that let you build a FaaS infrastructure on a bare metal server. So in cases like this there's an abstraction that hide most of your bare metal infrastructure details, but you're not committing to a fully managed and pay-per-use instance like an AWS lambda service would do.
Trying to do more research on this. Can you point to any sources? I found a presentation on doing this with rootless podman from a couple years ago but thats it so far. Is there a way to achieve functions as a service in kubernetes? Is there a specific OS or application you know of that helps achieve FaaS on prem? Thank you
@@ultravioletiris6241 it's raw, it's early, but someone is trying to achieve FaaS without committing to pay-per-ore. Search for "Apache OpenServerless" incubating project.
one single person against the serverless mafia - go Chris!!
Great video. But better beware of Betteridge's law 😉 en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines
your OF account?
You know why they give the first FKS implementation away on free tier? Because they don't really give two FKS for k8s.....
hmm... i was mislead.
Great video. Also, love the poster in the back ;) Pyra is the name of my ai assistant lol
I tried this platform, and this platform is unnecessary complex. However, their videos are awesome
It absolutely makes sense to run K8S even at small scale due to its nice ecosystem. You really get a lot of functionality for free. If you're not sure, give it a good try. It's not hard to use or install. You get free HA if you can abstract storage volumes. I love K8S.
No mention of NixOS, 2/10
Im glad you give us FKS. But you might want to run that one by marketing. How about FlyK? You can pronounce it flick.
If you're in a position to ask the question these days, the answer is probably yes, because the alternatives are probably worse. It's fairly easy to use k8s, esp. k3s for pretty much anything even on a single node. The complexity of k8s itself is vastly exaggerated, while the complexity of devops in general is often underestimated.
FKS demo please
So, the alternative to K8s is to couple one's application to the Fly service? Please correct me if my takeaway was wrong. I'm not arguing everyone needs K8s, but the only alternative you mentioned was to use Fly's built-in functionality instead, which seems a lot like jumping out of the frying pan (of complexity) and into the fire (of proprietary dependencies.)
Oh no, we fks'd up...
Kubernetes saves us metric bucketloads of money compared to renting PaaS/SaaS for things like kafka, cassandra, postgres etc. It's amazing for running infrastructure on
Kubernetes exists solely to give all the H1Bs something to do.
What means high density, can someone please elaborate?
Interesting approach :-)
I feel like horizontal scaling only becomes an issue early on with single threaded dynamic languages like Python/Node apps. With something simple like Go, html/template, and SQLite(WAL) it's amazing how far a monolith can be pushed on a VPS. Down the road under heavier loads though K8s can be nice if I'm not managing it so something like FKS could be a good option.
I'd still have a container on containerd inside your firecracker VM if I was using your stuff. I assume like Lambda this is an option. You'd be poor business people if you didn't support that model.