Fly․io
Fly․io
  • 91
  • 15 478 377
We built an orchestrator from scratch. Here's why.
Fly.io doesn't use a typical orchestrator - ours is *decentralized*, which is kind of weird! In this video I walk us through what you might typically need to build an orchestrator from scratch, why the mainstream ones like Kubernetes and Nomad didn't work for us, and what we did instead.
BLOG POST: Carving The Scheduler Out Of Our Orchestrator
fly.io/blog/carving-the-scheduler-out-of-our-orchestrator/
Video written and produced by Annie Sexton.
Переглядів: 1 827

Відео

Not your grandma's SQLite • feat. LiteFS + Litestream creator Ben Johnson
Переглядів 2,3 тис.14 годин тому
It's entirely possible to have strong disaster recover AND distributed replicas with SQLite when using Litestream and LiteFS. In this video I chat with Litestream and LiteFS creator Ben Johnson about how this can be done. Learn more about LiteFS: fly.io/docs/litefs/ Learn more about Litestream: litestream.io/ Interview conducted and editing by Annie Sexton.
Do you REALLY need Kubernetes?
Переглядів 17 тис.14 днів тому
If you're on the fence about using Kubernetes for your project, this video will help break things down. It's not a simple "yes" or "no" answer. Fly Kubernetes Service (FKS): fly.io/docs/kubernetes/ Video written and produced by Annie Sexton.
Is serverless actually better? - w/ Phoenix framework creator Chris McCord
Переглядів 8 тис.28 днів тому
Is using serverless functions actually easier and cheaper? In this video I chat with the creator of the Elixir framework Phoenix, Chris McCord, about why you might not need serverless functions to elastically scale. Interview conducted and editing by Annie Sexton.
We did something *weird* with our CLI.
Переглядів 11 тис.Місяць тому
Fly.io does some CRAZY STUFF to allow you to connect to your Machines, and you'd never know it. SSH-ing into an app instance seems simple enough, until you put that instance on a virtual private network. In this video, I share the weird steps we took to make SSH-ing into your app's Machine's a one-command ordeal. wireguard-go library git.zx2c4.com/wireguard-go gVisor's user space TCP/IP stack: ...
Trash talking the cloud w/ Fly.io CEO Kurt Mackey
Переглядів 203 тис.Місяць тому
Kurt Mackey and I chat about why public clouds generally suck. There's also a lot of (unwarranted?) hate on PaaS's built on top of these clouds. Kurt Mackey and I chat about why clouds aren't really built for developers. Interview conducted and edited by Annie Sexton.
We don't use containers, and here's why.
Переглядів 6 млн2 місяці тому
Fly.io runs your applications in light-weight micro-VMs called Fly Machines instead of containers, but why is this? The short answer: Isolation. In this video, we'll explore a few different categories of isolation and what kind of security they provide. An Intro to Fly Machines: fly.io/docs/machines/overview/ Machines API: fly.io/docs/machines/api/ Fly Kubernetes: fly.io/docs/kubernetes/ Video ...
Globally Distributed Postgres on Fly.io | Single leader replication + fly-replay
Переглядів 2,8 тис.2 місяці тому
Deploying your app to multiple regions is great - until you have to make long-distant calls to your ONE database that lives halfway across the world. In this video I'll show you a simple technique that makes it easy to deploy a globally distributed Postgres cluster using a really cool feature on Fly.io called "fly-replay". Fly-Replay Header: fly.io/docs/networking/dynamic-request-routing/ Bluep...
How Fly.io does routing at the edge | BGP + Anycast
Переглядів 1,6 тис.2 місяці тому
Learn how Fly.io serves your app close to your users by routing on the edge with BGP (border gateway protocol) and Anycast. Learn about Fly Domains: ua-cam.com/video/vU9xcRCX7-U/v-deo.html Learn about networking on Fly.io: fly.io/docs/networking/ Video written and produced by Annie Sexton.
Fresh Produce | July 2024
Переглядів 4582 місяці тому
A quick overview and demonstration of features introduced on fly.io in the month of July. Community Forum: community.fly.io flyctl Features - python scanner improvements: community.fly.io/t/even-more-python-support/20764 - meteor 3: community.fly.io/t/fly-launch-now-with-more-meteor/20837 - github actions: community.fly.io/t/automatic-github-push-to-deploy/20671 - into flag: github.com/superfly...
Learning AI Deployments
Переглядів 2672 місяці тому
I've been learning how to deploy and augment LLM models using Fly.io's GPUs. If you are building AI apps, here are five things I've learned as I get more familiar with everything: 1. Every LLM model should be its own application 2. Don't allow the public to access your models. 3. Don't redownload your base models on every run. 4. Don't wipe your database. 5. Stop idle GPUs. If you want to learn...
Deploying an LLM-Powered Django App | Ollama + Fly GPUs
Переглядів 1,1 тис.3 місяці тому
Learn how to run LLMs locally, integrate with your Python/Django apps, self-host Ollama with one file and finally deploy an LLM-Powered Django app using self-hosted Ollama running on Fly GPUs. Source code: github.com/katiayn/django-fridge Ollama demo app: github.com/katiayn/fly-katia-ollama Related videos: - Fly.io networking: ua-cam.com/video/vU9xcRCX7-U/v-deo.html Related links: - Ollama webs...
Scaling your GPU app in anger
Переглядів 6603 місяці тому
Scaling your GPU app in anger
Compose Apps with flyctl
Переглядів 5283 місяці тому
Compose Apps with flyctl
Supercharge your Laravel app with AI
Переглядів 1,1 тис.3 місяці тому
Supercharge your Laravel app with AI
Adding your own models to Ollama
Переглядів 1 тис.3 місяці тому
Adding your own models to Ollama
Stream Responses from an LLM | Ollama, React, Astro on Fly.io
Переглядів 1 тис.3 місяці тому
Stream Responses from an LLM | Ollama, React, Astro on Fly.io
Deploying an App using a Self-Hosted LLM | Fly GPUs + Ollama + Remix
Переглядів 4543 місяці тому
Deploying an App using a Self-Hosted LLM | Fly GPUs Ollama Remix
Scaling a social network on a whiteboard by Xe Iaso | DevOpsDays MSP
Переглядів 2,2 тис.3 місяці тому
Scaling a social network on a whiteboard by Xe Iaso | DevOpsDays MSP
Fly Domains Explained | .fly.dev, .internal, .flycast
Переглядів 3,3 тис.3 місяці тому
Fly Domains Explained | .fly.dev, .internal, .flycast
Build your Personal Language Assistant with an LLM | Fly.io GPUs + Ollama
Переглядів 6073 місяці тому
Build your Personal Language Assistant with an LLM | Fly.io GPUs Ollama
How to Self-Host an LLM | Fly GPUs + Ollama
Переглядів 2,5 тис.3 місяці тому
How to Self-Host an LLM | Fly GPUs Ollama
Globally-distributed Object Storage (no CDN required) | Tigris Data + Fly.io
Переглядів 2,4 тис.4 місяці тому
Globally-distributed Object Storage (no CDN required) | Tigris Data Fly.io
Boost Your App with Self-Hosted LLMs on Fly.io - Step-by-Step Guide
Переглядів 3 тис.4 місяці тому
Boost Your App with Self-Hosted LLMs on Fly.io - Step-by-Step Guide
Auto configuring Node (and Bun!) apps for Tigris
Переглядів 5994 місяці тому
Auto configuring Node (and Bun!) apps for Tigris
How Fly.io uses Docker (without Docker)
Переглядів 40 тис.4 місяці тому
How Fly.io uses Docker (without Docker)
Bounce traffic between Apps and Machines with fly-replay HTTP header
Переглядів 7194 місяці тому
Bounce traffic between Apps and Machines with fly-replay HTTP header
AWS Without Access Keys
Переглядів 1,7 тис.4 місяці тому
AWS Without Access Keys
Using AI to Boost Accessibility and SEO - Fly.io blog
Переглядів 1224 місяці тому
Using AI to Boost Accessibility and SEO - Fly.io blog
Chesstreaming: Game Ideas (Part 2)
Переглядів 1514 місяці тому
Chesstreaming: Game Ideas (Part 2)

КОМЕНТАРІ

  • @Jebusankel
    @Jebusankel Годину тому

    Shouldn't a decentralized orchestrator be called a choreographer?

  • @erkintek
    @erkintek Годину тому

    So flyctl keep track of empty workers to kill or in loaded region to create more workers?

  • @arieheinrich3457
    @arieheinrich3457 Годину тому

    Very intresting use case. Isnt this basically the same as the inner engine of GitLab / GitHub / ADO when they run pipelines with a pull instead of a push. Its exactly how runners / agents listen to broadcasted events from the main "API" service.

  • @Rachelebanham
    @Rachelebanham 2 години тому

    nice - out of interest have you open-sourced flyd?

  • @MadalinIgnisca
    @MadalinIgnisca 2 години тому

    😂😂😂😂 I did not know if to cry or to laugh. U know that real system engineers will not believe you?

  • @Dom-zy1qy
    @Dom-zy1qy 3 години тому

    So you wanted to perform bin-packing, but treat the machines' resources (vcores, ram, etc.) as the "capacity" of a container? When you were talking about how it's not ideal to try and fit as many applications as possible on a single worker, did "fit" refer to the size of the fly machine images on disk, or the expected resource utilization of an application? (Unsure how you could even predict resource utilization without relying on probalistic methods anyway, so I'm assuming it was the former)

  • @TannerBarcelos
    @TannerBarcelos 3 години тому

    This is a really cool approach! The bidding system is really clever.

  • @kaankeskin2214
    @kaankeskin2214 4 години тому

    thanks for watching? thanks for explaning :)

  • @FunkyELF
    @FunkyELF 5 годин тому

    I need to start playing with Firecracker. GitLab runners use Docker but the isolation isn't great. For instance running Docker within Docker isn't supported unless you expose your Docker socket from the host into the container (which isn't secure).

  • @h3techsme
    @h3techsme 7 годин тому

    Nice speed run through the problem space! 🏁

  • @DennisIvy
    @DennisIvy 7 годин тому

    Awesome video Annie! 🤩You are killing it!

  • @RaviPrakash-ix9dd
    @RaviPrakash-ix9dd 12 годин тому

    Kubernetes is dying. Our DevOps Team Stopped Using Kubernetes and They are Is Happier Than Ever

  • @sidiocity
    @sidiocity 2 дні тому

    Can we use any llama based model ? In the destination xan we use the llm we have downloaded ? Imean the custom llm based on llama ?

  • @mrjvc
    @mrjvc 4 дні тому

    Fantastic - great coverage of these tools

  • @madsciai
    @madsciai 4 дні тому

    Hi, thanks for these helpful videos to a seasoned programmer but Docker newer-ish. The video is about a year old, and I'm wondering if fly still doesn't support Silicon Mac use in build architecture, as I'd imagine you would have some support considering the GPUs available? Otherwise not sure how they're running with Intel.. do tell

  • @Beyblade0battles
    @Beyblade0battles 4 дні тому

    Great video! thank you

  • @KentCDodds-vids
    @KentCDodds-vids 4 дні тому

    Loved this chat! I've been using LiteFS on fly for years and it's been great!

  • @mwwhited
    @mwwhited 5 днів тому

    Virtualization doesn’t prevent hardware vulnerabilities.

  • @ramonpereira4460
    @ramonpereira4460 5 днів тому

    I was just wondering what rad means.

  • @dgdev69
    @dgdev69 5 днів тому

    I am a frontend engineer. I am learning SQLITE I love it just because of its simplicity.

  • @nightshade427
    @nightshade427 5 днів тому

    you can use libsql if you want sqlite but want local mode, server mode, distributed mode, replicas, embedded replicas, muti database schema management, etc

  • @awksedgreep
    @awksedgreep 5 днів тому

    Excellent show. Thanks for taking the time to put this together.

  • @JacksonMeires
    @JacksonMeires 6 днів тому

    Thanks for the tip, it helped a lot

  • @JohnNunemaker
    @JohnNunemaker 7 днів тому

    The rails people aren't recommending running your own database really. Most of us push people to use Crunchy or similar. It's more for stateless app/job servers.

  • @olddohko9037
    @olddohko9037 8 днів тому

    Watch out, the company I work for used their service, got scammed and it's about to go bankrupt. Lot of people are going to lose their jobs

  • @ha5anmukhlis
    @ha5anmukhlis 9 днів тому

    k8s would be cool if it weren't so expensive once you want to host it somewhere.

  • @adrianosleipnir854
    @adrianosleipnir854 10 днів тому

    Fly is great. But yes sometimes we need Kubernetes

  • @saiskanda
    @saiskanda 10 днів тому

    Whoa 4.7 million views

  • @VincentLAnderson
    @VincentLAnderson 10 днів тому

    Virtual machines are no more secure than containers and both technologies increase attack vectors by introducing unnecessary libraries and vulnerabilities either specific to the virtual machine employed or the containerized operating system. I know you won't do this, but just use Cosmopolitan C and it's clever way of employing PKZIP files which delivers all of the same benefits with less risk and greater portability across multiple architectures. If you feel you still need to run everything in a VM after exploring it, at least just compile your virtual machine in it. ;)

  • @venir_dev
    @venir_dev 11 днів тому

    A note about the first statement, about serverless and FaaS: they aren't synonims, and they can't be really interchanged. There are approaches that let you build a FaaS infrastructure on a bare metal server. So in cases like this there's an abstraction that hide most of your bare metal infrastructure details, but you're not committing to a fully managed and pay-per-use instance like an AWS lambda service would do.

    • @ultravioletiris6241
      @ultravioletiris6241 8 днів тому

      Trying to do more research on this. Can you point to any sources? I found a presentation on doing this with rootless podman from a couple years ago but thats it so far. Is there a way to achieve functions as a service in kubernetes? Is there a specific OS or application you know of that helps achieve FaaS on prem? Thank you

    • @venir_dev
      @venir_dev 8 днів тому

      @@ultravioletiris6241 it's raw, it's early, but someone is trying to achieve FaaS without committing to pay-per-ore. Search for "Apache OpenServerless" incubating project.

  • @venir_dev
    @venir_dev 11 днів тому

    one single person against the serverless mafia - go Chris!!

  • @simonmartinelli
    @simonmartinelli 11 днів тому

    Great video. But better beware of Betteridge's law 😉 en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines

  • @jutublizard
    @jutublizard 11 днів тому

    your OF account?

  • @roganl
    @roganl 11 днів тому

    You know why they give the first FKS implementation away on free tier? Because they don't really give two FKS for k8s.....

  • @pookiepats
    @pookiepats 11 днів тому

    hmm... i was mislead.

  • @thatMattW
    @thatMattW 11 днів тому

    Great video. Also, love the poster in the back ;) Pyra is the name of my ai assistant lol

  • @mayikx
    @mayikx 12 днів тому

    I tried this platform, and this platform is unnecessary complex. However, their videos are awesome

  • @cykatroopa
    @cykatroopa 12 днів тому

    It absolutely makes sense to run K8S even at small scale due to its nice ecosystem. You really get a lot of functionality for free. If you're not sure, give it a good try. It's not hard to use or install. You get free HA if you can abstract storage volumes. I love K8S.

  • @RealYethal
    @RealYethal 12 днів тому

    No mention of NixOS, 2/10

  • @pixelfairy
    @pixelfairy 13 днів тому

    Im glad you give us FKS. But you might want to run that one by marketing. How about FlyK? You can pronounce it flick.

  • @vicaya
    @vicaya 13 днів тому

    If you're in a position to ask the question these days, the answer is probably yes, because the alternatives are probably worse. It's fairly easy to use k8s, esp. k3s for pretty much anything even on a single node. The complexity of k8s itself is vastly exaggerated, while the complexity of devops in general is often underestimated.

  • @JAMaitim
    @JAMaitim 13 днів тому

    FKS demo please

  • @gearboxworks
    @gearboxworks 13 днів тому

    So, the alternative to K8s is to couple one's application to the Fly service? Please correct me if my takeaway was wrong. I'm not arguing everyone needs K8s, but the only alternative you mentioned was to use Fly's built-in functionality instead, which seems a lot like jumping out of the frying pan (of complexity) and into the fire (of proprietary dependencies.)

  • @digitalsparky
    @digitalsparky 13 днів тому

    Oh no, we fks'd up...

  • @lulzmachineify
    @lulzmachineify 13 днів тому

    Kubernetes saves us metric bucketloads of money compared to renting PaaS/SaaS for things like kafka, cassandra, postgres etc. It's amazing for running infrastructure on

  • @HerbieBancock
    @HerbieBancock 13 днів тому

    Kubernetes exists solely to give all the H1Bs something to do.

  • @vilpamisot
    @vilpamisot 13 днів тому

    What means high density, can someone please elaborate?

  • @fredio54
    @fredio54 13 днів тому

    Interesting approach :-)

  • @OldKing11100
    @OldKing11100 13 днів тому

    I feel like horizontal scaling only becomes an issue early on with single threaded dynamic languages like Python/Node apps. With something simple like Go, html/template, and SQLite(WAL) it's amazing how far a monolith can be pushed on a VPS. Down the road under heavier loads though K8s can be nice if I'm not managing it so something like FKS could be a good option.

  • @fredio54
    @fredio54 13 днів тому

    I'd still have a container on containerd inside your firecracker VM if I was using your stuff. I assume like Lambda this is an option. You'd be poor business people if you didn't support that model.