Offline AI on iOS and Android
Вставка
- Опубліковано 19 вер 2023
- I compiled a PyTorch AI model to run locally on iOS & Android! This keeps conversations private, free, & data compliant (both GDPR & HIPAA). In this video tutorial, i'll show you exactly how I did this for my ongoing healthcare project titled Doctor Dignity (previously titled DoctorGPT) so that you can do the same thing for any AI model, particularly Large Language Models. We start by building a simple language model in Python that runs on an Intel CPU, then accelerate it using CUDA for Nvidia GPUs, Metal for iOS GPUs, Vulkan for Android GPUs, and finally the Tensor Virtual Machine for both iOS and Android. We have a lot to GPU Programming cover in this video, comparing and contrasting different GPU programming techniques (computation vs allocation, abstraction vs transformation, stochastic optimization vs gradient based optimization). My next goal is to get a 3 billion parameter model to pass the US Medical Licensing exam and run on mobile devices. Don't worry if you're a beginner to all this, just sit back relax and enjoy the ride, I tried to make this as accessible as possible to all developers.
Star Doctor Dignity on GitHub:
github.com/llSourcell/Doctor-...
Please Subscribe! That's what keeps me going.
Want more AI/ML education? Connect with me here:
X: x.com/sirajraval
Instagram: / sirajraval
Facebook: / sirajology
Linkedin: / sirajraval
Tiktok: / sirajraval
Doctor Dignity Original Video:
• DoctorGPT: Offline & P...
Doctor Dignity iOS app: (Warning: this is a beta with bugs, please do not have high expectations, i promise it will get a lot better next month)
apps.apple.com/us/app/doctor-...
I Built a Trading bot with Bing and ChatGPT:
• I Built a Trading Bot ...
I Built a Sports Betting Bot with ChatGPT:
• I Built a Sports Betti...
I Built a Trading Bot with ChatGPT:
• I Built a Trading Bot ...
Watch ChatGPT Build an AI Startup:
• Watch ChatGPT Build an...
Watch ChatGPT Build a Finance Startup:
• Watch ChatGPT Build a ...
Learn Machine Learning in 3 Months:
• Learn Machine Learning...
Join my AI community:
gptschool.app Sign up for my AI Sports betting Bot, WagerGPT! (500 spots available):
www.wagergpt.co
can someone please explain to me about what is going on in this video
Great video Siraj. I like that you start talking about the tech from scratch and build up. However, a non-technical person will be lost for sure as there are a ton of concepts that require deep understanding, so there is not much point to it. Thank you for making everything OSS btw, wish you the best on your Doctor Dignity project!
Sounds awesome. Good luck!
Nice video, with detailed explanation. I have been having problems understanding Cuda and the rest before.❤
You are a great teacher, and based on what youve set out to do an absolute legend also lol. I really hope you get your dream off the ground. Cant wait to see your android tutorial and maybe something on finetuning and training LLMs so i can get my ideas off the ground. Can i request you clearly go through the steps if its not to inconvenient when u do them, im not a coder but id love to be able to follow along coz i think i almost could i think seeing ur so good at explaining, as long as theirs no assumption that people already know what things are and where they go, u know like following chatgpt, sometimes u gotta ask it where to put the code it spits out. Anyway thanks so much for this ive never seen someone appeal to the layman, really makes me feel like i could learn this stuff with some dedication and a dream.
Any progress on connecting SQL database RAG or source citation into mlc chat? Currently most interested in this for Android. Also this is such a great project. Big respect for democratizing access to this vital knowledge.
Amazing work!
Not only is it free but it helps me if im injured!😍😍 Thank you siraj.
Whens android version coming ?
Is it a possibility for you to implement support for older versions of iOS?
Still so excited for part 2 of this. Dont forget you were considering doing tutorials that people can follow step by step as well as your 5minute videos. I heard u say that somewhere years ago hehe
You should note in the app description that this will fail to run on iPhones with less than 6 GB of memory (assuming it runs on them). You should only mark it as available for download to the appropriate phones as I downloaded this on my iPhone 11 and it immediately said it can't allocate 3814 MB of VRAM. No surprise there on a 4GB phone, but I assumed this needed maybe 2 GB for the model - based on your monitoring app in the video showing 2 GB VRAM utilization.
Whatever happened to helping us make money with chat gpt? Thanks love your videos!
wow... definitely following this and gonna try build my own but based on Tongan culture. Sick
Thank you, waiting on the android version
love your voice bud
I love your channel is there anyway to hire you for a task we have to solve in my company as well as I would like to sponser if possible? Anyways great job!
please we want the android video
Can you accept the donation from outside of the US , I tried to join the membership, but google pause my membership after a month even declining my payment anymore, so strictly
This is gold
Can anyone provide the apk link?
Great how about loading llm on my workstation to do the heavy lifting . and control it from my android
You were zoomed in the entire time and moving around so much it was sometimes hard to follow. You're an incredible teacher though
what is this?
what about running it in a flash drive ? Would that work?
You are a very good UA-camr.
"for Android and iOS" -> "I can't get it to work on Android" so for iOS, You couldn't even get past setup.
"I need to set the right Java SDK working on that"! LOL 😂 WTF?!
please integrate memgpt
please ill seriously cry i cant figure it out
🔥🔥🔥
wtf??