Thank you. At last a tutorial about setting this up, that worked for me. I have zero experience with docker, and now I have it running on my computer! I'd like to learn even more about the usses of docker (just for hobby purposes at home).
I am on windows 11. I was having issues with it not loading anything when I typed a question. It was thinking and that was it. I found that after I went into the Admin Panel > Settings > Connections and turned off (OpenAI API) it started working. Just in case anyone else has this same issue.
Thank you! really like the simple step-by-step approach in an easy to follow ways. Managed to successfully set this up! Subscribed to your channel to support you!
@@rudhrateja4967 docker and vm are unnecessary overhead when there are alternatives to run stuff natively. I built a pretty solid a.i. ollama - with web UI no vm or docker! Works fast.
Thank you so much for this incredibly helpful tutorial! I really appreciate you taking the time to walk through all the steps in detail. Running a private AI chatbot on my own computer is going to be a game-changer. Can't wait to dive in and try it out!
Your vid is about the best I have seen for dummies like me. Step by step, nothing in between not added if you do this nothing but so this next. Write it out. It would be nice if it's written out where I can copy it and print it out. How do you copy and paste in terminal?
I installed it on my linux box running on 12600k and 64GB RAM, its a tad bit slower but completely workable till 35B model. Nice fun tool. I will add a GPU to enhance perf but waiting on Nvidia RTX 5000 series to see how much better they are than 4000 series. But its workable if you have a decent PC.
It took a few tries but I finally got the Linux items and Vm to run.. Could not find the VM setting in the bios without a few videos and one finally showed on a mother board close enough to mine and poof, one more restart and docker is running Llama3!!! Thank You!
Hello, does this Private GPT have something similar to ChatGPT’s custom instructions feature, or can I create separate specialty setups for statistics, Authoring, and/or Coding (COBOL, C#, Fortran, etc.)? Can writing (authoring ) samples of mine be uploaded for specific chat sessions?
I followed everything step by step and have it running on my windows 11 great. Time will tell how useful it truly is. Is there away to optimize some models? 25 gigs of memory to run some models and slow response seems a bit much.
The data sent to the chat bot is also sent to Meta, so if you're doing this for privacy purposes, think again. I mean it's just explicit cause you're using external open-source programs to do this. But I'll leave here the reminder aswell. "" We also use information shared on Meta's Products and services.""
Meta has an ai website called meta.ai which also uses llama 3. That’s where that quote comes from. This is a different thing. You can literally use this in a computer that has no access to the internet after you install it
It works with a lot of laptops. But the speed will vary. I think if you have a gaming gpu, it should be decent speed. The big speed issue I noticed is if I upload a lot of files to my knowledge base
yes any laptop with gaming gpu on it , ai loves nvidia gaming gpu even if you have 4gb vram like me , im able to run ai models but make sure if you have less then 8gb to download 1.5b and 3b and 7b other then that you 'll see alot of problems .
Super helpful tutorial! Worked like a charm! Question - I'm on a windows machine. What are the BASE specs and BEST specs for a windows machine to make this process as fast/efficient as possible? Thank you. Thank you. Thank you.
Can the ip for the machine this is running on be shared to others on the same internal network? I assume so... but just making sure before I give this a try.. Also, is there a way to train these models from here or no? Thanks for this BTW.. subbed.
my pc has not been asking me for my name it just saves and moves on. now when i get to the terminal it tells me ''docker : The term 'docker' is not recognized as the name of a cmdlet, function, script file, or operable program.''
Would this functionality allow me to upload screenshots of content and have the LLM be able to read through and respond to what it sees in those images?
If your computer doesn’t have the specs to handle it, you’ll be limited to using this on meta.ai. Running it private requires a beefy computer right now
Just ran across your vid. It's the one I need. However This code you need to copy and pase does not appear. I uninstalled ollama and reinstalled it no code. In terminal I typed "ollama did the space ollama3 nothing works. It does not recognize ollama 3. What am I doing wrong. I am a NON-GEEK. smile.
It was really easy to install, can you show how you use this with gcolab, using ngrok as a tunnel, and dockdesktop running locally and ngrok endpoint being the openWEBUI? This would allow everyone to use it on mobile as well, and also include setup to talk directly with the different models that support it, and explain how uncensored modell can effectively help you in ways that chatGPT cant..
Would I be able to use this functionality (or is there a better approach) to review/summarize/respond to specific, esoteric websites that are not typically included in the LLM databases? Could I provide specific URLs and ask for detailed feedback on what's found at the given URL?
the API key added itself automatically. It somehow picked the api key from somewhere.I didn't do anything. 😂 I was searching for this so that I could host an LLM locally and do organization-related tasks. Now I'm having doubts.
Thank you very much for such a clear and concise guide. I wish to use this to analyse confidential interview transcripts as a way to check my own manual analysis and interpretation of the data and this needs to be done offline so this is perfect. Please can you suggest if there is a model that would be best for this kind of analysis? BTW watching you use terminal took me back to my youth using MS DOS!
I don’t believe you can with this set up. The only LLM options I know for server based environments are the name brands like Gemini, and ChatGPT on azure OpenAI service. I’m sure a developer can set this type of thing up on a private server but I don’t get that far in my course
the knowledge base if your own documentation. Like blog posts your wrote, company policy, etc. It already has knowledge without your own documentations.
iv tried it it works well but mixtral has a extreme slow response time compare to lama for me and i have a 9950x with a 4090 gpu to be exact im running mixtral 8x22b but regular mixtral runs fast and smooth
Ok, so once everything is install and you play with it and it all works, what do you do the next time you want to run it? Let's say, my computer has been turned off and then on again. What do I need to run to start the ai again? Just docker? I would like instructions in how to restart everything. What are the step?
@@SkillLeapAI Thank you! Just got Docker up and running. Alright, everything is up and running! Unfortunately on my 2019 ryzen apu even phi3 is slow as molasses. Oh well, I thought that might be the case. I have to give you great props on this tutorial. I've seen a number of these olama-webui install tutorials and your tutorial was the most clear. The best! Thanks for your help!
If i host this in a private server for my team .. will my private files and data i used for this llm gets stored or trained by the models of the other person using the llm other than me and my team..?
I don’t see any limit but it hasn’t worked for me with more than 10. It lets me upload them but doesn’t give me useful responses. I’ll see if they have documentation on this.
Does Window Terminal work the same? I'm going to make a new video tomorrow with my Windows. My main computer is a Mac and the documentation said it was the same process.
I believe installing these apps turns your computer into a remote zombie that receives instructions to run a web based network. So in short your power is contributing to the network with out your knowledge. This is what bitcoin is doing… people think their computer was solving math problems but in reality they were part of a huge network that’s creating value with bitcoin…. I tried telling people but nobody understands how information works.. if you ever get bored look at what info is being shared from your computer without your knowledge…. Now imagine that times 1k! I have a computer that will turn on I the middle of the night if I sneeze… thats how sensitive it is.
Privacy. Lot of people can’t just upload documents to ChatGPT. It’s either for privacy, work rules, sensitive info, etc. this is local and private. You’re not sharing anything with any company
Of course it’s easy to follow your good step by step instructions, but the real magic is as usual happening behind the curtains… 🪄 So how’s ones privacy guaranteed and kept private in the long run…?
Its not at all the same for windows with setting up. Plus it looks like the weblinks you provided have updated. I did not get any window to copy and paste code. When I downloaded Ollama3 it was an application in my downloads to install. Which I did. But I do not know where it installed it and how to access it in windows terminal. I downloaded installed Docker from the link you provided, and it looks nothing like the docker you are using and is asking about containers and installing containers. You should really change this video to how to install on a MAC, because it is not at all helpful for PC.
So interesting thing just happened ...Chatbot got math wrong when I was trying to ask it an easy conversion of how much of a 3qt bag of activated charcoal to add to a 2.6 pound bag of coco coir/perlite 79/30 mix to get a 70/30/10 mix ..the answer is the entire bag ...first it kept thinking it was 2.6lbs of coir and perlite each ...but the big thing was it tried to tell me 3 quarts is equal to 2.086 pounds ...the. it admitted it made a mistake...I didn't think thay was possible?
I have it running in windows 10.. It's so easy even I did it in minutes and wrote half a book of 20 chapters. Life is simple when you think and believe it is simple.
Perhaps this is too old, but there are a number of places where your explanations do not match reality. Firstly you use Apple as your point of reference. I'm on windows so things don't match in some places. Next, your instructions about installing Docker don't look much like what is presented on the website. There is a missing step about installing WSL and updating it, but that was not a problem. When it came to running Docker, things were so radically different that I just have no patience to continue with your dead bang drop dead easy instructions. There is no where enough time to fight through it. I would have to find another source of instructions, or do a very deep dive into what the hell Docker is and how I might want it set up. So for now, thanks for the effort, but I'm off to find another explanation to run the LLM. Perhaps one that does not involve so many programs and logins and setups.
Get access to our new Private AI chatbot course as well as twenty other AI courses here: bit.ly/skillleap
I have this setup behinds a VPN on my local machine at home and it works like a charm on my phone remotely!
Any tut on how to do that?
@nicolaspietri4730
Please tell me how to do that ?
@@masdiqieset up a vpn like open vpn or similar product.
Try to replace localhost on ur phone with ur local pc adress it might work
Thank you. At last a tutorial about setting this up, that worked for me. I have zero experience with docker, and now I have it running on my computer! I'd like to learn even more about the usses of docker (just for hobby purposes at home).
man, when easier is better, thank you so much for your assistance with this.
This is working like charm, you deserve 100, thank you so much.
that's promissing. I'll try it out tonight.
Loved this video! I will be leveraging your instructions with my students as I introduce AI into their course content!
I am on windows 11. I was having issues with it not loading anything when I typed a question. It was thinking and that was it. I found that after I went into the Admin Panel > Settings > Connections and turned off (OpenAI API) it started working. Just in case anyone else has this same issue.
may I know what do you mean by admin panel??
Thank you
Thank you! really like the simple step-by-step approach in an easy to follow ways. Managed to successfully set this up! Subscribed to your channel to support you!
Thank you
I hate using docker, as it is too much overhead, but most other methods I tried did not work, or were problematic
thanks for this video
Docker is way better than other vm box
@@rudhrateja4967 docker and vm are unnecessary overhead when there are alternatives to run stuff natively. I built a pretty solid a.i. ollama - with web UI no vm or docker! Works fast.
Very well made video for anyone to get started quickly 😊👍🏻
Thank you so much for this incredibly helpful tutorial! I really appreciate you taking the time to walk through all the steps in detail. Running a private AI chatbot on my own computer is going to be a game-changer. Can't wait to dive in and try it out!
I just installed the 7B 4 GB model, very lightweight and useful in grid down scenarios.
Thank you very much for this comprehensive and handy tutorial.
My pleasure
Downloaded and working perfectly. Thanks.
Very thorough breakdown, thank you!
Your vid is about the best I have seen for dummies like me. Step by step, nothing in between not added if you do this nothing but so this next. Write it out. It would be nice if it's written out where I can copy it and print it out. How do you copy and paste in terminal?
Definitely like, everything work, thanks man
This is great! Thanks for sharing!!!
I installed it on my linux box running on 12600k and 64GB RAM, its a tad bit slower but completely workable till 35B model. Nice fun tool. I will add a GPU to enhance perf but waiting on Nvidia RTX 5000 series to see how much better they are than 4000 series. But its workable if you have a decent PC.
Thank you for this easy-to-follow tutorial. How would you deinstall a model that is to big to work with your hardware equipment?
It took a few tries but I finally got the Linux items and Vm to run.. Could not find the VM setting in the bios without a few videos and one finally showed on a mother board close enough to mine and poof, one more restart and docker is running Llama3!!! Thank You!
sorry but after i shut my pc down how do i get it started under the webui again?
Just launch docker and Ollama again and then click the IP link in docker
Can it connect to sharepoint sites to read from documents on the sites?
Hello, does this Private GPT have something similar to ChatGPT’s custom instructions feature, or can I create separate specialty setups for statistics, Authoring, and/or Coding (COBOL, C#, Fortran, etc.)?
Can writing (authoring ) samples of mine be uploaded for specific chat sessions?
Can we train it for our jobs, school or projects ?
I followed everything step by step and have it running on my windows 11 great. Time will tell how useful it truly is.
Is there away to optimize some models? 25 gigs of memory to run some models and slow response seems a bit much.
thank you a lot! I will try it in my own PC
Since this doesn't like running on potatoes, how do i uninstall this thing, do i just uninstall docker and ollama?
Inside of the settings of OpenWebUI, you can uninstall each LLM you installed first. Then you just uninstall docker and ollama.
why do the code dont com upp after i install ollama in step 1
from 2;18 i didn't get the code. Could this be the initial reason why it's not responding? if so, alternatively where can i get the code?
The data sent to the chat bot is also sent to Meta, so if you're doing this for privacy purposes, think again.
I mean it's just explicit cause you're using external open-source programs to do this. But I'll leave here the reminder aswell.
"" We also use information shared on Meta's Products and services.""
Meta has an ai website called meta.ai which also uses llama 3. That’s where that quote comes from. This is a different thing. You can literally use this in a computer that has no access to the internet after you install it
what kind of laptop hardware do we need? will it work with normal office thinkpad without a high end gaming gpu for the laptop?
It works with a lot of laptops. But the speed will vary. I think if you have a gaming gpu, it should be decent speed. The big speed issue I noticed is if I upload a lot of files to my knowledge base
yes any laptop with gaming gpu on it , ai loves nvidia gaming gpu even if you have 4gb vram like me , im able to run ai models but make sure if you have less then 8gb to download 1.5b and 3b and 7b other then that you 'll see alot of problems .
Super helpful tutorial! Worked like a charm! Question - I'm on a windows machine. What are the BASE specs and BEST specs for a windows machine to make this process as fast/efficient as possible? Thank you. Thank you. Thank you.
Can the ip for the machine this is running on be shared to others on the same internal network? I assume so... but just making sure before I give this a try.. Also, is there a way to train these models from here or no? Thanks for this BTW.. subbed.
I enjoyed your video. thanks and kudos. What about updates? Should we uninstall and reinstall for an update?
turn off the wifi to make sure this data isnt going anywhere...until you reconnect to the internet.
my pc has not been asking me for my name it just saves and moves on. now when i get to the terminal it tells me ''docker : The term 'docker' is not recognized as the name of a cmdlet, function, script file, or operable program.''
Would this functionality allow me to upload screenshots of content and have the LLM be able to read through and respond to what it sees in those images?
Great video,never even knew about this stuff! Only problem, my computer memory isn't large enough. Where do i even start?
If your computer doesn’t have the specs to handle it, you’ll be limited to using this on meta.ai. Running it private requires a beefy computer right now
Just ran across your vid. It's the one I need. However This code you need to copy and pase does not appear. I uninstalled ollama and reinstalled it no code. In terminal I typed "ollama did the space ollama3 nothing works. It does not recognize ollama 3. What am I doing wrong. I am a NON-GEEK. smile.
It was really easy to install, can you show how you use this with gcolab, using ngrok as a tunnel, and dockdesktop running locally and ngrok endpoint being the openWEBUI? This would allow everyone to use it on mobile as well, and also include setup to talk directly with the different models that support it, and explain how uncensored modell can effectively help you in ways that chatGPT cant..
Would I be able to use this functionality (or is there a better approach) to review/summarize/respond to specific, esoteric websites that are not typically included in the LLM databases? Could I provide specific URLs and ask for detailed feedback on what's found at the given URL?
When I open the web UI along with my local LLM Lllama 2, I can see GPT-4 and GPT-4o in the choose LLM dropdown. Do you know what is causing this?
If you add OpenAI api under settings, those appear in the menu. That’s the only way I know of
the API key added itself automatically. It somehow picked the api key from somewhere.I didn't do anything. 😂 I was searching for this so that I could host an LLM locally and do organization-related tasks. Now I'm having doubts.
Thank you very much for such a clear and concise guide. I wish to use this to analyse confidential interview transcripts as a way to check my own manual analysis and interpretation of the data and this needs to be done offline so this is perfect. Please can you suggest if there is a model that would be best for this kind of analysis? BTW watching you use terminal took me back to my youth using MS DOS!
Llama 3 or Mixtral are the two models I recommend. You can use both at the same time and compare results
Nothing happens on my pc, i cvan download & install it, but it wont run. :(
So how do you make it use 2 ore more GPU:s?
How can I share this on a home server for family members to access?
Great video! Can I make this work using Perplexity?
No it won’t work with that one.
I just bought an HP laptop the day I saw this video. I'm stuck after downloading llama... it won't open after download for me to continue...
before I take your course: can I use this in a server environment, so all of my employees can work with the chatbotˋ?
I don’t believe you can with this set up. The only LLM options I know for server based environments are the name brands like Gemini, and ChatGPT on azure OpenAI service. I’m sure a developer can set this type of thing up on a private server but I don’t get that far in my course
Very very good. Thank you.
How do you get the files for knowledge base . If yiu want your bot to focus on certain things . Or overall knowlwdge
the knowledge base if your own documentation. Like blog posts your wrote, company policy, etc. It already has knowledge without your own documentations.
@@SkillLeapAI thank you. Ok so I need to research my own knowledge base. I'm guessing perplexity is good for that or any other
thank you so much sir🌸
iv tried it it works well but mixtral has a extreme slow response time compare to lama for me and i have a 9950x with a 4090 gpu to be exact im running mixtral 8x22b but regular mixtral runs fast and smooth
is it possible to use this uncensored?
Ok, so once everything is install and you play with it and it all works, what do you do the next time you want to run it?
Let's say, my computer has been turned off and then on again. What do I need to run to start the ai again? Just docker? I would like instructions in how to restart everything. What are the step?
Launch docker and click on the port to launch it. You can also press stop on docker to stop it
@@SkillLeapAI Thank you! Just got Docker up and running.
Alright, everything is up and running! Unfortunately on my 2019 ryzen apu even phi3 is slow as molasses. Oh well, I thought that might be the case.
I have to give you great props on this tutorial. I've seen a number of these olama-webui install tutorials and your tutorial was the most clear. The best!
Thanks for your help!
is the MacBook Air m3 10 core gpu good enough for the last thing?
Yea I have M3 and have had no issues yet
@@SkillLeapAI thanks a lot
Why wouldnt you use something like LM Studio instead of doing this via localhost and a docker instance?
That's a very good option too. I was more familiar with this option and it's the version I've set up on my computer.
can you connect stable diffusion to lm studio
Can you talk with Ai ,your plans day by day in this programme
If i host this in a private server for my team .. will my private files and data i used for this llm gets stored or trained by the models of the other person using the llm other than me and my team..?
What’s the limit to the number of documents you can upload? Is there a limit on size or count?
I don’t see any limit but it hasn’t worked for me with more than 10. It lets me upload them but doesn’t give me useful responses. I’ll see if they have documentation on this.
Windows Powershell update requires installs alternate powershell.
Does Window Terminal work the same? I'm going to make a new video tomorrow with my Windows. My main computer is a Mac and the documentation said it was the same process.
Im so frustrated... when i run the app after intalling it, nothing happens! nothing opens! what is going on?
I believe installing these apps turns your computer into a remote zombie that receives instructions to run a web based network. So in short your power is contributing to the network with out your knowledge. This is what bitcoin is doing… people think their computer was solving math problems but in reality they were part of a huge network that’s creating value with bitcoin…. I tried telling people but nobody understands how information works.. if you ever get bored look at what info is being shared from your computer without your knowledge…. Now imagine that times 1k! I have a computer that will turn on I the middle of the night if I sneeze… thats how sensitive it is.
You are amazing!
Thank you!
@@SkillLeapAI Yw, keep making these!
Install a ton of apps to get the functionality. Why would I not go directly to chatgpt?
Privacy. Lot of people can’t just upload documents to ChatGPT. It’s either for privacy, work rules, sensitive info, etc. this is local and private. You’re not sharing anything with any company
I made this video for those people that need a private solution due to work rules or privacy concerns
Of course it’s easy to follow your good step by step instructions, but the real magic is as usual happening behind the curtains… 🪄 So how’s ones privacy guaranteed and kept private in the long run…?
can you upload a custom ai
Would you make a video about interact with documents, Pls
You got it
I have terminal on my phone, so actually it would work then to
Windows only ?
Its not at all the same for windows with setting up. Plus it looks like the weblinks you provided have updated. I did not get any window to copy and paste code. When I downloaded Ollama3 it was an application in my downloads to install. Which I did. But I do not know where it installed it and how to access it in windows terminal. I downloaded installed Docker from the link you provided, and it looks nothing like the docker you are using and is asking about containers and installing containers.
You should really change this video to how to install on a MAC, because it is not at all helpful for PC.
how beefy should my computer be?
Docker doesnt work on older macs.
God truly blessed us
even the "Hi" did not give me a response, only said it was not recognized.
So interesting thing just happened ...Chatbot got math wrong when I was trying to ask it an easy conversion of how much of a 3qt bag of activated charcoal to add to a 2.6 pound bag of coco coir/perlite 79/30 mix to get a 70/30/10 mix ..the answer is the entire bag ...first it kept thinking it was 2.6lbs of coir and perlite each ...but the big thing was it tried to tell me 3 quarts is equal to 2.086 pounds ...the. it admitted it made a mistake...I didn't think thay was possible?
I’ve heard language models aren’t good at math. it’s a language model not a math model
Not even close to the same install/setup process on Win 11 PC.
I’ll run it on my windows machine tomorrow
I have it running in windows 10.. It's so easy even I did it in minutes and wrote half a book of 20 chapters. Life is simple when you think and believe it is simple.
error 500????
can u upload pics?
Perhaps this is too old, but there are a number of places where your explanations do not match reality. Firstly you use Apple as your point of reference. I'm on windows so things don't match in some places. Next, your instructions about installing Docker don't look much like what is presented on the website. There is a missing step about installing WSL and updating it, but that was not a problem. When it came to running Docker, things were so radically different that I just have no patience to continue with your dead bang drop dead easy instructions. There is no where enough time to fight through it. I would have to find another source of instructions, or do a very deep dive into what the hell Docker is and how I might want it set up. So for now, thanks for the effort, but I'm off to find another explanation to run the LLM. Perhaps one that does not involve so many programs and logins and setups.
Meta and Privacy don't go hand in hand.
Completely private chat client inside spying Chrome inside spying macOS. Nice.
Not for M3…
I have M3 mac
@@SkillLeapAI interesting it errored out and said something about the intel chipset not compatible. I’ll have to retry again
When installing it from the Docker website, choose Apple silicon instead of intel.
A "Windows Preview" version that only installs Ollama over and over again. Thanks for catering to Mackies only. UNsubscribed.
Are you a troll. I installed it on windows 10 easily. Running perfect and fast.
Using the same steps, I successfully installed and run it on Windows too. Hope this helps.
@@ProfessorCrumbs65 Yes, Win10, but not Win11. Doesn't work for me either
“you just lost an important customer” ahh
Oh, man , final solution!
Thanks for this great video!