This could have proably have saved me two days, not on the Linux part which only took a few minutes, but with this sh*tty Windows. I had to update it first to 2H22 because otherwise my GPU wasn't avaiable under WSL2. But it couldn't do because of some generic error with some number that can have like 100 reasons and after wasting a lot of time with fix-your-windows guides, I found an assistent tool that finally updated my windows. Than I had another day fun with Ollama Webui in terms of portmapping from its docker container to the Windows host and then the LAN. I got it working first but another Windows update broke it as it installed some new iphelper service at port 8080 which is the exact port under which WebUI is exposed. With the Guide of ChatGPT I finally disabled that and installed a reverse proxy. What a pain! I'll almost certainly make a dedicated Linux AI machine this year. Btw. you could have mentioned WebUI for Ollama perhabs unless I just missed it when you did. ;-)
@@snuwan Yeah, I'm just in the process of deciding whether I bite the bullet and go all-in to get a car-prized workstation GPU for like 4x its actual worth or if I try old cheap server GPUs. It'd be 2x24GB P40 or M40 or even 4x. I must be sure though that this can run a 70b model in a decent manner.
@@PromptEngineer48 I do pip install ollama in my system but I'm trying to automate my ollama to take inputs using voice and in give that input in terminal using os.system but after activating ollama it not taking os.system input
I love Ollama. It's integrate is easier. esp. now with embedding models, integration with colabs, windows, linux, mac (local and on the cloud) doing multiple projects etc. I love Ollama again
It is very nice that finally they added support to windows. But this preview they have is really slow. It is slower than when we run it using wsl subsystem in windows. I have used it successfully within wsl subsystem. A lot of users have been seeing the same thing. Hopefully they will address these performance issues. But I am glad it is finally supported in windows.
@@PromptEngineer48 thanks your comment told me loads of information....... What makes it better LM studio was just one example of many pieces of software including releasing it in a premade conda environment. personally I don't have a problem running any llm as I have my own home built GPU server rack that I mainly use for cracking hashes... I'm curious because it would be nice to carry one around in a laptop and not worry about it compromising the CPU GPU and the ram which it does do to the average consumer laptop!!!!. If you play with autogen or open interpreter I suggest you do it on a device you don't use on a regular basis as they cause broken links and compromise the OS security..
Hey thanks for the vid! Hey maybe im missing something (and maybe its due to it being just the preview too) but do you know how I can watch the real-time logs? When using this via docker I was able to see the backend of it and it really helped with troubleshooting. IM sure theres a way im just blind haha
This could have proably have saved me two days, not on the Linux part which only took a few minutes, but with this sh*tty Windows. I had to update it first to 2H22 because otherwise my GPU wasn't avaiable under WSL2. But it couldn't do because of some generic error with some number that can have like 100 reasons and after wasting a lot of time with fix-your-windows guides, I found an assistent tool that finally updated my windows. Than I had another day fun with Ollama Webui in terms of portmapping from its docker container to the Windows host and then the LAN. I got it working first but another Windows update broke it as it installed some new iphelper service at port 8080 which is the exact port under which WebUI is exposed. With the Guide of ChatGPT I finally disabled that and installed a reverse proxy. What a pain! I'll almost certainly make a dedicated Linux AI machine this year. Btw. you could have mentioned WebUI for Ollama perhabs unless I just missed it when you did. ;-)
Thanks for going through all the pain and sharing your story. :) We all will benefit from your experience.
I also plan to build a dedicated Linux Ai machine with a good GPU. I think most will run faster on Linux.
RIght. For me I host Ollama on a Cloud GPU for keeping my projects running.
@@snuwan Yeah, I'm just in the process of deciding whether I bite the bullet and go all-in to get a car-prized workstation GPU for like 4x its actual worth or if I try old cheap server GPUs. It'd be 2x24GB P40 or M40 or even 4x. I must be sure though that this can run a 70b model in a decent manner.
I like the GPU option as things evolve and my bought system gets old very quickly
Please make a tutorial on web UI on windows
Why im not able to use ollama In my vs code custom environment
!pip install ollama I think u should not have missed this.
@@PromptEngineer48 I do pip install ollama in my system but I'm trying to automate my ollama to take inputs using voice and in give that input in terminal using os.system but after activating ollama it not taking os.system input
@@GlosbeNatural are you working in your local system or colabs notebook etc.??
@@PromptEngineer48 local system
can I chat with my docs with windows version?
Yep
Thank you for the video! This is so exciting!
Glad you enjoyed it!
Thank you so much for all your videos. What is your thinking on using Ollama vs. LocalAI?
I love Ollama. It's integrate is easier. esp. now with embedding models, integration with colabs, windows, linux, mac (local and on the cloud) doing multiple projects etc. I love Ollama again
It is very nice that finally they added support to windows. But this preview they have is really slow. It is slower than when we run it using wsl subsystem in windows. I have used it successfully within wsl subsystem. A lot of users have been seeing the same thing. Hopefully they will address these performance issues. But I am glad it is finally supported in windows.
True. WSL is faster as of now. It's just a preview. Let's just wait for a week and see the development! Wait wont be that long.
anybody having ollama run painfully slow on windows 11? if run in linux/mac its so fast, any suggestion how to fix slowness?
I agree with that. there is nothing we can ado apart from ripping the Ollama code itself. or wait for the Ollama developers to make the changes.
Can you make a tutorial for Maestro
by Maestro, u mean mobile UI testing framework??
Very cool!
Thanks !
So what's new about this we've been able to run llama in LM studio and run open interpreter an auto Gen.???????
Lm studio is nothing compared to ollama
@@PromptEngineer48 thanks your comment told me loads of information....... What makes it better LM studio was just one example of many pieces of software including releasing it in a premade conda environment. personally I don't have a problem running any llm as I have my own home built GPU server rack that I mainly use for cracking hashes... I'm curious because it would be nice to carry one around in a laptop and not worry about it compromising the CPU GPU and the ram which it does do to the average consumer laptop!!!!. If you play with autogen or open interpreter I suggest you do it on a device you don't use on a regular basis as they cause broken links and compromise the OS security..
Andy Kaufman?
Hey thanks for the vid!
Hey maybe im missing something (and maybe its due to it being just the preview too) but do you know how I can watch the real-time logs? When using this via docker I was able to see the backend of it and it really helped with troubleshooting. IM sure theres a way im just blind haha
Not direclty but 3 party services like Splunk could be used.
Oh good idea thanks alot for that!@@PromptEngineer48