This is awesome. Thanks for the tutorial Don. Hopefully you will do a more in depth video on the different features and extensions that are available. Speaking of extensions, any chance you could tell me where I can add elevenlabs? Would love to have TTS available.
Just installed it on my Ubuntu server and it seems to have gone well. Except the web URL can't be reached. I can't use the localhost IP (since the server doesn't have a GUI) so i'm trying from a remote system connected to the same network. . It says the site can't be reached. I have UFW turned off. What am i missing here?
I installed oobabogaa webui with docker-compose and llama 4 bit model is about 4gb im using the gpu 1070 ti on linux. What i noticed is that is to slow and when i try to run alpaca model runs out of memory error. Maybe if i use cpu is going to be faster? I have i5 8400 and 16gb ram.
Sadly im running AMD on both my good rigs so its probably going to be a pain. Someone was telling me it will chug without decent GPU acceleration and a GPU with a fair amount of memory also.
Great stuff😊
This is awesome. Thanks for the tutorial Don. Hopefully you will do a more in depth video on the different features and extensions that are available. Speaking of extensions, any chance you could tell me where I can add elevenlabs? Would love to have TTS available.
Awesome video :D
Albony
@@skullpoly1967 y e s ?
Thanks Don.
Awesome Don, can You show us how to use it with our own data, I've a couple of pdf and text files, how can I use them to answer my prompts?
Just installed it on my Ubuntu server and it seems to have gone well. Except the web URL can't be reached. I can't use the localhost IP (since the server doesn't have a GUI) so i'm trying from a remote system connected to the same network. . It says the site can't be reached. I have UFW turned off. What am i missing here?
edit the webui.py file on line 146 add "--listen" after "--chat" to enable listening on local ip
@@NovaspiritTech Awesome, thanks for the quick reply!
Where can i go if i am having problems with this install scrypt?
I installed oobabogaa webui with docker-compose and llama 4 bit model is about 4gb im using the gpu 1070 ti on linux. What i noticed is that is to slow and when i try to run alpaca model runs out of memory error. Maybe if i use cpu is going to be faster? I have i5 8400 and 16gb ram.
How much vram of GPU do in need to run this maybe 16gb ??
Yea at least, I was having trouble running 7B with my 8gb 1070
Can we use coral ai accelerator to RPI and run llama?
Sadly im running AMD on both my good rigs so its probably going to be a pain.
Someone was telling me it will chug without decent GPU acceleration and a GPU with a fair amount of memory also.
Not wrong, 8gb 1070 still not enough ram to run 7B
😢
Yep like all these chat ais, its broken, the chat ai community has a lot of catching up compared to art ai
But will it run on a Pi? I'm thinking the answer will be "yes, eventually."
yes, acutally tested some stuff on rpi and on khadas edge2. if it wasn't for the limited ram, you are stuck on 7B or lower
wait rpi got vram?
@@NovaspiritTech Which executable/zip file did you use for your khadas? I'm attempting this on the orange pi 5b :b