Hello Prashant, yes I believe that's possible! I am pretty new to OpenWebUI, and am more familiar with using Docker commands, so that's why I went this route in the video. Great point!
@@TrevorSullivan that’s really very useful tutorial, probably you’re the first (and only yet) to make tutorial on this. Thanks a lot! Hosting my own LLM soon, I do have 1 qq though. I was reading through some AWS posts and found some details about Inf1 and Trn1 (Trainium and Inferentia) types AWS recommending, and wanted to know your thoughts on which would be the best option to go with for the instance type Inf1 or the G type ? Would love to hear your thoughts on this 🙂
Guys at 11:47 When it came time to install nvidia-utils, I had problems with the library and driver version (nvidia-utils-550) [message: Failed to initialize NVML: Driver/library version mismatch NVML library version: 550.107]. The solution was to check the package names and I saw that "nvidia-utils-550-server" existed and that's the one I installed and it worked. I assume that the distributions have been updated and now it has to match the driver that was installed (nvidia-driver-550-server).
great tutorial, excellent explanations! thank you sir!
Brilliant!
Thanks so much, Andrew! I'm glad this was helpful. 🙏🏻
Very great tutorial, was looking for the exact same thing (y)
can you share all your install commands and docker compose yaml file?
What’s is the cost for acceptable spec.
Can't we use the hosted OpenWebUI web interface to download additional or remove existing model instead of exec'ing into the docker container ?
Hello Prashant, yes I believe that's possible! I am pretty new to OpenWebUI, and am more familiar with using Docker commands, so that's why I went this route in the video. Great point!
@@TrevorSullivan that’s really very useful tutorial, probably you’re the first (and only yet) to make tutorial on this.
Thanks a lot! Hosting my own LLM soon, I do have 1 qq though. I was reading through some AWS posts and found some details about Inf1 and Trn1 (Trainium and Inferentia) types AWS recommending, and wanted to know your thoughts on which would be the best option to go with for the instance type Inf1 or the G type ?
Would love to hear your thoughts on this 🙂
You definitely can. I have been doing that a lot.
@@AndrewCLatham thank you 👍🏻
Guys at 11:47 When it came time to install nvidia-utils, I had problems with the library and driver version (nvidia-utils-550) [message: Failed to initialize NVML: Driver/library version mismatch
NVML library version: 550.107]. The solution was to check the package names and I saw that "nvidia-utils-550-server" existed and that's the one I installed and it worked. I assume that the distributions have been updated and now it has to match the driver that was installed (nvidia-driver-550-server).
Thanks for sharing your solution! I think I've run into this before, where the driver and utility versions mismatched.