from home page of your webUI localhost3000 in your browser, click on your account name in the lower left, then click settings, then "models", then you can pull llama3.1 by typing it in the "pull" box and clicking the download button. when it completes, close webUI and reopen it. then i had the option to select 3.1 8B from the models list
Note for 405B: We are releasing multiple versions of the 405B model to accommodate its large size and facilitate multiple deployment options: MP16 (Model Parallel 16) is the full version of BF16 weights. These weights can only be served on multiple nodes using pipelined parallel inference. At minimum it would need 2 nodes of 8 GPUs to serve. MP8 (Model Parallel 8) is also the full version of BF16 weights, but can be served on a single node with 8 GPUs by using dynamic FP8 (Floating Point 8) quantization. We are providing reference code for it. You can download these weights and experiment with different quantization techniques outside of what we are providing. FP8 (Floating Point 8) is a quantized version of the weights. These weights can be served on a single node with 8 GPUs by using the static FP quantization. We have provided reference code for it as well. 405B model requires significant storage and computational resources, occupying approximately 750GB of disk storage space and necessitating two nodes on MP16 for inferencing.
finally setup open webui thanks to you. i'd approached it, seen "docker" and left it on my todo list for weeks/months. I'm running gemma2 2b on my gtx 1060 6gb vram. any suggestions on good models for my size?
hello. After installing OpenWebUI, I am unable to find OLLAM under 'Select a Model'. Is this due to a specific configuration? For information, my system is running Ubuntu 24.04.
2 videos in 1 day? Woah! Thanks
Interesting tutorial with Web UI and Ollama, Thanks!!!
Great one Dan! Keep ups updated on the AI stuff!
Ollama should integrate a feature like artifact that allow you to test your html css code in a mini webview
How can I connect my local ollama3 with webUi, My webUI couldn't find the locally running ollama3
same problem
from home page of your webUI localhost3000 in your browser, click on your account name in the lower left, then click settings, then "models", then you can pull llama3.1 by typing it in the "pull" box and clicking the download button. when it completes, close webUI and reopen it. then i had the option to select 3.1 8B from the models list
@@MURD3R3D i found that happen due to docker networking.
A perfect tutorial.
This is great. Thank you
Love your terminal, which tools do you use to customize it?
Note for 405B:
We are releasing multiple versions of the 405B model to accommodate its large size and facilitate multiple deployment options:
MP16 (Model Parallel 16) is the full version of BF16 weights. These weights can only be served on multiple nodes using pipelined parallel inference. At minimum it would need 2 nodes of 8 GPUs to serve.
MP8 (Model Parallel 8) is also the full version of BF16 weights, but can be served on a single node with 8 GPUs by using dynamic FP8 (Floating Point 8) quantization. We are providing reference code for it. You can download these weights and experiment with different quantization techniques outside of what we are providing.
FP8 (Floating Point 8) is a quantized version of the weights. These weights can be served on a single node with 8 GPUs by using the static FP quantization. We have provided reference code for it as well.
405B model requires significant storage and computational resources, occupying approximately 750GB of disk storage space and necessitating two nodes on MP16 for inferencing.
and what about 70B? How it could be served? Could some of llama 3.1 be used by simple 16-cores laptop with integrated GPU and 32GB ram?
When you say "we" do you work for meta?
Dan, what the specs for your local machine?
Would make a video on how to integrate llama 3 to wordpress website, making chatbot or co pilot
Hey, could you make a video on how to edit the login page? I want to make the login page to my liking.
in Ollama Is there an admin dashboard for tuning the model, sir?
finally setup open webui thanks to you. i'd approached it, seen "docker" and left it on my todo list for weeks/months. I'm running gemma2 2b on my gtx 1060 6gb vram. any suggestions on good models for my size?
Bro you the G
Big thanks from Palestine
❤💚🖤
be safe
Stay strong
How we can tune a model with custom data?
hello, any idea how to set keep_alive when running the windows exe ?
Is WebUI a replacement for aider?
hello. After installing OpenWebUI, I am unable to find OLLAM under 'Select a Model'. Is this due to a specific configuration? For information, my system is running Ubuntu
24.04.
my ollama running same model is deadslow, running in laptop i5 11th gen without GPU 26GB Ram.
Is it because of no dedicated GPU?
Thank you, I tried it but it is very slow, running it on a laptop with 16GB RAM!
Is there an integration for Open WebUI + Spring AI?
6 months behind everyone else.
Davis Michelle Clark Melissa Miller Frank