So technically, If I put a second hard drive on my PC running Ubuntu, with my RTX 3070ti I could host this myself and use it as an api endpoint for my web app to do translation, and other various tasks, calling whatever model I need based on the task I need done. All this for free ?
Great video man!!! 🎉
Glad you liked it, thank you for watching!
amazing content as always... thanks for sharing!!!
Thank you for watching!
Thanks.
Thank You for watching!
So technically, If I put a second hard drive on my PC running Ubuntu, with my RTX 3070ti I could host this myself and use it as an api endpoint for my web app to do translation, and other various tasks, calling whatever model I need based on the task I need done. All this for free ?
Yes exactly! as long as you gpu can hold the modle in it's vram, I will run smoothly, thank you for watching!
@@distrodomain could I run 3.2 ? Probably 11b is cutting it close ? I just need rewording texts, translation and some coding
Great content.
Can be done also with the Google Coral tpu?
I have not tested that chip, but if it has enough vrm and you can get the drivers working it could work, thank you for watching!
@@distrodomain Thanks
Hope you may do a video soon on it :-P
Can this be done on a Proxmox VM w/ GPU passthrough?
@@aiwa501 Yes as long as ollama has access to the gpu it should work
Jenns! pretty good
Thank you fpr watching!
Oh yeah, AI! 💯
;) thank you for watching!