GPU vs CPU: Running Small Language Models with Ollama & C#

[@RomuloMagalhaesAutoTOPO]

👍

[@bdanuw]

Another great video from ElBruno :)
Thank you Bruno.

[@elbruno]

Glad you liked it!

[@eugene5096]

Thank you Bruno, interesting as usual !!!

[@elbruno]

@@eugene5096 Thanks! The CPU vs GPU is a wow one 😁

[@muluhsenet7582]

❤❤❤

[@samirou976]

Hi ! This is very interesting but I wonder how it would perform on an NPU ? Is it possible to make it run on NPU ?

[@cuachristine]

NPU is a GPU with all the graphics bits removed.

[@samirou976]

@@cuachristine Yes I know thank you but that was not my question

[@elbruno]

Ohh that's a great question! I still don't have access to an NPU machine, however, if docker desktop allows access to the NPU, it should work. Let me ask Justin (a fellow CA.NET), who rocks the docker world to see what he can share about this.

[@samirou976]

@@elbruno Thanks man I would realy apreciate that 🙂

[@bilalbilal7674]

Bilal here😊, I think you should go for creating an extension then it will be good and easy to accessible

[@elbruno]

There is one in the Aspire Community Toolkit: github.com/CommunityToolkit/Aspire/tree/main
I may record a video about that one!
Best

[@jimmymac601]

Does this support multiple GPUs ?

[@elbruno]

I'm not sure, I'll say no, we may need to check with the ollama team