I enjoyed the video. Easy to understand and most importantly showing what you can do without to much hassle with a not too powerful MacBook. From the video I believe I have the same model as the one you used. I do like the idea of setting preset for the 'engine'. I do use the Copilot Apps. I can then check how both perform for the same question. I have just tested deepseek-coder-v2 with the same questions as you... Funny thing, it is not exactly the same answer. Also on my 16Gb Mac,,, The Memory activity get a nice yellow colour. Sadly contrary to the Mac in the video, I got more stuff running in the background like Dropbox, etc... Which I cannot really kill just for the sake of it,
if i understood correctly. The idea could be to create an app for macOS that includes some function that requires a LLM. The app is distributed without the LLM. The user is notified that said function will only be available if download the model. This message should be implemented in a View that contains a button that will download the file and configure the macOS app to start its use.
This is not working for iOS. If you want to run LLM on an iPhone you will need to use a smaller model which usually dont perform so well. Most iPhones have less than 8GB Ram. That is also why Apple Intelligence will process more advanced complex task in the cloud
Ollamac and OllamaKit creator here! 👋🏼 Great video, Karin!! ❤
more on these please
Really informative, something I kind of was in need. Thanks for showing things off.
Thank you for the video, and sharing your knowledge.
I use '/bye' to exit out of the Ollama cli
I enjoyed the video. Easy to understand and most importantly showing what you can do without to much hassle with a not too powerful MacBook. From the video I believe I have the same model as the one you used. I do like the idea of setting preset for the 'engine'. I do use the Copilot Apps. I can then check how both perform for the same question. I have just tested deepseek-coder-v2 with the same questions as you... Funny thing, it is not exactly the same answer. Also on my 16Gb Mac,,, The Memory activity get a nice yellow colour. Sadly contrary to the Mac in the video, I got more stuff running in the background like Dropbox, etc... Which I cannot really kill just for the sake of it,
Nice! 👍
wondering what it'd take to get something running on iOS. Even with 2B it might prove useful
Great video!
Dear Karin, Could you please advise on how to put my entire Xcode project into a context window and ask the model about my entire codebase?
I've tried using the LLM's locally, but I only have 8gb of ram. Great video!
Unfortunately, Apple made the base models with 8GB. A lot of people have the same problem as you.
I feel like you can still run the Phi3 model on your device.
if i understood correctly. The idea could be to create an app for macOS that includes some function that requires a LLM. The app is distributed without the LLM. The user is notified that said function will only be available if download the model. This message should be implemented in a View that contains a button that will download the file and configure the macOS app to start its use.
The awesome video!
Great video. Thank you. I am interested to know if any developers are using this in their iOS apps.
This is not working for iOS. If you want to run LLM on an iPhone you will need to use a smaller model which usually dont perform so well. Most iPhones have less than 8GB Ram. That is also why Apple Intelligence will process more advanced complex task in the cloud
Any way of running it on iOS?
Only watchos sorry
LLMs have a long way to go. 4GB to run a simple question is a no go. The have to reduce it to 20MB and people will start paying attention.