It really is such a good combination of tooling, exactly what I use myself. I have personally found that Qwen 2.5 14B tends to follow instructions better than Gemma and it seems more than twice as fast despite being a 14B. My System prompt: You are a tab completion style assistant in the Obsidian editor. After the user provided content you will continue writing the next few sentences of the text as if you were the original writer. Use British English Spelling. IMPORTANT: Write in the same style and tone of the user unless asked to do otherwise. Do not begin the text with any extra characters or '...' and don't summarise the text. My User prompt: {{#context}}Context: {{context}} ================================= {{/context}} Continue the following paragraph: {{last_line}} temperature 0.2 - Note: I set this in the Modelfile, not the Companion extension as I found the Companion extension would incorrectly send the parameter to Ollama as "temp" rather than "temperature" top_p 0.85 (or 1.0 with min_p set to 0.9) num_ctx 16384 num_batch 1024 K/V cache quantisation is set to q8_0
Thanks, Matt! At first, I was very hesitant about everything to do with AI and now, thanks to your great video content, I can't get away from it. Despite free tools and LLMs, it is slowly becoming an expensive pleasure. Not because of the hardware. I have to invite my fiancée to dinner more often so that she doesn't feel resentful when I fall asleep with my head on the keyboard. 😅All jokes aside, as a dev, I constantly have new projects in my mind. Keep it up, thanks for your work!
This is amazing! I love obsidian - I have it on all my machines! Question: if you run obsidian on a computer separate from the ollama server machine, can the plug-in work on the remote machine?
Thanks, Matt. Not for me, but I passed your video on to a friend with a channel he is building. I was the guy who said you have a very smooth delivery. I figured if you're using this, it has be decent and not just hype. Thanks again.
I heard about it but until you nudged me with this video didn't set it up. Like you, I prefer local model. It is really nice, I can't feel Ollama working in the background which is nice. You are right about Llama 3.2 not really cutting it. Off to try Gemma. Oh boy this is fun.
Nice! i was already using the "continue" extension in VS Code and thought it would be nice to have autocomplete in obsidian too. Thanks for the heads-up, will be trying this out right away 🙂
Man i just became obsidian fan as well. I am using the copilot for obsidian. Really good as well working with Ollama. But companion seems to be even better. Thanks Matt.
If you have Ollama built with K/V cache quantisation, and set it to q8_0 - the context will use 50% less memory and the generations won't slow down nearly as much towards the end of a larger document.
Awesome video as always! I use it with Groq and it's super fast. But sometimes the line between "help me articulate" and "parrot the AI" gets blurry, we tend to choose the path of least friction and forgot to think hard on what we write, it's just human nature. That's why I was hesitant to introduce inline functionalities to my Copilot for Obsidian plugin. At the time I created it, I preferred interacting with AI on the side and not let it directly modify what I write. But people have requested inline features a lot, I'll probably introduce it anyways.
Matt, forgive my ignorance. I'm a bit of a noob in the AI space as well as obsidian. I have been using obsidian and AI tools for a few months... I can't find an obsidian this configuration page you're talking about. Can you or someone in the chat tell me what subcategory or whatever in the configuration this is under
Hey Matt, while this works well on desktop, on my M1 Air 8Gb, it really hogs it, can you suggest a model I could use from Ollama, I can always use external but there is beauty to use local one. Maybe Phi3.5?
isn't user prompt the actual prompt the user needs to enter? it would be helpful to understand those keywords of {{#context}} vs {{context}} {{/context}} and why do i need the line ====== line and new lines.
can you make an ollama based version of Claude with computer use capabilities.? if yeah show how to video for this :P we need local open source free version of it :D
You are probably the most wholesome and humble of all AI content creators! Thank you for Ollama and these awesome videos!
It really is such a good combination of tooling, exactly what I use myself. I have personally found that Qwen 2.5 14B tends to follow instructions better than Gemma and it seems more than twice as fast despite being a 14B.
My System prompt:
You are a tab completion style assistant in the Obsidian editor. After the user provided content you will continue writing the next few sentences of the text as if you were the original writer. Use British English Spelling. IMPORTANT: Write in the same style and tone of the user unless asked to do otherwise. Do not begin the text with any extra characters or '...' and don't summarise the text.
My User prompt:
{{#context}}Context:
{{context}}
=================================
{{/context}}
Continue the following paragraph:
{{last_line}}
temperature 0.2 - Note: I set this in the Modelfile, not the Companion extension as I found the Companion extension would incorrectly send the parameter to Ollama as "temp" rather than "temperature"
top_p 0.85 (or 1.0 with min_p set to 0.9)
num_ctx 16384
num_batch 1024
K/V cache quantisation is set to q8_0
wow it works very well. thank you so much
Thanks, Matt! At first, I was very hesitant about everything to do with AI and now, thanks to your great video content, I can't get away from it. Despite free tools and LLMs, it is slowly becoming an expensive pleasure. Not because of the hardware. I have to invite my fiancée to dinner more often so that she doesn't feel resentful when I fall asleep with my head on the keyboard. 😅All jokes aside, as a dev, I constantly have new projects in my mind. Keep it up, thanks for your work!
This is amazing! I love obsidian - I have it on all my machines! Question: if you run obsidian on a computer separate from the ollama server machine, can the plug-in work on the remote machine?
This is absolutely brilliant. I discovered you 2 weeks ago and what I learned since then it's ... incredible. Thank you.
Copilot for Obsidian is also a great one. Combine these two with the “Run Code” plugin makes Obsidian like a Jupyter Notebook + AI
Thanks, Matt. Not for me, but I passed your video on to a friend with a channel he is building. I was the guy who said you have a very smooth delivery. I figured if you're using this, it has be decent and not just hype. Thanks again.
Great content! real world uses with great UI
Thanks for all your great videos... and super humor :-) Been looking for this plugin for Obsedian (without knowing it, before your video).
Tanks Matt! Great channel, great speaking speed, easy to understand and finally a nice way of spending time on internet.
As an obsidian user, I would love to see you make a plugin!
I heard about it but until you nudged me with this video didn't set it up. Like you, I prefer local model. It is really nice, I can't feel Ollama working in the background which is nice. You are right about Llama 3.2 not really cutting it. Off to try Gemma. Oh boy this is fun.
Nice! i was already using the "continue" extension in VS Code and thought it would be nice to have autocomplete in obsidian too. Thanks for the heads-up, will be trying this out right away 🙂
Will you be doing a video on how you trigger the checkmark on the script in obsidian triggers n8n?
Man i just became obsidian fan as well. I am using the copilot for obsidian. Really good as well working with Ollama. But companion seems to be even better. Thanks Matt.
You rock, enjoyed it, learned a bit too, thk you, and keep it up!
I actually have a rock that says 'You Rock' on it. Former manager when I lived in Amsterdam got it for me.
If you have Ollama built with K/V cache quantisation, and set it to q8_0 - the context will use 50% less memory and the generations won't slow down nearly as much towards the end of a larger document.
don't think I am seeing those improvements, but maybe I am using a bad model for it.
Thanks for sharing!
I like your videos, and the way you are presenting complex topics in such a calm and easy to understand way 👌
Thank you soo much Matt! I've been looking for something like this for ages! +1 for you writing your own version of the plugin,
Sounds like a good use for the free tier of Gemini!
Awesome video as always! I use it with Groq and it's super fast. But sometimes the line between "help me articulate" and "parrot the AI" gets blurry, we tend to choose the path of least friction and forgot to think hard on what we write, it's just human nature. That's why I was hesitant to introduce inline functionalities to my Copilot for Obsidian plugin. At the time I created it, I preferred interacting with AI on the side and not let it directly modify what I write. But people have requested inline features a lot, I'll probably introduce it anyways.
Great work Logan. Keep focus on agents, it's the future.
Matt, forgive my ignorance. I'm a bit of a noob in the AI space as well as obsidian. I have been using obsidian and AI tools for a few months... I can't find an obsidian this configuration page you're talking about. Can you or someone in the chat tell me what subcategory or whatever in the configuration this is under
Nevermind, what i was missing is that it "Companion" is a plug in...
Is there a Notion equivalent? :-) Hoping to crowd-source the brain trust here.
I am running Ollama as 0.0.0.0 using a Tailscale IP but I can’t seem to get the plugin to work on mobile. Have you had any luck with this or tried it?
Great. Please do more obsidian content ❤
Hey Matt, while this works well on desktop, on my M1 Air 8Gb, it really hogs it, can you suggest a model I could use from Ollama, I can always use external but there is beauty to use local one. Maybe Phi3.5?
I mentioned llama3.2 3b, but 1b is also good. Not sure about others.
This idea is fucking awesome
Oh, missing water bottle!!!
isn't user prompt the actual prompt the user needs to enter? it would be helpful to understand those keywords of {{#context}} vs {{context}} {{/context}} and why do i need the line ====== line and new lines.
can you make an ollama based version of Claude with computer use capabilities.? if yeah show how to video for this :P we need local open source free version of it :D
That existed before Claude did it.
Did you have to call me out with the FOSS Obsidian?
Come on...
I get it, yes Obsidian is obviously the better tool.
You have won :(