A local AI Voice Assistant using Granite-7B, Ollama, InstructLab and AlwaysReddy

I had an idea today to be able to chat with my local LLM that is powered by the granite foundation model from the InstructLab Project. Luckily, I was able to find an open source project called AlwaysReddy that fit the bill. In this video, I go from zero to chatting with the granite open source model in under 10 minutes. ALL IN REAL TIME with no cuts or edits.
github.com/ins...
github.com/ILi...
ollama.com/dow...
huggingface.co...

[@HarishPillay]

Do please do another demo on Fedora.

[@billychen2103]

awesome! thanks for sharing!

[@Avalon19511]

It would be nice if this had a gradio ui

[@heysherlock]

Exactly, this open UI seems like some cheap rip off of chatGPT

[@jerryyuan3958]

How many GPU on your window? Whisper also need GPU if you want to do a quick convert text to audio.

[@gshipley21]

Oh sorry - I didn't know that. I do have a RTX 4090 in my windows machine.