Building an AI assistant that listens and sees the world (Step by step tutorial)
Вставка
- Опубліковано 5 вер 2024
- I built an AI assistant that listens to my commands and uses my webcam to understand the world around me. I used Python to build it, and this video will give you a step-by-step overview of how it works.
Here is the link to the source code: github.com/svp...
I teach a live, interactive program that'll help you build production-ready Machine Learning systems from the ground up. Check it out here:
www.ml.school
To keep up with my content:
• Twitter/X: / svpino
• LinkedIn: / svpino
🔔 Subscribe for more stories: / @underfitted
Magic. Another demonstration of that fact that computer science is just piling layers of abstraction on top of each other. Astounding. Well done.
I love how you explain your code!
Thank you so much for your hard work and passionate explanation. More than the code walkthrough, what I learn most from you is to how to break down a use case into the parts of the workflow.
You're literally building my project faster than I can! 😂❤️
You mentioned tipping $5... lol. 100% you'll hear back from me with way more gratitude than just $5. 😉 You're giving me hope!
Also, if anyone wants longer responses from the LLM but doesn't want to sacrifice speed, you can stream the tokens, and have a function running on a separate thread that chunks these into sentences and adds them to a queue, then you can have another function on a separate thread checking this queue and processing them into speech. This way, the bot can start speaking while the streaming is still going on.
Yes
😊
Sweet! Amazing Video!
Can you create a second video like this one , but this time only with local models running on ollama, faster whisper, etc ?
I really like how you explain what the code is doing. Looking forward watching other tutorials.
The way you you explain that code!!!! Goose Bumps
Maestro! Very well explained Santiago, mis 10+ 😊
This has some awesome, awesome, awesome applications.
Thanks for sharing this knowledge, man
It's a fascinating quest to build an AI assistant that can see and hear the world! There are more tools available if you want to go further; they could provide new features and advantages to improve your AI projects. #TechTutorial #Innovation #AIAssistant
Love your tutorials. You keep it so straight forward and simple while still explaining what each part does.
Thanks !
this is amazing! I was able get it running on Windows easily. Thanks very much. I'll post my demo on youtube and tag you! An issue that I find was that it sometimes catch some background sounds and transcribe it to some weird sentences.
I am also working to build this kind of project. In my case it's a chat process. This video will be helpful. I am trying to use the multimodal which supports video, which will make the process easy.I am gonna do it.
Good afternoon! This was a very nice educational video. Not short, not long example, just enough to understand the idea. And I'd like also mention the passion and your interest to the job you are doing! Just thank you, and do, please, more)
Thanks!
Like? Whoa!! Bro? 😳😍Thanks for taking the time to share. Much appreciation. Subscribed!!!
Immediately i saw this video i knew i had to subscribe ASAP. I tried it and its up and running though i had some errors cos i am not a python programmer. But i want to use it to build an Ai that can be installed on a vessel (ship). Thanks for sharing ❤
This is AWESOME!!! "goose bumps"👍
Trying it out with local Llava model which is a multimodal one available with Ollama. Getting a ValueError saying image_url can only be string.
In the source code, if you want to use openai model, you'll need to import ChatOpenAI.
Yeah
Wow, I just saw Wes Roth's demo of it. Incredible.
I'm in the middle of integrating this into CrewAI and AutoGroq.... with the Autogroq model acting as the main agent assistant that is able to use crewai as its agency over a given task... it seems simple enough (famous last words) ... two classes in this script... although the alloy voice stinks... in fact all the openAI voices stink and Elevenlabs are still my go to.
@@mickelodiansurname9578 Oh my, you'll have to let me know how well it plays with CrewAI and Autogroq! If you can pull that off, should be awesome.
@@mickelodiansurname9578 did you succeed with your demo/project?
Excellent ! Fantastic explanation of the code ! Thank you Santiago !
very cool idea. well executed and great explanation. subscribed
Kudos for your clear explanation, inspiring 🤘
¡Gracias!
Thanks
Great video Santiago!! Congratulations!
Very simple but impressive
Thank you for this. Learnt a lot from you
Your explanation is amazing ... Can I use 11labs voice here ?
Hi I really liked your video I took inspiration from it and made the same thing without the camera integration I used the local llm llama3 to run it.I could not pay for openai api key so had to make use of that.If there is any new model which can do the webcam integration locally please recreate this video using that
Keep making more.
LOVED IT. God Bless You
Amazing 🎉
wow This is amazing!!!!
Great demo...😎
Im not very good at coding but can I add the LM Studio local server with a vision model instead of openAI or gemini? Also different local models for text to audio and audio to text. This would be great to have all local like the rabbit device....but free, would be great for ppl that cant see.
I am a fan immediately
Excellent content!! Amazing! One question: Why didn't you use Langchain in the TTS phase? Does Langchain only support Text-to-Text?
I could have, but it was just a one line of code.
@@underfitted thank you for clarifying. Keep doing content like this 🙌🏼
Incredible
This is very amazing
Good stuff!
greate JOB Thanks a million for share
i cant hear you
What happens? Too quiet the video?
@@underfitted yes i can hear sounds not what the sounds are saying
Nice!
I swapped out gemini for llama3 and got an error, 'ValueError: Only string image_url content parts are supported.' Seems like swapping out models won't be as straightforward yet despite the goosebumps! Any idea why this happenend?
Could it be used with local lava model through webui that now has an api key?
excellent
Hi Santiago, would it be possible for you to create a project which combines a ML with a AI methods (preferably with open source tools) to solve one real word problem? In case if the other chanell members are interested on my proposal they may jump in and provide their comments and suggestions. Thank you in advance.
You're the Goat 🐐
Thank you,,,
WHY?: RuntimeError: Unrecognized CachingAllocator option: 0
I hate these issues...😬
So look if I wanted the AI to JOIN a zoom call or a facetime session or something.... well okay how do we do that?
Magic 🧝♂️ 😅
CC it is then
u are so cool
I can't hear video
How come?
❤❤🥳🥳🥳🥳🥳
I don't see you and I hope I never will.Don't recommend channel option is not working!
Great PoC | tutorial!
is langchain >> llamaindex ??? or they are different?
Similar. Pick the one you like the most.
¡Gracias!
🙏🏻