I CREATED My NEW Voice Assistant w/ On-Device Wake Word Detection 🚀| Home Assistant | ESP32 S3
Вставка
- Опубліковано 30 чер 2024
- Let's look at how I created my own local voice assistant with On-Device Wake Word Detection using an ESP32 S3 and Micro Wake word with Home Assistant
Setup Guide & Circuit Diagram and component buying links : smarthomecircle.com/How-I-cre...
Setup Voice Assist Pipeline : • How I Setup LOCAL VOIC...
===========
Components used in the video
===========
ESP32 S3 N16R8 :
AliExpress s.click.aliexpress.com/e/_DFo...
Amazon US: amzn.to/3WM24je
Amazon EU: amzn.to/3WPJYwC
INMP441 Microphone:
AliExpress : s.click.aliexpress.com/e/_Dmn...
Amazon US: amzn.to/3GdxO7o
Amazon EU: amzn.to/47uScws
MAX98357a Audio Amplifier:
AliExpress : s.click.aliexpress.com/e/_Ddy...
Amazon US: amzn.to/46wKTmW
Amazon EU: amzn.to/47vEMjH
COB WS2812B 5mm LED Strip:
AliExpress : s.click.aliexpress.com/e/_DDt...
Amazon EU (Not the Exact Model): amzn.to/3yqR7t7
4 Ohm 3W Speakers :
AliExpress : s.click.aliexpress.com/e/_DBD...
Amazon EU : amzn.to/3xafVpd
Amazon US : amzn.to/4chdboN
Mute Button : s.click.aliexpress.com/e/_DEh...
===========
Support via :
===========
Buy Me a Coffee : www.buymeacoffee.com/amrutprabhu
Patreon : / amrutprabhu
PayPal : www.paypal.com/paypalme/amrut15
Social Links
===========
Follow me on X : / smarthomecircle
Instagram : / smarthomecircle
===========
===========
Smart Home related board To Buy:
===========
ESP32 WROOM : AliExpress: s.click.aliexpress.com/e/_DB4...
Amazon US: amzn.to/45vab4p
Amazon DE: amzn.to/3FbreOc
Raspberry Pi 5 : AliExpress : s.click.aliexpress.com/e/_DEG...
Amazon US: amzn.to/3xp4CZL
Amazon EU: amzn.to/3xeOB8S
===========
Gear I use :
===========
Sony Alpha a6000 : Amazon India : amzn.to/3RM7QMJ
Germany : amzn.to/3G1Iw24
US : amzn.to/3cpKZ9E
UK : amzn.to/3J0g5Ry
Razer Microphone : Amazon India : amzn.to/3aQxI9C
Germany : amzn.to/3DRRrAh
US : amzn.to/3ITKu3Q
UK : amzn.to/3v3RHbU
===== Chapters ======
00:00 - On-Device Wake Word Detection Demo
00:25 - Creating The Voice Assistant Device
06:37 - Creating ESP32 S3 Device in ESPHome
07:07 - Understanding On-Device Wake Word YAML Configuration
07:41 - Mute Button Configuration
08:02 - Hardware Mute Button Configuration
08:36 - LED Strip Configuration
08:56 - I2s Microphone & Audio Configuration
09:02 - Microphone & Speaker Configuration
09:51 - On-Device Micro Wake Word Configuration
10:45 - Voice Assistant YAML Configuration
11:25 - Flashing Binary On The ESP32 S3
13:31 - Connecting ESP32 S3 Voice Assistant to Home Assistant
13:53 - Finding IP Address Of The Device
14:50 - Viewing the Voice Assistant Device In Home Assistant
15:10 - Testing The Mute Buttons
15:35 - Testing The Hardware Mute Buttons
16:00 - Setup Voice Assist Pipeline Components
17:23 - Testing the On-Device Wake Word Detection
18:40 - Whats Coming Next? - Наука та технологія
Thanks a lot for the video... and I wait for the customization of the miro-wake-word :-)
Currently trying to find the best and easy way to create custom wake words without needing powerful hardware.
hi! great content, as always! subscribed and fan! tku for sharing this. ordered already components in ali and got my cousin to 3d print the case, after all done I will share the results with you...
btw , did you manage to train a new wakeword in another language for the microwakeword? i spent hours with colab w/o success :(
Thank you for such kind words. 😄
Regarding training the wakeword for microwake word, there is no colab available.
The colab playground is only for the wakeword used in OpenWake word.
I’ve found that I was having difficulty with the assistant hearing not just my voice but background voices, like from the tv, and therefore not recognising my voice commands. I went and changed the settings on noise_suppression _level: from 0 to 2 it’s working much better now
Ohh yes.. I had set it to 2 before. Not sure when I set it back to 0. 🤔
But thanks for this. I will update the article again. 😄
Very nice and informative video. I am trying to copy it for my voice assistant.
Wouldn't it be possible to use the touch ability of the esp32 instead of the hardware button?
Have you ever tried that?
I'm planning to use an ikea lamp with a metal stand for my voice assistant. It would be great to mute and unmute with just touching the stand.
Ohh that sounds really interesting 😃.
I think it would work. Give it a go and see how it works..
Very cool, Thx!. How set my sefl .tflite - wake word?)
I think you are talking about custom wake words, I will be working on that for my next video.
AFAIK to setup custom wake word for microwakeword, you'll have to train it in pretty specific way. Also i can't see the mechanism of setting it in the configuration so far - it seems like microwakeword is pointing to ESPHome GitHub, where these words are.
@@SmartHomeCircle I can't wait to see that! Amazing job you did!
My pre-soldered INMP441 microphone came with the connections on the other side, do you think that will be a problem? So the hole for the microphone is on the bottom in my case. Not sure if I can flushmount it now or that I need to give it some clearance so sound can come from beneath as well.
As far as I know, the voice is captured from that hole.
You might have to check how clearly the voice gets recorded with it.
Hey id like to also build your voice assistant. What speaker did you use? Do you have a recommendation for me? :) didnt found it in the description
Thank you :)
I updated the description and added links to it.
I also have this article which has everything in it : smarthomecircle.com/How-I-created-my-voice-assistant-with-on-device-wake-word-using-home-assistant
What an amazing video. I followed your guide but I can't active the esp32-s3 by hot word. I've check log in esp and nothing happen when I speak "hey jarvis", tried with other model like alexa, okay nabu with same result. I have 2 esp32 s3 and 4 inmp441 but none of the combinations are successful. Is there any tip to using hotwords?
Does it show in the logs that the the micro wake word is jarvis is used?
Also can you check the wiring of the Microphone , specially the L/R. Also make sure that the channel in the yaml code is set correctly according to the yaml code in the article that I have linked in the description of the video.
Hope that helps. 🤞
Also have to add logger: to the ymal
Ohh yes.. Missed that. I'll update the article. Thank you 😊
Thanks for another informative video. May I ask what issues you found if any. I followed your ESP32 voice assistant only to find it did not handle both microphone and speaker on the same device well. Any limitations on the s3?
None that I have seen. Also the issue of the cracking sound on boot has disappeared that existed in the ESP32 setup.
If it can not handle microphone and speaker in the same time that means that you don't have the s3 variant as only the s3 have two i2s buses.
He meant about the ESP32 board and not the S3 variant 😄
Great video! Got me interested in trying this out ") how does it handle when you have 200 over devices though? Would it still be useable? Or would it lag I wonder? Thanks! And cheers
Do you mean 200 devices in Home Assistant?
I haven't got so many devices yet😄
If you use the assist window and try to write a command to invoke one of those 200 devices, and if that is fast then it would be fast also.
This device just takes the input and gives it to HA.
It will be just fine. If you mean voice satellites - it's good to have local wake word, otherwise they all would be bombing HA with audio streams to find wake word.
@@formatBCE Yes, true. reduces some resources on the HA server also.
Thanks heaps, followed all your previous videos successfully, so looking forward to starting on this one
@@clairerovic thank you 🙂
Thanks
Thank you so much for your kind appreciation. 😃. Definitely helps me keep going.
May I add I stumbled initially once the device was set up, the problem I found was I had to go into voice assistant settings and set up a new assist profile, then leaving wake word empty
@@nigelbye7965 ohh yes.. you have to not set any wake word in the assist pipeline.
Hi erstmal ein tolles Video hat bei mir auf Anhieb geklappt das einzige was stört ist das die sprachausgabe unvollständig ist also nach dem ich ein Befehl gegeben habe bricht die antwort ab
I am glad the video helped. 🙂
My German is not that good, but I what I understood is that the voice response is incomplete or gets stuck.
I dint face issues with the response being incomplete or stuck. I had problems with the speaker crackling on boot, so I moved the amplifier input from 5v to 3.3v , as I think the LED strip was interfering with it.
Does that give you some help?
Genau das Problem es bricht einfach ab
Thanks for the Button and the great Video
Glad you liked it. 😊
What type of button is it and where do I connect it to on the board?
The circuit diagram in the article shows how to connect the mute button.
This is the button I used : s.click.aliexpress.com/e/_DEhJfB1
Checked the article and definitely does not show the button in the parts list the circuit diagram or connection list. It’s in the yaml GPIO10 but where do I connect the other wire? Sorry to be a pain. p.s. I love your work and have been building your things for sometime
I watched the video and managed to pause it at the exact point where I could see it’s connected to GND
Hey, thanks a lot for the video ! It's been a week that i'm doing some research to do just this. I have Ollama running on my local server and plugged in my Home Assistant Assist. I have an Ulanzi Pixel Clock where the answers from the Home Assistant (local AI in that case) or displayed in scroll. Looks pretty nice, you can check the last video on my channel. I have a few questions please,
Is it possible to play sound on this device like we can do on Amazon Echo Dot for example ? Or should it only be a satellite to ask to play audio on another device ? I'd really like an all-in-one personal A.I home assistant.
If so, do you know where I could find a very good quality speaker (or 2 for stereo) and microphone for the project
You've put a resistor for the speaker. Cannot you set the volume in the code itself ?
Thanks again !
First of all, nice to see you are using AI for the responses. Kudos on that. 👍
I was trying to make the same for work for me. But it was not very reliable. Will try ollama this time.
To answer your question,
To get a better sounding speaker, I haven't got one, but may be you can redirect the audio from the response to alexa or google home mini. Check this video ua-cam.com/video/RIpnyTJmJJ8/v-deo.html . I haven't tried it with this device, but it was working well with my older voice assistant with an ESP32
I had to use the resistor to reduce the volume as I could not adjust the level in Home Assistant. To provide volume controls, we have to use `media_player` type, but its not supported for IDF frameworks. Hence the option was the resistor.
@@SmartHomeCircle Thx for your Feedback. Why Alexa or Google home, why not a Bluetooth speaker ?
Bluetooth speakers are difficult to connect to Home Assistant. The cast option with google mini allows for easier integration.
whats the maximum distance for the mic to be heared?
I could get it to work around 3 mts away. So good for a room..
Are there no controls for the speaker volume?
No.. it's currently not possible with the speaker type in the yaml. And media_player is not supported for idf frameworks
@@SmartHomeCircle thanks, btw can i use the same i2c pin for light sensor (bh1750) or temp sensor (ath10)
I think you could use it, but you will have to try it out.
I remember when I was using the light the co2 sensor, I had to make use of daisy chain connection for it to work. Check this video : ua-cam.com/video/bwVttXh4Lk4/v-deo.html
I would say first try with normal parallel connection and if it does not work, then try the daisy chain connection approach.
Hope this helps.
ESP logs show you IP address.
Its an internal IP so its fine.
@@SmartHomeCircle I meant you don't need any IP scanners, if you have ESPHome console. :)
@@formatBCE ohh yes.. good point.
For me since I run esphome using docker, the auto discovery fails because of absence of mDNS. Hence I would usually need the ip scanner.
@@SmartHomeCircle yeah i have mDNS turned off too.
What i actually meant, when you open device logs in your ESPHome dashboard, it reports system info (purple lines), and among that lines there's current IP address - so you can take it from there instead of guessing from long list of unknown IP addresses in IP scanner.
For myself, i have big pool of static IP addresses, that i use for my smart devices. This way i always know that my DHCP lease didn't expire and device didn't change their IPs.
@@formatBCE ohh yes.. I use static IPs too. It's just in the video that I didn't want it to be more complex with static IPs.. as different folks would have different subnets configured in the router. So I tried to at least give people a tool that shows a blue icon next to an active ip address and test if it belongs to the device.
i followed your guide, but when i copy past the code and then save and click on install it get the following error .
INFO ESPHome 2024.6.1
INFO Reading configuration /config/esphome/esp-ww-lounge.yaml...
Failed config
ota.unknown: [source /config/esphome/esp-ww-lounge.yaml:44]
'ota' requires a 'platform' key but it was not specified.
Ohh yes.. here are some breaking changes in the new ESPHome.
You need to replace the ota section with this.
ota:
- platform: esphome
password: xxxxxxx
I will update my article with this.