I Installed Chat GPT In Home Assistant And The Results Were Amazing!
Вставка
- Опубліковано 25 кві 2023
- In this video, I review my analysis of using Open AI's GPT-3 model (also known as Chat GPT) in home assistant. I show examples of what GPT-3 can do and demo a few automations that take advantage of GPT-3's strengths. After watching this, you will definitely leave with your mind blown and excited with new ideas you can implement.
What isn't in this video? I do not go over how to install GPT-3 in home assistant. I will create a separate video deep diving into the installation process and how to use it in an automation.
Would you like to show support? Buy me a coffee!
bmc.link/mmontaque
#chatgpt #homeassistant #smarthome #nodered #homeautomation - Наука та технологія
HA turn on the tv. (HAgpt) No! I've determined it's your bedtime and sending your robot vacuum to attack you!
😂😂
read this is Paul Hibbert's voice 😂😂
@david 😂😂😂 - 🤯
"I just wanted to see if I could do it" is the motivation behind half the stuff I do with Home Assistant. Great video, man! Thanks!
Glad you like it!!!
Great video! Always great to find more unique tech/smart home creators. Hope this one blows you up!
Thanks, me too! 😁
So glad UA-cam recommended this. Love this type of content
Come on in the water is warm!
Happy you found the channel 😁
Another great video. I'd definitely would like to see a more technical how to on how you integrated GPT3. I always thought it would be fun if you could take something like GPT3 and run it locally. One more step to getting my very own Jarvis. 😏
I feel you on that!! I have a new series that I’m gonna drop that goes through the process so stay tuned
Great video. You sure smashed that 1k subscriber goal. Well deserved. This was an interesting experiment and a well laid out video.
I appreciate the compliment 🙏🏾
Found a new HA channel! Love your style. Love your humor. Love the knowledge and content. Looking forward to more Home Automation/ AI / Node-Red. SUBSCRIBED
Thanks & Glad you found the channel 😁
Great video and the link has been saved for future play. Thanks for this - subscribed!
Thank you and I’m glad you find the content interesting! 🙏🏾
Subbed with a bell and making a mark here at 1.02K subs. You’ve got some interesting ideas. It’ll be fun to watch you grow.
Thanks! i appreciate the support 🙏🏾
Great video, and congrats on cracking the 1k subs - I have a feeling there'll be a lot more of those to come :-)
Thanks! I’m grateful for the interest ppl are showing 🙏🏾
This is really awesome. Here are some ideas, first, Home Assistant is looking to incorporate voice assistant, so you can make your own speakers, completely leaving google or alexa out of your house, I'd love to see a AI get incorporated into that voice assistant. Secondly, with the use of things like Alpaca and Orca, it might be possible to run our own version of a language based AI model, similar to chatGPT, this bypasses your whole issue with having to pay for openAI. I'll be looking into this stuff. I am in the process of setting up home assistant now, switching away from google assistant. Though I dont know when I will be able to implement a self hosted AI, due to the physical requirements of running such a thing. My current setup certainly doesnt have a hope of running it, but Orca shouldnt be too bad.
I loved the video, good energy as well, keep up.
I Appreciate it 🙏🏾
Man, great video! Definitely earned my sub.
Glad you enjoyed!!
Only got 4 minutes into the video and felt like I had to pause it to like and sub; less I forget! I know I'm going to love the content, but I really love the presentation too!
I appreciate the support and I’m glad u enjoyed the content.!
New here and here to stay,loving the automation
Thank you & Welcome aboard!
Awesome content, thanks very much 🙏🌈 almost at 5k 🎉
So close!
You're doin your thing homie, keep it up. 💯 Forget the comment haters, I like the casual style and lol'd for real a couple of times, I had to sub. Keep grinding!
Bet! I really appreciate the support!
Sub earned! Great video. I was able to follow most of it, but I wonder if you could make future videos a bit more user-accessible (i.e. don't need to understand programming logic to follow your explanation).
I think one area in which I'd love to see ChatGPT integrated into HA is as a programming assistant for people (like me) who have a very limited knowledge of programming. So I could say, "I have this IKEA remote. I would like it to control the lights in this room through Zigbee2MQTT. I want button 1 to do this, button 2 to do this" etc. etc. To me, that would be a game changer for HA mass adoption.
Keep up the great work!
Thanks for the comment and a very interesting thought. I’m currently looking into this but there are some technical and security challenges. Since this tech is still in its infancy, it won’t be user friendly for a while and would appeal to those who don’t mind getting dirty in code BUT I’m confident it will get to that point and people like those at Nabu Casa will simplify it so everyone can enjoy :)
Here, take my engagement.
Thanks UA-cam for the recommend
hahaha! Glad you like it 😁
Sir, I have to tell you that the clickbaity title had me doubting the content, but mother trucker that was good info. Ty, for posting this. Sub earned.
Haha 😂 glad I could deliver!!
I am relatively new on the Home Assistant scene and I am in the process of developing my interface and portals throughout our house. Since I am NOT a programmer by trade (I’m a musician 😜) I have a working understanding of coding operations and some syntax (but that changes from platform to platform. Anyhow, I’m encountering lots of hurdles, conflicts when it comes to certain devices not talking to the whole, and pinpointing what may need to be done or even where to start! I can’t help but think that this is where our “home assistants” no longer limited to Siri, or Alexa or Google, and begin to take on their own characteristics and “Jarvis-isms” 😅 Which is where I’d love the integration of something such as GPT3 to head towards.
Anyhow, just happened on here my chance and I’ll stick around for more insight and tidbits! 😊
Welcome and I hope you find the content helpful!
Only 5600 subscribers? You deserve more!
😂 let’s hope others feel the same!
Nice video! You’ve got a new subscriber!
Thanks and welcome aboard!
Great video. Great project
Thanks!
Amazing man, you’ll hit ten thousand subscribers soon if you keep it up. Algorithm recommended this
😁
ChatGPT also supports plugins, though you need the go through a waitlist for the beta (and maybe requires GPT4, though not 100% on that). Will likely make a lot of the integration pins a lot simpler in the future. But with plugins what is possible is kind of mind blowing. You could maybe do wild stuff like ask it to order food and then schedule turning on the porch light five minutes before the estimated delivery
I would love to build a plug-in for it to explore further!
Things are moving quickly. I can now simply ask chat GPt to write code-red code for complex situations. I first describe my room and everything in it then give it a scenario and ask it to give me the logic flow, which it does instantly. Very soon we won’t need to write any sort of code.
This is a great idea. I want chatgpt in the Google assistant speaker so that is can learn to be a character, remember things told to it, and interact as that character. Even if it is a one off type entity apart form Google.
I’m not sure if you’ve seen these set of videos but this may help push you in the direction you seek: How To Build A Telegram Chatbot That Controls Your Smart Home
ua-cam.com/video/wl8ZWBjpBoA/v-deo.html
I think the powerhouse use case would be for GPT3 to interpret conversational commands (like "turn off the light") and "normalize" the command (based on context like "command came from Den, the light that is on is the table lamp") and toss that specific command to HA Assist ("Turn off the table lamp in the Den"). My wife and daughter don't know the specific names of devices in order to send those commands, and so get frustrated - hell, most of the time I don't get it right. No need to generate code on the fly to do what the system already is capable of, just massage the command so that it will work within the constraints of the system.
That’s a very good point! I feel like this is definitely something gpt3 could do. the tricky part would be keeping track of relevant contexts like the fact you are in a specific location or it’s a specific time of day. But I’ll experiment to see what I come up with 🤔
@@technithusiast Based on today's HA livestream, it seems this is the way they're heading natively anyway, even using OpenAI Whisper! ua-cam.com/video/Tk-pnm7FY7c/v-deo.html
Thanks for putting me on to this. I purposely design my automations to be extendable so when new integrations/implementations are released I can easily take advantage of it. I’ll check this out!
This is where these voice assistants have utterly failed so far. Alexa is as dumb as rocks and I swear these things have made me start questioning whether I'm an abusive person with anger issues or is it just that the system is so poorly designed that it's enough to make anyone frazzled and exasperated. I can't wait for the day I can trash these things and have a proper integration that I can go in and tweak when things go wrong. As it stands, the Alexa team seems to routinely and arbitrarily change things in a way that makes it so that commands you set up the other week suddenly stop working the next. Maybe my setup is an isolated case, but I somehow doubt it. I mean, how hard is it to look at a list of devices and see that there is only one light with the word "overhead" in it so I don't need to specify the room as well? Sheesh
Alexa can do that kind of thing already if you have an alexa in each room paired to the same zone as lights. I can just say ‘turn off the light’ and it’ll do it in the room I’m in only.
Hey @Technithusiast, do you have any video going over the dashboard you show in the thumbnail? I would proper love to know more about how you set it up.
Thanks for inquiring! I REALLY like this dashboard too. I found it from a user on a home assistant forum but they didn’t share the code. They only shared screenshots of their implementation :(
Nice Open bro!
Much appreciated!
Like some others, the algorithm sent me your video, certainly because at one point I searched « chatGPT and HA »…
So, you earned a like and a sub… and I’m going to take some time to watch your previous vids.
(I noticed problem of sync between image and sound, difficult for me :(, and the background music, as someone already said, it’s a no no for me, it doesn’t add anything, and it’s harder to follow what you says (not English native here, I’m French, my English is pretty good, but I have adhd, and the music certainly don’t help…)
Thx man, keep up, you’ll soon have a lot more than 1k sub of the content is as good as this one !
Welcome and thanks for the feedback! I’m working on getting betting sound equipment so I can control the audio better!
New Subscriber here. Really like your style. 👍😊
Glad you like it! Welcome aboard!
Can you share a little more about the intent mapping? How did you train GPT what the verbal intent was for specific HA actions you have? Further, can you add utterances to that intent to make it more robust?
Ie perhaps GPT understands “turn the temp up” as intent=increase temp, but you also want to associate something like “make it toasty” or “it’s an ice box in here” as utterances that have the same end result (intent=increase temp)
Great question. The core of my solution is that I use GPT as a decision engine to determine my intent. Here is a general outline of the process:
- first I created trigger node that allows me to write a vague description of when the automation should fire (ex: “this alters the temperature using the thermostat”).
- attached to the trigger node is the rest of the automation for controlling the temp. This can be as hard-coding how high/low to change the temp, or as complex as enabling the automation to ask questions to specifically determine how to change it.
- lastly, the custom trigger function, I mentioned earlier, provides the vague description to GPT whenever I speak to home assistant. I basically tell GPT to let me know if what I say relates to the intent description.
The above orchestration is watered down as there are checks and balances in place as the above implementation gets wild the more intents you create. I’m releasing a video talking about this exact process in a few weeks that gives a walkthrough and I’ll provide some of the code that makes it run. In the meantime I do have some videos out that teaches the concepts that build up to what I describe above and it provides you the code to build your own. It’s my AI Masterclass series: ua-cam.com/video/1tq9UbLfDtY/v-deo.html
I never comment on videos, but I've been trying to accomplish something similar to what you have done in my HA setup, but in a completely different way with custom automations components and such. I love Node Red and would love to setup something similar, but I'm missing details here on what you shared. What is the "Get the intent" node? Any chance you would share your template for this?
Hey! So I’m currently putting together an AI Master class series where I show how to set it all up and explain some of those custom nodes. Those particular nodes will be discussed in the 4th video as they’re a bit complex and I want to work out as much kinks as possible before sharing.
Are you going to/or willing to share your flows or a starting point to implement this. Great stuff man!!
Check out Chaperone: montaque22.github.io/
I don't have every automation there but im slowly adding all of the stable ones
Google will definitely implement Bard into Assistant and IoT in the near future. They are also part of the Matter consortium after all.
Im sure they will too. The more pressure we can apply to these big corps the more they will have to bend to us... or buy us out😛
I agree with you about pricing, although I think that you can add chat gpt with speech to text recognition and text to speech so you can ask with your voice to do things and can reply with voice which will make things simpler. But you will need home assistant cloud I believe so that will again extract some money off your wallet.
Yup... but where there's will there's a way 😬. I'm hoping to combine some techniques i've discovered to create a pseudo speech to text mechanism until the Home Assistant team makes it more user-friendly
Let’s drop a 199 nd gel on that rear window. Content is fantastic!
Lol thanks 🙏🏾 Tbh, I need better lights but so I can drop the exposure 😅
@@technithusiast just gel the window, your lights are fine.
A ND .6 film would do the trick.
loving the content, subscribed - feedback - much of what you are doing is in NODE RED and is not native HA functionality. personally i think you need to address this in your content. Keep up the good work
Hey thanks for the sub and the feedback!
You need to do a follow up with 4o
I think you’re right!!!
Just a note, the background music was overwhelming the narration at times with headphones in. Thanks for the videos sir!
Thanks for the feedback!!
What's this sexy dashboard in the thumbnail? Can you share this theme/set-up? Thnx man!
I wished mine looked this dope! It’s just a thumbnail I created. The inspiration is someone else’s design that they did not make available.
Is that a real theme for HA in your thumbnail or just a concept? I need this
I need it too :(
It's technically a real dashboard but the dev only showed it off via pics.
Now that vicuna is pretty much just as good or better than ChatGPT-3.5 I would like to see integration with your own home ran LLM integrated with HA and using a better voice model for communication. Maybe eleven labs?
I never heard of vicuna but it seems like something I would try out once it migrates over to JS.
respecting your statement. "..theres no need to use chatgpt in home assistant .. (only), if i can do it."
I like pushing boundaries even if they don’t need pushing😬
Do you have a recommendation for the best "app" version of Chat GPT? I'm beta testing Bard, but honestly I might as well be working for them, because I have to always explain why their result is horrible.
Truthfully I haven’t tested or used any of the other apps cuz I find it takes time for the tech landscape to settle and the reliable options become obvious. I just have a strong suspicion that GPT will withstand this volatility so I’m experimenting with them
Now Cyberdine has full controll of your home movements and calender.
Don’t forget bank account lol 😆
My Favorite part of this whole video is "There is no need to use ChatGpt in HA, just to see if i could do it"
😆
I didn't understand how you integrated gpt3 with the home assistant from the video. cool tho.
No worries! Check out this video below. It walks you through how to use GPT in Your automations.
Create an Epic AI Chatbot Using GPT | Node-Red / Home Assistant / Telegram
ua-cam.com/video/1tq9UbLfDtY/v-deo.html
I wonder can you host a gpt inside your home and then give it access to your HA? 🤔
You could! If you can find an open source project for GPT you could host it on your own server and then use the api node or the web-hook to communicate with it
is there a way to import this into your own ha without building it?
Yes. I have a site where I am putting documentation for some of the automations I talk about in my videos.
montaque22.github.io
Currently the site only works on desktop but the mobile-friendly version will be ready soon. I’ll be updating it regularly so check back later
Can you use GPT to sync the audio with the video? 🤔
😂 yeah the timing was a bit off but info is solid :)
Ok I've liked. I've subbed. You appear to know your shit :). It'd dead interesting if you could say collect your overall energy consumption then say your socket draw and light automations then get insights like ("are there any areas of my automations that could optimize my electricity bills and is there any phantom / possibly unnecessary usage. Aswell a legit way to feed an API data say your fit data from your smart watch to identify what music makes your feel how. Conversational AI is really bad at creating but amazingly adept at explaining all kinds of labeled data. Idk if all that's possible on home assisstant but langchain may work.
You’ve hit the mail on the head! I’m currently exploring how to collect historical information for home assistant and creating a smart analytics that can not only give you insights into usage but alert you when you’ve breached thresholds. Glad your hear for this ride :)
You seem like a person that has decent hardware, gpt4all and local models might be beneficial to you. Also langchain (and its nocode GUI langflow) might help you connect the different models and commands. Also autogpt is making strides in automating the chatbot itself
Interesting 🤔 I’ll check those out. Gpt3 has some limitations that prevents me from easily implementing what I would like so if there are alternatives, I don’t mind exploring those.
Have you already made the how to install video on this?
Yup you can start here! How To Build A Telegram Chatbot That Controls Your Smart Home
ua-cam.com/video/wl8ZWBjpBoA/v-deo.html
If you are using the webpage prompt and have a subscription, you have access to GPT3.5 and GPT4, both with their own strengths and weaknesses.
What’s the webpage prompt?
You can do all of this with alarmo
I’ve heard of Alarmo and looked into it but I don’t remember why I wasn’t able to use it. I could take a second look at it but still doing it manually was a fun exercise 😬
What is that smart screen device? Alexa? How are you displaying a dashboard?
Google nest hub
@@evilleader1991 Ah, nice. Is that a web page in kiosk mode, or the Android Home Assistant app?
@@MacDork No idea as I do not currently own a nest hub, but I will have it pretty soon :P
Yup it’s just a plain ‘ol’ nest hub
You know what is complex? Setting up all the logic in node red to do what you want. I.e.v train your system by converting your ideas into logic.
Better instead to train the system using natural language. "Whenever someone enters the bathroom, turn the light on at the lowest dimmer setting, then turn it off after 10 minutes. But keep the light off during an electrical storm. Now everyone in the house is a trainer, not just the devs
Interesting.. if I understand you correctly, are u suggesting using something like GPT to write the automations for you based on situations you dictate?
@@technithusiast Sort of, but really more than that. Make an AI engine be the keeper of the logic as well as the logic engine. You train it using natural language input and a combination of config for inputs and outputs, it gets input from sensors, etc, and it controls the whole system via the outputs. It replaces all your automations and node-red implementations. It is your own personal local skynet.
I came to the comments section fully expecting it to be filled with attacks from angry Home Assistant nerds hating on you, but I see that people here have been positive and respectful. I need to stop wasting so much time on Reddit.
I'm interested in using a chat AI with my HA system as well, but I'll wait until I can use a totally locally hosted solution. There are already scaled down versions similar to ChatGPT available, though I'm not sure how they compare to the real deal since I haven't experimented with any of them yet. Take a look at LLaMA if this piques your interest.
My ultimate goal is to have a system that listens to everything I say, processes it *locally*, determines exactly what is meant to be addressed to the system and acts on only those commands or questions. For example, it's unlikely that I would be barking a command to turn on the lights to a guest or my cat. Pretty easy for a good AI to figure that out. Also, once advanced enough it can decide that I did not ask my visitor "what solvent can I use to thin wood filler?", for example.
I plan on experimenting with llama in the future as I explore this field. I would love to see a user friendly local AI cuz it would compete directly with the big 3 and force them to innovate in our favor
@@technithusiast I've long given up on "the big 3", and you should too. Have you ever had a girlfriend that keeps telling you that you just need to have patience while she tries to "get better"? For, like, YEARS... and years...? Yeah. I don't think so, thanks. Time to move on and broaden our horizons.
😂😂😂
It's nice that it's possible, but there is no chance in the world I'd willingly give my data to chatgpt for my home automation.
Lolol. So you don’t want to give it your bank info to manage your finances? 🤣🤣
But I feel you on that sentiment though. Today we give it the status of our lights, tomorrow we may give it our life choices. Where do we draw the line? understanding the bounds of any tech is crucial as it will unlock new boundaries of society. And some boundaries aren’t meant to be crossed.
@@technithusiast I’ve been, historically, quite optimistic and enthusiast about new technologies. I however, for the first time, feel very wary and uneasy about the latest AI developments.
Personally, for AI, if I don’t run it locally, I don’t trust it.
I respect that 🫡
But no install video?
Check out my AI Master class where I walkthrough the ideas and implement it within Node Red/Home Assistant
How To Build A Telegram Chatbot That Controls Your Smart Home
ua-cam.com/video/wl8ZWBjpBoA/v-deo.html
Of course, any bugs or errors, along with successful implementation - are placed in the GPT dataset.
Throw in some fun things, and it will get better for all of us !
Free sex, high powered cars, cheap vacations - we’re all good to go.
The dead air at the end, black screen, kept waiting for something funny, never came lol.
Lol my bad I didn’t realize it was there until after I published so I had to commit 🙈
@@technithusiast least you know some watch to the end!
Video and Audio are a little off
Yeah, I had some camera issue 😑
"I want it to be free"
Yeah... but it's not free for them to run it...
So why should they give you something for free, when it costs them money?
big tech give devs a minimal amount of their resources for free for testing and hobby projects and charge for anything more or commercial. I feel openAI could’ve adopted a similar model. 🤔But I ain’t mad at them though 🤷🏾♂️
could have been worth watching but the noisy background soundtrack put me off - makes your
explanation hard to hear
Hi Chris. Thanks for the feedback. if you felt this video was distracting but worth watching feel free to try some of the other content on this channel. The concept and ideas follow similar themes :)
On a serious note, you are wasting your as well as viewers time. What the title says and what are the rubbish you discussing is totally a kind of nonsense. Ask chatGPT for good topics and always to make videos on that. Think twice before make viewers fool.
Ohhh my second super critical UA-cam comment! How exciting😮
I’ll pass but thanks for your feedback though 😬
Bro 4:28mins in and you keep wasting time with lame humor just get it the point geeze
My first negative comment! I’ll cherish this milestone 😁
A tip:
- If you format your prompts (look into Langchain for easy ways to do this) you can inject real-time-ish data into GPT's context/prompt and it can then use that data to respond _with commands!_
For example:
####Prompt####
You are a home personal assistant. Warn the user if any doors or windows are open before arming the security system. You have access to the following devices and their data:
Bedroom Lights: ON
Bathroom Lights: OFF
Garage Door: OPEN
(etc...)
You also have access to the following commands, which you can run by entering them in square brackets [ ... ], and a text parsing agent will extract and run those commands.
[Arm]
[Disarm]
[Authenticate]
[Lights;;]
[Doors;;]
(etc...)
Chat history:
...
User: Please turn on the lights in the bedroom.
Agent: Of course, User! [Lights;Bedroom;ON] I've turned the lights on in the Bedroom.
User: What is the state of the garage door?
Agent: The Garage Door is OPEN.
User: Please arm the security system.
Agent: User, I can not arm the security system, because the Garage Door is OPEN. (OR "Would you like me to close the Garage Door, first?")
User: Please close the garage door.
Agent:
###End example prompt###
I've experimented with this style of prompting, and it works great! You will, of course, need to parse the return text for commands and handle each of them, as well as tweaking the prompt until it behaves in the way you want. Additionally, and most importantly, you need to have some way for the Agent to identify the speaker as trusted. I used passphrase authentication and a simple time-out logic. "Hello User! It has been greater than 10 minutes since your last interaction, so you'll need to reauthenticate. Please speak your passphrase to continue." Security is a WIP, and I'm hoping that voice authentication makes a few strides this year, as there doesn't seem to be a system which can accurately identify a voice, nor pick a voice out of a crowd, yet.
For a deeper dive, try implementing long-term memory, via either/or/and of Summarization, or Vectors/Embeddings, either of which Langchain can provide, and which give your Agent and effectively infinite 'context' to work with.
Ahh you’ve been jumped ahead of me!!! 😁
In my AI master class series I uses these exact techniques to enable GPT to make decisions on which automations to run and what actions to take within home assistant. It’s dope to know someone else has been experimenting in this fashion too. If you’re interested you can check out the series to see my implementation of this. ua-cam.com/video/1tq9UbLfDtY/v-deo.html
@@technithusiast I'm totally blown away by how powerful these things are! Been really looking at them as a 'Cognitive Engine' and seeing them in that view really makes their capabilities pop!
hey if you want i can show you some python where you can use gpt3 with custom tools so if you asked turn of the lights it would ask gpt "from my list of tools could any of these help" it would see you have a tool to do whatever and execute it. its what ive been using for my assistant im working on if you want i can give you my discord
Sounds dope. I’m not that well-versed with python but if you have a node-is implementation I would love to check that out. I have a master class series where i explore similar functionalities you outlined above ua-cam.com/video/1tq9UbLfDtY/v-deo.html