This is a great and clear presentation. I already had Mosquitto and node-red running on a pi4 to be always on. With the help of this video, I installed influxDB (which I didn't know existed) and Grafana on a proxmox container. It all works together, and I can have a single graph with the inside temp from a little ESP-32 project and my outside temperature courtesy of my Acurite weather station. Thanks so much!
Thanks Gutsy, I'm glad it helped! This video is quite out of date now regarding the specifics of installation steps, but I think the value is in the overall structure and how the various elements work together. Once you understand that, setting up each piece of the puzzle can be done using their specific instructions.
As an introduction into software stack this was extremely helpful as i just bought some I/O from emerson and there was no "guide" that didn't require me to upfront company money. I watched this whole thing and it was very helpful on understanding how it all works together. Thank you very much!
Nice tutorial. I’m a big fan of mqtt, node red, influxdb and Grafana and wild agree that this is the way to go to support future projects. 100k tantalisingly close - well done Jon, well deserved.
That is by far the best video I have seen on Node Red! Most “How to” videos are so quick that it is really hard to follow, yours is perfectly paced and beautifully explained. I now feel confident in setting up InfluxdDB and Grafana. Thank you so much
Clean and simple demonstration of all the essential tools importantly with good documentation to allow me to follow it - Fantastic kick start for automation and data recording - Thanks
You can also take the json object and send to influx. If you do that you dont have to split each value Up. And if your sensors follow a Common system in terms of json data vs topic then you can have 1 single Line that takes all data from your sensors to the influx. Then you dont have to care when you add more sensors :) i do that for both tasmota, My own sensors and others. The data ends Up tagged and with proper series names IF done properly :)
@@SuperHouseTV also use tags/Keys in influx. Makes it easier to Group and show data without having to do queries for each sensor but instead use filters :) i have way more than 1000 data sources above is a must :) you could argue to send the data to influx directly or use telegraf but i like having all My data distributed via mqtt :) since i also use alot of the data for automations where they dont exist in ha. (I have used node red longer than ha)
Dear Jonathan. Thank you for making this video and making it available for us. I had been looking around for a informative video on the topic, since past few days. You have made it easy for us to understand the flow and enabled us to implement it ourselves.
SPLENDID, JON! THANK-YOU SO MUCH! I very much needed a detailed walk-through like this to get from zero to first-working. Trying to do this kind of first-time integration on my own is beyond painful, especially given how little spare time I have after my day-job. It takes weeks instead of hours for me to get things done starting from scratch, so not very many things get done! My wish-list item would be for one more level of time savings: Automation to create database entries on the fly for each mqtt data item the first time it is seen, then add line charts to a dashboard to display them. I'd very much rather prune unwanted items than add the wanted ones one at a time! I've never used NodeRED, but I suspect it should be able to handle this task. NodeRED looks to be a powerful OpenSource alternative to some of the National Instruments LabVIEW tools I've used in the past (and learned to love to hate). And I see PyNodeRED (pypi.org/project/pynodered/) will let me create and import modules written in my language of choice, Python, which I use at work for data transformation and analysis (and everything Bash won't do). Again, this video is awesome. It feels as if it were created just for me, to greatly accelerate my learning curve and make me instantly productive. I didn't even know the questions to ask, and you answered them all. THANKS! Next step: On to the web page for this episode! Edit: Really, this combined with your Tasmota walk-throughs gives me a soup-to-nuts recipe for data acquisition, transmission, archive and display. Thanks!
Hi BobC, thanks for your positive comments! Being able to automate the end-to-end setup of a new data point would be fantastic. Since recording this video I've been told about a way to simplify the JSON-to-InfluxDB step, so I'm going to add that info to the tutorial page as I'm finishing it off. It doesn't automate the process of creating dashboard widgets, but at least it could save some time setting up Node-RED for each individual measurement.
Oh man, you saved my day! Formating the data with json in Nodered was exactly was i was looking for - THE WHOLE DAY! Now my Jelly switch transfers numbers and not strings anymore. Thank you SOOO MUCH!
Thank you for sharing . I am working on a project related the internet of farmingbased on what we call a '' Plant Speaking Approach" and I find your video as great plus . I am using (influx db and grafana) home assistant. As I just bought a dragino gateway with LoRa. Definitely , I will get in touch with you for more advises . I have a question regarding the storage capacity.Can we use an external storage to store the data as I am looking to use for statistical modeling in the future. Thank you once again for sharing and believing that the open source is great way to shape the world in a sustainable way.
Thanks LiveArt! For long term storage a very important thing to set up is InfluxDB data retention policies. By default, InfluxDB just keeps everything you throw at it. Over time the amount of data can become unworkable. Retention policies allow you to specify things like averaging data across periods (effectively "thinning it out") so that you have detailed data for recent events, and for older periods of time you have less detail. For example, you might have 1 minute samples for the last month, 5 minute samples from 1 month to 12 months, and then hourly samples beyond 12 months. This prevents your storage getting out of control and keeps queries running fast.
@@SuperHouseTV Thank you for your reply and I appreciate your effort of sharing and spreading knowledge . Thank you once again. Happy Holidays to you and family .Stay safe.Cheers.
Excellent video. Yesterday I added InfluxDB, Grafana and Telegram to my OpenHab + NodeRed + Zigbee2mqtt setup and I was interested to see if I missed anything. I didn't, but it this tutorial would have definitely saved me time in working out how to set it all up. I'm using node-red as the data bridge and rules engine with telegram only for alerts (and, later, maybe specific controls) - mainly because node-red is very powerful and local, whereas telegram relies on the internet. So for my water leak sensors and mains water valve cutoff system, I want that to operate simply and locally.
Very good explanation of how to setup an extensive logging and visualization platform. Thanks! I indeed came here for informations on telegraf, but I'm also relieved that one does not have to use telegraf.
Wery useful video, thank you Jon! I would like to noticed: if someone wanna use the authentication from the version of mosquitto 1.5 , then need to write an additional row into the mosquitto.conf: per_listener_settings true If this row is missing, than the authentication will not work (actually without password will work the mosquitto_sub example (at 4:13).
31:03 when i select measurements, it shows nothing. I have data of weather. I configure influxdb and successfully( data comming from sensor via mqtt in nodered) store data of weather sensors. I can see it by commands but in grafana dashboard i didn't get data. Even influxdb database is configured in grafana successfully. I have 3 type of data in influxdb database(temperature, humidity, pressure). Help me if possible please...
Got it working very nice instructions. The only issue is that when you share Grafana, any random user can edit the Grafana dashboard. I could share just the plots (Kiosk mode), but when you hit Esc you go back to dashboard.
Thanks excellent vid, I used it to get my water sensor graphing, only thing I got caught out by was InfluxDB creating the values as a string so your example didnt work, more googling and a change expression $number(payload) in between the mqtt and influx nodes sorted things out
Hi PhatPizza, im have the exact same issue. Which node did you use to change it to numbers to fix this? Ive been battling for about 3 weeks with this now.
@@garyjoseph6260 As explained I used "Change" Function then created a rule to set msg.payload to an j:expression of $number(payload). Note, I had to drop the measurement as I couldnt find out a way to change it from a string. It can be checked by running the command SHOW FIELD KEYS FROM "your measurement"
Fantastic video. I wish I had know more about grafana two years ago. I spent years creating a new concept for my company adding embedded “senors” to monitor the operation of a real-time core 5G network component and the only missing element was a presentation layer (GUI). All the data (sensors) were in a database yet we couldn’t get funding for the final missing GUI/presentation layer. I’m actually gutted I couldn’t complete my concept that would have revolutionised the monitoring of this critical product ☹️ Open Source is frankly amazing! 🥰
Pretty god damn awesome. If you didn't do code reviews as in the previous video in this series I wouldn't have been as interested. I think the format of doing code walk throughs of microcontroller projects is one of the best ways to teach your viewers what's happening! One tip in general for the code you do as a good habit to develop. Instead of calling a function again and again and again in your main loop that always returns again and again the same return call data value such as when you are doing all the sprintf function calls to get the ESP ID move the function call out of the main execution loop() and up into the one time setup section. Then you only need one single call to that function for the entire time the microcontroller project is running -- assign the return value of the single ESP ID function call in setup and assign the data value in the return call to a variable -- or the hexadecimal equivalent that you want for this particular example. Then in your main loop() just reference the variable directly to save repetitive calls which when you design all your code this way can lead to faster execution times, faster time to reach that critical next sleep call and therefore more power savings -- less critical when powered by a wall wart power supply but very critical and good habit for when you start to build battery powered microcontroller solutions where every ounce of runtime life you can squeeze out per battery charge counts for something.
Super cool tutorial. Pulling data from a Sensirion SCD30 into a 3 gauge cluster for CO2, RH, and temperature. I need to learn how to combine outputs on Arduino to create one json instead of three. Thanks again.
Thank you this !!! This is really great. Explanations are clear, easy to follow, and the seed is great. You explained so much in a single video. Please keep posting!
Very nice video. Thanks. It's exactly like my setup. I hope there will be a sequel. I have been trying for a long time to figure out how to handle long-term data. For example, when I want to draw an annual graph, Raspberry is weak, it takes a very long time and it heats up a lot. So it is necessary to downsample the data. But this downsampled data goes into new metrics, and that means a new graph for long-term data. That seems silly to me.
Have you looked into InfluxDB retention policies? As part of the retention policy you average older data points so to minimise space at the expenses of accuracy
Yes, retention policies are something I should cover in a follow-up. It's very important on any datalogging system, but particularly on low-end devices like a Pi.
I did this. Excellent video mate. After all, I needed to know my beer fridg ewas always at the right temp.HeHe. Not joking about that either. Grafana says my beer is -0.69C right this second.
Excellent video! However, I'd like to know what happened to the presenter's shirt between 2:45 and 2:58... one moment the presenter's shirt looks brand new, then suddenly it looks old as if it's been worn for about 20 years... 😅
Great work Jon!! I have seen performance issue using Nodered. Rest of the components works flawless. I had switched to telegraf and the performance was far better.
Great tutorial! I've always wondered about Node Red and now this makes sense on how to set it up. I'm probably going to do this as a container in ProxMox as am concerned about the excessive writing to the SD card on the Raspberry Pi.
Good tutorial to follow... I would just add don't use the RPi for Grafana if you are going to have multiple dashboards/panels... After few dashboards and multiple panels (don't ask my why but I have 10+ graphs per dashboard) the interface slows down quite a lot. And a good idea would be to move everything to a USB drive on the RPi.
I wouldnt say its the Rasp as such that is the the issue. The challenge here is to get the data out and if you havent inserted the data in a good way into Influx and you pull big data sets you will kill the IO. Its common that people use 1sec data and then want to pull days or weeks of data out and ignoring the built in functions to make sure its possible :D And if you then use slow SD card you get even more issues :D I use SSD hooked up to my raspberry pi systems today and you can have 100 graphs easy without issues on same dashboard if done correctly. Yes a dedicated server with NVME is faster of course :D
Built in aggregation queries and data retention policies are very important, and I'd love to cover them in a follow-up video. Hopefully this is enough to get people started though :-)
Great video Jon, thanks for filling in a lot of gaps for me in the whole sensor to presentation stream. Node Red and Grafana look like really useful tools for this application. Much appreciated.
Excellent video/ tutorial as always Jonathan. Thank you for all the time you spend creating them. I'm just about to teach my students IOT, what a better way of backing up my teaching by pointing them at your videos... Hope life is getting easier in Sunny Australia..
That's fantastic, thanks John! I'm glad the videos are useful. Since you teach this sort of thing professionally (I'm just an amateur who makes it up as I go along) if there are specific topics or ideas that would be useful for your students, please let me know.
Hi, I came across your channel today. I've to ask a favor: in which video do you explain the json setup for your sensor? I like to do that for my BME280. THX for that super video. Now I now where and how my weather station project goes.
In my case i get these message from mqtt - "{"Time":"2022-01-26T07:26:16","ANALOG":{"A0":185}}". how can I convert this string to a number (A0). I cannot use the string in influx db.Kindly HELP !!!!!
hi, i have mqtt sensors data sends every 5 minutes from esp32, but i want to save some data/not all sensors data to influx only at hourly based, the 5 minutes frequency only for node red dashboard displays.. need to know how to do this
Hi, I really like your videos and have followed them for years. I always learn a lot from you. On this one i will suggest that you use a SSD drive in stead of a SD card for the system and the database. in this way you increase the reliability of the system over time.
I personally run this software stack in Docker on a Synology NAS, and I may do a follow-up video showing how to do it that way. For this video I wanted to make a tutorial that explains just the software itself, without having to also explain containers. Also, containers are a big time-saver, but it can also get people into the habit of treating each container as a black box, not knowing what's happening inside it.
@@SuperHouseTV theres a great project called IOTstack that has a gui for installing pre-setup containers for all of this and more (for instance i also use zigbee2mqtt with a zigbee dongle to get prebuilt zigbee sensors into influx) sensorsiot.github.io/IOTstack/
Can I ask a question, how do you enable the USB serial ports in docker, I want to use Arduino nodes to collect and send data over serial, is there a file I can edit for the Node Red container?
Hi Jonathan thanks for this great tutorial .I've achieved to send the data from my Higrow sensor to Influx and Grafana.Now I need your help .How can we send the data to a sensor in Homeassistant and see it in Lovelace.
Nice video! For step 7, is there a way to automatically split all the JSON items and store each one in the DB without having to manually copy/paste/change each entry?
I have some Sonoff compatible breaker switches that it is not possible to flash with Tasmota. They are controlled by my Home Assistant server using Sonoff Lan. Is there any way I can connect them to my MQTT broker? I would like to get the power consumption and switch state data from them into Grafana and thought that using the MQTT broker would work but have run into a problem.
Hi, excellent video 👍 One question: If I wanted to add actuators, what would I have to add to that architecture? What software or technology would I be missing? I would appreciate your answer. 👍🇵🇪
Nice video! I actually did connect home assistant to influxdb but node red is also a very good option! The only problem I've come across is that after a certain period of time tje data in influxdb gets corrupted. Long story short: Rpi OS actually works natively in 32bits while influxdb is actually 100% 64bits software. Since then I actually use Ubuntu Server instead of rpi os, which does work in 64bits. I also did make a docker-compose file so that I can easily recreate the stack whenever I need, "the guy with a swiss accent" (andreas spiess) has some pretty good videos on that.
Good points, thanks. I personally run these software components in Docker on a Synology NAS, but for this tutorial I wanted to provide a basic step-by-step guide that would show a system that didn't introduce more things to explain. I'd like to do another guide which is showing the same 4 software components, but installed using Docker instead and using external mount points to retain data.
This is a great tutorial - i'm going to give it a spin. Is there any way to have user-clickable actions on the dashboard to send messages to IoT relays etc to complete the experience?
Hi Jon. I far the issues are being compounded...Influxdb no longer installs using this process at all because influxdata appear to have revoked the repository key...Is there any fix do you think...?
Hi Jon, really thanks, very nice video!!! I have a question, in the case we want to connect some devise outside our network (Ex: a sonoff with tasmota in my brother house) and I have Raspi in my house, How we configurate the mqtt link to the broker? Could you help me? Thanks in advance. Sebastian
You've a choice between direct (e.g. a VPN tunnel or secure passthru) between the sites or indirect, e.g. using another messaging service like telegram, that both sites communicate with. Personally, I'd try telegram first as its easier to set up, if you are comfortable with the level of security you can achieve.
Excellent video! Thanks a lot! I'm actually working on something just like this...but I'm getting an error with influxdb (i guess). After installing infkuxdb and adding some sensors to store data (from PLC) the Raspberry Pi disconnects from internet and stops storing data.
This is a great and clear presentation. I already had Mosquitto and node-red running on a pi4 to be always on. With the help of this video, I installed influxDB (which I didn't know existed) and Grafana on a proxmox container. It all works together, and I can have a single graph with the inside temp from a little ESP-32 project and my outside temperature courtesy of my Acurite weather station. Thanks so much!
Thanks Gutsy, I'm glad it helped! This video is quite out of date now regarding the specifics of installation steps, but I think the value is in the overall structure and how the various elements work together. Once you understand that, setting up each piece of the puzzle can be done using their specific instructions.
As an introduction into software stack this was extremely helpful as i just bought some I/O from emerson and there was no "guide" that didn't require me to upfront company money. I watched this whole thing and it was very helpful on understanding how it all works together. Thank you very much!
Nice tutorial. I’m a big fan of mqtt, node red, influxdb and Grafana and wild agree that this is the way to go to support future projects. 100k tantalisingly close - well done Jon, well deserved.
Thanks Adam!
I was waiting for someone to make a video like this a long time ago! With exactly all this elements, thank you so much!!
That is by far the best video I have seen on Node Red! Most “How to” videos are so quick that it is really hard to follow, yours is perfectly paced and beautifully explained. I now feel confident in setting up InfluxdDB and Grafana. Thank you so much
Clean and simple demonstration of all the essential tools importantly with good documentation to allow me to follow it - Fantastic kick start for automation and data recording - Thanks
You can also take the json object and send to influx. If you do that you dont have to split each value Up. And if your sensors follow a Common system in terms of json data vs topic then you can have 1 single Line that takes all data from your sensors to the influx. Then you dont have to care when you add more sensors :) i do that for both tasmota, My own sensors and others. The data ends Up tagged and with proper series names IF done properly :)
That's a fantastic tip, thanks!
@@SuperHouseTV also use tags/Keys in influx. Makes it easier to Group and show data without having to do queries for each sensor but instead use filters :) i have way more than 1000 data sources above is a must :) you could argue to send the data to influx directly or use telegraf but i like having all My data distributed via mqtt :) since i also use alot of the data for automations where they dont exist in ha. (I have used node red longer than ha)
I wish this video existed months ago when I was trying to figure this out. Great job! Thank you!!
This is an excellent, detailed yet understandable walk through of this whole setup. Absolutely awesome work!
Dear Jonathan. Thank you for making this video and making it available for us. I had been looking around for a informative video on the topic, since past few days. You have made it easy for us to understand the flow and enabled us to implement it ourselves.
SPLENDID, JON! THANK-YOU SO MUCH!
I very much needed a detailed walk-through like this to get from zero to first-working. Trying to do this kind of first-time integration on my own is beyond painful, especially given how little spare time I have after my day-job. It takes weeks instead of hours for me to get things done starting from scratch, so not very many things get done!
My wish-list item would be for one more level of time savings: Automation to create database entries on the fly for each mqtt data item the first time it is seen, then add line charts to a dashboard to display them. I'd very much rather prune unwanted items than add the wanted ones one at a time! I've never used NodeRED, but I suspect it should be able to handle this task.
NodeRED looks to be a powerful OpenSource alternative to some of the National Instruments LabVIEW tools I've used in the past (and learned to love to hate). And I see PyNodeRED (pypi.org/project/pynodered/) will let me create and import modules written in my language of choice, Python, which I use at work for data transformation and analysis (and everything Bash won't do).
Again, this video is awesome. It feels as if it were created just for me, to greatly accelerate my learning curve and make me instantly productive. I didn't even know the questions to ask, and you answered them all. THANKS!
Next step: On to the web page for this episode!
Edit: Really, this combined with your Tasmota walk-throughs gives me a soup-to-nuts recipe for data acquisition, transmission, archive and display. Thanks!
Hi BobC, thanks for your positive comments! Being able to automate the end-to-end setup of a new data point would be fantastic. Since recording this video I've been told about a way to simplify the JSON-to-InfluxDB step, so I'm going to add that info to the tutorial page as I'm finishing it off. It doesn't automate the process of creating dashboard widgets, but at least it could save some time setting up Node-RED for each individual measurement.
This is by far THE BEST video on "how to use" .... all these appts !!!
Nice walk through!
I am just starting with node red. You can set the format on the mqtt input node to JSON, so you don't need the JSON node.
Oh man, you saved my day! Formating the data with json in Nodered was exactly was i was looking for - THE WHOLE DAY! Now my Jelly switch transfers numbers and not strings anymore. Thank you SOOO MUCH!
Thank you for sharing . I am working on a project related the internet of farmingbased on what we call a '' Plant Speaking Approach" and I find your video as great plus . I am using (influx db and grafana) home assistant.
As I just bought a dragino gateway with LoRa. Definitely , I will get in touch with you for more advises . I have a question regarding the storage capacity.Can we use an external storage to store the data as I am looking to use for statistical modeling in the future.
Thank you once again for sharing and believing that the open source is great way to shape the world in a sustainable way.
Thanks LiveArt! For long term storage a very important thing to set up is InfluxDB data retention policies. By default, InfluxDB just keeps everything you throw at it. Over time the amount of data can become unworkable. Retention policies allow you to specify things like averaging data across periods (effectively "thinning it out") so that you have detailed data for recent events, and for older periods of time you have less detail. For example, you might have 1 minute samples for the last month, 5 minute samples from 1 month to 12 months, and then hourly samples beyond 12 months. This prevents your storage getting out of control and keeps queries running fast.
@@SuperHouseTV Thank you for your reply and I appreciate your effort of sharing and spreading knowledge . Thank you once again. Happy Holidays to you and family .Stay safe.Cheers.
Excellent video. Yesterday I added InfluxDB, Grafana and Telegram to my OpenHab + NodeRed + Zigbee2mqtt setup and I was interested to see if I missed anything. I didn't, but it this tutorial would have definitely saved me time in working out how to set it all up. I'm using node-red as the data bridge and rules engine with telegram only for alerts (and, later, maybe specific controls) - mainly because node-red is very powerful and local, whereas telegram relies on the internet. So for my water leak sensors and mains water valve cutoff system, I want that to operate simply and locally.
Thanks for the video. I heard about Node-Red, InfluxDB and Grafana combination and never explore it. This video make the process easy to understand.
Having played with Node-RED a few years back and all that, this was a beautiful refresher and extension adding InfluxDB and Grafana. Thanks!
Stumbled on this video just now… It’s one of the most helpfull vids i’ve seen on this topic! My compliments and thanks! .. subscribed immediately!!
Very good explanation of how to setup an extensive logging and visualization platform. Thanks!
I indeed came here for informations on telegraf, but I'm also relieved that one does not have to use telegraf.
Wery useful video, thank you Jon!
I would like to noticed: if someone wanna use the authentication from the version of mosquitto 1.5 , then need to write an additional row into the mosquitto.conf:
per_listener_settings true
If this row is missing, than the authentication will not work (actually without password will work the mosquitto_sub example (at 4:13).
this is like a one stop shop tutorial. Brilliant!
Danke!
Excellent tutorial. Brilliantly explained. You should be a teacher.
Great tutorial! Thank you! I was struggling with Nr and Grafana.
31:03 when i select measurements, it shows nothing. I have data of weather. I configure influxdb and successfully( data comming from sensor via mqtt in nodered) store data of weather sensors. I can see it by commands but in grafana dashboard i didn't get data. Even influxdb database is configured in grafana successfully. I have 3 type of data in influxdb database(temperature, humidity, pressure). Help me if possible please...
Got it working very nice instructions. The only issue is that when you share Grafana, any random user can edit the Grafana dashboard. I could share just the plots (Kiosk mode), but when you hit Esc you go back to dashboard.
Thanks excellent vid, I used it to get my water sensor graphing, only thing I got caught out by was InfluxDB creating the values as a string so your example didnt work, more googling and a change expression $number(payload) in between the mqtt and influx nodes sorted things out
Hi PhatPizza, im have the exact same issue. Which node did you use to change it to numbers to fix this?
Ive been battling for about 3 weeks with this now.
@@garyjoseph6260 As explained I used "Change" Function then created a rule to set msg.payload to an j:expression of $number(payload). Note, I had to drop the measurement as I couldnt find out a way to change it from a string. It can be checked by running the command SHOW FIELD KEYS FROM "your measurement"
Excellent n cool presentation. I just wanted to ask,,,,,while node-red has its own dashboard then why did we use Grafana here?
Excellents video. Quick question, what would be the procedure if I wanted to access Grafana from a website instead of locally?
Fantastic video. I wish I had know more about grafana two years ago.
I spent years creating a new concept for my company adding embedded “senors” to monitor the operation of a real-time core 5G network component and the only missing element was a presentation layer (GUI).
All the data (sensors) were in a database yet we couldn’t get funding for the final missing GUI/presentation layer.
I’m actually gutted I couldn’t complete my concept that would have revolutionised the monitoring of this critical product ☹️
Open Source is frankly amazing! 🥰
Pretty god damn awesome. If you didn't do code reviews as in the previous video in this series I wouldn't have been as interested. I think the format of doing code walk throughs of microcontroller projects is one of the best ways to teach your viewers what's happening! One tip in general for the code you do as a good habit to develop. Instead of calling a function again and again and again in your main loop that always returns again and again the same return call data value such as when you are doing all the sprintf function calls to get the ESP ID move the function call out of the main execution loop() and up into the one time setup section. Then you only need one single call to that function for the entire time the microcontroller project is running -- assign the return value of the single ESP ID function call in setup and assign the data value in the return call to a variable -- or the hexadecimal equivalent that you want for this particular example. Then in your main loop() just reference the variable directly to save repetitive calls which when you design all your code this way can lead to faster execution times, faster time to reach that critical next sleep call and therefore more power savings -- less critical when powered by a wall wart power supply but very critical and good habit for when you start to build battery powered microcontroller solutions where every ounce of runtime life you can squeeze out per battery charge counts for something.
Super cool tutorial. Pulling data from a Sensirion SCD30 into a 3 gauge cluster for CO2, RH, and temperature. I need to learn how to combine outputs on Arduino to create one json instead of three. Thanks again.
Thank you this !!! This is really great. Explanations are clear, easy to follow, and the seed is great. You explained so much in a single video. Please keep posting!
Very nice video. Thanks. It's exactly like my setup. I hope there will be a sequel.
I have been trying for a long time to figure out how to handle long-term data. For example, when I want to draw an annual graph, Raspberry is weak, it takes a very long time and it heats up a lot. So it is necessary to downsample the data. But this downsampled data goes into new metrics, and that means a new graph for long-term data. That seems silly to me.
Have you looked into InfluxDB retention policies? As part of the retention policy you average older data points so to minimise space at the expenses of accuracy
Yes, retention policies are something I should cover in a follow-up. It's very important on any datalogging system, but particularly on low-end devices like a Pi.
Glad to have found your channel Johnathon, I really enjoyed this and hope to put it into practice.
Great! You did an awesome job explaining everything in a way anyone can follow!
I did this. Excellent video mate. After all, I needed to know my beer fridg ewas always at the right temp.HeHe. Not joking about that either. Grafana says my beer is -0.69C right this second.
Excellent video! However, I'd like to know what happened to the presenter's shirt between 2:45 and 2:58... one moment the presenter's shirt looks brand new, then suddenly it looks old as if it's been worn for about 20 years... 😅
This makes me want to have another go at Node-Red. Thanks for the great information.
On the website the step 6 and 7 are missing information.
If you can complete that it would be great.
Hi thanks a lot for the video. Also what model is your air quality sensor please
Great work Jon!! I have seen performance issue using Nodered. Rest of the components works flawless. I had switched to telegraf and the performance was far better.
Great tutorial! I've always wondered about Node Red and now this makes sense on how to set it up. I'm probably going to do this as a container in ProxMox as am concerned about the excessive writing to the SD card on the Raspberry Pi.
Good tutorial to follow...
I would just add don't use the RPi for Grafana if you are going to have multiple dashboards/panels... After few dashboards and multiple panels (don't ask my why but I have 10+ graphs per dashboard) the interface slows down quite a lot.
And a good idea would be to move everything to a USB drive on the RPi.
I wouldnt say its the Rasp as such that is the the issue. The challenge here is to get the data out and if you havent inserted the data in a good way into Influx and you pull big data sets you will kill the IO. Its common that people use 1sec data and then want to pull days or weeks of data out and ignoring the built in functions to make sure its possible :D And if you then use slow SD card you get even more issues :D I use SSD hooked up to my raspberry pi systems today and you can have 100 graphs easy without issues on same dashboard if done correctly. Yes a dedicated server with NVME is faster of course :D
Built in aggregation queries and data retention policies are very important, and I'd love to cover them in a follow-up video. Hopefully this is enough to get people started though :-)
I just wanted to say thank you for this very in-depth and great explanation! definitely gained a new subscriber.
Great video Jon, thanks for filling in a lot of gaps for me in the whole sensor to presentation stream.
Node Red and Grafana look like really useful tools for this application. Much appreciated.
Excellent video/ tutorial as always Jonathan. Thank you for all the time you spend creating them. I'm just about to teach my students IOT, what a better way of backing up my teaching by pointing them at your videos... Hope life is getting easier in Sunny Australia..
That's fantastic, thanks John! I'm glad the videos are useful. Since you teach this sort of thing professionally (I'm just an amateur who makes it up as I go along) if there are specific topics or ideas that would be useful for your students, please let me know.
How can I watch the dashboards in any place or device?
Hi, I came across your channel today. I've to ask a favor: in which video do you explain the json setup for your sensor? I like to do that for my BME280. THX for that super video. Now I now where and how my weather station project goes.
In my case i get these message from mqtt - "{"Time":"2022-01-26T07:26:16","ANALOG":{"A0":185}}". how can I convert this string to a number (A0). I cannot use the string in influx db.Kindly HELP !!!!!
Bravo
You are a very good 'teacher' also for non uk native language like me.
Thanks
Great video. What Air Quality Sensor are you using? I'm looking for something similar with a MQTT output
hi, i have mqtt sensors data sends every 5 minutes from esp32, but i want to save some data/not all sensors data to influx only at hourly based, the 5 minutes frequency only for node red dashboard displays.. need to know how to do this
Hi, I really like your videos and have followed them for years. I always learn a lot from you. On this one i will suggest that you use a SSD drive in stead of a SD card for the system and the database. in this way you increase the reliability of the system over time.
Fantastic walk-through - exactly what I have been looking for. Thanks!
Wow. It finaly makes sense :) Succinct and clear explanations. I'm in! Thank you.
Thank you for that great tutorial! I am sure I can use the information directly as I am redesigning my smart home.
Super video... yes, long but worthwhile, educational and tells the story perfectly
Any reason you didn't use docker?
I personally run this software stack in Docker on a Synology NAS, and I may do a follow-up video showing how to do it that way. For this video I wanted to make a tutorial that explains just the software itself, without having to also explain containers. Also, containers are a big time-saver, but it can also get people into the habit of treating each container as a black box, not knowing what's happening inside it.
@@SuperHouseTV it was a great video and I enjoyed it!!
@@SuperHouseTV theres a great project called IOTstack that has a gui for installing pre-setup containers for all of this and more (for instance i also use zigbee2mqtt with a zigbee dongle to get prebuilt zigbee sensors into influx) sensorsiot.github.io/IOTstack/
Excellent! Thoughtful balance of flexibility vs complexity. Can’t wait to build my own stack on my RPi. Well done, great demo!
Can I ask a question, how do you enable the USB serial ports in docker, I want to use Arduino nodes to collect and send data over serial, is there a file I can edit for the Node Red container?
would be easy to extract each device data from a single influx Database? like, i would have to build several flows for each device inside Node-red?
Hi Jonathan thanks for this great tutorial .I've achieved to send the data from my Higrow sensor to Influx and Grafana.Now I need your help .How can we send the data to a sensor in Homeassistant and see it in Lovelace.
Nice video! For step 7, is there a way to automatically split all the JSON items and store each one in the DB without having to manually copy/paste/change each entry?
Is it scalable? And can we send commands from the grafana dashboard to sensor to carry out a task?
I have some Sonoff compatible breaker switches that it is not possible to flash with Tasmota. They are controlled by my Home Assistant server using Sonoff Lan. Is there any way I can connect them to my MQTT broker? I would like to get the power consumption and switch state data from them into Grafana and thought that using the MQTT broker would work but have run into a problem.
Very good explanation for the combined software projects. It helped me a lot.
Thanx for your work!!!
Hi, excellent video 👍
One question:
If I wanted to add actuators, what would I have to add to that architecture?
What software or technology would I be missing?
I would appreciate your answer.
👍🇵🇪
*Really good and comprehensive tutorial. Thank You very much!*
Could you make a video using Iobroker as a alternative with MQTT, InfluxDB and Grafana?
Great video, just what I was looking for, Thanks very much Jonathan!
Nice video! I actually did connect home assistant to influxdb but node red is also a very good option!
The only problem I've come across is that after a certain period of time tje data in influxdb gets corrupted. Long story short: Rpi OS actually works natively in 32bits while influxdb is actually 100% 64bits software. Since then I actually use Ubuntu Server instead of rpi os, which does work in 64bits.
I also did make a docker-compose file so that I can easily recreate the stack whenever I need, "the guy with a swiss accent" (andreas spiess) has some pretty good videos on that.
Good points, thanks. I personally run these software components in Docker on a Synology NAS, but for this tutorial I wanted to provide a basic step-by-step guide that would show a system that didn't introduce more things to explain. I'd like to do another guide which is showing the same 4 software components, but installed using Docker instead and using external mount points to retain data.
nice overview and video... now I want to build an air-quality monitor as well!
Excellent explanation 👍
great video, you mention a discord server, but there is no link in description
This is a great tutorial - i'm going to give it a spin. Is there any way to have user-clickable actions on the dashboard to send messages to IoT relays etc to complete the experience?
do you have solution to your question?
Absolutely fantastic video as everyone else is saying. Fun, inspiring and useful!
You really made a very good job, in all of your video. It's very interesting and very useful. Many thanks !
Bravo, Got me thinking again
Hi Jon. I far the issues are being compounded...Influxdb no longer installs using this process at all because influxdata appear to have revoked the repository key...Is there any fix do you think...?
Great vid! Does Grafana come with non-editable view mode?
You create the viewer account.
Excellent content and well presented
Very well explained. Thank you
Great video really good explanation 👍
Hi Jon, really thanks, very nice video!!! I have a question, in the case we want to connect some devise outside our network (Ex: a sonoff with tasmota in my brother house) and I have Raspi in my house, How we configurate the mqtt link to the broker? Could you help me? Thanks in advance. Sebastian
You've a choice between direct (e.g. a VPN tunnel or secure passthru) between the sites or indirect, e.g. using another messaging service like telegram, that both sites communicate with. Personally, I'd try telegram first as its easier to set up, if you are comfortable with the level of security you can achieve.
can this be installed on the same raspberry pi that I have homeassistant on ?
Congrats my friend! Very nice 👍🏼
I love you. Thats exactly what I was looking for.
Or in German: Danke bester, genau nach dem Zeug mit deinem Vorgehen habe ich gesucht! :D
Super Video from Superhouse....big fan 👍👍👍
Hi, how can iget data of an atlas scientific temperature sensor RTD in node red and raspberry pi 4?
Excellent video! Thanks a lot! I'm actually working on something just like this...but I'm getting an error with influxdb (i guess). After installing infkuxdb and adding some sensors to store data (from PLC) the Raspberry Pi disconnects from internet and stops storing data.
Superb video and information!!!
Nice, I learned a lot, Could you make a mobie app and show the chart of Gurfana in the next video?
Thank you so much for putting this together!
Wouldn't installing Home Assistant be a lot easier then you have a GUI to work home?
Really helpful and easy to follow tutorial!!! 😃😃
Great detailed video! uUst what I needed to know.
You have a link for the mods for the ikea tables for that enclosure? :)
Excellent video ! Great Job !