Вставка
- Опубліковано 10 вер 2024
- What is going on with my ESP32's WiFi connection? And how to use a Circular Buffer.
JLCPCB Only $2 for PCB Prototype any colour jlcpcb.com/
So we look at WiFi connectivity issues with my ESP32 Dev Board (Revision 0) and then compare its Internet performance with a faultless Wemos Lolin NodeMCU V3 ESP8266..
We also take a good look at circular buffers which are built into the ESP32 core.
All this and more (like the ESP8266 sketch) in my GitHub:
github.com/Ral...
INFORMATION
Thingspeak - an IoT cloud-based company for makers to upload data easily
thingspeak.com/
The 30-pin ESP32-WROOM pinout and more information
circuits4you.c...
Important information about which GPIO pins can be used on an ESP32
desire.gieseck...
List of all my videos
(Special thanks to Michael Kurt Vogel for compiling this)
bit.ly/UA-camV...
All about creating circular buffers - but we have the capability to use the ESP32's built-in version
embeddedartist...
See the code for the ESP32 core version (no example, though)
github.com/esp...
All this and more (like the ESP8266 sketch) in my GitHub:
github.com/Ral...
PRODUCTS
*To support my channel please do click on the affiliate links below before ordering ANYTHING from Banggood, AliExpress or Amazon. It expires 24 hours after clicking it so do come back and re-click at any time!*
The 2.8" TFT touch screen used in my demo for just $6.66 (plus $2.05 shipping) or from USA & UK direct:
2.8 Inch ILI9341 240x320 SPI TFT LCD Display Touch Panel SPI Serial Port Module
www.banggood.c...
38-pin ESP32 with headers for just $4.99 plus $2.25 shipping ($9.00 plus 0.99 shipping if ordered from USA):
www.banggood.c...
**NOTE the 30-pin works just as well, the extra pins on the 38-pin variant are not for end-user use anyway!
Amazon UK sell this 2.8" screen (in the UK) for just £12.50 + free delivery!
amzn.to/38utf96
NB: Make sure you select the TOUCH version!!!
Amazon UK also sell the 38-pin ESP32 DEv Kit for just £5.55 in the UK:
amzn.to/3mFhy30
AliExpress offer both 30-pin and 38-pin ($3.36 + $1.04 shipping) of the ESP32 Dev Kit:
www.aliexpress...
I couldn't get an affiliate link for above, click here first: s.click.aliexp...
That's a link for a bare ESP32 with a whopping 16Mb on board! But needs an expansion board/PCB.
If you like this video please give it a thumbs up, share it and if you're not already subscribed please consider doing so and joining me on my Arduinite journey
My channel, GitHub and blog are here:
\------------------------------------------------------------------
/ ralphbacon
ralphbacon.blog
github.com/Ral...
\------------------------------------------------------------------
I had a very similar issue. Turns out I was using too much heap memory and when trying to connect the radio internals didn't have enough RAM to properly initialize. With all the other bits and pieces you probably need about 25-30kb of free heap to get a stable connection. I am fortunate though because I was using a dev-board that I could switch on PSRAM. I use the IDF and you can also configure the WIFI to use PSRAM. After a bit of memory juggling, its now rock solid stable. In any case maybe check the memory usage and free internal heap space just before connecting.
I did have some reboots (kernel panic) when I was experimenting with my ring buffer using normal heap memory (hence I could only use 10K) but with my WROVER unit I have allocated 150K (10 seconds) and all is well in the ESP32 kingdom. So far. Early days yet.
I'm confused. The ESP32 Rev 0 is from more than 4 years ago and the Rev 0 has been KNOWN to be buggy since introduction which is why it was quickly followed by the Rev 1 which solved those initial problems. Why are you using known buggy hardware from several years past and then telling ExpressIF they need to "up their game" when they already did almost 4 years ago???? LOL! Look at the number of comments accepting your assessment as if it relates to any hardware produced in the last 3 years. You do a real disservice when you don't start a video like this with a disclaimer such as, "I'm using known buggy old hardware that was fixed years ago by the manufacturer with a silicon revision, but I'm going to act like that's not the case." Somebody needs to up their game indeed.
He did mention the possibility that there cou,could, be a flaw on 0. He also said he looked and couldn't find anything specifically addressing g the issue, so KNOWN, appears to be a relative quantity. Do you have a reference that addresses this issue?
@@kentswanson2807 The broken disposition of rev 0 silicon is absolutely KNOWN since late 2016. Totally absolutely KNOWN. Zero questions. They're broken. It has several silicon bugs affecting data corruption due to the bus multiplexer which has the job of sharing the RAM and internal peripherals between the two cores. This exposed problems occuring in the pipelines which their IDE tried to counter by simply inserting NOPs to delay things and give the pipe a chance to clear. These are data bus corruption issues. You can't characterize them so easily because they depend on the particular data bits in play and code timing so looking for the software issues presented in this video by name is like looking for a explanation of why you can't order fancy French meals or trapeze lessons from a plumbing company. It should be obvious to most everyone but the plumbers probably won't advertise that they don't offer trapeze lessons. For a starter, see 3.3, 3.10, 3.16, and 3.18 in www.espressif.com/sites/default/files/documentation/eco_and_workarounds_for_bugs_in_esp32_en.pdf
@Peter S Please I think you can now stop replying to everyone with the same comment over and over, we've all heard you!
We cannot be sure what revision we'll be buying (from the Far East, anyway) as Espressif _allegedly_ dumped their Rev 0 chips for a song - which means we can still get them now and again.
Secondly, my Revision 1 chip also misbehaves - but ran for nearly two days without error and I thought I'd found the solution. It just lulled me into a false sense of security.
Now I'm running on a WROVER (same ESP32 Rev 1 chip of course) just to prove the circular buffer theory and it's all going well... so far, 1 hour in. Just like the previous test.
So I thought maybe it might be my router (although power cycling that didn't help either). So I tried connecting to my phone as a hotspot (WiFi turned off) and it had the same errors. Sigh. But I still have theories, not yet tested. We shall see. But please, no more comments about Rev 0 vs Rev 1.
@@RalphBacon Ralph, I was replying to individual persons who were under the impression that you are bringing up a relevant concern with the integrity of the ESP32 and with EspressIF. As of this post, almost 1500 people have heard you assess this chip based on known bad silicon which was replaced almost 4 years ago. If you want to question a product's integrity, you would be suited to do so with current hardware that isn't known to be broken in published errata from 2016. Did you see how you've influenced your viewers of this video? They're questioning the ESP32 platform because of the level of trust you've built. I think that trust in you is well deserved, but in this specific instance, not so much. Rev 0 silicon was only manufactured between August 2016 and January 2017 so the chances of getting a rev 0 in any legitimate product is zero.
Peter, I understand your point of view but I think they are right to question this product; who's to say what revision they have tucked away or what we will get in the post from the Far Eastern warehouses - and whether that cures these problems? Despite your continued affirmation that Rev 1 solves all problems, it didn't for me. Same error. AUTH_FAIL plus DNS failures .. just like the Rev 0, albeit after a couple of days. If I'd had this with my ESP8266 project it would never have got built. Until I get this working faultlessly (which I certainly hope I do) I can only stand by my empirical evidence. You don't work for Espressif do you?
An excellent explanation and graphic illustration about the ring buffer principle, well worth listening to you Ralph, well done..
Glad you enjoyed it
Hi Ralph, I know how frustrating it can be when something like that happens. This is a very interesting series on the esp32. Thanks for your hard work.
Glad you enjoyed it, Joseph!
I've got an ESP32 using WLED to drive a bunch of individually addressable LEDs. its been running for about 2 months without a reboot. I use hyperion to stream updates to it via UDP at 60 frames a second, comparable to what you'd see streaming music. Never had wifi connectivity issues with it.
Go on, Trey, make me jealous! Oh, you have done! Well, so far, 24 hours after I received it, my ESP32 Rev 1 WROVER (with the extra 4MB PSRAM) has performed faultlessly but it's early days yet. I hope to be able to report what you have experienced soon.
I have never streamed to or from an ESP32 but the time I have had "funny" WiFi problems with an ESPx it has been down to power supplies and/or decoupling. That's where I'd start looking. Also check that the regulator feeding the circuit is rated for the max. current draw.
Possibly also consider interference from the SPI when the screen updates causing the WiFi to drop out.
I, too, concentrated on the power (from a powered USB hub) as a source of issues, Pete, especially as I would get a brown-out reboot now and again. It's now running from a dedicated 5v 2A supply with a whopping big capacitor across VCC and GND. The result? Well, that and more in next week's video! Spoiler: looking good but that was not the only change I made.
New subscriber here. I made the comment about the caps on the VS1053 board last week. I think you should try to minimize the amount of devices and wires to the ESP32 to check if it is not an RF(I) issue which is causing this.
Also a different power supply and cable (management) can help if RFI is the issue. (rfi ferrite rings)
In my house I can hardly connect anything on lower 2.4GHz channels. The signal from my AP is strong, but it is no guarantee that reception is perfect. In my case I suspect a wireless camera in the neighborhood. The 2.4GHz band is free so anything goes within the limits I would say even if it is not wifi. Wifi analyzers on devices only show wifi signals but that doesn't mean that there are no other signals messing it up.
Looking around on your channel I have a lot to learn about those modules. Thanks for the nice tutorials!
Good suggestions you make, Luc (and thanks for the sub too). Certainly the wires are a mess and introduce lots of electrical noise. I have switched to a dedicated power supply which helped quite a bit. I also updated the ESP32 to a Revision 1 and that helped too. Watch next week's video for an update and some clarifications.
I feel your pain Ralph. Been there and done that with rev0 chips. Now using rev3 on some of my boards without problems.
The problem is how do we know what we will get? I'd love a Rev 3 chip if I knew that was what I was going to get, but they are rarer than hen's teeth right now. And if you find one, $$$.
@@RalphBacon I know what you mean Ralph. I stopped buying from china. Many of these are not the real deal. Some won't run at 240MHz etc. I get mine from Mouser now. Maybe a $ more but the cost is really in the noise if you value your time.
It is strange
I have no problems with WIFI
connection
Maybe the problem is with a specific manufacturer of the board?
It _could_ be the dev board, I suppose, the quality of the components, for example, but in this case I've found it was definitely the revision of the chip that makes Rev 0 and Rev 1 like chalk and cheese, in reliability terms. Watch the next video for more!
.... just a hint, not sure it will work here.
i had similar problems when is started using wifi based chips. i did not read your code, but my problem was that my functions where taking too long and the processor needs time to do some sort of a "heartbeat" to keep the connection alive. my solution was to use some "yield();" calls, which allows the esp to do his job to keep the connection alive. so if you can find some places to stick some yield() calls, you could try it.
yield() is an esp8266 requirement not needed by esp32 as it runs wifi on a different chip (core) to the sketch - which i suspect is the real issue
When I ported this to the ESP8266 I had to introduce yields or else I got panics from it, as I had not coded the ESP32 with that in mind, knowing the WiFi was on Core 0. But the ESP8266 was (and still is) rock solid. But sadly not enough pins.
Had the same kind of problems with my ESP32 weather station until I changed my router... after that everything works fine. The ESP8266 though worked fine with both routers.
You're another viewer who said this. Perhaps my ancient Cisco router is just too old now for modern electronics and needs an update?
That's the version 2 with esp 32 of my heating controller out the window, at least for now. Back to the esp8266 version with external pins using an pcf8574 (also thanks to you Ralph, by far the best intro to this smal chip).. Now I just have to figure out how to implement fast analog for RMS sampling from at least two inputs (phases)..
Ole - ADS1115 for your analog inputs. Great chip, connects via I2C and is hellish fast :)
Don't jump so quickly. This video is based on rev 0 hardware which has been known to be broken for over 4 years. That's why rev 0 was replaced by rev 1 in February of 2017. This video does more harm than good as evidenced by it impacting your design process.
@@Peter_S_I will just put that revision on hold for now, I will still pursue using Esp32, but as for Ralph I have had a solution with Esp8266 functioning rock solid for more than a year, slow and steady adding more and more sensors and functionality. But maybe I'll better just continue with both, and for this season keep running the old and trusted 8266, and when the heating season is over, switch to esp32 for stability testing. But I will also add that I have had stability problems with esp32 (revision 1) in the past, playing around with it, but newer had problems with esp8266, but then again this could be board manufacturing quality (Esp32 Devkit V1)..
@Ole Eide What you could do is source an ESP32 (Rev 1 or better) and write a simple sketch to connect every few minutes while calculating prime numbers or something just to keep it amused, and see how long that works for. The you will know whether it would be good for your project.
keep in mind where this esp32 came from.
very well may be a QC reject.
Absolutely! In fact, the rumour mill has it that the ESP32 Rev 0 chips were dumped and found their way into these modules we buy from China, probably without anybody in the chain being particularly aware of potential issues. Except now I am very much aware!
wouldnt be surprised if they using them rejects on ESP32-Cam boards... ive had nothin but trouble with them... forever power issues..killed 3 just trying to get enough power in them to run the camera without stupid lines through the image
ive had the same WIFI issues... had to go back a version on the arduino ESP32 core..
Interesting. Do you know what version of the ESP32 you have (it's shown at the start of every upload)?
i have multiple ESP32s..
ESP32-PICO-D4 (revision 1) on ESP32VGA
ESP32D0WDQ5 (revision 1)
on TTGO T-18
ESP32-cam (not sure on the chip but im sure its a rev 1)..
im 100% sure its nothing to do with the hardware itself.. its the esp32 core.. boards would work fine then all of a sudden wont connect..some more than others.. i found a forum post saying to try downgrading the core from 1.0.4 to 1.0.3... did that and havent had an issue since..
A new core, 1.0.5 RC2 is available on GitHub, and I would have used it if I had known how to coerce PlatformIO into using it. But, as it happens things have settled down - watch next week's video!
I think the critical thing is the esp32-arduino library version. Last summer I had these endless wifi connection problems when developing my ESP32-CAM-Video-Recorder, but they all disappeared with arduino-esp32 v1.0.4 in October 2019. I am using ESP32-CAM modules which are AI-Thinker WROVER modules "Chip is ESP32D0WDQ6 (revision 1)", and one has been running for 7-8 months without problems. The router has restarted a couple times, but the ESP32 reconnects without intervention. I normally use the Arduino IDE which lists "Using library WiFi at version 1.0 in folder: C:\Users\James\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\WiFi "
I think PlatformIO has that feature of fetching libraries for you -- if you are switching and keeping local copies -- you might be getting a 1.0.3 library mixed into the pot somehow?
I'm going to check that, right now...
[some time later]
I have to admit that within PlatformIO I don't know how to even check what version I have, let alone update it.
I don't have the libraries in my "lib" folder so I guess it's all dynamic. I wanted to test out the release candidate "ESP32 Arduino 1.0.5 RC2" to see whether this stabilized things too but I can't figure out how to get a non-release library into PlatformIO.
I could do this in Arduino-Land but I don't want to move all my code (again). Sigh. So much to learn.
@@RalphBacon the library wfi of the esp32 should be on C:\Users\\.platformio\packages and look for esp32 framework. Another way ist to use "Go to definition" in Visual Code on #include "WiFi.h". Anyway great video, thank you very much for sharing it.
@@cuperativa @Ralph S Bacon It is all very confusing.
In C:\Users\James\.platformio\packages\framework-arduinoespressif32\package.json
I get "name": "framework-arduinoespressif32", "version": "3.10004.201016"
which means 2020 Oct 16 ?? and maybe the 10004 is a version of 1.0.4 ? Don't know what "3" is.
The objects are here, but not version labeled that I can see?
C:\Users\James\Documents\PlatformIO\Projects\james-program\.pio\build\esp32cam\FrameworkArduino
espressif
/esp-idf is current v4.1
platformio
/platform-espressif32 is current v2.0.0
espressif
/arduino-esp32 is current v.1.0.4 with the v1.05-rc2 development
I've tried v1.0.5-rc2 for camera issues, but not wifi.
The old bug was that the wifi connection would fail, but the esp32 wifi did not fully realize this, so you had to do a wifi.disconnect() to underscore the fact, and then things would all resolve. This is the code that I used at the time:
eventID = WiFi.onEvent([](WiFiEvent_t event, WiFiEventInfo_t info) {
Serial.print("WiFi lost connection. Reason: ");
Serial.println(info.disconnected.reason);
if (WiFi.status() == WL_CONNECTED) {
Serial.println("*** connected/disconnected issue! WiFi disconnected ???...");
WiFi.disconnect();
} else {
Serial.println("*** WiFi disconnected ???...");
}
}, WiFiEvent_t::SYSTEM_EVENT_STA_DISCONNECTED);
@cuperativa Thank you for guiding me on this. I found the library that Platformio is using where you said it would be. It's revision 3.10004.201016. so quite old really. I'm surprised it's not 1.0.4. I might try just splatting it with the V1.05-RC5 or wait awhile, now that my setup is working very reliably.
Hi I use an ESP32 as an internet weather station, it's never missed a beat Wi-Fi never drops out. Not sure what version board it is has ESP-WROOM-32 on the can.
Go on, rub salt into the wound why don't you! Actually all is well in the Bacon workshop this week, watch Friday's video.
Excellent description of ring buffers. Thank you.
Glad it was helpful!
Could it be an antenna issue? Some of the ESP32 boards don't have the copper ground plane removed under the antenna section of the PCB. Others have the jumper set to use an external antenna. I've not had any issues once I've made sure the antenna is OK but I'm using the ESP32-IDF not Arduino.
I suspect changing to a different board might fix things but due to layout and not silicon rev.
He's using rev 0 silicon which has been KNOWN TO BE BROKEN for over 4 years and was quickly superseded by rev 1 silicon.
I read a report last night about using ESP-IDF which seems to things differently than the ESP32-Arduino core. Made me think it might be worth installing just to test this theory out. Ideally on a Rev 1 chip but worth trying with both. We shall see if I have time to do this.
I had ESP8266 boards (several of the same type) not being stable with Wifi, where same code in same location reproducably runs fine for months on a different board. Which makes me think it‘s not necessarily the chip, but its implementation (board)...
Indeed, and power is one thing I am testing here as the ESP32 is quite power hungry in the WiFi area - a sudden drop in available current might make it unstable.
There is always a reason for WiFi to fail. While error logging is not always perfect, here are a few things you can do:
1. Enable internal WiFi driver logging with 'esp_log_level_set("wifi", ESP_LOG_VERBOSE);'
2. Print available heap from time to time and when an issue happens
3. If none of the above helps you find the cause, post an issue with the logs here: github.com/espressif/arduino-esp32/issues/new/choose
I've already done (1) and (2) and the heap is always fine (100K+ free at any one time). When I increased the ring buffer too much I got a panic and reboot.
But I've made several changes to fix this (not least changing the dev board to one that uses a Rev 1 chip) so keep tuned!
👍Nice presentation and good sleuthing some interesting facts, and discussion in comments. I stand taught. Very patient too, as I would have likely hit the point of ditching in frustration over enjoyment vs need vs cost of real vs suitable intrinsic reward. Sometimes projects require personal reevaluation. 😉
I'm told "ring buffer" has a very different meaning on other 'Tubes, . LOL 😁
Glad you liked this one too, Scott. It was quite frustrating but I guess I won't give in too early in the development cycle. I have no idea what else that could mean and I don't think I want to find out!
How much does buffering make sense for a real-time stream? Yes it prevents gaps of silence but it introduces jumps in the stream. Example: halfway a song to the next song because the data in between is missing. I dont know which of the two is more annoying.....
Well, the weird thing is that when nothing comes down the line (client.available == 0) and I therefore cannot buffer/play, when it does resume then no information is missing, like the server (or Internet) had been buffering on my behalf. Weird. No gaps. So a nice large embedded buffer would work well. I have the WROVER version (with a 150K buffer) running right now, but it's far too early to say if it will work any better!
You can make really cool stuff with an ESP32-Wrover. I used the High-Speed DMA-SPI to drive a 480*320 Display. The additional PS-RAM can be used for big images which makes it possible to create modern GUIs on an ESP32 (I used the LittleVGL library for the UI)
An excellent use of the PSRAM, Fabian. I wonder if SPIFFS could be made to use it. We could then store lots of web pages there.
I've got a 150K buffer for this project now running on the WROVER and it seems fine (buffer always around the 98-99% full, which is about 10 seconds, depending on bit rate).
Ralph,
My simple version of the Web Radio is working. Now I'm in the process of porting it to FreeRTOS and I wonder.
What are your thoughts [is your motive] by populating the ring buffer with 100 byte chunks from the stream and reading them from the ring buffer into the VS1053 buffer by 32 byte chunks?
This is important because when we use a queue to communicate between tasks this Q can only hold 100 bytes OR 32 bytes. Thanks in advance.
My reasoning for this was that we can only give the VSW1053 32-bytes but we can read from the URL stream many hundreds of bytes at a time (probably much more). Therefore, to ensure the buffer was always as full as I could make it, I chose the arbitrary value of 100-bytes. I could have made it larger, really, but that was a "sensible" amount to test with - and it stayed like that.
Remember that I have a separate task reading the circular buffer, taking 32-bytes at a time and feeding it to the VS1053 so as long as there are 32-bytes available to it, it is happy. That task doesn't block the ring buffer at all (neither does the main task) on the basis that one is reading and one is writing and therefore updating different values in the circular buffer (tail/end/contents).
Any of this make sense?
I have made a esp32 deauth detector for a multitude of reasons. But in my experience things like the soldering iron can send out the smallest deauth signal.
I've upgraded to Revision 1 of the chip and it seems (much) more stable.
Thanks for the warning. I haven't run into that in my projects yet but I'm not streaming. I look forward to your investigations!
His problems stem from using known broken rev 0 silicon from 2016 that nobody should waste a moment using. Espressif released revision 1 silicon in February 2017 and today ships revision 3.
Indeed, maybe if I didn't stream it might not be discovered. But then again, if I switched the WiFi on and off (like my HomeAlone project) I wonder if it would connect reliably, every time?
Ralph, I know this is off topic, but I have been looking for a command to use on the esp32 that would turn off all GPIOs at once as opposed to having to digitalwrite each one to 0. Do you know of any easy way to do this? I have a esp32 connected to a number of relays in my irrigation controller with a node-red flow that automates it thru mqtt commands. I need a way to turn off all the relays at the end of a irrigation cycle.
You _can_ do this but it requires some lower-level programming. You can directly modify the register that controls whether a pin is on or off (as well as whether OUTPUT or INPUT but I've not covered that here).
To SET pins 18 and 27 HIGH you would use:
REG_WRITE(GPIO_OUT_W1TS_REG, ((1
@@RalphBacon Thank you Ralph for your very detailed answer. Sorry I have been away from the pc for a few days. I will play around with your code and see if it works for me.
I had the same problem with ESP8266 chips, because of that I have never used one of those again.
Interesting. And disappointing, as I've never had an issue with the ESP8266 chips (Wemos D1 mini). Try a newer revision? (But I don't know how to check that, they do not report it like the ESP32 does).
I've also had difficulty with esp32 connection stability. Are you using espressif or Arduino code?
I'm using Arduino-ESP32 core. My next step was to implement ESP-IDF but I haven't got to that point yet as my WROVER has arrived!
Ralph,
In you presentation you said: "If the buffer is full, the stream has to stop writing into the buffer until some data has been read out".
Is that possible; to stop the data from coming in?
No, it isn't possible to prevent the transmission - what it means is that you miss (don't read) that input which means it's lost forever unless you have another buffer that continuously reads the input, regardless, possibly overwriting the oldest data.
I Like it! Every time You showed new software is so exciting .. this time CoolTerm, good, thumbs up!!!!
Glad you like it! Yes, Roger (of CoolTerm fame) deserves all the support he can get, he's not a pro developer just does it as a hobby but it is really good.
Ralph , I enjoy your presentation style and your topics are very instructive. keep it up. Weisser , Schwarzwald
Thanks, will do!
We had issues at work with this. It turned out to be faulty firmware. Wasn't involved personally, just heard my collegues talk about it. So you have to rewrite part of the firmware or wait until a patched firmware is available. No ifs and buts about it. If I recall correctly it had something to do with some kind of wait /delay not handled correctly, meaning some routers and wifi s not being compatible even though they should.
Indeed, that is what Peter S is stating: that I am using a Rev 0 module with known firmware issues that were (mainly) corrected in the Rev 1 version. (Even newer versions are available but getting a V3 is like getting water in the desert right now.) It's just a pity he said it far too many times with the inference I should somehow have known that Rev 0 was buggy! I think we got the message.
@@RalphBacon hehe, yeah we sure as heck didn't know. So I wouldn't feel too bad about it except It's annoying that it doesn't work as it should. Good luck.
Hi Ralph, there are WEMOS 2866 with 16M but less pins the display is less desirable, the HW SPI is also slower. I wouldn't like to multiplex the SPI, so another processor for the display as the information rate is not great, perhaps via I2C. Graham
I have a Nextion which I've been meaning to try out! Now's my chance!
Question: how are you assigning the IP for your ESP32? The reason I ask is I've seen some DHCP servers assign IPs for only an hour. If there's an issue renewing the lease, that can cause connectivity issues. A quick test would be to use a static IP entered into the ESP32, so there's no negotiating for a lease. That would explain why you still couldn't connect after switching APs - the ESP32 still had the same MAC address and the same lease issue with the DHCP.
I've used DHCP and a static configuration (as part of diagnosing this) and none if it made any difference. The lease on any DHCP address is "forever" too. I might be much, much closer to a working model if 48 hours of (so far) uninterrupted running is anything to do by.
I often use ring buffers, but I normally use 3 control values: in, out, count
As you show the count is not needed, but I like using it and it has the advantages that the consumer knows how many bytes is available (It can be computer with only in/out).
Depending on data producer/consumer each entry do not have to be one byte, but could be 32 bytes or a pointer to a data block. It is something you will have to decide when coding, depending on what is best for the case.
In some programming languages you do not need to code these buffer, but can instead use library routines (Often called queues), but due to overhead they are best used when each entry is a data block, not single bytes.
The ESP32 Core (for Arduino, and I guess, for ESP-IDF) has a built-in circular buffer class that I have hacked to use PSRAM if it is available. It determines empty/full/head/tail the way I showed it (pretty much) and I like that too without having external variables but if you like them then why not?
I haven't used many ESP32 Dev boards as I use nearly all LOLIN32 boards and the one I checked this morning was Rev 1. Having said that I have had intermittent problems on one project, no disconnects but delays with data sometimes which was shown with a continuous ping where most pings were 10mS or less but around every 20 to 30 seconds one ping would be 100mS or more, this is caused by the WiFi Power Saving Mode being on by default. You can disable this by adding esp_wifi_set_ps (WIFI_PS_NONE); into setup before you start the WiFi, maybe worth a try but it does consume more power but that would only be an issue with a battery powered projects.
I just checked I already do this (remembering issues with my ESP8266 Home Alone project that went to sleep) and I do execute:
WiFi.setSleep(false);
which translates to:
esp_wifi_set_ps(WIFI_PS_NONE)
which is just as you suggest. As you rightly say, unless this was battery powered (not using this dev board!) it's best to keep things awake and eager to spring into life at a moment's notice.
That said, I have _reduced_ the power from the TX side of things as I am on top of the AP with:
WiFi.setTxPower(WIFI_POWER_MINUS_1dBm);
I'll see how that goes an up the power if I need to.
Question< I love the ESP32. But is there a reason the pinouts are scrambled and not in numeric order? Weird. i know a couple are but you have to jump around to find a GPIO. I'm sure they could have routed the circuit board better.
Unless you go for much more expensive multilayer boards, it's not easy to route the traces differently when that's the order they come out of the chip. Also none really cares about the order of the pins, because when you draw your schematic it doesn't really matter. At board layout the software keeps track of them.
It is true that routing pins is harder than you might expect on a standard 2-layer PCB (unless you want umpteen vias jumping from top layer to bottom layer). Additionally, the board manufacturers try to keep logical pins together if they can (eg I2C, SPI) even if the physical chips pin are spread about.
As @superdau says, it doesn't matter (unless you confuse VCC with GND, then it might) just have the pinout in front of you at all times (as I do).
Did you try the esp32 v2? I have many and i have lots of issues with smart home things that i have to keep re-register them often. Its annoying and the company just tells me how to re-register rather than a proper answer
Apparently, Revision 3 chips correct everything wrong with Rev 1 and Rev 2 chips, but it is still annoying, agreed. I don't have a Rev 2 to test (never officially released, apparently, just some beta tests) and I certainly cannot get a Rev 3. But they will appear one day in our Chinese modules and then all will be well with the world, you'll see!
15:58, I don't like that ONE BIT :-)
Love that!
Great intro to circular buffers
Hope for a happy end for your project!!
Thanks for sharing :-)
Thank you! Cheers! It's all working now, as you will see on Friday! Just don't tell anyone else, it's our little secret.
Had a similar problem last week it was the router playing silly and rebooting it or back to factory still didn't fix it.
My router was giving them a IP address , but blocking all traffic .
Had to turn it into a access point and use my spare router to get thing working again .
Crazy! Unfortunately I used my phone as a Hot Spot (no WiFi) and although it connected it would not get past the AUTH_FAIL issue. But the WROVER version is now running so we will see!
I don't use the board, just thinking about your explanations: or it is within the hardware revision or it is something else and must be within the program. I explain, how can a ringbuffer program that start and stops disconnect a system from internet. Indeed it can disconnect from the stream and has problems to reconnect but the wifi connection should stay stable, not? The only thing I can imagine if it's not the own program that causes this issue is that it's one of the libraries where there is some kind of 'buffer overflow' that is writing into parts of the wifi environment.
The ring buffer was introduced well after problems appeared with this device so I'm pretty sure that didn't affect it, although I like your problem-solving thinking, Marc.
However, I don't want to jinx anything but changing the board to a rev 1 (and other changes) has made it rock solid, now running for (over) 48 hours without a glitch - and believe me I tried to break it, from switching off my AP (ring buffer working well) to rebooting a few dozen times too.
The ring buffer. There are many type of circular array/ring buffer. The one you are demonstrating is a bounded FIFO. When it's full it's full. This is fine in some scenarios, particularly when the data is important and such a buffer overrun would be considered a fault. In embedded systems processing realtime data in a small memory footprint however it is usually not the best choice. Consider audio. Is it worth holding on to old data when there is newer data arriving and the buffer is "full", the ultimate result of doing so is buffer backlog and a failed stream at source.
The approach there is that the buffer can never be considered full. A source asking for somewhere to put data MUST be honoured. The simplest way to adapt yours to this approach is to just write the record to the current head position and not moving the head pointer. Thus continuing to overwrite the last element of the buffer until the slow consumer catches up.
All of this spawns around the classic problem in software engineering, which you can google, of slow/fast consumers. It gets nasty :)
Which brings me to a learning point I had recently around all of this. The simple "layman's buffer" you demonstrate will only work if the reader and write are running in the same execution thread. This is easy to control on small single core micros with the ability to disable or non-re-entrant your interrupts. The instant this stops being the case the circular array buffer becomes a LOT harder to implement. Often requiring private and public head and tail pointers and mutexes or other IPC.
In Uni we ended up with a circular array buffer which supported many readers and a single writer concurrently in the same buffer, running on multi-cores. It would take a lot longer than a few minutes to explain that! :) A very fun exercise. In Java it wasn't too hard, but in C/C++ it was a nightmare trying to protect pointers from incorrect, accidental or optimistic use from overrunning the other threads reserved areas.
The ring buffer I demonstrated here (as far as I can remember, anyway) was pretty much what I ended up with because, would you Adam & Eve it, Espressif had already implemented such a ring (circular) buffer.
Well, almost. When used with PlatformIO (on VSC) it could not access the PSRAM properly so I had to hack their code (it's still there!) but once that was done it worked flawlessly. I can allocate anything from zero to the full PSRAM memory. I found that 150K gave me about 8 seconds of buffered audio which was enough to cater for most Internet 'glitches' (drop-outs). And that was low enough for the hourly time signal and news not to be too far off reality!
I read the Internet audio stream and try and put it into the buffer; if that fails I try again a bit later (a millisecond or so). Some stations give me 100% buffering (i.e. using all 150KB) others only use 30% at best. I guess the radio stations' servers buffer more (or less) according to their individual setups.
Anecdote: Sky TV had lots of complaints from viewers during the 2018 world cup, apparently. Those watching via Sky would hear their neighbours watching on live terrestrial shouting "GOOOAAAL" well before it happened on _their_ screen. So much for buffering!
Hi Ralph,
You do not say how long your radio player runs before you hit the WiFi problem. In support of the ESP32 I will say that I have been running a weather station , that runs on a Lolin D32 that Arduino IDE say it is version 1, for around 18 months without any problems. But, it does only run for a few seconds every half hour and then go into deep sleep, so perhaps this reestablishing the WiFi link is hiding the problem. I am currently making a monitor to display data from MQTT that is also running on a Lolin D2 and the 2.8 TFT screen that you are using. Whilst it is not finished I have just started a soak test, what it is doing is receiving data from ESP8266 driven sensors that update data to MQTT every 15 mins and also it checks NTP time ever hour. I will leave it running from a battery bank and just see what happens.
A couple of questions, How do you do your logging and could the output go to SPIFFS or Littlefs? Can you please do a video on PlatformIO. I can install it on Windows and do Blink, but when I tried to add you Internet Radio Pgm it started asking for software from Github that I could not find.
Sorry for the long post.
The problem (that I had, no longer) occurred randomly, after a few minutes to after 2 days. However, I'll put a lid on this in this week's video.
The log output (using the ESP32-compatible library Arduino-Log by Thijs Elenbaas) allows logging to anywhere you want, including SD card or SPIFFS. It came with the VS1053.h library I used but I can't find it on GitHub by itself.
By default it is "print"ed but that could go somewhere else. You can get lot of logging in 3Mb but so much more on SD card, which my (and many other) boards comes with, so a 2GB card would contain all logging since the beginning of time, if you wanted. Maybe I should do a quick video on this for everyone?
Regarding PlatformIO, I hear you. It is sometimes a mystery to me too, but I will consider doing an intro for it. But for that Web Radio project (if you just added the entire thing to your PlatformIO projects folder and edited the project list file so it appeared in your workspace, it should find everything it needs. I did nothing out of the ordinary because I would not know how (yet!)
Long answer, I hope it wasn't a case of TL;DR?
Ralph, have you watched Andreas Spiess' video on the ESP32 radio?
I watched a video where he _wanted_ to make a radio but I think he decided on the Edzel fully-featured, all-singing design? I don't know any more than that as I had already decided to design my own (watch next week's video for a more positive update - shh, don't tell anyone else).
Do I detect a small amount of grumpiness today, Mr Bacon?
You think? I thought I had hidden it so well 🤣
I've had an absolutely bizarre problem with ESP8266 WiFi when the module was rebooting itself every few seconds with *any* code loaded (my code, SDK examples, etc). One day I found all of my ESP8266 devices has stopped working. At first I thought it was a power problem. I've changed the batteries, added electrolytic and tantalum capacitors, hooked up to the lab PSU - no effect at all. I've spent a few weeks testing different software and hardware configurations until I realised that that the device is 'magically' starts working again in some areas of the house (like on the floor in the bathroom, etc) or if I put the device outdoor or put it in a cooking pan. Long story short: apparently there is a some kind of interfering WiFi signal there that makes ESP crazy. It resolved when I switched to another ESP SDK version (I've tried a few and found a working one).
It's hard-to-diagnose problems like this that drive us crazy! Interesting that a different version of the software fixed it though.
Sir can you provide the code with ring buffer for esp32 (revision 1) with no METADATA.
There is a ring buffer built into the ESP32 framework. To use it just do:
cbuf circbuffer(SIZE);
Keep the size below 10K unless you have PSRAM and know how to "force" it to be used (does this automatically in the PlatformIO environment.
Very interesting. I have had similar problems with an ESP32 which Wifi worked for months and then not for weeks until it started working again..... The local Wifi networks were not changed. I can only assume that interference from an external Wifi network caused the problem???
I wish I knew. My ESP32 Revision 1 chip worked fine for nearly two days then started playing up just the same. Perhaps it is my router / Internet connection...
I only have rev1 esp32's and haven't seen this problem, although I do sometimes see it complain it can't connect and then immediatel;y reporting it received an IP address.
I just ignore the error. I am very hopeful the WROVER boards will work for you.
Yes, I've seen that behaviour too, very odd. My WROVER module is running fine with about a 150K buffer (just under 10 seconds) and all is well... for now. I've leave it running for a couple of days and try a few reboots to see if it can handle reconnecting to the WiFi!
I only have experience with AI thinker esp32-s boards, but most of the time they are fine. However they also don't recover well from disconnects and the best solution is a reboot. So that makes you think the issue is in the ESP-IDF framework. What I do is send a heartbeat signal on MQTT and check on the esp32 if it has not received it in the last 5 minutes, if so it does a reboot. The esphome project also uses this strategy.
That sounds like you are work round the problem of a device not being reliable. Well, I guess it depends on how often you have to reboot the AI-Thinker board. But it doesn't exactly build confidence, does it?
@@RalphBacon No you are right, but I'm hoping it may get fixed in software down the line. It happens about once or twice a day for me.
I had to put a capacitor on the 3volt and ground pin to get wifi to connect when I was working with an esp32. That turned me off of the esp32 rather quickly. I went back to simpler arduino and esp01 chips. I am currently working on esp01s because most of my projects require only 3 pins at most.
If you still have that ESP32 do check its revision. But in my Web Radio project I do put a largish cap between 5V and GND because the screen and mp3 decoder are all connected to that line. I think it's sensible to do that.
This may be the fix. @sbense I notice that esp32 firmware version 1.9.0 has just been released on platformio and it sounds like it has a couple of important fixes. Might be worth it to update everything before doing any further testing.
It seems to have worked for mesh project.
As a result of your nudge, Glenn, I've just upgraded some core libraries to 2.0.1 (something like that) but I've not had a chance to test things out yet.
Yep, I’ve had problems with ESP32 (revision 1) and WiFi, but no problems with ESP8266.
My ESP32 rev 1 chip are stable (enough) but I do have some rev 3 chips on order. But, like you, Craig, my ESP8266 has never given me a problem!
Aah these good Rev0 chips. Remember ATMega256. I was fighting few weeks to strange bugs. booted 2-3 timer per week. Cause: one asm command was executed twice if stack pointer is external ram and there was incoming interrupt. Change to newer revision of chip and all was good
Edit:
Best. ATMega128 also had same bug in first rev. nice copy paste rtl :)
Whilst I recognise that problems can occur once a product has been released, we wouldn't leave a car with faulty brakes to be discovered by its owner so why don't the makers of chips put a big ERRATA or similar page up? Honesty is always the best policy!
Excellent video Ralph. Fortunately I haven't got around to ESP32 wifi projects because the ESP8266 works so well. :-) :-) Still, if it ain't broke don't fix it eh?
BTW, Time for you to buy yourself a Nextion display Ralph. VCC and ground an one Software Serial port to connect it. I'm already looking forward to your tutorial on the Nextion GUI developer!!!
Keep the faith buddy.
I have a Nextion display, never got round to using it. Just a single serial input? That might work with my ESP8266 version of this project then. Hmm. Food for thought!
We have about 2500 in the field with no WiFi issues at all. We have them using Soft-AP as well as hopping on to Ubiquiti, Grandstream and Mikrotik access points. They stay up for months on end. We do connect to specific BSSIDs though after scanning and finding the access point with the strongest signal.
You're the 2nd person to comment on use large numbers of these devices in the field, Neil, without issues. I wonder if you use the bare ESP32 or is it already mounted on a dev board? I am using Rev 0 of the board which was known (apparently) to be a bit flaky so I'm guessing yours are Rev 1 (or even higher)? Whatever, I'm very glad to hear my original faith in the ESP32 is not unfounded.
@@RalphBacon Connecting to specific BSSIDs determined by an explicit scan solved similar WiFi issues on rev 1 for us. See github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/387
Also, if you're about to use PSRAM for your ring buffer, be aware of a hardware bug in revisions 0-2. You need to use the recently released toolchain including the refined workaround for this (having a little but non-severe impact on PSRAM performance) and enable the workaround. You'll need to figure out how to apply that to the Arduino environment. For details, see: github.com/espressif/esp-idf/issues/2892
@@RalphBacon We're using the ESP32-Wroom and I would say they are all Rev 1, we started production in Jan 2019 although some of the boards I still use were hand flowed my me ( oh the joys of prototyping ) in late 2018. We do ensure that all boards are genuine Espressif marked units and on the boards I have checked, they test out as rev 1 according to Andreas's check. TBH I find the ESP32 incredibly stable. In installations where we have > 10 of them we are forced to use the MIkrotik Access points due to Espressif's refusal to allow configurable > 10 clients in Soft-AP mode, the devices happily communicate on the network whereas Android devices have issues. I think this is more to do with ARP mechanics though. We do find we have to be quite careful with the amount of packets placed on the queue though as you can't inspect the tx queue depth leading to dropped UDP packets.
@@bgdexter That's interesting that BSSID stabilises other connectivity troubles, we use BSSID due to the ESP32's lazy behaviour - it merely joins the first AP it finds advertising the specified SSID. In multi-AP setups can cause it to join a network on a channel with weak RSSI.
@@iPeel My findings on multi AP networks looked like a round-robin scheme on the first connect. I've described the bug here: lists.openvehicles.com/pipermail/ovmsdev/2020-July/014269.html
Ralph,
This episode's accompanying program contains quite a lot of information about significant parts of the streaming data and their meaning.
Will you share the source of that information with us, please?
Jan
You're referring to the streaming .mp3 data and the integrated track info?
Yes, took me two days to get my head round that but now I have cracked it and wondered why it took so long. I'll see what I can do to clarify this in a future video.
Listening to my ESP32 web radio right now, working like a dream.
Had massive problems with esp32-cam wifi. Very bad wifi connection. Pinching the antenna between 2 fingers would restore wifi. Never found the problem.
Work around was to install an external antenna and changing the antenna selection resistor.
Oh dear, I have a ESP32-CAM but haven't used it yet. I will bear this in mind.
Hi Ralph,
Just a thought from the top of my head: I know that the '32 can't run Bluetooth at the same time as WiFi. Maybe (just maybe) switching everything BT off can make WiFi more stable? Nope, no demo code right here, but that's googleable.
Talking about broken hardware: I recently had 10 (!) WeMos D1 / ESP8266 with a broken WiFi from one seller in china - these boards can't connect to anything. Since that, I'm running a small function test on every new hardware I buy.
Keep up the great videos!!!
It makes me wonder where the sellers in China get their hardware sometimes - the "reject bin", perhaps?
The ESP32 can effectively run both BLE and WiFi at the same time. You don’t get simultaneous use of the radio, but if you’re not maxing out bandwidth on each it works just fine. My devices stay on a WiFi connection for hours/days (battery portable device so usual case is about 11 hours of uptime). When powered via usb, I can maintain WiFi for days, with occasional BLE connections. I’m using esp-idf 4.1.
:-) do you have some times the problem, the conection will be lost after some minutes. ? Soeme strams are runing very stable, other fails after 10 - 30 minutes. DIsd you found an solution ?
For the ringbuffer, i have done this here ( 32kbytes ) , ist nested.
void loop_main()
{
int chunk_count = client.available();
if (chunk_count > 512 ) chunk_count = 512;
Serial.println("A 000 " + String(chunk_count) + " " + String(100 * client.connected()) + " " + String(rbuff_count / 100)); //+ " " + String(rbuff_count));
while ((chunk_count-- > 0) && ( rbuff_count < ringpufferSize ))
{ //rbuff_count gibt an, wieviel verfügbar ist
uint8_t bytesread = client.read(mp3buff, 1); // read from stram
*(ringpuffer + rbuff_write_index) = mp3buff[0]; // Put byte in ringbuffer
Because I am using the ESP32 core (which has a circular buffer implemented) I just hacked it to detect the presence of PSRAM and use the PSRAM rather than SRAM and it works as expected. 10 seconds of buffer uses about 150K of PSRAM which means there is lots left over for other purposes.
@@RalphBacon my buffer runs fine, but i lost some time the connect to the URL. The wifi runs fine, but i must reconect the URL ...
if (client.connect(host_.c_str(), port_) )
{
client.print(String("GET /") + path_.c_str() + " HTTP/1.0
" +
"Host: " + host_.c_str() + "
" +
"User-Agent: Mozilla/5.0
" +
"Connection: close
");
delay(600);
I think there are some incompatibilities with ESP’s and some Wi-Fi access points. I had some ESP-01 modules and they worked fine (rock solid) the newer ESP-01S module had awful issues with the Wi-Fi running exactly the same code. I changed routers and both work fine now. I have an ESP-32 in a garden light project which drops off my Wi-Fi (Both of the access points) and I’m about to replace it with dual ESP-01 units. Interestingly when my ESP-01S was going wrong, it was giving me Auth issues.
Is it the APs that are fault or the ESP's WiFi code, Andrew? Whether that is firmware or software? If you do swap out your ESPs, upload a blink sketch or something equally small and see that revision the compiler gives back to you at the point of uploading. Probably Revision 0. It is is Revision 1 then we're in real trouble.
@@RalphBacon I've uploaded a sketch but can't see any revision number returned
Executable segment sizes:
IROM : 273496 - code in flash (default or ICACHE_FLASH_ATTR)
IRAM : 28648 / 32768 - code in IRAM (ICACHE_RAM_ATTR, ISRs...)
DATA : 1280 ) - initialized variables (global, static) in RAM/HEAP
RODATA : 3328 ) / 81920 - constants (global, static) in RAM/HEAP
BSS : 25648 ) - zeroed variables (global, static) in RAM/HEAP
Sketch uses 306752 bytes (32%) of program storage space. Maximum is 958448 bytes.
Global variables use 30256 bytes (36%) of dynamic memory, leaving 51664 bytes for local variables. Maximum is 81920 bytes.
esptool.py v2.8
Serial port COM3
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: bc:dd:c2:7b:06:03
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 1MB
Erasing flash (this may take a while)...
Chip erase completed successfully in 3.4s
Compressed 310912 bytes to 225819...
Writing at 0x00000000... (7 %)
..
Writing at 0x00034000... (100 %)
Wrote 310912 bytes (225819 compressed) at 0x00000000 in 20.0 seconds (effective 124.1 kbit/s)...
Hash of data verified.
Leaving...
Soft resetting...
It's shown right at the start of the upload process - but save any frustration as I'll be demoing exactly this in my video on Friday.
Edit: Hang on, you're compiling for an ESP8266, this works for an ESP32!
Are you doing analog reads? Are you using IDF or arduino ?
No analog reads, Arduino-ESP32 core. But the problem is no more, watch Friday's video!
@@RalphBacon Sweet
Easier to do bufferIndex%BuffSize in conjunction with a counter (To check if buffer is empty or full) this counter increments on a write and decrements on a read.
No doubt you're right (I won't check) but the ESP32 library does have a function to check whether the buffer is full (or how full) so I'm using that now.
Has there been any progress on this problem?
Yes, all well and good now. Just ensure your ESP32 is NOT a revision 0 (zero) (it shows this at boot up) and all is well.
as a network technician this sounds like a MAC address conflict to me... is the ESP32 a clone or has expressive cheaped out and put the same MAC on all it's devices thinking why the heck it's just for development anyway and not used in productions
Each ESP32 has 4 unique MAC addresses which use a common 46 bit wide prefix.
The lowest 2 bits are used to specify the individual MAC units (%00) WiFi station, (%01) WiFi soft AP, (%10) BT, (%11) Ethernet
Many companies use the ESP32 in production use (not the dev board, just the soldered on raw device) but many have said they have had to abandon that idea due to the WiFi problems I outlined. I don't know whether they returned to the fold, so to speak, when Rev 1/2/3 came on the scene. As we know it's hard to acquire customers but very easy to lose them.
are you using ESP-ADF? I also know Bitluni I think had a look at ESP-32 and audio, he did something to use some parts of the IC to overcome something, it's all kind of hazy but I think ESP-ADF and some of Bitluni's past content with the ESP-32 might help you with this project...
I just don't think it's Espressif's fault, you seem to be blaming them for their poor design but I'm not sure it's their design that's the problem...
Apparently it _is_ Espressif's "fault" inasmuch that Rev 0 had such dire issues that they had to release Rev 1 just a few months later. That sounds like insufficient unit tests on the Rev 0 to me! But I'll check out Bitluni's content and see what he did, thanks for the heads up!
And welcome back Dejan to Ralph's videos... :)
Where have you been, Dejan? Glad to have you here again!
@@RalphBacon I just pulled my nose from the schematics, pcbs, etc... :)
You really would not need a boolean to use every element in the buffer. You can just set tail to be out of range e.g. tail = -1; to show that it is empty.
ps. I always used 1 pointer and a length as I find that more understandable but that is probably just me. I think the machine code took less clock cycles for filling the buffer to but it was a long time ago so could be wrong.
You could, of course, use a pointer (to the start of the array) and add on the length used so far, but I think many of my viewers would run screaming from such a notion! I'm not sure how Espressif do it in their code for the ESP32 (cbuf) but it works just fine.
Hi Ralph, Great explanation of a "ring buffer"..as always and thanks ! regarding you project I would ask you to have a look at 'Tiny Pico' I'm no expert but it's an up to date ESP32 DEV board that I have watched being developed from day one, it has a fantastic community and the guy who designed it (his name is Seon ) is very passionate about this subject and extremely Knowledgeable when it comes to firmware, he streams on Tuesdays for us (he's in OZ wednesday morning for him) would love to see you there!....anyhow it's good your radio is moving in the right direction and I always look forward to your next installment...cheers!...oh yeah" Unexpected Maker" check out his store, I think you may be interested.
Thanks for the heads-up UA-camrs, I shall deffo take a look. Glad you now are an expert on ring buffers! Simple when you know, isn't it?
@@RalphBacon Breaking it down helps enormously, cos in real time its massive amounts of data ...cheers.
Anybody tried if the new 1.0.6 release (26th march 2021 released) solves this wifi problem ? Im now testing for 1 day on 2 ESP32 devices; so far so good....
Yes, anything other than the original release seems quite stable; the newest version is supposed to be the best yet but I don't have a sample to test. All in good time, I'm sure.
Not sure if this is your problem and you probably already know this, but your not supposed to run lots of tasks on the same core that the wifi runs on. Forget which core does the wifi but it is fixed. I dont have any problems with my esp32 wifi. Stays connected for months. Also rember that the esp32/freertos has concurrency features such as mutexes that you will probably have to use in your buffer implementation.
The WiFi runs on core 0 (along with BT) Mike so I'm safe running anything on Core 1 (which is where the loop() runs). I'm interested that your ESP32 stays connected for months at a time. I don't suppose you know the chips revision off the top of your head?
@@RalphBacon Not in a position to problematically check the fuses. I don't know how to tell visually. If you know how to tell visually, it's easy for me to look. It's part of a PID controlled temperature fan controller that is running inside a metal enclosure so it's got the version of the chip with the U.FL connector on it for an external antenna. Not sure if the external connector would make any difference. I suspect it's a current rev since the dev board was purchased in 2020 and was Made in the USA (www.ezsbc.com/product/wifi01u/). Hope that helps. So far I really like the ESP32. Want to try the LittleFS that you mention to serve HTML over WIFI. My fan controller project simply returns temp, fan speed, etc via JSON at the moment. But I want to write a front end in ReactJS or Bootstrap or something. LittleFS would help serve that up.
As it is so recent (and bought in the USA) is undoubtedly a more up-to-date revision than mine, for sure. The standard ESP32 has 4MB of flash memory of which we probably use 1MB for the sketch so you would have 3MB for SPIFF-related items. Or get a 16MB version and the world is your oyster.
Not sure if this matters, but the ESP32 will be connecting with 2.4Ghz WIFI, whereas your phone will almost certainly be using 5Ghz wifi on most (semi-recent) access points. This can lead to a lot of confusion, because they are actually completely different wifi networks, but sometimes are given the same name. In practice this occasionally means that the 2.4 Ghz wifi is totally misconfigured and you might not even know. It sounds like this may not be the issue in this case, but it is often a problem I see with WIFI.
the esp8266 is also 2.4Ghz and works fine.
I have the same issue with an ESP8266 losing connection. My phone does not drop connection, neither do any of my tablets or my laptop and they are all 2.4Ghz WiFi devices, I do not own a 5Ghz device so it can not be that.
Perhaps I have been really lucky with my ESP8266 Wemos D1 mini devices as I've never, ever had an issue with WiFi. And I tested them to (almost) destruction as they run in my mum's apartment several hundred miles away so no possible way of fixing them if they go wrong!
Does your WiFi router have logs? Maybe the problem is on that end.
My WiFi router is from the arc, Chris, so it barely runs on electricity (about time I upgraded, I know) and there are no logs. But it turns out the silicon revision of the ESP32 has solved (almost) all the issues together with an upgrade on the PSU. For more details watch this week's video!
Mr Bacon, I really enjoy your video on the various Arduino project you have made over the years. This one video caught my eye, using a ring buffer or ring memory on the ESP8266. I am using an ESP8266 to monitor the moisture of my plants using the SPIFFS, I am taking readings hourly, I have set aside about 1 Meg for the SPIFFS, granted I know that is a lot of data to be stored over a period of time, but I would like to implement a Ring buffer to keep the data circulating for display on a Web Page it shows the moisture value over time in a graph. The intent here is to keep storing the values without having to worry about it failing due to lack of storage. I was looking at your radio example and I was hoping you had an example that would show how to store time and soil moisture so that when I bring up the Web Page I can display the contents as if I did not have a ring buffer if that makes sense.
If you are actually writing to a file in SPIFFS (you should really use the in-built LITTLEFS now, more resilient) then a ring buffer will be somewhat difficult to implement.
When in SRAM memory, all that happens in a ring buffer is that the underlying code keeps track of where the start (head) and end (tail) of the buffer is. When the head reaches the maximum value (that you set) it starts again at the beginning (chasing the tail, but never reaching it, of course).
In a filestore situation, you might be able to emulate this (I'm sure you are not the first person to ask, but I haven't tried it so can't comment).
I do implement a web-page ring buffer in my most recent project (not [yet] published that keeps the last 20 messages; it does that by using a _vector_ (a type of array but easier to program). I just delete the first (oldest) message in the vector array before writing a new message to the end of the vector array. That way I only ever require space for 20 messages. I could increase the number of messages depending on my SRAM availability of course. Just be aware (or beware) that the messages disappear in the event of a power out or restart unless you save them to filestore!
I really should explain this in a video... one day! If you want more info, please do email me.
At the risk of sounding creepy, you look good in that shirt.
If you say so, and my dog Dougle thinks I look good in anything as long as I have a treat ready.
@@RalphBacon LMAO, Smart dog
Beautifully explained as always, Ralph, but I don't get how the circular buffer can help with connection drops or the times when the head catches up with the tail. In those circumstances the data is lost so when the tail is read in order to be played, there's a hole in the steam. Or have I missed the point and the circular buffer isn't meant to help in those times? Or do the received packets have overlapping data, or are they signed and send several times so missing ones can be reconstructed or slotted into the buffer? So many questions!
It depends what causes the buffer to run down, Andy.
1. If I switch off my WiFi then it uses all the buffer up and waits until it can connect again. We've missed all the intervening audio, of course.
2. If there is a glitch in the transport layer of the information it can "recover" to a point and we're none the wiser (apart from a possible period of silence).
3. If the radio stations server is overloaded (or has some other problem) we may lose the data or catch up. We may not notice this either.
What I have noticed is that some radio stations fill my 10-second buffer, whilst others hover around 38% (I don't start playing audio until the buffer is 1/3rd full to give us some wriggle room). This I don't fully understand. Perhaps something to do with the bit rate (ie how much data is coming down the wire.
TCP/IP which is what we use in the Internet, does ensure that the order of the packets arriving are in the correct order, so even if some took a circuitous route (eg an Internet router failed and they got redirected via Hong Kong) we still get the audio in the correct order. If there is a missing packet it will be re-requested, although whether this applies to streaming audio I don't know!
I'm going to expect you are not actually using the Espressif IDF for your code. So I'm honestly a little confused as to why you think espressif is to blame? You know the chip has no Wifi drivers or Wifi code on it and you upload ALL of that code from your IDE based on whatever versions and configuration there of it chooses for you.
The log file to me looks like a concurrency issue, probably corrupted memory causing the Wifi Task to get corrupted. Especially when you start to see duplicate errors.
Also... that log is very expensive. Is it being DMA'd, Interrupted out or are you blocking a core while writing it out? If the later and that core happens to also be the Wifi core, it's going to cause all manor of timing trouble for the rtos (if in use) and the timing and interrupts of the Wifi layer.
I have found the ESP32 is great at doing one, maybe two things at a time and for two you need to know what you are doing and do it carefully or... you will find the Wifi and/or BT layers coming apart and faulting.
Also.... why are you not logging out the ACTUAL debug Uart on GPIO3? That's the proper logging which is hidden in Arduino space too often. It will have the low level UART logging from the actal Wifi driver.
As an example... An ESP32 as a A2DP sink, streamed out I2S to an audio-codec = fine. Introduce an FFT calculation on the user core and boom, core dump, "Forbidden Access", "Corrrupt back trace" hard fault. Most likely a process writing past some memory map boundary and into some other part of SRAM where the core trips an ACCESS fault to that memory segment.
If you want to have more sanity and less issues, just buy the official Espressif modules and use a programming harness. Then you get ALL GPIOs, full access to the chip and no devboard stupidity to get in the way.
On one more aspect. That's a pretty nasty birds nest of some non-trival speed digital connections. That TFT is running on SPI at probably 40Mbit/s and it's wires are criss crossing directly over the Wifi antenna. I'm surprised it works at all!
TL;DR? It was a hardware issue, now resolved!
Taking your last point first, Paul, if I didn't breathe too hard next to the bird's nest of DuPont wires it all worked very well - once the Wi-Fi issues were resolved.
The Wi-Fi issue was traced back to the ESP32 revision I was using. It was a Revision 0 which apparently had "known" Wi-Fi connectivity issues (well, not _that_ well known unless you dug deep) so they released Revision 1. Rev 1 resolved the Wi-Fi issues but introduced other issues, notably with the PSRAM which they resolved with a software patch. Rev 2 swiftly followed but was not released to the general public (they obviously found something dire) and Rev 3 was the last revision issued, but these are, for some reason, hard to get hold of.
So most people, me included, are stuck with Revision 1 samples and they work "well enough".
Incidentally your comments about Wi-Fi and BT being blocked on a core would indicate that you use Core 0 for your tasks; this is not something to be recommended for the very reasons you mention. Developers should stick to pinning their tasks to Core 1 where RTOS does the magic.
BTW the Arduino framework from Espressif works very well, although ultimately not as flexible as the Espressif IDF - but then again, we Arduinites probably don't need that level of granularity anyway.
Incidentally, this investigative project (from more than 2 years ago) was the precursor to my ESP32 Web Radio which was designed for a PCB and several of my viewers have built it to their satisfaction and delight - my one is running faultlessly here in the workshop almost 24x7!
@@RalphBacon Thanks. My message was a little off character, I didnt have my coffee yet!
I did indeed discover my code was running in the event handler of the BT task on core 0. Nothing in the loop on core 1.
I dont really trust the ESP32s as much as others and it's mostly due to the sheer bulk of the frameworks and configurations. The BT core alone takes up half the flash and includes everything under the sun even if you just want to play a BTClassic A2DP stream.
In other frameworks like STM HAL they are a lot more selective of what gets added to the project and what doesn't and what gets compiled and linked and what doesn't. Less code, less problems, less bugs, less memory.
The IDF and Arduino Framework wrapper of seems to just wrap everything into static archives and then link them into monster binaries full of 90% unused code. /rant :)
Even a simple "Blink" program uses hundreds of kilobytes on an ESP32, for sure, as it loads in all that framework. But it doesn't get exponentially bigger, the more code you write; it's a one-off overhead.
I have to admit I'm happy with the ESP code framework. They normally do a good job of porting their native IDF C code to the Arduino C++ format and do revisions on a regular basis too.
A lot of my projects use an ESP32 thanks to the inbuilt Wi-Fi and/or BT. The modules are cheap enough, and as long as the ESP32 module is revision 1 or greater, no Wi-Fi issues! 😉
@@RalphBacon A project of mine is to try and open up the STM32 to be more useful for home automation by creating a board and library, with an ESP32 as a dumb Wifi module. Actually an MQTT client.
You just call a library function like "connectWifi(); subscribe("my/topic");"
Then just loop calling "getReadyBuffer" and low and behold MQTT messages arrive in the buffer!
Will wait till I get the first prototype boards manufactured and the code on public github and I might ping you for a review?
On the ring buffer. It's fine it will work. It offers no protection for multi-threading though.
When I had a writer on core 1 and my reader on core 0 it didn't work at all.
The writer asked for a buffer, which incremented head. Before it even got to start writing into it however the other core had started reading, as it was waiting on head not being equal to tail.
Making buffer use synchronous with both "request buffer" and "release buffer" calls can help that and let clients hold exclusive access to buffers. That an a few mutexes or semiphores to make control variable "test-set" operations atomic.
No time to watch yet but files winging their way to you now. All the best.
Mudgel's the very generous chap who generates my complete list of videos in PDF format that I put a link to in each of my video descriptions and GitHub.
i think your neighbour's spamming deauths and alloc your MAC after 😉
Cheap broadband for my neighbour?
@@RalphBacon perhaps? i would airdump a while to see whats going on. could be interesting regarding the esp's beaviour thou.
it isnt the 192.168.4.1 problem .. i had that , were my internet provider also had a server on that ip .. check default "AP" ip of that device , and the need to clear that data out of the device and make it a client only .
I don't store the details in the device's NVR (all part of my testing to see what I could do to fix this). But watch the next video to see it working!
You could also use a shiftregister for the screen
You're not talking about the circular buffer? What were you referring to?
@@RalphBacon
Hi Ralph - I think he's suggesting a way around your worry (@20:40) that you're going to run out of pins trying to connect the display to your ESP8266 module. A common(-ish) suggestion is to use a shift-register IC to extend a few pins to many. There's lots of information out there, but here's one of the clearest tutorials I've come across: ua-cam.com/video/Ys2fu4NINrA/v-deo.html
And in this one, Karen gives more detail on how they work - the internals: ua-cam.com/video/Ys2fu4NINrA/v-deo.html&t=452s
Of course, you suggested another solution in one of your own videos from several years ago (#49 in 2016) about an I2C pin extender: ua-cam.com/video/kQZU8kE3ksU/v-deo.html :-)
Cheers!
Keep the faith old boy, put it away for a whole week then go back at it fresh.
Even if he puts it away for a year, it will still be known broken rev 0 silicon from 2016 that is known to not work due to design errors. Rev 1 fixed these errors in February 2017.
@Mark Grecro Even if I can't "fix" the Rev 0 chip it will still be a very fast Arduino Nano! And maybe the BT works more reliably. All is not lost!
Hi Ralph, just wondered if you've tried the ESP32-S2 modules? I have one from Olimex, mainly because it boasts low sleep current consumption, but it maybe Expressif have resolved it with this new board revision.
The S2 variant, Mike, is a cheaper, single core device (so you will have to use yield() in loops and the like to avoid a panic) but apart from that it should operate as a standard ESP32 (Rev 1 or higher) and hopefully won't have this issue.
The ESP32 is not an ESP8266 .... BTW there is a DNS issue in the lib of the ESP32 - it's easy to work around. Found it reasonably reliable... runs about 68 days between reboots
I don't know what you mean by "the ESP32 is not an ESP8266", Dougal. Apart from the obvious do you mean in reliability or capability? I only mentioned the two chips to compare reliability with WiFi, they are from the same design factory after all!
It would be helpful if you would post the DNS issue workaround here (or a link) for everyone to see, remember "sharing is caring".
@@RalphBacon If you look in the source of the UDP lib you will find it makes a different DNS name resolution call and if your DNS is not reachable then the CPU reboots or hangs. To avoid this my approach was to make a name resolution call before you send UDP packets and always send the packets by IP address not by name.
My comment about them being the same was made as people seem to expect they are 100% compatible. I would say they are more like siblings with there own personality and charms. The ESP32 just needs a bit more TLC in the wifi area but you can achieve the same results in the end. I think people may be disappointed as so much is interchangeable but the last little bit is different.
I actually wrote a project about swapping from ESP8266 to ESP32
www.hackster.io/dougal
Source at
github.com/Dougal121
I noticed that Espressif have got a new Arduino-ESP32 library (1.0.5-RC2) waiting in the wings, perhaps that will fix the problem you outline.
As it happens I'm just using TCP/IP not UDP for my project so I think it should be OK (especially as my ISP uses Google for DNS resolution).
As a general point, I wouldn't want to use IP addresses as they can change at any time - hence DNS!
I saw your website, very impressive indeed! You're obviously an experienced maker/developer. Always glad to have your input.
@@RalphBacon Um you sort of missed the point. Yes I also always use DNS not IP, just use the call that works not the one that destabilizes your WiFi.... Get the IP address first from DNS using the right call then use this in the TCP or UDP packets.
Check if yor access point has something called WMM. Disabling it might help.
I checked, but as my AP is quite old I don't think it understands anything about QoS or things like that. Possibly time I upgraded...
@@RalphBacon You can get bt home hub v5 type A and flash it with openwrt. They are on ebay for around £10-£20. I will be buying a few more an try to setup multiple access points around the house with 802.11r. Let me know if You need any help as they are quite fiddly to flash.
I use at least 50 esp32 for all kind of projects, never seenthese issues
I think it will have to do something with your Arduino setup
I am using Wroom 32
I'm happy to hear that you have so many all working fine, Patrick! That, at least gives me confidence going forward. I'm pretty sure my Arduino setup (PlatformIO, actually) is all correct but I am using a Rev 0 of the ESP32 which might be the problem... or power, or the dev board... or? Stay tuned for any updates!
@@RalphBacon what you could od is a quick setup of ESPHOME and try to see if these connect properly
thats a good baseline test
I do suspect all the jumper wires acting as little high noise antennas is making thing harder on the chips
My esp32 cam board only connects d transmits if I touch it on the right spot on the pcb areal
Me again, couldn't one use the esp8266 to to the wifi bit, and connect that via say bluetooth to an ESP32 to do the heavy lifting, processing, and user interface... (for those cases where you want to he power of the esp32)
He's using known broken silicon which was replaced 4 years ago by Espressif, because it was known to be broken. Using known good silicon might be a better start.
@@Peter_S_ Thanks, I'm still pretty new. But I'm amazed how easy some parts of programming became... things I for years couldn't get my head around, I now throw around with abandon using graphical interfaces like tuniot and the MIT app maker.. Some days I feel like a real programmer...
I do have an antenna on the way for my cam32 board. hope it fixes the situation... me holding the board isn't very stealthy.
Crap!.. I've just recently got 6 ESP32 first revision (30-pin) boards with intention to make some 24/7 online data-logging and monitoring stations for some equipment in the lab... 🤬
rev 1 should work fine. I ahve several of them working with no issues.
@@peut Um... Which one has the bug then?.. Also, I can't be sure I didn't get some random Chinese clone... It looks exactly like DevKitV1, and has the same pinout, at least...
@@eliotcougar If you uoload a sketch to it the esptool tool tells you which revision you have. There are no known clones of the ESP processors, only some better or worse clones of devboards ( so differences in voltage regulators and USB to UART chips (CP2102/4 etc) . I have seen no real problems with the revision 1 chips I have. I wouldn't worry too much.
From the comments here, Ilya, anything higher than Rev 0 should behave. But it's worth creating a simple sketch and letting it run overnight or longer, just to make sure it stays connected for hours and then prove it can reconnect many times with the dreaded AUTH_FAIL error (or some DNS failure).
Do you suppose there could be a Plonker in the neighbourhood amusing itself issuing Wi-Fi deauthentication attacks.
It seems unlikely, Nick but I'm willing to believe that aliens are responsible too! As it happens my WROVER unit (with 10-second ring buffer) has been behaving nicely since yesterday. Fingers crossed.
I'm curious about a comment from someone who lauchnech couple hundread devices on the market. That is more accurate answear wheter ESP32 is a fail, than comment that someone have one device and it works fine.
That comment was what made me think that Rev 1 of this device was good (enough) for production use - and that has been borne out.
Hi Ralph. Did you see G6EJD's latest video on WiFi output? Not sure if helpful
Not yet, I've been busy with my ESP32 (Rev 1) WROVER unit as it's behaving itself (so far). But I will add it to the list. Thanks for heads-up.
Watched it. Not specific to my problem but very useful nonetheless as my ESP32 is just 1m away from the AP. I've added the following:
WiFi.setTxPower(WIFI_POWER_MINUS_1dBm);
As I should not really need hardly any power to connect. But I'll modify if there are issues! Thanks for the link.
@@RalphBacon well, I wasn't sure if it would help in any way, but seemed relevant to the current topic.
Thanks for the heads up Ralph, once again great video. I've found my esp8266 based weather station which uploads readings to ThingSpeak to be robust and was planning to move it to an ESP32, so your comments are noted. I'd read somewhere that if you run on CPU1 (which I assume is the default) then you can avoid the need to yield as the wifi stack runs on CPU0 and theoretically makes the esp32 far more stable than the esp8266, but I guess not. Someone also mentioned that the wifiscan.ino sample shows much worse scan results on their ESP32 than their ESP8266, so worth looking that that too (antenna issue as already mentioned by someone today). Your errors maybe point more at their wifi hardware encryption (faulty firmware there?) rather that a signal loss issue. I imagine that if it turns out to be a Rev 0 v Rev 1 issue there won't be way of doing a low level firmware update and they just expect people to bin the device and buy a new one.
The big problem with rev 0 was not the firmware, it was the bus arbiter corrupting data. This has been known since late 2016 and its covered in EspressIF errata. They corrected most of the early bugs by February 2017 when rev 1 was released. This video has likely done more harm than good because everyone in the development community at the time knew the rev 0 silicon was broken to the point you could not design reliable product around them. That situation was obviously remedied years ago.
@Juan Herrero I guess any of the reasons you mention could be the reason but I'm retesting with a Rev 1 device and a WROVER (also Rev 1) so we shall see. My ESP8266 (also connecting to ThingSpeak) runs great but there are not so many pins.
Esp32 Is All broken. It's far less mature than esp,8266
Well, that's rather harsh, Davide. Although I was quite (very) cross with Espressif when I had lots of problems with their Rev 0 chip, since then it's been working very well. It's a powerful chip and now that the ESP32-S3 has been launched (still dual core, but 10 more pins) it can only get better.
expert opinion here lol, as a matter of fact millions of consumers products in the field are based on esp32
@@lo2740 most ESP32 users does not even know why they use ESP32 over ESP8266. for most projects ESP32 is simply overkill and it brings no benefits. Libraries are less mature so cons are often more than pros.
hmmm seems you got yourself a challenge or two. Good exercises for the gray cells.
The noise is probably your cabling, hose jumpers never had the best connection and the breadboard don't make it any better. I am sure it would get better when you get around to making a PCB with the probber shielding.
There is already a lot of people on here having problems with WiFi on ESP32 and no one with an answer. I don't have an answer. I don't have a clue if it hardware or software but......😁
Could the problem be 3rd. party drivers?
On WiFi there is a lot of handshakes going back and forth, are they being handle correctly?
You might have to build a new driver for WiFi.
Just wondering, if it is a hardware problem, why haven't the manufacture answered on those issues?
I hope you find answers soon to cheer you up. Good luck.
I'm saddened by this. He's using rev 0 silicon which has been KNOWN TO BE BROKEN for over 4 years and was quickly superseded by rev 1 silicon.
Those cables (and the breadboard) are rubbish for this type of project. The connections are flaky and introduce all sorts of noise, buzzing and artifacts. The ESP8266 was better in this respect as I didn't connect the screen so fewer cables, had firmer connections and a better all round experience. I'll progress the problems going forward.
The 8266 has a software MAC while in ESP32 MAC is hardware implemented.
I wonder if software MAC can be enforced on ESP32, because that can be a solution to the problem.
You can set the MAC address of an ESP32 , ( just like you can on theESP8266) if that is what you mean.
I don'tthink itwould solve the issue.
Using current rev 3 silicon that's known to work instead of 4 year old rev 0 silicon which is known to be quite buggy and was fixed years ago would be a good start.
@@Peter_S_ how do I know which units to buy? I have Wroom-32 on my lolin board with battery input and also have a few more units on the way as I had something in mind that needs a swarm behaviour.
@@poepflater The good news is that Espressif released revision 1 silicon in February 2017 so anything made today with fresh parts should be fine when it comes to the issues presented here. I already replied in your other post, but check out the Tiny Pico as one ESP32 family option. www.adafruit.com/product/4335 Its designer's channel is ua-cam.com/channels/u94OHbBYVUXLYPh4NBu10w.html and he live streams constantly.
@Vasily I'm not sure the MAC entry would make much difference but it's something to think about. Funny how Espressif can design an ESP8266 that is rock solid and then get it so wrong with a Rev 0 ESP32.
Interesting video. You seem to be as much a youtuber as developer, so you really need to get some control over your comments section. You can ban and remove comments if you wish, and this Peter S guy is a regular pain in the butt. I'm sure from your demeanour that you don't like to ban commenters just because they imply criticism of you, as you seem a pretty humble man really, but trolls are trolls and need muting. Maybe it's Rev 0 vs Rev 1, maybe it isn't, it's a fair question but not fair to destroy a reasonable comments section. Boot him out.
My finger was poised to do just that, believe me. As you say, though, I don't want to censor comments unless they promote "adult" sites or are abusive (all caught by UA-cam). But in this case, he sounded so determined I was convinced he was employed by Espressif; I mean why does he care so much? Move on, is my advice to him. Anyway, everyone gets one gentle warning from me then snip!