One aspect of RTOS development you somewhat glossed over is the amount of memory to assign for each task's stack. FreeRTOS has features to determine stack usage / detect stack overflow, so that the stack size for each task can be set appropriately. For development it's useful to use a processor with plenty of RAM so that task stacks can be set much larger than needed and then reduced so the application can run on a part with less RAM. Another important FreeRTOS specific detail is enabling the RTOS features you want in the FreeRTOS config header file. Finally, for those using STM32 processors, avoid using both STMCube HAL and FreeRTOS together. They do not play nice!
Thank you for your additional info. Indeed, I did not know how much stack is enough. Fortunately, the ESP32 has enough RAM for my application. So I did not need to answer the question...
@@AndreasSpiess I found this on the ESP-IDF docs that might produce an answer: - At runtime, call the function uxTaskGetStackHighWaterMark() with the handle of any task where you think there is unused stack memory. This function returns the minimum lifetime free stack memory in bytes. The easiest time to call this is from the task itself: call uxTaskGetStackHighWaterMark(NULL) to get the current task’s high water mark after the time that the task has achieved its peak stack usage (i.e. if there is a main loop, execute the main loop a number of times with all possible states and then call uxTaskGetStackHighWaterMark()). Often, it’s possible to subtract almost the entire value returned here from the total stack size of a task, but allow some safety margin to account for unexpected small increases in stack usage at runtime. - Call uxTaskGetSystemState() at runtime to get a summary of all tasks in the system. This includes their individual stack “high watermark” values. It's in the RAM Usage section: docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/performance/ram-usage.html Thanks for your wonderfully informative and well made videos!
I can't even begin to tell you how much I value the work you (and other educational youtubers who are truly educated and articulate) do for the comunity and myself..... Tryin to jump in with arduino and programming to make my dream projects come true would have overwhelmed me and I would have definitely given up without the resources you provide. Sometimes when your teaching yourself things the hardest part is not knowing what you don't know, and not knowing how to ask the right questions, so when you go out of your way to provide context and redundant examples along the way without delight too deep into those other topics it really elevates the content from infotainment to genuinely valuable educational material.
Thanks for referencing my vid Andreas! I’m self taught so I didn’t know there are more elegant ways to do multitasking. I haven’t watched all of this video yet but I will definitely come back to it. I’m getting back into coding to make a synthesizer so this is just what I need.
You are welcome! You are right, this one was not easy. Also because I had to concentrate very much on the important things. The script took many iterations...
A great intro to RTOS. I find it good practice to use it all of the time with the ESP32 and just have the OTA in the "loop" function. It's a good way to learn how to program many micro controllers in the big industry and not just hobbyist style coding. As I've also been use RTOS on many other microcontrollers like ARM and PIC for many years in industry. This makes porting code from other microcontrollers to the ESP range very easy. ...Keep up the great work. - Enjoying watching from the UK. 😉👌
Andreas, excellent instructional video. Your explanation of time slicing, task creation by breaking down the requirements of the program, communication queues was well explained. The analysis aspect of programming is a crucial skill that is required when incorporating the additional control of a multitasking environment into a program solution. Your big picture overview to small detail approach, embedded humour (pun) ensure your technical videos are a great learning tool and a joy to watch. Well done!
Excellent tutorial Andreas! I wanted to start off with RTOS and I remembered that you had a video on this. So I went straight into it as I know you would do the best job is getting me started. Thank you so much. Now I can go through the rest of documentation and other tutorials :)
I almost never critique UA-cam videos simply because they are what they are, a freely given effort to inform people of something. But your videos are exceptionally good due to your intellect, consummate skills, and your meticulous presentation. So because of that, and at the risk of being perceived as overly critical I will point out that you said 'seconds' instead of 'milliseconds' at 23:07 into the video. I do so because I think that to you, it matters. You actually want to be clear, concise and correct about everything you tell us, and you work at it throughout your videos. That is the reason why I watch ALL of your videos, even on subjects in which I have little interest, because they all exhibit what few other UA-cam videos do - genuine quality. I appreciate the quality, and I can guess how much mental and physical effort is involved in creating and maintaining it, video after video. Thank you Andreas.
"If you are good, you get more work!" Not going to lie, that line gave me flash backs to my Air Force days. haha As always, love the videos and I always learn a lot from you!
20:25 I think the mutex is used when you can have concurrent access and not just multiple writes, assume one task is reading a struct, and in the middle of reading, the other task modifies the struct.
If the processor can read/write 32 bit variables atomically you don't need to protect them (if you have only one writer). If your processor cannot access the data in one instruction, you indeed need protect read and write.
yes, when the display task needs to print the variable onto the screen, the writing task should not change it until the display task finished printing.
Exactly, you may have single-writer-multi-reader-configs when the shared resource can be written atomically, like 32-bit words on a 32-bit processor, but most other things require some protection even when only one thread is writing. Atomic operations (Compare and Swap etc) are also a useful thing to look into. They can help avoid mutexes and therefore overhead when building queues or counters.
Thank you for the discussion. I would like to add another aspect: Because the ESP32 is so fast, none of my tasks needed more than a fraction of 1ms and I was always able to actively pass the control back to the scheduler. Is a mutex still needed?
@@AndreasSpiess and thanks again to you dear Andreas for your great contents. Hadnt thought about it this way, your assuming since there is no preemption in your case, the mutex is not needed, which I think is right. although I would always add them as good practice since you don't know what future holds...
Never knew the ESP32 had a full FreeRTOS running in the background always. It is well hidden, not showing when we write simple serial Arduino programs. And not only it manages the WiFi on one core, but it gives us a really easy way to write parallel code with tasks and queues and locks. Brilliant.
Another great video Andreas.! I liked your history lesson at the beginning. In the late '80s I worked with Forth which was perfectly capable of multitasking with 8bit microprocessors like the 1802, Z80, 6809 & 8051. It used a cooperative round robin model which meant the program designer could allocate as much cpu time as was needed for each part of a task. With no interrupt and time slicing the task switching overhead was tiny. One benefit was the ability to test and alter code from the keyboard with existing tasks running in the background.
You are right. We currently re-discover things we had in the 80s like inline debugging. But of course, today it is much easier because of the resources an ESP32 offers. It has nearly no limitations for an average project. Which makes things much easier. No optimization needed, just straight programming...
Theres a lot of really informative material in this video THANKS ! I didnt realize it was so straightforward to run freeRTOS on an ESP32. My first freeRTOS app was running within 15mins of completing this video. Really encouraged now to delve deeper.
Excellent comprehensive example with code examination. I've used FreeRTOS on 10 or so projects. What you've covered here is probably 90% of FreeRTOS projects. If users can get through your example understanding these concepts, their spaghetti code will become readable, understandable and maintainable. One last thing, do NOT use arduino delay. It affects the scheduler and your code will stop working. Search your code and change delay to vTaskDelay. Thanks from Colorado.
Thanks for sharing your experience! One question: What does the Arduino delay() harm? I thought, it just takes up all the time in the task's slot (which is, of course, not ideal).
@@AndreasSpiess I looked at the ESP32 FreeRTOS delay code. It implements Arduino delay with vTaskDelay, so on the ESP32 using delay should work fine. I believe my comment above deals the smaller Atmel parts, 328P, with limited timers. Seems like I remember using Arduino delay changes TIMER2 (may be incorrect here), that's the same timer used by FreeRTOS's scheduler. It may be different now, but back then Arduino's delay affected the scheduler. If this is incorrect or has changed, please comment below. I've switched to using STM32's for my designs which runs FreeRTOS just fine. Thanks again for the video, I'm always interested in what your doing (keeps me from having to figure things out), you do a wonderful and complete exploration.
Ein Video an meinem Geburtstag über eines meiner aktuellen Lieblings-Bauelemente. Das ist ein extrem interessantes Thema, -werde mich da demnächst mehr mit beschäftigen. Danke für's zeigen!
At first! Thank you for your great explanations on all the different topics. They are fabulous! Just a minor remark: I think that RTOS is not about "fast" responses to an event, but about guaranteed (!) response (in a specified time). How fast the RTOS has to react depends on the physical properties of the system under control.
Great presentation ! I love RTOS, by first was VxWorks, they really make embedded coding easier in most cases. Your single writer, multiple readers comment about mutex not being needed is only true for simple variables - if you need to update many parts of a struct, for example the x and y parts of coordinate, then either both reads and writes always need a mutex (if writes are common relative to reads) or a spin lock on write is needed (faster if writes are much less common than reads), or some odd stuff might be seen ! General rule is mutex unless performance is an issue then selectively remove them with massive comments in code - better safe than sorry . Defensive coding is always the RTOS coders friend !
Thank you for the addition about mutexes. I did not think about those cases because all my tasks only use a fraction of a slot and I always actively pass control back to the scheduler. But I agree: Better safe than sorry...
As a fulltime ESP32 programmer (for 3 years now), I've found that if you want to save yourself a lot of headaches and constantly hitting limitations then either use Arduino as a component under ESP-IDF, or just don't use it at all and use ESP-IDF functions directly. Also utilizing the ESP-IDF & FreeRTOS documentation is quite important, as you learn a lot of crucial things like that a delay of 65ms (15:40 :)) is not actually a thing, it will probably block for 70ms :) you can find out how on your own edit: my last point may be inaccurate, as after a bit of research it seems that arduino might have changed the default value (100) to 1000hz, making 1ms tick resolution possible.
You can be sure I checked the timing of my Morse signals ;-) You also saw it on the logic analyzer. Anyway, using manufacturer's IDE is for sure a better decision if you only work with one architecture. I use at least three architectures. So , the Arduino IDE is ok for me. And I also do not create sellable products.
@@AndreasSpiess Good points indeed, I guess my situation is more of an exception. Also yeah, I edited the comment when I saw the 1ms resolution on the scope ;)
To be fair, the NodeMCU (Lua) firmware has already been offering an async model on the ESP8266/ESP32 since 2015. Not based on FreeRTOS but on the Espressif NonOS SDK.
Thanks Andreas. Great summary of the extra power and features FreeRTOS provides! Further: In principle, mutexes should be used for all shared resources, no exceptions. However, if the resource is simple, e.g. a boolean variable, chances of corruption due to concurrent access are negligable or maybe even non-existent, depending on the code and the compiler. The same applies for a mutex with only one writer and one or more readers. However, taking the risk of corruption should be a conscious one and taken with a lot of care. Small tip: Taking and giving mutexes is "expensive" in terms of performance. In a time critical loop it may be help to use xSemaphoreGetMutexHolder( nameOfSemaphore ) to check if a Mutex has been taken. Apparently, this generates less overhead than the xSemaphoreTake function. An example would be a mutex set by a task to stop a tight loop in another task.
Thank you for the additional info! You are right with the Mutex because we do not know when the millisecond is over (maybe in the middle of an update statement). However, the ESP32 is mindboggling fast. I checked how long my tasks run. Even the longest one (searching for the next letter) only needed a fraction of the 1ms slot. So I always know when I pass the control to the scheduler. I think, with such a fast processor we can avoid many situations a slower CPU could generate. I also proposed to only use one priority based on this fact. Also here we reduce the chance for problematic situations without any harm.
Right? Wrong!!! hahahaha. Andreas, your videos are excellent, and a great source of information. Congratulations, one of the best channels on microcontrollers and iot!!!
Another great presentation!... RTOS in a nutshell 🙂 --> BTW Anderas when you use Visual Studio there is an easy way to move the heavy .vs forder out of your Solution root: Tools > Options > Text Editor > C/C++ > Advanced In the section "Browsing database backup..." change the option "Use backup folder" to "True" and indicate a temporary forlder on your PC as "Backup folder" (the section names may be a bit different, my VS is in French...)
Thank you Andreas. I learn something from every video of yours. Today, I learned QLF 😀. Also, that debugging hook up @ 12:35 caught my interest and maybe suggests a future video. See you next Sunday.
Hallo Andreas! Ja, it's strange to me that, when you start talking OS, people right away think of Linux, Windows, OS X, et al, when we were using RTOS, even in my early days ;-) on 8-bit micro-controllers, and MC68000-based SBC's, to help with reliability and productivity, by providing a "framework" to bolt-on the 'tasks', almost everything running on prioritized interrupt handlers. Well, in truth, the whole idea of an RTOS is that it is based on interrupts! The RTOS reduces a lot of the burden around ensuring "timely" response of the software to "hard real-time" operations, and can really aid in troubleshooting systems. Now, I'll watch the rest of another excellent (I can tell, schon!) video! (well, after taking my insistent cat out for a walk :-) )
You are right. They existed for a long time. But now with the ESP32s they are much easier to use because you will hardly encounter memory or speed limitations. This project only was possible with only a handful RTOS commands because the longest task ran in only a fraction of 1ms and I was able to actively pass the control back to the scheduler. I also was able to increase stack size without thinking to an amount for sure big enough... This was harder in the past.
@@AndreasSpiess you are so, so right, Andreas! In 'the olden days', i designed a hospital bed controller in 512 bytes of code / 256 bytes of RAM, including "faking" a 13-bit ADC, using PWM, RC, and an opamp, with the 8-bit onboard ADC (MC68HC11); but, in many ways, that was more interesting, maybe more 'fun' doing "the impossible" ; -) With today's €4 (or less) tiny 'SBC's, it's amazing what can be done, almost trivially! "We stand on the shoulders of giants".... Thanks Andreas, I'm positive that *you* will have an interesting (challenging) week, coming up with another video for 'us'! Alles gute!
Today we have just to chose higher goals than an automatic bed and it is still challenging, I think. Recently I had a conversation with a young man who builds a earthquake warning system for poorer countries using Maker technology. It seems to work and only cost a fraction of the government established systems in Japan or the US... As you wrote: We always find new projects if we keep our eyes open.
@@AndreasSpiess it was actually a special bed for changing sleeping and rest positions for quadraplegic patients, to avoid the very nasty ulcers that form when there are long term pressure points; it had adjustable movement rates, and adjustable timing between moves, and needed a high-resolution drive control for accurate positioning and quiet operation... just wanted to clarify that the project was not some crappy DIY, or retail toy, but a beneficial medical device, with a few challenges; -) Are there any links the earthquake sensor/prediction system? One of my former colleagues worked on ways to use existing mobile telephones to detect lightning from storms, to predict the storm path, and, simultaneously, provide warnings to handset users, of an incoming storm, or possible tornado... yep, lots of interesting things yet to be done!
I assumed it was a useful bed (my wife works in the hospital). I just meant that today we can build global networks as a hobby project, which was not possible when I was young. Concerning the earthquake project: I plan a video on it where I show the different sensors etc.
Excellent video. The secret of arduino setup() & loop() being RTOS task is very helpful. Learning such tips earlier help a lot. @12:27 the schema and explanation gave a great deal of insight on project design and state machine concept. 👍 There is another saying "He won't get a break if he learns his lesson". I think it is a universal fact that good people are kept busy 🤣
Thank you for this amazing video! I like to believe that me commenting about FreeRTOS in your "The All-New Arduino IDE 2.0: Introduction and Test " video had a small role it its creation. 😊
Great video, much to learn as always. After you last video on ESP32 using two cores I have built several multi tasking systems but I wasn't aware of this level of detail. After this I switched from std:: mutex to xSemaphoreCreateMutex() which saves some memory. FYI If you are porting existing code there is no need to go back and change the calls to delay(); In the framework delay(ms) simply calls vTaskDelay(ms / portTICK_PERIOD_MS);
Danke schön, noch einmal. I managed to get through a whole career in embedded controls without ever using a true RTOS. I started with bare-metal assembler in the early 70s, and retired the same way. I did during that time make many very simple superloop systems, and never wrote blocking code. As I have just transitioned from Arduino to ESP32, and in retirement started learning C++, I found this very interesting. One day I may just have to give it a spin.
As you write, you can do without it. But it is much easier to do it with one. We already used RTOS in the 80s for räsonable complex products. I am curious what your verdict would be after trying it...
I feel the same i.e. Is it worth the time to learn it and how much time to be fully understood? I think if you are into robotics where there are a lot of things going on it could be helpful. Most of my projects are pretty simple though.
been there a month ago.. take care and get an oxymeter to check on your saturation. If it goes lower than 90% please go to a doctor.. Even if you dont feel bad breathing, trust me
@@adilcoelho thanks for your tips. luckily my oxygen level was within safe limits. but i had fever for a very long time and consequently weakness..Here in India, this thing has taken worst direction, almost every fourth person is affected and hospitals are unable to cope up due to sudden increase especially in the last 2 weeks. Last year situation was better.
I use RTOS on the ESP32 for a gps-logger with a e-paper screen (GPS-speedsurfing). As the display routine was blocking (>0.5s), I was stucked... Then I discovered RTOS, and this was a fantastic tool to fix this problem ! Now the esp32 can receive, parse and store the gps data @ 10 Hz on a sd-card ! The display gives me the actual speed and distance. I still work with a lot of global variables, works fine for me !
Excellent video, as always, Andreas. I found myself working through and learning how RTOS worked when working through and debugging the code for the ESP32 Internet radio project that you discussed previously. However this cleared up a few things I was not able to get my head around from the Free RTOS documentation. Thanks again.
Thanks for another great video, Andreas! I have a project which is sitting around for over a year now, which is built on ESP32 and needs to perform several tasks separate from eachother. I got some great ideas on how to accomplish this in a better when than it is managed now.
@@AndreasSpiess Is VERY surprising the level of resistance people have against using a RTOS, specially in camps like CubeIDE. It's like they prefer DOS than Windows
Thank you for the video. Just one observation on using shared variables. The change in one task and read in many works fine if the variable is simple and is written without possibility of the rtos scheduler interrupting it. If you use a more complex data type or structure you need to be careful and be sure you know when its contents is consistent.
Cane here to say this. Basically boils down to if it can be read in a single instruction. A single int doing a single read you can probably get away with. Any sort of array or more complex maths and you're boned. But very very rarely so you probably won't find the bug until you have thousands of units out there. Short version always protect reads and writes to shared variables. If you're planning on doing many operations consider copying the variable into a local variable and working on it there. If you don't, then consider what might happen if that variable changed part way through your function.
I used to do real-time programming 40 years ago using RSX-11 on PDP-11s. I can confirm that debugging can be challenging! One case I remember well - a multithreaded spooler sending output across a network to line printers all over the place. Once in a while a line for one would show up in the middle of output for another! Had to sit and watch for hours until spotting it happening - and send a message across the network to freeze the program so I could arrive in time to get a memory dump! Got it in the end!
Great presentation, as usual Mr. S! Interesting trivia: Back in the 80's, there were multiuser OSes like MPM (a multi-user version of CPM) that didn't time slice. It's true! Each user had a dedicated CPU/RAM card, and the OS managed authentication, physical board allocation, and the shared permanent storage (SMD disks, mag tapes, and even 12-inch laserdiscs). Their 15MB disks were tiny miracles of miniaturization: usually 16 inches wide, 20 inches long, and 10 inches high.
You are right. We even had RTOS systems for one CPU back then. I never heard of such a concept where you had different CPUs managed by one operating system. For me this came later with the VAX/VMS multi-processor systems.
MP/m really did use time slicing on a single CPU. At least in most configurations. MP/m 8-16 seems to have supported multiple processors (typically one 8-bit and one 16-bit). I'd guess it also supported premption and time slicing. Multi-user without premption and time slicing is not really workable in general, but if you really dedicated one core per user it could work. What was the name of the computer system that you used.
@@AndreasSpiess commercial mainframes multiprocessors were available by the early 1960s. For example, the Burroughs D825. As far as I remember, the first multiprocessor VAXes were home-made at Purdue University and ran UNIX.
great stuff Andreas, thanks a lot. But at 20:41 there was a blocked resource in the spellchecker, it seems ;) Thanks a lot for diving into such not obvious topics!
not only I was in the army but also I learned there Morse code as it was still a time to send all the communication over it - what a time it was!!! ;-)
Why Arduino IDE? That's one of the worst IDEs I've tried. Granted, I mostly use Teensy, STM32 and now getting into Pi Pico. PlatformIO with VSC is nice, though Pico support is still very new on that; and STM32Cube is far better for their platform (though it's eclipse based, which is horribly slow).
Greetings from snow covered South Dakota. We have just finished a 3 day blizzard here, lots of snow banks and blocked roads. No mail for 3 days as movement was impossiable even for our mail men. The snow plow just passed my house now at 1400! At last we can get out and shovel snow! Well the folks who can still shovel will get out. I am pretty much stationary any more. I am back among the living once more after having sunken down to the bottom of a well of depression when my lovely wife of 51 years died of cancer. I never knew one could fall so deeply so quickly! Thankfully I have a strong family and good doctors who dug me out of the well and have me again playing with arduinos and ESP units. I have been building my WIFI radios again, and have some nice looking products around my home built in old watch boxes as well as rebuilt old IPOD amplifiers where I built the wifi units and pulled power from their amp power supplies giving me some wonderfully deep sounding systems. I have them programed with music and programs from the 40's through the 80's limiting my station load to around 50 discrete channels. I wanted to thank you for the prayers and caring when I was caring for my wife, it helped.
I am glad to see you here again! Bad luck with the snow in your area. Here we had a very mild winter and now spring is here with nice temperatures. So I am pretty sure that the snow also will go away soon where you live. And I am sure it will help you if you see the flowers and can go outside again. And of course, good music is a very important ingredient!
The first time I heard about the concept of an RTOS was from someone I used to hang out with online who was part of the development team for the RTOS on some fighter jet (the F22, I think), then seeing RTOS as an option when compiling the Linux kernel, and, finally, FreeRTOS on the ESP32.
20:25 A resource does not need to be protected if (i) only one task reads/writes it or (ii) many tasks read it (but none write to it). All other cases need protection. Critically, if one task updates a resource while others read it, you still need protection, as otherwise they might read garbage (a mixture of the 'old' and the 'new' value).
Thank you for your info. I probably oversaw this case because also my longest task only ran a fraction of the 1ms and I was always able to actively pass control to the scheduler.
Love your videos! I've heard the word RTOS many times and thought it's something very complicated. Your explanation gives a very good intro to it. Now there is so many use cases coming in my mind 😁
@RobotsReloaded You don't even think on using a PC as baremetal. An embedded system is basically the same case even if the level of complexity is lower
@@AndreasSpiess I think you found a nice balance between important information and not making it too long. Every single one of the included topics has enough aspects to talk about for a whole day, but not everything is important to know if you just want to use it on an esp. (At least that's when I (re)watch your videos. When I start a project and want to freshen up my knowledge.)
I didn't realize it is such a short jump from ESP32+Arduino IDE to using an RTOS! It was right there all along. Anything that lets us keep our design complexity lower is potentially a huge win.
Great video! Never knew the ESP32 already hat an RTOS. This will be really useful. Minor correction: do the spinlock functions at 21:14 really expect a parameter - the documentation in your next shot suggests they are "void void" functions.
7:52 I don't know the specific problem but crashing the whole program because the wifi management code did miss some implicit deadlines is a bug. the crashing is avertable - everything else is an excuse. handling an issue of any peripheral like the wifi controller, a baseband modem etc. can and has to be dealt with gracefully.
Very nice Andreas, greetings to you and the laboratory manager fur (Feline) from South Africa. We are also concerned with speed, and is why we have parallax flip boards in our possession, 8 cogs to do multi tasking. And then I noticed the Teensy 4 boards, faaaaast. And the the 4.1 can store data to a sd card for recording. This is more industrial type applications for us, so we are not bothered about the 100mA of current to run these mcu's. So much nice stuff, a solution looking for a problem. I enjoyed the video, something to look forward to on a Sunday. See you next week, Kind regards, Duncan
Greetings back to SA. Yes, those boards are very fast. For my applications often much too fast. For this one, I checked and found that none of the tasks needed more than a few percent of one time slot. BTW: Next week there is no video from me. Only 3 per 4 weeks...
I'm here delirious after my second covid shot, wishing I were more coherent to understand all of this. Thanks, Andreas! A topic I've always wondered about.
I loved the Einstein quote re resource allocation to problem vs. solution. And of course your observation about how the typical engineer allocates time is 100% "on target" in my experience. But you missed an opportunity I think - to remind engineers that the quality of their solution often improves when they allocate resources more like Einstein. :) Great video, BTW.
For a great fork, look at Qnx (now a Blackberry Company). 18 years ago they posed the single floppy challenge to all OS manufacturers. They packed a complete OS with IP stack and graphical UI with mouse support on a single 1.44 meg floppy. Brilliant and Canadian.
@@AndreasSpiess You are correct, it is proprietary. I just thought you'd enjoy the perspective of a great RTOS. It does run my current BB Z10 but they gave up on it in favour of white labelling other products. QNX is achieving great industrial success even in the automotive world. All the best Mr. Funny Hat (meant respectfully). Paul N.
Another video with a swiss accent, it must be Sunday! Thanks for providing a regular time reference in COVID induced "days without much distinction" otherwise.
I would add that you probably NEED RTOS as soon you start writing state machines to allow multiple parts of your sketch running parallel. That's what I was doing before knowing RTOS !
When one task is writing to a shared resource and another could read from it, protection with a mutex/semaphore might still be needed, depending on the size of the resource and lots of other details. Best always protect any access to a variable shared between different tasks/threads. I also think there was a problem with your struct queue example @ 18:43. I think it was pushing a pointer to a local struct (so allocated on the stack) into the queue (rather than a copy of that struct). This is dangerous because the pointer may remain in the queue after the sending task has already exited the function this local struct was defined in and hence removed it from the stack. That could raise lots of weird behavior (sometimes working fine, sometimes getting totally different data or even crashing).
Thank you for the info! I did not think too much about the Mutexes because my tasks run on the same core, only use a fraction of a slot and I always actively give control back to th scheduler. The struct is defined outside of the tasks and so far, I did not encounter problems with it.
Hello Andreas, here is the guy from Chile from ESP32 sitting in the first row of your class. I love the tests done with leds and semaphores. However, those of us who develop IoT systems need to have a template for REAL applications that separates slow tasks from fast ones through two threads. Example of slow tasks: - Establish a bluetooth connection - Connect and reconnect to WiFi and MQTT - Validate if there is MQTT connection - In case of alert, send an e-mail - Reception of orders or parameters from MQTT Example of quick tasks: - Sensor reading - Calculations with information from the sensors - Preparation of information that needs to be sent Do you have an example in which in Core 0, you handle the WiFi connection and the rest is handled in Core 1 (or vice versa)? The sketch is booted for me when I try to separate the WiFi connection and put it in the slow core. I have tried putting it in core 0 and core 1. Can you give us a solution that connects WiFi on a separate thread? It would be magnificent!!
I do not think that slow or fast is a good classification. I see all tasks as what they are: A task has prerequisites to run and leads to results. This is the first thing to design. Second is to find out whether some of the tasks are so time-critical that they have to be able to interrupt others. These get a higher priority. Third is "load balancing" where you distribute and assign tasks to cores. Often this is not needed and can be left to the RTOS. With the ESP32 we have to consider that some tasks already exist (like Wi-Fi) and run on a core with a high priority. They can influence your decisions. I would only go to step 2 and even 3 if you foresee or encounter problems. So your "slow" tasks all are low priority tasks and "quick" tasks have a higher priority. Keep in mind that no task runs for long because it is always interrupted by RTOS.
4:13 Odd to see so many processes with identical virtual and resident memory usages, totalling up to probably more than your machine can handle. Are they separate threads sharing the same memory context?
At 7:57 there is a reference to ESP8266 crashes due to keeping the processor busy. I found this was discussed in a previous video #33: ua-cam.com/video/D_7ciW_TCac/v-deo.html
I am not a big fan of interrupts and when writing more involved programs I am always trying to keep the loop time a low as possible. I need to do this project and just play around with the few commands that you used. Thanks for trimming down a very large amount of info into something we can use.
How about the fact that the 60s seems like it should be only 40 years ago rather than 60 and that I, born in 1982, am closer to middle age (45) than I am to my 20s (7 vs 8 years)?
I used RTOS on the ESP32. It is a nice clean approach. But it is not easier compared to a standard sketch. Just much cleaner for software architecture. I like it.
Thanks for explaining FreeRTOS with well suited example as well. One thing I want to bring to your attention that FreeRTOS is not designed to support multiple CPU cores. You when you said that scheduler is giving time to core0 for network related tasks that is not completely correct. Yes scheduler manages core0 tasks as well but that is separate scheduler dedicated for core0. It means there are two separate schedulers running for core0 and core1.
Thanks! Will certainly help to rethink my switch debouncing motor controlling code that crashes all the time. At 21:42, Would it be wrong to say that we use "RTOS" all the time, even for a "hello world". But the question is when to use more advanced features like multitasking.
I think the best time to use multitasking is when your project is trying to do more than one thing at a time, and you can logically break it into parts. The code in each task becomes way simpler, and you can decouple one process from another process. The way Andreas was able to split his morse solution into simpler parts, is I think the best part of having a rtos.
Oh your info on this RTOS system reminds me of my college days back in the early 90's. Our computer labs were made up of 486 machines running DOS and WINDOWS 3.1 that nobody used much except to play with as a sort of DOS file system manager. Then we had Intro to UNIX! That was cool because using those old machines we could go into the network and sign onto the School's Mainframe machine in UNEX and do some real computing in an Operating system where all who were in the lab at one time 25 computers could sign into the Mainframe and tinker around with the Unix system. I still keep a Linux partition on my laptop and get into it when my anger at Windows exceeds my desire to get work done.
Correction: ISR is Interrupt Service Routine, not internet service routine
This must be an April fools joke.
I already posted a pinned correction comment ;-)
Another great tutorial and reason to love the esp32 even more, thanks!
ua-cam.com/video/LaYXrdASR8M/v-deo.html
Correct, great teacher
Shawn did a really good series on freertos, I’m glad you reference it.
Agreed. Probably a little "oversophisticated" versus the end...
He has put out a bunch of good STM32 CubeMX videos too.
Who’s Shawn ? Could you send a link please
@@iSuperMC Shawn Hymell (Digi-Key). He does a lot of instructional videos but he's a bit of an egg-head for most of us.
One aspect of RTOS development you somewhat glossed over is the amount of memory to assign for each task's stack. FreeRTOS has features to determine stack usage / detect stack overflow, so that the stack size for each task can be set appropriately. For development it's useful to use a processor with plenty of RAM so that task stacks can be set much larger than needed and then reduced so the application can run on a part with less RAM.
Another important FreeRTOS specific detail is enabling the RTOS features you want in the FreeRTOS config header file.
Finally, for those using STM32 processors, avoid using both STMCube HAL and FreeRTOS together. They do not play nice!
Thank you for your additional info. Indeed, I did not know how much stack is enough. Fortunately, the ESP32 has enough RAM for my application. So I did not need to answer the question...
Hello what problems did you observ with rtos and HAL library. Please Share your experiences.
@@yagizhanugur cant be used together as they both try to play the OS part and do scheduling and shit like that
they are flat out incompatible
@@AndreasSpiess I found this on the ESP-IDF docs that might produce an answer:
- At runtime, call the function uxTaskGetStackHighWaterMark() with the handle of any task where you think there is unused stack memory. This function returns the minimum lifetime free stack memory in bytes. The easiest time to call this is from the task itself: call uxTaskGetStackHighWaterMark(NULL) to get the current task’s high water mark after the time that the task has achieved its peak stack usage (i.e. if there is a main loop, execute the main loop a number of times with all possible states and then call uxTaskGetStackHighWaterMark()). Often, it’s possible to subtract almost the entire value returned here from the total stack size of a task, but allow some safety margin to account for unexpected small increases in stack usage at runtime.
- Call uxTaskGetSystemState() at runtime to get a summary of all tasks in the system. This includes their individual stack “high watermark” values.
It's in the RAM Usage section: docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/performance/ram-usage.html
Thanks for your wonderfully informative and well made videos!
I can't even begin to tell you how much I value the work you (and other educational youtubers who are truly educated and articulate) do for the comunity and myself..... Tryin to jump in with arduino and programming to make my dream projects come true would have overwhelmed me and I would have definitely given up without the resources you provide. Sometimes when your teaching yourself things the hardest part is not knowing what you don't know, and not knowing how to ask the right questions, so when you go out of your way to provide context and redundant examples along the way without delight too deep into those other topics it really elevates the content from infotainment to genuinely valuable educational material.
Thank you for the nice words! I became a UA-camr because I learned a lot when I started with the Arduino myself...
Great explanation, and thanks for the shout out!
You are welcome!
Thanks for referencing my vid Andreas! I’m self taught so I didn’t know there are more elegant ways to do multitasking. I haven’t watched all of this video yet but I will definitely come back to it. I’m getting back into coding to make a synthesizer so this is just what I need.
You are welcome. You will see, RTOS is a different world because most things are "under the hood" and your code is easy to read.
Thank you very much! I can't even imagine all the work behind this video! Great job.
You are welcome!
You are right, this one was not easy. Also because I had to concentrate very much on the important things. The script took many iterations...
One of the best RTOS video to start with, than you so much for making these! Also, love the jokes and the metaphors!
You are welcome! Thank you for your kind words.
A great intro to RTOS.
I find it good practice to use it all of the time with the ESP32 and just have the OTA in the "loop" function. It's a good way to learn how to program many micro controllers in the big industry and not just hobbyist style coding.
As I've also been use RTOS on many other microcontrollers like ARM and PIC for many years in industry. This makes porting code from other microcontrollers to the ESP range very easy.
...Keep up the great work.
- Enjoying watching from the UK.
😉👌
Thank you for your feedback. For sure it makes a lot simpler...
incredibly explained, as always, thanks!, I wish I had a teacher like you when I was studying at the university
Happy to help! And thank you for your nice words.
Andreas, excellent instructional video. Your explanation of time slicing, task creation by breaking down the requirements of the program, communication queues was well explained. The analysis aspect of programming is a crucial skill that is required when incorporating the additional control of a multitasking environment into a program solution. Your big picture overview to small detail approach, embedded humour (pun) ensure your technical videos are a great learning tool and a joy to watch. Well done!
Thank you for your nice words!
Excellent tutorial Andreas! I wanted to start off with RTOS and I remembered that you had a video on this. So I went straight into it as I know you would do the best job is getting me started. Thank you so much. Now I can go through the rest of documentation and other tutorials :)
Glad to read that. A good start is all I wanted to provide ;-)
I almost never critique UA-cam videos simply because they are what they are, a freely given effort to inform people of something. But your videos are exceptionally good due to your intellect, consummate skills, and your meticulous presentation. So because of that, and at the risk of being perceived as overly critical I will point out that you said 'seconds' instead of 'milliseconds' at 23:07 into the video. I do so because I think that to you, it matters. You actually want to be clear, concise and correct about everything you tell us, and you work at it throughout your videos. That is the reason why I watch ALL of your videos, even on subjects in which I have little interest, because they all exhibit what few other UA-cam videos do - genuine quality. I appreciate the quality, and I can guess how much mental and physical effort is involved in creating and maintaining it, video after video. Thank you Andreas.
Thank you for the info about my slip of the tongue . And thank you for your nice words. Glad you like the content of the channel.
"If you are good, you get more work!" Not going to lie, that line gave me flash backs to my Air Force days. haha As always, love the videos and I always learn a lot from you!
:-))
20:25 I think the mutex is used when you can have concurrent access and not just multiple writes,
assume one task is reading a struct, and in the middle of reading, the other task modifies the struct.
If the processor can read/write 32 bit variables atomically you don't need to protect them (if you have only one writer). If your processor cannot access the data in one instruction, you indeed need protect read and write.
yes, when the display task needs to print the variable onto the screen, the writing task should not change it until the display task finished printing.
Exactly, you may have single-writer-multi-reader-configs when the shared resource can be written atomically, like 32-bit words on a 32-bit processor, but most other things require some protection even when only one thread is writing. Atomic operations (Compare and Swap etc) are also a useful thing to look into. They can help avoid mutexes and therefore overhead when building queues or counters.
Thank you for the discussion. I would like to add another aspect: Because the ESP32 is so fast, none of my tasks needed more than a fraction of 1ms and I was always able to actively pass the control back to the scheduler. Is a mutex still needed?
@@AndreasSpiess and thanks again to you dear Andreas for your great contents.
Hadnt thought about it this way, your assuming since there is no preemption in your case, the mutex is not needed, which I think is right.
although I would always add them as good practice since you don't know what future holds...
Never knew the ESP32 had a full FreeRTOS running in the background always. It is well hidden, not showing when we write simple serial Arduino programs. And not only it manages the WiFi on one core, but it gives us a really easy way to write parallel code with tasks and queues and locks. Brilliant.
I agree!
Was waiting for something like this .
Thank you
:-)
lately every time I have an interest in a topic and search youtube you've made a video on it already. Thanks once again :)
Happy to hear that!
Another great video Andreas.! I liked your history lesson at the beginning. In the late '80s I worked with Forth which was perfectly capable of multitasking with 8bit microprocessors like the 1802, Z80, 6809 & 8051. It used a cooperative round robin model which meant the program designer could allocate as much cpu time as was needed for each part of a task. With no interrupt and time slicing the task switching overhead was tiny. One benefit was the ability to test and alter code from the keyboard with existing tasks running in the background.
You are right. We currently re-discover things we had in the 80s like inline debugging. But of course, today it is much easier because of the resources an ESP32 offers. It has nearly no limitations for an average project. Which makes things much easier. No optimization needed, just straight programming...
Theres a lot of really informative material in this video THANKS ! I didnt realize it was so straightforward to run freeRTOS on an ESP32. My first freeRTOS app was running within 15mins of completing this video. Really encouraged now to delve deeper.
I think a deep dive will be rewarding. Enjoy!
Excellent comprehensive example with code examination. I've used FreeRTOS on 10 or so projects. What you've covered here is probably 90% of FreeRTOS projects. If users can get through your example understanding these concepts, their spaghetti code will become readable, understandable and maintainable. One last thing, do NOT use arduino delay. It affects the scheduler and your code will stop working. Search your code and change delay to vTaskDelay. Thanks from Colorado.
Thanks for sharing your experience! One question: What does the Arduino delay() harm? I thought, it just takes up all the time in the task's slot (which is, of course, not ideal).
@@AndreasSpiess I looked at the ESP32 FreeRTOS delay code. It implements Arduino delay with vTaskDelay, so on the ESP32 using delay should work fine. I believe my comment above deals the smaller Atmel parts, 328P, with limited timers. Seems like I remember using Arduino delay changes TIMER2 (may be incorrect here), that's the same timer used by FreeRTOS's scheduler. It may be different now, but back then Arduino's delay affected the scheduler. If this is incorrect or has changed, please comment below. I've switched to using STM32's for my designs which runs FreeRTOS just fine. Thanks again for the video, I'm always interested in what your doing (keeps me from having to figure things out), you do a wonderful and complete exploration.
On Linux Arduino 1.8.13, ESP32 delay implementation is here.
~/.arduino15/packages/esp32/hardware/esp32/1.0.6/cores/esp32/esp32-hal-misc.c
void delay(uint32_t ms)
{
vTaskDelay(ms / portTICK_PERIOD_MS);
}
Ein Video an meinem Geburtstag über eines meiner aktuellen Lieblings-Bauelemente. Das ist ein extrem interessantes Thema, -werde mich da demnächst mehr mit beschäftigen. Danke für's zeigen!
Bitte, gern geschehen, Und herzliche Glückwünsche!
At first! Thank you for your great explanations on all the different topics. They are fabulous! Just a minor remark: I think that RTOS is not about "fast" responses to an event, but about guaranteed (!) response (in a specified time). How fast the RTOS has to react depends on the physical properties of the system under control.
Good point!
Thanks a lot for this compact introduction in such an important topic as FreeRTos on ESP32!
You are welcome! Like hardware debugging, this is game changing for me...
Really impressive! The best channel on this subject in Internet.
I wish my brain could multitask his knowledge without all the deadlocks! Great content as usual Andreas liked vid.
Wow, thanks!
Great presentation ! I love RTOS, by first was VxWorks, they really make embedded coding easier in most cases. Your single writer, multiple readers comment about mutex not being needed is only true for simple variables - if you need to update many parts of a struct, for example the x and y parts of coordinate, then either both reads and writes always need a mutex (if writes are common relative to reads) or a spin lock on write is needed (faster if writes are much less common than reads), or some odd stuff might be seen ! General rule is mutex unless performance is an issue then selectively remove them with massive comments in code - better safe than sorry . Defensive coding is always the RTOS coders friend !
Thank you for the addition about mutexes. I did not think about those cases because all my tasks only use a fraction of a slot and I always actively pass control back to the scheduler. But I agree: Better safe than sorry...
As a fulltime ESP32 programmer (for 3 years now), I've found that if you want to save yourself a lot of headaches and constantly hitting limitations then either use Arduino as a component under ESP-IDF, or just don't use it at all and use ESP-IDF functions directly.
Also utilizing the ESP-IDF & FreeRTOS documentation is quite important, as you learn a lot of crucial things like that a delay of 65ms (15:40 :)) is not actually a thing, it will probably block for 70ms :) you can find out how on your own
edit: my last point may be inaccurate, as after a bit of research it seems that arduino might have changed the default value (100) to 1000hz, making 1ms tick resolution possible.
You can be sure I checked the timing of my Morse signals ;-) You also saw it on the logic analyzer.
Anyway, using manufacturer's IDE is for sure a better decision if you only work with one architecture. I use at least three architectures. So , the Arduino IDE is ok for me. And I also do not create sellable products.
@@AndreasSpiess Good points indeed, I guess my situation is more of an exception. Also yeah, I edited the comment when I saw the 1ms resolution on the scope ;)
I believe this video should be fundamental for every maker! Thanks Andreas!
Glad you think so!
Cool! Appreciate that you trying to push asynchronous programing into embedded DIY areas.
Thank you! Maybe after this video a "pull" will result ;-)
To be fair, the NodeMCU (Lua) firmware has already been offering an async model on the ESP8266/ESP32 since 2015. Not based on FreeRTOS but on the Espressif NonOS SDK.
Thanks Andreas. Great summary of the extra power and features FreeRTOS provides!
Further: In principle, mutexes should be used for all shared resources, no exceptions. However, if the resource is simple, e.g. a boolean variable, chances of corruption due to concurrent access are negligable or maybe even non-existent, depending on the code and the compiler. The same applies for a mutex with only one writer and one or more readers. However, taking the risk of corruption
should be a conscious one and taken with a lot of care.
Small tip: Taking and giving mutexes is "expensive" in terms of performance. In a time critical loop it may be help to use xSemaphoreGetMutexHolder( nameOfSemaphore ) to check if a Mutex has been taken. Apparently, this generates less overhead than the xSemaphoreTake function. An example would be a mutex set by a task to stop a tight loop in another task.
Thank you for the additional info! You are right with the Mutex because we do not know when the millisecond is over (maybe in the middle of an update statement).
However, the ESP32 is mindboggling fast. I checked how long my tasks run. Even the longest one (searching for the next letter) only needed a fraction of the 1ms slot. So I always know when I pass the control to the scheduler.
I think, with such a fast processor we can avoid many situations a slower CPU could generate. I also proposed to only use one priority based on this fact. Also here we reduce the chance for problematic situations without any harm.
Right? Wrong!!! hahahaha. Andreas, your videos are excellent, and a great source of information. Congratulations, one of the best channels on microcontrollers and iot!!!
Thank you for your nice words!
Another great presentation!... RTOS in a nutshell 🙂
--> BTW Anderas when you use Visual Studio there is an easy way to move the heavy .vs forder out of your Solution root:
Tools > Options > Text Editor > C/C++ > Advanced
In the section "Browsing database backup..." change the option "Use backup folder" to "True" and indicate a temporary forlder on your PC as "Backup folder"
(the section names may be a bit different, my VS is in French...)
Thanks for the tip!
Never die my hero! a lot of love from Argentina
:-)
Thank you Andreas. I learn something from every video of yours. Today, I learned QLF 😀.
Also, that debugging hook up @ 12:35 caught my interest and maybe suggests a future video. See you next Sunday.
When I listen to the bands I hear a lot of QLF...
I already made a video about hardware debugging with PlatformIO.
Thank you for taking time to recurd such an instructive video. Keep up your wonderful work!
My pleasure! And thank you for your nice words!
Hallo Andreas! Ja, it's strange to me that, when you start talking OS, people right away think of Linux, Windows, OS X, et al, when we were using RTOS, even in my early days ;-) on 8-bit micro-controllers, and MC68000-based SBC's, to help with reliability and productivity, by providing a "framework" to bolt-on the 'tasks', almost everything running on prioritized interrupt handlers. Well, in truth, the whole idea of an RTOS is that it is based on interrupts! The RTOS reduces a lot of the burden around ensuring "timely" response of the software to "hard real-time" operations, and can really aid in troubleshooting systems. Now, I'll watch the rest of another excellent (I can tell, schon!) video! (well, after taking my insistent cat out for a walk :-) )
You are right. They existed for a long time. But now with the ESP32s they are much easier to use because you will hardly encounter memory or speed limitations. This project only was possible with only a handful RTOS commands because the longest task ran in only a fraction of 1ms and I was able to actively pass the control back to the scheduler. I also was able to increase stack size without thinking to an amount for sure big enough...
This was harder in the past.
@@AndreasSpiess you are so, so right, Andreas! In 'the olden days', i designed a hospital bed controller in 512 bytes of code / 256 bytes of RAM, including "faking" a 13-bit ADC, using PWM, RC, and an opamp, with the 8-bit onboard ADC (MC68HC11); but, in many ways, that was more interesting, maybe more 'fun' doing "the impossible" ; -)
With today's €4 (or less) tiny 'SBC's, it's amazing what can be done, almost trivially!
"We stand on the shoulders of giants"....
Thanks Andreas, I'm positive that *you* will have an interesting (challenging) week, coming up with another video for 'us'! Alles gute!
Today we have just to chose higher goals than an automatic bed and it is still challenging, I think. Recently I had a conversation with a young man who builds a earthquake warning system for poorer countries using Maker technology. It seems to work and only cost a fraction of the government established systems in Japan or the US... As you wrote: We always find new projects if we keep our eyes open.
@@AndreasSpiess it was actually a special bed for changing sleeping and rest positions for quadraplegic patients, to avoid the very nasty ulcers that form when there are long term pressure points; it had adjustable movement rates, and adjustable timing between moves, and needed a high-resolution drive control for accurate positioning and quiet operation... just wanted to clarify that the project was not some crappy DIY, or retail toy, but a beneficial medical device, with a few challenges; -)
Are there any links the earthquake sensor/prediction system? One of my former colleagues worked on ways to use existing mobile telephones to detect lightning from storms, to predict the storm path, and, simultaneously, provide warnings to handset users, of an incoming storm, or possible tornado... yep, lots of interesting things yet to be done!
I assumed it was a useful bed (my wife works in the hospital). I just meant that today we can build global networks as a hobby project, which was not possible when I was young.
Concerning the earthquake project: I plan a video on it where I show the different sensors etc.
Nice and sufficiently accurate description of a difficult subject. Note that shared resources also include your display.
You are right. I thought, the display was on the list of shared resources...
Excellent video. The secret of arduino setup() & loop() being RTOS task is very helpful. Learning such tips earlier help a lot.
@12:27 the schema and explanation gave a great deal of insight on project design and state machine concept. 👍
There is another saying "He won't get a break if he learns his lesson". I think it is a universal fact that good people are kept busy 🤣
Thank you. Glad it was helpful!
Thank you for this amazing video!
I like to believe that me commenting about FreeRTOS in your "The All-New Arduino IDE 2.0: Introduction and Test
" video had a small role it its creation. 😊
:-)
Great video, much to learn as always. After you last video on ESP32 using two cores I have built several multi tasking systems but I wasn't aware of this level of detail. After this I switched from std:: mutex to xSemaphoreCreateMutex() which saves some memory.
FYI If you are porting existing code there is no need to go back and change the calls to delay(); In the framework delay(ms) simply calls vTaskDelay(ms / portTICK_PERIOD_MS);
Thank you for the tip about delay(). I was not sure.
Danke schön, noch einmal.
I managed to get through a whole career in embedded controls without ever using a true RTOS. I started with bare-metal assembler in the early 70s, and retired the same way. I did during that time make many very simple superloop systems, and never wrote blocking code. As I have just transitioned from Arduino to ESP32, and in retirement started learning C++, I found this very interesting. One day I may just have to give it a spin.
As you write, you can do without it. But it is much easier to do it with one. We already used RTOS in the 80s for räsonable complex products. I am curious what your verdict would be after trying it...
I feel the same i.e. Is it worth the time to learn it and how much time to be fully understood? I think if you are into robotics where there are a lot of things going on it could be helpful. Most of my projects are pretty simple though.
Interesting topic. i will have to watch it again when i am more fit..
Just came out of covid fever, feeling weak..
been there a month ago.. take care and get an oxymeter to check on your saturation. If it goes lower than 90% please go to a doctor.. Even if you dont feel bad breathing, trust me
Glad to read that both of you survived! So far I had no problems and now, I go my first shot and some hope that I can avoid your experiences.
Good to know you're coming out of it, and to have enough strength to post about it.
@@AndreasSpiess the shot will surely help.. Glad youre getting it! Your videos are awesome
@@adilcoelho thanks for your tips. luckily my oxygen level was within safe limits. but i had fever for a very long time and consequently weakness..Here in India, this thing has taken worst direction, almost every fourth person is affected and hospitals are unable to cope up due to sudden increase especially in the last 2 weeks. Last year situation was better.
This is so much more elegant than the mess of millis(). Thank you for the clear explanations, as always.
Indeed! Much easier to read.
I use RTOS on the ESP32 for a gps-logger with a e-paper screen (GPS-speedsurfing). As the display routine was blocking (>0.5s), I was stucked... Then I discovered RTOS, and this was a fantastic tool to fix this problem ! Now the esp32 can receive, parse and store the gps data @ 10 Hz on a sd-card ! The display gives me the actual speed and distance. I still work with a lot of global variables, works fine for me !
A good application for an RTOS, indeed!
That was a great introduction. It would be great to see other FreeRTOS videos. Thanks
We will see. Maybe as a part of a project.
Excellent video, as always, Andreas. I found myself working through and learning how RTOS worked when working through and debugging the code for the ESP32 Internet radio project that you discussed previously. However this cleared up a few things I was not able to get my head around from the Free RTOS documentation. Thanks again.
You are welcome! I forgot that the internet radio also used RTOS. I am getting old...
nice video! today i saw a little Tensorflow on an ESP32cam and then this video! So much to learn!
True. But a lot of opportunities!
Thanks for another great video, Andreas! I have a project which is sitting around for over a year now, which is built on ESP32 and needs to perform several tasks separate from eachother. I got some great ideas on how to accomplish this in a better when than it is managed now.
You will see that, if you get the design of the different modules right, things get much simpler than with standard programming.
@@AndreasSpiess Is VERY surprising the level of resistance people have against using a RTOS, specially in camps like CubeIDE. It's like they prefer DOS than Windows
As always: Great Work, Andreas! Thx!
You are welcome!
Amazing! Thank you so much Andreas! And you have great humor also!
I try my best to keep the humor in those times ;-)
Grüezi Andreas 🇨🇭
I have to give this a try. Thanks for sharing ✌️
Hope you enjoy
Once again a very detailed and useful video, including a wealth of information and tips ... Thank you :)
My pleasure!
Thank you for the video. Just one observation on using shared variables. The change in one task and read in many works fine if the variable is simple and is written without possibility of the rtos scheduler interrupting it. If you use a more complex data type or structure you need to be careful and be sure you know when its contents is consistent.
You are right. I forgot about that fact. Thank you.
Cane here to say this. Basically boils down to if it can be read in a single instruction. A single int doing a single read you can probably get away with. Any sort of array or more complex maths and you're boned. But very very rarely so you probably won't find the bug until you have thousands of units out there.
Short version always protect reads and writes to shared variables. If you're planning on doing many operations consider copying the variable into a local variable and working on it there.
If you don't, then consider what might happen if that variable changed part way through your function.
I used to do real-time programming 40 years ago using RSX-11 on PDP-11s. I can confirm that debugging can be challenging! One case I remember well - a multithreaded spooler sending output across a network to line printers all over the place. Once in a while a line for one would show up in the middle of output for another! Had to sit and watch for hours until spotting it happening - and send a message across the network to freeze the program so I could arrive in time to get a memory dump! Got it in the end!
Thank you for telling us the nice story! It shows exactly what I wanted to say.
Great presentation, as usual Mr. S! Interesting trivia: Back in the 80's, there were multiuser OSes like MPM (a multi-user version of CPM) that didn't time slice. It's true! Each user had a dedicated CPU/RAM card, and the OS managed authentication, physical board allocation, and the shared permanent storage (SMD disks, mag tapes, and even 12-inch laserdiscs). Their 15MB disks were tiny miracles of miniaturization: usually 16 inches wide, 20 inches long, and 10 inches high.
You are right. We even had RTOS systems for one CPU back then. I never heard of such a concept where you had different CPUs managed by one operating system. For me this came later with the VAX/VMS multi-processor systems.
MP/m really did use time slicing on a single CPU. At least in most configurations. MP/m 8-16 seems to have supported multiple processors (typically one 8-bit and one 16-bit). I'd guess it also supported premption and time slicing.
Multi-user without premption and time slicing is not really workable in general, but if you really dedicated one core per user it could work.
What was the name of the computer system that you used.
@@AndreasSpiess commercial mainframes multiprocessors were available by the early 1960s. For example, the Burroughs D825.
As far as I remember, the first multiprocessor VAXes were home-made at Purdue University and ran UNIX.
great stuff Andreas, thanks a lot. But at 20:41 there was a blocked resource in the spellchecker, it seems ;) Thanks a lot for diving into such not obvious topics!
:-)) You are right. Unfortunately here the problem was between my ears...
Impressive video as always 👍
Thanks for sharing your experience with all of us 👍😀
Thank you! This one was not too simple to understand, I think...
excellent video. Multitasking is finally a reality in my projects!
Fantastic!
Thanks for taking the time on making these videos. You are a great great teacher! I've learned a lot from you. Thanks for all!
You're welcome!
not only I was in the army but also I learned there Morse code as it was still a time to send all the communication over it - what a time it was!!! ;-)
I learned Morse in military, too!
Why Arduino IDE? That's one of the worst IDEs I've tried. Granted, I mostly use Teensy, STM32 and now getting into Pi Pico. PlatformIO with VSC is nice, though Pico support is still very new on that; and STM32Cube is far better for their platform (though it's eclipse based, which is horribly slow).
We covered this topic in the last video, I think.
I thought that i will never use FreeRTOS because of its pain in a.. , but this video changed my decision. Thanks a lot
You will love it!
Greetings from snow covered South Dakota. We have just finished a 3 day blizzard here, lots of snow banks and blocked roads. No mail for 3 days as movement was impossiable even for our mail men. The snow plow just passed my house now at 1400! At last we can get out and shovel snow! Well the folks who can still shovel will get out. I am pretty much stationary any more. I am back among the living once more after having sunken down to the bottom of a well of depression when my lovely wife of 51 years died of cancer. I never knew one could fall so deeply so quickly! Thankfully I have a strong family and good doctors who dug me out of the well and have me again playing with arduinos and ESP units. I have been building my WIFI radios again, and have some nice looking products around my home built in old watch boxes as well as rebuilt old IPOD amplifiers where I built the wifi units and pulled power from their amp power supplies giving me some wonderfully deep sounding systems. I have them programed with music and programs from the 40's through the 80's limiting my station load to around 50 discrete channels. I wanted to thank you for the prayers and caring when I was caring for my wife, it helped.
I am glad to see you here again! Bad luck with the snow in your area. Here we had a very mild winter and now spring is here with nice temperatures. So I am pretty sure that the snow also will go away soon where you live. And I am sure it will help you if you see the flowers and can go outside again. And of course, good music is a very important ingredient!
The first time I heard about the concept of an RTOS was from someone I used to hang out with online who was part of the development team for the RTOS on some fighter jet (the F22, I think), then seeing RTOS as an option when compiling the Linux kernel, and, finally, FreeRTOS on the ESP32.
The concept is old (I used one in the 80s). But now with the abundance resources it is very easy to use...
20:25 A resource does not need to be protected if (i) only one task reads/writes it or (ii) many tasks read it (but none write to it). All other cases need protection.
Critically, if one task updates a resource while others read it, you still need protection, as otherwise they might read garbage (a mixture of the 'old' and the 'new' value).
Thank you for your info. I probably oversaw this case because also my longest task only ran a fraction of the 1ms and I was always able to actively pass control to the scheduler.
Very nicely made video of the project. Good scope. I like the design of the deck too.
Thank you!
Love your videos! I've heard the word RTOS many times and thought it's something very complicated. Your explanation gives a very good intro to it. Now there is so many use cases coming in my mind 😁
Glad it was helpful!
@RobotsReloaded You don't even think on using a PC as baremetal. An embedded system is basically the same case even if the level of complexity is lower
Thanks for the video. You managed to scrape by a whole semester of a programming course in just 23 minutes.
You are welcome! I had to leave a lot of details out, though...
@@AndreasSpiess I think you found a nice balance between important information and not making it too long.
Every single one of the included topics has enough aspects to talk about for a whole day, but not everything is important to know if you just want to use it on an esp.
(At least that's when I (re)watch your videos. When I start a project and want to freshen up my knowledge.)
Indeed! A whole semester in just 23 minutes - completely explained in a perfect way!
Thank you Andreas!
Well done as usual!
I didn't realize it is such a short jump from ESP32+Arduino IDE to using an RTOS! It was right there all along. Anything that lets us keep our design complexity lower is potentially a huge win.
Indeed it is a short jump for a start. Maybe it needs some practice...
thank you for talking about the hidden esp32 secrets and wonders :)
You are welcome!
Great video! Never knew the ESP32 already hat an RTOS. This will be really useful. Minor correction: do the spinlock functions at 21:14 really expect a parameter - the documentation in your next shot suggests they are "void void" functions.
In the ESP32 implementation they expect (or allow?) a parameter. But I am no specialist.
7:52 I don't know the specific problem but crashing the whole program because the wifi management code did miss some implicit deadlines is a bug. the crashing is avertable - everything else is an excuse. handling an issue of any peripheral like the wifi controller, a baseband modem etc. can and has to be dealt with gracefully.
These problems are not nice, but can easily be resolved. So no problem for most of us.
Very nice Andreas, greetings to you and the laboratory manager fur (Feline) from South Africa.
We are also concerned with speed, and is why we have parallax flip boards in our possession, 8 cogs to do multi tasking.
And then I noticed the Teensy 4 boards, faaaaast.
And the the 4.1 can store data to a sd card for recording.
This is more industrial type applications for us, so we are not bothered about the 100mA of current to run these mcu's.
So much nice stuff, a solution looking for a problem.
I enjoyed the video, something to look forward to on a Sunday.
See you next week,
Kind regards,
Duncan
Greetings back to SA. Yes, those boards are very fast. For my applications often much too fast. For this one, I checked and found that none of the tasks needed more than a few percent of one time slot.
BTW: Next week there is no video from me. Only 3 per 4 weeks...
I'm here delirious after my second covid shot, wishing I were more coherent to understand all of this. Thanks, Andreas! A topic I've always wondered about.
The advantage of UA-cam is that you can re-view videos later if you want...
@@AndreasSpiess what will they think of next?! 😂
no one cares.
Wow ! That was really an eye opener for me. I had no idea we could use that. Thank you weird accent guy !
Glad you liked it!
I loved the Einstein quote re resource allocation to problem vs. solution. And of course your observation about how the typical engineer allocates time is 100% "on target" in my experience. But you missed an opportunity I think - to remind engineers that the quality of their solution often improves when they allocate resources more like Einstein. :) Great video, BTW.
You are right. But I do not like to tell people what to do. I hope some learned from Einstein ;-)
Thanks Andreas, for yet another great educational video !!
Glad you liked it!
Thanks for starting with RTOS, always wanted to learn it!
Go for it. It is worthwhile.
Thanks. Great as usual, that was a nice introduction and overview of RTOS for me.
Glad it was helpful!
For a great fork, look at Qnx (now a Blackberry Company). 18 years ago they posed the single floppy challenge to all OS manufacturers. They packed a complete OS with IP stack and graphical UI with mouse support on a single 1.44 meg floppy. Brilliant and Canadian.
I still remember my first Blackberry when it was a status symbol ;-) According to Wikipedia, QNX is proprietary and does not run on our MCUs.
@@AndreasSpiess You are correct, it is proprietary. I just thought you'd enjoy the perspective of a great RTOS. It does run my current BB Z10 but they gave up on it in favour of white labelling other products. QNX is achieving great industrial success even in the automotive world. All the best Mr. Funny Hat (meant respectfully). Paul N.
Thank you for the lesson!
You are welcome!
Another video with a swiss accent, it must be Sunday! Thanks for providing a regular time reference in COVID induced "days without much distinction" otherwise.
You are welcome! Indeed it sometimes in not easy to distinguish weekends and weekdays with all this Home Office...
I would add that you probably NEED RTOS as soon you start writing state machines to allow multiple parts of your sketch running parallel. That's what I was doing before knowing RTOS !
Uhm you definitely don't need it for a state machine.
Great stuff, Andreas! I appreciate your high-quality information~
My pleasure!
"Guru Meditation Error" is a gag from the old Commodore Amiga. I like it.
I did not know that. Thanks!
@@AndreasSpiess whenever the diskette didn't work or a program crashed. That was a gimmick by the programmers.
When one task is writing to a shared resource and another could read from it, protection with a mutex/semaphore might still be needed, depending on the size of the resource and lots of other details. Best always protect any access to a variable shared between different tasks/threads.
I also think there was a problem with your struct queue example @ 18:43. I think it was pushing a pointer to a local struct (so allocated on the stack) into the queue (rather than a copy of that struct). This is dangerous because the pointer may remain in the queue after the sending task has already exited the function this local struct was defined in and hence removed it from the stack. That could raise lots of weird behavior (sometimes working fine, sometimes getting totally different data or even crashing).
Thank you for the info! I did not think too much about the Mutexes because my tasks run on the same core, only use a fraction of a slot and I always actively give control back to th scheduler.
The struct is defined outside of the tasks and so far, I did not encounter problems with it.
21:08 Spinlocks are commonly used for multiprocessor synchronization. The Linux kernel has them.
I want more videos like this! Thanks for this video
You are welcome! We will see...
Very nice! Thank you for this video.
Glad you liked it!
Hello Andreas, here is the guy from Chile from ESP32 sitting in the first row of your class. I love the tests done with leds and semaphores. However, those of us who develop IoT systems need to have a template for REAL applications that separates slow tasks from fast ones through two threads.
Example of slow tasks:
- Establish a bluetooth connection
- Connect and reconnect to WiFi and MQTT
- Validate if there is MQTT connection
- In case of alert, send an e-mail
- Reception of orders or parameters from MQTT
Example of quick tasks:
- Sensor reading
- Calculations with information from the sensors
- Preparation of information that needs to be sent
Do you have an example in which in Core 0, you handle the WiFi connection and the rest is handled in Core 1 (or vice versa)? The sketch is booted for me when I try to separate the WiFi connection and put it in the slow core. I have tried putting it in core 0 and core 1.
Can you give us a solution that connects WiFi on a separate thread? It would be magnificent!!
I do not think that slow or fast is a good classification. I see all tasks as what they are: A task has prerequisites to run and leads to results. This is the first thing to design.
Second is to find out whether some of the tasks are so time-critical that they have to be able to interrupt others. These get a higher priority.
Third is "load balancing" where you distribute and assign tasks to cores. Often this is not needed and can be left to the RTOS.
With the ESP32 we have to consider that some tasks already exist (like Wi-Fi) and run on a core with a high priority. They can influence your decisions.
I would only go to step 2 and even 3 if you foresee or encounter problems.
So your "slow" tasks all are low priority tasks and "quick" tasks have a higher priority. Keep in mind that no task runs for long because it is always interrupted by RTOS.
Interesting as usual.
Thanks
Pity I don't have time now to test this out.
It takes some time to get acquainted... At least for me.
4:13 Odd to see so many processes with identical virtual and resident memory usages, totalling up to probably more than your machine can handle. Are they separate threads sharing the same memory context?
It is a Docker system, that is all I know...
At 7:57 there is a reference to ESP8266 crashes due to keeping the processor busy. I found this was discussed in a previous video #33: ua-cam.com/video/D_7ciW_TCac/v-deo.html
You are right. Thanks for the reference.
I am not a big fan of interrupts and when writing more involved programs I am always trying to keep the loop time a low as possible. I need to do this project and just play around with the few commands that you used. Thanks for trimming down a very large amount of info into something we can use.
I also did not need interrupts (with the exception of the PS/2 library. 1ms was fast enough.
"The '60's of the last century" - That hurt.
Why do you think?
The tuesday of last week was the last Tuesday ever in this mindset...until tomorrow
How about the fact that the 60s seems like it should be only 40 years ago rather than 60 and that I, born in 1982, am closer to middle age (45) than I am to my 20s (7 vs 8 years)?
Life goes on, Dude! Would the 50's of the last century hurt you more?!
I used RTOS on the ESP32. It is a nice clean approach. But it is not easier compared to a standard sketch. Just much cleaner for software architecture. I like it.
I agree!
Thanks for explaining FreeRTOS with well suited example as well.
One thing I want to bring to your attention that FreeRTOS is not designed to support multiple CPU cores. You when you said that scheduler is giving time to core0 for network related tasks that is not completely correct. Yes scheduler manages core0 tasks as well but that is separate scheduler dedicated for core0. It means there are two separate schedulers running for core0 and core1.
You are right. In addition, Espressif added a few "bells and whistles" for two cores which are not standard.
Thanks! Will certainly help to rethink my switch debouncing motor controlling code that crashes all the time. At 21:42, Would it be wrong to say that we use "RTOS" all the time, even for a "hello world". But the question is when to use more advanced features like multitasking.
You are right, the Arduino framework on the ESP32 always uses RTOS. But we are not aware of it if we just use setup and loop.
I think the best time to use multitasking is when your project is trying to do more than one thing at a time, and you can logically break it into parts. The code in each task becomes way simpler, and you can decouple one process from another process. The way Andreas was able to split his morse solution into simpler parts, is I think the best part of having a rtos.
Really well explained, thank you very much! Thumbs up!
Glad you enjoyed it!
Oh your info on this RTOS system reminds me of my college days back in the early 90's. Our computer labs were made up of 486 machines running DOS and WINDOWS 3.1 that nobody used much except to play with as a sort of DOS file system manager. Then we had Intro to UNIX! That was cool because using those old machines we could go into the network and sign onto the School's Mainframe machine in UNEX and do some real computing in an Operating system where all who were in the lab at one time 25 computers could sign into the Mainframe and tinker around with the Unix system. I still keep a Linux partition on my laptop and get into it when my anger at Windows exceeds my desire to get work done.
I still do not use Linux a lot. Only on my Raspberry.