Coding a Sound Driver on the ESP32 ULP
Вставка
- Опубліковано 25 вер 2024
- This video shows how to code a sound driver for the ULP on the ESP32 and overcome hardware limitations with some creativity.
Code:
github.com/bit...
Docs:
docs.espressif...
espressif.com/...
en.wikipedia.o...)
Hardware (affiliate links):
ESP32 MINI KIT (~$4.90) s.click.aliexpr...
TTGO ESP32 ($4.90+) s.click.aliexpr...
plz share :-)
Consider supporting my lab on Patreon for some extras and frequent updates:
/ bitluni
I'm also thankful for any donation on PayPal:
paypal.me/bitluni
Twitter: @bitluni
reddit: r/bitluni
#esp32 #ulp #sound #bitluni
Reminds me with the old days of software programming, especially for game programming on the pre-Pentium machines where you need to be a scientist in math, physics and computer hardware to get over the limitations of PC resources available at that time.
and all games were enjoyable back then .
Brilliant work and excellent presented. I really enjoy and learn a lot from your videos about pushing the limits of a Esp32.
Thanks!!
You're doing ESP like the demo scene used to do 80s home computers. I love it! Kinda get the feeling you'll have written your own operating system for the ESP by the time you're done though. :D
Mind blown again Bitluni... I can't keep up with your genius pulling the ESP32 down, to make it do what seems to be - anything!!
Your projects never cease to amaze me! Would you be able to use this in conjunction with vga output?
It will be adde dto the esp32 lib. but for vga there is still i2s0 free for that task
@@bitluni madness lol genius dude
Where did you get the audio clip from? The one that says "This is an emergency broad cast, nuclear missile inbound detected, seek shelter, duck and cover. Embrace for impact." ?
I'd like to get this audio clip too.
Very interesting use of the ULP.
Amazing work, I was working with this board to develop a BT a2DP source code, your is brilliant!
Your ESP32 work is brilliant and inspiring. Thanks for sharing with us!
Very nice work, getting the audio to work with these annoying limitations of the ESP32! The video is great too, you are improving with every upload. Keep it up! Your VGA board is now running happily here, the next version of the board will have an audio jack I assume?
Thanks! the version after has a jack. but I have to test if it works with smd caps properly. You could use the pin directly with a line input since most audio interfaces remove the dc offset anyways but I just destroyed one DAC while plugging the audio jack in w/o the cap. it seems to short for a moment. the dac is very sensitive
have come back to this one having learnt a bit more about the ULP. Still a great one.
For this, you could use the DMA to write your samples to the DAC instead. The DMA is actually very capable...
Assuming that the DMAs are needed for something else and must use the ULP; I think it is more efficient to "compile" the sample in the main CPU to the ULP machinecode, so that each byte to be written is the actual opcode to write that data out, so plus a call to a delay, and you should be able to get ~4k samples out of it, right?
Write the code where each instruction explicitly writes each sample to the DAC... one instruction for each sample. WOW! Would that actually work - not checked the instruction lengths for that. That's uber-mega-hacky, and I love it! But to be honest, you're completely right with the DMA. That's probably the best way to go.
I'm curios to know the advantage of this solution vs using DMA and route I2S to DAC. This other way saves CPU and frees the low power core
Another great project!! "Congratulations!" from India :) Now you have to consider remaining peripherals of the esp32 to achieve the best performance to price ratio!
Awesome stuff! I'd love to see people doing more with the BLE and possibly BLE mesh
I read the whole week about BLE.. :-D but classic is also very interesting
@@bitluni Do you think it's possible to make the ULP "listen" for specific BLE messages and wake the main CPU when something specific arrives? This would allow it to spare a LOT of battery!
I ordered and received a TTGo 2020 Watch which features an ESP32, however, the battery runtime is only about 8 hours with BLE polling done with the main processor. Bummer...
Amazing work! I love the fact you've used the ULP to offload processing. And with self-modifying code too! These truly are the oldest tricks in the book, and I'm delighted that you're keeping the concepts alive - because sometimes, they're necessary. While I see the good, I also believe the world has gone bonkers with virtual machines, runtime virtualisation, containerisation etc. VMs inside VMs inside VMs.... WTAF. While tech like Kubernetes / Google Cloud / Azure work for BIG industries, it's costly in human effort to get right, and I know of start-ups that burnt their money trying to "do it the right way" from the outset. I know for a fact that while the "latest" concepts / best-practices mean well, their implementations (or usage of) cause way more issues than they solve. Constraints breed creative and innovative solutions. The Commodore 64 had similar constraints, and timing CPU instructions was critical for stretching its hardware to its absolute limits. Those were the happy days!
Great stuff, I've been intending to move my DACAudio library over to the co-pro, this may be the kick I need to get moving on it.
I was thinking about your stuff while watching. Still intending to try it out some time.
@@WistrelChianti I'm re-writing as I2S and getting the co-pro to do the work. Working on it towards Christmas
This is amazing. I could have never created this code, but you've presented it in a way that I should be able to use it in projects. Well done Luni.
Hey bitluni, I love your work, and this ROCKS! Curious why not use an individual instruction for each possible value you can send to the DAC, only 256 needed, jump table takes 1k? Another 1k for code? Then use bottom 16 bits of the rest of the memory, 2 bytes, so 6k * 2 or 12k sample buffer.
This is genius,
I hadn't event considered that the ULP could be fast enough to work with I2S, I'm working on a project that could definitely use this, do you think the the ULP is powerful enough to do a volume level calculation of the i2s stream?
My goal/hope/dream is to be able to wake the main cores using the ULP processor whenever sounds above ambient are picked up with a I2S mic, so i can save as much power as possible.
This is a very fine work with esp32, a total inspiration to start to work with it
It would be interesting to also see the difference in power consumption :)
Can't you use the timer to trigger a DMA transfer from memory to the second DAC's register?
Thanks this is really helpful, i was looking for streaming audio via esp-32. mesh or over esp-now. This one solves one part of the puzzle.
This was an excellent explanation, and I appreciate the graphics!
perfect for filling the 7 minutes until makercast :)
ua-cam.com/video/igUd8rgy_0g/v-deo.html
Nice work and amusing video :). But why not just use the DMA engine to send I2S samples to the DAC?
Haha! I knew you would not give up that easily.
Impressive and cool work, I enjoy your projects.
I got a lot of ESP32 experience with our Arduino ECO Power board, mainly focusing on battery operation and LoRa networking. Maybe we can meet at Maker Faire Berlin next month.
Regards Helmut from Arduino Hannover
Wow.. Du bist echt ein Brain. Geile Arbeit. Weiter sooo
brilliant! you pushed the limit of this chip again!
2:10 that was so unexpected I spit out my drink xD
Always with great mood, thanks for sharing!
Great work! And I really enjoy your moving diagrams in this video (like the one at 1:00), what tools do you use to create them?
I like the idea of having the main ESP32 CPU cores just write the audio samples as a series of move-to-peripheral instructions; no lookup table needs because no look-ups :-)
I wonder if it's possible to build a little FORTH implementation for the ULP. That might be fun, and a more general way to be able to use it to do stuff..
Can you make a deep dive into esp32, because most videos i see they just consider it as an arduino with wifi/Bluetooth but following your projects it has more stuffs and different pitfalls
That was this most brilliant thing I’ve ever seen............
Brilliant work! Makes me wonder how to modify this for ws2812 LEDs...
you got a point! it was always a hassle with the rtos! I MUST try that!
@@bitluni If you do that I will upgrade my lights project to use your library, you have my word! As of right now I'm dealing with the jitter from handling interrupts from a web socket connection. I'm sure many other people who like to torture their esp's processor would appreciate a fix for the jitter!!!
I solved issues with the neopixels on the esp32 using i2s for the pov lightsaber. You should check out the code
This Hack is insane, totally love it!
Very smart usage of the esp. I noticed you need an upgrade to a 32 bits Bitluni's lab soon... Almost 16 bits of subscribers 😜
3:05 Ok, ok...
Google listener: Yes?
Me: Ghost?
O_O so this is what the next level looks like...
Did the designers have a particular use in mind for the coprocessor? It seems weak to me.
Mostly to be running some small operations during sleep of the main core
You could have done the self-modifying code thing if you were happy with 7-bit samples though, right?
You are an embedded systems wizard!
exciting! subscribed already!
Hello dear
I wanna to connect bluetooth audio module to esp32. Can you make library for that ? How much it will coast me?
Hi bitluni ,can you do a Bluetooth esp32 audio receiver outputting it as is 24 or 32 bit?
Hey, they say in datasheet, that there is 16k rtc memory. Is it not the same as ULP memory?
Loved the special effects.
Did you use Maya for that? 😏😀
Outstanding work!
Takes me back to getting a 640k DOS program written in x86 Assembly to work in just 64k of real memory by swapping out chunks of data in and out of the extended memory... Ah... those were the days.
I want to make a Theremin using two I2C ToF sensors and an ESP32 can the ULC be easily used as a wave generator with varying wave length and amplitude?
Cool video! Reminds me of writing self modifying for C64 raster interrupt code a long time ago.
Just bought a M5 Stack Core 2 with 2 sensors that seem fast, accurate and sensitive enough. Just worried if it is still ok when controlling playing audio.
crazy, this video made me forget to drink my morning coffee
Hello bitluni, great project nad congratz for solving the riddle! Do you think it would be possible to read adc and drive one pwm from ulp with main cores sleeping?
Thank you, for detailed explanation. Is it possible to take mic data via ADC and stream out using ULP ?
Should be possible, the ULP has full access to ADC.
In fact there is a dedicated command to get the current adc value.
Hey bitluni how do you know so many things ? Did you learn all this in college 👀
Just following my interests and gathering infos from the webz as I go
would you consider to code some ULP Fifo buffer as freelancer for us?
In what language is programming? Arduino IDE or mrcroPhyton? Also, what of this two you would recommend us to learn? Thnks and keep going on
GreatScott brought me here
How is the Synchronization done between the two processors?
Patrick Knöbel the youtuber Andreas Spiess has a good video about the ulp
The sample buffer is a ringbuffer. The pointer to the current sample is written back to the memory. The main cpu simply remembers the last pointer it has filled and continues to load samples until it reached the pointer of the ulp. This refill happens each few milisecond s. I'll add some comments to the code. The scetch isn't that long. RTC_SLOW_MEM is an array of 32bit words mapped to the rtc memory so the main cpu can easily read and modify it.
Nice video! Congratulations and thanks!
Super cool, well done!
Do you have a link to that sound byte?
you make me smarter and feel stupid at the same time.
"Oh cool ESP32..."
"Oh shit assembly, what is going on!?!"
what is that "emergency broadcast" sound bite u used?
This is brilliantly executed, but wouldn't it be a lot easier to run an I2S DAC off an available GPIO pin and feed its buffer from the loop? That works very well, even on a single-core ESP8266, and in stereo. I know that it works, because I made a raccoon proof cat feeder that plays the Darth Vader March (VERY LOUDLY) when it closes off the feeding dish, and plays ice cream truck jingle music when it opens. It was funny, at first, but after a few days it became very annoying. But it works: the ESP does almost nothing and the raccoons go hungry.
You should try hitting up SpriteTM on Twitter maybe he has some tricks to help. Great work on this it's incredible what you can do with these chips.
He has truly some better insights on the esp sitting at the source. I hope to meet him this year. maybe supercon
@@bitluni *waves* Not sure how many insights I personally have , but I can maybe poke the digital team every now and then. Also, if you're going to the Supercon, we should indeed meet up. Edit: Also, forgot to praise you: excellent work, both on the ULP sound driver as well as everything you manage to get out of the ESP32 in general!
Do you have issues with jitter?
Can you not shift the bits in the buffer right 16 bits to get the upper high? From the RTC memory to the register that is. So two sequences. One for the lower 16 bits, then shift, and process the upper 16 bits. If it is out of alignment for some reason you could just shuffle the way you load the data... Hmmm.... dunno.
Shifting is done by the ALU. The source and the target need to be registers. All registers are 16bits only. There is unfortunately no way to get the high 16 bits. Even the technical reference states that.
According to the doc, the write instruction also writes some stuff in the upper 16 bits of the target memory word (the current value of the program counter), as is shown in the video. Doc mentions this can be use by the application 'to know which instruction stored that piece of data here'. So perhaps there is also a way to read the upper 16 bits, but that does not look very practical here if we can not control what we write there :). Unless the main cpu can write the full 32 bits, and we only read ?
nice, I think it is great
keep sharing them =D
thanks for this
Brilliant!!
Masha'allah 🧠🤯
What abou the other DAC output??
I see that you have a new intro! ;)
Jokes aside, Amazing work!
Hi dude! Nice video. Could you please tell me a good board with esp32 using Arduino ide? Is Expressif board the best choice or Arduino board is better?
Hi. Do you also teach to build circuit
Nice, thanks for sharing😀👍
would this be useful for making a small sampler with sample playback? or is it better to use only the main cpu?
Once it's included in the sound engine it doesn't really. The dac i2s examples from the espressif docs are sufficient, too
Great Work! Send this immediately to marco reps, so he could forge a stepper-motor controller for us.
---
Schick das bitte sofort an Marco Reps, damit er uns daraus ne nen Stepper Controller bastelt. Super interessantes Video.
True, realtime on the esp is a rarity :-)
awesome
Brilliant ! I knew you could do it 👏👏👏👏👏👏👏👏 (8-BIT CLAP for you)
Tachchen, sag mal bist Du bei der Makerfaire in Berlin?
YES, I'm at Maker Faire in Berlin and even have a stand this time. You'll find me :-)
Woooow amazing!!!!
I love you bitluni
He's an awesome youtuber. I really enjoy his content, it's in depth and well explained. Am german too btw :)
Simply Genius!!!!
thx.
Started working with ULP assembly about a month ago, it’s pure horror
I saw there is a compiler etc. But I really enjoyed using it with the macros and absolute jumps. The instruction set is so basic. there is not much you can do wrong, except for the limitations
@@bitluni don't get me wrong, I enjoy it, but it takes a lot of blood, sweat and tears...
Best. Video. Ever! 😁
Nice job...can esp32 work as drum machine!!!!
probably. I might try implement a simple synth on it
And I love esp32
Cool
high level stuff
OMG this is extream!
You are super!
Seems nice! :3
what software do you use
You will be surprised: inkscape to create the graphics and magix video to move some parts
@@bitluni can you someday make a tutorial on how you make your videos basically an overview of "bitluni style video editing" . 😃
Simply genius. I think I've understood the concept, but could you tell my, how you came up with 0x1D4C0121 | (i
Black magic. I'm convinced Assembly is black magic.
Holy shit man
Genius!!
I like the way you use 'retarded'