You're doing an amazing job explaining everything in a clear way. Have you in mind to explain also good practice and design pattern regarding asyncio? Thank you for your efforts😀
David Beazley's ground up work on callbacks and coroutines were phenomenal. Your work on asyncio history concludes the whole story and connects all the dots. This is amazing and can't thank you enough for the gap you have filled. Perfect closure for me. BTW, I think personally dark theme (easy on eyes), professionally while presenting white background theme.
Dark theme is good. This series is incredible, other explanations refer to he event loop vaguely but seeing exactly what it does definitely grounds things.
Fantastic tutorial.Really appreciate your effort and grace in putting all this valuable and high quality content for free for everyone. I prefer the black theme
Watched on an iPad, the white background seemed clearer and easier on the eyes. Thank you so much for your videos. Fantastic work, enjoyed it tremendously.
00:00 Introduction. What this video is about. 03:29 Callback call diagram. 03:51 Examples of how the event loop works. 09:42 select 11:09 IOCP - Input/output completion port 12:24 Abstraction levels 15:41 unix_events.py kqueue, epoll, poll, select 18:08 How to define a selector yourself 20:47 Library code analysis 27:04 Analysis through set_debug 29:03 uvloop 32:55 Briefly about the above 34:03 About the next part of the video: Using Coroutines
Again: amazing, thank you so much! :) As a beginner in python, I am happy to found this video. AsyncIO is so great to use and now, I understand what is happening under the hood.
this was brilliantly articulated and explained... learnt so many OS related things and low level design patterns... thanks for sharing this knowledge...
What a high quality series! I have watched a lot of videos about asyncio and none of them discussed in such detail like you did. They just showed how to use the APIs... Will you publish new series about other topics? I'm looking forward to it!
This is marvelous. It explains things in a clear and precise way. It should be included in the official asyncio documentation. Thanks a lot! -- Dark theme, please.
Black background for the win. I think you'll be covering how asyncio web servers work, I would be interested in that. Also some insight on how the asynchronous interaction with the database is handled would be very nice. Thank you so much for the precious work!
Thank you again Lukasz. To answer your questions: 1. probably Dark but I don't really mind. Seems like most people prefer dark which is good option. Perhaps it's related to what they are used to programming with day to day. 2. I also want to learn how to use this within a python webserver which is on your list. Specifically around websockets. I'd like to know how to read from some external sources and pipe them to clients connected to my own server.
These series are priceless, thank you so much to putting this out!. Dark theme for the IDE looks a bit better, but no harm on the white one thou. By the way, does asyncio supports io_uring on linux (I've read is something kind of analogous to iocp, not sure if that's true)?
I have a preference for dark theme. It would be good to go into further details about the differences between asyncio and curio/trio, and to talk about their influence on asyncio.
This is awesome! Thanks for creating this video series/course! I giggled with the "some random dude use it in his video", yeah, sure, "some random dude"... 😂 I like the dark color scheme better 😎 I have a couple of requests 😬: It's probably already planned, but I would like to know what would be the best approach to handle CPU-bound code that doesn't really block on I/O but hogs the event loop, like Numpy and Cryptography stuff. run_in_executor? Are there some guidelines to decide when to specifically send stuff to run_in_executor even if it's not blocking "waiting" but just fully using the CPU? Also, when dealing with code that uses extensions or Cython, if it uses threads, I'm not really sure if that code has access to the same thread locals from threading.local, would it be dangerous to use it and share objects with different functions passed to run_in_executor for the same task/context? Or when dealing with compiled code is there some additional level of abstractions to handle that that doesn't interfere with thread locals? I'm not even sure I'm asking the question correctly, but any insight you could share about that and how to interact with contextvars and compiled/Cython code would be much appreciated!
Thank you for your work! You were able to keep great quality :) Thou I use light color schema for everyday work I think dark one is better for YT watching (especially at big TV screen).
I was trying call_later and even in your video I noticed that it is not delaying for the specified number of seconds. It actually waits for lesser seconds than specified. Why is that?
For batteries included module, would love to see hundreds of async calls to databases or webservers... Thanks for your help and work in explaining this ever evolving topic in Python!
Thank you for this amazing series! Can’t wait for the next episodes. Please, use light theme - it’s easier on eyes when watching in a well-lit environment
Thanks for sharing your really deep knowledge. As an real-life example application I would like to see the application like GoogleAnalytics. It should gather data from millions of sites (or at least thousands) simultaneously (clicks, events, mouse moves etc). And to be even better than GA. GA displays all the statistics with an about 24-hour delay. I want to be able to see that information in live mode. Theme - not important. This is strange that while coding I usually use the white one, but I really like your black :) By the way - what code editor do you use for everyday codding?
06:08 - I checked the docs for "loop.call_later" - why did the loop only run for 4 seconds and not 8 - as you've specified as parameter to the loop.call_later" call? I noticed the loop terminates at around 06:12 and thought that maybe you did something at 06:12 - 00:08 = 06:04 and you did... namely the "loop.call_later" call... but that call is not supposed to start a new loop, right? Is the call at 06:04 related to why the loop does not last for 8 seconds as you've specified?
how can one see what is scheduled to run on an event loop? I did "dir(asyncio.get_event_loop())" but I cannot see anything useful... is it a static method, maybe?
Wow, I've been working with Python for almost 4 years and had no idea that 'underscore variable' is actually storing the latest created object or something like this :D
hello, thank you for this video. I learned a lot, and will re watch your videos several times. I subscribed to EdgeDB bc of your content. 1. dark theme is easier on the eyes. could you just make parentheses and dots etc more visible? 2. everything you did here with the “event loop”, you can just use a while loop. What’s the difference between event loop and a regular (while) loop? In other words, to do what you showed us, I can just use something like: while (until 20 seconds from now) First() Second() Third() Hog() and the result is the same. Both “event loop” and “regular/nonevent loop” blocks while Hog() takes it time. What different thing will learning asyncio allow me to do? 3. why is the trampoline in loop.call_soon(trampoline) called a “callback”? It (at least the first time it’s called) is not being called after anything. It’s just being called like any regular function. 4. conceptually, is loop.run_until_complete(task1) different from loop_call_soon(task1) different? is task 1 a “callback” in both situations? Is task1 processed any differently? Thank you again and I am waiting for your next videos.
If asyncio fits the functional programming paradigm (no advanced stuff like functors, monads etc) it would be nice to see some basic examples like function composition, iterators etc
9:07 "we avoid doing very long operations at one given moment". "making sure you are doing our work in the smallest chunk possible". Is there no simple way to force big chunks of work to pause at regular intervals ? I feel awkward having to manually divide big chunks in smaller chunks, particularly if the smaller chunks are hidden under several layers of function calls.
That is great quality content. Thank you! I'm fine with Dark theme. I'm a bit confused about the various keywords you have in your bpython like print_now() and range(10_000).
print_now() is a function we defined ourselves: it just prints the current date and time. 10_000 is a way to write the number 10000 in Python 3. With the underscore it's way faster to read what value our number has.
In some Python environments, such as the standard Python interpreter or Jupyter notebooks, the underscore _ is automatically assigned to the result of the last executed statement. This allows for easy access to the result of the previous operation. However, in other environments, like certain Python shells or IDEs, the underscore _ might not be automatically assigned in the same way. This could explain why you're encountering a NameError in the Python interpreter. It's worth noting that the behavior of the underscore _ can be influenced by various factors, including the version of Python you're using and any customizations or settings in your environment. Therefore, it's not guaranteed to behave consistently across all Python environments.
Amazing job. Best part of the video is seeing your enthusiasm! How is it that the event loop doesn't run "hot" while inside a `while True` loop? github.com/python/cpython/blob/4454057269b995341b04d13f0bf97f96080f27d0/Lib/asyncio/base_events.py#L595-L598 My guess is it's the timeout value inside _run_once, which is then passed into the selector or maybe it's also handled by the selector itself? I guess if you could talk about that a little more it might clear things up for me? As for what could be discussed in the batteries included episode, I'm sure you'd likely get to it, but I think discussing how running synchronous code inside the run_in_executor works, and what, if any, practical limits you get by doing this (for instance, if you have too many of those calls, do you end up back with all downsides of threading or if by doing them inside an event loop it's somehow not as bad?) Also, and now this is getting pretty specific, but if i had a subprocess call (let's say zip a file), and I wanted to stream (PIPE) the standard output of that subprocess back to a user, can you do this using asyncio? Might be a fun example to show? As for theme - very slight preference for dark theme.
Thanks alot for the video. I love this so much. I think either backgrounds works for me. I do prefer the dark one. Personally. And regarding the batteries included. I would like to see how would a worker, a loop maybe could work along side a web app that can consume events from a stream of events asynchronously. Something like Kafka or Redis. I have done this using the help of a library called Faust. But it would be nice to see how would that work along side a webapp. Thank you so much for the videos again.
Dark theme for the IDE is fine.
Thanks for this amazing series!
I prefer to see a dark theme in this series. :)
The light one is too light.
You're doing an amazing job explaining everything in a clear way. Have you in mind to explain also good practice and design pattern regarding asyncio? Thank you for your efforts😀
Thank you! We'll make sure to cover some good practices and design patterns in Episode 6 while developing the example Web application.
David Beazley's ground up work on callbacks and coroutines were phenomenal. Your work on asyncio history concludes the whole story and connects all the dots. This is amazing and can't thank you enough for the gap you have filled. Perfect closure for me.
BTW, I think personally dark theme (easy on eyes), professionally while presenting white background theme.
Dark theme is good. This series is incredible, other explanations refer to he event loop vaguely but seeing exactly what it does definitely grounds things.
Fantastic tutorial.Really appreciate your effort and grace in putting all this valuable and high quality content for free for everyone. I prefer the black theme
it's rare to find videos like this, detailed and deep. amazing work, thank you
Watched on an iPad, the white background seemed clearer and easier on the eyes. Thank you so much for your videos. Fantastic work, enjoyed it tremendously.
00:00 Introduction. What this video is about.
03:29 Callback call diagram.
03:51 Examples of how the event loop works.
09:42 select
11:09 IOCP - Input/output completion port
12:24 Abstraction levels
15:41 unix_events.py kqueue, epoll, poll, select
18:08 How to define a selector yourself
20:47 Library code analysis
27:04 Analysis through set_debug
29:03 uvloop
32:55 Briefly about the above
34:03 About the next part of the video: Using Coroutines
event_list = self._selector.select(timeout)
Let's take a short moment to appreciate this wonder ✨🙏🏽🙏🏽
Amazing. Best description of how the internals of asyncio works. Thanks
Excellent videos. Thank you for making these so clear and not making huge assumptions. Dark theme.
Wow! Amazed to have found this Thank you so much! Also, Dark! I am hooked.
Again: amazing, thank you so much! :)
As a beginner in python, I am happy to found this video. AsyncIO is so great to use and now, I understand what is happening under the hood.
this was brilliantly articulated and explained... learnt so many OS related things and low level design patterns... thanks for sharing this knowledge...
Amazing! thank you so much for this :) I like the dark theme more.
Thank you so much for this video series. I am okay with the dark background. Looking forward to Co-routines.
Informative series! I prefer the dark theme. It's easy on the eyes when watching on my phone or 💻
What a high quality series!
I have watched a lot of videos about asyncio and none of them discussed in such detail like you did.
They just showed how to use the APIs...
Will you publish new series about other topics?
I'm looking forward to it!
Thanks for the high-quality content. I also prefer to see the codes in a dark theme.
Great talk, I'm waiting for next episodes! Thanks! I would like to know what is best way to handle byte streams from sockets using aysncio.
Thank you very much for the series, Lukasz! One qeustion: what's with the underscore assignment at 04:00?
This is marvelous. It explains things in a clear and precise way. It should be included in the official asyncio documentation. Thanks a lot! -- Dark theme, please.
thx for this, mate !
btw. vote 4 dark theme
still the best video series explaining asyncio. excellent job well done Łukasz.
Thanks for this videos, they are very useful. I prefer the dark theme :)
Your series mets more than expected. Thank you
Black background for the win. I think you'll be covering how asyncio web servers work, I would be interested in that. Also some insight on how the asynchronous interaction with the database is handled would be very nice. Thank you so much for the precious work!
Best asyncio series in the Internet. And the Internet is a big place.
Thanks so much for your effort! This is by far the best material on this topic.
Good video.
I point out at 3:18, you write UNIXes; the plural form of Unix is Unices.
Thank you again Lukasz. To answer your questions: 1. probably Dark but I don't really mind. Seems like most people prefer dark which is good option. Perhaps it's related to what they are used to programming with day to day. 2. I also want to learn how to use this within a python webserver which is on your list. Specifically around websockets. I'd like to know how to read from some external sources and pipe them to clients connected to my own server.
These series are priceless, thank you so much to putting this out!. Dark theme for the IDE looks a bit better, but no harm on the white one thou.
By the way, does asyncio supports io_uring on linux (I've read is something kind of analogous to iocp, not sure if that's true)?
I have a preference for dark theme. It would be good to go into further details about the differences between asyncio and curio/trio, and to talk about their influence on asyncio.
Incredible stuff, I am so glad I found your channel. Going to watch all of these for sure.
+1 for white theme. Thanks for this series!
Amazing content !! Thank you for such deep coverage of the topic !
This is awesome! Thanks for creating this video series/course!
I giggled with the "some random dude use it in his video", yeah, sure, "some random dude"... 😂
I like the dark color scheme better 😎
I have a couple of requests 😬:
It's probably already planned, but I would like to know what would be the best approach to handle CPU-bound code that doesn't really block on I/O but hogs the event loop, like Numpy and Cryptography stuff. run_in_executor? Are there some guidelines to decide when to specifically send stuff to run_in_executor even if it's not blocking "waiting" but just fully using the CPU?
Also, when dealing with code that uses extensions or Cython, if it uses threads, I'm not really sure if that code has access to the same thread locals from threading.local, would it be dangerous to use it and share objects with different functions passed to run_in_executor for the same task/context? Or when dealing with compiled code is there some additional level of abstractions to handle that that doesn't interfere with thread locals? I'm not even sure I'm asking the question correctly, but any insight you could share about that and how to interact with contextvars and compiled/Cython code would be much appreciated!
Yo..big fan!!
Thank you for your work! You were able to keep great quality :)
Thou I use light color schema for everyday work I think dark one is better for YT watching (especially at big TV screen).
I was trying call_later and even in your video I noticed that it is not delaying for the specified number of seconds. It actually waits for lesser seconds than specified. Why is that?
did you manage to find an answer to that?
For batteries included module, would love to see hundreds of async calls to databases or webservers... Thanks for your help and work in explaining this ever evolving topic in Python!
Awesome series, 1st time understood how eventloop, multiplexer, selector, etc work under the hood in python...
Thank you very much for this. Looking forward to the next tutorial. Also, I'm OK with the background
This is an amazing explanation about event loop. Thank you so much!
I think that dark theme is better.
This is great, super informative. Love the dark theme. Thank you for making this series.
why is audience small? This is the best series of explaining ayncIO
Thank you for this amazing series! Can’t wait for the next episodes.
Please, use light theme - it’s easier on eyes when watching in a well-lit environment
well most programmers hides in not-well-lit env
Jesteś absolutnie świetnym nauczycielem :D
This is a great series so far. I vote for dark color scheme.
Thanks for sharing your really deep knowledge.
As an real-life example application I would like to see the application like GoogleAnalytics. It should gather data from millions of sites (or at least thousands) simultaneously (clicks, events, mouse moves etc). And to be even better than GA. GA displays all the statistics with an about 24-hour delay. I want to be able to see that information in live mode.
Theme - not important. This is strange that while coding I usually use the white one, but I really like your black :)
By the way - what code editor do you use for everyday codding?
bravo, excellent job!!! exactly what I was looking for
This is an *amazing* series
06:08 - I checked the docs for "loop.call_later" - why did the loop only run for 4 seconds and not 8 - as you've specified as parameter to the loop.call_later" call? I noticed the loop terminates at around 06:12 and thought that maybe you did something at 06:12 - 00:08 = 06:04 and you did... namely the "loop.call_later" call... but that call is not supposed to start a new loop, right? Is the call at 06:04 related to why the loop does not last for 8 seconds as you've specified?
how can one see what is scheduled to run on an event loop? I did "dir(asyncio.get_event_loop())" but I cannot see anything useful... is it a static method, maybe?
best resource on the asyncio, thank you
Congratulations for the amazing content! BTW, I would prefer the dark theme
what causes this on the Spyder? RuntimeError: asyncio.run() cannot be called from a running event loop
Your explication is fantastic. Good Job.
Best tutorial playlist!
Excellent video, thank you!
Wow, I've been working with Python for almost 4 years and had no idea that 'underscore variable' is actually storing the latest created object or something like this :D
great detailed explanation
Thanks so much make me more clearly from ground-up and why.
Great explanation! 👏
Amazing, thank you so much for this
Thanks for the video. Really great. I prefer the dark theme.
hello, thank you for this video. I learned a lot, and will re watch your videos several times. I subscribed to EdgeDB bc of your content.
1. dark theme is easier on the eyes. could you just make parentheses and dots etc more visible?
2. everything you did here with the “event loop”, you can just use a while loop. What’s the difference between event loop and a regular (while) loop? In other words, to do what you showed us, I can just use something like:
while (until 20 seconds from now)
First()
Second()
Third()
Hog()
and the result is the same. Both “event loop” and “regular/nonevent loop” blocks while Hog() takes it time. What different thing will learning asyncio allow me to do?
3. why is the trampoline in loop.call_soon(trampoline) called a “callback”? It (at least the first time it’s called) is not being called after anything. It’s just being called like any regular function.
4. conceptually, is loop.run_until_complete(task1) different from loop_call_soon(task1) different? is task 1 a “callback” in both situations? Is task1 processed any differently?
Thank you again and I am waiting for your next videos.
excellent video, I can't wait for the next.
How can I send you what I'm working on to see how to apply it. It is about algorithmic trading
You really do something great, Thanks for your video. BTW, I like black theme.
If asyncio fits the functional programming paradigm (no advanced stuff like functors, monads etc) it would be nice to see some basic examples like function composition, iterators etc
Of course thank you a lot for this amazing material
We'll be getting to that as well, good point!
9:07 "we avoid doing very long operations at one given moment". "making sure you are doing our work in the smallest chunk possible". Is there no simple way to force big chunks of work to pause at regular intervals ? I feel awkward having to manually divide big chunks in smaller chunks, particularly if the smaller chunks are hidden under several layers of function calls.
i dont get how you were calling the async functions without declaring async first in the examples ? (for example trampoline).
excellent video, is there git examples for this video?
I prefer to see a dark theme in this series
I took a small moment to appreciate it. :D
💯AsyncIO from scratch better explained ✌🏾🙏🏾
background theme: ⚫️
8:16 Why after second 11 the hog function kicked in when it was scheduled to start after 15 secs?
What kind of app are you using at 19:00?
i fucking love this series already, sorry for the language but I really needed to emphasise my feeling ;)
+1 for white theme
and thanks for your work!
Thank you for your beautiful video
Thanks for that, I'm waiting for the next Video.
Please use the Dark Theme background.
This video is very helpful!
Thanks so much. Black background works for me, I'm on mac & use youtube dark mode.
That is great quality content. Thank you! I'm fine with Dark theme. I'm a bit confused about the various keywords you have in your bpython like print_now() and range(10_000).
print_now() is a function we defined ourselves: it just prints the current date and time. 10_000 is a way to write the number 10000 in Python 3. With the underscore it's way faster to read what value our number has.
Why is it that only the python IDLE accepts loop = _ while the python interpreter returns a NameError: name '_' is not defined error?
In some Python environments, such as the standard Python interpreter or Jupyter notebooks, the underscore _ is automatically assigned to the result of the last executed statement. This allows for easy access to the result of the previous operation.
However, in other environments, like certain Python shells or IDEs, the underscore _ might not be automatically assigned in the same way. This could explain why you're encountering a NameError in the Python interpreter.
It's worth noting that the behavior of the underscore _ can be influenced by various factors, including the version of Python you're using and any customizations or settings in your environment. Therefore, it's not guaranteed to behave consistently across all Python environments.
which code editor is he using?
bpython, running in a terminal
Amazing job. Best part of the video is seeing your enthusiasm!
How is it that the event loop doesn't run "hot" while inside a `while True` loop? github.com/python/cpython/blob/4454057269b995341b04d13f0bf97f96080f27d0/Lib/asyncio/base_events.py#L595-L598 My guess is it's the timeout value inside _run_once, which is then passed into the selector or maybe it's also handled by the selector itself? I guess if you could talk about that a little more it might clear things up for me?
As for what could be discussed in the batteries included episode, I'm sure you'd likely get to it, but I think discussing how running synchronous code inside the run_in_executor works, and what, if any, practical limits you get by doing this (for instance, if you have too many of those calls, do you end up back with all downsides of threading or if by doing them inside an event loop it's somehow not as bad?)
Also, and now this is getting pretty specific, but if i had a subprocess call (let's say zip a file), and I wanted to stream (PIPE) the standard output of that subprocess back to a user, can you do this using asyncio? Might be a fun example to show?
As for theme - very slight preference for dark theme.
Does pygame event loop uses asyncio??
Which Color theme is this ?
Thanks alot for the video. I love this so much.
I think either backgrounds works for me. I do prefer the dark one. Personally.
And regarding the batteries included. I would like to see how would a worker, a loop maybe could work along side a web app that can consume events from a stream of events asynchronously. Something like Kafka or Redis.
I have done this using the help of a library called Faust.
But it would be nice to see how would that work along side a webapp.
Thank you so much for the videos again.
I like dark background (My girl is making me say this. SH(W)E likes dark background. I actually like the light one more)
Thanks for this amazing series. I prefer to see a white theme.
My favourite moment of the whole series: ua-cam.com/video/E7Yn5biBZ58/v-deo.html 😀
You rock Łukasz!
brilliant!
Crystal clear explanations. Light theme is better !
I would like to see how edge db leverages async for some crazy task😅
Thank you very much!
Why are only 3 more videos left 😭😭😭
this is great