amazing! as an AI researcher I missed these videos back in the days when I studied convolutions, hope they'll bring more understanding to the people just coming to the field!
An absolute pinnacle of online education materials in the field, when it comes to giving a real gut intution of what do operation look like 🙌 its a real talent you got there, thank you on behalf of the rest of the internet for using it well
I just wanted to say a huge THANK YOU for all the incredible animations you've been creating. Your work has been a game-changer for me, making complex concepts so much easier to understand.
Absolutely loved the way the instructor used animations to explain concepts like Groups, Depthwise, and Depthwise-Separable Convolution. It made understanding the topic so much easier and engaging. Keep up the great work!
Do remember in future vids to invite viewers to smash the like button, as it improves your ranking as per the Algorithm. I just realised I watched half a dozen of these without hitting it.
Please sir, also make visualizations like these on RNN, LSTM and most importantly Transformers. Would be really thankful to you. And also, your videos on CNN are just gems in the ocean of youtube.
Hi Animated AI, for clarification, are the stacks of cubes in the first 30 seconds of the video feature maps? Also, how exactly did the depth increase as we get into the deeper layers? Based on my understanding, the lecture you provided was focused more on maintaining the depth while increasing its efficiency. I hope to hear from you soon! Your work is great!
That's correct, they're the feature maps which are the inputs/outputs of the layers. The depth of a feature map is equal to the number of filters in the convolutional layer that created it. So the depth increases that you're seeing are simply layers that have more filters than the number of features in their input. Let me know if that isn't what you meant by your question. This video shows the depth staying the same in a depth-wise separable convolution, but you can still depth-wise separate a layer that increases the number of filters and get the performance benefits. You can just take the input depth and use twice (or some other multiple of) that value for the filter count in both the depth-wise and point-wise convolutions.
@@animatedai I see, I see. So if the input is an RGB image, and the first convolutional layer uses 5 filters, then the depth of the feature map will be 5. If that feature map goes to another convolutional layer with 5 filters, will the output contain a feature map with a depth of 25?
In that example, both outputs would have a depth of 5 because both layers have a filter count of 5. My video on filter count might help you visualize the relationship there: ua-cam.com/video/YSNLMNnlNw8/v-deo.html These videos are both part of this playlist on convolution: ua-cam.com/play/PLZDCDMGmelH-pHt-Ij0nImVrOmj8DYKbB.html
Soo the output has the same number of channels as the input? Or can you modify that by 1x1 convolution at the end ? Also doesn't this double the required storage for feature maps ?
In practice, it probably doesn't make a huge difference where you increase the number of channels. You could increase the channels in the depthwise convolution as long as you wanted the output channel count to be a multiple of the input. EfficientNet actually increases the number of channels with an extra pointwise convolution before the depthwise convolution. Yes, it increases the storage required during training in TensorFlow and PyTorch. Post-training, you don't necessarily need to keep around all the intermediate feature maps. So whether or not it doubles the required storage is dependent on the library (if any) that you're using for deployment.
You could theoretically separate any kernel into a depthwise-separated one. But you'd need a lot more filters in the depthwise convolution, so the result would be about the same performance. The performance improvement comes from training the network to take advantage of depthwise-separated convolutions.
These videos are excellent, but I suspect your ability to discern adjacent colors on a color wheel greatly outpaces mine. I have to pause and stare back and forth between blocks. It would be nice it were easier to see. Tools like Viz Palette can help pick better colors for data visualization.
I appreciate your feedback! I could rant for hours about how hard it is to pick colors :). I have two clarification questions: 1) Which part of the video did you pause to stare back and forth between blocks? 2) Which feature of Viz Palatte do you think would have helped pick colors for that instance?
Please don't stop making videos. They are of great help. Thank you for your efforts.
amazing! as an AI researcher I missed these videos back in the days when I studied convolutions, hope they'll bring more understanding to the people just coming to the field!
An absolute pinnacle of online education materials in the field, when it comes to giving a real gut intution of what do operation look like 🙌 its a real talent you got there, thank you on behalf of the rest of the internet for using it well
I just wanted to say a huge THANK YOU for all the incredible animations you've been creating. Your work has been a game-changer for me, making complex concepts so much easier to understand.
This is amazing! There is a lot of great material out there, and your channel is a really solid and valuable contribution to that. Thanks a batch!
This helped so much, you can't understand how thankful I am
Absolutely loved the way the instructor used animations to explain concepts like Groups, Depthwise, and Depthwise-Separable Convolution. It made understanding the topic so much easier and engaging. Keep up the great work!
By far the best explanation of depthwise-separable convolutions I found! This is a service, thanks!
One of the best channels on DL I’ve seen so far. Please publish more!
This was an incredible video. You can see the the amount of work and dedication; and you explain really good! Thanks, please keep on doing this videos
Great Work. I am a Master's student in ML and I your animations are really helpful in understanding this concept!! Thanks a lot.
Great stuff... the algorithm should give your content more attention!
This is so underestimated channel, will share it as much as I can. Thank you, Mr AI Animator!
Please continue make such amzing videos....they really helped me
This is great stuff. Please continue to make more, you are saving new scholar lives here!
The best conv layer visualization so far👍
Thank you for your great work💥💥
Your work is truly amazing, please keep enlighting us!
game changer for anyone learning neural networks
Thanks, the explanation of this mechanic is exceedingly lucid.
Hey, I want to thank you for spending time making great content animated! I’ve been using depth wise for a time and it has been always a little hazy
You are a freaking saint. I gotta sub for the effort you put in.
Incredible video! Brilliant visualisations and perfect explanation. Keep it up
You got a new subscriber. You are 3b1b of AI. Thanks for existing.
Thank you! Such a great video!
Incridibly helpful, keep up the good work!
My second favourites 3Brown1Blue channel
Do remember in future vids to invite viewers to smash the like button, as it improves your ranking as per the Algorithm. I just realised I watched half a dozen of these without hitting it.
absolutely love these videos! doing gods work
That was so intuitive. Thanks for that!
Amazing video, so well explained and to the point.
Very intuitive to understand, thank you.
thankyou very much brother this video means a lot for people like me 😍
Thanks!
Thank you for making videos like this.
Please sir, also make visualizations like these on RNN, LSTM and most importantly Transformers. Would be really thankful to you. And also, your videos on CNN are just gems in the ocean of youtube.
Congratulations for amazing class
Fantastic explanations, even though I understand the paper diagrams, this makes it super clearer. Would you cover cascaded/DenseNet someday?
this animation really helps me , thanks!
Finally understood. Thanks. Really helpful videos.
Another great video. Can't wait for you to go into animating Transformers!
underrated channel
This is so great!
Thank you very much for your sharing. It helps me a lot. I would really appreciate it if you could add subtitles
I want to know how can you make this video , what tools of software you used ?
Hi Animated AI, for clarification, are the stacks of cubes in the first 30 seconds of the video feature maps? Also, how exactly did the depth increase as we get into the deeper layers? Based on my understanding, the lecture you provided was focused more on maintaining the depth while increasing its efficiency. I hope to hear from you soon! Your work is great!
That's correct, they're the feature maps which are the inputs/outputs of the layers.
The depth of a feature map is equal to the number of filters in the convolutional layer that created it. So the depth increases that you're seeing are simply layers that have more filters than the number of features in their input. Let me know if that isn't what you meant by your question.
This video shows the depth staying the same in a depth-wise separable convolution, but you can still depth-wise separate a layer that increases the number of filters and get the performance benefits. You can just take the input depth and use twice (or some other multiple of) that value for the filter count in both the depth-wise and point-wise convolutions.
@@animatedai I see, I see. So if the input is an RGB image, and the first convolutional layer uses 5 filters, then the depth of the feature map will be 5. If that feature map goes to another convolutional layer with 5 filters, will the output contain a feature map with a depth of 25?
In that example, both outputs would have a depth of 5 because both layers have a filter count of 5. My video on filter count might help you visualize the relationship there: ua-cam.com/video/YSNLMNnlNw8/v-deo.html
These videos are both part of this playlist on convolution: ua-cam.com/play/PLZDCDMGmelH-pHt-Ij0nImVrOmj8DYKbB.html
@@animatedai Hi animatedAI! I'll check the link out. I hope I'll get it afterwards haha. Thanks for sharing!
Awesome job, I have a quesion out of the box, how you are did this work? which programs used in this video to produce it?
Thanks! Best explanation ever
Great video! Keep up the good work
תודה!
Great video... keep it goining..Thanks a lot
Truly awesome!
Thanks! Could you also cover convolutions with processing audio?
Great vidoe! Your website will be a very usefull ressource.
May I ask you what tool you are using for creating these animations?
I'm using Blender and relying heavily on the Geometry Nodes feature.
this is too cool to handle!!!!
From your example, it could be nice to give the number of computations as example of +/- 9x faster :)
so so good
Soo the output has the same number of channels as the input? Or can you modify that by 1x1 convolution at the end ? Also doesn't this double the required storage for feature maps ?
In practice, it probably doesn't make a huge difference where you increase the number of channels. You could increase the channels in the depthwise convolution as long as you wanted the output channel count to be a multiple of the input. EfficientNet actually increases the number of channels with an extra pointwise convolution before the depthwise convolution.
Yes, it increases the storage required during training in TensorFlow and PyTorch. Post-training, you don't necessarily need to keep around all the intermediate feature maps. So whether or not it doubles the required storage is dependent on the library (if any) that you're using for deployment.
great material!
Great work, thank you
amazing job !!!
It should be noted that this will not scale well with tensor cores and may even be slower.
Great work, thanks !
Maybe FPNs next time ? :-D
What are groups?
Fantastic!
awesome thanks!
Thank you
Nice job!
Excelent!!
great video
Thank you!
Can we convert trained standard Convnets to depth wise ones ?
You could theoretically separate any kernel into a depthwise-separated one. But you'd need a lot more filters in the depthwise convolution, so the result would be about the same performance. The performance improvement comes from training the network to take advantage of depthwise-separated convolutions.
These videos are excellent, but I suspect your ability to discern adjacent colors on a color wheel greatly outpaces mine. I have to pause and stare back and forth between blocks. It would be nice it were easier to see. Tools like Viz Palette can help pick better colors for data visualization.
I appreciate your feedback! I could rant for hours about how hard it is to pick colors :). I have two clarification questions: 1) Which part of the video did you pause to stare back and forth between blocks? 2) Which feature of Viz Palatte do you think would have helped pick colors for that instance?
thx!!
fye
ok visualize transforms next please, Vision Transforms would be nice.
please be more productive . Your videos are amazing
❤
hi
jiff
ok visualize transforms next please, Vision Transforms would be nice.
yes, a good visualization of transformers would be great
Also graphnet