Super nice tutorial, very insightful! Small remark, a convolution is actually a correlation with a reflected kernel, not a rotated one, i.e. convolution and correlation are related by a change of variables: corr(f, k) = \sum_i=1^N f(i)k(i) then conv(f, k) = \sum_j = 1 ^ N f(i)k(N - i). In the case of a 2x2 kernel this happens to coincide with a 180 deg rotation, but this is not true in general e.g. for a 3x3 kernel a flip and 180 deg rotation are not the same. See: en.wikipedia.org/wiki/Convolution#:~:text=Some%20features%20of,the%20convolution%20operator.
This is one of the best explanation of CNN on the internet for me, and that 3b1b video format is cherry on the cake. Please keep on making these videos.
Honestly, this is barely an explanation. He just showed you the steps to achieve CNN from scratch. He did not explain why we did some of the stuff we did, like the cross-correlation and stuff.
@@mysticlunala8020that's what other videos out there already do. The focus of this video was how to actually put all the concepts into code concisely and intuitively.
I know it’s a bit late, but I thought I should mention how well this video is paced and structured. The listing and crossing out of what topics are to be covered makes the video very clear, concise and easy to follow.
I can't thank you enough; you made everything super simple. Please keep making these videos if you can apply the same method to machine learning models. Thank you so much.
This is for real one of the best videos related to any type of NN I've ever seen. Most videos just scratch the surface of how these NNs work, but you went deeper and in an understandable way. Congratulations and keep the good work!
Cannot believe that tutorials like this exist. Thank you so much. I have been looking for a tutorial for a long time and I finally found it. This is definitely one of the best tutorials out there!
I am making a CNN from scratch and I was a little bit stuck on how to find the gradients of convolutional layers but that little digression about how the equation of a convolutional layer is really just a more general version of the equation of the dense layer output really made it clear for me! This video is gold
Thank you!! I'm making a machine learning library from scratch for fun and I've been confused with some details that thanks to you now I understand. It's my favorite explanation of CNNs on youtube
Thank you so much! I've been searching for this kind of explanation of CNN, especially the backprop process. I'll for sure cite this video in my thesis. Thank you!
Thank you so much oh my god, I've been working on fully implementing a U-Net from scratch and I've been going mad trying to find the equation you reveal at 23:28, since I didn't know how to handle the multiple matrices produced by the d amounts of convolutions (I was going to guess summation but guessing felt bad). And here is this beautiful video with less than 200k views, which not only is a very very solid explanation of convolutional layers to full depth, but also contains a little puzzle piece I cannot for the life of me find anywhere else. Thank you!
Finally, a tutorial where I got to know how the 3-channel RGB is being mapped mathematically into features. It is surprising to watch so many tutorials and none mentioned that for every channel there is a corresponding kernel and the summation of the convolutions was used to get the result for the next step. They all show h*w*3 and then a single 3*3 kernel. Example this video: C4W1L08 Simple Convolutional Network Example
You are a gifted teacher bro. I can't believe you've only got 50k views. But then again with how esoteric the content you're teaching is, it's impressive that your videos are so popular! Thank you
Man, I hope you channel become very huge. Thanks, this is the one of the best videos on youtube and not just about this topic, it is in general one of the best video in youtube
Amazing video, watched it multiple times. The only feedback I can give is, you can be a little more mindful of your words specifically, it felt like you were strongly implying the nature of CNNs to have a forward pass function that is more general than a FFNN which is an interpretation. Regardless, the best video I've seen on CNNs. Thank you so much
I've been reading about CNN and image recognition for a while to make my own one for my project idea, but never thought or found something that brought me light into how to implement a CNN, because I want to do it from scratch, with the maths and all staff. You have thought me a lot on 33min of video, now I know how I can make my own CNN, and also that I need to go over derivatives UwU Thanks a lot!!!
would be useful in future to create an outro where you give suggestions on how to implement other numbers or improve performance;like, look at this function instead of cross entropy... or smth like that; that would work like an exercise and engage us in excitement to see how you'll do it in future and differences between our codes
I have started my AI journey a month back and I have lots of confusion as how these CNN are getting parameters and how is it passing through layers and why reshaping and many more queries. I give full star to clear all the doubts on this video. This is saviour for me in my AI journey.
It is really help me to understand the whole concept of Convolutional Network. Especially the backpropagation. Please make some video on RNN, LSTN. Thank you.
Your "from scratch" videos are great! I was able to convert them into c++/cuda neural net classes and they work better than my old code. Thank you! Also, is there any way you can do one for an "Unconvolutional" layer? I would love to mess around with different types of autoencoders for images. :)
You could implement the hypermatrix operation you talked about in the beginning of the video in order to simplify the forward and backward functions of the Convolutional layer, removing all the loops
Everything well explained, thank you. I think all the code still can be simplified and made faster by using numpy ndarray functionalties. Instead of using all data sets in each epoch just use batches of data so that you can train all set without needing using only two class of data. Using numpy ndarray functions will remove almost all loops and hence your code will be faster.
Wow super impressive. Actually the only presentation on the internet of how actually CNN is built. What was your source of information? Some CNN science papers? Or any book?
Really a great video ❤️❤️ I also liked the neural network video very much bcz it cleared all my doubts and I was really amazed to see how we can easily implement a network by building each layer in the internet there are many implementations but this one is the best and easiest one and this CNN is the most amazing video bcz I haven't found any article where CNN is explained in such depth it was a great video🔥🔥🔥animations are also really helpful specially its 3b1b style is one thing i liked very much. I have a request that you should also make a video on RNN bcz after watching this video I think there are many deep understanding in RNN which I don:t know and also the implementation will be really helpful.
Highest quality I have ever seen. At 23:45, shouldn't the sum go from i =1 to i = d (not n ). Thank you for such a great video, I hope you find the time to continue explaining these kind of topics.
@@independentcode Sorry to contradict you but if you see the equation at 23:40, i goes from 1 to d (j is fixed, going from 1 to n input channels). Then in 23:45, i goes now from 1 to n, so Yi goes from Y1 to Yn, but there are not Yn elements. For each input channel, we need d terms in the sum because Xj contributes to all Ys.
@@chinin97 My bad I was actually looking at the wrong part of the video, few seconds before. You are correct, 23:45 should be i=0 to d. Thanks for pointing this out :)
Super nice tutorial, very insightful!
Small remark, a convolution is actually a correlation with a reflected kernel, not a rotated one, i.e. convolution and correlation are related by a change of variables: corr(f, k) = \sum_i=1^N f(i)k(i) then conv(f, k) = \sum_j = 1 ^ N f(i)k(N - i). In the case of a 2x2 kernel this happens to coincide with a 180 deg rotation, but this is not true in general e.g. for a 3x3 kernel a flip and 180 deg rotation are not the same. See: en.wikipedia.org/wiki/Convolution#:~:text=Some%20features%20of,the%20convolution%20operator.
Thanks a lot for pointing this out! I'm pinning your comment :)
This is one of the best explanation of CNN on the internet for me, and that 3b1b video format is cherry on the cake. Please keep on making these videos.
Yeah , for sure !
Honestly, this is barely an explanation. He just showed you the steps to achieve CNN from scratch. He did not explain why we did some of the stuff we did, like the cross-correlation and stuff.
@@mysticlunala8020that's what other videos out there already do. The focus of this video was how to actually put all the concepts into code concisely and intuitively.
I know it’s a bit late, but I thought I should mention how well this video is paced and structured. The listing and crossing out of what topics are to be covered makes the video very clear, concise and easy to follow.
Thank you :)
I'm an undergrad student studying CS at Georgia Tech. This video explained the backprop in CNN's better than my professors. A true gem.
This is THE BEST video where I completely learned about forward and backward props , in such an in depth manner! Thank you so much!!
After going through many blogs, this helped me just fully understand these networks. Such a great teacher you are!!!
Please produce more high quality videos like this. Your 30-minute video explains CNN better than my 1-semester AI class in college
I really love this explanation of CNNs. It almost makes them look easy
This is the best and calm explanation in NNs that I have ever seen on Internet! Amazing work, definitely sharing it to my colleagues
I can't thank you enough; you made everything super simple. Please keep making these videos if you can apply the same method to machine learning models. Thank you so much.
Best video on CNN. Period. Liked. Subscribed. Shared.
This is too much underrated.
Keep doing the good work. I really appreciate your contribution.
This is for real one of the best videos related to any type of NN I've ever seen. Most videos just scratch the surface of how these NNs work, but you went deeper and in an understandable way. Congratulations and keep the good work!
the "from scratch" series you made is pure gold!!
Cannot believe that tutorials like this exist. Thank you so much. I have been looking for a tutorial for a long time and I finally found it. This is definitely one of the best tutorials out there!
I am making a CNN from scratch and I was a little bit stuck on how to find the gradients of convolutional layers but that little digression about how the equation of a convolutional layer is really just a more general version of the equation of the dense layer output really made it clear for me! This video is gold
Thank you! Thank you! Thank you! Thank you!
Thank you! I appreciate the support :)
Thank you!! I'm making a machine learning library from scratch for fun and I've been confused with some details that thanks to you now I understand. It's my favorite explanation of CNNs on youtube
Thank you so much! I've been searching for this kind of explanation of CNN, especially the backprop process. I'll for sure cite this video in my thesis. Thank you!
True that. After reading so many blogs on Medium, none could solve all my doubts. You did it. Kudos to you.
Thank you so much oh my god, I've been working on fully implementing a U-Net from scratch and I've been going mad trying to find the equation you reveal at 23:28, since I didn't know how to handle the multiple matrices produced by the d amounts of convolutions (I was going to guess summation but guessing felt bad). And here is this beautiful video with less than 200k views, which not only is a very very solid explanation of convolutional layers to full depth, but also contains a little puzzle piece I cannot for the life of me find anywhere else. Thank you!
Thank you for the kind words, I'm glad the video was helpful!
I wouldn't have felt dumb, if i had watched this tutorial earlier
One of the best explanation so far for me.
One of the best tutorial so far, Great job
This video is amazing, It really helped me understand the math behind CNNs - thank you!
One of the best explanations for CNN on internet! Your channel would be Big in future.
This is an absolutely wonderful video you have here. Your explanations are clear and you helped out a graduate student a great bit!
Finally, a tutorial where I got to know how the 3-channel RGB is being mapped mathematically into features. It is surprising to watch so many tutorials and none mentioned that for every channel there is a corresponding kernel and the summation of the convolutions was used to get the result for the next step. They all show h*w*3 and then a single 3*3 kernel. Example this video: C4W1L08 Simple Convolutional Network Example
Great thanks!This is the clearest video about CNN's on the whole internet!😀
The best explanation of CNN I have ever seen. Thank you!
please please please keep making more videos. This is so insightful and relaxing to watch.
You are a gifted teacher bro. I can't believe you've only got 50k views. But then again with how esoteric the content you're teaching is, it's impressive that your videos are so popular! Thank you
this is definitely one of the better videos on the topic, surprised it doesn't have more views (:
Man, I hope you channel become very huge. Thanks, this is the one of the best videos on youtube and not just about this topic, it is in general one of the best video in youtube
Crazy video, i haven't seen anything better in this topic !
Amazing video, watched it multiple times. The only feedback I can give is, you can be a little more mindful of your words specifically, it felt like you were strongly implying the nature of CNNs to have a forward pass function that is more general than a FFNN which is an interpretation. Regardless, the best video I've seen on CNNs. Thank you so much
I dont know if its the music but this video is incredibly calming, thanks for this!
Came across this while trying to code Resnet in pure CUDA
The best explanation on the topic!
Great Thanks!!
One of the best tutorial I have gone through. Thank you so much.
I've been reading about CNN and image recognition for a while to make my own one for my project idea, but never thought or found something that brought me light into how to implement a CNN, because I want to do it from scratch, with the maths and all staff.
You have thought me a lot on 33min of video, now I know how I can make my own CNN, and also that I need to go over derivatives UwU
Thanks a lot!!!
Thank you for the kind message, I'm really glad if it helped :)
Thank you so much for those tutorials. They are really clear and well explained. Pls do that in every domain you know, even if its plumbery
Numpy needs to add this operation and give it a name for real 8:46
Edit: Ammmazing video btw!!
3b1b video format & amazing calming voice
OMG, you are a treasure
heavy logical equations like poetry, the only channel I activate the bell :D thanks for all.
Best ever video on CNN, hats off!
would be useful in future to create an outro where you give suggestions on how to implement other numbers or improve performance;like, look at this function instead of cross entropy... or smth like that; that would work like an exercise and engage us in excitement to see how you'll do it in future and differences between our codes
I have started my AI journey a month back and I have lots of confusion as how these CNN are getting parameters and how is it passing through layers and why reshaping and many more queries. I give full star to clear all the doubts on this video. This is saviour for me in my AI journey.
Finally understood backprop of conv. Thank you for the great video!
It's really the best explanation I have ever seen about convolution neural network
Thank you pro.
Man , I am mind blown by your content . Such a great video 👌
The best-ever tutorial. thank you.
You are one of the best had ever explained this topic. Keep up your easy and succinct style. thumbs up
Very much interesting!
Thank you for deep explanation ❤
Your lessons are works of art!!!
best video on Convolutional layer. Good job!
Thanks for making this. You're really good at doing what you do.
This in a very good tutorial to learn about CNN. Thank you so much.
Your animation reminded me of 3Blue1Brown videos. Awesome stuffs! :)
Thanks for sharing!
It's because I'm using his library :)
github.com/3b1b/manim
I had so many aha moments here! this is awesome
Thanks a lot for this video. Couldn't be more grateful!
GREATEST LECTURE EVER ON CORE DEEP LEARNING.... THANK YOU MATE
really found this video very interesting and informative . I really appreciate it a lot. Thanks!
Great video - first I saw that really shows how to thing about implementation
Best Explanation... lv ur teaching style and animations
Perfect Perfect i like this channel. Bravo, i found what i was looking for. Really thank you Sir.💚
awsome, thank you for the tutorial it really help me out to understand about cnn with easily
Simply beautiful videos, days of struggling for understanding backprop has just ended. Would you be willing to make a video on transformers?
This is amazing man. Very informative!
best CNN video on youtube by far!
Great sir !! i find about backward kernel so long time thank you
very high quality video and amazing explanation!
thank you very much for this masterclass
It is really help me to understand the whole concept of Convolutional Network. Especially the backpropagation. Please make some video on RNN, LSTN. Thank you.
I feel like I don't deserve to get such content for free.. Amazing job!
Very nice video! Just a question: are pooling layers used in your model?
No pooling layers, I only used what I showed in the video :)
Your "from scratch" videos are great! I was able to convert them into c++/cuda neural net classes and they work better than my old code. Thank you!
Also, is there any way you can do one for an "Unconvolutional" layer? I would love to mess around with different types of autoencoders for images. :)
Deconvolution is just a convolution, you just pick your convolved image, and use a bigger filter and add some padding.
im currently working on a c++/cuda implementation aswell. how did it go?
best video on cnn ever
Please make a video on Recurrent Neural Networks... This was sooo goooddd!
excellent video on CNN ever thanks buddy. Do more videos like this!!
Wait what?! You only have 4 videos? No no no no we need more "from scratch" videos
You could implement the hypermatrix operation you talked about in the beginning of the video in order to simplify the forward and backward functions of the Convolutional layer, removing all the loops
Exceptional explanation, thank you for sharing this.
Thank you for your time and effort, this is the best so far for me
Everything well explained, thank you. I think all the code still can be simplified and made faster by using numpy ndarray functionalties. Instead of using all data sets in each epoch just use batches of data so that you can train all set without needing using only two class of data. Using numpy ndarray functions will remove almost all loops and hence your code will be faster.
Your just hiding the loops in the function call
@@polyfoxgames9006 No.Not really. Most of numpy functions are optimized C functions.
@@polyfoxgames9006 Vectorized Python code can be as fast as C or Julia.
@@michaelpieters1844 no way haha. Pytorch is fast because it is cuda, being called by python
2:01 the formula is given to be y=i-k+1
as the stride is one but there is no existence of it in the example given
Thanks a lot!
Another piece of art! Thank you
Thank you so much of those two excellent videos !!!
Thanks, This vidio was perfect also had some beauty of mathematics
Thank you for this! great explaination.
I will request you to do "Attention" next if possible.
Wow super impressive. Actually the only presentation on the internet of how actually CNN is built.
What was your source of information? Some CNN science papers? Or any book?
This is extremely in-depth and just what I needed... All the best for future videos..
Just amazing work ❤
Thank you for explaining very clearly!
Incredible video!
Really a great video ❤️❤️ I also liked the neural network video very much bcz it cleared all my doubts and I was really amazed to see how we can easily implement a network by building each layer in the internet there are many implementations but this one is the best and easiest one and this CNN is the most amazing video bcz I haven't found any article where CNN is explained in such depth it was a great video🔥🔥🔥animations are also really helpful specially its 3b1b style is one thing i liked very much. I have a request that you should also make a video on RNN bcz after watching this video I think there are many deep understanding in RNN which I don:t know and also the implementation will be really helpful.
Highest quality I have ever seen. At 23:45, shouldn't the sum go from i =1 to i = d (not n ). Thank you for such a great video, I hope you find the time to continue explaining these kind of topics.
Thank you :)
Notice there is Yi in the sum, and Y goes from Y1 to Yd. There are d equations, with n terms in each.
@@independentcode Sorry to contradict you but if you see the equation at 23:40, i goes from 1 to d (j is fixed, going from 1 to n input channels). Then in 23:45, i goes now from 1 to n, so Yi goes from Y1 to Yn, but there are not Yn elements.
For each input channel, we need d terms in the sum because Xj contributes to all Ys.
@@chinin97 My bad I was actually looking at the wrong part of the video, few seconds before. You are correct, 23:45 should be i=0 to d. Thanks for pointing this out :)
@@omaraflak Perfect, no problem 😀. Thank you again for the quality of your explanation!
BEST video! Thanks a ton!
Wonderful explaination love from India🇮🇳
Wow your content is super awesome,
it would be super cool if you would also code RNN, LSTM, GRU and all that.. 🙂
your bias term depends on the input size, which would be nice to mention