Alfredo made an absolutely excellent explanation. There is also the paper "A Gentle Tutorial of Recurrent Neural Network with Error Backpropagation. Gang Chen" maybe it will be useful to someone for a better understanding of the BPTT.
Grazie Mille Prof Canziani! I think that when one loads zero to the abstract concept state vector, its not only loading an arbitrary value to initialize an allocated memory, but defining zero as the absense of information, so the distance from the origin somehow represent the level of information you have. Just thinking a little about this class part! Thank you!
Are the answers to the homeworks available anywhere? I'm having significant trouble getting my alphabetic signal echo RNN to perform well without significant overfitting
In your animation (it's awesome btw!) starting at 6:35 how do you get a smooth continuous morphing of the space? Since you have 5 hidden layers shouldn't the animation just be 5 discrete images of the space as it appears after that layer? Edit: wait I think you literally answer that right after the animation... asked too soon
@@alfcnz I meant like when we use a 1 hidden layer neural net the output neuron is just taking the nonlinear function learned by neurons of 1st hidden layer and do the weighted sum of those functions to approximate a nonlinear function ,that's what i meant by combination of function
Hi Alfredo, Your videos are really wonderful and I want to thank you for uploading such quality material. However, I am unable to understand the data flow in RNN networks. For instance, I have a sentence which is 4 words long. If I am using a Fully Connected Network of 12 neurons, each of my 4 words will be fed into the 12 neurons. I want to understand what will be the data flow if I use a LSTM with 12 hidden states. Thanks.
Alf, Thank you for making all these interesting lectures and materials available. I'm just loving this. BTW, you keep mentioning Yann's lectures. Can I view those too? Were they recorded?
@@alfcnz Thank you, Alf. I don't know why I didn't see all the other lectures. I'm glad I asked. Even though I'm getting the material from your lectures, I'm a firm believer in reinforcing work through repetition. Cheers!
These videos keep getting better! Excellent explanations and animations! Btw, are you familiar with the book 'Mathematics for Machine Learning' by Deisenroth et. al.? I'm really enjoying reading it. Might be a good additional resource for your students.
@@alfcnz Zero shot anticipation for instructional activities. But I want to create a GAN which will take in a recipe text and create a video for it. Using Tasty dataset
hey Alf, how is it going? I have a question from this week's homework that I worked super hard to answer (I spent about a month trying to answer the question) but couldn't find the answer, so I said maybe I can ask you. however, I hate wasting someone important's time, like your time so if there's any place that I can ask these questions so that I don't take your valuable time, please guide me. that being said, let me state my question. In the theory part of HW, Question 1.1, part C, subpart (ii), you asked about the derivative of F with respect to weight matrix W. first of all I worked out the dimension of the F function to be 1 * 1 * 2. There my problem begins, I know how to differentiate a vector with respect to a matrix but those two extra dimensions (first and second dimensions that are both 1) are giving me a hard time in figuring out the dimension of the derivative. I know that the dimension of the derivative should be the number of elements in the output, times the weight matrix dimensions transposed, and I worked it out to be 1 * 1 * 2 * 1 * 3 * 2, but it really seems wrong to me. am I missing something?
@@alfcnz Particularly ones that are similar to 3Blue1Brown, and ones where you can animate what happens with data structures. (isn't matplotlib used for python).
Nicely explained ... thanks a lot for uploading these to UA-cam. Quick questions though ... any idea when practica #1 will be available and Lecture #4 is missing from the 2020 playlist (ua-cam.com/play/PLLHTzKZzVU9eaEyErdV26ikyolxOsz6mq.html)?
There was no lecture 4 in 2020 (it was President Day). Practica 1 is basically me going over the syllabus and Yann giving an intro on the history. Not sure if it's worth adding it.
@@alfcnz Yep. You sometime reply to the chat and it breaks the flow a bit since we don't see it. That being said, again, it's nothing major. This is amazing content and it's free so I'm not complaining (: Just wanted to suggest a possible improvement.
@@paolomontesel these are my live lessons, where my NYU students ask anything during class. I read their questions out loud if they are relevant. So, I'm not 100% sure I understand why I should have the text on screen. Can you expand a little? I'm really interested in understanding your opinion.
Nice lecture and impressive animations, mate. Your dedication to teaching is remarkable. Well done!
Yay! 🥳🥳🥳
I just wanted to thank you because I really struggled before to learn these concepts but when you say it, it seems easy although it isn't. thanks
🙂🙂🙂
Wow! High quality content and super entertaining to watch!
I'm glad you found it enjoyable. 😊
Love the animations at 7:20 professor! Thank you for the materials.
🤓🤓🤓
The production quality is amazing
I've put some hours of work towards a few new skills 😇😇😇
Very impressive animations, Alfredo! I think that this is the best explanation of the ANN which i have seen. Thank you for your work
❤️❤️❤️
best lecture with visualization ive found on youtube. Thanks a lot!
You're very welcome! 😊
Thank you for your hard work in editing. recording, and the effort for making a new effect🎇✨.
Hehe, I'm having quite some fun! Also, working on brand new lessons as well.
these animations in the video are just something amazing for understanding what's going on
P.S. I've made 10000 woooows during watching
Yay! 🥳🥳🥳
Alfredo made an absolutely excellent explanation. There is also the paper "A Gentle Tutorial of Recurrent Neural Network with Error
Backpropagation. Gang Chen" maybe it will be useful to someone for a better understanding of the BPTT.
I'll look it up, thanks!
Grazie Mille Prof Canziani! I think that when one loads zero to the abstract concept state vector, its not only loading an arbitrary value to initialize an allocated memory, but defining zero as the absense of information, so the distance from the origin somehow represent the level of information you have. Just thinking a little about this class part! Thank you!
You're welcome. You need to add a timestamp if you refer to any part of the video.
@@alfcnz 50:20
Not necessarily. There's a bias there as well, which can counteract any kind of initial value you pick (as long as it's constant).
Amazing content.
🥳🥳🥳
Are the answers to the homeworks available anywhere? I'm having significant trouble getting my alphabetic signal echo RNN to perform well without significant overfitting
No answers available (we may have similar exercises during future semesters). Feel free to ask in the Discord channel.
@@alfcnz oh I forgot there was a discord channel! That's a great idea, thanks! And thanks for uploading the course on YT!
In your animation (it's awesome btw!) starting at 6:35 how do you get a smooth continuous morphing of the space? Since you have 5 hidden layers shouldn't the animation just be 5 discrete images of the space as it appears after that layer?
Edit: wait I think you literally answer that right after the animation... asked too soon
Yup. Also, in the next lesson I explain that further.
amazing class!
I'm glad you're enjoying it!
Why deep neural nets require less number of neurons than shallower ones to approximate a function in terms of function combination and composition?
You can consider a 1,000 neurons single layer to be roughly equivalent to 3 10 neurons layers stacked on top of each other.
@@alfcnz But why?and also can we write a compostion of functions as some combination of functions
@@Anujkumar-my1wi what does combination of function mean?
@@alfcnz I meant like when we use a 1 hidden layer neural net the output neuron is just taking the nonlinear function learned by neurons of 1st hidden layer and do the weighted sum of those functions to approximate a nonlinear function ,that's what i meant by combination of function
That's called composition.
So, your question asks «how to write a composition of functions as composition of functions?» which does not make sense.
You're a legend!
Aww 🥰
It is so worthy to watch 05:05 one thousand times... so how did you make it, so interesting... ( I know that you used a mouse🐭😉)
Hi Alfredo,
Your videos are really wonderful and I want to thank you for uploading such quality material. However, I am unable to understand the data flow in RNN networks. For instance, I have a sentence which is 4 words long. If I am using a Fully Connected Network of 12 neurons, each of my 4 words will be fed into the 12 neurons. I want to understand what will be the data flow if I use a LSTM with 12 hidden states.
Thanks.
Perhaps you may want to check once again the «Training example» section, at 47:52.
Amazing... Thank you 😊
You're most welcome ☺️
Alf, Thank you for making all these interesting lectures and materials available. I'm just loving this. BTW, you keep mentioning Yann's lectures. Can I view those too? Were they recorded?
Just follow the playlist. Every lecture from Yann has one of mine. The link is in the video description.
Let me know if you cannot find it.
@@alfcnz Thank you, Alf. I don't know why I didn't see all the other lectures. I'm glad I asked. Even though I'm getting the material from your lectures, I'm a firm believer in reinforcing work through repetition. Cheers!
These videos keep getting better! Excellent explanations and animations! Btw, are you familiar with the book 'Mathematics for Machine Learning' by Deisenroth et. al.? I'm really enjoying reading it. Might be a good additional resource for your students.
Thank you 😌
And yes, we've been recommending Marc's book to our students already (especially the chapter on gradients).
Hi brother. Could you please make a video on Text to video using Gans. Specifically for videos which have annotations
Interesting. What paper are you talking about?
@@alfcnz Zero shot anticipation for instructional activities. But I want to create a GAN which will take in a recipe text and create a video for it. Using Tasty dataset
hey Alf, how is it going? I have a question from this week's homework that I worked super hard to answer (I spent about a month trying to answer the question) but couldn't find the answer, so I said maybe I can ask you. however, I hate wasting someone important's time, like your time so if there's any place that I can ask these questions so that I don't take your valuable time, please guide me.
that being said, let me state my question. In the theory part of HW, Question 1.1, part C, subpart (ii), you asked about the derivative of F with respect to weight matrix W. first of all I worked out the dimension of the F function to be 1 * 1 * 2. There my problem begins, I know how to differentiate a vector with respect to a matrix but those two extra dimensions (first and second dimensions that are both 1) are giving me a hard time in figuring out the dimension of the derivative. I know that the dimension of the derivative should be the number of elements in the output, times the weight matrix dimensions transposed, and I worked it out to be 1 * 1 * 2 * 1 * 3 * 2, but it really seems wrong to me. am I missing something?
We have a Discord channel for homework discussions. 😀
@@alfcnz oh thank you, that's such a relief!
do we get certificate after watching those videos :D
Haha, only if you can write the notebook from scratch 😉
What do you use for your animations
Matplotlib, PowerPoint, Adobe AfterEffects. Depends what animations you're talking about.
@@alfcnz Particularly ones that are similar to 3Blue1Brown, and ones where you can animate what happens with data structures. (isn't matplotlib used for python).
Can you add minutes:seconds so I understand what you're talking about?
@@alfcnz @6:48, the animation of the plane ?
Yeah, matplotlib.
Nicely explained ... thanks a lot for uploading these to UA-cam. Quick questions though ... any idea when practica #1 will be available and Lecture #4 is missing from the 2020 playlist (ua-cam.com/play/PLLHTzKZzVU9eaEyErdV26ikyolxOsz6mq.html)?
There was no lecture 4 in 2020 (it was President Day). Practica 1 is basically me going over the syllabus and Yann giving an intro on the history. Not sure if it's worth adding it.
Great lecture, loved the examples. You can be even better if you work on your articulation and accent!
Hahaha 😸😸😸 My Italian is perfect! 😛😛😛
Gg e grazie per l'effort. Maybe show an anonymized chat in future streams? Not necessary ofc.
The students' chat, you mean?
@@alfcnz Yep. You sometime reply to the chat and it breaks the flow a bit since we don't see it. That being said, again, it's nothing major. This is amazing content and it's free so I'm not complaining (: Just wanted to suggest a possible improvement.
@@paolomontesel these are my live lessons, where my NYU students ask anything during class. I read their questions out loud if they are relevant. So, I'm not 100% sure I understand why I should have the text on screen. Can you expand a little? I'm really interested in understanding your opinion.
NYU SHOULD GIVE YOU A FUCKING RAISE OR PROMOTION FOR THIS!!!
Hahaha 😅 I agree 😅 I don't have a stable job nor a comfortable salary 🤣 but at least I do what I love!
@@alfcnz we need you here!!!
that sounded like the intro of a scifi erotic novel lol
😮😮😮
Hes got 99 likes but a dislike aint one.
Haters haven't come over just yet 🤣