@@hugomougard Thanks ;) This is the best video I have seen to explain universal approximation theorem, it's better than 1k words. Thanks for making it.
Hello, thank you for your video, it helps to understand, but I have a question : How does the NN choose the "steps" of cut ? Because from what I understood, for next layer (i.e here f hat), we simply do the sum, as you did in blue below. Because if we do the real sum of the functions, we'll get simply a non linear function but that looks like a ReLU right ?
Hi Y., For each hidden nueron, the cut is picked by tweaking the parameters (each neuron has 2 parameters, one that multiplies the input, and one that is just added to the multiplied input). The way those parameters are tweaked is usually by gradient descent. Sorry I'm aware that my response looks very generic, but going into more details is basically doing a full course on neural networks. I can recommend the videos by 3blue1brown on the subject if you want more details!
So good, so clear, so concise.
I can see the ReLU function *_*
Indeed, good job :)
How do you see that?
Great visualization !
very good, easy to understand 👍
hmm..makes sense ...so it finds the best possible linear function then the activation..n then finally add them up together to join them all
Thanks for the clear visualization! In this case, the activation function is ReLU right? Sigmoid will looks different
Well spotted! Indeed the activation in this case is ReLU, and SIgmoid would have looked smoother :)
@@hugomougard Thanks ;) This is the best video I have seen to explain universal approximation theorem, it's better than 1k words. Thanks for making it.
@@jingli9995 Thank you for your kind words.
Hello, thank you for your video, it helps to understand, but I have a question : How does the NN choose the "steps" of cut ? Because from what I understood, for next layer (i.e here f hat), we simply do the sum, as you did in blue below. Because if we do the real sum of the functions, we'll get simply a non linear function but that looks like a ReLU right ?
Hi Y.,
For each hidden nueron, the cut is picked by tweaking the parameters (each neuron has 2 parameters, one that multiplies the input, and one that is just added to the multiplied input).
The way those parameters are tweaked is usually by gradient descent.
Sorry I'm aware that my response looks very generic, but going into more details is basically doing a full course on neural networks. I can recommend the videos by 3blue1brown on the subject if you want more details!
Can you do this with more layers? I want to know how adding more of them can increase the complexity of the function
Could you visualize the same thing but with multiple hidden layers?
I do not plan to make a visualization about that soon but I'll keep your suggestion in mind :)
now approximate sin(x^2)
how you do these visualizations please help.
I use github.com/ManimCommunity/manim :)
i wish there was gif version haha
How to implement in matlab ???
yes