The Independent Code
The Independent Code
  • 4
  • 365 004
Softmax Layer from Scratch | Mathematics & Python Code
In this video we go through the mathematics of the widely used Softmax Layer. We then proceed to implement the layer based on the code we wrote in last videos.
😺 GitHub: github.com/TheIndependentCode/Neural-Network
EIDT: I decided to deactivate my @CodeIndependent Twitter account. Instead I'll use my personal account to tweet about upcoming videos :)
🐦 Twitter: omar_aflak
Chapters:
00:00 Introduction
00:30 Forward
01:31 Forward Code
02:14 Backward
05:47 Backward Code
06:21 Conclusion
Переглядів: 17 717

Відео

The unknown trick to solve these polynomials: symmetrical coefficients
Переглядів 3,5 тис.3 роки тому
In this video I show you a trick to divide by two the degree of some special sets of polynomials: the symmetrical coefficients polynomial. 😺 GitHub: github.com/TheIndependentCode/Neural-Network 🐦 Twitter: omar_aflak Chapters: 00:00 The method 01:52 Degree 6 03:00 Generalisation 04:28 Proof 05:28 Summary
Convolutional Neural Network from Scratch | Mathematics & Python Code
Переглядів 199 тис.3 роки тому
In this video we'll create a Convolutional Neural Network (or CNN), from scratch in Python. We'll go fully through the mathematics of that layer and then implement it. We'll also implement the Reshape Layer, the Binary Cross Entropy Loss, and the Sigmoid Activation. Finally, we'll use all these objects to make a neural network capable of classifying hand written digits from the MNIST dataset. 😺...
Neural Network from Scratch | Mathematics & Python Code
Переглядів 145 тис.4 роки тому
In this video we'll see how to create our own Machine Learning library, like Keras, from scratch in Python. The goal is to be able to create various neural network architectures in a lego-fashion way. We'll see how we should architecture the code so that we can create one class per layer. We will go through the mathematics of every layer that we implement, namely the Dense or Fully Connected la...

КОМЕНТАРІ

  • @L2.Lagrange
    @L2.Lagrange 5 днів тому

    Very useful, thank you

  • @ash_80
    @ash_80 6 днів тому

    Awesome explanation and amazing animations

  • @battycrease221
    @battycrease221 16 днів тому

    wouldn't the output have the same dimensions as the input or is that only if you do a full convolution with a 3x-dim filter matrix?

  • @ExXGod
    @ExXGod 22 дні тому

    I just watched your CNN video, the next one and I couldn't resist watching this one. Although I knew most things in this video, watching everything work from scratch felt amazing.

  • @ExXGod
    @ExXGod 23 дні тому

    Best video on CNN. Period. Liked. Subscribed. Shared.

  • @Nightwolfjor
    @Nightwolfjor 28 днів тому

    One of the best tutorial so far, Great job

  • @riccvven2078
    @riccvven2078 Місяць тому

    would be useful in future to create an outro where you give suggestions on how to implement other numbers or improve performance;like, look at this function instead of cross entropy... or smth like that; that would work like an exercise and engage us in excitement to see how you'll do it in future and differences between our codes

  • @thiagoborn
    @thiagoborn Місяць тому

    by far, the best video of this topic that I saw in the whole platform

  • @riccvven2078
    @riccvven2078 Місяць тому

    thks i didn't get the connection between convolution and nn

  • @baongocnguyenhong5674
    @baongocnguyenhong5674 Місяць тому

    i've taken inspirations from your code and cited your channel for my neural network paper for a college project, im just letting you know this here and hope that you won't feel particularly mind for it. btw, thank you so much for the video, 3blue1brown's series on neural network is great and all, but it is your video that makes all the computations really sink in and make actual sense, representing the gradients as linear algebra operations just ties everything together so neatly, compared to individual derivative formulas for the weights and bias, which is how it's usually written. And the choice of seperating the dense layers and the activation layers was, to put mildly, fucking brilliant.

    • @independentcode
      @independentcode Місяць тому

      Of course! Thank you for the kind words :)

  • @samarthdhingra7474
    @samarthdhingra7474 Місяць тому

    2:01 the formula is given to be y=i-k+1 as the stride is one but there is no existence of it in the example given

  • @Adityaaaa0408
    @Adityaaaa0408 Місяць тому

    I have been struggling with backpropagation in MLP from 2 weeks and when I was just searching for a video which can help me understand the process mathematically this video grabbed my attention and in this video I was able to understand the whole process both conceptually and mathematically actually the code given by you was the same code given by our mentor to us but he he was unable to explain clearly and the animations shown in the video were really great finally thank you for posting this video!!!!🛐I CAN ADVANCE IN MY PROJECT FURTHER!!!

  • @vidyaviju1675
    @vidyaviju1675 Місяць тому

    This is THE BEST video where I completely learned about forward and backward props , in such an in depth manner! Thank you so much!!

  • @tanweerriyadh3785
    @tanweerriyadh3785 Місяць тому

    Please make a video on Recurrent Neural Networks... This was sooo goooddd!

  • @aminemeskar7564
    @aminemeskar7564 2 місяці тому

    I can't thank you enough; you made everything super simple. Please keep making these videos if you can apply the same method to machine learning models. Thank you so much.

  • @rezNezami
    @rezNezami 2 місяці тому

    are you fucking serious?? 5 min of one commercial (Wix Studio) at the start?!!!!

  • @noahwall6970
    @noahwall6970 2 місяці тому

    GOATED video

  • @victorantoniotorrescastill2534
    @victorantoniotorrescastill2534 2 місяці тому

    Very nice video! Just a question: are pooling layers used in your model?

    • @independentcode
      @independentcode 2 місяці тому

      No pooling layers, I only used what I showed in the video :)

  • @davidk5589
    @davidk5589 2 місяці тому

    Super nice tutorial, very insightful! Small remark, a convolution is actually a correlation with a reflected kernel, not a rotated one, i.e. convolution and correlation are related by a change of variables: corr(f, k) = \sum_i=1^N f(i)k(i) then conv(f, k) = \sum_j = 1 ^ N f(i)k(N - i). In the case of a 2x2 kernel this happens to coincide with a 180 deg rotation, but this is not true in general e.g. for a 3x3 kernel a flip and 180 deg rotation are not the same. See: en.wikipedia.org/wiki/Convolution#:~:text=Some%20features%20of,the%20convolution%20operator.

    • @independentcode
      @independentcode 2 місяці тому

      Thanks a lot for pointing this out! I'm pinning your comment :)

  • @owl166
    @owl166 2 місяці тому

    OMAAARRRRR where are you MAAAANNNNNNNMN

  • @davidbarrero525
    @davidbarrero525 2 місяці тому

    Someone pinch me because this is so good I can’t believe it’s real! Thank you so much for this :D

  • @dontlistenme
    @dontlistenme 2 місяці тому

    At 7:10, you called the "d" the layer depth and the output depth. but, here, the output is dependent on the number of Kernels instead of the depth of kernels or the input (channels). could you please clarify this?

    • @omaraflak
      @omaraflak 2 місяці тому

      That's correct. Each kernel has to extend to full depth of the input. So what I called the "layer depth" is the number of kernels, which is also the depth of the output.

  • @emm1757
    @emm1757 3 місяці тому

    only 0.00000003875% of people know this trick

  • @bidd2278
    @bidd2278 3 місяці тому

    05:00 shouldn't the size of bias be one real valued number for each kernel features, i.e b1 for feature 1 and b2 for feature 2 and so on....

  • @baongocnguyenhong5674
    @baongocnguyenhong5674 3 місяці тому

    This video is godsend, thank you.

  • @Rustincohle88
    @Rustincohle88 3 місяці тому

    This is literally a masterpiece

  • @Ryanxyang
    @Ryanxyang 3 місяці тому

    Great video! At 17:45, last row of matrix W' (transpose of W), subscript got a bit messed up. w_1j, w_2j and w_ij should be w_1i, w_2i and w_ji, i.e., j rows and i columns.

  • @working7343
    @working7343 3 місяці тому

    Great video Thank you, can you please reduce the background music volume. Thank You

  • @joshp8820
    @joshp8820 3 місяці тому

    Thank you so much oh my god, I've been working on fully implementing a U-Net from scratch and I've been going mad trying to find the equation you reveal at 23:28, since I didn't know how to handle the multiple matrices produced by the d amounts of convolutions (I was going to guess summation but guessing felt bad). And here is this beautiful video with less than 200k views, which not only is a very very solid explanation of convolutional layers to full depth, but also contains a little puzzle piece I cannot for the life of me find anywhere else. Thank you!

    • @independentcode
      @independentcode 3 місяці тому

      Thank you for the kind words, I'm glad the video was helpful!

  • @akbardehghannezhad2378
    @akbardehghannezhad2378 3 місяці тому

    Thank you so much!

  • @SanjiPinon
    @SanjiPinon 3 місяці тому

    There is a mistake. The bias is a single term and not a matrix. For every filter there is one scalar bias added to all elements in feature map.

  • @dandan1364
    @dandan1364 4 місяці тому

    Amazing. Couldn’t follow all of it, very dense, but still very good. What’s really annoying is you did back prop in 4 lines of code, my dense ANN back prop in C# is like 50 lines of code. Good goal for me to see how to compress the code.

  • @BhavinMakwana-b3o
    @BhavinMakwana-b3o 4 місяці тому

    How do i make it for the all classes? This is only for the two classes? Bdw great video. I am Phd student and this is gold for me.

  • @dhudach
    @dhudach 4 місяці тому

    This is an unbelievably clear and concise video. It answers all of the questions that linger after watching dozens of other videos. WELL DONE!!

  • @HimanshuShekhar-e7o
    @HimanshuShekhar-e7o 4 місяці тому

    Crazy video, i haven't seen anything better in this topic !

  • @Param3021
    @Param3021 4 місяці тому

    Very much interesting! Thank you for deep explanation ❤

  • @maurod6180
    @maurod6180 4 місяці тому

    excellent!!

  • @samyakbharsakle
    @samyakbharsakle 4 місяці тому

    4:41 what exactly does it mean by having 3 dimensional data? what could be an example?

    • @independentcode
      @independentcode 4 місяці тому

      An image is a 3D input: width x height x channels

  • @vijaybarhate7769
    @vijaybarhate7769 4 місяці тому

  • @minecraftermad
    @minecraftermad 4 місяці тому

    hmm. so it's just a dnn neuron with a matrices for weights, bias and input, the input being a sampling of the previous layer. or maybe it'd be better to say each filter is a small mlp, that's slowly dragged over the entire input field to produce and output field?

  • @Vishal-d6j
    @Vishal-d6j 5 місяців тому

    You are a GOD 🙇🙇

  • @brenojust6436
    @brenojust6436 5 місяців тому

    Hi, I'm naive to math and coding, can anyone explain where in the def backward part of the dense layer code the derivatives are computed? The video explains that the derivatives are there, but I was expecting to see a function to compute it. Where exactly does the derivative appear there?

  • @LouisDuran
    @LouisDuran 5 місяців тому

    Thank you! Thank you! Thank you! Thank you!

    • @independentcode
      @independentcode 5 місяців тому

      Thank you! I appreciate the support :)

  • @Leo-dw6gk
    @Leo-dw6gk 5 місяців тому

    This video should be the first video you see when you search neural network.

  • @Dynamyalo
    @Dynamyalo 5 місяців тому

    this has to be the single best neural network explaining video I have ever watched

  • @matthewhernandez9152
    @matthewhernandez9152 5 місяців тому

    This is an absolutely wonderful video you have here. Your explanations are clear and you helped out a graduate student a great bit!

  • @omegaui
    @omegaui 6 місяців тому

    Such a great video. Really helped me to understand the basics.

  • @luigibombastico7641
    @luigibombastico7641 6 місяців тому

    I think this is a little to complicated for me... i just wanted an AI that plays the simple game of Nim but now im here...

  • @sayantikachatterjee5032
    @sayantikachatterjee5032 6 місяців тому

    here you are doing the SGD right? because for every training instance you are updating the weights. This is not batch GD.

  • @minakaviani
    @minakaviani 6 місяців тому

    There is a mistake of matrix W.t in last row (w1i, w2im,,,,wji), in the time of 17:49....