Neural Networks | E02: predictions (unfinished series)
Вставка
- Опубліковано 3 чер 2018
- In this episode we look at how neural networks can be represented with matrices, and create a simple feedforward network in python.
Note: at 2:31 the second bias vector should only have two rows, not three.
Code:
github.com/SebLague/Neural-Ne...
I owe a lot to this excellent online book on neural networks:
neuralnetworksanddeeplearning....
Learn more about weight initialization:
cs231n.github.io/neural-networ...
(note: numpy.random.randn and numpy.random.standard_normal are functionally equivalent, the latter just takes a tuple for the shape parameter).
Support the creation of more tutorials:
/ sebastianlague
www.paypal.me/SebastianLague
Hi everyone, sorry for leaving the series unfinished. I do however have a new video out now which covers everything I intended to talk about in this series (although the code is in C# instead of python). You can watch it here if you’re interested:
ua-cam.com/video/hfMk-kjRv4c/v-deo.html
thank you sebastian,
I just watched your new video, and suddenly want to come back to this series
@@MusobarMedia After watching this video and next, I still want to finish watching this series before going to the new one!
NOTE: at 2:31 the second bias vector should only have two rows, not three.
Hello, sorry for the terribly long wait, future episodes will (hopefully) be out much quicker!
I'm trying to decide if I should do an episode covering the basic calculus understanding required for future episodes, or whether to just assume familiarity with things like the power and chain rule. Would love to hear your thoughts on this.
Sebastian Lague please just make a short video on basics of calculus that is needed for it.
Those unfamiliar with the calculus required can easily learn it online, I think you should skip it.
Ye, Im also with the idea on short calculus video
Please make a short video about it. We could learn it somewhere else, but you are a better teacher so it will be easier to understand/follow.
YESSSS calculus!!!
The most relaxing accent to listen in the world.
Awe dis Safrican boet
@@theepicosityofpizza Josh boet, can we please agree that Cape Town is still superior in the accent game?
@@shortsupply4760 of course! Those Joburgers don't talk too nice neh
@@theepicosityofpizza ja no lekka bru
YEs
The calm voice is passively mocking me as ai have no idea what is going on
I see what you did there...
Probably my favorite explanation video on the internet.
1) SO relaxing.
2) Concise.
3) Your visuals provide so much in the way of explanation.
I cannot say how i happy i am that you are doing machine learning! You are the primary reason how i learn programming!
I did not know this channel had a neural networks tutorial! Time to go to episode one
You are absolutely amazing at teaching incredibly complex processes..
"Everywhere I go.. I see his number.."
"Who's?"
"Euler's."
Whose* 🙃
@@IAmBael 🙃🙃🙂🙂🙂😀😀😀😢😢😫😫😤😤😠😠😡😡🤬🤬
“Everywhere I go.. I see his number..”
“Who is?”
“Euler’s.”
Keep it up, literally everything you upload is of interest to me!
Thanks! That's such an amazing series.
Already looking forward the next videos.
This was great! I've been looking forward to this video for a while.
An amazing explanation, I’ll no doubt be coming back here to make sure I’ve got a solid grasp of it.
OMG this is a miracle, we need more such videos. Maybe videos with ML models, their principles.
Jesus christ I wish I knew what you were doing. Intriguing.
You could *listen* to the video and pay attention? It's really not that hard.
@@milesrout I'm so proud of myself, I listened.
@@milesrout my very adhd brain that has been following Sebastian's introduction to unity tutorials for a week without any math knowledge whatsoever can tell you two things.
1) I'm learning a lot of stuff I didn't know before.
2) listening and paying attention to stuff you don't immediately understand is very hard.
you are a born teacher! hope to see more tutorial other than game programming
I had a lot of terrible experiences in calculus and precal classes, my paws are getting sweaty just looking at those brackets and when you showed that 2 by 3 matrix my hard stopped! This is really informative, almost soothing, but at the same time my fight or flight response is getting activated.
Him: (quantum rocket brain mechanics)
Me, who has no idea what's going on: *nodding*
wow yet again making this so easy to understand
This is such a good working knowledge explanation of how machine learning works, thank you
Amazing video. The voice accent is soothing too
Man, you're the one who makes best algorithms ! Thanks a lot man.
I really enjoy your videos , very good tutorials :)
Looking forward to the next lesson!
I wanted to learn neural networking and here I go !
Very good explanation 😍 please continue the series on neutral nets
Great video ! I hope to be able to do something like this in the future!
Multilayer NN in 17 lines? That's quite impressive :D
That numpy sintax simply shorten my 15 line of code lol
Not really because it is only feed forward and libraries shorten it
I just discovered your channel, it's amazing! I'll be watching every single video this week!
However, my feedback for this one is that the font is too small to read on my phone xD
2:45 - shouldn't b2 have 2 values only?
Oops, yes, absolutely!
@@SebastianLague Timestamp: 2:30 - I paused the moment I saw b2 vector with 3 elements to search for this correction! :D Could you add that correction to the video as a floating note?? I have a small understanding of neural networks, but people seeing this for the first time might think that that vector is legit and misunderstand this.
@@vial7912 Floating Notes haven't been a part of UA-cam for like 8 years
"episode covering the basic", yes, I would like to have it. Basics are also good, if someone is too good in this, those videos can be skipped but in the opposite, it is quite hard when something is missing and you have to search for info somewhere else.
Do not get me wrong but I treat your channel as a place where you can start learning something new and do not expect deep hardcode right away, so I would vote for basics to be.
This is the basics.
this is as basic as it can get
Very clear and concise explanation. Much props to you. I wonder what your thoughts are on the trend towards using ReLU instead of sigmoid as the activation function for hidden layers.
finally the easiest ANN tutorial to learn
You are the best man
Great video, just finished first year Computer science at university and need something to learn over summer. Keep it up!
Although any recommendations on how to learn neural networks and machine learning properly, would love to get a full grasp of it but there's so much material out there its hard to know where to start.
Hey! Great video! Thanks!
He is one of the only two programmers (the other on being `code bullet` ) whom I saw actually code the neural network because they know how that actually works instead of importing neat-python and doing the steps in documentation line by line. Now I know what on earth is an activation function...
The thumbnail is the real deal 😎😎😎...
Incredibly well made video, I mistakenly read 150k views... 15k views on a video with such clear explanation and visualization?
Explaining these networks so anyone can understand is quite difficult. What I am finding is than many 'items' are assumed understood. You do quite well. What is important to all, is to understand the 'newness' of this subject. At any time there may be an introduction that 'enhances' this topic for everyone. This needs to be concluded with "you're do fine job."
Great, now time to do it in c#
For a person accustomed to more formal languages such as C++, C#, and Java, Python is pure sorcery. I'm going to have to spend some time learning this.
if you know C/C++ on relatively good level, learning python won't be a problem at all. don't worry)
@@evgenyaleksandrov1206 there is no reason to try python if you know c/c++.
Zytyk Clppw Why would there be no reason to learn another language? C++ isn’t the perfect language for most tasks. Languages are built for specific purposes, and for things like AI and Neural Networks, C++ would be the wrong choice for the job.
Python is considered the easiest language so it shouldn’t be hard it learn
Excellent video. Quick question though: what do you call that technique in python that you do to instantiate weight_shapes list? I'd like to learn more about how that works.
what will be the pink purple graph be look like, if say we have 10 predicting output class? like will be 10 colours?
Hi, please make a video of evaluation techniques, and the best methods
I dont think I retained a single thing from this but I feel educated either way
*like before watch!*
Just getting into neural networks and these videos are great visualization. Does anyone know how he was visualizing his models? Was it a graphical python library or something else?
What is predict function and why it's returning single value not vector?
Omg neural networks are so awsome i tryed to made one before and i hop you can help me please more
Hi! Im trying to implement your method as just a script, without classes. But output layer always seems to be as big as the hidden layer, not the output layer. So if my layer_size is (3,5,10), the output layer gives me 5 instead of 10. Heres the code.. Could you explain? Thanks !
import numpy as np
layer_size = (1,2,5)
weight_shape = [(a,b) for a,b in zip(layer_size[1:], layer_size[:-1])]
weights = [np.random.standard_normal(s)/s[1]**.5 for s in weight_shape]
biases = [np.zeros((s,1)) for s in layer_size[1:]]
def predict(a):
for w,b in zip(weights, biases):
a = activation(np.matmul(w,a) + b)
return a
def activation(x):
return 1/(1+np.exp(-x))
x = np.ones((layer_size[0],1))
prediction = predict(x)
print(prediction)
Output
-----------------------------------
[[0.44016988]
[0.42551257]]
How did you learn to program?
@ 7:53 Is there any reason to us e over any other rational or irrational constant (such as pi or 2) in your activation function?
@S. Choudhary I get that I didn't actually ask "why," but I thought it was fairly well implied by my obvious ignorance of a reason.
I hope your channel UA-cam can be suces
Hello Sebastian, do you use Python for convenience, or is there no such library in csharp as numpy? How do you then use the network in Unity? Assume you train in python and then only import the resulting weights and biases into unity, correct?
Lovely explanation with wonderful animations.
However, I think you have a small mistake at time 2:33 concerning the vector b2 because the size of the vector should be 2×1 instead of 3×1.
Sebastian plz reply
Ever since episode 19, (or episode 18) the terrain has started to fail generating and i dont know why. The console says there are no errors, and don't ask me about the viewer, because i've already set it. It's really annoying and i really need your help. The edge chunks are still not alligned and the terrain only generates 1 chunk. This is from procedural terrain generation. I'm replying here as well because its a new video.
What was the "neuralnetwork" module you were using, I cannot seem to find it?
Thanks
It's the script created in the first part of the video.
Oh, of course, I feel like such a buffoon!
I love you
Episode 4! please!
Feels like I need get a degree to understand these videos. If someone can point out where should I start the process of learning the basics to follow this serie, any advice is well received
I'm curious about what colour scheme he's using for his code. Any ideas?
That's Monokai.
What program did you use for the python
I need some help, I've done everything as he did, but got stuck at ˜10:00 where he ran the second program which he made. Whenever I launch the code, it gives me an error saying that matmul doesn't exist in the module I have. I have numpy 1.16.2, and Python 3.7, and I checked, matmul is in the files of numpy.
Error code: AttributeError: 'module' object has no attribute 'matmul'
my code is the same as his, no changes. I even downloaded his neural network and tried it out. Same error.
I don't know what the problem could be, but if you are still stuck replace, his part of the code "np.matmul(a,b)" with "a@b", both do the same thing (matrix multiplication)
a little search on google 'cause I had same issue : if you are using sublimeText also you'll have to create a new build for python 3 because default python build is for version 2...
stackoverflow.com/questions/51061973/how-to-change-python-version-from-2-x-to-3-x-in-sublime-text-3
5:44 That sounds like a nasty flaw in how the training library implements adjusting, rather than "a terrible idea" to initialize with.
No this is just how math works. if you apply the same function on the same input it will always give the same output
Could someone make this clear to me? Im not sure I fully understood.
They way I understood it is that the hidden layer has 3 neurons. These neurons all have values of 0-1. This value is calculated and represented as a1, a 3x1 matrix that holds the 3 values.
I then dont undserstand how the 2 output neurones (matrix 2x1) is calculated.
I understand you multiply a1 by w2 and add b2, (represented with matrixes: 3x1 X 1x3 + 2x1) and then do the sigmoid function to normalize the result and make it fit between 0 and 1. I just dont understand how the 2x1 output matrix is created. I did it on paper and got a 3x1 matrix.
I apreciate the help.
It's w2 × a1 + b2 so it will be (2×3 X 3×1 + 2×1) = 2×1 which is the output.
@@bharathikannannithyanantha7841 (2x3 x 3x1 matrix gives a 3x3. How could you then add the 2x1 matrix to that?
Sublime text :)
Are you ever gonna continue the unity procedural generation series? :)
doubtful, it seems to be stuck at a deadend
I got the error numpy has no attribute matmul in NeuralNetwork
How to create a 2d bike game on unity
Good bidio
i´ll always get this:
line 15, in predict
for w,b in zip(self.weights,self.biases):
AttributeError: 'NeuralNetwork' object has no attribute 'weights'
That means in the class neuraNetwork .....it cant find weights...
Maybe you spelled it wrong there or forgot to put self.weights there
So THIS is why I should have payed attention in calculus
4:18 why "(a,b) for a,b in zip(..." instead of "(a) for a in zip(..." ?
or even list(zip(layer_sizes[1:],layer_sizes[:-1]))
@@Achmedsander you can just use plain zip() as size argument for np.random.standard_normal().
Which ide you are using,
Is it pycharm or vs
The IDE you use generally doesn't matter that much. Mostly, as long as you are using a modern one (which both of those are) it's just a matter of preference. Personally, I'd suggest using VS Code (code.visualstudio.com/) because it's more modular, as in it works better with many different programming languages (HTML, Java, JS, C#, ect). However, if you're just getting started then honestly choose whichever you think looks best. After all, you'll be spending a lot of time looking at it ;).
Can anyone give me a good place to learn python? I know only a little bit of the language, and id really like to know it better in order to do this...
Little Narwhal Take the Ultimate Python Course on Udemy. It's priced at only $10 now. Very comprehensive content and over 23 hours of video tutorials.
Wowwwwww 😲
I understood none of that, but I watched it anyway.
11:00 i don't think likelihood is the right term... if it were, all the outputs should sum to 1
some people might misunderstand that part
nonetheless, great video!
Ooh you use blender too
Why is python so much better for programming neural networks than C#? Is there a simple explanation for that?
Yes. Python is a higher level language than C#, that means it has more layers of abstraction (hides the complexity from the programmer). This makes it slower than C# though.
Can you do the other tutorials in unity / are the other tutorials in unity
Nikita Skobelevs ok thank you :)
Nikita Skobelevs PS: I thought he would make for example a little ai for a game
Before moving to E03 I just wanted to say:
As a person who knows python but nothing about Neural Networks (other than the previous episode E01) I can't able to understand anything about what this network predict here. :(
Let's see what will happen in E03!
I just checked out all your videos related to AI. I am new to AI and you can say a beginner. I want you to make a pattern recognition based AI or image recognition using AI for beginners with a small data set lets say 100 images with all guidance related to which software you would use and code as well step by step. Need guidance to start working on AI
I think I've seen similar classes in U of Alberta, but I think it's in 500 level :v
Episode 3 when?
Bruh it came out over a year before you made this comment
why does the thumbnail for this video animate when I scroll past it?
Found yourself an optical illusion here my friend.
I'm in python 3.8 and I'm getting totally different outputs.
nevermind, it was going a bit too fast for me and i missed a word
Technically, you could have just done list(zip(...)) 4:30
python > Python around 3:00
you went so fast over predict function ...I mean I think only people already knowing how it works would understand it or at least most of them. I want to be mean, I would just say that u should spend like 30 seconds explaining how it works, what matmul does and so on, because one thing is seeing it in matrix form, another is in code ...you should put those together and make sure everyone get it ...if you want to have noobie tutorial you should not skim through the most important part ! ...but as a person who already has some basics, I loved it ! Dont mean to hate, just some constructive criticism :)
As a JavaScript coder i am so incredibly lost
wow did a decade past?
or more like how much time passed
i'm not mad or something
but i'm curious why did it took so long
i may respond to myself it was family problems...
nani?
2:50 b2 Is 2x1 not 3x1
Hey guys ppssst... i didn't install numpy for this and created the calculations myself
I'm recreating that in Lua and I think I'm ready to die
Great tutorial, but your single letter naming of variables makes the code very unreadable. I had to re-watch alot of this multiple times just to remind me what some methods were acomplishing. Following good code practices should make each line self explanatory!
Thanks! I agree, if I were to do it over I’d definitely prioritise clarity over brevity.
@@SebastianLague Thank you :), I can fully understand with certain uses! (Shorterning Weights + Biases to w,b respectively) but for some I was losing the connection between the theory you provide and where the code was applying it I.E: (for s in weight_shapes, I presume s is a shape, but what was a shape in the theory again),( What was the x parameter for the activation meaning? ). This can just be my missunderstanding of simple concepts.
This is way too much at once.
I have no clue how Python works nor do I know how Neral Networks work and for my lack of understanding the video proceeds way too fast.
But still, it is really interesting. If i would not be that lazy as I am I would invest time into understanding it better. :D
The idea is that it expands ur knowledge if u already know how to programme
You missed a semicolon
Because i dont use python i have no idea what is going on. Just none.
I used it a lot in the past but this neural networks concept is just too complex for me