Hi, I actually have some experience in this field. The method grey is describing is called gradient descent. The weights have the derivative of their error function (basically a function for how accurate the model is) with respect to themselves multiplied by a learning rate (which either increases or decreases how drastic the changes to the weights are), then subtracted from themselves to create a new weight. This results in the error of the neural network gradually approaching 0. It's one of the more basic algorithms but it's great for beginners to understand before dipping their toes in more complicated optimization methods.
ohh so this is what was going on, we had to build a program with a polynomial that got closer and closer to the sin graph, i didn't really understand what or why we were doing what we were doing, but not it makes sense, at the time it was a lot to take in with all the matrices and differntiation stuff suddenly being thrown at us, now this video and the 3blue1brown video really puts into perspective if someone is in my scenario and still don't understand whats going on the constants in a function that you can change are the wire attachment things he was talking about the video, you take the derivitive to know how far away from the results they are and then with that you get closer and closer to the results to a stage where its almost perfect at the outputs it gives
For those who are interested in this topic, CaryKH has some interesting videos about this where randomly generated digital creatures learn how to walk, jump, steer etc.
Ahhh, but do ‘these’ comments still get through? Let’s find out, for science! Reply to me to let me know if this worked. Also… Grey, did you ‘change the name’ on this video to get more reactions? Interesting.
For my AI exam, I made a neural network that was designed to estimate, based on statistics on actors and their movies, how much a given movie would gross, based on actors, genre and director. It overestimated by about 100%, so a movie that grossed $50M, the network would say grossed $100M, or more. It was sort of adorable to watch it attempt (and fail miserably) at guessing movie revenue, but it's scary how easy it was for a noob like me to just sit down and write something that sorta-kinda worked.
I should probably try to redo it. Had to follow some guidelines which I disagree with, design-wise, for the exam, so I might be able to make something slightly better as a hobby project.
Zenyl I personally learned the most just messing around with all sorts of layer sizes and activation functions and the like so I would definitely recommend spending a little free time on it :)
Having accurately labeled pictures is big chore, and when training having more examples usually helps results. So the standard procedure is to take a labeled photo, then transform it and use the transformed versions as well: shift the object up, down, left, right, and scale it bigger and smaller, rotate it, even do things like blur it a bit, and of course combinations of all of these. That is much easier than having to find 20 different photos of bees in different orientations.
Damn. The first explanation was genetic evolution through natural selection. This second process is basically what happens post birth when synaptic connections get reinforced with use; this is the in vivo evolution of a brain from a mush of random meaningless connections to a network of purpose built ones. Are... Are we someone's AI?
GregTom2 knock knock, it sounds like religion is at the door with an answer to your question and answers to several follow up questions. Wanna open the door?
Great job! I had my concerns about the original video but this one cleans things up great. Also that clip of the the bottom throwing images into folders is my favourite thing in any of your videos ever.
@Mycel that was me when I was reading a set theory textbook that was at the time way too difficult for me. I walked up to my (smarter and more importantly older) friend and said: "Hey you should read this" "Ok, what is it about?" "Set theory" "More specific please?' "I have no idea what it's about, but you'd like it".
A good example I have seen for this is the MtG neural network generated cards. The dials have been tweaked to create a range of cards varying from "could've been made by the designers" to "not even close to plausible". I would definitely recommend checking them out
On the subject of machines identifying things in pictures, my phone just randomly spat out a “memory” with many of my pictures of my cats and titled it “four legged friends”
Yeey! The first (popular) video about artificial intelligence that is actually accurate about the methods used. Great job! Would have loved to see the animations for evolutionary training of networks, where the teacher bot makes the students brawl :)
CGP Grey this is what I’ve been working on for the last month (back propagation) and my friends always ask what I’m doing and how but I always have trouble explaining it. Thanks, now I can show them this video!
Wow I was literally complaining about your oversimplified and somewhat inaccurate metaphor of “bot factory” in your main video, and here it is, a perfectly explained footnote. For someone who still can’t do pointers, this is a surprisingly in-depth grasp of the neural network concept! I salute you!
If you are really interested in the details, I can highly recommend 3Blue1Brown's series on machine learning. He does a great job explaining the basic math behind deep learning.
Your video's story, graphics, animation, narration & production values are great, I'am very glad you did this piece and eluded to the fact that neural nets is where the 'hotness' is. The term 'Genetic Models' should have been mentioned in the first piece. It was the topic of the piece. Since neural networks are where the opportunities are, I hope you consider redoing both pieces, and reverse the order. . Neural nets can be described in simple terms, and you could be the one to do it best.
Interestingly in retrospect, this video was originally published not long after the "Attention is All You Need" paper, which made recursive neural networks less relevant. ChatGPT uses Transformers instead.
I really liked the way you endded the main video.. A good and creative way of both emphasizing why people always ask "us" to do this, while at the same time getting us to do it.
3b1b has an amazing explanation of how deep learning works if anyone's interested in digging in to some of the details. Still pretty basic compared to what's used in research today but a good place to start
CGP Grey getting closer and closer to complete existential fear of robot oppression.
I for one welcome our new robot overlords. They can see this, right?
randomguy8196 then it will be time for
FULLY
AUTOMATED
LUXURY
SPACE
COMMUNISM
Oor for Fully Automatic Miserable Corrupt Corporatocracy
Scorponox93 oof. Bad robo-bourgeoisie
HA the bots are really stupid, the amount of supposed keys goes up between visits. I'm still to smart for it..... still.....
Man's was 5 years ready for Chat GPT to blow up
Ahh, I see the thumbnail and title have been updated to current events. Grey, you continue to bless us with great knowledge.
Hi, I actually have some experience in this field. The method grey is describing is called gradient descent. The weights have the derivative of their error function (basically a function for how accurate the model is) with respect to themselves multiplied by a learning rate (which either increases or decreases how drastic the changes to the weights are), then subtracted from themselves to create a new weight. This results in the error of the neural network gradually approaching 0. It's one of the more basic algorithms but it's great for beginners to understand before dipping their toes in more complicated optimization methods.
ohh so this is what was going on, we had to build a program with a polynomial that got closer and closer to the sin graph, i didn't really understand what or why we were doing what we were doing, but not it makes sense, at the time it was a lot to take in with all the matrices and differntiation stuff suddenly being thrown at us, now this video and the 3blue1brown video really puts into perspective
if someone is in my scenario and still don't understand whats going on
the constants in a function that you can change are the wire attachment things he was talking about the video, you take the derivitive to know how far away from the results they are and then with that you get closer and closer to the results to a stage where its almost perfect at the outputs it gives
Mitochondria is the power house of the cell
yes
Ah yes, I definitely understood whatever you said here
The derivative here is basically doing a newton's method on the weights, right?
0:15 It's an older code but it checks out. Nice star wars reference
AwesomeMangoes lol, I saw that too. Clever, Grey, clever.
AwesomeMangoes It’s an older meme sir, but it still checks out.
AwesomeMangoes I know right
it's a trap.
Yep
Uh, of course we know what a car radio is, Grey, we are't stupid.
...what's a dial?
Garageband
I think it's a phone thing like... Dial phone?
"Don't touch that dial!" - radio DJs in the before time
It’s the uhhh the thing on Call of the Dead from Black ops 1
I've seen one of those once. They use it to tell time via the sun.
“..is an older code, but it still checks out.” Nice one Grey.
I have liked and commented. Have I pleased the learning overlords?
I have liked and commented as well. Now I only have to watch the video.
The bots like your viewer retention.
NO
CGP Grey your obviously not the real one so stop!
Shhh.... Ignore, report...
00:10 "marry-o"
what accent is that? perhaps the answer to this question will reveal Grey's secret identity.
He lives in England and has done so for at least 10 years
@@thehiddenninja3428 And he grew up on Long Island, New York.
What?
Nah
My family all said it that way when I was young. We lived an hour outside of NYC.
Man is gigabraining these rebrandings
I can't be the only one who wants a Bot Baby plushie right?
Tweet at Brady I'm sure he's our only hope. :P
They are so freaking cute
DOOOO IT GRAY
I want a plushieeee
Please Grey I beg of you!
For those who are interested in this topic, CaryKH has some interesting videos about this where randomly generated digital creatures learn how to walk, jump, steer etc.
yeah, and 3blue1brown explained pretty well the recursive learning in newer algorithms.
+
Yeah, those videos are a good way to see machine learning in action.
The bot which chose your comment to be so high up is doing a great job since I am now intrigued to check out those channels...... THEY ARE EVERYWHERE
+
“An older code, but it still checks out”. Hehehe nice one Grey.
I couldn't find it anywhere on github.
+CGP Grey
false
Dude. Stop it.
ididnotgetthatreference.gif
Hamish Woodland
Wut?
Ahhh, but do ‘these’ comments still get through? Let’s find out, for science!
Reply to me to let me know if this worked.
Also… Grey, did you ‘change the name’ on this video to get more reactions? Interesting.
"Mayrio"
CGP GREY 2017
No
Look at the Description
Someone Made an AI play Mario called Mar/IO
For my AI exam, I made a neural network that was designed to estimate, based on statistics on actors and their movies, how much a given movie would gross, based on actors, genre and director. It overestimated by about 100%, so a movie that grossed $50M, the network would say grossed $100M, or more.
It was sort of adorable to watch it attempt (and fail miserably) at guessing movie revenue, but it's scary how easy it was for a noob like me to just sit down and write something that sorta-kinda worked.
Why didn't you just divide the output by 2? Wouldn't that have made it more accurate?
The idea was for the network to figure that out on its own, no handholding.
Adjust a few hyperparameters and maybe tweak the structure a bit and I'd bet you would be within a few million
I should probably try to redo it. Had to follow some guidelines which I disagree with, design-wise, for the exam, so I might be able to make something slightly better as a hobby project.
Zenyl I personally learned the most just messing around with all sorts of layer sizes and activation functions and the like so I would definitely recommend spending a little free time on it :)
CGP Grey is the bot sent to us to teach us about the bots
That probably makes him a bit of a traitor in the bot's eyes...
But if every one knows how to make bots, more bots will be made. They need every one to make lots so they can have a huge army.
@@jpr90 Why? He's an Envoy.
Having accurately labeled pictures is big chore, and when training having more examples usually helps results. So the standard procedure is to take a labeled photo, then transform it and use the transformed versions as well: shift the object up, down, left, right, and scale it bigger and smaller, rotate it, even do things like blur it a bit, and of course combinations of all of these. That is much easier than having to find 20 different photos of bees in different orientations.
Damn.
The first explanation was genetic evolution through natural selection.
This second process is basically what happens post birth when synaptic connections get reinforced with use; this is the in vivo evolution of a brain from a mush of random meaningless connections to a network of purpose built ones.
Are...
Are we someone's AI?
GregTom2 knock knock, it sounds like religion is at the door with an answer to your question and answers to several follow up questions. Wanna open the door?
- answer that door -
the reason it is like this is likely due to the fact that these learning models are based on ourselves because that is what we know works.
Yes we are someone’s AI. It is not known exactly who, but yes.
@@Rottilargo55 *you can make a religion out of this*
Great job! I had my concerns about the original video but this one cleans things up great. Also that clip of the the bottom throwing images into folders is my favourite thing in any of your videos ever.
If you want to see the math behind this in detail, I recommend 3blue1brown's series on this
It's actually the top recommended video now. The bots are always watching.
Mohammed Saud We’re all doing the bots’ jobs.
@Mycel that was me when I was reading a set theory textbook that was at the time way too difficult for me. I walked up to my (smarter and more importantly older) friend and said:
"Hey you should read this"
"Ok, what is it about?"
"Set theory"
"More specific please?'
"I have no idea what it's about, but you'd like it".
@@Toviewthisvideoyou not for me. Remember, targeted suggestions are different for everyone. Which I personally find even more creepy.
yeah CGP grey's explanation is ridiculously spot on, it's crazy how seeing good the analogy is after watching 3b1b's series.
Grey knows how AI works, because grey knows how he was programmed. Also, how he pronounces "merry-yo" cracks me up
CGP Grey normally I'd be suspicious but since it's CGP Grey posting this link it must be legit
seems legit
No, no it's not.
Grey uploaded a video? Everyone, take your wishes! This only happens once per millenia.
how - why?? BOTTTTSSSSSSSS
I'm fine with that. I prefer quality over quantity
Did you just imply that Grey is a Jirachi? Cuz I'm totally down with that
Well what do you expect from a podcaster who only does UA-cam on the side?
+
0:11 How did you just pronounce Mario?
marry-0
you're like a german complaining about austrian dialect
may-rye-ah.
But, seriously, it's MAH-ree-oh
MEHRYO
*Mary O.*
dang i miss smm1
For those interested: 3Blue1Brown has an amazing 3 part series on this topic. I'm surprised Grey didn't mention it because it's really good.
Now he did.
Every over-simplification I didn't like in the main video was simply answered in this follow-up video. CGP Grey, you are the Master Explainer.
A good example I have seen for this is the MtG neural network generated cards. The dials have been tweaked to create a range of cards varying from "could've been made by the designers" to "not even close to plausible". I would definitely recommend checking them out
Maerio!? MAERIO?!?!? *I had respect for you*
0:11 .... it’s Mario.
I was looking for this comment. I too felt the pain when "maerio" was said
One of his signature catchphrases is him saying his name.
It's-a me, Mario!
It's pronounced Ma-ree-o
How do so may people get it wrong?
2:01
You understand, mechanical hands, are the future of everything
Very nice Footnote! Very good visualization of AI algorithms in the two videos as well!
As a machine learning researcher this was awesome, best simple explanation I've seen on the internet.
Valkyrix Github or Shutup
github.com/valkyrix/Deep-Red-Framework/blob/master/Dissertation/dissertationmain.pdf
On the subject of machines identifying things in pictures, my phone just randomly spat out a “memory” with many of my pictures of my cats and titled it “four legged friends”
...How many photos of your cats do you have? ...How many cats do you have?
LCNoir it was only about a dozen shots, mostly for sending to my sister when she has a bad day at work.
I am impressed with your skill to condense such complex systems and concepts into short entertaining videos.
I see what you did there Grey
My, my. He understands the algrorythm.
Yeey! The first (popular) video about artificial intelligence that is actually accurate about the methods used. Great job! Would have loved to see the animations for evolutionary training of networks, where the teacher bot makes the students brawl :)
00:10 "MARY-OH"
Nah. Nahnahnahnahnah. Mario doesn't deserve that, you monster.
Look at the Description
Someone Made an AI play Mario called Mar/IO
@@MarcTelang yes and it’s not pronounced that way either lol
CGP Grey this is what I’ve been working on for the last month (back propagation) and my friends always ask what I’m doing and how but I always have trouble explaining it. Thanks, now I can show them this video!
Wow I was literally complaining about your oversimplified and somewhat inaccurate metaphor of “bot factory” in your main video, and here it is, a perfectly explained footnote. For someone who still can’t do pointers, this is a surprisingly in-depth grasp of the neural network concept! I salute you!
If you are really interested in the details, I can highly recommend 3Blue1Brown's series on machine learning. He does a great job explaining the basic math behind deep learning.
How the tables turn...
WHAT IF THERE IS A BEE SHAPED AS A NUMBER 3??/ WILL THE MACHINE COMMIT SUICIDE BY REMOVING ITS BATTERIES?
No it will have an existential crisis leading it to be self aware, killing all mankind.
Machine: neither. Wait NO, three. No its a bee... hold on, isn't that a thr- I'm done.
Machine: NEITHER!!!
No, it will probably just get it wrong.
Games14159 r/woooosh
Blyzel oh no
I like this video much more than its parent video. More in-depth and more oriented towards today's tech.
5 years later ChatGPT is crazy.
*May the bots have mercy on us all.*
Very nice Star Wars VI quote at 0:13
thank you for footnotes on your videos. you are a wonderful teacher.
It's an older code, Sir, but it checks out... I was about to clear them
Thanks for this video! It's such a perfect addition because it's really the only thing a computer scientist would want to add after your first video.
Really appreciate the work you put in, love the footnotes.
A very good explanation of how NNs work. Thank you for this!
this feels like the epilogue to the story of how skynet came to be.
*prologue
Thank you for this footnote. This helped me understanding it much better. IMHO, this should have been the actual explanation.
blocki +1
+
Your video's story, graphics, animation, narration & production values are great,
I'am very glad you did this piece and eluded to the fact that neural nets is where the 'hotness' is.
The term 'Genetic Models' should have been mentioned in the first piece. It was the topic of the piece.
Since neural networks are where the opportunities are, I hope you consider redoing both pieces, and reverse the order.
.
Neural nets can be described in simple terms, and you could be the one to do it best.
I like the Star Wars quote “it’s an old code but it checks out”
0:13 - "A Genetic Code is an older code, but it still checks out."
I see what you did there, Grey...
Interestingly in retrospect, this video was originally published not long after the "Attention is All You Need" paper, which made recursive neural networks less relevant. ChatGPT uses Transformers instead.
The bot-autogenerated subtitles are fun. It thinks this video is in portugese.
I really liked the way you endded the main video.. A good and creative way of both emphasizing why people always ask "us" to do this, while at the same time getting us to do it.
Really enjoy your videos. Always look forward to seeing them in my subscription box.
Gonna watch this footnote before the main video.
deviaNT
Drew LeBow also watch “rules for rulers” followed by “humans need not apply” and ask yourself what happens if a ruler’s keys to power are mostly bots.
SOMEONE GET THIS MAN UNDER CONTROL
CGP Grey Fuck off
"An older Code but it checks out"
Subtle Star Wars references. I like them.
MAY THE BOTS HAVE MERCY ON US ALL
The pause at the end gave me the creeps.
That was super cool glad you added that footnote.
Who else has this video randomly pop up on their homepage, also ChatGPT 4.0 just released.
You mean GPT-4
OH MY GAWD HE ACTUALLY UPLOADED
I would recommend watching Code Bullet, he uses this stuff to make AIs that play games.
0:14 Star Wars reference? (pretty sure that line is in Return of The Jedi)
I’m really inspired right now, I’m going to go look into neural networks and hopefully won’t get overwhelmed immediately and give up
I love these videos,( and every other one of your videos) keep up the work!
Liked for the subtle Star Wars joke
Nope, at 0:13
Please explain for the uninitiated.
just google "older code but it checks out" :)
"subtle"
Nice
heyyyyy i didn't know you watched cgp, how's the new edit going?
I wonder what the bots think of memes
Oh, hey, what's up dad
Unexpected meetings
MEME LORD, WE PRAISE YOU
1:48 just take every frame of the video and check if it's a bee or a three. if the majority of the frames is a bee then it's a bee
"as compute power approaches 'crazy-pants' " 🤣
This makes me feel old. Crazy-pants have come and gone, we are ~ crazy-pants ^ 16...
In the words of the "play the machines" option in a machine uprising in Stellaris:
"It's too late for you, organics."
There is a great video on this on 3blue1brown
Yes, fantastic channel
HE'S ALIVE!
00:11 Can't unsee the puppy ears!
Fuck You.
I can’t see puppy ears...
Kevin Benoit The nintendo switch controller!
Snowleopardy j
I enjoyed the 10 second pause of suspense. Was expecting a jumpscare of the bot variety, but, it never came o_o
“It's an older code, but it still checks out." Shuttle Tydirium, you are clear to land.
1:05
Not for much longer!
holy shit, this is the best explanation of gradient descent I've ever heard.
SEES CGP GREY HAS UPLOADED A VID. *clicks on it immediately*
The dial adjustment bot adjusted the dials to give you CGP videos faster.
Me too ... You should check out the video's of Tom Scott too.
"Vid" has become quite the common term. Linguistics majors may have a topic for their PHDs ...
I appreciated the Star Wars reference at 0:13
This couldn't be more perfectly timed, trying to figure out how to make a trading bot and this was a pretty helpful starter
"Play Merio"
oreosmooshy Mair ee oh
marry-o
"Hans Stuck has collided with MERIO Andretti's Lotus!"
How interesting, yes!
“may the bots have mercy on us all”
wait what?
Cars still have radios, just not dials.
What car radio do you have? My parents have '16 SUV and it still has dials.
Gotta admit, that reference was so smooth I had not noticed it before.
1:13 "it's an older code but it still checks out" that's a Star wars fan thank you
"Mare-io"
Grey, you're killing me slowly.
Look at the Description
Someone Made an AI play Mario called Mar/IO
May the bots have mercy on us!
When people ask what I do, I will show them this video, nice work!
Can I just say the animation on these videos have been fantastic!
0:13 I see what you did there
Mare-io? And here I was thinking that you were an intellectual, Grey
well ua-cam.com/video/qv6UVOQ0F44/v-deo.html
That's probably due to Grey living in England for so long. From my perspective, Americans pronounce 'Mario' wrong
Bees or threes is probably gonna be a meme soon
...
Squid or kid?
Dick or stick?
Leeches or bitches?
Oh no. You guys have began listing off Japanese game shows, haven't you?
What do you mean?
"It's an older code but it checks out"
Very well played, CGP Grey. Well Played.
Love your vids man, fun to watch.
May the bots not have mercy on me, I'm looking forward to heaven ya' know
Felt that
I know this won't mean much but please don't die.
sorry to dissapoint but there is no heaven and death is probably temporary
@@nobodyinparticular968 just respect other religions/religion in general
@@trainmaster997 i cant respect religions at all
3b1b has an amazing explanation of how deep learning works if anyone's interested in digging in to some of the details. Still pretty basic compared to what's used in research today but a good place to start