Neural Network Learns to Play Snake
Вставка
- Опубліковано 3 лис 2018
- In this project I built a neural network and trained it to play Snake using a genetic algorithm.
Thanks for watching! Subscribe if you enjoyed and Share if you know anyone who would be interested!
GitHub Repo: github.com/greerviau/SnakeAI
Twitter: / greerviau
Support me on Patreon: / greerviau
Thanks to Josh Cominelli for the music!
Soundcloud: / josh-cominelli - Наука та технологія
You guys think that the snake died because of the lack of left turns, but in reality the snake evolved to the point where it got consciousness and understood that life dedicated to running in circles is not worth living.
No One ur joke is so dark, it darken my life
WoW
almost cut myself on all that edge
No One agreed
That's deep
2:26, that’s literally just a dog
literally
literally
Literally
literally
yllaretiL
ME: "Hello World">>20 errors found.
Funny: Funny! Funny ... ?
@@NoctumusTV what?
@@NoctumusTV Can you pls explain again. Thanks in advance.
@@NoctumusTV oh ok
@@marls3518 Explain what again?
My favorite part is every time you think the AI finally has it down, then runs into a wall for no reason
Every time
that's what God said, when watching humanity
there is a reason, the human element is a fuck up disgrace in this case
Maybe time for revolution
@@user84074 Then God killed the program
Same as humans)
For anyone who wonders why it seems to prefer right turns, I believe that is because it started at the top-left, going towards the right. There was no way for it to turn left. So with 2000 snakes per generation, a LOT of those learned that left is death. Since right worked every time, it simply had no reason to learn that turning left after leaving the wall would be safe.
I believe that is also why you got those wiggly motions. That's it trying to turn left, but then immediately turning right again, so its profile won't go any further to the right than the starting position.
It might be interesting to see what happens if the starting position is randomised
What about the middle ?
@@trex70 middle and just go down, snake will choose Left or right way by red points
About what I thought, but does this mean it is unable to get significantly better scores, because it will suicide inevitably by coiling up instead of folding itself?
@@badkingjohn5235 The most likely scenario is that it will discover that it can fold itself in that direction, which makes it survive for some time longer.
It will therefore take days, possibly even weeks, to simulate it to the point where it learns to fold in both directions, or fold and twist around.
I love that the reason it failed is because that's the one flaw of the technique it's honed from the start
Lock in. You hit on a successful strategy which gets you all the way to the point where it is no longer successful, but by then you can't do anything else. A typical failure mode these kinds of systems, from corporations to civilizations.
Benjamin Feddersen
While you’re correct that adaptation is a very necessary skill. Michaels point was more about BAD HABITS than an inability to adapt to new circumstance.
Bebolife A bad habit can inhibit adaption.
@@benjaminfeddersen7937 Dude this shit is deep as fuck.. It's the epistemological concept of paradigm. Any paradigm in order get surpassed need first to collapse on its own rules, unable to explain or resolve newer problems
There are no bad habits. Just bad outcomes.
I think another one reason why this doesn’t get higher because in input it gets -
1. Distance to food
2. Distance to wall
3. Distance to tail
Wait but what about its whole body ??
So that’s why snake trap around it’s own body.
Just a guess though 🤔
thought the same, but could the lenght just be another input neuron ?
We could probabaly include the previous outputs as an input like an LSTM or GRU
Perfectly correct. If you feed it the location of the whole body in terms of a matrix it will evolve to the point where its better than any human playing the game
Nice observation
Yup, using a CNN would be a good approach for this problem, I think. Use a different weights for the head, tail and the location of food.
Humans: *computers will take over the world and destroy us all*
Computer: *hehe line go zoom*
not funny didn’t laugh
My disappointment is immeasurable and my day is ruined
snake go brrr
@@mesq999 And this is why no one likes you at school
@@nahimafing Just because he has an opinion slightly different than your opinion, it means no one likes him? You are a fucking asshole
Left turns: *Am I a joke to you?*
I observed the same thing; is that a design flaw?
It's not an ambiturner.
Neural networks is a lazy algorithm and will take the shortest route to achieve its goal. My guess is that the input of the distance from the left wall from the snake plays a significant importance to its decision making.
You can use dropout which will force other nodes to train that never gets a chance when the whole network gets trained
@@pocketrocket27 God damn you Ivan - you beat me to it. Damn you to hell :D
but it took left turns
This neural network is incredibly inefficient. Right from the beginning, it learned to not turn left by any means. This video is perfect as a demonstration that neural networks can easily get stuck on a very wrong local optimum.
bit of an analogous for humanity, don't you think?
It's not inefficient - it has an energy cost of 0, there's nothing to constrain it's time. If there was an energy cost (negative reward function) for turning, it would optimize its routes.
@@manzell good point!
@@superpantman not really, as humans use a mixture on neural networks and symbol manipulation. That’s why AI (recently overly focused on association and deep learning) are not progressing as fast as hoped.
@@manzell And perhaps adding in more genetic variances from generation to generation to allow novel ideas to die or take hold.
I can't imagine how happy would be the first guy who developed these algorithms.... ❤️❤️
Yes
Yes
yes
Yes
We
Generation 30: *dies*
Me: YOU WERE THE CHOSEN ONE
I have deeply learned that in the end, nothing is left.
You're right
Lmao
This is hilarious
But don't massacre your clan in future
@@re_claimer_ how about you go watch shippuden? clearly you dont know shit
congratulations for the big work you've done, not only at the algorithmic part, but the visual part which i can see it's a huge effort to present us your job.
What I think is most fascinating about this project is that the neural network never learned the dimensions of the game board and kept returning to the start
"What is my purpose ?"
"You pass butter...."
I think it probably would have learned better if you had started off with a lower number of moves left (maybe like 60?) so that it doesn't have so much security to take its time.
That is definitely possible
@@GreerViau Also, If you want them to evolve how to avoid hitting themselves better try making the map small so that they encounter tat problem sooner
@@arthurfacredyn That's especially true if the improvement yield was already capped, with a lot of room still available around.
You could also add a small negative score for each frame, so that it prefers to die rather than do nothing, but it might get stuck in a local optimum of immediately killing itself.
@@arthurfacredyn Or making the snake longer right at the beginning so it can develop strategies for avoiding his body
The mind of the Snake in the first few generations, spinning to infinity a pixel away from the food
"FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD FOOD"
sciencesyfy this actually made me laugh and not just breathe fast out of my nose, gg
I laughed at this
6:07 love how the snake eating the food is perfectly synced up to the songs snare until around 6:22
lmfaoo thats pretty cool
Lol neato
The snake evolved into being able to understand the music
actually also at the start of gen 30 (from around 5:00 onwards) it’s synced up in some ways
😂😂😂
No one:
UA-cam when my lil brother uses Wi-Fi 1:22
he must be downloading extra ram
@@Faisalamin01 no he was downloading graphics card 😂😂😁
It takes a few generations for any significant progress to be made
It's so fascinating to look at a neural network learn and it be visualized, it's like a mini brain in a computer learning and reacting to their surroundings, telling a machine that only follows orders to figure it out themself
Could you make a video where you explain your code an how you determined fitness and the mutation and crossing over procedure?
I looked at the code a little and while I don't know the language, most of it is rather simple.
The weights are stored using a self-written matrix class, which is a 2-dimensional array with a few methods to do matrix stuff and for mutating and crossover.
Mutating adds some random gaussian noise to every weight. You can look that up in the github repo in the file Matrix.pde
The crossover method selects a random coordinate inside the matrix. Anything above or left of that coordinate uses the values from partner A, anything below or to the right of that coordinate uses the values from partner B.
The fitness is just the length of all the snakes in a generation added up.
I learned this kind of stuff in university and this project goes against a lot of what I learned. For a practical application, these functions would be pretty bad and most importantly, very slow. But the whole thing still works very well, so well in fact that without knowledge of the subject, most people wouldn't be able to tell it apart from a more professional approach.
It shows that machine learning isn't hard on its own, but the tools that are used nowadays are pretty complex.
@@nottheengineer4957 in which program or app can I do these kind of stuff?
@@mauriciomontalvo5885 Well u can use any programming language i presume, though some are better than others for these kind of things. If u want to hard code it yourself i would use something fast, but you won't likely achieve great performance unless u really know how to optimize the hell out of it. What you can do is use NEAT or tensorflow for example in python. Combined with pygame you could do all kinds of things like this. NEAT is extremely easy to use, to the point that you barely have to understand what is going on.
@@nottheengineer4957 Where to learn about more professional ways and tools they use? Just for curiosity and learning purposes (obviously without getting into uni, too old and too broke now for that).
Hi Kant☺️👋
The song works so well with this video. I am feeling so calm right now lol.
This made me genuinely happy, thnx for posting
I would like to see this but also with an adversarial neural network placing the next food piece.
or two snakes, each racing for the food
@@JohnSmith-xf6nb I feel like allowing it to change size would result in it shrinking the board as small as it can to reduce the number of points available.
@@JohnSmith-xf6nb I think that might go to far the other way, because a bigger board would mean less points per apple. Maybe if the board is smaller than whatever the "standard" is, then the points awarded increases in proportion to the number of points lost?
If you're trying to add a new thing for the adversarial network to do to try and mess up the main one maybe it could also spawn "bad" apples that either kill the snake or remove points. I think that would be interesting because then the snake couldn't always just navigate directly to the apple, it might need to avoid something it it's way and the adversary could try to place them in choke points and such.
Theyd get further with more information. You forgot a key piece. Direction of "motion" of its tail. While not immediately obvious in game its something human players take full advantage of when they get stuck on inner loops
becominghuman.ai/designing-ai-solving-snake-with-evolution-f3dd6a9da867
Do you think it would perform better if the input to the network was the grid array containing all the information about the game state. eg a 50x50 array of numbers 0, for empty, 1 for snake body and 2 for food. Or is it better to explicitly tell it the distance from the food, is it unlikely to work it out itself?
@@dananderson8459 using convolutional layers instead of fully connected layers probably yes, otherwise probably only with a significantly larger network
what if there was one value for head position, one for head direction, one for food position and a vector for the entire body
I'm a noob but I think it could do very well with this
if the network also used some recurrent design (such as an LSTM) it could possibly compute motion and have better planning abilities
Thank you so much for making this educational video! Well done! We are so grateful
More intense than any latest action movie fight scenes :D Respect!
Nice visualization combination of the neural network firing and its effect.
Let's say I also want to create such visualisation, how should I do it?
This is what happens when you don’t consider the “time-to-solution” in your fitness algorithm!
That's exactly what I was thinking, along with the fact that the player usually is not the snake, so there should be a couple of input neurons more with the position x-y of the food
@@yurimrt Yes and the cartesian distance to the food sqrt((Xsnake-Xfruit)²+(Ysnake-Yfruit)²)
@@uwu_senpai yeah but that works only for whne the snake itself is not blocking the path, there needs to be a priority set that it just need to find the shortest path to is next objective, like going out of the block by the snake which can be obtained by looking if the snake is on the x way and the y way to the food and if it is look for the shortest path possible for that not to happen.
@@uwu_senpai Euclidian? Sounds like a bad idea as you cannot reach the fruit in less than |xsnake-xfruit| + |ysnake-yfruit| ticks
@@mirabilis but you know if there the snake doesn't block that path, it's the fastest way possible, there is no faster way, it's just math.
I love how it likes to return to top left before making next manoeuvre, shows the training
A quick suggestion: don't constrain the neural net so much. Give it the entire 38 by 38 grid with three possible values for each location (off, snake, apple) and train using those inputs. It can even be considered a vision problem at that point, and modern ML libraries can solve it with a convolutional neural net pretty effectively.
Wouldn't that be far, far more computationally intensive? Genuinely asking
@@Caffeine_Addict_2020 not really, considering modern hardware can comfortably run CNNs on proper images, 32x32 grid of pixels is nothing
@@Caffeine_Addict_2020 relative to this model? Yeah. But it still wouldn't run slow on modern hardware by any means
This is so amazing. Next topic to learn - Genetic Algorithms
Heres to where youtube recommendations lead me to today during quarantine :D
Wow awesome stuff man!
Wow! Great video! ❤️ Neural networks rules! 👍🤓
2:31 me when i play tetris and i know im gonna lose
UA-cam Algorithm: Dis looks guud, lemme recommend it to everyone
It would be interesting to see colors for the hidden layer nodes as well, colored for their activation level
Plus a gradient for the weights instead of just blue/red
AWESOME man this was so awesome !!!
Anyone else deeply in love with the first song ? It’s so calm and nostalgic
Awesome! Now make it two AI-players: Your snake vs AI that places the food with the opposite target: Reward if the snake dies. That would be an interesting experiment :)
i just love that part when the 30th is synced with the music as is turning on walls
At 2:20 starts feeling like I'm watching a movie about a guy who was weak at the beginning but he starts training more and more despite his failures and finally he comes to success
You might have gotten better results had you let it start from the middle or from different places every time ^^
Great video! I glad more people are taking interest in neural networks
I'd have said that the problem might be that he is selecting only the best out of the 2000 snakes. That leads to a strategy which is only a local maximum. That's also the reason he doesn't get better results by training further. It's hard to get out of that when you don't allow the chance of exploring other strategies which are not locally the best. His population was too small and the mutation rate too low to fix this issue. You'd probably get better results by selecting a small group of snakes with equally distributed fitness.
I mean, this then adds an "RNG" variable, which you really don't want, no? A snake may have better fitness because it got a lucky placement, and you don't want to breed for luck because that will be completely random
People: omg ai is going to dominate the entire world
Meanwhile, the AI: gonna go get max scoring in snake
This comment hasn’t aged well :))
Beautiful story. Never give up!
This is mind blowing!!
Great work @Greer Viau
One way it can avoid this kind of a death is if the locations of all the pixels of the snake are given as input to the neural network and not just the start and the end.
You can see that it learns that when the head is just above the tail (or above and diagonally left of the tail), then it has to go right to avoid eating itself. But, when it gets stuck in a loop of its own body, it does not know that it's body is there. So, it would become very difficult for it to learn that it should avoid its own body when the tail is far away.
The strategy backfired if it becomes long enough, a rule telling it should calculate its length vs the size of the field before making a move should be applied
You should nickname your snake Derek Zoolander because it appears to struggle to turn left
it doesnt
@@user-kx5es4kr4x it really does
Underrated comment
I wish you a speedy recovery and hope all god will makes everything goes well for you.
This is the first video I saw of you!
The limiting factor is the input vector IMO. If the snake operates only on relative distances then no matter what - it'll end up encircling itself and getting stuck.
this is the coolest thing i have seen in my entire 18 years of existence
At the end it felt like the snake was synced up with the music and dancing along that was pretty groovy
Nice work bro
In such simple networks, the encoding of inputs can make all the difference. Representing distance in some sort of a grey or logarithmic code may be worth a try to speed things up :)
Thank you, your code help me understand well about the AI. I am a newbie :
This is oddly philosophical. No matter how much we advance, we will keep progressing, all while securing our own downfall.
The music is so nice!
5:48 it's starts eating the red dot on the beat
6:17 begins the killer moves to the beat 😂😂
Samir, you are breaking the snake.
Samir, you are not listening !
What the deuce !!
Who are you ?
@@stewiegriffin6503 that's what I am supposed to ask. Who are you! And why do we look same
Shut up, dont tell me who to drive
Looks like Stewie has been messing with the time machine again.
The perceptrons glowing is really cool and I don't know why
I think using some form of DeepRL coupled with CompVision could yield great results for Snake. Of course, for such a simple game you could skip the CV component, but I feel it'd be more fun that way. Also to avoid biasing, you could perhaps pick a random spot as a starting point.
The playful ones are especially cute OML XD
Given your input layers it makes sense that it started to struggle when the worm got to big. It doesn't have the input layers to detect spatial availability like that.
Wow! Really cool! Thank you
1st generation : I'm hungry
30th generation : solved the hunger problem
100th generation : discover the network
500th generation : taking over the network
1000th generation : human extension.
*extinction
@@vibinv8905
let it go man 😂😂
@@adomustafa1777 The OCD just took over :D
@@vibinv8905 Butt in your moment of "OCD" did you notice the choice ?
You see even though it might not feel like it (regardless if you have this so called OCD or not) there is always a moment where you have the choice. The thing that told you that you wanted to correct him is simply an impulse and you have complete control over your impulses. It no longer works to say oh blame it on my OCD because YOU chose to listen to it. Whatever reason you have for making the decision, it always comes down to you.
A habit is just a choice you keep making.
@@PsychoBackflip thanks for the pep talk.
Thanks man!
Your example is absolutely beautiful. Most AI/ML courses are missing this stuff. It should be taught before moving on with Tensorflow and other high level libraries
It really loves that top left corner
*Me:* * _sees the thumbnails_ *
*"Wait, That's illegal!"*
Hello, amazing video, thoroughly enjoyable. I'm very interested in starting to program stuff like this, can you point me what direction I should go to start learning to write programs like these? I already have myself familiar with both neural networks and a few optimization algorithms but this program seems to be a mix of both of them., since it doesn't really have any training data and relies on generations and random behavior to train the neural network.
nice opening track. really enjoyed it.
Nice work fella
Я всё ждал, что нейронная сеть будет управлять змейкой по оптимальному и короткому пути, в том числе по диагонали! 😃
В конечном итоге я дождался другого, когда нейронная сеть будет проигрывать из-за столкновения змеи об саму себя. 😅
на самом деле это оптимальный вариант движения по кругу, т.к. змейка может быть ограничена только размером карты, движения по диагонали уменьшает свободную площадь от 10 до 50%.
привет от диванных РУвойск
__
вот вот_ чето автор логику игры не допилил _ когда змейка заходит во внутринний круг то конец сразу __ хотя может лишние проверки.. а нужна была производительсть .. хотя... хотя...
the snake consistantley modes clockwise.
Amazing work
it's so facinating to see that even such a simple network is capable to evolve to solve this task. What can 1000 times more complex one do?
zoolander bot only turns left
At last he did it!
You mean right. It only turns right. But on the bright side, it is RIDICULOUSLY good looking.
And eat only from top to bottom.
Reinforcement learning (RL) can lead to better results in lower time, compared to using genetic algorithms. Google, openai and other research teams, like the one i'm part of (RoboCIn) are using RL to play soccer, dota, starcraft...
Great initiative to solve the problem, make a video and share the code! 👏👏👏
wow cool ,are codes public for that?
this video helps me to understand the nn better.thank you
its amazes me to watch this an inanimate being learning to survive through experience, just like humans, it's incredible to witness it live
2:39 - The snake has evolved into a dog.
Woah, great visualization
Can you name-drop some of the tools used to create this?
ㅇㄷ
There aren’t any “tools”. You should start by learning about neural networks and deep learning, and try out a few simple networks to learn how to program them. Once you’ve got a grasp of neural network programming you can pretty much adapt them for any problem, and expand the hidden layers and neurona where necessary.
@@26dimensions70 Thanks for replying!
I am in fact a computer science graduate, and I also have a degree in Industrial Design.
My question was about the visualization tools you used to produce the video, I'm fascinated by the animations and would love to learn how to produce similar demos of my own ML research :)
You're welcome to take a look at one of my projects (an artificial intelligence constructing objects from a 'LEGO' like building block I designed) where I used Python's mplot3d to create a set of images I converted to animated gifs to visualize the algorithm.
www.razkarl.com/projects/kawaz
@@razkarl Take a look at OpenAI gym. It is a virtual environment used for reinforcement learning
The start of generation 17 plus the music makes it seem like it’s a main character in a superhero coming of age film and it’s gotten the hang of its powers so well it can do cool tricks
6:27 The snake is so long that it forms an enclosed space, and the new prey is outside. There is a way to use the inner space to get out of the narrow gap.
It`s cool to see how this is going :D
But sad to say that I`ve played way more rounds and never get a better score than 7 :(
😂
what software do you use to display the state of the network?
I think it's all done in Processing but not 100% sure
btw, awesome job mate!!
Holy shit! This is amazing
I don't know if you observed or not but snake is doing clockwise rotation most of the time
Basically I believe that the snake is using the wall as a map, the neural network doesn't know where it is on the screen, only distance to wall, tail, and food, so it travels around the edge because it's a significant boundary, then when it gets the closest to the food, it travels in a straight line until it meets another wall, with some small differences in between depending on distance to tail.
The snake is Republican
@@MybeautifulandamazingPrincess lmao xD
Optimal strategy often is not the entertaining one.
Interesting question. Could be random. A successful generation introduced it randomly. Or it has some deeper sense. More Galaxies are rotating counter clockwise.
Wonderful demonstration! To train a condescending, plural-array You’ll always need the “imaginary side-node”. It’s a fictional response that always concurs with the ideal national response. In this case, the network must revolve first left then no other direction, or vice-versa. The side-imagined node will condescend any alternate output, here. You’ll see the snake “win”, but you could’ve done that simply. A “national, variance-norm” sweeping network is not about the output but the internal shape- it’s a complex geometry very easily, sending product information all over! Imagine a space-station inside a server-network on Earth like this! Take care, My Child.
@Dale Owens A “neural network”, including the human brain, is only lightly about the output, and much, much more about the path.
Specific to “winning”, I also said, the ideal “snake game” behavior is simple: turn only left or right, and so any AI playing snake (or anything else either) doesn’t need more than a ruleset to always win.
I’m a Researcher, having a PhD in Game Theory, Science of Matt-Brainology, and another, more.
I'd have said that the problem might be that he is selecting only the best out of the 2000 snakes. That leads to a strategy which is only a local maximum. That's also the reason he doesn't get better results by training further. It's hard to get out of that when you don't allow the chance of exploring other strategies which are not locally the best. His population was too small and the mutation rate too low to fix this issue. You'd probably get better results by selecting a small group of snakes with equally distributed fitness.
At least he is trying a evolutional algorithmic approach from computational intelligence field. I totally missed that he forgot the bias node.
Of course there are several ways to solve that snake game problem. His approach is not useless though.
Super vidéo ! Parfait pour un pic nic entre potes pour se relaxer. MERCI
Awesome trained snake :D
Pleas make explaination for control kamikaze drone next!
Terminator: ....SnakeNet begins to learn at a geometric rate. It becomes self-aware at 2:14 AM, Eastern time, August 29th. In a panic, they try to pull the plug.
really enjoyed this
This is stunning.
The 30x speed felt like I was playing osu for some reason...
*DADADADADADADADA INTENSIFIES*
this isn't really a great way to train a network, but it does get better, just very slowly compared to using backpropageation and natural deduction. these would improve the learning rate as well as extend the scope of its intelligence.