CPUs troll AI with Items | Mario Kart Wii
Вставка
- Опубліковано 19 чер 2024
- #ai #mariokart #reinforcementlearning
By popular demand, here an AI will learn to play Mario Kart Wii with Items ENABLED! The AI in this video will be using a variant of the Deep Reinforcement Learning algorithm Rainbow DQN, to teach itself how to play the game with no prior knowledge, and no human input.
0:00 Intro
2:50 Training Starts
7:20 Final AI
If you are a C++ developer please help on this GitHub, as this is what I use to allow AI to interact with the game! I am currently looking for someone to add functionality for the Nunchuck and accelerometer, as this would allow me to play other games like Super Mario Galaxy. Any improvements (especially speed) would be very much appreciated though!
github.com/Felk/dolphin/issue...
Also thank you to my editor for editing this video!
@benji.botterill
www.benji-bott.com
Music:
Tiger Tracks - Lexica
Fights - AGST
Through the Prism - Ava Low
Backseat rider - Lupus Nocte - Наука та технологія
I'm amazed to see that the AIs have gone from just self-sabotaging for speed to learning how to bag efficiently
The AI in this video received drastically less reward for really high speeds compared to the other video where the AI sandbagged, but it still didn't seem to solve it haha. I guess it worked a little though as you mentioned since it didn't completely self-destruct!
@@aitangoMaybe teach the ai some basic strategies to evolve faster including taking shortcut with mushroom or when to turn/drift etc. You can also maybe cut down the unnecessary visual of the screen in order to maybe give him colour and higher resolution, including the lap and Time count.
the ai already knows when to turn and when to drift lol ?@@QnjtGWonQNqVsbYyzjx4
@@aitangoI don't think the sandbagging is a reward problem its a lack of opponent difficulty problem. Assuming you are giving it more reward for being closer to first place, the fact that the CPU opponents don't pose enough of a challenge means the only way to get more reward when you know you can beat the CPUs with driving skills alone is to fish for items.
@@Jackpkmn Smart observation
The fact the ai tend to pickup tactics that actual people sometimes use always interests me, that and seeing them forced into weird situations and still make their way out of them. Honestly plenty of ideas to be had even just for Mario Kart.
I love seeing AIs find tactics humans use. I remember years ago a project called AlphaZero which learned chess, and while playing it would learn human openings and then discard the ones it thought were bad haha
I think a potentially better way to give the AI visual input would be to palletize the screen - that is, instead of reducing it to black and white, reduce it to say, a 32- or 64-color palette, so the AI just receives the color index of each pixel. That way it'd be easier for it to tell the difference between offroad and road, since they're usually very distinct colors, even if they're about the same shade. Although I could see palletizing every frame could be a bit more computationally expensive.
On another note, as you may be starting to recognize me as being more critical, I want to start by saying don't take it personally obviously. I do enjoy seeing what neural networks and machine learning is capable of, and these videos ARE entertaining. The criticism I have for today is the stock footage used in the intros of each video. It's not really useful for the video, and it's often the same clips used in previous videos, which can get a bit confusing, perhaps making people second guess whether they've already watched this particular video. I don't really have a proposed better solution, but I think just having you talk over gameplay rather than random stock footage would probably work fine.
Also detect bananas
Also might help with the bananas
I really love the graph showing the AI's decision making in real-time and how confident it is in its decisions!
Glad you like it, I wasn't too sure how many people actually look at it! I like to include it though, just to give some idea of what the AI is "thinking"
@@aitango as a data analyst, I watch multiple parts of the videos two times for that reason
One idea is, train an AI off using just no items with CPUs alone, and then add items while keeping the values for other actions the same. So it will know how to drive and can then deal with items
That's actually what I did in a previous video! (This AI went WILD) That works pretty well, the only issue is that I have to predetermine the number of actions before I create the AI, so I would have to give it an action for using items that it wouldn't be able to use for the first part. Another options I've been considering is training it in both situations at once simultaneously, but I really have no idea what effect that will have
Maybe you could add rewards for time spent in a high position and items in their proximity being destroyed
@@aitangoYou could concurrently run a driving model and weapons handling model using the current convolutional input setup, but give each of them an additional input set with the other networks options and current preferred action
@@aitango you could use weight masking so that the model contains the weights and outputs needed for item usage, but won't actually train or use those weights for inference until the second phase.
The fact the blooper actually did its job correctly at 5:20 is just incredible
This gota be good
It's awesome to see this creator get some traction. I've been a fan for a while now but I'm happy the previous video got ~900k views
As far as the input data to the AI goes, I somewhat recall that one semi-common strategy is to train an autoencoder on gameplay footage, and then use that autoencoder's encoded representation as input.
Another cool thing that the AI was doing is dodging POWs
Yeah in lots of the videos it picked that up pretty quickly! I guess its just something that's easy to do "accidentally", and then it can learn to do it on purpose
@@aitango Also thanks for reporting what the model takes as input, in the other video there was a debate about it ahah
Oh yeah probably best to settle that haha
lets go more mario kart ai!! i love these videos havent watched yet but ik itll be a certified BANGER
It's super fascinating to see the AI actually begin to adapt to item usage and even begin to overcome the blooper, which I always assumed would just be an insurmountable obstacle to the AI due to how it's designed to take in information, a number of things I'd suggest:
1) Increase the AI's viewpoint resolution, as you mentioned in the video the AI seemed to be severely handicapped by its available resolution and this made it very difficult for it to identify and avoid things such as bananas on the track even as it learned to do so, I don't believe this'd take a large upscaling and perhaps you could sidestep this issue by simply taking a snippet of the middle of the screen, upsizing it and moving it off to the side (Perhaps the top right corner since the AI doesn't seem to need to be able to view its lap count and the race timer) so the AI has a sort of "magnifying glass" it can reference for things further down the track without increasing the overall number of pixels the AI has to process, not really sure how well this would work through.
2) Give the AI a limited number of coloured pixels over its item box so that it can distinguish between green and red shells, this would give it a better idea of how these items are separate and what they both do, since right now it views them both as the same item due to their sprites being otherwise identical.
3) Change the item input so that it's a toggle hold button instead, firing the input once causes the AI to hold the current item behind itself and firing it again causes the AI to release that item, allowing it to defend itself from items and still use them without increasing the overall number of inputs the neutral network needs to process, although the AI would need to adapt to this process of using items and it'd probably have to learn the difference between when an item is being held behind itself and when it's not, so it might lengthen the learning process a bit.
4) This probably isn't practical either way but I figured I'd throw this in the suggestions regardless; giving the AI a reward for hitting an enemy with one of its items would massively improve its item usage by outright incentivising it to use and hit enemies with items, humans use the cheer sound the driver makes to know when they've hit someone, but obviously training the AI on sounds would exponentially increase the scope of this well beyond the realm of reality, so you'd probably need to figure out some way to access MKWii's internal variables to see when this cheer sound is playing and then wire that back to the AI.
I really enjoyed this video, can't wait to see more content!
Thank you very much for the thoughtful comment! Increasing the AI's resolution is something I'm actually currently looking to do, however it does present some challenges to do this without massively tanking the training speed, but its a work in progress. Finding a way to distinguish between shells would be very useful; as the AI is setup only to accept black and white, but there may be a way that rgb can be converted to grayscale such that the resulting image appears differently. Changing item usage to a toggle will be done for sure, as this is pretty critical. I think I could track down the variable for hitting a CPU with an item, however the main problem I have is that every new track I play on will have a different memory address for that value, so it means finding it over and over again if I wanted to do multiple tracks (hint hint)
@@aitangoI was wondering that instead of increasing the resolution as it will increase the weights and make the model run much more slowly to instead input RGB frames at the same resolution, this will only triplicate the weights of the first conv layer leaving the rest of the architecture identical and shouldn't slow down the model too much. Using RGB frames of course makes it much easier for the model to discern bananas as they are a yellow thing on a gray background (the road), instead of being a shade of gray on another shade of gray. It will also fix the problem with the shells.
@@aitango (Red+Blue*255+Green*255*255)/16581375
Would give you "all" colors compressed into 256, reducing the amount of color fidelity by an extreme amount.
Not the most efficient approach for sure, but would be fast even in a shader.
The worries are though it will lose a different level of detail that is important.
You could also consider giving it "FoV" detail differences like humans, with incrementally more detailed vision the closer to the center.
What if you took all of your Mario cart AI's and put them all in a race against each other
That would be utter mayhem haha. I'm hoping to set something like that up at some point, but its definitely going to take a little while!
The AI learning to play semi competitively is completely unexpected imo
Your videos are truly amazing! As someone who is striving to learn about neural networks and machine learning, you manage to make the subject incredibly engaging and enjoyable. I wouldn't mind if you dedicated a small section of your videos to provide a more in-depth explanation of how to create an AI for a game. However, I understand that not everyone may share my interest in this topic.
I'm really glad you like the videos! In future videos would like to go a little more in technical depth, however I'm trying to figure out how to do so while still keeping lots of people interested!
I really enjoy even just listening to how you set up the AI and watching it learn. It's also really cool to hear your retrospectives on what could have been improved to help the AI be better (like higher resolution might resolve the banana issue). Gives my a ton of interesting insight into creating AI's / neural networks. 😊
You deserve so much more attention, I found one video and basically binged your whole channel. This video kicked ass, Tango.
Really glad you enjoyed the video! I'm still blown away by how well my last video did, and its been great to see so many people explore the rest of my channel after watching that video!
great content. i am interested in this kind of stuff (as i believe alot of people are), and id even be interested in pure learning videoes regarding the neural network. Seeing a combination of games and teaching makes this very easy to consume and also fun to watch
Really glad you like it! I'm trying to find a good balance between explanation and watching the AI learn! I think in each video I'm going to try and discuss something new about how the whole thing works to keep it new and interesting!
Would be really nice to see a behind the scenes on how you are running this training and setup. Also tools and other things needed if someone would like to try it out on its own. Great video.
awesome video keep up the great work!
Thanks so much, really glad you enjoyed!
great video I love all of your work since the last year
Considering the issues the AI has perceiving bananas, you could use a two-channel image stream (luma aka brightness and chroma aka hue) instead of a single channel image stream (grayscale) so it could more easily identify the distinct yellow hue of a banana.
You could also consider training the AI in two separate pieces, a visual classifier/object tracker and the actual gaming agent. The object tracker simply identifies what's on the screen and where they are, using supervised learning. The gaming agent takes the interpretations of the object tracker and decides what controller inputs to use, using reinforcement learning as you've done here.
Well said
Ah yes, I loved the last items AI video you did, I’m glad you released this one too. Keep up the good work!! 🎉
The AI reacted to the blooper the way Nintendo thought people would.
great video, great editing, keep it up
Can't wait for the next one! Keep it up man!
Your editing and narration got even better !
Congrata on the great video
You are making me wanna get onto smaller scale silly AI projects again, and I love you for it. Keep up the beautiful AI stuff my friend!
Neat analysis video! Thanks for uploading!
Finally, someone who uses visual input in these things
I think letting the AI read from memory is cheating haha
Love the content. Can’t wait for more.
Love your videos! Glad I found your channel. One small bit of feedback-watching your last couple videos I wish the music was a bit quieter compared to your narration. For me at least it makes it harder to understand what you are saying.
Fun fact: the circuit used is a mariokart version of the Monza Circuit, albeit without the chicanes and a banked Parabolica
your videos are awesome ! keep it up !!!
Great job! I have a suggestion for the next AI, though I'm not sure if it would work: a reward for being in first place. This might negate the reward it gets for bagging, and encourage it to actually win the race faster.
Honestly wasn’t expecting this, all those calculations and RNG calls that has to be made are insane! Surely this is the way forward for future games!
I'd really love to see this type of AI used in future games for people to play against, as I think it would provide such interesting and dynamic gameplay! Obviously the purpose of AI in games isn't to obliterate the human player, but Id be interested to see what could be done by taking different versions of the AI throughout training to vary the difficulty
So much awesome. Thank you again.
Next I’d love to see the AI trained on a multitude of tracks and see if it can win a full grand prix
love your work man
Really neat video, im would be curious to see adverserial networks battle 1v1, although it would probably take a lot of compute
Im exited!
I love these vids, just wished they’d be longer.
oh hell yeah, can't WAIT!
My new favorite channel ❤
Wow, I did not realise that your last video got so many views. Congratulations on your efforts finally paying off after a longer absence :)
Thank you very much!
Nice video! In my opinion would be much interesting i you did a livestream wich shows all the process that takes you to create an ai like this
Thanks! I did a livestream a while ago of the training process, but I’ve never done one for the actual code development. I never thought me programming for hours would be that interesting haha
Amazing video❤🎉
These are great! I wonder if you could overlay the minimaps to give an idea of how the Ai progresses over time? It could be cool to have that to show how far the first AIs made it in the time it took the final Ai to finish the course
I don't know anything about Mario Kart Wii's memory, but would there be a memory address you could read that would determine how far around the track you are? I know there's invisible checkpoints you have to hit so that the game correctly counts laps (mostly to prevent you from cheating), but a "you are X units away from hitting the next checkpoint" could help with knowing when to reset.
There might also be a memory address that determines if the player has been hit by an item or not. That could also help the training program determine if it's not making progress due to shit driving, or not making progress because it's being smashed with items. That could allow faster resets to squeeze in more training.
And of course if you want to cheat, you could just feed those to the AI so it'd "know" without having to analyze the screen to work out what its current state is.
But these sorts of videos are always fascinating, especially yours. I tried playing around with RNN with Mario World on the SNES. After a bunch of hours of training, it could finish the first level, but as soon as it hit Yoshi's Island 3 (where it would have to move upwards before it could move forwards), it just got stuck endlessly, but I still had a blast doing it.
Part of the learning process is figuring out what items do and learning how and when to use items to get a new one. Those are important skills for competitive racing.
So the AI isn’t just butchering the humans and disposing it’s learning how to deal with unexpected issues such as human resistance.
AI overlords we salute you.
This channel will blow up.
I like to think that the AI is actually having fun sandbagging. Basically, like any of us having fun, there's an activity that gives it a reward in its brain and it just focuses on doing that, even if it doesn't win
holy moly its a sequel, most tubers would have just not done anything for a year and then redo the original video and play it off as a sequel
I’d love to see how this AI performs on a track it’s never seen before, if it would even work at all. Great video!
Coming Soon...
@@aitangolet me guess: teaching it to identify what could be considered a road so it can be more adaptable.
I would love to see how the AI behaves when given more control over items! great video man
Really glad you liked it! When I do another video that uses items, I'll come up with an improved system for sure!
I can't wait for him to blow up so he can run three NASA 4090ti supercomputers in a chilled server and simulate an AI beating a Mario Kart online match.
this is amazing 🎉
I’m so excited
Fantastic video as always! I'd be curious to see how far an AI could learn before it plateau's if it were to race against other similar AI's, with rewards being given to the winners. With compeition on the same level as itself, it'd be forced to learn to play even better in order to come in first, yeah? ^^
Amazing Vid fr 👏
Glad you enjoyed!
8:46 THE AI IS BAGGING!!!!!!!!!!
Wow this video was a great follow up id love to see if an ai could adapt to the battle stages
I like how the ai at the beginning is like on the off-road and just being (probably) happy about being able to drive on the off-road and not being punished for that
After my other videos where it get instantly deleted for doing that, I imagine it must be a relief! That said, its amazing how much it slows down training though, I guess harsh teaching has its advantages
@@aitango yeah, but let the ai sometime just mess around and keep it often just in the video! because it can sometimes show how every ai has its own will when doing that!
Now make him compete in the cups on 150cc!!
Perhaps multiple tracks will be coming soon... :)
To prove to people saying it just memorizes tracks try to make it play a different track after it’s done with training (maybe a similar track visually though)
Love your mkwii content
This is awesome but maybe you could give the ai a reward if it hits a cpu or player with an item (bomb, shells, bananas etc.)
Amazing video
You should try randomizing the tracks after every attempt and leave it running for a very long time to create an ai that can play all of the tracks
I may not be too experienced in the field of AI, but I think it might be a good idea to give it a small reward every frame based on its position. I think this would help it learn to efficiently use its items to sabotage opponents.
The music was way too loud Bro, anyways nice vid! Love it
i wonder if you could train an AI on MKwii world records, and try to see if it could match those lines? itd be really cool to see! your content is really cool man
First we got AI learning to play sports, now we get AI learning to play MarioKart. This is amazing.
All that skill and awesome work all these people do. Incredible.
Thank you so much, always amazing to hear such kind support!
I'd love to see an AI that just has the exact same controls that a human would have while playing. I know it would be a lengthy training process but itd be amazing
It would be interesting to try to help the AI changing a bit the architecture of the network to help it structure the data on screen better. Instead of feeding it just the raw pixel footage, you can use simple scripts or networks to more accurately and less effortlessly (in terms of computation required) give it a better idea of what's happening on screen. Some of the systems that may be insteresting to implement are:
- Items: decide if it has an item (and which one it is) or if it's in the process of getting an item. With the 'rolling an item', you may be able to start seeing the AI tap the item button to get the item earlier (maybe even slightly reward it).
- Item alerts: whether it is a red shell or a bullet bill coming for the AI, it doesn't have much time to accurately learn about how to handle the alerts it gets. Implementing a system that recognizes and classifies the alerts, then feeds the AI their position, type, time/frames since it appeared on screen and time/frames since it disappeared (the moment you want to take out your banana to block a red shell is the moment the alert disappears and shows on screen, as you didn't make the AI able to hold an item behind itself).
- Position: it is rather easy to tell the numbers apart and give the position as an input to the network. Both this and the item think will probably make it so that the AI largely ignores pixels in those areas, focusing more on the meat of the game.
- Map: the map info is probably hard to digest for the AI, so implementing a system focused in just doing that will probably help the AI anticipating turns and learning how to drive under blooper much faster.
- Handling of noise: instead of always giving the AI the clean input of the screen, you can block some pixels/change their color. This guarantees the AI can learn to follow the track even if its view is obstructed by the CPUs or a blooper (maybe remove this noise when a blooper is detected). This is may be helpful for chaotic situations in which the AI seems to just prefer avoiding the chaos, fall back and get an item (which is risky, especially if there are a lot of drivers really far behind) instead of out-driving the other cars.
- Items on the ground: pattern-match both harmul items in the ground and reward accordingly when avoided/hit. Good items can also be accounted for, like when the AI got shocked in the sand with mushrooms but instead of using the now-on-the-ground mushroom to go back to the sand, it simply drove normally through the sand back on the track.
- Do item-specific races. To test the understanding of each item of the AI, you can force it to play with a reduced itemset (just shells, just bananas, just bombs...). It would be beneficial to do if you realized the AI can't learn an item properly (so force it to deal with it) or if you simply want to test some of the above mentioned systems.
I hope you find my comment helpful. Great video! :)
We all know this is gonna be good.
Your an actual genius you made an AI to start bagging ❤
Thanks haha
Hey Tango! I’ve been following your channel for a while, and your A.I. Mario Kart videos have really caught my interest. I had some questions about the A.I.
1. I noticed you always use Funky Kong/Flame Rider combo for these videos. Would the A.I. preform the same if given a different character/vehicle combination? Why haven’t you tried using different combinations in your videos? If you’ve done it off-camera, I’d like to hear how that faired.
2. Seeing how you feed the A.I. information through B&W minimized frames, I’m curious what would happen if you did this in vs. Would it be possible to train 2 separate A.I., or even 3 or 4? It’d be interesting to see multiple A.I. trained to compete against each other, or even just seeing how a fully trained A.I. could stand a chance against a human player.
3. Do you plan to release the A.I. for others to experiment with? I’d enjoy racing against it, and I’m sure others would too, especially if it’s able to use multiple character/vehicle combinations. (Me personally, I use a lot of miis and I always find myself wishing the CPUs could choose miis.)
I look forward to seeing what you’re able to achieve in the future, I will watch your career with great interest!
This is really cool! Please let it play a harder course
I just watched the other, and seent he comment asking for this. I HAD to see if you did the video. Low and behold. Better turn on notis.
"AI will take over the world!!"
"hehe funny ai get destroyed in mario cart"
After watching this video, I now finally know the use of the blooper item
A good idea for the next video is to have more ai race against eachother
Cool stuff.
These videos are so interesting. I have a question though, is rainbow road possible
I wonder what will happen if you had more than one AI learning. It would defiantly be interesting to see. Especially if the AI got more reward for being closer to first place so they actually race against eachother when both are skilled.
Ill be waiting for ai trolls cpus with items. Definitely need to allow multiple inputs
Bro how is your channel this underrated?
Thanks haha, although I'm not sure its still underrated after how my last video did!
Interesting video. You should test it on every track
Your so underrated!❤❤❤❤❤❤❤❤
We need to get this legend a proper office.
You should use one of the pretrained ais on another track so it can learn to distinguish between different road types
For item usage, perhaps aggressive play, reward the AI for every time it trips someone up with a banana, or hit with a shell, and give and deduct points for blocking shells and getting hit by them respectively. You could probably train it to dodge blue shells too using mushrooms
What a time to be alive
Hello there, new subscriber!
i love ai learns a game videos, especially when im good at the game. im 9k vr on mario kart wii
You should try to train the AI with some combo other than funky-flame runner. As this is the best combo it will have an easier time getting first places, specially in luigi circuit, where it probably gains a big advantage just by wheeling in the straightaways.
You could test some of the middle range inside drifting bikes to see how it performs with less of a speed advantage.
It might be a good idea to use the collision models of the track as visual input instead of those with textures as that well generalize better across different tracks.
Hey man love the videos. I'm doing my Masters in Computing Science and following some Machine Learning and Deep Learning courses next semester so I'll definitly watch your future videos. I was wondering how difficult (if even possible) it would be to create an AI that learns to play one track but can also perform decent on another track.
Thanks, and good to hear more people are getting into Deep Learning! Maybe you'll find out in a future video :)
@@aitango I'm excited
Bro ai now knows item bagging, they be ballin out now