Training a Neural Network to operate drones using Genetic Algorithm
Вставка
- Опубліковано 10 лют 2025
- After my first try with flappy I wanted to see how would a genetic algorithm handle more complex situations.
Github github.com/joh...
Music used
freepd.com/mus...
freepd.com/mus...
freepd.com/mus...
Good idea, and I like your smoke!
Thanks! I think smoke is where I spent the most time :D
@@PezzzasWork Why do we get hung up on those small sidequests?
@@mendelovitch it’s an easy way to procrastinate the main problem
How or where you stimulate this in unity or special software
I love how they seem to move so organically even though it seems like a relatively simple model. I bet there's some really interesting optimisation problems and extra restrictions you could throw at this.
Also thanks for uploading the demo and source code, very fun to play around with!
I think that the need to center themselves perfectly with the sphere is what makes them not become speed machines. Because when they reach the target they always gotta somehow "dock". And that requires their inertia to be 0 when they reach that point so they have to slow down. If somehow this was changed by making the drones to just need to touch the point at any part and maybe making the orb bigger I would certainly expect that there would be more speedy manoeuvres to just arrive at the target and pass through it. Perhaps even in an elliptical patrolling. Would be certainly interesting to see.
im currently working on the same thing but with more inputs;
I will try ours too;
@@00swinter21 Don't forget to post the result on your UA-cam channel !
Exactly my thoughts. It looks like the target requires pixel perfect precision to count as a success. Careful approach is the only way when the targeting criteria are so unnecessarily strict.
@@Wock__ I believe you are right. On one of their videos, there is an actual clock face that counts down on top of the target, like a circular loading bar.
The goal is to dock, not to touch the target. Changing the goals to achieve a better outcome does not mean that your model improved. Making them just have to touch the target so they could go really fast does not mean that they are suddenly better. Your thinking is flawed.
5:28, gen 900: Ok, you guys are too good and I'm tired now. Bye!!!
true
"I have to go now, my planet needs me"
I'd love to see a game where your enemies are all neural network trained AI, and the higher the difficulty, the more trained AI variant you will have to face
Give it 10 years
imagine if the AI is being trained while you play. The better you play the less hard the ai is, but if you slow down the difficulty increases
@@ChunkyWaterisReal it's already possible now lol
@@marfitrblx AI has been shit since the 64 hush yourself.
Or even the player being an AI - I can totally see a 2D game with your cursor being the target point, and the more you play/the more enemies you defeat/etc. the smarter your character gets
This channel is a gem
Would be really interesting to add fuel consumption to the mix and watch them optimize their fuel economy
and give them more fuel for every target they reach as more reward for doing that
Very cool stuff, well done!
Now make them go through an obstacle course 😁
I am working on it ;)
a combination of the ants finding the optimal path and then the drones following that? :)
@@PezzzasWork where's the video 🗿
These vids are awsome and very inspiring! It's amazing how a neuron network can adapt for a specific task!!!
Great work. I think many people would appreciate seeing background of the work.
love this channel. what separates this guy from others is his consistent ability to make his sims look cool.
This is one of the coolest projects I've ever seen. Would be awesome to extend to add walls and an environment! Great work.
This is one of the coolest implementations i've seen. Nj!
Dude I love it when they get sooo roofless! So fun to watch!
I wrote my autopilot cargo drone for space engineers and still i am impressed by the work
The end of play lineup was a cute touch. Nice work!
5:56 The drone in the left down corner synchronized with the beat in the music. Perfection.
That end result with the live-tracking is so good! I wonder how viable it is to train simple neural networks like this for game enemy AI
Depends on the game, but on games with a clear goal, it is fairly trivial and will quickly surpass humans.
@@originalbillyspeed1 i guess for different difficulty levels game designer can use agents (enemies) from different generations, for example "easy" = generation 400, medium = generation 500, hard=generation 1000.
@@AB-bp9fi I don't think that would work for most applications. When you want to make enemy AI easier or harder, you always have to think of it in relation to the player - for instance, in a stealth game, harder AI could mean it detects you faster - which pushes the player to improve and be more careful. That won't happen if you just made the enemies drunk (which is basically what would happen if you pick bad neural networks) - it just adds randomness which can be annoying to deal with. Maybe it could work better in things like racing games though.
I'm now imagining a game cloud coordinating through the internet. The AI uses background CPU while the game is running to simulate and evolve against itself, spits its best results against the player to see how they fare, and takes those results as more data to go back to the cloud with to keep working. The bots will start laughably bad at first, but they'll learn how players act, and make players devise new tactics... You might even get good teammate and wingman AI out of it if you put those AIs on the player's side.
@@commenturthegreat2915 What about training AI to match the certain level of intelligence? Like if AI detects a player too fast, then it failed the test.
Props to Gen 300 and 400 for beings underdogs and yet surviving for so long
OMG This is so cool, your video actually change my attitude toward neural network from hate to love.
Impressive Stuff! Had my hands on GAs too for my Bachelor Thesis but with a 6 DOF 3D acting robotic arm. Kinda addicting when you dive deep down in ML :)!
very nice! I'd love to see the same tests, but with added random disturbances like wind gusts from the side, to see how well they can adapt to that!
You are amazing. Thank you for sharing your fascinating work.
Getting some strong Factorio vibes at 4:57
I like how it learned to turn off its thrusters to arrest upward motion and to speed up descent.
It would be great to have a remake of this one
I am actually working on a follow up :)
@@PezzzasWork noice! I will certainly watch it
Thanks for the video! It's really inspiring.
give a consolation prize to generation 300!
It deserves it all
Have you ever tried using a neural network on a hardware platform?
1:58 that faint Vader "noooooo" put me on the floor for some reason
Very nice! Please make more such content, with neural network and drones! :)
somewhat smaller models and policy gradient following might have increased convergence speed. MLPs are differentiable, so you could just backpropagate through them, sampling distance to the target at every frame and accumulating rewards over the trajectory for an unbiased estimate of a policy’s optimality. you could even use a decay term to incentivize the robots to move faster by downweighting rewards acquired later in the trajectory: distance to the target is ideally the same in the end, but according to the gradient of this reward function, faster would be better.
the only thing left would be running the simulations in parallel or faster than real-time by simply not fully rendering the state of the environment at every training step
Adding a fuel allowance would probably add a more varied result, possibly get those burn hard drones quicker. Also maybe increase your destination bubble a fraction ? This increase the prize rate and hopefully the drones would tighten up the homecoming naturally like the ants do for food routes
Hello,
very interesting work !
Did you think about testing scenarios with obstacles ?
It would be also interesting to compare the last trajectories and controls with optimal control algorithms solutions.
Cheers.
Other than giving us almost 20 seconds to read 6 words at 4:39 this was very enjoyable to watch :p
I tryed the mouse controlled vesion what you uploaded on github. And i saw that it's easy to confuse the A.I. in that way to lose controll and fall off the map. I think if you crate a small Trainer A.I. for the target control what best interest to confuse the drone and make it fall off the map, it can train the drone to not fall off no matter how the target moves.
Yes I did a more robust version that I can upload as well
gen 400 is like that one kid in your class that cant stand still when waiting in a queue
I’d love to see an algorithm where you simply add the direction from the current target point to the next, and see if it, with only that information learns to steer ahead of time.
oooh idea. Space Invaders: Drones Addition. Different levels use different generations of drones as enemies.
I suggest to add more then just time to the fitness equation. Fe. Energy use, pressicion, stability of flight and adding external forces like wind. with these factors the movement would become smooth like silk. But nice project anyway
The current fitness evaluation takes speed, precision and stability into account. I tried to add wind after the training was done and it worked quite well :)
@@PezzzasWorkahh I see, but the angled engines while hovering still seem very inefficient to me :)
@@DeepRafterGaming Yes you're right and I don't really know why they do this. My assumption is that it is a way to reduce power, as if they couldn't go very close to 0 power so it is easier to add angle. This could be avoided by taking energy into account in the fitness function. If I increase gravity, they don't angle the thrusters to gain more power. Here is a windows demo with a config file if you want to try it out github.com/johnBuffer/AutoDrone/releases/tag/v1
@@PezzzasWork Yeah it's hard to tell why. The fitness function is the most complicated part of any neural network.
I would allway advocate for implementing energy use in any neural network because, if you think about it, if the network doesn't have to bother with the used energy it will always come up with unnecessary movement patterns that look jenky. It's more important than speed I'd say ^^
@@PezzzasWork if you watch the way generation 5500 flys sideways, it ends up with one thruster almost horizontal and the other almost vertical. They might like tilting the thrusters because its kind of an inbetween state between flying right and left. So when it gets a new target, it can start flying towards the target sooner. That might be part of the reason anyway.
xDDD the "ok..." almost kills me
The memes are fun on this vid
7:25 the music moves to your left and right ear as the drone in the top right moves it's power to it's left and right thruster.
Pls make more vids like this I love them
You could turn the target tracking into a game, try to get the drone to lose control as quickly as possible, using your mouse as the target! Or, just play with it. It looks fun.
Give the target to another network that tries to learn how to get the drones to crash while the drones learn how not to crash
@@DogsRNice oh no the ai wars
I like these projects !
7:24 Loved how the Gen-400's legs synced with the music...
Btw, How do we decide the size of the hidden layers? Is there some rule or formula for the best size approximation?
Beginning of the video: LOL!! those squeaks as they fall are really funny
End of the video: let's run to buy some food cans before they come for me!!!
Imagine spending hours and hours trying to get to something and then when you finally get there you just have to go to another one
This would be a great premise for a game where the character tracks the mouse so instead of controlling the character you're directing it and it gets better as you play through AI learning
Very nice result!
i have no idea about how you did it ..but it seems like something fun to learn
Machine learning is extremely fun and addictive :)
@@PezzzasWork can confirm
400 was such a trooper
The target tracking would be cool for a background
This video felt like it's 30 minutes because I somehow kept falling asleep every ten seconds or so.
And it's not boring and no I am not high, idk I guess I just got tired or something
Totally amazing!!!
That drone that got yeeted at 5:30 had me dieing 😂
Which parameters give the drone positive or negative feedback?
Is flying time a positive or a negative parameter? An acceleration to the target?
im more impressed by the smoke, great project though!
That was really cool.
I dont understand the sin and cos part in the inputs can someone explain?
basically you take the angle the drone is currently at and get the sinus and cosinus of that angle
Love that video
I'm kind of upset that you didn't publish the thing at the end on itch. Its so satisfying to see the drone follow your mouse and I want to play around with it. Great video!
You can download the control demo here github.com/johnBuffer/AutoDrone/releases/tag/v1
@@PezzzasWork thank you! :)
Wonderful!
Dude the physics look so polished. This is amazing!
Acceleration (gravity, mass an inertia) is probably the simplest physics properties to program. Literally just adding or subtracting numbers. He does not require your compliments on the physics.
@@UnitSe7en ?
@@UnitSe7en shut the fuck up, he’s giving him a compliment
Now create an additional network which positions the orange dot (target) to navigate around obstacles on its own.
What about creating new variables? Like saving fuel or energy consumption, or giving priorities like speed over energy/fuel consumption
Great video! How long have you been training them? Greetings from Uruguay!
Ok, now make these drones fight in groups of 5, they can kill other drones in 2 ways one is to ram into enemy drones (killing both of them instantaneously), or shooting them with miniguns (only killing the target if it is hit X amount of times). But every time when they die they respawn, smarter, faster, more accurate, etc.
After a few tweaks, I have a feeling this could have real-world use.
Amazing
if you had an body orientation/angle input they would have been able to recover from a spin out or even fly upsidedown
Great video! I've been trying to make a similar recreation of this project in Python but while I get some decent results, I'm struggling with local minima trapping and have failed to get the kind of 'brutal' drones you got at the end of training. Tried having a look at the source code but I'm not too familiar with C++. Just wanna know, what did you use for your fitness function and how did you mutate your networks? A reply would be very much appreciated!
the target tracking drone would be a really cool and distracting extension, it follows your cursor around where ever you put it lol
I love this! I'm gonna implement it right now in Python. What genetic algorithm were you using? I'm planning on using Neat
how did you get this environment in Python? I want to test policy gradient RL algorithms
@@CE-ov7of not sure if you still need this question answering however i'll give it my shot. My guess is hes implementing the basic algorithm of the envirment in python using pygame and and numpy. Then for the AI my second guess is he'll be using NEAT Python library or custom AI/NN algorithm for the agent and training. That's my guess however if you want any question just reply and i'll do my best to help. Python isn't my strongest language however but i'll try my best.
Hey @@j_owatson , unfortunately this is not something I have time/interest for anymore.
But I really appreciate your willingness to help! This is what makes the software/tech community great!
Very cool
Gen 5500 appears to display knowing how to fall rather than turning the thrusters to push itself down.
how do you tune the weight and bias using GA,? do you intercept the backward process with GA?
Amazing 😮😮😮
Wowwww I'm amazed
You should make a game out of this, it looks very funny!!
Nice work!
What mutation/crossover did you use?
Would be interesting to have a drone sumo where they can collide and try to shove each other out of a ring.
Really cool project!!!
I was wondering what fitness function you used?
Can u make a tutorial how to choose the best inputs depend on sample ?
you should place the targets randomly and not in a specific order. And for more challenge, they only have a specific time to reach the target. After the time the target disappears. And finally, the targets are fuel. If they miss too often they run out of fuel.
Edit: maybe even add obstacles.
In the video the targets are in a specific order to be able to benchmark the different generations, for the training I used random sequences
@@PezzzasWork Ok, that makes sense
You may have to select more aggressively for speed. They seem a bit slower than what the optimal handmade algorithm could do
This is cool.
"Im a Hovercraft like my Father before me and his before him!"
why not add a fuel limitation (which would refill once they get to a point) forcing them to switch between points as quickly as possible from the beginning
wouldn't be necessary, they're already rewarded for speed
Guys, how do I make a game or whatever these robots are in the video? What application? OpenGL, Unity, Unreal Engine... If someone can tell me.
man this is so cool. a bit off topic but how are you rendering the thruster particles and smoke?
The smoke is just made out of static sprites and the thruster particles are baked into the flame's texture
What is being passed at cos Angle and sin Angle? the angle to the target or just the function sin and cos?
the angle of the drone to the world
its really beautiful.... can you please suggest how do I learn all this. What I learn in what seuqence ??
Wow that's amazing and looks amazing, how did you cross the two neural networks?
Gen 2600 was a big leap in speed and control.
I would love for you to make an eco system like the bibites using those drones
you should make a game where you control a small ship like asteroids and your goal is to juke out the drones and cause them to crash or see how long you can survive before they hit you or something
it's cool to see your using dropout, so it learns better
great Awesome!!👌👌😀. where did you learn to do this?
While it was nice for the visual of all the different generations together, I feel like it would have been better to randomize the dot locations so that they have to learn to adapt to a new path every time
Nice work! Can you propose me material so that I can understand in practice how to build a neural network? Something with examples.
That's a good tutorial idea, I will think about it :)