[Timestamps]: 00:00 - Sneak-peak of final AI 00:08 - Introduction 00:32 - Making AI's environment 02:00 - Designing AI itself 02:45 - Training AI 04:55 - Montage of final AI bombing missile launchers 06:27 - Ending
Found your video by an unrelated UA-cam search, I stayed for the quality production value, subscribed for the enjoyable content. Keep up the good work!
This channel, with only 2 videos, is already one of my new favorites! I love to watch the AI learn, and to learn how it works, so I naturally love this. Keep it up!
Oh sick, I'm glad youtube's algorithm is helping me out here ahahah. So far most of my views have come from Reddit. Anyway, I'm really glad you liked it!
I remade this as a means of learning hah! - I'm curious what results vs time you would get with the newer versions of ml-agents - You should make a tutorial or release a github of this example - It was such a great way to learn various bits and pieces of unitys agents. Nice job!
That's great to hear, I hope it went well. And yeah I've noticed the newer versions of ML-Agents seem far more optimised. Right now I dont have any plans to return to this project but in future projects I'll take some time to formalise it and put it on github for people like you who also want to give it a shot. Anyway glad you enjoyed it!
@@randomman1000tweeny1 I would love to see them down the road! It's fun to see how others solve their problems - if you ever do create a git for this project - I would love to see how you tackled the observers - finding a formula to calculate the proper rate for distance took some fine tuning. As always, its a pleasure to see your content! -Thanks!
@@NSTATS You can download the project here: www.mediafire.com/file/n120iqtrynlmr07/AI_Jet_Bomber.zip/file It's quite a mess but finding the AgentJet script shouldn't be too hard. Anyway thanks again and best of luck with your future projects!
Yeah it sure was. Unity can actually make a lot of really cool things. It's just that since it is so easy for anyone to use a lot of people use it to make trash games that they put no effort into.
I experimented with giving the jet the position of the closest missile but that didn't help since there were many missiles. I assumed that giving the jet the position of all the missiles would make training take too long and so I literally just trained the jets without any indications of missiles. Luckily, the jets then developed a really erratic style of movement which naturally made them able to dodge missiles since they never travelled in a straight line (the missiles work by predicting the jets position in the future and going to that point for the same time the jet gets there, this only works if the jet doesn't change direction). Also cheers, I'm glad you liked the editing!
ahahah yeah that would be pretty sick. Maybe one day if I figure out how to hack game engines to get them to run on 5x speed I could train an AI on it lol
Sure is, here is my project files: www.mediafire.com/file/n120iqtrynlmr07/AI_Jet_Bomber.zip/file Hopefully the hyperparameters I used are included but if not sadly I no longer have them on my computer. Though tbh I didn't test many hyperparameters so I doubt mine were optimum anyway.
Interesting, 3h and planes learned to pitch up a little bit? That's too slow for such a simple task. I trained a racing car in 45 min. that was driving almost perfect and better then me ^^ I think there must be something not ideal about setup, maybe hyperparameters etc. Otherwise VERY cool project! Something I wanted to do in a long time (not exactly this but a very similar idea of a plane bombing moving targets on the ground).
Yeah, I reckon it's to do with the complicated reward function and the fact the jet is given quite a few extra variables as inputs which it needs to correctly drop the bomb. I suspect if I simply rewarded the jet for being alive and removed most of its inputs it would learn to pitch up within the first 20 minutes too. However, I am also sceptical of my hyperparameters though. Recently most of my projects have been taking a long time to do simple tasks. Oh well, anyway it's cool to hear you've had success with AI too! And yeah maybe I might revisit this project with moving targets but I fear it would take significantly longer to train.
The AI has a score, when I reward the AI it adds points to this score, when I punish it I take away points to the score. The AI sees when points are added or taken to its score. Every frame the AI will look at its inputs and based on them produce some outputs. If it sees points then being added it knows whatever calculations it did to those inputs to make the outputs where good. If it is punished, or gets no reward, it'll assume that whatever the calculations it did to the inputs was not correct and will change the calculations it performs. When I say calculations I'm referring to a neural network btw. It's hard to explain in words and for a better understanding check out my previous AI video where I taught a drone to fly.
This is amazing. Does unity come with ML packages for training? Did you also have to create the visual assets? Also, did you have to write functions to capture the coordinates of the rockets being sent at the planes or does unity come with a bunch of packages where you can set the reward function?
There is a Unity asset to do the ML training, here it is github.com/Unity-Technologies/ml-agents/blob/master/docs/Readme.md In regards to the other stuff the jet and missile launcher 3d models I got from the internet (I did however design the poker table lol). I experimented with giving the jet the position of the closest missile but that didn't help since there were many missiles. I assumed that giving the jet the position of all the missiles would make training take too long and so I literally just trained the jets without any indications of missiles. Luckily, the jets then developed a really erratic style of movement which naturally made them able to dodge missiles since they never travelled in a straight line (the missiles work by predicting the jets position in the future and going to that point for the same time the jet gets there, this only works if the jet doesn't change direction). You do also have to design the rewards yourself and it's the most important part of the project. It will define whether the project will be a success or failure. In my case I used a pretty whacky function to reward the jet 0.0005 * (1 / ((0.000005 * x^2) + 0.3f)) where x is the distance between the jet and target. Luckily, it really doesn't matter much what function you choose, 1/(x^2+0.1) could have probs done the job.
I am using a GTX 970 however this AI was trained on my CPU being an AMD FX-8350 Eight-Core 4GHz. It took about 1 day and 8 hours to train if I remember right.
Oddly enough that was actually the original plan for the video. In the end I had to scrap the idea because the missile did really trash compared to just using a simple algorithm. I could probably have made them work if I spent more time on it but I decided the jets would be more fun to use AI on. Maybe in the future I might revisit it and have the two AI's compete.
[Timestamps]:
00:00 - Sneak-peak of final AI
00:08 - Introduction
00:32 - Making AI's environment
02:00 - Designing AI itself
02:45 - Training AI
04:55 - Montage of final AI bombing missile launchers
06:27 - Ending
Do you have a fast-forward button that turns 10 minutes into 10 hours?
Found your channel on reddit, impressive production quality! Keep up the good work :)
Thanks, it means a lot to see people are enjoying this!
Underrated channel. The editing and the content are both beautiful. Wow.
Okay man amazing job!
Found your video by an unrelated UA-cam search, I stayed for the quality production value, subscribed for the enjoyable content.
Keep up the good work!
Thanks mate, it means a lot to hear!
This channel, with only 2 videos, is already one of my new favorites! I love to watch the AI learn, and to learn how it works, so I naturally love this. Keep it up!
Thanks, I'm glad to hear you like! I'll certainly keep on making them :D
Omg, I didn't want to leave this video without telling you how grateful I am for this.
Thank you so much.
You are really inspiring :3
Thanks I'm glad you enjoyed it! It means a lot to see it
These videos are so good. Great work, Ethan, keep it up!
Cheers mate and I will do, vape nash doooog!
Starxel: Today we're gonna bomb a missile launcher with AI powered Jet
AI/ML engineers working in military basements: Im I a joke to you?
Best AI training video!
This is very well explained, shot and edited. Keep up the good work. I look forward to see what you come up with next.
Would love to see more AI learning videos! Good job.
I dig what you do. Keep it up.
Cheers man, will do!
@@randomman1000tweeny1 I like to use rare CGI for my beats. Is it okay to sample with credit?
instagram.com/lashlashbeats/
@@jimmyjim9935 Yeah sure thing, I'd be more than happy for you to do that. Nice beats btw.
Amazing content, your channel is going to blow up, that's a promise. Subscribed, keep it coming mate =D
cool fighter models!
An ai arms race would be cool, where the missile launchers also get to evolve
Possible tutorial??
Dude this is amazing! Great work and editing!
Incredible animation
this is quality content. please keep it up
I do not know why youtube decided I should watch your video but I did and I liked it.
Oh sick, I'm glad youtube's algorithm is helping me out here ahahah. So far most of my views have come from Reddit. Anyway, I'm really glad you liked it!
tutorial?
Incredible work
This was amazing
I know a channel with strong potential when I see one. Include me in the 100k screencap.
Thank you. I enjoyed this. :)
That's great to hear, I'm glad you liked it!
I remade this as a means of learning hah! - I'm curious what results vs time you would get with the newer versions of ml-agents - You should make a tutorial or release a github of this example - It was such a great way to learn various bits and pieces of unitys agents. Nice job!
That's great to hear, I hope it went well. And yeah I've noticed the newer versions of ML-Agents seem far more optimised.
Right now I dont have any plans to return to this project but in future projects I'll take some time to formalise it and put it on github for people like you who also want to give it a shot. Anyway glad you enjoyed it!
@@randomman1000tweeny1 I would love to see them down the road! It's fun to see how others solve their problems - if you ever do create a git for this project - I would love to see how you tackled the observers - finding a formula to calculate the proper rate for distance took some fine tuning. As always, its a pleasure to see your content! -Thanks!
@@NSTATS You can download the project here:
www.mediafire.com/file/n120iqtrynlmr07/AI_Jet_Bomber.zip/file
It's quite a mess but finding the AgentJet script shouldn't be too hard. Anyway thanks again and best of luck with your future projects!
AI has evolved a lot past 4 years
It sure has. Now is a particularly exciting time to be in AI but there's definitely a lot of chaos. Hard to tell what new tech is useful and not.
All done in Unity3D? Wow
didn't tought unity capable for ai things
Yeah it sure was. Unity can actually make a lot of really cool things. It's just that since it is so easy for anyone to use a lot of people use it to make trash games that they put no effort into.
What function did you use for the distance based reward and how many max steps did you use?
I downloaded the code but have no idea how to use it / get it working please help
“The ai starts getting better”
Every jet: *dies*
Holycrap. Teach me senpai!
You must first learn the art of programming on Unity3D. It is only then will you be ready to tackle AI head on.
bro this project is so impressive , i enjoyed watch ,
if it will be possible to update to RLFH
u r Underrated 😢
Post more
Hi,How to train the AI Jet? Thanks.
awesome video which languages you used to code this project?
Thanks, glad you liked it! All simulation parts were done on Unity3D using C#. The code for training the AI is done in python and uses tensorflow.
@@randomman1000tweeny1 oh ok thanks
Please continue making videos!
Thank you, I definitely will!
Does the jet have any input from the missile locations? or is it avoiding them by coincidence? Really neat project and your video editing is on point.
I experimented with giving the jet the position of the closest missile but that didn't help since there were many missiles. I assumed that giving the jet the position of all the missiles would make training take too long and so I literally just trained the jets without any indications of missiles. Luckily, the jets then developed a really erratic style of movement which naturally made them able to dodge missiles since they never travelled in a straight line (the missiles work by predicting the jets position in the future and going to that point for the same time the jet gets there, this only works if the jet doesn't change direction). Also cheers, I'm glad you liked the editing!
@@randomman1000tweeny1 Ouwh :( that means the one plane that did the barrel roll did it perfectly by chance :/
Could you share the project,it seems like so interesting!
How does the plane know when to drop the bomb ? You said the outputs are pitch and roll, so what decides when the bomb has to be released?
Thank you:)
that's amazing..can't you share the details with tutorials
train an ai to do cas in war thunder while having rolands and tunguskas after it? ;)
ahahah yeah that would be pretty sick. Maybe one day if I figure out how to hack game engines to get them to run on 5x speed I could train an AI on it lol
Hey is this project is in open source .. how can I reproduce your results
Sure is, here is my project files: www.mediafire.com/file/n120iqtrynlmr07/AI_Jet_Bomber.zip/file Hopefully the hyperparameters I used are included but if not sadly I no longer have them on my computer. Though tbh I didn't test many hyperparameters so I doubt mine were optimum anyway.
Interesting, 3h and planes learned to pitch up a little bit? That's too slow for such a simple task. I trained a racing car in 45 min. that was driving almost perfect and better then me ^^ I think there must be something not ideal about setup, maybe hyperparameters etc. Otherwise VERY cool project! Something I wanted to do in a long time (not exactly this but a very similar idea of a plane bombing moving targets on the ground).
Yeah, I reckon it's to do with the complicated reward function and the fact the jet is given quite a few extra variables as inputs which it needs to correctly drop the bomb. I suspect if I simply rewarded the jet for being alive and removed most of its inputs it would learn to pitch up within the first 20 minutes too. However, I am also sceptical of my hyperparameters though. Recently most of my projects have been taking a long time to do simple tasks. Oh well, anyway it's cool to hear you've had success with AI too! And yeah maybe I might revisit this project with moving targets but I fear it would take significantly longer to train.
Now see how it goes after 2 weeks training.
How do you "reward" the AI for a job well done? How does it know a reward is good?
The AI has a score, when I reward the AI it adds points to this score, when I punish it I take away points to the score. The AI sees when points are added or taken to its score. Every frame the AI will look at its inputs and based on them produce some outputs. If it sees points then being added it knows whatever calculations it did to those inputs to make the outputs where good. If it is punished, or gets no reward, it'll assume that whatever the calculations it did to the inputs was not correct and will change the calculations it performs. When I say calculations I'm referring to a neural network btw. It's hard to explain in words and for a better understanding check out my previous AI video where I taught a drone to fly.
@@randomman1000tweeny1 Thanks for taking the time to explain.
this is fucking epic
This is amazing. Does unity come with ML packages for training? Did you also have to create the visual assets? Also, did you have to write functions to capture the coordinates of the rockets being sent at the planes or does unity come with a bunch of packages where you can set the reward function?
There is a Unity asset to do the ML training, here it is github.com/Unity-Technologies/ml-agents/blob/master/docs/Readme.md
In regards to the other stuff the jet and missile launcher 3d models I got from the internet (I did however design the poker table lol).
I experimented with giving the jet the position of the closest missile but that didn't help since there were many missiles. I assumed that giving the jet the position of all the missiles would make training take too long and so I literally just trained the jets without any indications of missiles. Luckily, the jets then developed a really erratic style of movement which naturally made them able to dodge missiles since they never travelled in a straight line (the missiles work by predicting the jets position in the future and going to that point for the same time the jet gets there, this only works if the jet doesn't change direction).
You do also have to design the rewards yourself and it's the most important part of the project. It will define whether the project will be a success or failure. In my case I used a pretty whacky function to reward the jet 0.0005 * (1 / ((0.000005 * x^2) + 0.3f)) where x is the distance between the jet and target. Luckily, it really doesn't matter much what function you choose, 1/(x^2+0.1) could have probs done the job.
can u explain the bomb-dropping mechanism ?
btw if u can, can u show me the code?
Which gpu/cpu are you using? How long did it took to fully train?
I am using a GTX 970 however this AI was trained on my CPU being an AMD FX-8350 Eight-Core 4GHz. It took about 1 day and 8 hours to train if I remember right.
forgot to sub yesterday. I wanna see them videos! Once I get out of my gpu world generation.. ai is next!
For sure, I'm sure you'll love getting into it! I'm yet to meet someone who regretted getting into AI.
I wanna see if someone can make a game with ai that is only trained like not programmed just player controls
should add ai logic to the sams as well.
Oddly enough that was actually the original plan for the video. In the end I had to scrap the idea because the missile did really trash compared to just using a simple algorithm. I could probably have made them work if I spent more time on it but I decided the jets would be more fun to use AI on. Maybe in the future I might revisit it and have the two AI's compete.
5:34 Highlight kill of the week.
👍
good
and can you share de code?
Here it is: www.mediafire.com/file/n120iqtrynlmr07/AI_Jet_Bomber.zip/file
America: entresteng
Interface your nn to DCS world!
NATO is looking for you to train their drone
Tutorial?