Teaching a computer to strafe jump in Quake with reinforcement learning
Вставка
- Опубліковано 1 чер 2024
- In this video, I use RLLib to train a reinforcement learning agent to strafe jump in the classic video game Quake
Chapters
00:00 Introduction
01:11 What is Reinforcement Learning?
02:42 Quake RL environment
04:41 Results
08:22 Comparison with human WR
09:54 Future direction - Ігри
![[柴犬ASMR]曼玉Manyu&小白Bai 毛发护理Spa asmr](http://i.ytimg.com/vi/0TsXQ7z2Dh4/mqdefault.jpg)
Using low difficulty defrag maps might let you focus more on the movement than dealing with enemies and figuring out how to press buttons
i am so pleased to have found your channel through this video; please keep it up! this is my favorite content i've absorbed recently, as a long time speedrunner of quake-based games, its really wonderful to get raw insight down to basic physics and code to understand why im doing the actions i'm doing
This video is amazing! Your creativity and practicality with the production and visuals is so helpful for viewer comprehension. For example, I had never understood powerbunnying before other than as just a strange engine quirk that made quakeguy go faster, but getting to see the graph displaying the vectors frame by frame totally illuminated the reason this tactic works and allows players to optimize their runs. But of course, the real treat-watching the software complete a quicker 100m demo than crashfort, one of quake/goldsrc/source engine's most well respected and skilled movement players despite some limitations on keypresses and mousespeed. Truly awesome.
genuinely hope your channel grows
Your Quake videos are excellent. Many thanks for the informative content.
Incredible channel, just finding it now. Subbed!
This was very interesting to watch, thank you for sharing it!
Wonderful suggestion about the potential for reinforcement learning to find new techniques! I'm stoked!
Fantastic video Matt! Would love to see how far you can take this and really push the engines limits.
The video is oddly enough, very well composed and polished. Subscribed, please provide more content XD
Amazing video. Super detailed information!
This is nuts dude, good job, I love it
thank you for your work, great job.
Awesome video. I wish you had shown how fast it could go without the restrictions.
Great job man! I hope to see more
I studied ai at university and played a lot of qw, so i find your work very intresting. I have had been writing a bot, but my enthusiasm just exhausted after 4 months of work. But i hardly thought of making the bot move through ai and just took moves from pro-players' demos ( i had 16gb of them). I wanted to use some ai for more common behaviour such as attack or chase, or retreat..camp etc..
We want more of these stuff !
This is super neat. I searched “ai bhopping” and found your video. Really enjoyed it. Do you think this is could be repurposed with other games that allow bhopping / strife jumping?
I LIKE THE PART WHERE HE JUMPS
This is sick you should really upload more this stuff is really cool definitely a subscribe from me
Amazing work
a new matt video?
on easter?!
my year is complete
Hello ! Impressive work. I was wondering which algorithm you are using ? Is it ppo2 with baselines and did you tweek the hyperparameters ? Also, which policy and how many timesteps it took ? Which python version and others (gym, tensorflow, baselines/stable-baselines, etc). I would realy like to know more ! :)
This is awesome!
Any chance you'll make your code publicly available? ^^
Yes please
I've now cleaned up my code and put it here: github.com/matthewearl/q1physrl
@@MattsRamblings Matt, first of all I'd like to say, I hope you are doing great and have a Merry Christmas and Happy New Year! I have a few questions.
1. Do you think it would be possible to create an AI- bot with reinforcement learning for *Quake Live* Duel?
2. If so, would it be possible to write the code to limit it to only shoot around a certain % of LG and machine gun? Any gun with track aiming. Obviously, it wouldn't be too fair if the AI bot could shoot 100 LG or completely have lock on aim, so it would be interesting to see how it would learn to take fights and learn how to get frags if it could only shoot 30 percent lg or up to 40 or 50.
Personally, I think the lower the track aiming accuracy the AI bot would have would be way more interesting to see because it could teach you combat techniques that could work for ordinary humans since only the top .1 % of players were able to shoot super high 40 to 50% LG during duels.
If you could create an AI bot like this I would love to buy or rent it off of you so I could plug it into my servers and play it to sharpen my game in duel, because right now I'm only around 2100 elo and would like to get to around 2500, 2600, elo if possible and be at a higher level.
Interesting project! What kind of visualization library did you use for the diagram in 6:10? It doesn't look like the standard matplotlib.
nice video!
your voice is very relaxing
this is totally insanity I like it
This is super interesting! I'd love to see more videos like this, being a developer myself.
Do you have a recommendation for learning this lib?
Also, alternative titel: "Computer teaches Quake novice to strafe jump" lol
Check out gym.openai.com/docs/ , if you are interested in the theory behind it search for "RL Course by David Silver"
I would love to see this ML approach applied to a full run of Quake or Half Life!
awesome
Content like this is fascinating because regular developers created a video game and people are learning how to get the best results out of the rules that the developers have written. Humans working within other humans' expressed boundaries.
Is it necessary to use learning and stuff? SJ consist of combination of buttons and mouse. Program one strafe jump, program how to chain them together, wouldnt it be easier?
Cool
I wonder how this can discover new tech in AM2R
Its crazy that the human wr is the almost the same as RL.
Interesting. You've said it would be interesting to see how a hand-crafted agent would compare; I've got a video of the theoretically perfect completion of 100m here: ua-cam.com/video/rejtyHCf120/v-deo.html No demos on that video unfortunately, but I can ask a friend of mine (@Jukspa) to make them using his version of the tools: github.com/Jukspa/TASQuake/
Got someone on Q2DM8 once.. bunnyhopping from bottom stairs, through the gap and into the tunnel under the dropdown with megahealth..
As I'm passing through the gap between boxes, I hear someone spawn to the top right.. kept bouncing into the megahealth tunnel and flicked up, behind right, about.. there..
and back to the bouncing.. guessing he just saw railgun come out my ass as I was going past 🤣🤣🤣🤣🤣🤣🤣🤣🤣
Matt!!! Please pleeeeeeeeeease.. There's this immensely addictive web-game i've being playing called "agario". its simple: just keys w and spacebar + mouse move. it's a super simple and at the same time super complex game: you start as a little blob and you need to eat pellets spread around the field in order to grow, and eat smaller blobs to keep growing, while running away from bigger blobs which are always faster than smaller ones, and finally there are Viruses that are fixed blobs that explode you if you touch them, and you can also shoot pellets (using W) on those virus in the direction of other players so they explode in smaller pieces that you can eat. Players have invented multiple complex movements along the past many years. Could you implement an AI agent to play this game?? (there are already many "bots" in this game, but they are all dumb)
Can you post a link to the track map?
speeddemosarchive _d0tt _c0mm_ _slash_ quake _slash_ maps _slash_ 100m _d0tt_ zip
The link for downloading this map?
speeddemosarchive _d0tt_ _c0mm_ _slash_ quake _slash_ maps _slash_ 100m _d0tt_ zip
have you ever looked into csgo hns
may I know why you used reinforcement learning insted of neat..
:)
All i can remember is phoon from cs 1.6
He is doing it sideways!
phoon was from cs source , not 1.6
@@vabalokis he also played 1.6
A new kind of fuzzing 🤔
I grew up on Quake.
holy crap the compression youtube does make it looks like the video had smallpox
Tame your plosives. and employ a de-esser, I feel like I am watching an ASMR channel.....
pop filter