Reverse Engineering the AI of Age of Empires

Поділитися
Вставка
  • Опубліковано 23 лис 2024

КОМЕНТАРІ • 118

  • @nathanbaggs
    @nathanbaggs  4 місяці тому +5

    Want early access to new videos and some behind the scenes content? Consider becoming a channel member ua-cam.com/channels/QvW_89l7f-hCMP1pzGm4xw.htmljoin
    Other videos you might enjoy:
    ua-cam.com/video/DvJLIWAGno4/v-deo.html
    ua-cam.com/video/ILY7tYdIS2Y/v-deo.html
    ua-cam.com/video/_2GObcrrWq8/v-deo.html

  • @DavePottinger-x1z
    @DavePottinger-x1z 2 місяці тому +27

    Well, I can shed some light on this... since I wrote the "AI" for Age1. I'm putting AI in quotes there because it was not really much of an AI.
    * Only the low level graphics and tiny bit of the pathfinding was optimized into assembly. Everything else was C++ modulo a few interface setups for various packages/libraries.
    * SNs are "Strategic Numbers". It was a list of control variables that gave the designers (mostly Rick Goodman at that point and then me when I went into tune things) ways to control the overall strategies and tactics. Stuff like "Oh, you want to rush, then use one of these units if you're Egyptian". Stuff you'd produce with dynamic analysis nowadays, but well, this was a while ago back when AI was considerably less fancy.
    * Spot on with logging getting stripped and human players having null/unfilled hooks for AI logic. Fun fact, apart from the AI logic, all the player data/commands are the same between human and AI players in all the original Ensemble games. When the AI wanted to attack a player, it was required to interface with the rest of the game in the same was as the humans. Obv can't speak to the re-releases;)
    * Awesome video, BTW.

    • @nathanbaggs
      @nathanbaggs  2 місяці тому +7

      Thanks so much for reaching out, what a great insight. Glad I a got a few things right (:

  • @jpgdallas
    @jpgdallas 4 місяці тому +19

    In 1996 I worked for Ensemble which was an IT consulting firm before Tony launched Ensemble Studios. So all the people listed in the Wiki page I knew back then. Angelo spent time explaining DirectX to me. Was fun to watch them make the game. Good times.

  • @wilfridtaylor
    @wilfridtaylor 4 місяці тому +92

    Wolololo indeed. Love this series keep up the good work.

  • @JerryThings
    @JerryThings 4 місяці тому +15

    Facinating to see an old game not using behavior trees, but instead it takes a dynamic approach. Love it!

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +4

      I think one of the Halo games was the first to use behaviour trees (but I may be misremembering)

  • @نونيم-ي4ح
    @نونيم-ي4ح 4 місяці тому +68

    "i get outsmarted by a 26 year old program" bro fear 2005 ai is like alien technology AI compared to most modern triple A enemy AIs.

    • @AEGISAOE
      @AEGISAOE 4 місяці тому +6

      there is a open ai machine learning, but for aoe2

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +27

      It doesn't help that I'm just bad at games

    • @looc3Z
      @looc3Z 4 місяці тому +3

      You don't want to know alien tech - trust me.

    • @1AEGIS
      @1AEGIS 4 місяці тому +1

      @@looc3Z what makes u think they are so advanced?

    • @looc3Z
      @looc3Z 4 місяці тому +1

      @@1AEGIS Well something created this world you know... We live in matrix.

  • @akioasakura3624
    @akioasakura3624 4 місяці тому +13

    underrated youtube legend. ive learnt so much. most ppl who try to teach reverse engineering programs, assembly thingies etc do an absolute crap job at it and expect you to "figure it out" and "practice". but u cant practice something u cant do!! seeing u do this for real and explaining/commenting all the way was invigorating for lack of a better word xd. thank u so much sir 🔥🔥

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +4

      I'm really enjoying just sitting down, solving problems and then taking people on that journey

  • @burgtaro
    @burgtaro 4 місяці тому +21

    So bizarre that this series came up in my recommended, I’ve been trying to find any reverse engineering online for AOE 1 for ages (similar to how OpenRCT has been done)
    This is awesome, keep it up

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +3

      Thanks! I've got some videos reverse engineering some other games you might find interesting

    • @burgtaro
      @burgtaro 4 місяці тому

      @@nathanbaggs The discovered the RCT one, amazing

  • @horsethi3f
    @horsethi3f 4 місяці тому +2

    I had fun modding StarCraft1 ai back in the day. You just give it a buildings list and unit list for it to build at each step. You can check for enemy composition and build specific counters by using goto commands. Fun stuff.

  • @LosjuegosdeDani
    @LosjuegosdeDani 4 місяці тому +1

    4:59 The purple hue means that you are watching a function that ghidra couldn't find any direct references to from other functions, and so it doesn't know where the function begins
    This also explains the fact that picking one of the switch cases causes ghidra to reload the function showing that case, since it doesn't actually know that the function starts earlier
    Also, the "rubbish" from 5:25 is part of the switch case, those are values which represent the location of each case of the switch statement that the function jumps to
    Keep up your nice work! 😄

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +1

      All makes sense, thanks for the insight

  • @GerinoMorn
    @GerinoMorn 4 місяці тому +8

    That's not a video I expected this evening, and I fully intended to click off after just checking what's the approach, but your narration is so good I stayed. Also had no idea AoE was written in assembly, I knew about the coaster and some other "big" (as in complexity) games. I guess I expected MS to just go with straight C.

    • @jrcarlsen
      @jrcarlsen 4 місяці тому +3

      Microsoft didn't develop AoE, it was developed by Ensemble Studios which was later bought by Microsoft.

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +6

      It's nice when the UA-cam algorithm works

  • @lor3
    @lor3 4 місяці тому +3

    As a fellow reverse engineerer I salute your perseverance . Don’t give up! :)

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +2

      Got to keep plugging away at these problems, can't let the computers win

  • @JimNichols
    @JimNichols 3 місяці тому

    About 43 years ago I made a star trek game in Basic+ it was a test from my computer science teacher. Todays code is basically the same as the code then as we also programmed in assy. I remember so much but sitting down to learn C or Python at 64 I find that I don't have that spark that you have decompiling this game any longer. I love listening to your vids and you have a great oration skill, thanks for the many memories that return while watching your channel. :)

    • @nathanbaggs
      @nathanbaggs  3 місяці тому

      Thanks for anecdote (: glad you’re enjoying that videos!

  • @mattmurphy7030
    @mattmurphy7030 4 місяці тому +5

    My favorite kind of content about my favorite game? Yes please!

  • @crypticsea
    @crypticsea 4 місяці тому +8

    I'm pretty sure most of the game was written in C++. and assembly was only used for certain graphics routines

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +4

      I am fully prepared to admit that my research stopped at Wikipedia: "Age of Empires was mostly written in x86 32-bit assembly code totalling approximately 13,000 lines".
      Might be fun to dig into the graphics and have a look though

  • @Galic44
    @Galic44 4 місяці тому

    I've just discovered your channel and saw only your two videos about Age of Empires, but they are absolutely brilliant. I really hope that this will be a regular series, it's interesting and really well done. Subscribed! Next up: cheating in UT99 followed by the Worms 2 videos!

  • @HinkHall
    @HinkHall 4 місяці тому +6

    Love this series

    • @nathanbaggs
      @nathanbaggs  4 місяці тому

      Glad you're enjoying it (:

  • @js-123
    @js-123 4 місяці тому +17

    The statement "The entire game is written in 13k lines of assembly" must be wrong - that'd only be 13k instructions. That's by far not enough. Probably some tight loop was written in assembly and is 13k lines, that would make a lot more sense.

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +9

      I am fully prepared to admit that my research stopped at Wikipedia: "Age of Empires was mostly written in x86 32-bit assembly code totalling approximately 13,000 lines".
      Might be fun to dig into the graphics and have a look though

    • @voodoo1094
      @voodoo1094 4 місяці тому +8

      ​@@nathanbaggs I remember reading an interview years ago where a dev stated that most of the rendering pipeline was written in assembly for performance reasons. So I imagine most of the rest of the game was written in C or C++ and they used inline assembly for specific parts.

    • @teracraged320
      @teracraged320 4 місяці тому +1

      Imagine writing today a game pure in assembly this people are goats

    • @RandomUser2401
      @RandomUser2401 3 місяці тому

      one could compare the number of lines for Rollercoaster Tycoon to do some sanity checks

  • @chrisdickens4862
    @chrisdickens4862 4 місяці тому

    I really enjoy your editing, narrative style, and sense of humour. Another great video. Thanks!

  • @quicksilver_x
    @quicksilver_x Місяць тому

    Hi Nathan, I'm trying to open a savegame file (.gmx) from the demo version of a Rise of Rome campaign called, First Punic War. The thing is, this campaign (.cpx) does not exist in the full version of the game and it is shipped only with the demo version. So I copied the campaign and apparently, it works in the full version! Now my challenge is to open a 20-year-old savegame originally created in the demo version where I spent 8-12 hours and finally, get a screenshot in glorious 1440p. You don't need that much time to beat the scenario ... this save is special for various reasons and I don't want to spoil the surprise.
    My initial attempts to open this savegame have failed as the full version sees the save but complains that there was a problem opening the file. I have made sure that I beat the previous scenarios leading up to the scenario in question, to finally unlock.
    The last thing I tried was to create a hex dump of a new savegame (.gmx) from the same scenario using this full-version and another hex dump of the old save from the demo version. I couldn't get them to match at the header and footer so that I could at least transplant the contents of the old save into the new save, but I will keep trying.
    If you'd like, I can report back after trying the tools mentioned in your video description, and better yet talk to you over email or discord someday when you get a chance.
    I absolutely love these videos about C++ and Age of Empires, and I'm going to save this video for future.

  • @petemc4190
    @petemc4190 4 місяці тому

    that music is back, in pog form, learning me up good in why AoE is that good

  • @kumaravelviswanath3553
    @kumaravelviswanath3553 4 місяці тому

    Great video. Keep up this series. Also can you work your magic on Soldiers of fortune. Thanks a lot

    • @nathanbaggs
      @nathanbaggs  4 місяці тому

      I’ve got some more games in the pipeline

  • @lsf3og
    @lsf3og 4 місяці тому +1

    Thanks for the follow-up

  • @imqqmi
    @imqqmi 4 місяці тому +3

    I wouldn't be surprised that the AI runs stuff faster than you, or needs lower resources so it can reach certain stages faster. I was wondering the same with Steel Empires and the manual actually gives a few clues. In that game the AI earns more money, about twice as much on the highest difficulty. I was only able to defeat it by saving state in the emulator each round and reload if I lost a battle and try again with a different strategy.
    There must be an optimum build order in AOE that it's using in order to defeat the human player, and it probably doesn't have to explore as much as it can look up where the resources and enemies are. Also AI can do stuff simultaneously where a human needs to divide the attention. The number of things it can do per 'turn' or 'tick' is probably a variable that relates to difficulty level. There's probably also lists for each unit and in what order it can be built and what the optimum numbers are resources vs time to build vs power etc. AI will be limited in how it can counteract different strategies and some just aren't covered and can be exploited.
    Interesting project!

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +1

      I’ve seen no evidence of the AI cheating, it looks like it has a bunch of initial build orders (based on some files). I suspect the difficulty is just how aggressive it is in attacking you and pursuing victory conditions (like wonders)

    • @mitchlindgren
      @mitchlindgren 4 місяці тому

      I am almost certain that the AI cheats on higher difficulties in the original releases of AOE I and II. This is widely believed to be the case in the community, although I don’t know if anyone has provided proof via source code or disassembly. For AOE II DE, the new AIs were lauded for being very skilled despite not cheating, which also implies that the original AI did cheat. Will be interesting to see if you can confirm this for AOE I via your analysis.

    • @araarathisyomama787
      @araarathisyomama787 4 місяці тому +1

      @@mitchlindgren I vividly remember spawning an "Impossible" difficulty AI in AoE2 editor like 20 years ago on a map where the AI did not have any means of producing anything, TC, nor vills, just military and other buildings or some similar condition where it's impossible to grow and I reached it after a few minutes and it randomly had all of those things.

  • @theexplosionist2019
    @theexplosionist2019 4 місяці тому

    SN means strategic number. Those are values set by the AI script that dictate how the "tactical AI" behaves.

  • @teamruddy611
    @teamruddy611 4 місяці тому

    This is a great way to learn how to play a new game. You also learn how the game works at the same time.

    • @nathanbaggs
      @nathanbaggs  4 місяці тому

      I’m still terrible at it though (:

  • @phitsf5475
    @phitsf5475 4 місяці тому +3

    Could you compare it to the AI in AoE2? Aoe2 has original and upgraded AIs, and then extra difficult custom AI players have developed

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +3

      I think that's an interesting idea for a video...

  • @heyimflowers
    @heyimflowers 4 місяці тому +3

    This game has ruined me so many times! Fantastic video!

  • @rodrigobogado653
    @rodrigobogado653 4 місяці тому

    wololooooooo. Your videos are a mixture of nostalgia and learning, I came looking for copper and I found gold, I really like AOE, thanks to playing it I prevented them from breaking into my house. It turns out that as a teenager I always played it every morning and on one of those, I saw a thief trying to break in!

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +1

      Glad you found the channel and are enjoying it (and thanks for becoming a member!)

  • @AgentM124
    @AgentM124 4 місяці тому

    If the game is written in assembly and you assemble it and then disassemble it. Does it closely match the original source code? Or does it still completely destroy the structure like it would with higher level languages getting compiled down, therefore being a one way function?

    • @donpalmera
      @donpalmera 4 місяці тому

      I think it would look similar but you'd lose all of the stuff assembly source contains like macros.

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +1

      Depends if there's any transformative process (like macro expansion) along the way

    • @AgentM124
      @AgentM124 4 місяці тому

      @@nathanbaggs sure, Macros are lost, comments are lost. Formatting if you can really call that important is lost.
      Names of labels is probably lost? Since it converts to addresses and offsets.
      I don't know if assemblers have any optimizations in them like compilers do, but that might also affect it.

  • @khatdubell
    @khatdubell 2 місяці тому

    "the game is hand rolled assembly"
    Take another look. It says "mostly".
    I think, especially with the context of the rest of the sentence, that its referring to the graphical part of the game is most likely assembly and the other parts are c++.

  • @bmqww223
    @bmqww223 4 місяці тому

    EXcellent video, please do make a tutorial on how you do this... can i use this kind of knowledge to extract how a racing game implements its physics?or how a character controller is implrementd in a shooter game?

    • @nathanbaggs
      @nathanbaggs  4 місяці тому

      I live stream low level stuff most Sundays if you’re interested

  • @Exotic69420
    @Exotic69420 4 місяці тому +1

    Great video!

  • @realdotty5356
    @realdotty5356 26 днів тому

    Where do you actually get this game?

  • @kkolakowski
    @kkolakowski 4 місяці тому

    I don't think the article said that ENTIRE AoE was written in hand-written assembly, but mostly sprites rendering code. Which would make sense, given that 13k loc, especially assembly - is not really enough IMO for a game such complex like AoE 😅
    I think they wrote most of the game in regular C/C++ using Visual C++ with parts hand-optimized in assembly.
    That would explain vtables and __thiscall right? 🙂

    • @nathanbaggs
      @nathanbaggs  4 місяці тому

      I am fully prepared to admit that I did not do a lot of research on that part, I pretty much stopped at the Wikipedia article

  • @aeonitis
    @aeonitis 4 місяці тому +6

    Love your work 😁
    On the ?? Sections of Assembly you referred to them as rubbish or caves. Can you please educate me on what they really are, and what their presence may indicate?
    I know they're called Mnemonics but not much else 😢

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +3

      Honestly not sure what the nonsense was, probably some hand rolled optimisation. A code cave is just an unused section of a binary you can poke your own code

    • @Bobbias
      @Bobbias 4 місяці тому +3

      Mnemonics refers to the (often 3, but sometimes longer) letter assembly instruction names, like "mov", "adc", "xor", etc. "??" in the mnemonic area means that that ghidra isn't confident what that data is supposed to be, as it doesn't appear to be either intentional data or meaningful code. The "??" itself is not a defined mnemonic with any meaning, it's just an indicator that ghidra doesn't know what that data is. If you were to try to run the data there as code, it could do anything, because it's effectively completely random data. As you might imagine, treating random numbers as code is generally speaking not a good idea.
      Code caves are an informal name given to chunks of unused memory assigned to your process that don't contain actual code or data. They are usually a byproduct of something needing things to be aligned to some memory size. They're particularly useful for hacking or reverse engineering like this because nothing uses that data, so we're free to overwrite it with our own data. This lets us insert some instructions into the process that we might not otherwise be able to insert without breaking something else. If there was no free space in memory to insert your own code, you'd have to overwrite existing code, potentially completely breaking the game. Using code caves, you can insert your own code without overwriting things and make only a small edit somewhere to make execution jump into your code where appropriate.

    • @Artentus
      @Artentus 4 місяці тому

      My guess is it's some kind of lookup table the compiler inserted inline for the switch statement.

    • @mattmurphy7030
      @mattmurphy7030 4 місяці тому

      @@Artentuscompiler? Its hand written assembly

    • @Artentus
      @Artentus 4 місяці тому

      @@mattmurphy7030 according to what I could find on the internet, it was mostly the graphics code that was written in assembly, everything else is C++.

  • @SivakD
    @SivakD 4 місяці тому

    13,000 lines of assembly? I wonder if that's instructions only and other things like assets and directives are separate. Still, it's interesting. I wonder if all the AI code is in the same spot in the binary or spread out all over the place. Luckily you don't have to deal with bank switching.

    • @nathanbaggs
      @nathanbaggs  4 місяці тому

      Something I want to dig more into, some people have suggested that the assembly is just for the rendering code

  • @AEGISAOE
    @AEGISAOE 4 місяці тому +4

    USERPATCH FOR AOE1 WHEN?

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +3

      I'd rather spend the time making more videos (:

    • @1AEGIS
      @1AEGIS 4 місяці тому

      @@nathanbaggs w userpatch

  • @gunnarbradvik8169
    @gunnarbradvik8169 3 місяці тому

    Interesting!

  • @JohnAlanWoods
    @JohnAlanWoods 4 місяці тому

    Another banger. Great content.

  • @DOOM-Videos
    @DOOM-Videos 4 місяці тому +1

    can u reverse doom AI?

  • @bjduncc
    @bjduncc 4 місяці тому +1

    Could reverse engineering the multiplayer next?

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +1

      What an interesting idea...

  • @tmbarral664
    @tmbarral664 4 місяці тому

    Oh oh ! Brilliant ;)
    I did some reverse a few years ago… to … hum…. Cheat 😇
    But that’s another level !
    Love it !

    • @nathanbaggs
      @nathanbaggs  4 місяці тому

      Glad you found and enjoyed this! (I have other videos on cheating...)

  • @GGRS
    @GGRS 4 місяці тому

    Always interesting stuff 👍

  • @ericdanielski4802
    @ericdanielski4802 4 місяці тому +6

    Nice video.

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +2

      You must be watching this on 100x speed (:

    • @ericdanielski4802
      @ericdanielski4802 4 місяці тому +3

      @@nathanbaggs I reverse engineered your video.

  • @Ayymoss
    @Ayymoss 4 місяці тому

    Nice thumbnail!

  • @iamtimsson
    @iamtimsson 4 місяці тому

    2:00 ha ha lol

  • @mrboblox4444
    @mrboblox4444 4 місяці тому +1

    Hello Dearest Sirs, enjoy the video :D

  • @chainbreaker
    @chainbreaker 4 місяці тому

    playing AoE in 'programmer' difficulty, lol. this is awesome content

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +1

      More fun than actually playing the game

  • @user36583658
    @user36583658 3 місяці тому

    i sugest the game "Total Annihilation" cavedog 1997 ,

  • @RubenKelevra
    @RubenKelevra 4 місяці тому +1

    2:15 friendly reminder to register your Sublime text editor, mate. :)

  • @brandonlittle6444
    @brandonlittle6444 3 місяці тому

    There is a rather large following around the Game Command and Conquer, Kanes Wrath,.
    The members are active, I suggest working with that community for a place to make contributions that can be enjoyed actively by others!

  • @iamtimsson
    @iamtimsson 4 місяці тому

    10:30

  • @ButtonBashOfficial
    @ButtonBashOfficial 4 місяці тому +5

    Next stop: Age Of Empires 2 for the 25th anniversary!

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +3

      There's still more to figure out in AoE, but I would like to look at 2 at some point

  • @katyk9483
    @katyk9483 4 місяці тому +3

    So....if the game was written in assembly, why are you trying to Ghidra it? Shouldn't you be reading the disassembly instead? This actually seems like a stroke of luck since there really ISN'T another source you can't see (other than labels etc) and hand-written 32-bit x86 is usually way easier to read than compiled code. I'm just thinking that trying to render a program as C that was never really C might mangle some common assembly techniques and lead to confusion.

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +2

      You’re right and I was surprised how well ghidra handled it. I guess the developers chose to stick to a well known ABI, which helps with RE

    • @minirop
      @minirop 4 місяці тому +4

      @@nathanbaggs the game wasn't 100% assembly, there was some C++, hence thiscall. Most of the assembly was for rendering.

  • @RubenKelevra
    @RubenKelevra 4 місяці тому

    Weird, I always thought the AI is super easy to defeat. Do I remember this wrongly, or was I just good as a kid? 🤔

    • @nathanbaggs
      @nathanbaggs  4 місяці тому +2

      I think maybe I'm just bad at the game

    • @RubenKelevra
      @RubenKelevra 4 місяці тому

      @@nathanbaggs maybe it helps if you know how the AI is thinking 😏