Arcanum Optimization
Вставка
- Опубліковано 7 лют 2025
- I talk about how we optimized Arcanum for both time and space considerations.
Videos I reference:
Optimization: • Optimization
Arcanum Prototypes & Structures: • Arcanum Prototypes & S...
GNW & TIG, My OS Abstraction Libraries: • GNW & TIG, My OS Abstr...
Game Engines: • Game Engines
Please don’t ever stop doing these - your perspective is extremely valuable even to non developers. Thanks for another great video Tim!
Malloc and Calloc sound like ambiguously evil warlocks that control Arcanum from the shadows
There is a third: Realloc.
Malloc as an evil warlock checks out
@@BoardGameMaker4108 Not many people know this, but 'Malloc' is actually short for 'Malicious Warlock'.
@@danilafoxpro2603And Calloc is short for Callous Warlock
Malloc the Greedy, keeper of heaps and hoarder of all things.
He cares not what burdens his great vile excess places upon the world, nor who originally owned the things he takes.
I need more videos on Arcanum's codebase! Great as always Tim!
Thank you for existing TIm!
I'm happy about that too!
@@CainOnGames Splendid :)!
Can confirm, existing is pretty dope.
@@ThatsRaf Yes, but mighty Tim existing is dope on a secret boss dropping the best item in the game level of dope!
As an end user ngl this sounded like a wizard explaining how their spells work
It's so fascinating
I thought I was going crazy hearing sounds and then realized it is the dog sleeping lmao
Yes ! i'm on headphones took me 10mn to really notice :D
Omg thatsssssss what it was LOOOL!!! I came to the comments cause I KNEW I wasn't the only one that heard it. Dang ADHD and getting me distracted at EVERYTHING
If you go through the catalog. The hound is snoozing in a LOT of them 😊
A similar video for VTMB would be much appreciated Tim! That community has been thrown for a loop in need a decent(and Ideally semi-sanctioned) patch for things like the sewer shortcut, fixing the werewolf fight, maybe some combat balances if feeling silly.
Absolutely love all you do, even if this gets ignored, I’m glad to give you some engagement and to be able to watch these fascinating videos!
I'm not really a programmer, more of a Jack of all trades hobbyist, but I love videos like this! The programs I've worked on are so simple, modern computers so powerful, that optimizations are only so important. I love hearing about how the games I grew up with were made. Pushing the envelope and forcing PC's to do things I didn't think were possible! Thanks Tim!
I think that computers and engines are way stronger than what I need for my game anyway. If I could make something that looks as good as Skyrim I would be happy as a Khajiit jester :)
I've got to say, if Tim isn't looking into a career in teaching game design/programming, it almost feels like wasted potential. But this channel's still amazing for all the information we get from it.
Thank you. I have taught a class before…I’ve considered doing it again.
My University Game Development Class
ua-cam.com/video/RY_tqjx_iFc/v-deo.html
Man, the bit fields thing explains so much.
I had to reverse engineer PROTO files with a hex editor back before there were community-made tools.
Almost all the weird things I remember struggling with trying to modify items can be explained with your bitfield system. I remember pulling my hair out trying to figure out why some properties just didn't exist or were in a different order. I did eventually figure out they were being pulled from the PROTO files, but I never figured out how the formatting worked!
I wish you'd started this channel a decade or two earlier!
Thanks for the geeat video, i love these more technical ones!
Very informative! Thank you! I'd be thrilled to see videos on a "toy" isometric rpg, where you discuss/demonstrate different subjects you find interesting.
Come on Tim, spill it, release the fixes, release the source code... the feds aren't watching.
On a serious note, thanks for everything you do, Tim.
Between Tim and Mortismal, they’re definitely up to something. Doubt they could say
Thank you, I loved Arcanum, tough I never finished it. And this explanation gave me insight into the technical puzzle of game making.
Recently, I have been enthralled by Lost Kingdoms, a game for Gamecube, by From Software. Basically, Yu-Gi-Oh, but the cards summon creatures that fight in real time. I would like to make something like that some day, perhaps. :)
You definitely can control where memory allocation goes in C++.
You can redefine your memory allocation routines (much the same as rewriting malloc in C), you can use "in-place" constructors to construct an object at a given address (provided you allocated memory there correctly before). You have full control on memory fragmentation in this language.
C++ has got flaws, but not this one.
The one question I expected when I saw the title of the video and you didn't answer is the following:
Initially, the game slowed to a crawl in certain areas (particularly the elven forest in the woods). If I remember well, the fact that some roofs/walls/trees/overhead graphics were shown caused the game to slow down a lot. As in it litterally took minutes for my character to move from one end of the screen to the other.
I don't remember if you patched that or if I got it fixed when I changed computer (the latter being more likely).
So:
How did you optimize or not optimize this kind of issues?
It may have been hardware-related (certain graphics cards) but I could, at the time, almost find what algorithm you were using to paint the map because it was easy to see that you were painting stuff over other stuff and that took forever (either that or computed that you must hide the thing, can't remember now, it's been a while).
And in today's context, how would you optimize for that kind of problem, in particular if it's hardware-related?
I suppose Early Access gives access to a lot more hardware you would otherwise be able to test, but what solutions do you have?
Isn't using a tested Engine a way to minimize this dramatic performance issues risk?
This was really good. Had to watch bits a couple of times to let it really sink in.
Man i wish you could make that arcanum sequel you've got so much passion for the game and the world.
That prototyping system is fascinating. I looked up vtune and I was really surprised to see that it works with Rust (my language of choice for my engine)
VTune was so good! I used it so long ago it would show you U and V execution pipe stuff on ye olde pentium.
Great answers! My plan was to learn something new/get insight from you, so could use it on my own game project.
I didn't understand any of this! It was awesome!
You absolutely can do your own manual memory allocation and management in C++, it's just not taught that way, for some reason. Many of the other C++ programmers I've run into seem to lean on classes when it's not at all necessary, or even might be counterproductive.
My terminal CRPG had a single, long char buffer for the screen. I used a formula to translate the memory block into 2D coordinates and back, and any adjacent map data like interiors was stored in a similar heap-allocated format that I wrote a special allocator for it to maintain the layout and data.
I prefer to have total control over my code, so my C++ looks a lot like C code most of the time.
I gotta say though, I'm really glad for OOP features like classes. When I do need them, they save SO much time and effort. I just wish more programmers would work at a lower level and not rely so heavily on it.
That's actually the reason I stopped using Unity, I didn't like how little control I had over things.
Your solution to that was really cool! I'll have to keep it in mind if have to use another engine or library again
The Prototype System sounds alot like how Parent and Child Blueprint classes work in Unreal.
Closer to entity component system (ecs) brian bucklew has a great video about it, he co-created caves of cud a roguelike game. Unity is closer.
ua-cam.com/video/U03XXzcThGU/v-deo.htmlsi=34ypJyNb-cD4Lyoq
Darth Dog is clearly in the background.
I have never* really had to allocate or manage memory, but honestly this makes me kind of curious and excited to try!
*Except for that one time when the Citrix team accused my app of taking up 8 TB of memory across the farm, but then when I couldn’t replicate it turned out they were off the mark… by 3 orders of magnitude 😅
That was some good cake I got as an apology 🍰
Well, a 't' does looks like a 'g' doesn't it?
@ Totally. Practically the same letters.
My god
At least they apologized properly, it sounds like.
@@Anubis1101 After being shamed into it by my boss, but yeah 😉
very cool! optimization is my favorite part of game development so far, I just can't help myself but skinny down those bits y'know?!
Great info, you never fail to make me wanna make games. You can use C++ for only some of the features, but not use classes. Then you can have complete grip on the memory thing, correct?
Almost any C code is valid C++ code, so it's more important you build good practices. If you're interested in memory management, you can learn C first, and transfer that knowledge into C++ without much trouble.
Just because classes exist doesn't mean you have to use them; learn when they're useful and when they're not.
I'm no programmer, but only storing the delta from the base version sounds a lot like Unity Prefabs. Was that not a common optimization that many years ago? I feel like that is so obvious to me, but maybe that's just my view standing on the shoulders of giants.
It's Scriptable Objects in Unity.
It was common to split shared data and instanced state. I think what Tim refers to is a very specific way to store data in relative offsets, which probably means different instances could have a different amount of used memory, which would not be possible by just having a struct with only some fields in use.
Thank you Tim!
Bit fields are a neat way to store information for up to 32/64 booleans. Alternatively you could simply have booleans like "HasHitpoints." Another "trick" is to utilize more of their bits. If you take 4 bits, you have 2*2*2*2=16 potential values. With 8 bits you have 256. For example I have 64 bit IDs for solar systems, from which I deterministically generate names for solar systems. The first 4 bits determine the format, the other 60 are used for that specific format. For example you want to generate 3 characters, you take 20 bits for each. So you'll get 2^20 (~1M) potential values per character, you run a modulo over it (10 digits + 26 characters), convert the result into a character, and voilà. But that's an overkill, 6-8 bits are sufficient for a character (the lower the potential values, the worse the distribution).
Anyway, this (and more) is how I deterministically generate solar system names that are... unique enough. Like: "X54-Wolfram TQ"
Did you know, that the whole universe of the first Elite game was stored in 3 x 2 Byte values? The rest or each sector was procedurally generated from these 6 bytes.
@@OpenGL4ever 6 bytes can represent 280 trillion numbers... that is sufficient to have unique IDs. But that's not how a universe is "stored", that's just the seed, which is optional.
The keys are the algorithms. And it's conceptually simple: You split the galaxy into chunks (3D cubes) and get X,Y,Z integer coordinates. Then you can take these, plus the ID (=seed) of the galaxy, and you can fill individual chunks with solar systems (amount, position, type, size, ID) using the X,Y,Z-coordinates and the galaxy ID.
Practically (doing all of it and making it actually work well) it was of extreme difficulty. But not as difficult as doing the scalable single-realm global multiplayer system. Starfield comes close to it, allegedly.
For anyone interested, the object compression is the same as prototype subclassing in JavaScript.
Hum. So you implemented something akin to JavaScript prototypal inheritance. Very interesting. I might steal that :D
Please Tim, don't let Matt Hansen's rage destroy whole Obsidian, I love that company.
Trying to finish Arcanum for the first time at the moment. Have 130 hrs in, last time i got farther but didn't finish.
Honestly gave up linux gaming to play and replay your games. Namely Fallout and arcanum. That's how good they are as art. Even the "less good" ones. If i win a big lottery, we're putting together a team.
arcanum works fine with wine, I've played it this summer 🤷
Arcanum works great with wine/proton, Fallout works out of the box on Steam thanks to proton (I just haven't tried any other version)
Funnily enough, I found getting older games to work easier via steam's proton selection than on windows.
Not sure if you mentioned in your prototype and struct video but when you did +1 sword prototype (as an example) was it a prototype of a prototype or did you create a clean prototype with all its fields filled in?
In our system, you could either make a +1 sword item that was an object that referenced the sword prototype with an override on the damage and to-hit values, or you could copy the sword prototype and change its damage and to-hit values to make a +1 sword prototype that items would reference. While I think our system could have been extended to make nested prototypes, I think the small space savings would have been way less useful than the time overhead to go through the hierarchy.
I always find it interesting when despite all the memory available nowadays a programmer still optimizes the code. A good example this year is Animal Well. Theres a lot in that game but the total file size is 33mb for the whole thing. Fair enough it took seven years to make but in all that time they were consciously making sure they weren't wasting resources they didn't need to.
I get that its less important but when I see some games that are 100GB to download and they aren't monstrously big games or really pushing graphical fidelity I do wonder how much of that space could be cut but just isn't because people don't feel the need. I wonder if performance would be significantly improved on some of them if people were just more careful with their code.
Big agree, I think there's a lot of waste.
But it's hard to tell for sure since we don't have access to source code.
If I had to guess, I'd say textures. High-fidelity models in AAA games can have a dozen textures, maps, and other data that really adds up.
We can hear the dog. Can we see him too?
Coming this Friday…
Also, you can opt to not update your Unreal version (which is something they advise if you are well into development in a given engine version).
4:10 Would you still recommend storing data as individual bits for boolean values and querying them using bit masks or would you simply use a whole byte? After all, today's computers have many times more RAM and if I am not mistaken accessing a byte is usually faster than accessing a single bit.
9:47 Today we would use libsan with fsanitze for this purpose to find such memory bugs.
12:26 Have you also grouped NPCs that are further away so that you only have to perform a single hardbeat calculation routine for the whole group instead of for each NPC individually?
Thank u so much for more Arcanum videos 🥹
What are your thoughts on the recent news about obsidians racist and sexist hiring practices? I was shocked when i read that this morning😱
Much love tim, enjoyable video as always!
I looked this up in case anyone's curious. It's just Elon Musk and a bunch of creeps getting upset at an Obsidian employee going on twitter and saying there should be more diversity in game development. The outraged parties are Grummz, Elon Musk and Asmongold. If you think any of those people are credible sources then I encourage you to go outside and remember that life is short and you're wasting your fucking time with these culture wars.
This all started because Avowed lets you select pronouns. Yeah. Who gives a fuck? I'm gonna go ride my bike.
@PlaylistGeneral not sure what you're rambling about friend
Arcanum need the remaster so badly T__T one of the best game i ever played =b
Was it possible for the prototype to also refer to its prototype? So that the +1 Sword can refer to the default Sword for all its properties that are the same as the regular Sword, add its overrides for hitpoints, colour, whatever.
It is possible and preferable with my little understanding, i wonder if they did this for arcanum
@@benhickson6149 Yes exactly, that's what I mean. I'm curious if they did it that was for Arcanum?
I know it's possible in principle.
In our system, you could either make a +1 sword item that was an object that referenced the sword prototype with an override on the damage and to-hit values, or you could copy the sword prototype and change its damage and to-hit values to make a +1 sword prototype that items would reference. While I think our system could have been extended to make nested prototypes, I think the small space savings would have been way less useful than the time overhead to go through the hierarchy.
@ Thanks for satisfying my curiosity! I understood from the video the two options you gave here were how it was generally done in the game - I was just curious about pushing it further.
But apparently the savings wouldn’t be worth it vs the overhead- always a trade off, it does sound like the overhead is already considerable.
What's going on over at Obsidian
They're very busy, I think
I wonder if Tim would do a video about "game boards", like hex grid vs square grid vs free motion in strategic RPGs (or if he has already)
Would you react/give your opinion on N+2 programmer video? We need it. The video title is:
Casey Muratori Smart-Pointers, RAII, ZII? Becoming an N+2 programmer
Hehe you can hear the dog sleeping in the background
Even just stuff like reducing texture resolution and making sure meshes aren't too high poly makes a huge difference over every model. A broom model with 60k triangles probably isn't a good idea and not necassery at all, same with a tyre on a car having 8k textures.
Hey Tim, after recent events, I have to ask. What do you think about game devs making intentional design choices in a way that will upset those of the player base with whom they have personal disagreements?
So object prototypes mentioned here are basically the same the Prefabs in unity?
They look like a lot but I think they are more optimised.
The prefabs, in-game, have instances of all the components you put on them. For instance if you have a collider, you will get an instance of the collider component even though you only changed your object's transform, you still have the weight of the whole collider with you.
Personnally, for items in unity, I have items reference an itemdefinition (type) rather than use prefabs. I instantiate a prefab with the stuff the engine needs (graphics stuff, transform) but all game data is on a non GameObject which I can then manage. I haven't gone as far as using bitfields and optimizing memory because, until I measure it to be a problem, that would be a huge waste of time, but it would be possible to do only because I'm not using a GameObject.
It called entity component system (ecs) in the industry
ua-cam.com/video/U03XXzcThGU/v-deo.htmlsi=34ypJyNb-cD4Lyoq
As someone who plays Arcanum at least every 2 years, I gotta say, what optimisation? :D
I'm curious. The Arcanum system described here sounds like an entity component system rather than having an OOP approach. If so, does anyone know if there is a signifiant saving by having entity varients using this bitfield method to point to a template entity compared to just having all entities contain the full list of enums and values even if they are very similar?
I would assume so, ecs is so much better than oop by the way, but modern games do not need to worry so much due to more memory these days. Sounds like you are dabbling in game design
@benhickson6149 Been doing game design as a hobby for around 3 years now but had a fascination it for decades. First started mapping for the original Quake and then progressed through to dabbling in modelling, audio design and then programming. Didn't really have the time to invest into it as a hobby until my kids became late-teens and could provide for their own needs.
The indie community is actually very supportive so I came across ECS during talks with a professional developer and brought me up to speed on it. Fascinsting stuff but not as cost effective as you think since the overheads caused by running everything on the tick only begin to outperform OOP as the simulation grows more complex. So for something like an FPS not ideal, but for the kind of open world simulation that Tim describes here it would be great.
Fun Friday suggestion: Which or who is the ugliest, most unsettling creature from any game you've ever made ?
Your such a inspiration i want to make games so bad, but just cant sit down and try to learn the techniques or how to use the engine i have (unreal 5). I can conceptualize things but even trying to do a GDD its difficult. Wish i had a group to talk with.
How do you decide which systems/ menu views are the core of your game after you’ve planned an RPG stats system?
I do my best to describe that here:
ua-cam.com/video/aDE69s3pp7M/v-deo.html
Speaking about optimization, do you familiar with Id software "Fast inverse square root"?
I would really like a new Arcanum but then in 3D more like the Outerworlds. Would that even be possible now that Microsoft owns the license.
Talking about one of my greatest Gems :) ( Not saying Game esspecially :) , it's a well carved shiny object ) ;
I just wondering something Tim I don't know if you even remember or would be possible to talk about it or maybe comparing with Arcanum would be great .
I have a modder friend who is one of the modders working on some Fallout 2 modding project . And he is trying to figure out how to encountering system works in Fallout 2 and there are quite a bit documentation of scripting and also like how monster types flagged in Fallout 2 .
1) When encountering something if i understand it right within the code i was looking not 100% sure on that some of the codes are referenced functions so can't see what's it's logic inside but it seems that there is first encountering roll i assume ? Like X percentage if player rolls between these values ? player is in encountering section , then enemy type list triggers it's related into where you are in map right now like X_TYPE_ENEMIES_1 list calls and there are some percentage numbers in there it's kind of weird maybe the scripting part may be wrong but when summing all those percentages it's most likely above on 100% but there is also sub function that has in those enemy types which calls after these rollings , those enemy spawning percentages summing up exactly into 100% , kind of mixed algorithm behind it maybe and would like to curious about does the Arcanum has kind of same logic behind these encountering system as in Fallout ?
2) Someone made a stealing graph between NPC and Player based on how the percentages most likely work in Fallout 2 like shows your caught chance , if it's true it seems that NPC has a steal skill as well as the player so when the NPC skill is high in some percentage player skill goes almost exactly in same direction of being caught of chance , like if NPC has 50% Steal Skill and Player Has 50% skill also it's colored like Red , shows like you most likely get caught stealing . But if player skill is exceeding 75% no matter what NPC Stealing skill is it's most likely colored as in orange like there might be chance to get caught by NPC . How Arcanum apporach into this methodolgy also if i remember right there was Pick Pocket in Arcanum and it has like 20 points limit into ? and you became master of it when maxed out .
Prototypes = scriptable objects in unity?
Hi Tim!
Arcanum is blocked from the Steam shop in germany right now. If you know someone who can push the game through the Steam age rating verification form please let them at least know.
Thank you!
Can we petition MS to allow you to release You updated Arcanum? or will that get you in trouble for making it in the first place?
By "Arcanum", are you refering to Arcanum: Of Steamworks and Magick Obscura?
Wait there's another one?
@@Anubis1101 No. What other game would I be talking about? Its the first search item and the only game with such name that appears whenever I search "Arcanum".
@@OnyeNacho yes, that was the joke; it's the only game with that title, and this is the channel of its principal creator.
So I wanted to know why you were asking.
At best, I was hoping there was another game with that title that i'd just never heard of before. I could use another good fantasy RPG.
@@Anubis1101 I wasn't sure. At first I thought you were trolling, but then someone else literally asked me in Discord if there was another Arcanum game coming soon, so then I took another look and suspected maybe you were genuinely curious.
This is the decade of remakes, remasters, re-imaginings, re-releases, and exploitative sequels after all.
@@Anubis1101 Yeah, I started playing it this month after buying it on New Year's Eve last year on discount. I am really loving it. I am not usually a fan of isometric RPGs, but Septerra Core managed to win me over, so I reckoned it was no longer a surprise that after discovering just how unique Arcanum was, I practically fell in love with it as well.
Not since Septerra Core in 2023 (and by extension since Fallout Tactics in 2013) that I ever enjoyed a classic isometric RPG game. These classics surprise and satisfy me more than 90% of modern titles these days (yes, even the indie games).
Doggo is sleeping while Tim talks to his bleep bloop box so he can maximise his awake time with dad
I've felt like a Hobbit seeing Gandalf talk about how to magic. I mean, I can understand the concept, but the idea of things moving in memory, how a handle can know where to point, slide a block to free a memory... Actually, drop the medieval metaphor, this feels much more like cyberpunk (2020/RED/2077).
I might be biased, but it doesn't seem so hard if you thinking about memory as an array of bytes, and pointers as indices into that array. For handles, there could be an array of handle slots, to which each handle points to. And each memory allocation, in turn, has a pointer to the slot it would need to change if moved.
Arcanum in title? Thats an insta click from me bud
Hi Tim.
Is someone snoring in the background?
there needs to be a npc in arcanum 2 (c'mon tim, just do it) that understands through "mystic dreams" that the world is run on code, or has some sorta metaphysical understanding of things that we the audience will see is a metaphor for code and how computers/programs work. Obviously everyone considers them a lunatic. Also they should have a quest where they send the pc to find a "prototype" object and then perform a magical ritual to destroy it, if the pc does so they remove all instances based on the prototype from the rest of the game.
I'm not able to see my comment after I posted it, but I hope it went through :(
What is that breathing? A dog?
Is there a dog or something snoring in the background? 😋
Moar of this please!
Of course you can control memory in C++ thats like the only reason anyone puts up with it anyway.
Am i going crazy or i hear someone snoring in the background 🤣
Your not crazy I can hear it too!
Arcanum was optimized?
arcane has jump the shark and you still talking about aracanum