Such is the life of a coder. Tho to be fair, it's easy to blame such mishaps on the programmer when really it's quite hard to figure out such extreme scenarios as presented here without extended periods of testing against wild theories. Needless to say, you usually don't have that kind of time or money to cover for this, whether you work on a game or otherwise professional software.
This is why I’ve made it a habit to always check for all stupid, implausible edge cases, but hide them behind a constexpr if so they can be disabled if performance is a concern. Modern C++ makes balancing performance and safety fun and easy
"these conditions should never happen in the first place, right?" Also said some nuclear reactor designers, aerospace engineers,... Any complicated system have loopholes where glitches can occurs. And the more the system is complex, the smaller is the possibility to get it 100% bulletproof from the get go.
One obscure correction: At 21:58, you state that the reason Mario immediately drops the cork box while the bobomb thinks it's held is because the two objects are colliding, but the actual reason is unrelated and IMO somewhat interesting. When a bobomb is held normally, it has a fuse timer that counts up every frame. Once this timer is greater than a critical value, the bobomb sets a bit every frame telling Mario to drop whatever he's holding (expected to be the bobomb) so that it can explode. This forced drop only requires the bobomb to think it is being held, and so when you go to grab the cork box, the bobomb still tells Mario to drop it. This can be circumvented by overflowing the bobomb's fuse timer, at which point you can grab the cork box or any other object normally, and the bobomb will still be invisibly suspended in the air before Mario. Unfortunately, it takes over two *years* for this overflow to occur, so nobody's actually done it on a real console. Everything else in the SM64 section seemed pretty accurate to me though, and is a great explanation of what's going on!
tbh I love that you put human-made captions on your videos, it's super nice for me as a hearing-impaired person, especially in the wake of UA-cam removing community contributed captions
@@RGMechEx I know this has been most likely asked before probably, but it would be really awesome if you could make a series about the Nintendo64, similar to the SNES series as well.
@Lorash I guess that depends on the language it was written in, and the compiler. Like maybe you compile a C Hellos World with some buggy compiler and the resulting machine code can operate in weird ways under given conditions? So not even the simple hello world is saved from bug hell :)
....... right? Wrong! You should perform a bounds check. But that takes more CPU cycles, more code and thus more space. Oh, well in that case. It should be fine, it will always be valid anyway...
The thing with bounds-checks are the classic "You don't need them if you just program well. 4Head" If everything worked in the code for the Pokemon-itemmenu example, then you wouldn't need these theoretical max boundries of 0 and 20, because it's normally impossible to get above 20+cancel or below 0 anyways. The bigger question would be to figure out a way to only let the ID instead of the quantity be recognised as a potential terminator. After that, it works, all other errorcases are caused by unrelated glitches messing up the redundancy-value directly and in that case, we should just fix that instead of wasting time and romspace by placing unnessesary failsaves.
The first Gen Pokémon games were a marvel -- they were technologically very clever to pull off everything they did, but that cleverness left us with a lot of... These situations.
@@n3rdv10l3nc3 Yeah, the cleverness unfortunately involved cutting tons of corners and making lots of assumptions -- but it's okay because it makes for some very entertaining glitches :]
@@n3rdv10l3nc3 Yeah, all the cleverness came at the cost of cutting tons of corners and making lots of assumptions -- it does make for some very entertaining glitches though
the sm64 part was really fascinating. i've been watching pannenkoek2012's and his uncommentated channel videos explaining unintended mechanics and glitches in that game for quite a long time. while he did explained some things in great detail, the "cloning" mechanic and the whole "hands-free" ordeal weren't explored with this much detail. this video really clears things up. i went from "i.. kinda understand...?" to "oh, i see!". good illustration, clear analogy, and great presentation. all around.
At least the aspect of holding things and cloning is explained somewhat better in his newer video "How Holding Objects Really Works" (compared to "The Science of Cloning" on his main channel, which is older and more convoluted), which uses the terms "ticket" resp. "slot" to talk about a pointer to the held object. ua-cam.com/video/BZyUGT9YPhg/v-deo.html
@najwan Yep, it would also be cool to see a full-on co-op video on SM64 by pannenkoek2012 and Retro Game Mechanics Explained. They *DID* interact in the comment section of RGME’s video on ”Credits Warp” on SMW, on his other channel, and Pannenkoek even helped in making this video, so they’re *DEFINITELY* at least somewhat familiar with each other.
@Serrara Willow Mayfield - aka Fluttershy Makes sense. Another alternative would be ”Wrong Grab”, as in DK64. Also, bad_boot has also called ”cloning” ”a bit of a misnomer”.
@@donalynlavarias7385 It’s a reference to how buggy both games can be, and people thought that was funny. Not sure why you felt the need to ask honestly.
A great year for games with exploits speed runners would use years down the line. Data Redundancy errors is a frequent source for the glitches they use
My main question is why do objects that aren't meant to be thrown stop working when they are thrown. Pannen describes these phenomenon as the "one touch rule" and that clones "can't send or receive signals". Like those are nice colloquial explanations, but I was hoping for a more technical explanation. Edit: So because I've yet to hear a sufficient explanation, I just looked at the disassembly source code and figured out myself. In Mario 64, every object contains a behavior. This behavior is merely a pointer to an array of small scripts. As the game runs, each object runs through its list of scripts, most of which just call that objects loop function to update it's state. How throwing objects work is by the method `obj_set_held_state` which takes an object and a BehaviorScript. The behavior scripts used (bhvCarrySomething3-5) in question actually do nothing: they begin, then immediately end. What `obj_set_held_state` does is check if the object is holdable, then if it is, it checks if the passed BehaviorScript is any of those three. If it is, it sets that object's `HeldState` to either held, thrown, or dropped. This is done without every running the script However, if the object is not holdable, it will overwrite that objects behaviour variable with the passed behaviour, which remember from above, does nothing. For example, with the coin, this results is the coin not being able to perform its update loop, as its behaviour is now the do nothing behaviour. Now despite this, why can we still "collect" the coin, and why doesn't it disappear. That is because collision is handled completely separately from an objects behaviour. When an object is initialize, it registers a hit-box along with an interaction type to it. A completely separate method is used to handle interactions. When Mario interacts with a coin, that coin's interacted flag is set to INTERACTED and it is here where Mario's coin counter is incremented, and 100 coin start spawns. Normally during the coins update loop, it checks if it is interacted, and if so, spawns sparkles and unloaded. However because the coin's behaviour was overwritten, the loop never runs, hence never de-spawning. Now why are you only allowed to touch it once, that's because the code to handle interactions completely ignore interactions between Mario and an object whose interacted flag is set to INTERACTED, and similarly to despawning, it is most object's update loop which un-sets this flag.
My first guess would be that these objects don't have a behaviour associated when they're being thrown (since they're not supposed to) and they are set to a state where no action is defined (like moving, updating sprite, despawning). So they exist but have no way of doing anything. Unfortunately, I find it hard to find the code that'd be relevent in the decompilation and am not even sure I could even read what it would do. So these are just assumptions
@@sacrijuts7265 i thought that too, but I feel like that should crash the game if it didn't have some for of error handling. And if they did have error handling, their decision on how to handle it seems weird to me, which makes me think that there isn't any error handling. That confusion is why I want the amswer
@@zeropointer125 maybe it's related to the physics then. Maybe it says for them to be pushed in a certain way, but they aren't supposed to move so they don't.
Congrats on the channel's 4 year anniverary :) Honestly one of my favourite channels with such well edited video's every time! (Even though I don't fully understand them all xD;; But they're still very enjoyable!)
Over a decade ago I saw a glitch compilation where somebody teleported while holding the penguin, causing the penguin to be seemingly erased out of existence despite still crying away constantly and having its mother be thankful for their return (but not actually give a star for it). Great to finally an in-depth and clear explanation as to what the heck was going on there!
"erased out of existence despite still crying away constantly and having its mother be thankful for their return (but not actually give a star for it)." 15 Most Heartbreaking Anime Deaths Of All Time
it would be interesting to have a modified ROM of Pokemon Red that has issues like these fixed. then give them to speedrunners and see if they can find more exploits... repeat this cycle until you have a near-bug free game.
Redundancies can't just be "fixed". But you can certainly add in sanity checks that intentionally error/crash when something that shouldn't happen does happen. If the check is only in the debug build, it's usually called an "assertion", from the pseudo function assert in C commonly used to implement it.
@@henke37 but why can't it be fixed? if you for example change the "remove item" function to check the Item count before actually starting the fuction, and only check the item ID for 255, and stuff like that. why isn't that "fixing" them, when it does fix the issue? it's just error checking to make sure it never gets into an invalid state.
Fixing this bug wouldn't do much. For it to even happen, the game already has to be messed up. It would be better to just fix the shore tiles, which they actually did in later revisions.
This exact issue appears in the Japanese version of the GameCube game "Megaman X Command Mission." The Force Metal UI screen (the game's non-weapon equipment) is one box larger than what the list is programmed to be. The final box is "shared" with the Main Weapon list which is stored directly after it. Placing an item in the final slot will allow you to "take out" said item as a weapon on another screen. Most famously, Force Metal #25 "Thunder Guard" becomes Weapon #25 "X-Buster Mk 3", which is X's best weapon.
your production quality was already stellar and this video and the previous one are again on an entirely different level. you really deserve 10 times the number of subs you currently do, easily.
For a spectacular example of things like this going wrong in a modern game, look up a Hollow Knight Any% All Glitches speedrun, which has example after example of game state being desynchronized intentionally to benefit the speedrun, breaking the game completely and allowing them to beat it in less than 15 minutes with no real bossfight along the way. (They only "fight" the Hollow Knight, and do so by duping a room from Crystal Peaks into his arena, causing him to commit suicide on the crystals.)
It shows you just how good Team Cherry is at debugging and patching...and how bad they were in the past about sanity checking. I'm confident Silksong will be WAY more concrete in its codebase.
I imagine passion has something to do with it. It’s easier to be super passionate about a single lecture that you spend months preparing, but if you have to do it again and again every six months, it gets to be more effort than it’s ultimately worth to do that much work. A lot of my professors were also pretty new, so maybe that’s also a factor?
@Lorash As an engineer who was a professor, it's more likely that they didn't want to settle for the money a professorship has to offer. As a senior Instructor at a state university I was making 35k with benefits, as a principal architect at my current tech firm i make 125k with annual bonus and benefits. The difference is drastic sometimes, you need to be okay with making a lot less money.
At school, number of potential teachers = N. On youtube, number of potential teachers = N*1000000. Larger body to choose from = greater ability to find outliers.
@@B3Band thats a problem with your country (presumably its the USA), my degree in germany costs me 6 semsters *300€, for which i even get 6*150€ government assisted
I was actally about to say that many of the issues described in this video are still prominent today. This video is great at describing them, so I think there is some merrit to what you're suggesting.
(I'm around 4:46) oh, so the count variable comes first! That explains the weird glitch I did in Virtual Console with my transfer box 😅 I forget if it was Pokémon Gold or Crystal, but basically I discovered that if you change boxes to any box other than box 1, and turn the power off when it says not to, the Pokétransporter app will read Pokémon data from this box instead of box 1 (which it's hard coded to do), and will subtract the number of Pokémon in this new box from box 1. My box 1 is currently at a negative number I think and it's eating into the Pokémon Center Vs data table
funny enough, I encountered a data redundancy error similar to the inventory one just a day or so before watching this. A Minecraft mod known as Not Enough Items (or NEI for short) provides server admins and others with access to the "/give " command a GUI for doing so, but instead of just dropping the item on you like that /give command does, it inserts it directly into your inventory. Inventory can have at least 36 slots, so finding an empty slot wouldn't take too long, but to save time anyway, the mod keeps track of the last slot it inserted an item into, incrementing it each time it inserts. However, this redundant pointer is not bounds checked to make sure it doesn't go past the end of the "main inventory", resulting in storing junk into your armor slots and possibly eventually overflowing the data structure completely. Fortunately the pointer is reset and starts again at the first empty slot if it reaches a full slot, but I didn't figure out how to make it do that until yesterday, just hours before watching this video for the first time. I swear, the UA-cam's algorithm is psychic sometimes!
now, the question is... does Minecraft 1.7.10 (the version I was on) qualify as a retro game? I mean, it's not the version everyone is playing, but it's still playable... jk, if this channel does a video on ANYTHING involving the inner workings of a Java program, I'm going to worry about isoFreeze's sanity
Your visuals have so much polish and effort it's insane. I wouldn't understand very well if you had just explained, but the detailed visuals add a whole new layer. Favorite content on youtube hands down.
I love these videos, despite them being a bit beyond me. And I gotta say that your animation game is stellar, very slick production without being distracting. Very well done.
This is also what Stale Reference Manipulation in Ocarina of Time is. It is very similar to how grabbing an object in SM64 functions, except the actual object is being moved by updating its position relative to Link's position.
Have you ever thought about looking into 8/16 bit era passwords that were used for saving game state? Such as Super Hang-On, Metroid, MegaMan, Castlevania etc? There's some interesting systems that programmers came up with for generating/validating (or in some cases not) the passwords.
TheZZAZZGlitch did a similar video -- he didn't really explain exactly what was happening, but he did show off some silly passwords he managed to generate for a number of old games
Thanks for this amazing video and thanks again for letting me use your Pokémon background! You really understand the art of teaching CS with fun, games and glitches.
Thanks for this. Pannen's explanations of things are great, but for me personally, having it explained via code terminology allows me to finally actually get it.
Hi! I just wanted to say I really enjoy your content and all the effort you put into understanding and presenting all of this technical information. The Missingno glitch videos are some of my favorites, which is why I wanted to leave a suggestion about another glitch I think would be nice to look into: the Sword Brothers Chapel Exit glitch, for Castlevania Symphony of the Night. It's supposedly one of the first glitches discovered in the game, in which you simply perform a a spell in the Royal Chapel area, and that's enough for the door's transition trigger to stop working for some reason, allowing you to fly out of bounds and gain some extra map completion. I couldn't find any technical explanation as to why this happens, though, so I thought I could mention it here :)
Man, as a music teacher, I'm in awe of your teaching skills. I don't know much about programming at all, and I feel like I followed you the whole time.
i still admire how well you can explaing pretty complex things by neatly demonstrating it in the actual game right next to diagrams and charts that are animated in sync! i love that!
Great video and explanation as usual! Although in my mind the term "Data Redundancy Error" has a very different meaning in other CS contexts (such as in relation to parity checking, Reed-Solomon coding, etc.), so was a bit perplexed at first. lol
As a developer..wow..your videos are just insane.. so detailed, well animated and explained in a clear way that anyone should be able to understand, at least on some level. Instant sub.
Honestly, showing what goes wrong is a great way to demonstrate how safety is done. Like race conditions, which you might not even think of even know of when coming across one for the first time.
I love this channel. It's like...interesting answers to questions I never asked. I really miss the simplicity of old hardware. Such easy to manipulate memory. The days of switching on my NES game genie and entering gibberish codes to see what we can mess up or make interesting, almost never resulting in a simple crash, but almost always changing some weird data that causes the screen to erupt into weirdness or what have you.
Something about the way this video presented the Pokemon list actually made me figure out a potential way to fix a database issue I'd been having in an app I've been working on! ...I wasn't even having a redundancy problem. XP
Watched a few videos, subscribed; this is exactly the kind of deep _technical_ insight into games that I find fascinating as a perfect supplement to all the game design essay channels I am subscribed to.
Whenever someone starts talking about invalid states in Pokemon, I know that it's going to come back to the mother of all glitches, Missingno.. The moment you started talking about the item list and the terminator being 255, I knew where you leading; plus the "not "less than" but "not equal to" was a pretty solid hint that it's going to be a value outside the expected table - which happens with the item part of the Missingno. experience.
2:10 this actually brings up a big debate in CompSci over whether it's better to have a list (such as a string, which is just a list of characters) keep track of its length or whether the end should just be indicated with a special byte (usually 0x0, aka NULL). Both have their own advantages and drawbacks, and there really is no "right" answer. A null-terminated list could theoretically be as long as it wanted, while a length-specified list would either be limited by the number of bytes used to store the length, or would have to figure out a way to have a variable-length list length indicator. Of course this isn't terribly relevant in modern programming or game development - since most coding today takes place in high-level languages that handle stuff like that for you, most programmers don't have to care whether or not a list's length property is calculated on-the-fly or read out of memory.
these videos are so well-explained! i can imagine exactly what the code would look like, and i think that even people who don't understand code would be able to get a sense of what's going on.
When a video is so good you don't even realize 20+ minutes have passed. Seriously, I was here expecting the next example and boom the video ended. It's been a while since this last happened to me.
My favourite dynamic list/array header in 64bit mode overloads the array pointer with an item count field.. I'm also developing a 'clean object, dirty pointer' object pointer system too... I've gone with a 44 bit address field and 20 bit count for my main list type. Million items is enough for most lists. Million classes is a lot too.. Only takes a shift or an and mask to extract a field and it's FASTER than storing the size field at the start of the array.. Class system is also no slower than putting a class pointer at the start of an object (with virtual methods) as it uses a constant array partitioned by 2 power n bytes so just an extra shift in total.. Bit wasteful as each class with a virtual method table takes up as much space as the largest VMT but saves 64bits per object so easily makes up for it.. Access is as fast as a standard C++ object. All dirty pointer ops are inlined in the base class so no procedure calls added when accessing the overloaded pointer fields...
The game Human Resource Machine is really great at making you think like an old console programmer - your resources are VERY limited and you have to do some crazy-risky stuff to meet their performance targets. It's a great puzzle-programming game that has made me appreciate those who program in assembly.
19:44 - Cloning can be abused in ways that are pretty broken as the HOLP is also screwed - i.e. it doesn't update, so when it is thrown, it will go to the HOLP which is no where near Mario.
2:30 For a linked list, maybe. For an array-based list (that isn't null-terminated), there is no simple way to tell the difference between a list item and random nonsense, so that external count is actually the only reliable way to size the list, so I'd say that calling it "redundant" is a bit disingenuous.
its "redundant" because there is a list terminator; similar to how storing the length of a c-string is technically redundant because it is terminated by a null byte
The information about what each memory address was responsible for? I got that info from Glitch City Laboratories, but I think that website shut down so you can only find an archive of it now.
It's fascinating that the vagueness of a land border graphic causing a developer to forget to add encounter stats to a strip of land made it possible to manipulate an encounter of missingno, causing the index overflow, causing the item count to be in an invalid state that could be manipulated to expose all of RAM and allow arbitrary code execution.
As an amusing example, this is why, on the website Tumblr, for a long time your "follower count" could be larger than your actual list of followers, sometimes by a lot. The follower count was updated when someone followed or unfollowed you. But if you blocked someone's account, they would be removed from your followers list without the follower count being updated. I don't think it ever caused problems other than the number being wrong.
Thanks, now I finally understand the Mario 64 cloning glitch! You're right, the name really is misleading, considering it doesn't actually make objects duplicate like in the Pokemon glitch.
I have an idea for a video, I think that looking into super mario 64's "parallel universes" might be a cool video! It's probably really complicated but, it would be neat to finally know why that happens!
well, the redundant list is also used for your party pokemon: in memory, after 7 bytes are reserved for player's name (6+1 terminator), the next byte @ d05f is the amount of pokemon you have in your party (0-6), then d060-d066 is which pokemon you have (species id)... terminator FF byte follows after the end of the list (or at d067 if 6 pokemon) and then starts the data for each pokemon, which always starts with the species id. so, the species id of a pokemon is literally stored twice in memory in a system that has 8k of wram (gbc had 12k, cause you could bankswitch wramx (d000-dfff) in one of two banks, but first gen didn't use it, only crystal version did).
There isn't enough youtube channels ( as far as I know ) about game programming. I studied it at university over 10 years ago and I had to basically guess how most games worked. The course didn't really try to teach us any of that stuff... it was just a standard computer science course with digital media creation ( art ) modules thrown in. As a result I am really disillusioned with university, especially where its related to programming or video games.
These conditions should never happen in the first place right?...
Said every programmer at some point ever.
Such is the life of a coder. Tho to be fair, it's easy to blame such mishaps on the programmer when really it's quite hard to figure out such extreme scenarios as presented here without extended periods of testing against wild theories. Needless to say, you usually don't have that kind of time or money to cover for this, whether you work on a game or otherwise professional software.
@@KyoukiJuuno Yeah I pretty much just blame the user's data at this point.
Cheers bro, I'll drink to that
This is why I’ve made it a habit to always check for all stupid, implausible edge cases, but hide them behind a constexpr if so they can be disabled if performance is a concern. Modern C++ makes balancing performance and safety fun and easy
"these conditions should never happen in the first place, right?"
Also said some nuclear reactor designers, aerospace engineers,...
Any complicated system have loopholes where glitches can occurs. And the more the system is complex, the smaller is the possibility to get it 100% bulletproof from the get go.
One obscure correction: At 21:58, you state that the reason Mario immediately drops the cork box while the bobomb thinks it's held is because the two objects are colliding, but the actual reason is unrelated and IMO somewhat interesting. When a bobomb is held normally, it has a fuse timer that counts up every frame. Once this timer is greater than a critical value, the bobomb sets a bit every frame telling Mario to drop whatever he's holding (expected to be the bobomb) so that it can explode. This forced drop only requires the bobomb to think it is being held, and so when you go to grab the cork box, the bobomb still tells Mario to drop it. This can be circumvented by overflowing the bobomb's fuse timer, at which point you can grab the cork box or any other object normally, and the bobomb will still be invisibly suspended in the air before Mario. Unfortunately, it takes over two *years* for this overflow to occur, so nobody's actually done it on a real console.
Everything else in the SM64 section seemed pretty accurate to me though, and is a great explanation of what's going on!
I love the idea of letting your N64 run for two years in order to perform this glitch on authentic hardware in real time.
@@ZimmervisionCZ Someone needs to attempt this.
Your everyday UA-cam comment from SM64 *LEGEND,* ds273.
Somebody called pannenkoek2012 to do this!
@Trimint Pikachu I also had him in mind. I mean, he’s our go-to -guy, whenever there’s any glitches/analyzing of SM64 involved. 😎
tbh I love that you put human-made captions on your videos, it's super nice for me as a hearing-impaired person, especially in the wake of UA-cam removing community contributed captions
I'm glad you found them useful! I've always made sure to have accurate captions available for this reason.
Wait, they did what?! Why?? What's the point??? Holy shit, UA-cam has totally lost it...
@Sushiirull Allegedly. Nobody really had this problem from what i’ve heard
@@RGMechEx I know this has been most likely asked before probably, but it would be really awesome if you could make a series about the Nintendo64, similar to the SNES series as well.
@@ikidu1102 The nintendo 64 was famously confusing and hard to develop for, so if he did make a series it would probably be long and boring.
The simple way to have perfect redundancy against errors;
Don't have data.
The GNOME philosophy
Perfect redundancy
@Lorash I guess that depends on the language it was written in, and the compiler. Like maybe you compile a C Hellos World with some buggy compiler and the resulting machine code can operate in weird ways under given conditions?
So not even the simple hello world is saved from bug hell :)
@@manelcastillogimenez194 just pull a ben eater and do it in raw assembly
@@computer-love bug free until you add a number to the wrong register
"But that's fine, right?"
[long pause]
....... right?
Wrong! You should perform a bounds check.
But that takes more CPU cycles, more code and thus more space.
Oh, well in that case. It should be fine, it will always be valid anyway...
I got an ad during the long pause, maybe that’s what it was for?
"You should perform a bounds check."
It did bounds-checks, but not enough as it did not check against something messing with its internal memory.
The thing with bounds-checks are the classic "You don't need them if you just program well. 4Head"
If everything worked in the code for the Pokemon-itemmenu example, then you wouldn't need these theoretical max boundries of 0 and 20, because it's normally impossible to get above 20+cancel or below 0 anyways. The bigger question would be to figure out a way to only let the ID instead of the quantity be recognised as a potential terminator.
After that, it works, all other errorcases are caused by unrelated glitches messing up the redundancy-value directly and in that case, we should just fix that instead of wasting time and romspace by placing unnessesary failsaves.
My mind just pictures John Krasinsky mugging to camera during the pause
The list terminator literally *being* the CANCEL button in Pokémon is brilliant! They didn't have to include specific code to append that option.
It's pretty amazing yes, I wish more programs nowadays were like that.
And now I use it in all my profile, but I can't imagine implementing it so long ago.
The first Gen Pokémon games were a marvel -- they were technologically very clever to pull off everything they did, but that cleverness left us with a lot of... These situations.
@@n3rdv10l3nc3 Yeah, the cleverness unfortunately involved cutting tons of corners and making lots of assumptions -- but it's okay because it makes for some very entertaining glitches :]
@@n3rdv10l3nc3 Yeah, all the cleverness came at the cost of cutting tons of corners and making lots of assumptions -- it does make for some very entertaining glitches though
the sm64 part was really fascinating.
i've been watching pannenkoek2012's and his uncommentated channel videos explaining unintended mechanics and glitches in that game for quite a long time. while he did explained some things in great detail, the "cloning" mechanic and the whole "hands-free" ordeal weren't explored with this much detail. this video really clears things up. i went from "i.. kinda understand...?" to "oh, i see!".
good illustration, clear analogy, and great presentation. all around.
At least the aspect of holding things and cloning is explained somewhat better in his newer video "How Holding Objects Really Works" (compared to "The Science of Cloning" on his main channel, which is older and more convoluted), which uses the terms "ticket" resp. "slot" to talk about a pointer to the held object.
ua-cam.com/video/BZyUGT9YPhg/v-deo.html
Petition to rename "cloning", "superposition"? It's a more accurate name since the object is not being duplicated, just rendered twice.
@najwan Yep, it would also be cool to see a full-on co-op video on SM64 by pannenkoek2012 and Retro Game Mechanics Explained. They *DID* interact in the comment section of RGME’s video on ”Credits Warp” on SMW, on his other channel, and Pannenkoek even helped in making this video, so they’re *DEFINITELY* at least somewhat familiar with each other.
@Serrara Willow Mayfield - aka Fluttershy Makes sense. Another alternative would be ”Wrong Grab”, as in DK64. Also, bad_boot has also called ”cloning” ”a bit of a misnomer”.
@@serraramayfield9230 Ah yes, trying to replace Cloning with Superpositions. That complies with the No Cloning Theorem!
I appreciate you using the fonts / color schemes for the games you're talking about, as if the game itself was displaying this information to us.
i know i love it its so cool
Ah yes, gen 1 Pokemon and SM64, what a great episode this is...
*siiiiiip * Yep, 1996 sure was a great year for games
@@ohnoitschris A pretty big year for Nintendo in terms of software thanks to those two games alone.
How dis has so many likes???
@@donalynlavarias7385 It’s a reference to how buggy both games can be, and people thought that was funny. Not sure why you felt the need to ask honestly.
A great year for games with exploits speed runners would use years down the line. Data Redundancy errors is a frequent source for the glitches they use
My main question is why do objects that aren't meant to be thrown stop working when they are thrown.
Pannen describes these phenomenon as the "one touch rule" and that clones "can't send or receive signals".
Like those are nice colloquial explanations, but I was hoping for a more technical explanation.
Edit: So because I've yet to hear a sufficient explanation, I just looked at the disassembly source code and figured out myself.
In Mario 64, every object contains a behavior. This behavior is merely a pointer to an array of small scripts. As the game runs, each object runs through its list of scripts, most of which just call that objects loop function to update it's state.
How throwing objects work is by the method `obj_set_held_state` which takes an object and a BehaviorScript. The behavior scripts used (bhvCarrySomething3-5) in question actually do nothing: they begin, then immediately end.
What `obj_set_held_state` does is check if the object is holdable, then if it is, it checks if the passed BehaviorScript is any of those three. If it is, it sets that object's `HeldState` to either held, thrown, or dropped. This is done without every running the script
However, if the object is not holdable, it will overwrite that objects behaviour variable with the passed behaviour, which remember from above, does nothing.
For example, with the coin, this results is the coin not being able to perform its update loop, as its behaviour is now the do nothing behaviour.
Now despite this, why can we still "collect" the coin, and why doesn't it disappear. That is because collision is handled completely separately from an objects behaviour. When an object is initialize, it registers a hit-box along with an interaction type to it. A completely separate method is used to handle interactions. When Mario interacts with a coin, that coin's interacted flag is set to INTERACTED and it is here where Mario's coin counter is incremented, and 100 coin start spawns.
Normally during the coins update loop, it checks if it is interacted, and if so, spawns sparkles and unloaded. However because the coin's behaviour was overwritten, the loop never runs, hence never de-spawning.
Now why are you only allowed to touch it once, that's because the code to handle interactions completely ignore interactions between Mario and an object whose interacted flag is set to INTERACTED, and similarly to despawning, it is most object's update loop which un-sets this flag.
My first guess would be that these objects don't have a behaviour associated when they're being thrown (since they're not supposed to) and they are set to a state where no action is defined (like moving, updating sprite, despawning). So they exist but have no way of doing anything.
Unfortunately, I find it hard to find the code that'd be relevent in the decompilation and am not even sure I could even read what it would do. So these are just assumptions
@@sacrijuts7265 i thought that too, but I feel like that should crash the game if it didn't have some for of error handling. And if they did have error handling, their decision on how to handle it seems weird to me, which makes me think that there isn't any error handling.
That confusion is why I want the amswer
They probably don't have code to be thrown, as they weren't designed to be.
@@EvelynFTTE that much I already assumed.
@@zeropointer125 maybe it's related to the physics then. Maybe it says for them to be pushed in a certain way, but they aren't supposed to move so they don't.
Nitpick: the item list in the first example contains 21 items, since you start it at ID 0 and end it at ID 20.
GameDevYal Yep - at 4:04 for anyone else looking for it.
I hate you.
Ah, off-by-one errors, no matter how experienced you get as a coder you can never fully escape them.
@OskarN Are you sure you didn't see zero? Or did you see the zeroth one?
The 21st ”item” is likely the terminator, a.k.a. the ”Cancel”-button.
Congrats on the channel's 4 year anniverary :) Honestly one of my favourite channels with such well edited video's every time! (Even though I don't fully understand them all xD;; But they're still very enjoyable!)
(shows wet-dry world background) "there's an infamous-"
me: (grabs popcorn)
Over a decade ago I saw a glitch compilation where somebody teleported while holding the penguin, causing the penguin to be seemingly erased out of existence despite still crying away constantly and having its mother be thankful for their return (but not actually give a star for it).
Great to finally an in-depth and clear explanation as to what the heck was going on there!
"erased out of existence despite still crying away constantly and having its mother be thankful for their return (but not actually give a star for it)."
15 Most Heartbreaking Anime Deaths Of All Time
it would be interesting to have a modified ROM of Pokemon Red that has issues like these fixed.
then give them to speedrunners and see if they can find more exploits...
repeat this cycle until you have a near-bug free game.
Isn't that literally what the whole concept of beta testers are nowadays?
That sounds awesome
Redundancies can't just be "fixed". But you can certainly add in sanity checks that intentionally error/crash when something that shouldn't happen does happen. If the check is only in the debug build, it's usually called an "assertion", from the pseudo function assert in C commonly used to implement it.
@@henke37 but why can't it be fixed?
if you for example change the "remove item" function to check the Item count before actually starting the fuction, and only check the item ID for 255, and stuff like that. why isn't that "fixing" them, when it does fix the issue?
it's just error checking to make sure it never gets into an invalid state.
Fixing this bug wouldn't do much. For it to even happen, the game already has to be messed up. It would be better to just fix the shore tiles, which they actually did in later revisions.
This exact issue appears in the Japanese version of the GameCube game "Megaman X Command Mission."
The Force Metal UI screen (the game's non-weapon equipment) is one box larger than what the list is programmed to be. The final box is "shared" with the Main Weapon list which is stored directly after it. Placing an item in the final slot will allow you to "take out" said item as a weapon on another screen.
Most famously, Force Metal #25 "Thunder Guard" becomes Weapon #25 "X-Buster Mk 3", which is X's best weapon.
your production quality was already stellar and this video and the previous one are again on an entirely different level.
you really deserve 10 times the number of subs you currently do, easily.
For a spectacular example of things like this going wrong in a modern game, look up a Hollow Knight Any% All Glitches speedrun, which has example after example of game state being desynchronized intentionally to benefit the speedrun, breaking the game completely and allowing them to beat it in less than 15 minutes with no real bossfight along the way. (They only "fight" the Hollow Knight, and do so by duping a room from Crystal Peaks into his arena, causing him to commit suicide on the crystals.)
It shows you just how good Team Cherry is at debugging and patching...and how bad they were in the past about sanity checking. I'm confident Silksong will be WAY more concrete in its codebase.
@@serraramayfield9230 hopefully they can figure out how to account for the proton decay in whatever hardware it'll be running on
Man, why are most UA-camrs better at teaching that my actual technology teachers?
I imagine passion has something to do with it. It’s easier to be super passionate about a single lecture that you spend months preparing, but if you have to do it again and again every six months, it gets to be more effort than it’s ultimately worth to do that much work.
A lot of my professors were also pretty new, so maybe that’s also a factor?
You can relate more to Pokémon and Super Mario 64 than to what your teachers teach. So there's an emotional connection that makes learning easier.
@Lorash As an engineer who was a professor, it's more likely that they didn't want to settle for the money a professorship has to offer. As a senior Instructor at a state university I was making 35k with benefits, as a principal architect at my current tech firm i make 125k with annual bonus and benefits.
The difference is drastic sometimes, you need to be okay with making a lot less money.
At school, number of potential teachers = N. On youtube, number of potential teachers = N*1000000. Larger body to choose from = greater ability to find outliers.
@Lorash There's an old saying that goes "Those who can, do. Those who can't, teach."
honestly, best channel on youtube...
I wish old games were taught at university, they are so interesting
@@B3Band thats a problem with your country (presumably its the USA),
my degree in germany costs me 6 semsters *300€, for which i even get 6*150€ government assisted
@@CubemasterXD bUt ThAt'S cOmMuNiSm ThOuGh!!1!
I was actally about to say that many of the issues described in this video are still prominent today. This video is great at describing them, so I think there is some merrit to what you're suggesting.
@@General12th blame subsidies idiot
@@blocks4857 I blame reckless, unabashed greed.
(I'm around 4:46) oh, so the count variable comes first! That explains the weird glitch I did in Virtual Console with my transfer box 😅
I forget if it was Pokémon Gold or Crystal, but basically I discovered that if you change boxes to any box other than box 1, and turn the power off when it says not to, the Pokétransporter app will read Pokémon data from this box instead of box 1 (which it's hard coded to do), and will subtract the number of Pokémon in this new box from box 1. My box 1 is currently at a negative number I think and it's eating into the Pokémon Center Vs data table
funny enough, I encountered a data redundancy error similar to the inventory one just a day or so before watching this. A Minecraft mod known as Not Enough Items (or NEI for short) provides server admins and others with access to the "/give " command a GUI for doing so, but instead of just dropping the item on you like that /give command does, it inserts it directly into your inventory. Inventory can have at least 36 slots, so finding an empty slot wouldn't take too long, but to save time anyway, the mod keeps track of the last slot it inserted an item into, incrementing it each time it inserts.
However, this redundant pointer is not bounds checked to make sure it doesn't go past the end of the "main inventory", resulting in storing junk into your armor slots and possibly eventually overflowing the data structure completely. Fortunately the pointer is reset and starts again at the first empty slot if it reaches a full slot, but I didn't figure out how to make it do that until yesterday, just hours before watching this video for the first time. I swear, the UA-cam's algorithm is psychic sometimes!
now, the question is... does Minecraft 1.7.10 (the version I was on) qualify as a retro game? I mean, it's not the version everyone is playing, but it's still playable... jk, if this channel does a video on ANYTHING involving the inner workings of a Java program, I'm going to worry about isoFreeze's sanity
Your visuals have so much polish and effort it's insane. I wouldn't understand very well if you had just explained, but the detailed visuals add a whole new layer. Favorite content on youtube hands down.
I love these videos, despite them being a bit beyond me. And I gotta say that your animation game is stellar, very slick production without being distracting. Very well done.
only listed 7 consoles and one of them was the TurboGrafx-16. You sir have earned my respect
He lost my respect when he removed the Virtual Boy. Yeah, totally not a coincidence.
I had none, so no big deal, lol.
This is also what Stale Reference Manipulation in Ocarina of Time is. It is very similar to how grabbing an object in SM64 functions, except the actual object is being moved by updating its position relative to Link's position.
I was going to ask this question. Thanks for the explanation!
Yeah. They're so similar that Pannenkoek's cloning vid actually *really* helped me understand SRM.
Have you ever thought about looking into 8/16 bit era passwords that were used for saving game state? Such as Super Hang-On, Metroid, MegaMan, Castlevania etc?
There's some interesting systems that programmers came up with for generating/validating (or in some cases not) the passwords.
TheZZAZZGlitch did a similar video -- he didn't really explain exactly what was happening, but he did show off some silly passwords he managed to generate for a number of old games
Thanks for this amazing video and thanks again for letting me use your Pokémon background! You really understand the art of teaching CS with fun, games and glitches.
Thanks for this. Pannen's explanations of things are great, but for me personally, having it explained via code terminology allows me to finally actually get it.
i love your use of movement and color to connect what you're saying to the illustration.
it makes it so easy to follow along.
You spooked me with that not-black background. I thought I had screen burn.
I thought that video compression got wonky.
Hi! I just wanted to say I really enjoy your content and all the effort you put into understanding and presenting all of this technical information. The Missingno glitch videos are some of my favorites, which is why I wanted to leave a suggestion about another glitch I think would be nice to look into: the Sword Brothers Chapel Exit glitch, for Castlevania Symphony of the Night. It's supposedly one of the first glitches discovered in the game, in which you simply perform a a spell in the Royal Chapel area, and that's enough for the door's transition trigger to stop working for some reason, allowing you to fly out of bounds and gain some extra map completion. I couldn't find any technical explanation as to why this happens, though, so I thought I could mention it here :)
Man, as a music teacher, I'm in awe of your teaching skills. I don't know much about programming at all, and I feel like I followed you the whole time.
i still admire how well you can explaing pretty complex things by neatly demonstrating it in the actual game right next to diagrams and charts that are animated in sync! i love that!
Awesome explanations and great animations! You just taught me how the 0:00 speedrun for Pokemon is achieved
Great video and explanation as usual! Although in my mind the term "Data Redundancy Error" has a very different meaning in other CS contexts (such as in relation to parity checking, Reed-Solomon coding, etc.), so was a bit perplexed at first. lol
sync error would sound like something related to multithreaded execution and race conditions I think, "state inconsistency", maybe?
As a developer..wow..your videos are just insane.. so detailed, well animated and explained in a clear way that anyone should be able to understand, at least on some level. Instant sub.
10:00 when showing a list of bits, when the bit for the item gets changed, the item (still blurry in the bg) changes as well. Nice touch.
While watching the Mario 64 portion of the V
video the file selection music started playing in my head.. thanks for the great explanation!
This would be such a good lesson in a college class. "Here's how you make a linked list. Now here's how it can go _wrong!_"
Honestly, showing what goes wrong is a great way to demonstrate how safety is done. Like race conditions, which you might not even think of even know of when coming across one for the first time.
I love this channel. It's like...interesting answers to questions I never asked. I really miss the simplicity of old hardware. Such easy to manipulate memory. The days of switching on my NES game genie and entering gibberish codes to see what we can mess up or make interesting, almost never resulting in a simple crash, but almost always changing some weird data that causes the screen to erupt into weirdness or what have you.
Something about the way this video presented the Pokemon list actually made me figure out a potential way to fix a database issue I'd been having in an app I've been working on!
...I wasn't even having a redundancy problem. XP
I ADORE hearing how games perform certain functions, it's so interesting to hear and understand.
Watched a few videos, subscribed; this is exactly the kind of deep _technical_ insight into games that I find fascinating as a perfect supplement to all the game design essay channels I am subscribed to.
Whenever someone starts talking about invalid states in Pokemon, I know that it's going to come back to the mother of all glitches, Missingno.. The moment you started talking about the item list and the terminator being 255, I knew where you leading; plus the "not "less than" but "not equal to" was a pretty solid hint that it's going to be a value outside the expected table - which happens with the item part of the Missingno. experience.
2:10 this actually brings up a big debate in CompSci over whether it's better to have a list (such as a string, which is just a list of characters) keep track of its length or whether the end should just be indicated with a special byte (usually 0x0, aka NULL). Both have their own advantages and drawbacks, and there really is no "right" answer. A null-terminated list could theoretically be as long as it wanted, while a length-specified list would either be limited by the number of bytes used to store the length, or would have to figure out a way to have a variable-length list length indicator.
Of course this isn't terribly relevant in modern programming or game development - since most coding today takes place in high-level languages that handle stuff like that for you, most programmers don't have to care whether or not a list's length property is calculated on-the-fly or read out of memory.
these videos are so well-explained! i can imagine exactly what the code would look like, and i think that even people who don't understand code would be able to get a sense of what's going on.
Ah, the dangers of doubly linked lists.
7:35 "so it's fine right?"
The video: Cuts to ad
Lol yeah that happened to me too 🤣
You just adequately explained those two main weird things you see glitch showcasers and speedrunners do all the time in these games, thanks
I don't know what he's talking about in detail, but i did get an abstract understanding of how the game handles these things.
This is such a good video! It's very useful and did a much better job of explaining cloning than any other video I've ever seen on the topic!!!
I've been looking for a video like this since forever now, thank you so much!
I love this guy. I like how the autotranslate captions let me watch in french Aswell
When a video is so good you don't even realize 20+ minutes have passed. Seriously, I was here expecting the next example and boom the video ended. It's been a while since this last happened to me.
The amount of coordination between the audio and visual components of these videos must be insane.
I've just noticed the animation on the Quantity of Item #5 while it was blurred during the explanation of MissingNo setting the MSB of it.
13:04 suddenly, abstract Pikachu!
This still blows my mind. I never knew you were a UA-camr. I thought you were just a genius SMW speedrunner LMAO
This is a fabulous channel. I loved to find out why that happens. Cheers
My favourite dynamic list/array header in 64bit mode overloads the array pointer with an item count field.. I'm also developing a 'clean object, dirty pointer' object pointer system too... I've gone with a 44 bit address field and 20 bit count for my main list type. Million items is enough for most lists. Million classes is a lot too.. Only takes a shift or an and mask to extract a field and it's FASTER than storing the size field at the start of the array.. Class system is also no slower than putting a class pointer at the start of an object (with virtual methods) as it uses a constant array partitioned by 2 power n bytes so just an extra shift in total.. Bit wasteful as each class with a virtual method table takes up as much space as the largest VMT but saves 64bits per object so easily makes up for it.. Access is as fast as a standard C++ object. All dirty pointer ops are inlined in the base class so no procedure calls added when accessing the overloaded pointer fields...
I really like how you use the game's font for each example. Very nice.
Really interesting! Makes it a lot more clear exactly how/why these glitches work and it fits in well with what I've been learning in Computer Science
Really nice explanation of the List Underflow glitch! Thanks!
I did not know what to call this in the past. I had named them caching errors because I was failing to properly maintain cached values.
I love how 1920x1080 in Full Screen makes the each game simulation's pixel not softened
Omg!! This is crazy, thank you for showing me this! 😄 I've been wondering this for so long. Thanks for answering a request I've had good sir.
The original Super Mario Bros. for the NES also had some redundant data in its source code, and I hope you might look into that on a future episode.
2:00 Yarandev be like "I'm just gonna ignore that."
In the example at the beginning you sold the Virtual Boy. What a nice touch!
"But that's okay because these conditions should never happen in the first place, right?"
Unfortunately, MISSINGNO. wants to battle...
The game Human Resource Machine is really great at making you think like an old console programmer - your resources are VERY limited and you have to do some crazy-risky stuff to meet their performance targets. It's a great puzzle-programming game that has made me appreciate those who program in assembly.
19:44 - Cloning can be abused in ways that are pretty broken as the HOLP is also screwed - i.e. it doesn't update, so when it is thrown, it will go to the HOLP which is no where near Mario.
10:41 If your footnote video teaches people that underflow is specifically a floating point phenomenon, I will love you even more than I already do.
Unfortunately not, but maybe I will clarify the distinction in a later video ;)
2:30 For a linked list, maybe. For an array-based list (that isn't null-terminated), there is no simple way to tell the difference between a list item and random nonsense, so that external count is actually the only reliable way to size the list, so I'd say that calling it "redundant" is a bit disingenuous.
its "redundant" because there is a list terminator; similar to how storing the length of a c-string is technically redundant because it is terminated by a null byte
Love it, instructional and ealy to follow! Well done! You did it again!
This was a super interesting & thoughtfully presented video. Thank you!
This channel makes the most entertaining videos I don't understand
11:08 Where did you get that info about the underflowed slots?
The information about what each memory address was responsible for? I got that info from Glitch City Laboratories, but I think that website shut down so you can only find an archive of it now.
The last part was the most interesting part as it was so much more clearer what you actually mean by this,
1:48 It's Count-Along Time with Retro Game Mechanics! Let's count this list together, kids!
It's fascinating that the vagueness of a land border graphic causing a developer to forget to add encounter stats to a strip of land made it possible to manipulate an encounter of missingno, causing the index overflow, causing the item count to be in an invalid state that could be manipulated to expose all of RAM and allow arbitrary code execution.
10:46 Why are only 128 items accessible? Shouldn't it be 255?
“So, what message speed do you play at?”
“Master Ball”
As an amusing example, this is why, on the website Tumblr, for a long time your "follower count" could be larger than your actual list of followers, sometimes by a lot. The follower count was updated when someone followed or unfollowed you. But if you blocked someone's account, they would be removed from your followers list without the follower count being updated. I don't think it ever caused problems other than the number being wrong.
This video was fascinating, great job.
Since SM64 is in this video for analysis of redundant data, it’s safe to say: “An A press is an A press. You can’t say it’s only a half.”
UA-camr: *is going to talk about SM64
The pancake: hello
*The Dutch pancake
Thanks, now I finally understand the Mario 64 cloning glitch! You're right, the name really is misleading, considering it doesn't actually make objects duplicate like in the Pokemon glitch.
I have an idea for a video, I think that looking into super mario 64's "parallel universes" might be a cool video! It's probably really complicated but, it would be neat to finally know why that happens!
Just what I needed at 2am!
What is that font you used around 3:40 called? It looks like the one in Petscop.
Damn you can really feel time slow down in this video it's just so, informative
okay, now how do i intentionally cause this
This feels really useful as a modern programmer as well. Thanks!
Perfect length until I have to get back to class lol
excellent cohesive video as always!! love ur channel
So this is why Missingno interacts with the 6th item slot.
I had always wondered why it was specifically and consistently that item.
well, the redundant list is also used for your party pokemon: in memory, after 7 bytes are reserved for player's name (6+1 terminator), the next byte @ d05f is the amount of pokemon you have in your party (0-6), then d060-d066 is which pokemon you have (species id)... terminator FF byte follows after the end of the list (or at d067 if 6 pokemon) and then starts the data for each pokemon, which always starts with the species id.
so, the species id of a pokemon is literally stored twice in memory in a system that has 8k of wram (gbc had 12k, cause you could bankswitch wramx (d000-dfff) in one of two banks, but first gen didn't use it, only crystal version did).
There isn't enough youtube channels ( as far as I know ) about game programming. I studied it at university over 10 years ago and I had to basically guess how most games worked. The course didn't really try to teach us any of that stuff... it was just a standard computer science course with digital media creation ( art ) modules thrown in. As a result I am really disillusioned with university, especially where its related to programming or video games.