Remastering Classic Games with Texture Upscaling | AI and Games #61
Вставка
- Опубліковано 6 лип 2024
- Support AI and Games on Patreon:
/ ai_and_games
--
The recent release of Mass Effect Legendary Edition is one of the most high profile cases of AI being used to upscale game art for higher resolutions. In this episode, we take a look at the tech that makes it a reality and the modding communities that have pioneered this new approach to remastering games.
[00:00] Intro
[01:07] Graphics in Games
[06:30] This Sounds Like DLSS?
[08:08] How Texture Upscaling Works
[12:03] Modding with AI
[15:45] AAA Super Resolution
[18:44] Closing
If you want to learn more about what's covered in this episode, check out these links:
- Learn more about super-resolution (and more) over at @TwoMinutePapers
- Check out the game upscaling subreddit: / gameupscale
- Learn more about ESRGANs and grab the codebase: esrgan.readthedocs.io
- DOOM Neural Upscale 2X: www.doomworld.com/forum/topic...
- Max Payne Remastered: www.moddb.com/games/max-payne...
- Dolphin Texture Mods: github.com/rapka/dolphin-text...
- Deus Ex New Vision 2.0: www.moddb.com/mods/new-vision...
- Morrowind Enhanced Texture Mod: www.nexusmods.com/morrowind/m...
--
AI and Games is a UA-cam series on research and applications of Artificial Intelligence in video games. It's supported through and wouldn't be possible without the wonderful people who support it via Patreon and UA-cam memberships.
/ ai_and_games
/ @aiandgames
--
You can follow AI and Games on Twitter:
/ aiandgames
Join me for live streams over on Twitch:
/ aiandgames
Join our Discord Community:
bit.ly/AIandGamesDiscord
Get yourself an AI and Games t-shirt over on Teespring!
teespring.com/stores/aiandgames
#AIUpscaling #GameDev #DeepLearning - Ігри
If you're keen to try out the mods shown in the video, be sure to check out the video description where I've left links for you to check out.
In the meantime, episode 62 on DLSS is coming your way very soon. I hope you enjoyed this episode, it's an interesting change of pace from the regular stuff we look at around here.
Just to let you know, great approach to this material. I usually get insight into these types of things on Digital Foundry, and I felt that you filled in some gaps for me!
I'd guess this was achieved by presenting it from more of a gamer's perspective...
Just one small error in your Video. You suggest that DLSS needs to be trained for a specific game, afaik this was only true for early versions of DLSS.
@@MrNubix Indeed, as of version 2.0, the deep neural network is no longer trained per game. I explain that in detail in the DLSS video.
@@AIandGames
On it Captain.
One of the things I loved about Doom 3 is that the engine would sort the level's textures according to frequency of use before applying the compression necessary to fit them in a 64 meg texture budget. So the most-used textures (like a corridoor wall texture) would get the least amount of compression and a one-shot texture (like the scanner at the start of the game) would get the most. It really made the game look terrific.
@@Earo16 I think the book was called "The Making of Doom 3" by Steven L Kent. I haven't read it but the folks who quoted from it had some interesting trivia about the game. Id needed to fit the game's textures into a 64 meg texture budget because they wanted to release the game on XBox, as well as PC. (Even though most PC users had 128/256meg graphics cards at the time). As a result of that, there was some cut content. In the "Welcome to Mars City" level, there was a missing chapel and the Player character had to find his living quarters, before the game continued. By cutting that content, they freed up just enough resources to add the infirmary (where you see the injured marine being treated).
So you could say that using an AI requires alot of... Calibrations.
This was a super cool video thanks for sharing. It's clear you put a lot of time/research into it, and writing the script in a way players can understand!
Didn't Expect a SWTOR connoisseur in these comments
there is really like 3 piece of information crumb for the length of 20 minutes.
I'been watching the channel Two Minute Papers here on UA-cam for quite some time now and it mind is blown regularly. The technology being created surpasses even recent science fiction. Up until maybe a couple hundred years ago society changing technology would come about every several generations but now it seems to happen at least once a decade. In some fashion this is also apparent in the evolution of video games over the console generations. Man what a time to be alive!
It has been so long since UA-cam recommended one of your videos, I was checking my subscriptions and found this.
Great video.
This was a bit of a left-field topic, but one that *absolutely* applies when you consider this channel is about A.I and games! Who's to say this should always refer to the A.I that governs how games operate? ;)
8:15 - "Neural networks"
What I heard: "noodle network"
Me: giggles and scrolls back to check I heard that right.
This is great content, people should be more aware of this. I would love to play MGS1 with this enhancement!
Whenever anyone talks about remasters, my mind immediately starts thinking about MGS. MGS3 subsistence would be my top choice (although a remake in the fox engine would be better) just because MGS3, but MGS1 probably needs it more.
I saw that Xbox swap out! Lmao, good stuff.
I was interested in this fir a while now & u did a great job explaining thanks!
I wonder if it will be ever possible to apply tessellation or map displacements to specific modded textures.
Fascinating stuff, thanks for the video.
What I appreciate is how well Bioware was able to create 'Rounded Rear Curves' 17 years ago. True Artistry! ^_~
you mannaged to explain dlss in a one min throw away segment far better than any of the 10 min lectures ive seen manage i actualy get it now thank you
Just found your channel as Im interested in learning how Ai remasters old video footage?
Then I thought how Awesome it would be to have a higher quality version of Kingpin, I loved that game and Total war Shogun!!!!
Interesting that it uses a comparative GAN. I thought it would use a variation on an Autoencoder trained to transform downscaled images to their pre-downscaling form.
Could this type of upscaling possibly be done on meshes? I don't know much about this topic.
Great video !
Yes, a neural network could be trained to refine and increase the detail of a mesh beyond simply subdividing it. It'd be prudent to create different training sets for Humans, Organics, Machinery, etc, and could be further helped along by reading the texture / normal maps. That said, there's a limit to what can be realistically be done. I can't imagine that Mario as he appears in Super Mario 64 could ever be automatically updated to resemble the Mario of Super Mario Odyssey, but I do think neural networks could be used to bring a 2013 character up to 2021 standards.
I think it's important to remember that the models of yesteryear aren't simply high-res models that have been scaled down to fit the limitations of a given console; but instead specifically created for those limitations. To go further back to retro Arcade games- pixel artists leveraged the bleeding of pixels on CRT screens to their advantage, and many arcade games appear to lose detail when displayed on modern monitors because the pixels don't bleed; so the colors don't mix. This is similarly true with early 3D models- They're designed to be low-poly, and more detail may not be an improvement.
That is what tesselation essentially is, and it is available in almost all modern games after around 2011.
@@anteshell True
But hightmap for tesselation is created by humans :-)
Now we can generate hightmaps by ML so tesselated objects would have more details.
@@igorthelight Not true. Tesselation is not made with a height map but with a shader in particular place in the rendering pipeline.
The shader for creating the tesselation is written by humans and that is mostly due to the fact that it needs to be configures game by game basis depending on the model complexity and so on.
However, my point was not to say anything about creating that shader but rather the process of tesselating itself. There is no human in that process. Also, there are generalized tesselation algorithms available that pretty much cuts the human interaction to zero, if not counting placing it in the right place.
A height map can be used to feed data for the algorithm, but is not required. Besides, nowdays height maps are rarely made by hand.
@@anteshell Thanks for the info!
Anybody know any particular sites to get texture packs from
Great video Thanks
Really cool stuff. Excited for the DLSS episode -- never even heard of it before but it looks incredibly promising as a way to downsize game files and texture load times
Great video!
The possibilities of deep learning texture upscaling algorithms are exciting.
have been watching 2 min papers for quite some time, but just where can I grab a GAN model to tweak myself?
Google for them - some of them are free on Github as far as I know ;-)
Great video
I can only speak for Metroid Prime here, but a big problem with the AI upscaled textures for that game is that they tend to be massive, poorly optimized, and contain a ton of artifacts. There's another project working on manually recreating the Prime series' textures at a higher resolution, which promises smaller file sizes and (imo) looks much better, even if it's currently incomplete.
Liking, commenting, and just subscribed!
This presentation was very well done. Instant Sub. I can't wait until one day we have super AI scaler dongles that give us exactly what we want in real-time for any console.
the whole video is about how this thing isnt really possible. The information is not there, you see. upscaling only really works when the game was prepared for upscaling at development.
@@Omertahun I strongly disagree. I think the future will surprise you.
Very interesting... thx
My dream is to see SNK Samurai Shodown games upscaled, and depixalated. I love SNK games.
Thanks for the vid
What about the Hitman series from Codename 47 to Blood Money? Those missions need the Remaster treatment!
so what's keeping this same sort of thing from being applied to geometry?
Nothing.
It's just someone have to do that ;-)
do you force 16x anisotropic filtering on all games in the nvidia control panel? i’m looking at morrowind in the first part of the video.
You may do that.
Or just use OpenMW - a more modern recreation of Morrowind's engine :-)
@@igorthelight yes, i meant that anisotropic is useful in many older games that didn’t have it in the options menu.
@@RobNomad True.
In most cases you may force it in driver options. But there are modern implementations or old game engines where it works without additional headaches :-)
@@igorthelight i always leave 16x activated in the drivers options, there are no drawbacks
@@RobNomad True.
Videocard must be VERY old to take penalties for using AF 16x
I'm talking about 3dfx/Voodoo era :-)
As the second oldest A I channel, your channel being the oldest, I really enjoy your content.
I'd really like to do this to some Splinter Cell games. Have zero idea on how to do it though lol. I'll check out some of the links.
Imagine if they merged this kind of thing with something like Nvidia Canvas and used it on older games like ps1 or ps2. They could take the baron landscapes and turn them into something amazing. Or train an AI with a modern remake of an old game, like the Crash N Sane remake and it's original PS1 games, so that you might be able to feed it something like the original crash and it spits out something looking more like the remake.
6:17 Is that Tom Cruise lol, honestly its funny how I think it kinda looks like him
that was funny @3:47 lol
Kind of surprised there wasn't much consideration of the pushback from artists.
Artists on older games designed the art to work for low resolutions and different screens - they knew what they were making and adjusted for it, the same way cinematographers accounted for black-and-white film. You can't get out what the art might have looked like if it was made in a different medium, because the artist would have done things differently. The set for the Adam's Family was designed with bizarre colours so they'd show up well on film - if you recreated the colours from that, it would look awful. If you recreated the colours based on what you think the intended effect was, you're just guessing.
That would be fine if mistakes were always obviously wrong, but sometimes they camouflage themselves well. Take the problems of StyleGAN - it creates fairly realistic human faces, but biases towards white features. This led to much hilarity when the PULSE Face Depixelizer, using StyleGAN, was shown to upscale a lowres Barak Obama into a poorly-lit blue-eyed, pink-lipped white dude. Pretty convincing face if you didn't know who it was supposed to be!
It's not a problem when the AI interprets a tiny texture on a soda can as baked beans if it's just used for piles of trash, but becomes concerning when, after publishing it, it becomes clear it was also used on a desk in an office or bedroom, with the worrying implication that whoever sat there was drinking beans straight from the tin.
But I'm more worried about negligent publishers throwing their old PS1/2 games at a dev team who never worked on them and telling them it's being released Q3, just get it done, the GAN knows what it's doing.
While I think part of the fun for developers is to make these textures faithful to the original, I can see what you mean. This could possibly be an issue for some, yes. I think that it can also be a welcome addition to devs who wish they had created larger textures in the first place.
this was very well said
@@florianhoste6969 Yeah, the number of custom HD texture packs for old games shows dedication can have amazing results, don't get me wrong. All remakes and remasters have challenges like these. I'm not against them by any means and AI upscaling can be amazing. It's just a little odd to not mention these angles in the video.
I'm a narrative and game designer. I love your channel mostly for A.I. in games, but I love to keep an eye on all new tech in a casual way. You said you have resources for the more scholarly, and thats me, just not in this field.
DLSS sounds like an interesting piece of technology also the Mass Effect Legendary Edition looks amazing.
I've only played the first game so far (all three games are included) but it looks fantastic. They've done a great job.
@@AIandGames Except some strange "mood changes" like different time of day or weather on a planet we visit in the games.
Those changes are mostly negative in my opinion.
I recently upscaled all the textures for The Thing 2002 using A.I. If anyone wants I can give a link to the upscaled texture pack :)
This is something I've wanted to do but haven't had the chance yet
I wish we could use vector graphics for textures, then we wouldn't need to worry about scaling changing resolution.
True.
But vector graphics needs more computational power (every frame) because every detail is not just an array of pixels - it's a mathematical equation now ;-)
@@igorthelight ah..true. You'd need much less V-Ram, but one hell of a monster CPU. A hybrid system may work though, what if the textures were stored as vectors, but baked as rasters at the required resolution.
@@MoonlightFox I heard about that somewhere...
Not sure is that worked or not - someone may try to make it :-)
What I really want to know is can I get a game let’s say, conflict Vietnam (old PS2/PC game) and somehow get it updated on unreal engine 5.
I know solidworks, auto cad, spaceclaim inside and out and very familiar with cryengine but unreal engine looks better.
So my question is are there any tutorials out there on how to take a PS2 game, get the game files and then work with the textures and 3D assets to update the game? Surely this can be done without needing to remake the game and simply updating the assets. That’s all I want to do update the textures
Cool.
The artists orginal intent vs CDs 700MB size?
I saw you try to sneak the xbox one in there for a second when you said "original xbox" you sneaky sneaker you
I just wish someone could do it for the original dishonored, man that would rule
Videos like this are a stark reminder of how many people are much smarter than me 😖
That xbox joke, GOLD.
Hello to Jason Isaacs
kermode gang worldwide
AI is magic
Funny enough I'm playing the diablo 2 remaster while playing this and I don't think publishers realize just how much money there is to be made by just making a clone of the old game (with major bugs/glitches/crashes fixed ofc) and putting on a new 21st century coat of paint. I'd buy so many older games again if they got HD remakes.
can u upscale AION CLASSIC? :D
Loved your accent
some of the ai upscaled texture mods look weird like on old games clothes look weird
Those Morrowind textures used as an example at around 3:20 don't actually look that bad, and it detracts from the point of example for me.
I found this video through a ff14 upscale mod.
@AI and Games PLS correct disciples 2 :c
Jesus Christ it’s here boys
📢Bro i No this is an older video but please Get ahold of Rock star And break down and explain this stuff to them👍💯 For their grand theft auto trilogy Definitive addition
ua-cam.com/video/Fix6u4pksrg/v-deo.html Was that Xbox picture swapping a subtle jab at the confusing naming scheme of the consoles? 😁
I'm all for upscaling textures in older 3D games. But think classic/retro sprite art games should be left alone. just my two pennies worth.
"Wouldnt usually recommend reading reddit" sir how dare you.
Might just be me, bur the first image "upgrades looked rubbish (the two guys and the doom one). Seems like a nice idea but given the examples I guess it has mixed results?
The upscaling on Doom 1 honestly still looks pretty bad. :/ Even with all the manual clean-up. It looks like it took a lot of work and I'm sorry.
I feel like retro 3D is best treated like pixel art, but I might be alone in that. If the original game's visuals were designed with anti-aliasing in mind, that's when upscaling begins to look good to me.
4:33 40Gb RAM, dam
Yeah.... heh, that was my old machine too.
JUST GIMME GTA IV UPSCALE VANILLA TEXTURES AAAAAAH
Just interesting how much of a bottleneck hardware was then, 32 MG VRAM graphics cards couldn't pull much.
Well now i understand why it should have Been tested more for gta….
Zoom and enhance.
Now compare this to that shitty ai upscaling in GTA remastered 😂
The Mass Effect remaster is probably the worst example that you could've possibly came up with to show how AI upscaling can preserve the artists' original vision.
I think textures are good.
But "mood changes" on many game levels are really... unfortunate :-)
Ah.... The Max payne remaster is a modd... Fair enough
Up scaling textures only fixes half the problem,since you're usually still working with the same low-poly model as before
Soon, models could also be upscaled by ML :-)
Animations could be improved by key frames interpolations (simple math - no ML at all, but it may help)
@@AngelsInCircles Interpolation could be non-linear if you want them to ;-)
@@AngelsInCircles There will be :-)
LOL the lack of the aspect ratio fix in Max Payne is atrocious. If you couldn't patch it, at least do the proper 4:3 and leave black bars.
You could say this video contains 'Deep Learning'! bdum tish...Sorry I'll see myself out
That was terrible, but I'll allow it.
first, and a great video.
Tha hell is this accent I've never heard anything like that before
Sorry, but you definitely need to read more into how 3d art works. Saying "1080p texture" is completely incorrect. Screen resolution has nothing to do with texture resolution. High resolution screens may allow you to better see detail, but it does not "stretch" textures or manipulate them in any form. A texture that is considered "4k" is not "3840x2160" like your monitor, it is "4096x4096" which is a power-of-two texture. They need to be power-of-two for texture streaming.
Is the accent south african?
Thumbs Down
You didnt "Upcale" Deus Ex , you just used low and high settings
You're adorable 🤣
Wheres pong in 4k? I demand it
Great video
Cool.