Asset Packs

Поділитися
Вставка
  • Опубліковано 28 бер 2024
  • To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/TheCherno . You’ll also get 20% off an annual premium subscription.
    Hazel ► get.hazelengine.com
    Patreon ► / thecherno
    Instagram ► / thecherno
    Twitter ► / thecherno
    Discord ► / discord
    Hazel ► hazelengine.com
    🕹️ Play Dichotomy for FREE (made in Hazel!) ► studiocherno.itch.io/dichotomy
    🌏 Need web hosting? ► hostinger.com/cherno
    💰 Links to stuff I use:
    ⌨ Keyboard ► geni.us/T2J7
    🐭 Mouse ► geni.us/BuY7
    💻 Monitors ► geni.us/wZFSwSK
    This video is sponsored by Brilliant.

КОМЕНТАРІ • 81

  • @TheCherno
    @TheCherno  2 місяці тому +20

    FINALLY getting around to some of the topics you guys have asked for! What do you want to see next?
    In the meantime, you can try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/TheCherno . You’ll also get 20% off an annual premium subscription.

  • @thebutterappletutorials6553
    @thebutterappletutorials6553 2 місяці тому +20

    Would love to see a video on serializing the assets! Also really hoping we'll get to see how an Asset Packer is implemented in the Game Engine Series

  • @nephxio
    @nephxio 2 місяці тому +23

    I'd love to see a video on Cooking assets. It's always been a mysterious process to me

    • @Jorven
      @Jorven 2 місяці тому +1

      Yes, I haven’t got a clue as to how that works.

    • @neil_m899
      @neil_m899 2 місяці тому +4

      In my game engine, I have an AssetProcessor that takes in source assets and generates product assets (.asset file extension) for each individual asset. This way, I can keep each product asset as separate file. For final packaging, I can just create a zip file with all the product assets programmatically. This is my preferred method 🙂

  • @aj-jc4cv
    @aj-jc4cv 2 місяці тому +7

    +1 for binary files ser/deserialization video/s. Presumably FileStreamReader wraps ifstream and ostream calls. Great content.

  • @KojoBailey
    @KojoBailey 2 місяці тому +1

    Love this kind of content - extremely interesting, especially as a modder who deals with these kinds of "packs" frequently lol, but also as an aspiring software/game developer who has great interest in optimising game files like this

  • @kirammanenjoyer
    @kirammanenjoyer 2 місяці тому

    I was looking forward to this video. Thank you sooooooooo much Cherno

  • @syn2142
    @syn2142 2 місяці тому +4

    Good Video Cherno!
    A video on your approach to serialization/deserialization would be interesting.

  • @dealloc
    @dealloc 2 місяці тому +2

    I found that using generic packed file format (ZIP/PAK) makes things easier and scalable; and you get all the compression for free. Thinking about security through obscurity does make it harder to manage for very little benefit (if someone wants to, they _will_ figure it out anyway).

    • @hopperpl
      @hopperpl 2 місяці тому

      zip is not used anymore, because the format can only compress single files, but not compress over many files. modern game engines use a similar format to tar, omitting all irrelevant information like time stamps or names, and then compress the tar file chunk-wise. the format is not tar, but it is similar to tar. And no, you dont need to decompress the whole file from the start to find your asset, you calculate the needed chunk offset and decompress that chunk; usually chunks are 256 KiB in size and span over many files

    • @dealloc
      @dealloc 2 місяці тому +1

      @@hopperpl Indeed. I also meant in general archive formats with some examples. "packed" is maybe not the right use of the word here in context. And yes, random access decompression is definitely more ideal when you deal with larger archives.

  • @iAmTaki
    @iAmTaki 2 місяці тому

    Great video! Thanks!

  • @sadunozer2241
    @sadunozer2241 Місяць тому +1

    This is a lot like the Unreals stuff. I’m loving where Hazel is going

  • @dan-rk4
    @dan-rk4 2 місяці тому +2

    Thanks, very interesting. I'd like to hear more about the binary serialisation/deserialisation. I wrote a library for writing PSD files and want to know if I did it in a sane way.

  • @GreenJalapenjo
    @GreenJalapenjo 28 днів тому

    I wish you went into detail on *why* you have an asset pack in a single file rather than just having a folder with the optimized assets as individual files. Is it mainly performance? Is it about avoiding lots of small files? About having an "atomic" representation of all the assets, identified by build version?

  • @MaxRovensky
    @MaxRovensky 2 місяці тому +4

    I'm making a photo editing app and I had a very similar idea for presets recently
    Since I need to pack more that JSON-serializable parameter arrays for what I'm doing I've been on the fence between sth like LZ4 and this approach
    And since I'm already borrowing a lot of principles from game dev for this project - might as well do this too, now that someone broke down the approach with a nice flowchart 😌

    • @TurtleKwitty
      @TurtleKwitty 2 місяці тому

      You'd be well served by sqlite so you don't need to make your own format

    • @MaxRovensky
      @MaxRovensky 2 місяці тому

      @@TurtleKwitty I do, there's multiple concerns beyond just serializing the data

  • @Bit_Maximum
    @Bit_Maximum 2 місяці тому +2

    Thanks!

  • @henrijohansson2482
    @henrijohansson2482 2 місяці тому +11

    Nice the autogenerated subtitles say that your name is CH, not too far from the truth.

  • @Zvend
    @Zvend 2 місяці тому +1

    I really miss the outros you used to have 😭 It was like your signature for me 😂

  • @dj10schannel
    @dj10schannel 2 місяці тому +1

    Awesome thx 👍

  • @hukasu
    @hukasu 2 місяці тому

    i wonder if using vector and doing binary search would have any significant speed difference from using map, as the indirections of a map might cause more cache misses, at least this is something i hear on discussions of vector vs map, but this might be dependent on the size of the map

  • @IainEmslie
    @IainEmslie 2 місяці тому

    Nice t-shirt Cherno

  • @MeetMistry-ye5yk
    @MeetMistry-ye5yk 2 місяці тому

    Great video

  • @realishak
    @realishak 2 місяці тому

    Always the best!

  • @tapthelvete
    @tapthelvete 2 місяці тому +1

    Great video, Charno!

  • @clh2533
    @clh2533 Місяць тому

    Yes to 3D model data video please!

  • @ashwalker1
    @ashwalker1 2 місяці тому +2

    Hey Cherno can bring the C++ gamedev series back. like updated and with even more fun projects.

  • @KennyTutorials
    @KennyTutorials 2 місяці тому +2

    Duck is back, good content is guaranteed )

  • @jumpylionnn
    @jumpylionnn 2 місяці тому

    Amazing video as always :) it was really informative. one question i have is, how do you handle integer endianness? if you build the asset pack in a little endian machine and then someone wants to download the game on a big endian machine, will it still work? I have seen in the video you had a template function to deserialize the binary data and it takes whole structs. are the templates specialized for the different structs? thanks again

  • @BlackxesWasTaken
    @BlackxesWasTaken 2 місяці тому +1

    Yes, yes, yes binary serialization would be amazing! :D
    Really interesting video

  • @saeedbarari2207
    @saeedbarari2207 2 місяці тому +5

    how do you handle the byte order?

  • @RickMcMichael
    @RickMcMichael Місяць тому

    Is there a version of Hazel available for testing purposes? Also supports XR (OpenXR - not openVR)?

  • @AgentM124
    @AgentM124 2 місяці тому

    Will this system support modding? What if you want to allow mods to replace assets or add any?

  • @jimmylaessig2593
    @jimmylaessig2593 2 місяці тому +16

    You should add an app-specific signature at the end of the asset pack to ensure that the pack is not altered.

    • @futuremapper_
      @futuremapper_ 2 місяці тому +11

      someone really hates modding :)

    • @TurtleKwitty
      @TurtleKwitty 2 місяці тому +4

      @@futuremapper_ That wouldn't actually stop modding if the engine allows the runtime to continue anyways a simple popup "The games assets have been modified. Continue?"

  • @AgentM124
    @AgentM124 2 місяці тому

    What happens if you create a patch to the asset file over idk steam and it's like 1GB large. Can you make an efficient delta so the end user doesn't have to wait an hour to patch their game every time an update comes out?

  • @ItsSunnyMonster
    @ItsSunnyMonster 2 місяці тому +1

    what happens in the case of a corrupted file that still has a valid HZAP header? would you need to check for corruption for every single part of the file before loading them?

    • @mattiapezzano1713
      @mattiapezzano1713 2 місяці тому +1

      You might as well let it crash at that point 😂 is prob some cheater changing meshes and stuff 😅

  • @has_j
    @has_j Місяць тому

    I might be asking too much here, but I feel like it would help if you actually make a video of you doing something like this on a small demo project, at least just a way to be able to have exe's working without putting the actualy asset fbx files in the packaging folder.

  • @Speiger
    @Speiger 2 місяці тому +2

    A question comes to mind. If you have 1 big monolithic file for the game, how do you plan to make steam updates not basically force download the entire game with each update.
    Even if only 1 asset was changed.
    Ofc I don't know how steam deals with synchronization, they might have a system in place. But it would be interesting to know if that has been considered.

    • @Rhadamus
      @Rhadamus 2 місяці тому

      Steam doesn't redownload the entire file. It computes the difference between the old version and the new version and only downloads and applies that difference to the files.

    • @Speiger
      @Speiger 2 місяці тому

      @@Rhadamus ok, but he just tells we dump everything asset wise into 1 file.
      Isn't that a redownload of the entire game by that logic?

    • @IgnoreSolutions
      @IgnoreSolutions Місяць тому

      @@SpeigerA single file can still be “diff’ed”. When you do that you’re applying a patch to the file only changing the bytes that have been modified.

  • @owdoogames
    @owdoogames 2 місяці тому

    This is great and everything, but what I really want to know is who do I throw money at to get the duck shirt you are wearing?

  • @raffiepro
    @raffiepro 2 місяці тому +1

    How will you get the 8 byte integers working in 32 bit builds?

    • @anon_y_mousse
      @anon_y_mousse 2 місяці тому

      Probably the same way they already worked before. You could manipulate 64-bit integers in C and its various offshoots back in the 90's. Most of the time the compiler would generate special code inline to work with them, and then as compilers got better they would use SSE registers and then we got proper 64-bit a pretty long time back.

    • @raffiepro
      @raffiepro 2 місяці тому

      @@anon_y_mousse Okay thanks 🌚

    • @foobar4114
      @foobar4114 Місяць тому

      Hazel doesn't support 32 bit builds, iirc.

  • @pepguitar001
    @pepguitar001 2 місяці тому

    Yes, more on serialization pls
    Bring raincloud back :P

  • @lanchanoinguyen2914
    @lanchanoinguyen2914 Місяць тому

    build your own format is okay but i would like to put stuffs into a encrypted zip file instead of messing around.Yeah loading text based formats can be slow but not that much i doubt,look at how fast blender load files.

  • @showmeyourcritz321
    @showmeyourcritz321 Місяць тому

    What IDE is that? VS 2022 right?
    What is that written "not equal" symbol in there? @23:03
    How does this work? Is != translated into this?

  • @mrmati5
    @mrmati5 2 місяці тому

    Currently the App Binary decompiles perfectly and has all the names just like Unity games without il2cpp. Are you planning some kind of obfuscation?

  • @nahuelfernandez2352
    @nahuelfernandez2352 2 місяці тому

    A video about binary serialization would be awesome

  • @michelewang6983
    @michelewang6983 2 місяці тому

    Great ✨✨ 🔥☺

  • @nahuelfernandez2352
    @nahuelfernandez2352 Місяць тому

    A video about binary serialization woud be awesome.

  • @tanja_the_fox
    @tanja_the_fox 2 місяці тому

    7:00 if anyone is more of a foss guy, I've found a liking to ImHex

  • @anon_y_mousse
    @anon_y_mousse 2 місяці тому +4

    This is interesting, but why didn't you just use a .zip archive? There are libraries for dealing with them in C and I'm sure there are some C++ specific ones as well, and C# has it in the standard library. Also, you shouldn't store libraries inside the asset packs. It would be far faster to load the code if it was stored external to any asset packs. My suggestion is that you should have a file in the root directory of a .zip archive for all the metadata you wish to store and then just have the assets as if they were in a directory whose root is the location of the game, but keep all of the code separate from the asset pack. Do you have a Linux build working yet? Will you be accepting community code contributions?

  • @cawkcheck
    @cawkcheck 2 місяці тому +1

    daddy

  • @gsestream
    @gsestream 2 місяці тому +1

    any security you might think is smart will be broken, by the local access security policy. if you even once get the asset as plain data, its unsecure. people will hack it. any such efforts are only making more burdens to yourself. multiple duplicate work for no actual reason. would be better to just have all plain readable files. also for all debugging reasons during editor operations. yep same files. even encrypting would not benefit you at all. you are going decrypt and load the files on the local computer. and online DRM will shuggers. speed-up reasons are not valid either, you can do that smart using the original master files. or generic export files. you just gave two excuse reasons to make things harder for programmers. engineers keep it simple. why would you promote unnecessary work. bad design. same goes for gov and taxes. no use. only extra hand waving. dont pay taxes to your king. independence day. managers not needed or required. at all. three times advertising something not required. 3rd time the charm. hmh. wasted work. no wonder time spent doing stuff that would not be required to be done. ever. fluff. money makes you doing nonsense. cut it straight up. why you need a trainer to keep you in focus. omg. bloat ware, and shovel middle ware. integrated not sparse. I hope you are not talking your own, or copy from people. because you did not heed to God's advice.

    • @gsestream
      @gsestream 2 місяці тому

      even if you stream remotely, you know those casette tape copies, or just screen capture. or NVIDIA RTX remix on the fly. etc. also making law again only shows you did not learn your lesson. emphasis on yours. oh yeah, money to serve.

    • @kirillvorotov1169
      @kirillvorotov1169 2 місяці тому +2

      it is less about security and encryption and more about performance. You definitely don't want to load asset files one by one at runtime, it'll probably kill the performance of the game. Another thing, as being said, is that common general purpoise formats usually contain mode data than you need. For example, typical fbx file will contain scenes, dummies, lights, meshes and more. Instead of converting this data into a format that you actually need at runtime you can preprocess you data.
      Actually, scratch that. Your last sentences clearly show you a bloody lunatic.

    • @gsestream
      @gsestream 2 місяці тому

      at least I will not be implementing any of the nonsensical stuff for you. if you teach it you do it yourself. will not take any part in this. what, you cant code a fast text parser. even that binary program you use can do it. nonsense, anything you judge, comes on your head. no thanks.@@kirillvorotov1169

    • @gsestream
      @gsestream 2 місяці тому

      tanking performance? nope its just fine. its all the other issues, including extra non-useful work that tanks all money projects.@@kirillvorotov1169

    • @gsestream
      @gsestream 2 місяці тому

      what do you think about xml parsers or xhtml on the web. fast enough.@@kirillvorotov1169

  • @ahmadshbat8363
    @ahmadshbat8363 2 місяці тому +3

    dude, please don't make your vid personal-stories based, make them generic same as C++ series
    there might be a lot of ppl come to your channel for the first time and they find you talking about some stories of you engine in the middle
    make more of C++ tricks, vids, .... etc

  • @drunk3np1nky40
    @drunk3np1nky40 2 місяці тому

    But what about assets that are instantiated at runtime only? Do they belong to any scene?

  • @IvanStamenkovicSeemsIndie
    @IvanStamenkovicSeemsIndie 2 місяці тому +7

    No Cherno, I do want my file to be the raw modal file, that makes you game easily moddable. c'mon...

    • @IvanStamenkovicSeemsIndie
      @IvanStamenkovicSeemsIndie 2 місяці тому +1

      @tony01riggioni let's say you want to re-texture the existing game, if a file is a custom binary format good luck editing that in Photoshop or similar. Where if it was jpeg or similar you would be able to edit it quickly

    • @WhoLeb7
      @WhoLeb7 28 днів тому

      I guess then you would need to create a tool for that, which allows you to modify the said maps without allowing you to retrieve asset data, at least not in transfer states(gltf or obj or whatever)