This is amazing. Impressive how you got it to work with the official FAT filesystem! Now the hard part is reading and writing files at the individual byte level, you'll have to know the byte-seek value (32-bit number telling you where you are in the file in bytes) in relation to the cluster size to know exactly how many clusters to access and where you are in the cluster. Writing 500 bytes could mean crossing a cluster barrier if you're close to the end of a cluster.
@@weirdboyjim For me it took quite a while to figure it out how to do it reliably (especially with edge cases like single byte writes right at the edge). I had to first psuedocode it in C and then translate to assembly language (since I wrote it for my own homebrew 8-bit CPU with 64 KB of RAM), however it is the first time I've made filesystem drivers ever, so it takes a while anyway haha. The filesystem I have made has 16-bit linked-list FAT entries (0xFFFD for bad/inaccessible cluster, 0xFFFE for end of file, 0xFFFF for free cluster), 512-byte clusters (making 32 MB of storage). It's basically just a root directory (32-byte directory entries with 8.3 filenames), FAT table and data region from scratch and is incompatible with FAT. No boot sector, no subdirectories and no long-file-names yet. The hardware disk drivers in the BIOS are capable of IDE/CF-card or high-capacity (HC) SD-card in SPI mode (later I'll support 4-bit mode). I actually followed your tutorial for SD-cards!!! I'm thinking of making a boot sector next with a linear allocated space for the OS (before the filesystem-layer at the sector level) so that it can be loaded from disk rather than being in ROM, allowing for a very simple 2 KB BIOS to then load the 16 KB OS from disk. Subdirectories would just be a root directory structure but instead of being hard-coded at the start of disk, it's in a file (or a folder, which is a file with the folder attribute set to 1) and can be dynamically loaded in for recursive tree directory structures. Long filenames would have the LFN attribute set high, and would then allow for null-terminated ASCII strings amended at the end of the file entry. This is different from how FAT does it, but I'm making mine from scratch and solving the problems as I go.
A decent dive into the FAT filesystem. Just about right for this format! And impressive that you got it to work without any remade library, but rather rolling your own, as it were.
@@weirdboyjim Coming from a games programmer background where every cycle counts, that isn't surprising. Funny thing, mostly when a project winds down or changes direction I usually lose interest, but your blend of continued hardware development and software to make it work, shifted the focus in just the right way.
That's awesome! Really educational, too - you did a great job of explaining the basics of FAT16. It really seems like a great fit for the jam-1 - can't wait to see you expand your library's capabilities and write programs that can take advantage of it!
The quality of this series is incredible, every day I'm more in love with the details, and today to my surprise you brought a top theme, file system, damn, I want more haha, now seriously, I think it's also time to think about creating a small language at a slightly higher level for your architecture, something like a small compile for your assembly, would make it a lot easier, in any case I'm looking forward to the next video for file manipulation, this video was incredible, congratulations to you .
I really enjoyed this episode, James. I've always been fascinated with how a hardware project and some code actually become an operating system and a functioning general purpose computer. This series is just starting to get stuck into that now! 👌
Thanks George! Glad you are finding it interesting. Loading and running programs from the SD card is about as far as I'll go, but it will make the system much more capable!
this is so thorough tutorial for FAT that even my monke brain can understand the basic logic, maybe make it bit easier to find with tags for it. I'd hate to miss this information when searching for it.
And also by uploading this code to the SD card via an FTP server, which will require a network adapter on some ESP8266. Then it is possible to provide terminal access. Multitasking and... :)
Nice one James! Reminds me of reading through that massive ST Disk Drives: Inside and Out book back in the day. I got distracted making boot sector demos that were around 400 bytes - fun times!
Awesome. I always think the moment you can load and save to storage is a turning point in any homebrew project. Suddenly you find yourself looking to standardize the filesystem interface. This is also usually when the idea of a C runtime occurs to me. It's the promotion of a project from a hardware one to a software one. Seriously, well done!
So awesome! You amaze me and I really appreciate you. This was an absolutely fantastic breakdown and it answered many questions for me (about how I thought it worked).
Regarding that cache sector, it resembles the functionality of the Forth block word set. You have block (load block, except for the return value, since there's just one buffer), with one more flag of status (updated aka dirty) you could add buffer (map without read), update (mark modified) and save-buffers (save modified buffers). You can probably tell why 32x16 characters per screen was used.
This is fantastic. I've wanted to learn about file systems for a while, and this is a great primer! Thank you! I have a suggestion. Perhaps you could add a RAM chip that uses SPI for doing the caching (or for whatever use). I'm looking at a 23K256-I/SN from Microchip Technology on DigiKey that might work for you.
Nice work, James! It might not be super helpful, since it's assembly for a different architecture, but the x86 assembly source code for MSDOS is available. If you want to study how they do pattern matching, etc. Might give you a few ideas on how to implement similar functionality in your own SDOS (SD-card Operating System?).
It's always interesting to around the source to see how things were done. For my project memory is a big constraint, so working out functional solutions that don't consume too much space is always a big challenge.
Lol i hope he comes out with a book on computer architecture and operating systems. Most texts are just too detailed and dense but if they explained things simply like he does here, it'd be a great foundation to build on.
This is really coming along :D Will you be able to edit your games in notepad now on a PC then load them into this? Also, I'd love to see a floppy drive implementation :D
good old dos delete, remove the first char in the file name. reminds me of the fun we used to have back in the days before windoz. A TSR linked into the keyboard interrupt was always fun to throw an odd keyboard strike in to the user typing.
I forsee a future with lots of game cartridges, erm I mean SD Cards with custom printed labels. 😂 Years after USB Autostart was deprecated, I see james bringing it back to life with SD Cards
Would it be possible for you to implement system interrupts? You would be able to make a system timer and process timed events together with processing interrupts from external devices
@@weirdboyjim I have thought on implementing it. This is how I would have implemented it: At the end of a instruction, when a interrupt has occurred, a separate ROM starts executing instructions with its own PC. It executes push functions where all registers are pushed to the stack. Afterwards it will call a address where you can implement your interrupt handler. When the RETI command is executed, the interrupt control block triggers and it will pop all registers back into its original state. You can have a special purpose register where it is possible to have all interrupt sources as flags.
Can I find details about the CPU instruction set somewhere? And do you have a simulator for it or are you always running code directly on the real hardware? Thanks!
Hi James, I've noticed a few times now that you still have the entry nop at the beginning of each assembly file. I think at some point this was out of necessity for the pipeline initialization, but didn't you fix this "bug"? Why do you still keep this?
@@weirdboyjim Yeah but some of them had to have a whole extra computer in the floppy drive for it to prepare the data for the main computer. That seems to be the sollution I have most encountered in 8 bit computers. We never used the floppy drives on my grandfathers ABC 80 while it was in his possesion so I have no idea about those. But a c64 must have had a file system that the drive could read and re arrange it so that the computer could recieve it through the tape loading routines?
OT: feckin YT is deploying 4 minutes in 3 ads, on an unlisted video. It becomes more and more annoying. James, did you ever consider to allow your Patrons to view your videos outside youtube? On-topic: nice one, as always!
Thanks! That level of adverts sounds annoying! I know patron have a facility to host videos now which might be an option. I'll need to think about that, I want to avoid "overhead creep" in video production.
@@weirdboyjim Sure. I don't wanna push you into more work. And I don't wanna make you missing income. However, the amount of ads becomes more and more ridiculous.
Finally good video on the internet about FAT filesystem and some practical examples.
Glad you found it useful!
He's so incredibly good at explaining in simple terms and complements it with great visuals.
Thanks!
This is amazing. Impressive how you got it to work with the official FAT filesystem! Now the hard part is reading and writing files at the individual byte level, you'll have to know the byte-seek value (32-bit number telling you where you are in the file in bytes) in relation to the cluster size to know exactly how many clusters to access and where you are in the cluster. Writing 500 bytes could mean crossing a cluster barrier if you're close to the end of a cluster.
That shouldn’t be too difficult to build on top of what I have, it’s all about creating the right layers to build on.
@@weirdboyjim For me it took quite a while to figure it out how to do it reliably (especially with edge cases like single byte writes right at the edge). I had to first psuedocode it in C and then translate to assembly language (since I wrote it for my own homebrew 8-bit CPU with 64 KB of RAM), however it is the first time I've made filesystem drivers ever, so it takes a while anyway haha. The filesystem I have made has 16-bit linked-list FAT entries (0xFFFD for bad/inaccessible cluster, 0xFFFE for end of file, 0xFFFF for free cluster), 512-byte clusters (making 32 MB of storage). It's basically just a root directory (32-byte directory entries with 8.3 filenames), FAT table and data region from scratch and is incompatible with FAT. No boot sector, no subdirectories and no long-file-names yet. The hardware disk drivers in the BIOS are capable of IDE/CF-card or high-capacity (HC) SD-card in SPI mode (later I'll support 4-bit mode). I actually followed your tutorial for SD-cards!!!
I'm thinking of making a boot sector next with a linear allocated space for the OS (before the filesystem-layer at the sector level) so that it can be loaded from disk rather than being in ROM, allowing for a very simple 2 KB BIOS to then load the 16 KB OS from disk. Subdirectories would just be a root directory structure but instead of being hard-coded at the start of disk, it's in a file (or a folder, which is a file with the folder attribute set to 1) and can be dynamically loaded in for recursive tree directory structures. Long filenames would have the LFN attribute set high, and would then allow for null-terminated ASCII strings amended at the end of the file entry. This is different from how FAT does it, but I'm making mine from scratch and solving the problems as I go.
I will need to watch this a few times, because that is what you do to something this awe-inspiring and well done. Thanks James, you are a star!
A decent dive into the FAT filesystem. Just about right for this format! And impressive that you got it to work without any remade library, but rather rolling your own, as it were.
Glad you liked it. I do tend to do everything myself from the ground up!
@@weirdboyjim Coming from a games programmer background where every cycle counts, that isn't surprising. Funny thing, mostly when a project winds down or changes direction I usually lose interest, but your blend of continued hardware development and software to make it work, shifted the focus in just the right way.
That's awesome! Really educational, too - you did a great job of explaining the basics of FAT16. It really seems like a great fit for the jam-1 - can't wait to see you expand your library's capabilities and write programs that can take advantage of it!
Glad you enjoyed it!
The quality of this series is incredible, every day I'm more in love with the details, and today to my surprise you brought a top theme, file system, damn, I want more haha, now seriously, I think it's also time to think about creating a small language at a slightly higher level for your architecture, something like a small compile for your assembly, would make it a lot easier, in any case I'm looking forward to the next video for file manipulation, this video was incredible, congratulations to you .
I really enjoyed this episode, James. I've always been fascinated with how a hardware project and some code actually become an operating system and a functioning general purpose computer.
This series is just starting to get stuck into that now! 👌
Thanks George! Glad you are finding it interesting. Loading and running programs from the SD card is about as far as I'll go, but it will make the system much more capable!
@@weirdboyjim hehe! I hear ya. But that's the kind of feature set that ties the entire system together. Makes it from a bag of bits to a thing.
How to make a stone think. 🪨 💬
this is so thorough tutorial for FAT that even my monke brain can understand the basic logic, maybe make it bit easier to find with tags for it. I'd hate to miss this information when searching for it.
Glad you liked it. What tags would you suggest?
Loading code directly from the SD card, such a satisfying step forward!
And also by uploading this code to the SD card via an FTP server, which will require a network adapter on some ESP8266. Then it is possible to provide terminal access. Multitasking and... :)
Next step is make the monitor support executable files directly!
Nice one James! Reminds me of reading through that massive ST Disk Drives: Inside and Out book back in the day. I got distracted making boot sector demos that were around 400 bytes - fun times!
Ha! I did some similar things! Really long iteration time when you only have one pc though!
Awesome. I always think the moment you can load and save to storage is a turning point in any homebrew project. Suddenly you find yourself looking to standardize the filesystem interface. This is also usually when the idea of a C runtime occurs to me. It's the promotion of a project from a hardware one to a software one.
Seriously, well done!
I agree! Can’t wait to share the advances I’ve made since!
This was wonderful! For your next trick: synchronizing on $4489 and MFM decoding. You know you want to!
I’ll settle for a basic dir command 🤔
At long last, much of a BIOS, and much of the filesystem layer! A DOS cannot be far behind!
Watch this space!
So awesome! You amaze me and I really appreciate you. This was an absolutely fantastic breakdown and it answered many questions for me (about how I thought it worked).
Glad you found it interesting!
Regarding that cache sector, it resembles the functionality of the Forth block word set. You have block (load block, except for the return value, since there's just one buffer), with one more flag of status (updated aka dirty) you could add buffer (map without read), update (mark modified) and save-buffers (save modified buffers). You can probably tell why 32x16 characters per screen was used.
I actually added a dirty flag to support writing shortly after recording this video.
This is fantastic. I've wanted to learn about file systems for a while, and this is a great primer! Thank you! I have a suggestion. Perhaps you could add a RAM chip that uses SPI for doing the caching (or for whatever use). I'm looking at a 23K256-I/SN from Microchip Technology on DigiKey that might work for you.
I've actually looked at one of those ram chips as a possibility. In theory I can interface any spi device now.
Nice work, James!
It might not be super helpful, since it's assembly for a different architecture, but the x86 assembly source code for MSDOS is available. If you want to study how they do pattern matching, etc. Might give you a few ideas on how to implement similar functionality in your own SDOS (SD-card Operating System?).
It's always interesting to around the source to see how things were done. For my project memory is a big constraint, so working out functional solutions that don't consume too much space is always a big challenge.
Lol i hope he comes out with a book on computer architecture and operating systems. Most texts are just too detailed and dense but if they explained things simply like he does here, it'd be a great foundation to build on.
I'm unlikely to do a book, making videos already takes up far too much time!
Love this! the algorithm did me right recommending your channel
Awesome! And Welcome, I hope you enjoy!
This is really coming along :D Will you be able to edit your games in notepad now on a PC then load them into this?
Also, I'd love to see a floppy drive implementation :D
At the moment the assembler is still only on the pc, so I have to build the executable and then transfer them.
good old dos delete, remove the first char in the file name.
reminds me of the fun we used to have back in the days before windoz.
A TSR linked into the keyboard interrupt was always fun to throw an odd keyboard strike in to the user typing.
I used to have writing TSR programs that played pranks on people!
I forsee a future with lots of game cartridges, erm I mean SD Cards with custom printed labels. 😂
Years after USB Autostart was deprecated, I see james bringing it back to life with SD Cards
To be honest I’ll be able to fit every program ever written for this device on a single SF card 🫣
@@weirdboyjim Yes I know 😂 But SD Cards are cheap and it would be so cool 😂
You need to add the floppy seek sound.
Could probably manage a screech of some kind from the audio device.
The Mega 65 does that using a floppy drive. ;)
FAT started on an microsoft disk basic for intel 8080 or z80. MSX also used it, it survived into 32 bit era but i dont associate it with 16 bit era
My understanding was that fat was developed for the early pc’s 8086/8088 which are 16 bit. That’s definitely what I associate it with at least.
fire. Best, as ever
Thanks twobob!
Ohh very exciting!
Hope it doesn't disappoint!
Would it be possible for you to implement system interrupts? You would be able to make a system timer and process timed events together with processing interrupts from external devices
It’s possible but it would be tricky and I have no plans. I will be supporting interrupts in a future build though.
@@weirdboyjim I have thought on implementing it.
This is how I would have implemented it:
At the end of a instruction, when a interrupt has occurred, a separate ROM starts executing instructions with its own PC. It executes push functions where all registers are pushed to the stack. Afterwards it will call a address where you can implement your interrupt handler.
When the RETI command is executed, the interrupt control block triggers and it will pop all registers back into its original state.
You can have a special purpose register where it is possible to have all interrupt sources as flags.
Can I find details about the CPU instruction set somewhere?
And do you have a simulator for it or are you always running code directly on the real hardware?
Thanks!
Yes, I have a simulator I've written. You can get it on the #toolchain channel on the discord.
Hi James,
I've noticed a few times now that you still have the entry nop at the beginning of each assembly file.
I think at some point this was out of necessity for the pipeline initialization, but didn't you fix this "bug"?
Why do you still keep this?
It’s very much a legacy thing in most places. The first byte of the loader ROM still has a problem but it’s as much conventional as anything else now.
when are you going to add sprites to the vga,or did i miss it
Sprite hardware is the last bit of functionality that doesn’t exist yet. Will be starting it soon!
🤩
🙂
Did 8 bit computer have a file system or did they jus just utilize storage as part of the memory map?
Some did, most of the time it was as an optional extra floppy drive.
@@weirdboyjim Yeah but some of them had to have a whole extra computer in the floppy drive for it to prepare the data for the main computer. That seems to be the sollution I have most encountered in 8 bit computers. We never used the floppy drives on my grandfathers ABC 80 while it was in his possesion so I have no idea about those. But a c64 must have had a file system that the drive could read and re arrange it so that the computer could recieve it through the tape loading routines?
OT: feckin YT is deploying 4 minutes in 3 ads, on an unlisted video. It becomes more and more annoying. James, did you ever consider to allow your Patrons to view your videos outside youtube?
On-topic: nice one, as always!
Thanks! That level of adverts sounds annoying! I know patron have a facility to host videos now which might be an option. I'll need to think about that, I want to avoid "overhead creep" in video production.
@@weirdboyjim Sure. I don't wanna push you into more work. And I don't wanna make you missing income. However, the amount of ads becomes more and more ridiculous.
...firefox
ublock
thanx
You are welcome.
No need to defrag just yet...
Far less need to defrag on solid state media. There are some benefits to sequential reads but no cost like moving a physical head.