Also for anyone like me who does disk imaging, make sure you create flux images with multiple revolutions - preferably at least 5 revolutions. All the major flux imaging tools and hardware support this and more than 1 revolution is necessary to fully capture many types of copy protection. It makes the flux images bigger but more flux information is always better than not enough. For GreaseWeazle the --revs=5 parameter is all you need to ensure 5 revolutions are captured. Kryoflux captures 5 revolutions by default. I believe SuperCard Pro captures 2 revolutions by default but can be easily increased in the SCP software.
Yes, absolutely! I sold a copy protection scheme for the Atari ST in the 80s which had a sector which started at the very end of the track and went well over the index mark. It could be read just fine, because the FDC ignored the index mark during a sector read. If you stop reading right at the index mark, aligning the beginning of the track with the end is probably not possible.
@@markusfritze Some PCs back then had great control of the hardware like the APPLE ][ and the C64. The IBM also had fairly good control. Thanks for the info about Flux. I only used to use some software like Anadisk/Teledisk for IBM,locksmith (v6 is the best) for apple and some copy protection/disk copying manuals for Apple ][ that talked about track arching etc. You can still find good manuals online in archives like Asimov that discuss all of the protections of the day right upto the 1990s.
Geez... I thought I knew disk formats, having designed the hardware and written low-level device drivers for two (very) proprietary hard and floppy disk controllers in the 1980s, but this stuff just made my brain explode... Just brilliant, and huge respect for the level of research that you've put into understanding this subject and using these tools to their fullest capabilities.
Our university library is working on creating a digital preservation workstation, focused on older media at the institution first, but there's probably no restrictions on what they'd work on. This video was shared with them as it was the perfect length and detail for what they need as an introduction to working on floppies. A lot of the media that needs to be worked on is pre 2000. Seems the important flash drives and zip disks mostly got copied to network storage, but when storage was more at a premium, floppies archived a lot of stuff. Thanks for sharing this!
Realistically I'm personally never going to have any use for this information but your presentation just makes it so fascinating I can't help but watch! Thanks for the video and esoteric knowledge!
Disk formats are something that I watch from afar these days. Just not enough time for my regular systems job to work with them myself. Retirement is only few years away. Maybe I will come back to it.
Another use case for raw backups of floppy drives, is to recover discs that are very hard to get, by using several drives. I did it with some bad amiga discs. Thanks to the "guided raw" option of the kryoflux, you can dump, and check which track is good, and which one is bad. As every track is backup as a file, and different floppy drives, get different recovery success you can overwrite the successful tracks files on top of the bad ones, to get a perfect raw backup of the disc. and then, do a successful binary conversion.
Love my Kryoflux. So far I have had no problems using Kryoflux to achieve everything from archival quality flux copies to just converting the disks to use with an emulator. Of course, I use it just for Commodore (and maybe a Nabu if I can get a disk drive.)
Having had to work all this out for myself to preserve an extremely rare HH Tiger machine, I found this video is greatly reassuring and wished it had been around more than a year ago! Being able to rescue and even create new floppies for my Victor 9000 with its ZBR arrangement was a godsend!
We can add hard sector floppies to the to the mix. The North Star floppy disk system used ten sector disks with each sector marked with a hole marking each sector. A bit more than that, but that is the gist of it. They started out with 90k floppies, then 180k. but that was ok, as the system, at the time only had 24k of RAM. The controller was an interesting board, 1k of read only memory. I have a note on how that worked if anyone is interested.
Excellent video as always! I would add though there are formats out there where not even sector header information is valid which can be a problem with disk imaging software. The original Quantel Paintbox (DPB-7001) used 8" DSDD disks, the formatting essentially is one sector per track. When writing a track the machine looks for the physical index pulse, then writes a few 0x00 then 0x53 ('S' for sector) and then 19200 bytes of data payload and that is literally it... no sector IDs, no checksums literally nothing else. In later years they did add more checks into it, checksums etc but it's a big problem imaging and converting as you are stuck at working at the flux level.
I used to fiddle with sectors on disks back in the good old days on my C64 and Amiga. It was great fun and something I miss about old computers and tech gone by. This was a refreshing insight into the things I didn't get when I was learning about raw copies and sector meddling to copy those protected floppies. :) Man, I'm old.
Your explanation of this and the laserdisc archiving in previous videos are really excellent. You have a real talent for making this highly technical stuff understandable, and that's a valuable skill.
It's vanishingly unlikely I will ever be doing any of this myself, but this info and the way you discuss it is absolutely freakin' FASCINATING, and I am extremely grateful to those who are preserving the data on these disks for future generations. Also "Greaseweazle" is a freakin' awesome name for a piece of tech. ;)
Props to YT for recommending your video ! I pretty much only work with TRS-80 disks (I have 2 M1s, 4 M4s and a 4P), and use my greaseweazle to image to .scp files for the HxC software to decode for me. That dummy disk image view has been great for dealing with all the different kinds of disks I see. I've got pretty much the same drive setup, but keep the ds40 drive on a shelf for rare just-in-case use. I might copy your triple-drive setup, though. It looks pretty handy. And now it's time to go scroll through your entire videos list.
great video! As a suggestion, I would love a guide like this for CDs and DVDs, specially with dealing with DRM. I made an effort to make images of all my games, but I'm not sure if I did a good job, some times my imaging would fail and I resorted to manually the files and creating a new ISO.
I've got a TRS80 M1 giving me fits on the disk system. This video helped to solidify my understanding. I've been waiting for my GreaseWeazel to arrive from the other side of the pond. Thanks for making this video.
18:56 YES! jaut-76, this is also exactly what we see on your 8" SWTPC disks that I imaged for you, so take note! Shelby knocks this explanation out of the park...way better than I was able to articulate!
You're going to want to make a part 2 for this discussing the Copy 2 PC Option board, and further discussion of various copy protection schemes like weakly recorded bits, intentional bad sectors, and how the exact floppy controller might be required in order to read copy protected discs (preventing some clone and 3rd party controllers from working). Also you kind of omitted the fact that floppy discs are analog, and their controller is basically a software controlled mechanism. Oh, and the hacks we used to use to make 360KB discs hold 800KB. And an expansion on the 'booter' programs. Which correlates with why off-the-shelf USB floppy drives are a bad choice for archival purposes. The floppy rabbit hole goes pretty deep.
Shelby, I LOVE this topic...excellent video, thanks so much for creating this! This video is the BEST EXPLANATION I've seen on UA-cam YET on how flux encoding works on floppy disks...superbly done!!
This was a great video! I have a really hard time understanding computers because I usually do analog electronics and mechanics. Starting with the low level stuff really helps me understand!
It could be great if someone could simplify the command line interface by adding step by step screens that ask questions to the user, providing guidance as to what's being asked. This idea has already simplified many workflows like jailbreaking consoles, upgrading node modules, etc.
If it helps, I added some rough documentation of reading, converting, and writing to my written side of this to see some basic examples of the commands: wiki.techtangents.net/wiki/Floppy_Disk_Imaging#Format_Structure_and_Converting
Having listened to more than a few Jason Scott talks it's really nice to get a technical intro into this subject. I'm not worried about the magnetic artifacts though. The amount of data/media that has slipped through our fingers in this digital age is way scarier than getting a proper flux track of some 80s floppy. A few days ago the Yuzu repo imploded into a logic cloud. Git has the awesome characteristic of every clone being the full source of truth yet there's so much more to it (and with naive googling I haven't found a mirror so far,, maybe I just to wait till the smoke clears).
TY for the video it finally made me buy a Greaseweazle i was thinking about buying something alike for years. Also i like the style a lot, it feels like there are no "things you should've just known" left.
The amount of knowledge you have is simply astounding. I'm always amazed and I SO like to listen to you, although majority of this stuff will be of no use to me. Never the less, thank you for taking the time to explain!
This video was awesome. Helped me a ton to better understand disk formatting. In the process of trying to write downloaded images to physical floppies for multiple vintage machines, but mostly for Kaypro and TRS-80; Apple is easily taken care of with ADT Pro. Thanks for the education! 😃
Thanks for presenting. I remember Copy to PC and tools of such and made a BIN file of a Floppy and image of Zork (way back when). I still have some of the old floppies around.
If you're using a greaseweazle just use ` --tracks="step=2" ` to tell the software to double step the drive which will make it read every other track. That's really all there is to getting flux off of the disks with a 1.2MB drive.
I bought a Greaseweazle device a year or two back. I had full intention of backing up all my old PC floppies, BBC Micro floppies, Atari ST ones, and finally my Acorn Archimedes discs before they suffer any bit rot. I can do most of them, except the PC 5.25" ones as I can't find my old 5.25" drive. I really thought it was going to be in storage in my attic, but I emptied most of that recently to do some home renovation and can't find it. For now I'm not prepared to pay the silly prices that the PC 5.25" drives are listed for, so I'll just have to wait. One will cross my path eventually.
The analogy I use to explain the difference between informants and formatted capacity (2MB as opposed to 1.44MB) is this. Imagine you are going to use a garage wall for storage. The wall is ten feet high by twenty feet across. 10x20 gives 200 square feet of storage. Except, it doesn’t. By the time you cut the two by fours to size and arrange them to support the shelving and cupboards, you end up with about 144 square feet of storage. Floppy formatting is similar. Some of that magnetic real estate has to be used to set up the areas where data will be stored so that it doesn’t spill over into other data. The file allocation table and directory structure needs somewhere to sit and the physical limitations of the floppy drive head limits cannot reach the edges of the disk. For a while, the 2.88MB floppy disk was available. It relied on a floppy drive that could take shorter steps so that it had twice as many steps as a 1.44MB drive. 1.44MB floppy disks could be read in the 2.88MB drive but only 2.88MB disks would provide reliable storage due to the method to record the data and the hysteresis used to ‘fix’ the data. It was a step up from the same technology that allowed single sided, single density disks to be read in new double sided, single density drives. The 2.88MB format was superseded by the availability of ZIP and JAZ drives which provided 100MB of storage and had more in common with hard drive technology.
The analogy I like to describe why you would want a kryoflux image as opposed to a floppy image file is like the difference between a high resolution scan of some archival newspaper article as opposed to a text file with the text contents of that article in that newspaper.
I imagine that a reason you don't see that copy protection on modern software is that not a lot of people would want to reboot their computer just to run one game. Also with that weird track layout I can imagine there may have been a few drives or floppy controllers that could not recognize it. The last reason you don't see it is probably cost. I know other computers had floppy copy protection too. I know there is a popular DOS game for the PC that is protected. I knew someone who tried to copy the files from the disk to one of his own. All that happened was that his copy didn't run. It was unusual because it was something we didn't see a lot. I just know he never had any luck,. That game was Lemmings. It has some catchy music and some levels that seem to be just about impossible to me.
When you talk about DRM, I remember when you had to install Lotus-123, you had to set the clock on the computer to 8Mhz from 12MHz, for their copy protection to work.
Are any of these able to identify and/or write half/fuzzy/weak bits yet? FYI half/fuzzy/weak bits are when certain bits are purposefully written incorrectly so that the flux is halfway between a 0 and a 1. This makes it so that about half of the time it will be read as a 0 or a 1. The copy protection keeps track of those bits and if it sees that it’s been read as a 0 or 1 multiple times in a row (instead of alternating) then it triggers copy protection. It’s normally been impossible to deal with as most software can’t identify the half bits in the flux stream and if you don’t know where they belong you can’t even attempt to write the halfbits. I have quite a few X68000 games that I really want to back up for preservation but the half bytes make the images unusable without someone cracking the game which I don’t consider to be a true real backup.
Fascinating, but… I don't have a PC. I'm looking forward to requiring my Atari Home Computer Systems that were taken after I moved to California thirty years ago. The photo I saw showed at least one 1050 floppy drive and a couple of cases full of floppies. Originally, Atari DOS 2.0S formatted 5 1/4" floppies in single density. Then Atari DOS 3 came out and offered an "enhanced" density format. I don't remember now if it provided more sectors per disk or just stored more data per sector, but the result was about a 50% increase in data storage per disk. I added a bootleg of a third-party ROM to my floppy drive to provide true double density. If these old computers actually still work, I plan to see if I can read these disks and copy the files to an SD card using modern accessories for my old computers. I think I had some disk editor software back in the day and I have a detailed source listing of at least the original DOS 2.0S software.
The OSI 610-Floppy controller board was one truly weird beast. It basically just sent the output of 6850 serial uart to the disk heads (FM?) It didn't do sectors as far as I know, You basically just read "whole tracks" ... But given that a whole disk image (40 tracks) was a mere ~60k that was fine Loading a program was basically just reading in a track to memory :)
Thank you. A fine run down. Certainly agree about the GW. The kryoflux never seemed to work out for me. Perhaps copy protection could be covered elsewhere as the number one message you make is. Don’t just flux image, far too many raw images contain errors in the encoding and are useless. The GW encourages checking reads as central to its philosophy. Next part… writing to physical media?
11:22 So, since I always load my raw flux image file into HxC to view how good the read was, I also always use HxC to export known formats, by simply clicking the Export button and choosing the IMG file (.img) RAW Sector file format, which should accomplish the same thing you've demonstrated here. Aha, I see you also show that at 14:15, thank you! But, could you speak to why going to the extra step of using the GW command line to do this is beneficial? 14:24... Ah, this works in HxC "ONLY under ideal circumstances, though". Excellent that you explain why. Thank you! I also thought that HxC could allow you to specify the same parameters on export, thus fixing those strange circumstances you show, but apparently not? I should experiment with that more and report back. THANK YOU for showing how this is done in GW...I've certainly learned something, and I've also been working with strange format floppies for years...again, excellent content. I love your channel!
My first and favorite archival of game install discs would be for Tribes Vengeance, which used regular DVD's or the like... quite literally was worried due to the few scratches and many years since I played a cracked version, and simply copied the discs to an ISO which I then installed the game from. Not sure how much I went into getting the game to run though, as it required the first disc inserted to play, and almost immediately looked for a patch/workaround which I found. Was nice to get it working, and I still have the original discs in their rather readable state.
Should do an episode like this on zip disks at some point, I have a few that decided to stop working because I think the tracking sectors have gone bad. And of course they're the ones with data on them that's not backed up which I was trying to do when I found out.
That Mac software on an MFM disk is interesting! I think it won't run on anything older than a IIx, since the Integrated Woz Machine used up to that point only handled GCR and not MFM.
What about the half (or quarter) tracks of Apple ][ 5.25" disks where the stepper phase could be accessed in a manner that moves the head 1/2 (or even 1/4 or 3/4) between tracks to store copy protect keys checked by the software? The other big copy protect tactic was to alter the GCR header tags but that will get picked up by the flux.
5:36 "Multiple reads of each track". I've recently discovered that The Applesauce does excellent at this too... They are way more expensive and NOT open source like the GreaseWeazle, but worth mentioning...
I was going to say there were some weirdly formatted disks for copy protection. I think some early Mac programs and some Apple II programs used it. Also, what about the IBM 2.88 MB disks?
22:16 Microsoft Adventure - Copy protection example...cool! I wonder if the flux of this has been uploaded to archive? I should check... I'd kinda like to play with that... 23:19...Unless you want a REALLY big challenge.....YES...I DO, thank you!!!
You are definitely the OG for this stuff on UA-cam. Adrian Black is the OG for repairing old computers, down to board level components. Also CRTs. And the 8-Bit Guy is the OG for cleaning up and retro-brighting old systems.
Great tutorial. But how about hard sectored disks? Still haven’t found a way to preserve nor write them, I guess it’s a whole other challenge (and ressources still untapped)
It is a more difficult challenge that requires throwing out existing code that relies on the index pulse, over sampling the track flux, and manually trimming the start and end of the tracks by timing the index holes to see where the extra one is. I have read that it might be possible with existing hardware, but I've been focused on the fundamentals discussed here and haven't tried it yet. I do have some hard sector disks, so sometime I do need to try it some time to find out if it is possible.
I wonder if there are any flux readers/writers for mitsumi quickdisc (QD) drives, as found on some early akai and roland samplers, a few sequencers, and word-processing typewriters... also used on the famicom disk unit.
Of course, the "1.44 MB" labelling is also a misnomer because the standard format 80 x 2 x 18 x 512 = 1,474,560 byte is a bit over 1.47 decimal megabytes, or nearly 1.41 mebibytes.
Another weird case is the PC98. In many cases, they just ran the motors of the drives at 360 rpm instead of 300 rpm. Otherwise, they had normal MFM encoded.
3:38 374x type data entry. 3742 was typically a 'dual station entry", two people sitting face to face. 3741 was a single unit. I don't recall having dual drives on a single user machine. The image is older than my brain implant.
BEFORE 'Norton' (later Symantec) bought the rights to the program and then 'updated' it, altering the program when they comnercialised it, 'Ghost' was known as 'Pharr-Lapps Ghost' by Binary Research Labs - licenced to 'Pegasus Software' for corporate use. That program was great, and did both data-stream images and Sector-by-sector images flawlessly regardless of what the bytes were. Easy. It had its own Disk Operating system and commands, so it was very clean and efficient cos it didnt care what it was backing up. Then came commercial usage with its avoidance of copying protected data. Ah well.
Very interesting, especially since I have some Apple2 disks to save. What drives do you recomand to do so? (If you have a tool compatible with original Apple2 drives to do so, I have said drives.)
Funny how the internet know what I'm doing.... I just got a 10 bay ATX tower for a Grease Weasel project 360k single side 360 DD, 720K, 1.2HD 1.44HD 3" all on one ribbon connected to a Grease Weasle 4.1 usb to internal usb headers, Its a Pentium 3 1Ghz Asus CuSL2-C AGP 3DFX 16mb +SLI12mb +4mb PC passthough, IDE Zip250, IDE LS120, Pioneer Slot load DVD, 80GB IDE SSD , 768MB ram, SBLive sound, all into one tower for the purpose of archiving, preserving and writing games back to realy disks. I have Apple iis, C64, Amiga, Atari ST, and rare PC8801,PC9801 systems. Get a Grease Weasel you won't be sorry
I remember my Acorn BBC model B duel 40/80 track disc drive cost more than the BBC model B Computer, i remember being a child at the time feeling shock at this and thinking I could also buy a Commodore 64 computer for the price of the disc drive but upon using the disc drive i very soon understood how awesome a disc drive was compared to using a tape cassette and playing Elite on disc sold me to the BBC B and disc drives
great video. although I must admit, I spent half of it trying to figure out how you'd get a 5.25" disk with different formats on each side. My guess: they formatted it first in a system like a PC with a double-sided drive, then needed to reuse it on a single-sider like a C64/Apple II, and only bothered reformatting the side they used for that?
That's about my nearest guess, it could also easily happen with any single sided system if you put a fresh out of the box pre IBM formatted disk into it. I had to learn it the hard way with some disk images from the HP Museum that are for some reason HP 9121 formatted on one side and IBM 720k on the other and needed to get only the HP data.
Also for anyone like me who does disk imaging, make sure you create flux images with multiple revolutions - preferably at least 5 revolutions. All the major flux imaging tools and hardware support this and more than 1 revolution is necessary to fully capture many types of copy protection. It makes the flux images bigger but more flux information is always better than not enough.
For GreaseWeazle the --revs=5 parameter is all you need to ensure 5 revolutions are captured. Kryoflux captures 5 revolutions by default. I believe SuperCard Pro captures 2 revolutions by default but can be easily increased in the SCP software.
Yes, absolutely!
I sold a copy protection scheme for the Atari ST in the 80s which had a sector which started at the very end of the track and went well over the index mark. It could be read just fine, because the FDC ignored the index mark during a sector read. If you stop reading right at the index mark, aligning the beginning of the track with the end is probably not possible.
@@markusfritze Some PCs back then had great control of the hardware like the APPLE ][ and the C64. The IBM also had fairly good control. Thanks for the info about Flux. I only used to use some software like Anadisk/Teledisk for IBM,locksmith (v6 is the best) for apple and some copy protection/disk copying manuals for Apple ][ that talked about track arching etc. You can still find good manuals online in archives like Asimov that discuss all of the protections of the day right upto the 1990s.
Geez... I thought I knew disk formats, having designed the hardware and written low-level device drivers for two (very) proprietary hard and floppy disk controllers in the 1980s, but this stuff just made my brain explode... Just brilliant, and huge respect for the level of research that you've put into understanding this subject and using these tools to their fullest capabilities.
You want me to forget everything I know about how floppies store data... I'm way ahead of you, I forgot that over 20 years ago :)
I never knew anything to begin with. 😁
Our university library is working on creating a digital preservation workstation, focused on older media at the institution first, but there's probably no restrictions on what they'd work on. This video was shared with them as it was the perfect length and detail for what they need as an introduction to working on floppies. A lot of the media that needs to be worked on is pre 2000. Seems the important flash drives and zip disks mostly got copied to network storage, but when storage was more at a premium, floppies archived a lot of stuff. Thanks for sharing this!
Realistically I'm personally never going to have any use for this information but your presentation just makes it so fascinating I can't help but watch! Thanks for the video and esoteric knowledge!
Disk formats are something that I watch from afar these days. Just not enough time for my regular systems job to work with them myself. Retirement is only few years away. Maybe I will come back to it.
Another use case for raw backups of floppy drives, is to recover discs that are very hard to get, by using several drives.
I did it with some bad amiga discs.
Thanks to the "guided raw" option of the kryoflux, you can dump, and check which track is good, and which one is bad.
As every track is backup as a file, and different floppy drives, get different recovery success
you can overwrite the successful tracks files on top of the bad ones, to get a perfect raw backup of the disc.
and then, do a successful binary conversion.
Love my Kryoflux. So far I have had no problems using Kryoflux to achieve everything from archival quality flux copies to just converting the disks to use with an emulator. Of course, I use it just for Commodore (and maybe a Nabu if I can get a disk drive.)
Having had to work all this out for myself to preserve an extremely rare HH Tiger machine, I found this video is greatly reassuring and wished it had been around more than a year ago! Being able to rescue and even create new floppies for my Victor 9000 with its ZBR arrangement was a godsend!
We can add hard sector floppies to the to the mix.
The North Star floppy disk system used ten sector disks with each sector marked with a hole marking each sector. A bit more than that, but that is the gist of it. They started out with 90k floppies, then 180k. but that was ok, as the system, at the time only had 24k of RAM. The controller was an interesting board, 1k of read only memory. I have a note on how that worked if anyone is interested.
Excellent video as always! I would add though there are formats out there where not even sector header information is valid which can be a problem with disk imaging software. The original Quantel Paintbox (DPB-7001) used 8" DSDD disks, the formatting essentially is one sector per track. When writing a track the machine looks for the physical index pulse, then writes a few 0x00 then 0x53 ('S' for sector) and then 19200 bytes of data payload and that is literally it... no sector IDs, no checksums literally nothing else. In later years they did add more checks into it, checksums etc but it's a big problem imaging and converting as you are stuck at working at the flux level.
I used to fiddle with sectors on disks back in the good old days on my C64 and Amiga. It was great fun and something I miss about old computers and tech gone by. This was a refreshing insight into the things I didn't get when I was learning about raw copies and sector meddling to copy those protected floppies. :) Man, I'm old.
I recall that the hardware on our Amiga 1000 could leverage 82 tracks!
And now I understand why PC floppy drives cannot read 800k mac disks! Thank you so much for this! You're very good presenter ❤
DataViz Conversions Plus would read mac disks on PC and PC disks on Mac versions of the software. Windows 95/XP days if I remember correctly.
@@roydugger7303 the pc drive hardware could not read 800k mac disks.
@@BestSpatula The DataViz software would read 800k Mac disks on a PC 3.5" drive.
@@roydugger7303 you need special hardware like fluxengine to do it. Pc floppy controller cannot read 800k or 400k mac disks.
Wow! This is super timely with my own backup project underway. I cannot believe the massive depth this actually goes into.
Your explanation of this and the laserdisc archiving in previous videos are really excellent. You have a real talent for making this highly technical stuff understandable, and that's a valuable skill.
I have a bunch of Floppy disks in "T-Format" formatting, which worked perfectly in DOS and Win9x, but dead in the water in NT.
T-Format claimed 1.72 MB max capacity, with DriveSpace this could give you 5+ MB of capacity if you used basic Text documents.
It's vanishingly unlikely I will ever be doing any of this myself, but this info and the way you discuss it is absolutely freakin' FASCINATING, and I am extremely grateful to those who are preserving the data on these disks for future generations.
Also "Greaseweazle" is a freakin' awesome name for a piece of tech. ;)
Props to YT for recommending your video ! I pretty much only work with TRS-80 disks (I have 2 M1s, 4 M4s and a 4P), and use my greaseweazle to image to .scp files for the HxC software to decode for me. That dummy disk image view has been great for dealing with all the different kinds of disks I see. I've got pretty much the same drive setup, but keep the ds40 drive on a shelf for rare just-in-case use. I might copy your triple-drive setup, though. It looks pretty handy. And now it's time to go scroll through your entire videos list.
great video!
As a suggestion, I would love a guide like this for CDs and DVDs, specially with dealing with DRM. I made an effort to make images of all my games, but I'm not sure if I did a good job, some times my imaging would fail and I resorted to manually the files and creating a new ISO.
I've got a TRS80 M1 giving me fits on the disk system. This video helped to solidify my understanding. I've been waiting for my GreaseWeazel to arrive from the other side of the pond. Thanks for making this video.
18:56 YES! jaut-76, this is also exactly what we see on your 8" SWTPC disks that I imaged for you, so take note! Shelby knocks this explanation out of the park...way better than I was able to articulate!
Indeed he does. Hopefully this means we can have the disks available and working for the masses
I lived through this time of computing, admittedly as a dead end user.
I had no idea about any of these complexities.
Thank you. This was amazing.
I first started using floppies in (I think) 1986. And yet, you've taught me some things. Well done!
Thank you for this! I grew up at the tail end of floppy and have been struggling some trying to get into it.
I didn't understand a single word, and yet I loved every second. Shelby rules!
You're going to want to make a part 2 for this discussing the Copy 2 PC Option board, and further discussion of various copy protection schemes like weakly recorded bits, intentional bad sectors, and how the exact floppy controller might be required in order to read copy protected discs (preventing some clone and 3rd party controllers from working). Also you kind of omitted the fact that floppy discs are analog, and their controller is basically a software controlled mechanism. Oh, and the hacks we used to use to make 360KB discs hold 800KB. And an expansion on the 'booter' programs. Which correlates with why off-the-shelf USB floppy drives are a bad choice for archival purposes. The floppy rabbit hole goes pretty deep.
For some reason, the “floppy rabbit hole” made me picture those portable holes from Toontown.
Yeah, there's an endless probability of dark abyss scenarios created by the evolution and de-volution of data storage
Tell me you didn't watch the video without telling me you didn't watch the video.
@@davidwillmoreUA-cam has video now?? 🤔
I would very much like to see all this covered in a follow up video. Fascinating stuff!
Shelby, I LOVE this topic...excellent video, thanks so much for creating this! This video is the BEST EXPLANATION I've seen on UA-cam YET on how flux encoding works on floppy disks...superbly done!!
I bought a Catweasel 20 years ago to archive my old Commodore and Amiga disks. Nice to see contemporary tools to work with these images.
This was a great video! I have a really hard time understanding computers because I usually do analog electronics and mechanics. Starting with the low level stuff really helps me understand!
What a fantastic video. This will be so useful for generations to come. Make sure you backup the flux map of this video!
There was also M2FM format disks mainly used on some 8 inch drive systems. It was a hybrid FM MFM system
My linux brain lit up at the kde oxygen window decoration + nimbus GTK theme :D
I was like: Is this Java with Nimbus look and feel?
I found a box of old floppies, but didnt know a thing about reading them! Thanks for this perfect video!!
It could be great if someone could simplify the command line interface by adding step by step screens that ask questions to the user, providing guidance as to what's being asked.
This idea has already simplified many workflows like jailbreaking consoles, upgrading node modules, etc.
If it helps, I added some rough documentation of reading, converting, and writing to my written side of this to see some basic examples of the commands: wiki.techtangents.net/wiki/Floppy_Disk_Imaging#Format_Structure_and_Converting
Man, do i feel stupid now. I had no idea it was so complex. Thanks for the video!!
Having listened to more than a few Jason Scott talks it's really nice to get a technical intro into this subject. I'm not worried about the magnetic artifacts though. The amount of data/media that has slipped through our fingers in this digital age is way scarier than getting a proper flux track of some 80s floppy. A few days ago the Yuzu repo imploded into a logic cloud. Git has the awesome characteristic of every clone being the full source of truth yet there's so much more to it (and with naive googling I haven't found a mirror so far,, maybe I just to wait till the smoke clears).
Love these explainer videos. I don't own any retro computers but I enjoy watching them.
This is epic, what a community contribution both in explanation and projects!
TY for the video it finally made me buy a Greaseweazle i was thinking about buying something alike for years. Also i like the style a lot, it feels like there are no "things you should've just known" left.
Great video, thanks for sharing your scripts. I wouldn't have been able to archive the Zobex boot disks without your help.
Great guide that I've come back to multiple times. Thanks!
6:35 I've used this exact feature if HxC for YEARS...and I still love it! Use it all the time!
The amount of knowledge you have is simply astounding. I'm always amazed and I SO like to listen to you, although majority of this stuff will be of no use to me. Never the less, thank you for taking the time to explain!
Wow! Amazing explanation of how this all works! Now,I finally try some myself. Thanks for sharing
This video was awesome. Helped me a ton to better understand disk formatting. In the process of trying to write downloaded images to physical floppies for multiple vintage machines, but mostly for Kaypro and TRS-80; Apple is easily taken care of with ADT Pro. Thanks for the education! 😃
Fantastic review of use of disk preservation systems. Thanks!
Sector interleaving is quite interesting as it shows how slow the CPUs at that time were...
Thanks for presenting. I remember Copy to PC and tools of such and made a BIN file of a Floppy and image of Zork (way back when). I still have some of the old floppies around.
Thank you Shelby!!! Great content
I would love to see a detailed video on preserving Apple2 disks using an IBM 1.2MB drive, since finding 360K drives is a nightmare.
If you're using a greaseweazle just use ` --tracks="step=2" ` to tell the software to double step the drive which will make it read every other track. That's really all there is to getting flux off of the disks with a 1.2MB drive.
Great video, I have a few old 286 XT era shareware games on disk I would like to archive, this explains a lot that I was having issues with. thanks 👍
Cylinders are 3D tracks, basically multiple stacked tracks. That's why we talk about cylinders for hard drives and tracks for floppies.
man, it feels like i've been watching Druaga1 and AkBKukU forever now
I bought a Greaseweazle device a year or two back. I had full intention of backing up all my old PC floppies, BBC Micro floppies, Atari ST ones, and finally my Acorn Archimedes discs before they suffer any bit rot. I can do most of them, except the PC 5.25" ones as I can't find my old 5.25" drive. I really thought it was going to be in storage in my attic, but I emptied most of that recently to do some home renovation and can't find it.
For now I'm not prepared to pay the silly prices that the PC 5.25" drives are listed for, so I'll just have to wait. One will cross my path eventually.
Brings back memories looking at all your software. I feel like I'm at Fry's Electronics or CompUSA, those were cool times. 👍 👍
thankyou 1000x for making this intro video to this topic.
The analogy I use to explain the difference between informants and formatted capacity (2MB as opposed to 1.44MB) is this.
Imagine you are going to use a garage wall for storage. The wall is ten feet high by twenty feet across. 10x20 gives 200 square feet of storage.
Except, it doesn’t. By the time you cut the two by fours to size and arrange them to support the shelving and cupboards, you end up with about 144 square feet of storage.
Floppy formatting is similar.
Some of that magnetic real estate has to be used to set up the areas where data will be stored so that it doesn’t spill over into other data. The file allocation table and directory structure needs somewhere to sit and the physical limitations of the floppy drive head limits cannot reach the edges of the disk.
For a while, the 2.88MB floppy disk was available. It relied on a floppy drive that could take shorter steps so that it had twice as many steps as a 1.44MB drive.
1.44MB floppy disks could be read in the 2.88MB drive but only 2.88MB disks would provide reliable storage due to the method to record the data and the hysteresis used to ‘fix’ the data.
It was a step up from the same technology that allowed single sided, single density disks to be read in new double sided, single density drives.
The 2.88MB format was superseded by the availability of ZIP and JAZ drives which provided 100MB of storage and had more in common with hard drive technology.
Great video. Your video on this topic made it to the front page of Hacker News. Good job
The analogy I like to describe why you would want a kryoflux image as opposed to a floppy image file is like the difference between a high resolution scan of some archival newspaper article as opposed to a text file with the text contents of that article in that newspaper.
I imagine that a reason you don't see that copy protection on modern software is that not a lot of people would want to reboot their computer just to run one game. Also with that weird track layout I can imagine there may have been a few drives or floppy controllers that could not recognize it. The last reason you don't see it is probably cost. I know other computers had floppy copy protection too. I know there is a popular DOS game for the PC that is protected. I knew someone who tried to copy the files from the disk to one of his own. All that happened was that his copy didn't run. It was unusual because it was something we didn't see a lot. I just know he never had any luck,. That game was Lemmings. It has some catchy music and some levels that seem to be just about impossible to me.
When you talk about DRM, I remember when you had to install Lotus-123, you had to set the clock on the computer to 8Mhz from 12MHz, for their copy protection to work.
Are any of these able to identify and/or write half/fuzzy/weak bits yet? FYI half/fuzzy/weak bits are when certain bits are purposefully written incorrectly so that the flux is halfway between a 0 and a 1. This makes it so that about half of the time it will be read as a 0 or a 1. The copy protection keeps track of those bits and if it sees that it’s been read as a 0 or 1 multiple times in a row (instead of alternating) then it triggers copy protection. It’s normally been impossible to deal with as most software can’t identify the half bits in the flux stream and if you don’t know where they belong you can’t even attempt to write the halfbits. I have quite a few X68000 games that I really want to back up for preservation but the half bytes make the images unusable without someone cracking the game which I don’t consider to be a true real backup.
I see that copy of Nox in the back. Still one of my favorite games!
Fascinating, but… I don't have a PC.
I'm looking forward to requiring my Atari Home Computer Systems that were taken after I moved to California thirty years ago. The photo I saw showed at least one 1050 floppy drive and a couple of cases full of floppies.
Originally, Atari DOS 2.0S formatted 5 1/4" floppies in single density. Then Atari DOS 3 came out and offered an "enhanced" density format. I don't remember now if it provided more sectors per disk or just stored more data per sector, but the result was about a 50% increase in data storage per disk.
I added a bootleg of a third-party ROM to my floppy drive to provide true double density.
If these old computers actually still work, I plan to see if I can read these disks and copy the files to an SD card using modern accessories for my old computers. I think I had some disk editor software back in the day and I have a detailed source listing of at least the original DOS 2.0S software.
The OSI 610-Floppy controller board was one truly weird beast. It basically just sent the output of 6850 serial uart to the disk heads (FM?)
It didn't do sectors as far as I know, You basically just read "whole tracks" ... But given that a whole disk image (40 tracks) was a mere ~60k that was fine
Loading a program was basically just reading in a track to memory :)
Interesting video. I do agree that this is the best form of preservation. Kudos on this and keep up the good work.
Thanks for the guidance on HP Series 200 disks. I have a few from A Long Time Ago -- it'll be interesting to see if they're still readable.
This guys goatee and hair go hard I respect it
Wow great explanation! will have to watch again for it to sink in, lol. Does clear up alot for me though.
Thank you. A fine run down. Certainly agree about the GW. The kryoflux never seemed to work out for me. Perhaps copy protection could be covered elsewhere as the number one message you make is. Don’t just flux image, far too many raw images contain errors in the encoding and are useless. The GW encourages checking reads as central to its philosophy. Next part… writing to physical media?
11:22 So, since I always load my raw flux image file into HxC to view how good the read was, I also always use HxC to export known formats, by simply clicking the Export button and choosing the IMG file (.img) RAW Sector file format, which should accomplish the same thing you've demonstrated here. Aha, I see you also show that at 14:15, thank you! But, could you speak to why going to the extra step of using the GW command line to do this is beneficial? 14:24... Ah, this works in HxC "ONLY under ideal circumstances, though". Excellent that you explain why. Thank you!
I also thought that HxC could allow you to specify the same parameters on export, thus fixing those strange circumstances you show, but apparently not? I should experiment with that more and report back.
THANK YOU for showing how this is done in GW...I've certainly learned something, and I've also been working with strange format floppies for years...again, excellent content. I love your channel!
I love your videos, can you please do a video on using the domesday duplicator for VHS archival.
My first and favorite archival of game install discs would be for Tribes Vengeance, which used regular DVD's or the like... quite literally was worried due to the few scratches and many years since I played a cracked version, and simply copied the discs to an ISO which I then installed the game from. Not sure how much I went into getting the game to run though, as it required the first disc inserted to play, and almost immediately looked for a patch/workaround which I found. Was nice to get it working, and I still have the original discs in their rather readable state.
Should do an episode like this on zip disks at some point, I have a few that decided to stop working because I think the tracking sectors have gone bad. And of course they're the ones with data on them that's not backed up which I was trying to do when I found out.
I don’t think this is possible without hacking a drive to replace the onboard controller. You don’t get flux-level access through IDE or SCSI.
Great video dude, well done !
Gotta love that name: "Greaseweazle" 😀
Maybe alluding to the old Catweasel floppy controller (and the even older TV show Catweazle)?
I use 2 MB Floppies to save my Dumped Gameboy Games. Perfect size. I didn't seem to have any problems on Windows 10. Works for me.
Also may wish to cover Write-PreComp at track 43 on some 8" Floppy Drives. Ref the Centurion minicomputer from the 1970-1980's.
18:40 that computer look so cool!
I remember interleaving, and being stoked to finally have a PC that was fast enough to not need it anymore...
That Mac software on an MFM disk is interesting! I think it won't run on anything older than a IIx, since the Integrated Woz Machine used up to that point only handled GCR and not MFM.
What about the half (or quarter) tracks of Apple ][ 5.25" disks where the stepper phase could be accessed in a manner that moves the head 1/2 (or even 1/4 or 3/4) between tracks to store copy protect keys checked by the software? The other big copy protect tactic was to alter the GCR header tags but that will get picked up by the flux.
5:36 "Multiple reads of each track". I've recently discovered that The Applesauce does excellent at this too... They are way more expensive and NOT open source like the GreaseWeazle, but worth mentioning...
I was going to say there were some weirdly formatted disks for copy protection. I think some early Mac programs and some Apple II programs used it. Also, what about the IBM 2.88 MB disks?
22:16 Microsoft Adventure - Copy protection example...cool! I wonder if the flux of this has been uploaded to archive? I should check... I'd kinda like to play with that... 23:19...Unless you want a REALLY big challenge.....YES...I DO, thank you!!!
You are definitely the OG for this stuff on UA-cam. Adrian Black is the OG for repairing old computers, down to board level components. Also CRTs. And the 8-Bit Guy is the OG for cleaning up and retro-brighting old systems.
adrian is legit, i envy his abilities
The 8bit guy used to restore Clamshell G3 iBooks back in the day and nowadays writes software in assembly for vintage platforms.
Thought you were gonna say that 8bg is the OG for blowing up systems lol
@@TheEradorsame when I first read it.
Awesome video. Greaseweazle and HxC can be less than intuitive to someone new to flux reading.
Insert "don't copy that floppy" somewhere in here
Great tutorial. But how about hard sectored disks? Still haven’t found a way to preserve nor write them, I guess it’s a whole other challenge (and ressources still untapped)
It is a more difficult challenge that requires throwing out existing code that relies on the index pulse, over sampling the track flux, and manually trimming the start and end of the tracks by timing the index holes to see where the extra one is. I have read that it might be possible with existing hardware, but I've been focused on the fundamentals discussed here and haven't tried it yet. I do have some hard sector disks, so sometime I do need to try it some time to find out if it is possible.
I wonder if there are any flux readers/writers for mitsumi quickdisc (QD) drives, as found on some early akai and roland samplers, a few sequencers, and word-processing typewriters... also used on the famicom disk unit.
this is cool... well done...
Of course, the "1.44 MB" labelling is also a misnomer because the standard format 80 x 2 x 18 x 512 = 1,474,560 byte is a bit over 1.47 decimal megabytes, or nearly 1.41 mebibytes.
Another weird case is the PC98. In many cases, they just ran the motors of the drives at 360 rpm instead of 300 rpm. Otherwise, they had normal MFM encoded.
3:38 374x type data entry. 3742 was typically a 'dual station entry", two people sitting face to face. 3741 was a single unit. I don't recall having dual drives on a single user machine. The image is older than my brain implant.
BEFORE 'Norton' (later Symantec) bought the rights to the program and then 'updated' it, altering the program when they comnercialised it, 'Ghost' was known as 'Pharr-Lapps Ghost' by Binary Research Labs - licenced to 'Pegasus Software' for corporate use. That program was great, and did both data-stream images and Sector-by-sector images flawlessly regardless of what the bytes were. Easy. It had its own Disk Operating system and commands, so it was very clean and efficient cos it didnt care what it was backing up. Then came commercial usage with its avoidance of copying protected data. Ah well.
Very interesting, especially since I have some Apple2 disks to save. What drives do you recomand to do so? (If you have a tool compatible with original Apple2 drives to do so, I have said drives.)
Funny how the internet know what I'm doing.... I just got a 10 bay ATX tower for a Grease Weasel project 360k single side 360 DD, 720K, 1.2HD 1.44HD 3" all on one ribbon connected to a Grease Weasle 4.1 usb to internal usb headers, Its a Pentium 3 1Ghz Asus CuSL2-C AGP 3DFX 16mb +SLI12mb +4mb PC passthough, IDE Zip250, IDE LS120, Pioneer Slot load DVD, 80GB IDE SSD , 768MB ram, SBLive sound, all into one tower for the purpose of archiving, preserving and writing games back to realy disks. I have Apple iis, C64, Amiga, Atari ST, and rare PC8801,PC9801 systems. Get a Grease Weasel you won't be sorry
3" or 3.5"? 3" is rare and the only systems I know that used them were Amstrad and the Sinclair ZX Spectrum +3.
wonder if any hidden easter eggs in those old weird old floppy formats
“I want you to forget everything” *half-life death noise*
I remember my Acorn BBC model B duel 40/80 track disc drive cost more than the BBC model B Computer, i remember being a child at the time feeling shock at this and thinking I could also buy a Commodore 64 computer for the price of the disc drive
but upon using the disc drive i very soon understood how awesome a disc drive was compared to using a tape cassette and playing Elite on disc sold me to the BBC B and disc drives
great video. although I must admit, I spent half of it trying to figure out how you'd get a 5.25" disk with different formats on each side. My guess: they formatted it first in a system like a PC with a double-sided drive, then needed to reuse it on a single-sider like a C64/Apple II, and only bothered reformatting the side they used for that?
That's about my nearest guess, it could also easily happen with any single sided system if you put a fresh out of the box pre IBM formatted disk into it. I had to learn it the hard way with some disk images from the HP Museum that are for some reason HP 9121 formatted on one side and IBM 720k on the other and needed to get only the HP data.
There were specific bulk mastering hardware/software to achieve this.
@@jeff-networkI know, I have some. but still, the idea that someone would do it on purpose is silly! I have to assume it was done on accident