Should I use TAPE BACKUP in 2023? LTO-5 Drive with Proxmox Backup Server
Вставка
- Опубліковано 27 тра 2024
- As promised in a previous video about my Proxmox Backup Server, I have a Quantum LTO-5 tape drive that I'm going to try and use to implement a proper 3-tier backup strategy with offsite tapes.
I'm currently using Linode's object storage for backing up my personal data (~200G), and not backing up the video files outside of the two existing copies (on the storage server and the PBS server). With the affordability of tapes, I can keep the video files and personal data off-site reliably.
My blog:
www.apalrd.net/posts/2023/pbs...
Feel free to chat with me more on my Discord server:
/ discord
Support me on Ko-Fi if you enjoy my content and find it useful:
ko-fi.com/apalrd
Timestamps:
00:00 - Intro
00:41 - 3-2-1 Backup Rule
01:58 - The New Solution!
05:57 - Cloud Comparison
11:31 - Bigger Cloud Comparison
14:06 - Tape Drive Setup
24:50 - More Thoughts - Наука та технологія
I'm a big believer in tape for one main reason. Built a lot of basic machines with tape drives in them for the different departments in a hospital back in 1997 through about 1999/2000 or so. Last year, they had to dig up some records, couldn't find what they were looking for, and thought 'maybe the old tapes?' - Took dusting off an old system to do it, but cracked open one of their storage lockers, brought out some tapes, and they spun up no problem. I loved those things. 5-1/4 bay SCSI interface card. Nice compact tape that could hold an absolute insane amount of data for the time.
The technical term for that problem where the write buffer gets starved and the drive has to stop streaming data to the tape a reposition again is called 'shoe shinning' in the tape world.
technical would be buffer underrun
i.pinimg.com/originals/7b/b9/17/7bb917ccd41722996904c2f7b1bbabac.jpg
Technically, instead of "technically," he meant "colloquially." Dick swinging aside, I find the term "shoe shining" hilariously fitting.
Damn, LTO takes me back!!! It is funny how we thought tape was dead. But sheesh, for sheer bulk storage for backup, it is unmatched. Old things becoming new again. Love it
They still release new versions every few years, with higher end higher capacities, so it’s still relevant for *BIG* data.
Tape is most definitely NOT dead.
There are a LOT of enterprises that still use it.
I have deployed LTO-8 at home to backup upwards of around 216 TB of data and the cost to do that is tiny compared to how much it would cost to back all of that data up to the cloud.
LTO tape is used to store digital movie masters but for blockbuster films like "Oppenheimer" for example they still make a laser recorded YCM film archive. That being three separate black and white negatives. Monochrome film since the 1970s has an expected life of over 250 years if stored properly. Now find me an LTO drive in 2223, that is if the tape didn't already turn to iron dust!
I was at EMC and I remember the bumper stickers we had made that said that. I'm not there anymore, but people I know who are are still supporting tape based backup and recovery solutions. Who remembers DLT?
Tapes never went away at an enterprise level. Small businesses phased them out in favor of... not backing up, mostly 🤣
26:05 A quick time-saving tip with Windows calculator (and many/most physical calculators): if you want to repeat the same operation you've just done (in this case multiplying by 1024), just hit the enter/equals key again.
It is just important to keep in mind that you have to store the tape in good conditions for it to last long. I'm not sure what can affect them, but I guess humidity and temperature are two main factors.
Great video, I barely have any data to justify LTO drives
Yeah there's quite a bit of data you have to archive to break even vs HDDs
Realistically you're not doing backups. You're archiving. Tape is going to be the way to go in that scenario
How is that not a backup? 😅😅😅
Your computer corner looks like it is in a closet... That's interesting... Can it be closed, and how does it look?
@@LucasHartmann It doesn't look like it can be closed and I bet it's space efficient with the desk taking up less room.
@@hhkk6155 "A backup is never a backup until it's restored from -- until then it's just a copy."
If you’re working small time, optical is the way. Discs (good ones) can last 100 years or more if stored correctly.
Nice video - good to see someone else using LTO for homelab stuff.
I'm using mine as a hoard. 150 LTO-4 Tapes using an internal LTO-4 drive. Archiware allows me to index everything.
Archiware is really expensive!!
@@djole02 yeah it was a lockdown project for me. They had a promotion at the time where you could get their outgoing version for free.
@@deangray4502 Lucky you!! I missed out on that promotion by a few months. 😭
So cool to see you use tape! I have been in that industry since 1991 and still write tools for tape devices. LTO-5 and newer probably means the backup uses LTFS (Linear Tape File System), or at least some format that requires partitioning.
As for the shoe-shine issue from slow data transfer you described, modern LTO drives support speed matching, which means they can adapt within limits to the actual data speed. Tape has come a long way.
That's actually really interesting to know - I have an LTO-4 drive currently and I've considered upgrading just for LTFS alone. I was unaware the system was "efficient" enough to warrant running full backups with it. I thought it might work similar to an open-session CD/DVD-RW, where, yeah, you can throw multiple TOCs into the media, but it's inefficient and not the way the media was intended to be used. Of course, if you're writing a TOC for an entire tape's worth of data during a backup session, there's not much "inefficiency" in that. 😃
Knowing that modern drives can throttle themselves is a huge plus, and ties in extremely well with the "drag and drop" sort of mantra I think of when I picture LTFS. Similar to Plextor's BurnProof, not risking turning your tape into a "coaster" by a buffer underrun seems immensely useful (again, throwing analogies out there to optical media). Do you know what generation (or thereabouts) this feature was adapted?
When LTO was introduced, it was not using LTFS, but TAR format.
I'd also go the tape route, but a couple of things missing in the calculations are the costs of electricity, ups, networking and other stuff required for high availability. Also set-up, maintenance and upkeep is part of the cost. This is why comparing pricing is difficult. This is also never a GB for GB comparison because you are getting different things, like the ability to store the tapes for free, forever without having to keep paying to maintain your data and the ownership, privacy and security.
In this case, this is the third tier backup, and all of the power consumption / networking is handled by the existing second tier backup, so there's minimal additional power / hardware cost to the tape drive. The second tier backup has a running cost, but that's separate from this tier.
I consider my tapes as my "archive" set of data, once I put it on tape its normally put away and stored. Most modern tapes can be rolled about 150-200 times which is plenty of times for access and getting your data back for many years.
I plan on backing up those tapes every few years as well for longevity. Once I started using tapes, I just pretty much archived ALL of my data for the heck of it.
WOW! I was thinking of how to backup my data, and this was very helpful, had not even considered how ISPs throttle uploads lol
It may or may not matter long term depending on how you upload. For example, an incremental backup (like the Proxmox chunk format or zfs send/recv) is only sending changes, so once you've done the first 6 day full backup you aren't sending a massive amount every time, but even then it's not zero and you will probably need to heavily rate limit the upload to keep everyone else on the connection happy.
The good ISPs don't :D But sometimes you can't choose.
This is one of those things where it comes up now and again for homelab people and I don't think it ever really pans out unless you can get a drive cheap or free or if you have so little data that it's fine using old gen drives with little to no capacity.
For ~$10/TB you can buy SATA drives with warranties (even used drives often still have years left on them), and if you're buying Seagate's higher end drives you even get free data recovery. If you go the SATA offsite backup route you get higher density (assuming you aren't buying a $7k LTO-9 drive), faster speeds, random access, and a much more universal interface.
You'll most likely have to buy a second LTO drive in case of a fire/robbery where it's a total loss of your main copies which means there is a "recovery" fee with a backup restore. With SATA you can plug it into any motherboard or get a cheap USB adapter anywhere.
The only real advantage tape has over just buying bare drives is durability and life. However, I will point out that the super long lifespan tape is specifically in controlled, low humidity environments - chances are a home user won't be able to provide that. As long as you're backing up regularly, you'll most likely notice a failing drive before it's too late. Plus with all the money you saved on buying a tape drive you can just grab another ~$170 18TB drive to mirror your backups to. :D
I got used SAS drives for the backup server even cheaper (around $7/TB I think, partially because they are SAS and not SATA), with less than 3 years of power on time and ~75 cycles, so they are in better shape than any of the drives in my TrueNAS system that I bought new. They are definitely a good solution for backup, and I'm very happy with that path for the PBS server. SAS isn't common enough to plug into a motherboard, but it's also not hard or expensive to get a SAS HBA to recover the data.
For me at least, this route means not having to build another system for the backups or get a disk shelf / enclosure for the backup backup drives, and I'm happy enough with it.
Yeah, for me, I just basically pick up cheap SATA drives (say $10/TB) -- like @Peter says above, and it's quite fast and easy for backups. And, I split stuff so that I never will go over the size of the drive, which makes it much easier. Like @Aparlrd said above, he has got them for $7/TB. Haven't got drives that cheap as him recently, but I have bought some 4 TB drives at $10/TB at $40 bucks can't be beat. So, it makes backups fast and easy for me, since I don't have huge storage requirements at the moment.
Not always true about longevity, I've seen tapes sitting inside closets since the 1980s, still working and basically all of the data was recoverable.
Cheap drives are exactly that, cheap drives. I would never trust important data on them, especially if you are using them more than 3-5 times a week.
I like tapes because with modern technology, they last way longer than HDD/SDD drives and for archiving data, they just work so much better. Sometimes the more expensive option is the better option.
Awesome video. Love everything that you do, @Apalrd. You've got the exact content that I like watching and learning about. And, these are questions I've asked too -- about Tape Backups.
Backblaze is changing their pricing on 3rd of October: Storage pricing increases by 20%, but you will get free egress for three times your stored data volume. I guess this will change your calculations a bit, though it also highlights the risk of price changes if going for a cloud solution (I think this is the first price increase BB has had since I became a customer many years ago though).
Also, when looking at secondary backups I would look for options to make backed-up objects immutable for a set time - it could mean higher cost depending on your rate of change, but does provide an additional layer of protection against ransomware.
That was exciting to discover, thank you for sharing :)
Love LTO! Thank you for sharing :)
Thanks for the video on your LTO legwork. I always thought it was neat to use a tape in 2023, but didn't have a dataset large enough to make the buy. Cool seeing it in action.
Timely! I have a tape library and I know PBS supports it but I haven't set that up yet. It has LTO-5 drives but I recently managed to snag an LTO-6 drive for $200(!) so I think it's time to get PBS up and running!
You are putting the best homelab content out there. Congrats!
Thanks!
The internet throttling is definitely something to consider depending on how you are backing up. uploading just the deltas wouldn't be much of a big deal but getting that first initial set uploaded to the cloud could be a real problem. I've got friends that have 1gbt down but only get a few hundred mbt, if that, upload. I'd not previously considered a tape drive but I may need to. Thankfully I get 2.5gbt both ways but... the low cost is appealing.
I get 500/25 from Comcast, their gigabit plans only get around 30-50 up and that’s the only provider I have and gigabit is their highest plan here.
@@apalrdsadventures yeikes that would take FOREVER.
@@apalrdsadventures
Comcast sucks.
Had it when I was going to school in Michigan.
Now I have a local fiber ISP (in Canada) where their fastest tier is 1 Gbps down/750 Mbps up.
(But I'm on 250 Mbps down/250 Mbps up.)
Nice Demo! I have a couple of LTO5 drives that are FCAL over FO. I missed out on grabbing an old 16-port FCAL switch from work, so I have to have one FCAL card per drive.
I am a repair technician that services quantum tape devices and its always interesting to see how others approach this technology as before I had this job I had never seen this type of technology. I mainly service the robots in the quantum scalar i6000 libraries, but I've gotten to see a great deal of their enterprise storage devices. It has been interesting to see what type of industries use this technology for immense data backups we don't ordinarily think about .
Great introduction for those homelabers who never had a tape drive (or maybe never even heard of them)! And yes, it's shockingly plug&play. We're using PBS with a (small-ish) tape changer library at work, and it's just as plug and play. Just select the device for the changer that shows up and it just works. Including the integrated barcode scanner to identify tapes.
Only correction I have for the video: you don't need to go to the command line to format new tapes. It's done from the interface of your drive (so you click on "quantum" under Tape Backup). If a tape is in the drive, you can click to label, catalog or format it (or clean it if it's a cleaning cartridge). If the tape has a barcode and the drive (or tape changer) has a barcode reader, it'll auto-fill the barcode it read as the label, too. This is one of the very few areas of proxmox where you actually don't need the command line (shocking, I know!).
Since my drive doesn't have a barcode reader, it wasn't auto-filling the barcode and it would give me an error that the tape already had a label that didn't match what I typed. Hence the format.
@@apalrdsadventures I meant that there should also be a format button in the GUI (roughly next to the label button), doing the same thing as the console command. But no worries.
Tape drives belong to the same category as printers - mechanical = problems. I spend too much time in my youth fighting DDS3, DDS4 and LTE tapedrives. LTE was great, but far from perfect.
I use borg to a local repo which I then rsync to hetzner (storagebox) which allow me to mount the repo via smb for restore if I need to - without having to pull the whole repo.
I pay approx 12$ a month for 5TB - which is approx. 30 months with your calculation.
I've got an LTO4 tape drive (HP Ultrium) SAS in my proxmox tape backup server and it's working great!
Glad it's working well for you!
That is WAYYYY easier to deploy than I thought it would be.
I don't have a second tier backup server like you do, so I might have to spin up Proxmox Backup Server as a VM from my main server, pass the SAS card to it, and then run my backups this way.
Worth a shot, I think.
Do you know if the Proxmox Backup Server tape backup system does any kind of indexing of the contents that has been backed up?
(i.e. if you want to pull a single file or select files from the backup, will you be able to do that?)
Would love your insights in regards to this.
Thank you.
Great video!
Tapes are good. Used to use them back in 2007 at the NHS. We had a robot at one site and could watch through the window it picking the tapes up. The other site didn't have one, just the drive. Tapes were in a different fire zone in a fire proof safe. They never gave us a tray to carry them or anything and there were lots. At the main site, they'd get regularly dropped as I'd walk the long corridor with a stack of them. They still worked just fine though after being dropped several times.
Mar 2024: Thanks for all this info during my research. Based on drive costs these days, I've opted for BD-R (DL and XL) discs for now, for family data backups at multiple sites for storage.
I have a Lto 4 Tape Library 24 Tapes, bought it for scap and restored it. Works awesome
What's the total storage capacity?
I think a TS3100 goes up to LTO7, and I have seen some chassis/library going used for around 250-300, you'd just have to buy a drive for it, ..but iirc there's 3 different types of drives, sas, scsi or fiber channel, and sas is likely the most desired and so likely the most expensive too. The advantage here is that you wouldn't have to swap tapes manually so much.
One of my projects that's still pending due to my own lazyness is a off-site backup to a Hetzner storage box.
Hetzner is a pretty popular option here in Germany where you pay a flat 50€ for 20TB, no other costs involved.
But you know, transferring data over the atlantic probably isn't a fun experience.
Yeah, they don't offer most of their services in the US (since they don't own DCs here, they just colo for cloud services) so I'd be going to DE or FI for a storage box. Not really a problem for backups though, aside from my ISP being awful with upload speed.
As someone who works with LTO drives/tapes for dozens of clients, I approve - but with significant provisos and warnings. When a drive or tape fails, it fails hard. Stuck tapes, drive mechanical failures, tape degradation are all common occurrences. Because of that, you really need to pay attention to your backup statuses and LTO drive diagnostics.
When it works, it works great. When it fails, it can be catastrophic.
EDIT: Things are moving away from LTO to RDX. UNFORTUNATELY prices for RDX per GB is currently still very high, so not quite home-user ready. But it is something to keep an eye out for.
EDIT 2: Look at getting a local off-site location. Many of my clients use a safe deposit box at their local banks.
You realize RDX is just an overpriced standard shitty 2.5" laptop HDD in a fancy plastic casing. If your dead set on HDDs consider a 2.5" Icydock dock system available as a 5.25" or 3.5" native bay or USB 3 external dock. You can buy the sleds individually for whatever more reasonably priced source of 2.5" HDD you want.
No you don't get the same plastic/rubberized outer casing but if you plan on sticking them straight into a foam "pelican" case then I think the effect is close enough.
RDX is a cruel joke for ignorant customers. Also from what I hear the industry is moving back to tape due to HDD sizing potentially beginning to stagnate alongside tape capacities moving to 50TB per cartridge.
Tape is great for a secondary backup and for archive. I am fighting with my work currently to try to get a tape library setup.
Great video, thanks. Oddly enough I was just going though the same thought process but came to the conclusion that USB hard drives were a better choice. Tape is appealing but the price to performance doesn't feel like it's right for me.
I actually started back in ~2013 with 2TB USB3 drives, my original XBMC setup used two of them with two Raspberry Pi's. I still keep those two USB3 drives to archive my media collection, and they still appear work fine, although I can't confirm the data is still good since they are both ext4.
Aaaaand, the very first time you "forget" to off-site your tapes, compared to a predictable RPO for Object, every bit of that math cancels itself out. Compared to S3 on Wasabi, Backblaze, Linode...
In business, we'd also do the math on human hours for those management and maintenance tasks.
Great justification for a new piece of homelab kit though ^_^
Tape is nice and all, but I’d argue that if you have a good Internet connection, you only restore the full data set in exceptional circumstances and you preprocess your data, Glacier DA is the cheapest method for
LTO is definitely a good move - very solid
You can probably upgrade the LTO drive to newer LTO versions. They have a standard 5.25" LTO drive inside
Thanks!
Glad it's helpful to you!
Wow I can't believe there are still people working on this technology like me.
I have a HP MSL2024 library and LTO5 + LTO6 SAS tape.
Tape backups are great. BEWARE: If the data written on tape uses a special format and you lose the software that wrote it, your data is essentially gone! Make sure you keep your old archive tapes up to date with recent software to ensure a hardware or OS failure doesn't destroy your archival strategy. Been there when the software was upgraded and can no longer read the old format and had to dig up an old machine and software to unload the archive tapes to the reload with the new software.
Constant rotation fixes this problem, unless the drive itself is rare!
Oh boy. Thought I was going to catch him up on the 3-2-1 rule of backups. (not today) 😂
Man you seem to be a prophet! I was mulling over the possibility of organizing my data backup around LTO and are very interested into LTO5 tapes!
That wasn't the first time you are featuring some topic I am currently mulling over implementing in my home lab!
I had an old 8mm tape library that was the old 50-pin parallel SCSI. The tape changer was cool to watch but I never really got it working well and the tapes didn't hold much.
Thank you for sharing your thoughts and the cost comparison!
Do you have a strategy in place to check data integrity for the different data locations? Would be interesting topic for a video maybe?
This is very differrent depending on used filesystem and maybe filetypes?
I had the situation that random files in NTFS-Filesystem were unreadable to filesystem. This was only discovered by me on doing a full backup and I had to restore them individually manually 😅
I also wouldn't know how to check if a file is intact even when the filesystem can access it (ie bit wise errors in Video Files). As far as I know this can only be done on filesystems with checksums enabled. So how do you check data integrity regularly on the common Desktop-Filesystems (Win/MacOs/Linux)? And how so on servers?
All of the Linux systems use zfs and do monthly scrubs. This includes all of the Proxmox systems (including all of their boot drives), the cluster (except for Ceph, but that also does scrubs), TrueNAS, and Proxmox Backup Server. ZFS on desktop systems is also a thing, to the level of having a unique zfs dataset per home directory.
My Mac has APFS which has some of the features of ZFS but not all, and Apple also does their own thing at the block level with ECC in the flash controller that they believe to be as robust as the zfs block checksum.
The Windows system is out of luck, but most of the space is Steam games anyway. Other than the HUGE hassle of re-installing and setting defaults for software I don't think I would suffer any data loss if this were to fail.
I also generally don't keep important data outside of the Linux systems, this is why I edit off my NAS and directly work on personal files off the NAS as well, instead of keeping them locally and copying back/forth.
@@apalrdsadventures thanks on the insight!
In your scenario, It would be a simpler process to just use a couple of external drives. You can get a 10tb red nas pro drive for ~$250.00. external enclosures are really inexpensive. As you mentioned the size is pretty similar.
That really is the best way. I keep a "FIRE" drive that is an external 14TB, I backup all important things like pictures, PDFs, etc, incase I have to evacuate (California fires), so I can just grab it and go. Also backup those things to a friend's server overnight using restic.
Modern hard drives are unreliable, and nobody knows if they even work in a couple of years just sitting there. Tape is extremely reliable, very good stuff for a second backup
@@hhkk6155 Yeah, no. they don't just sit there you rotate them like tapes. tapes are super slow and become unusable once you have enough to back up that it can't be done in time. there are levels of quality in drives. some are shit some are not. a quality drive will last plenty long enough before it is time to be replaced. Also, if your actually serious about the "second" backup, it should be cloud. not tape or hard drive. but for me, if cloud was somehow not an option, its drive over tape all day.
@@netrospect6525 with modern drives you don't know if they would fail tomorrow or in 10 years. For a local and fast backup they are good (especially in raid 1). But for the long term cold storage tape is better.
Tapes have linear speeds around that of a HDD, or even faster (in modern tapes, lto-8+).
Old Lto-5/6 is cheap, fast enough, and good capacity for home/enthusiast use. Depending on the work you can fill a 1.5 tb tape in a week to a month, get 2 copies and ship one off-site
@@hhkk6155Great point! Though if you have a decent NAS, with either full disk parity which lets you mix and match drive sizes, or RAID 5, an offsite backup (and a local one), you're golden. I think I may pickup an LTO5 or 6 for the important things like my scanned docs in PDFs, and pics.
I remember learning about the 321 rule when I first started IT. So now I keep one of my backups in my car.
Somewhat compelling and I've considered this avenue myself but $30 for a mere 1.5 TB doesn't scale very long. It's really the hurdle of the drive purchase for the newer LTO specs that have me holding back on this whim a lot. That and I'd probably have to figure out how to use Bacula or something since I don't really virtualize at home. I kind of wish ZFS send had a tape friendly way to manage things (yes, you can send the incremental stream to files but you are undermining what something like Bacula is trying to do for granularity). The Sun Storage thumper devices had some supposed backup specification (NDMP) but I don't think it ever truly gained traction and I'm not even sure that it's open.
$50 for 18TB (pretty much all tapes are roughly the same price new) does scale though. And it's possible to buy tapes used like hard drives, most businesses using tapes will retire them after a fixed number of cycles and you can use them for archival without worrying.
If you are sending a non-incremental complete abckup, you can zfs send directly to the tape block device. zfs send isn't designed to be written to storage since it has no error correction, but it does have error detection, and will abort the read on error.
tar is also an option still, as is LTFS.
@@apalrdsadventures Yes, sort of. It's cheaper than hard drives, but so is buying thousands of standard sized BD-Rs (obviously tape is superior here but you get the analogy). That's a lot of buckets when you have like 40TB. 27 tapes * $30 is ~$810. A 20 TB hard drive is maybe what, just sub $300? In terms of raw capacity, that's $600 for the cold storage, albeit with all the caveats of a normal disk.
Edit: err wait, you meant for the newer LTO specs. Yeah, I can see that, but the drives do still cost a fortune and are unlikely to ever go down until maybe a decade after they are ubiquitous.
Yes, I've considered zfs incremental sends directly to the tape device without rewinds and it _might_ be a viable option but I guess there's still some bookkeeping that has to be done on the user end to know where to seek on the tape.
Certainly LTFS is an option as well if you have LTO tapes new enough. Even though the access pattern would be linear, I do wonder how abysmal that might perform. I'm a little bit surprised that outside of the vendor-locked enterprise space from Sun and now Oracle, we aren't seeing solutions popping up for ZFS replication that are a bit more tape-friendly.
The problem with LTO 5 is that it's a little too old and so the price per TB will not be very attractive same for LTO 6 and LTO 7, i think the sweet spot is around LTO 8 since you can also use M8 LTO 7 formatted cartridges that will give you 9TB/tape and 12TB for lto 8 and if you can might as well go with LTO 9 and get 18TB/tape compared to the 1.5 and 2.5TB of LTO 5/6 you can see the difference, also I personally like using LTFS and install the binaries on windows you can double click files on explorer say video files and play videos off the tape like an external hard drive just takes a little time to cue the file but once it's found you're set (anything from LTO 5 or newer can do that) :)
I also always use 2 copies per set of data, ideally also using different tape brands and having them at different locations for archive, in case something is wrong with one copy there's always the second copy elsewhere.
Nice move.
I running a neos T24 with a single LTO-9 drive loaded with lto-8 tapes
if memory serves, the most impaortant thing to keep in mind with tapes is that while they have a long shelf life they have a very short write endurance so after a few hundred full rewrites they are not reliable anymore. I don't know if this changed with more modern tapes bt it's something that should be looked at
Yeah, doing monthly backups and swapping between two sets of tapes I don't think I'd ever get to 100 rewrites on a single tape. More frequently maybe, but then when a tape gets to 50 rewrites I could move it to the archive vault to retire.
@@apalrdsadventures What about streamer reliability?
Surely anything involving magnetic heads and drive belts/o-rings is still probably a mechanical nightmare to maintain? Especially when you are using already outdated (not in production and probably not supported in terms of parts) LTO standard.
The 2 medium rule only implies it is stored on 2 different systems. If you have 2 copies on the same system and same pool it is just one medium. 2 servers with HDDs would still be to mediums. I'm not sure if production data could be called a backup copy. If you do ZFS snapshots than it could be different
snapshots (and raid) are not backups, and production data is a "copy" (if you have any backups that is).
So having three copies means having the production data + 1 backup on another system and one offsite backup
@@marcogenovesi8570 to a degree are ZFS snapshots a backup, they can be copied to a different server and can be used to "restore" the system to the time of the snapshot. It's not a backup in the traditional sense but more than VMware snapshots for example. The copy on write nature of ZFS allows you do that. You should still keep a copy because storing them on the same system could make them useless anyway
I'd consider a snapshot to be the first backup, so the backup server is the second and the tape is the third. First backup deals with user error, second deals with single machine level failure, third deals with datacenter level failure.
@@JohnSmith-yz7uh snapshots of any type (NTFS has snapshots too btw, Shadow Copies) are not backups. To be a backup it has to be independent and a snapshot is clearly not independent.
Yes it can be useful to restore data in some cases but so does a versioning system (like git) or just making copies of files in the same drives. That's NOT a backup.
I do appreciate you showing us your tape setup. I wish I could afford a LTO 7 Type M tape setup (9TB per tape, ~$4.8 per TB before drive costs).
At the $30 (assuming after tax) you paid for 1.5TB tape that's $20/TB, before the cost of your tape drive (and it's expected life expectancy/data volume over that time). Ideally you want your tape cost cheap enough to have two copies of each tape, despite the rest of your backup system having copies.
How many TB do you add per year (50GB/month x 12 = 600GB/year, ah why bother with tape), how many years will the tape drive last? Tape migration plan? Tape storage environmental controls? I don't know if a safety deposit box has humidity control for tape storage. I don't think I'd send any tapes via the postal service, do you want your tapes to spend 8 hours in the back of an aluminum box getting cooked at 110 degrees (spontaneously catching fire as the LLV trucks have been known to do)? Is your host system well spec'd / powerful enough to feed the tape drive with data fast enough? (another commenter mentioned 'shoe shinning')
$20/TB is more than current HDD pricing (and hard drives might be good for 3-5 years at minimum) (albeit it you're getting some things with tape that you desire, like offline storage that's less mechanically complex; if a hard drive breaks you've lost 10-20TB, if a tape drive breaks, data is just temporarily inaccessible not lost, if a tape itself breaks, hopefully it was cheap enough to have two and/or it's not your only copy).
I'm currently getting $15-18 per TB on HDD. I need cost per TB on tape (incl cost of drive and host system over it's life expectancy) to be far lower than disk, not roughly equal. With my data volumes, over maybe a 6 year window, I could perhaps see my costs on LTO 7 Type M as low as $10-12/TB, but that depends on finding a reasonable second hand drive, and tape production and prices holding out. Hard drives are at like $15-18/TB; my data volumes would increase with a tape setup as I could then afford, but still, the economics of tape for small deployments are challenging. It's difficult to say if this make sense, despite how "cool" having a tape setup is for us geeks.
AWS Glacier pricing is great so long as you don't need to recover anything (or only very small critical things and your file structure is designed accordingly). It's like data insurance. I definitely feel $12/TB/year is wonderful for data you CAN NOT afford to lose, and would pay ANYTHING to recover (so legal compliance data, last extant copy of anything). Even for data you generally might not be able to ever afford to recover, but still can't afford to lose (I have it, I can't get it, but it's not lost, so still safe). AWS bandwidth is the other annoyance. You can always run your data through a transformation workflow on your own host systems before pushing it to AWS if the 4MB size you're currently working with is a problem (don't pay for millions of 4MB requests, batch that up into some other format). Really, you might also want to afford yourself a way to delete older data in the future (if only for economic reasons), so perhaps the proxmox backup format isn't ideal for the cloud model anyway.
Backblaze Personal is useless/unreliable. Backblaze B2 ($60/TB/year) is a different model to Glacier cold storage, more online/nearline. OVH also has a Glacier like archive product, think it's like $27/TB/year. Wasabi is somewhere between these lot.
If you need multiple tiers of data storage, making one tier "mostly inaccessible but extant due to economics" for $12/TB/year isn't a bad thing. If you activity maintain the health of your local disks, then having AWS manage your "tape-style" tier for less than you can manage on your own (in terms of per-TB cost over lifespan) is probably not a bad thing.
All storage (cloud or local) has a life expectancy and has to be looked at over some time window. Your AWS cost calculator examples seemed to assume your drive lasts forever, or that tapes or disks don't need to be rotated after some window of time, be it 3 or 5 or 10 years.
You paid 250, or was it 300, you've said both, for the used tape drive/cleaning cartridge. You said you add 50GB/month, 600GB/year. Lets play a little bigger, lets say you add 2TB/year. If your tape drive lasts you 6 years... $300 tape drive / 6 years / 2TB data growth each year = $25/TB/year. Plus the cost of tape media, you said $30 for 1.5TB, so $20/TB just for tape, divide by 6 year life = $3.33/TB/year tape cost.
$25/TB/year drive cost + $3.33/TB tape cost = $28.33/TB/year over a six year window. At that cost, if you don't experience a data loss incident, AWS's $12/TB/year (before operations and bandwidth, inbound is free) is cheaper as the lowest tier compared to your homebrew solution. Your $28.33/TB/year is cheaper than B2 assuming no recovery, and comparable to OVH's ~$27-30/TB/year option.
Now, if your used tape drive lasts more than my guess of six years and your data volume is higher, the math might start to become even more interesting. $300 drive / 10 years of life of mechanical wear and tear / 12TB per year = $2.5/TB/year drive cost plus tape costs $3.33/TB/year = $5.83/TB/year, now you're getting somewhere. Double tapes for redundancy (cause one day your tape drive will eat a tape), $11.66/TB/year = roughly same cost as AWS Glacier's $11.88/TB/year, assuming no data loss incident.
I so want tape to be the right answer, but for 50GB/month, I'm not 100% seeing it. If your data needs grow a little, then I suppose for $300 and the same tape cost as disk, why not toy around and get into it as a technical project, as you've done. Of course, if an $18/TB hard drive will last me 6 years, that's $3/TB/year, two copies on two drives would be $6TB/year, and I'm not at the mercy of a used tape drive failing, and hard drive prices are mostly falling, while tape media access should stay good but could become difficult for older generation media.
Note, $30/tape is high, you should be able to get LTO-5 Tapes for closer to $18-22 each, so $20 / 1.5TB = $13.33/TB, over a 6 year window, about $2.22/TB/year. $300 drive/6 years/2tb year = $25/TB/year + $2.22/TB/year = $27.22/TB/year, I feel like that's the real cost of this setup for your current data scale. More than Glacier, cheaper to recover, comparable to OVH or maybe Wasabi. Honestly, this is also in the range of data that you could manage to shove onto an "unlimited" cloud service (while they last) for quite a cheap monthly rate.
I do still like what you've done here. I might copy you still. If I found a tape drive for $300, and it lasts 7 years, for my current data volume of 48TB/year, with $20 LTO-5 tapes, I could see costs as low as $2.79 (single tape) to $5.95 (double tape) per TB per year, but that's just putting me mostly on par with disk prices of $3 to $6/TB/year, and then there's the management overhead of tape. Of course, no reason the tapes only have to last 6-7 years, but everything does have a lifespan.
I think, currently, you need to hit 100 or 200+ TB/year data ingest for tape to really be worth the effort. If my needs doubled or quadrupled, it'd be worth it for me.
Well, anyway, here's to your new journey into the world of LTO. Keep us updated.
All you said is right obviously ;-), so I'll be short. You still don't cover the main point. The usefulness of LTO medias is ALSO in a solution for long-term ARCHIVING. This, in context of large files creation like post-production etc. For me, starting from LTO-5 with LTFS, the media is well suited for simple ARCHIVING of valuable post-prod assets. As of today, in this context and with used units, it seams we shouldn't go lower then LTO-6.
@@Patrick_AUBRY nope, I did understand it was for "archiving", that's why I said, "albeit it you're getting some things with tape that you desire, like offline storage that's less mechanically complex." 'offline storage' = archive ...But "ARCHIVING" (or in lowercase, "archiving") isn't some MAGIC WORD, you still make the same considerations: cost, media and total system lifespan, maintenance/media rotation, transport/storage, etc. You can archive on any media you like. You'd want to rotate tape (or hard drives or even blue rays) after some limited number of years for evolving system compatibility (business process and also LTO generation) and data integrity verification reasons, despite that you could, if you wanted to, keep tape in some old salt mine with minimal environmental controls for decades on end, again, if you really wanted to. Archiving isn't a magic word, and LTO is a media format to consider just like anything else. If you've got the data volumes for it, I think tape can make sense, but disks are catching up, with different tradeoffs for wear and tear, data size, life expectancy, overall cost. Media production is a great example where data footprints are ballooning and tape can still be the winner, especially as more UHD production emerges. Cheers!
I so badly wanted to TLDR, but holy shit was this a well-written and articulate comment. A couple things I might add are:
- The strong chance that his decision to adopt tape was "logically" conclusive based on a deep, inner-nerd compulsion, especially for those who have never used it before.
- You covered Blu-Ray but left out many key points that make it extremely enticing for the explicit purpose of "archiving" this amount of data (as also briefly mentioned in your reply to Patrick_AUBRY): 1.) Drives are ubiquitous, and they probably will be for the foreseeable future. They are an inexpensive commodity and there is no chance of needing to find or replace a "costly" drive to retrieve or store your data, should the need occur down the line. 2.) You mention wear and tear and life expectancy, but M-DISC has an expected lifespan of 1000 years. There will never be a need to refresh this archive. 3.) The only major downside is the physical interaction necessary to create the archive, but after an initial full backup has been made, incrementals will fit flawlessly into his
@@koozmusic Thanks for reading, otherwise I'm just shouting into the void. I also have that inner tech-geek compulsion for tape, so I won't knock him for the economics not being perfect :)
(and also, the idea of making a youtube video might also have been part of his desire to do it, was still a good project video for others to watch)
Yeah, I'm also interested in the physical properties and possible longevity of Blu-Ray as an archival option but the low data volumes (unless you use some kind of cartridge-type thing) and costs of discs weren't looking compelling enough to me, just personally; I want to be shifting TBs around not managing GBs. It is very true though, with his current 50GB target size, it does fit well to optical media capacities. That would really be a good fit for him at this moment.
I think every archive deserves auditing/performing integrity checks periodically, including disc formats. It is still sensible to re-process the archive, or at least a rotating subset of it, every so often as time wears on, even if the physical media doesn't strictly require it. Maintaining a familiarity with the recovery processes is important, as well as continually evaluating what parts of the overall business workflow/tooling/software do and don't survive different scenarios/incidents.
In some respects I wonder if magnetic tape is a less complex technology (over the years) than optical disc; just thinking about if the format (drive hardware) magically disappeared and you had to reconstruct the technology primitives to decode the data after some decades. It's not the most pressing of concerns, but I'm just always prone to thinking, if we had to recover this from scratch, how would we. Sidenote: An interesting article where X-Ray was used to recover images from, basically, a big pile of melted film stock of a historic TV programme: www.bbc.co.uk/rd/blog/2017-12-morecambe-wise-video-xray-microtomography
Anyhow, cheers!
Haha, you are my kinda guy! Jesus. It's like we were cut from the same cloth. Lol, right down to the idea that some day, we'd have intelligent lifeforms decrypting our ancient glyphs, which were crudely polarized into this carbon matter-based data store. 🤣
Anyway, I concur on all points, but would like to add that when it comes to periodic integrity checking, optical media does not suffer from the wear and tear aspect of this endeavor. Short of human mishandling (which is unavoidable until we've been 100% wiped out by our computer overlords), they are immune to any common effects from use, unlike any other physical media, that I'm aware of. 🤔
edit: also, the inner tech-geek compulsion for tape... I'm pretty sure we _all_ have it... 😧 It spreads RAPIDLY to our kind.
another edit: If they're smart enough to decode our binary data, I'm pretty sure they'll be able to grasp the difference between a spiral and a roll. 😁@@christianherald
I’d argue that if you have a good Internet connection, you only restore the full data set in exceptional circumstances and you preprocess your data, Glacier DA is the cheapest method for
I recently bought two mislabled ebay quantum tape libraries. Advertised as LTO-3 they were LTO-6 got them for $150 Australian dollars each and hold 16 tapes each. I can get LTO-6 tapes for $35(AU) each ( but got a box from the US for $20(AU) a tape) backing up my 50TB truenas box. Only have 10TB on it currently. Seem sto work great for now. Cheapest way to have a backup of my NAS I think.
Holy shit, what a score! 😮
@koozmusic just keep and eye out. Some ewaste recycling companies don't know much about tape stuff.
Great, just got bacula running only to find your video and seeing how easy it would have been with Proxmox Backup... well start over again I guess :D
Depends on your workload, PBS is more focused on backing up Proxmox clusters but it can do Linux hosts and Linux-hosted datasets as well.
@@apalrdsadventuresIt looks much easier to manage than bacula or amanda I tried to get running. I just want to manually backup from a bunch of NFS mounts.
Currently trying out PBS in an LXC Container on my Proxmox machine. I like how easy it makes managing the tape drive, but it looks like it’s not designed to di what I want, backing up arbitrary data.
I’d have to replace my backup NAS with PBS and set up a Linux host to backup my TN Core data to PBS.
One thing to be aware of with PBS is that tape backups must come from PBS backups, not directly from clients.
@@apalrdsadventuresThat's the key point. I guess for the time being I'll revert back to get things up with Bacula. But I found your Video about backing up TN datasets to PBS. Gave me a good idea how I could replace my backup NAS with PBS long term. Your channel is a wealth of great information!
Why not blue ray? For a single terabyte set it seems peefect
Getting an HP 8 tape library and refurbished LTO-6/7 drive is best way for 2023 if you have more data than 2 tapes can fit.
LTO-6 for the library are kinda same as external, but are more abundant and thus cheaper.
I personally really don't like backup that requires essentially same space as backup itself, and can't work directly off the ZFS snapshot =\
Backblaze and Amazon S3 are bad choices. Wasabi is $6 a month per terabyte. iDrive E2 is $20 per TB per year, which is $1.66 per month give or take. I get 60 MB/s on gigabit backing up to Wasabi or iDrive E2, so I don't quite need a tape drive. Granted, I also have a Synology I backup my primary NAS to and it's 125 miles away. I can just go grab it if I need it. The primary drawback of using cloud storage for enterprise is the seeding and pulling of data - Pulling a 10 TB SQL server down over gigabit is still pretty damn slow.
your obsession is more advanced than mine. but I tend to just dump money into internal 3.5 rust hard drives
awesome
Nice video. I still do not get what wrong with HDDs? I for example have a huge dataset of about 8TB to backup. What's wrong in storing that on a single 10TB HDD? Those are not that expensive compared to the 5 tapes I would have to buy, not to talk about the speed you mentioned. You should be able to make use of a VTL like QUADstor. If your backup server still has place and supports hot swapping (which mine does) it feels more convenient.
¿did you update the firmware on that quantum drive?, since it's so old it should have several fw revisions ahead
What are your thoughts on practicality of buying hard drives for cold storage with prices falling? They seem to have a decently long cold shelf life.
The lubricant will seep out of bearings and seals will leak even if they are not on (although it happens more slowly). A 10 Year old car is not new because it has 0 Miles on it and an HDD is rated for ~5 Years of operation. They are also designed to be turned on somewhat regularly so after a few years they might simply not be able to start up because the lubricant should have been spread by movement instead of pooling in one place. Even assuming thrice that expected period (15 years instead of 5) is half a tapes rated shelf life of 30 years
@@Momi_V in the video a much shorter working life was discussed for tapes. I was wondering if a hard drive powered on infrequently would be a more cost effective option for tens of TB over 5-8 ish years
@@thyme676 That might be the case. The point of current gen LTO is more about archiving 100s of TB for decades.
Backblaze is $7 a month for unlimited backup storage on a Personal Account and even the option of having your backups sent to you on physical medium like a USB key. It is what I use
Did you also include the travel costs to keep taking your tapes off site each day? :)
Which HDs are reliable Seagate or WD Red? How much storage per HD do you have? Also, isn't M Disc better? In addition, how do you prevent bit rot on your archrivals?
[Forgive me ... I think you covered this, but I wasn't expecting you to cover as much of this -- in such great detail. Very good job. You can ignore this ... Yes, you covered it all ...]
One thing, though, I wish you would talk about how much that tape drive costs. And, where you got it -- if you got it cheap. I looked into tape, and it seemed like it was outrageous few years back. I only wish they would drop to something like $300 or $400. Then, I'd use tape a lot for backups. [Sorry, okay, you start talking about it at 3:46. Forgive me -- only watched first few minutes before commenting ... Okay, I'll get back to your video. Sorry for the interruption.] Damn, $250 for tape drive ... where'd you get it? And, I hope you show us how to hook it up and use it.
i think a orange pi5 with 2.5gbe(usb) and a couple drive in raid1 would be preferable - put the backups in a safe out back in a shed - the tape is ok i guess but using same media makes things simpler, faster, cheaper, and more convenient - pls do a report on your new pc and upgrades/addons plus expansion plans - zen5 should be a good one and you have a good upgrade path - pls try to add 40g to dual nas (dual port 40g cards are 35 bucks) - this would be the best and fastest way to process backups imo bring time to do backups down significantly - 40g plus nvme raid0 - great for businesses who can emancipate themselves from the cloud and gain an order of efficiency - the isp is weak link followed by internal network deficiencies for the vast majority of smb
So currently the backup server has 4x10T SAS drives which together can't saturate 10G, so I can't do a backup faster than that. But, even more of a problem, the Microserver only has one PCIe slot and I'm using it for the HBA (the on-board HBA is SATA and I need SAS), so at best I have 2xUSB3 + 2xGbE for connectivity to that box. But it's not a problem since backups run daily or twice daily and are automatic and I don't care as long as they are all happening and finishing within their time window without using too many resources on the hypervisor side.
I'm also running into bottlenecks in the ingest process with the speed of my SD cards and card reader, so I can't upload more than about 70-80MB/s (even though I can do transfers NASPC's NVMe drive at 200+MB/s. So going to 40G wouldn't be a great advantage there either. Editing is really the only part of the workflow left that's fairly network-heavy, and it's currently not awful at 1080p60.
What would be an advantage is moving to Ceph so the distributed workloads can all share bandwidth better without me manually adding a bunch of point to point optimizations here and there (like the 3x links to the backup server), and I'm already starting to buy dual port 25G cards for that. I'm preferring 25G over 40G since it's a more modern standard and the newer used equipment and new Mikrotik stuff is going to 25G, plus it's easier to work with over fiber with 1x25G instead of 4x10G.
@@apalrdsadventures yep yep - eventually you will end up with bigger and better on 25g and then you could always goto 25g bonded to match up a bit better with nvme arrays - i think the key point is you want to do backups and have some offsite but closer to home you want some redundancy local so you never have to go grab those offsite backups which is why i harp on dual nas - it is just a smart thing to do and you can also backups the nas' to a usb hd as well if your backups are less than say 10tb. once you do decide to goto 25/40g there is a great upgrade path to 100g and 200g bonded - i think cots geat is slowly creeping to pint where this is actually a realistic option - the network is the weak link and when you have better fabric it really opens up path to better clusters and netfs - with advent of zen5 soon prices should drop even more for cpu like 5600g and 5700g which make for pretty decent nodes - low price, decent performance - am4 will get cheaper soon with the new parts - yeah it would be nice to build on am5 and zen5 or used epyc but not worth it for the money right now for nodes anyways - for ws yeah - am5 will be good for nodes in a couple of years when things start to move to pci-e v6
I think you should goto 40gbe cars even if they are older - this could be an advantage since they are faster and older which means cheaper - the tech is basically the same - you can convert mellanox cards to ethernet and use cheaper dac cables while still having option for fiber. check this one out too - This $250 Ryzen Pre-Built is a BEAST Home Server!
Wolfgang's Channel
maybe you can get (make) this cheaper for a nice cluster node and be able to have the cpu and pci to push more data vs older machines - i think networking is basically your weakest link - with faster networking you can have a better performing cluster and less overhead with not too much expense plus less cluster overhead and you can go with bonded connections on some machines or eventually upgrade to 100g at some point and then 200g bonded - you will gain better scalability with your nodes but to go beyond a 3 node cluster you probably will need a 500 dollar switch maybe just go with 2 3 node clusters and on your ws have 2.5 for management network and to connect to both clusters - maybe you can do content on a good siem machine - selks works - you do want 24/7 pkt cap
You can get hard drives for 12 dollars per TB these days. I love tape but it costs too much except for floor space density and really, really cold storage.
If I were packing a time capsule to be opened in 2 years I’d use hard drives, if for 5+ years I’d us tape. If I wanted extremely dense automated storage to be occasionally accessed for a complete database/dataset retrieval I’d use the most modern or 2nd most modern LTO tape.
I use two networked old low spec Desktop PC's that I only power on when I want to backup/restore, with 4TB SSD's in them. I could just use one old low spec PC with multiple backup drives in it, but I happen to have a number of machines in my network, so why not spread the backup process across them? Offsite storage is achieved by removal of an HD, take it off site and replace it with another. SSD's are more expensive than LTO5 tapes. However, I didn't need to buy a tape drive. SSD's are much faster, but I don't actually realise that benefit as my backup network is only 1Gb.
I would say there is 2 more advantages LTO has over ssd :
- backup softwares are aware of how tapes should be used/managed
- there is no electronics (except for a small nfc like tag) or complex mecanical parts in the tape, so lower risk of data loss over time
On the speed subject, lto5 drives has a steady 140MB/s raw which is already quite good when your usual hot backup storage is hdd backed (you tend to write a lot, ssd aren't that happy about it), but lto drives also come with an hw compression which can double (or more depending on your lto generation) both capacity and speed; wikipedia provides a nice table with stats : en.wikipedia.org/wiki/Linear_Tape-Open
note: tape backup becomes more and more relevant as scale goes up and/or backup policies complexity increase (e.g. GFS)
I learned fealy early on (about the late 1990's) that tape backup is dinosaur technology. It is slow, unreliable and very expensive. Not just the tapes are costly but the drives even more so.
The best way to backup data is to use (surprise surprise) hard drives (either a rotating version or a SSD or even a bunch of USB3.x sticks will do)
Obviously backing up multiple 10's of Terabytes is not feasible on USB drives but on hard drives or SSD drives it most certainly is.
Both media formats are fast, easy, relatively cheap and highly reliable. On top of that is storage of data on hard drive or SSD the longest lasting (as far as it is known, simply because it is not yet known how DVD or Blu Ray storage fares over a couple of decades. Hard drive storage (the mechanical kind) the reliability data is very much in. As long as the PCB and mechanical components of the mechanical drive are still working the data can be read even after 3 decades Of SSD's and USB drives such data is not yet available but it looks like it is in the same realm of reliability, especially if said drives are not powered on all the time...
My recommendation for your case would be to use a series of SSD drives and a replication script or application which can run the data copy when said data is not in use. You can mount the SSD drives in external cases, preferably interfaced by something faster than USB 3.x (or get a hot plug drive cabinet of sorts) and remove the drives to take off site...
Does it support encryption of the backups? Do you need to store those chunks on a server so that it can do diff backup? or those are just all changes since specific timestamp? So appending to a file will create whole file backup or just diff since specific timestamp?
Tape backups are created from PBS backups, which themselves can be encrypted. So yes to that.
And since tape backups must come from a PBS backup, the complete backup does need to be on a server. The backup is complete (not timestamped / incremental), but deduplicated, so duplicate data blocks between two backups will be stored in one place.
I did the same estimation process of tape vs Glacier and it was close but I ended up going with the cloud. Cloud backups are scripted to run automatically whereas tapes require manual steps. I'm essentially gambling that I will never have to pay the restore costs though because if I do I'll be wishing I had gone with the tapes. AWS really gouges you when getting data out and you show tape is easier to work with than expected.
Yeah, that's how they get the price so low. They are also likely using offline media themselves of some type, so they need time to bring the media back online for your export.
You should run through a worst-case scenario data retrieval test, and also add that into your comparison!
@@koozmusic I did. For low double digit TB it comes out to around $100/TB to download directly or $50/TB if exported using Snowcone. That includes Glacier Deep bulk retrieval and some time stored in S3 standard while downloading or exporting. If you don't need all the data immediately you might restore over multiple months to spread out the cost.
I'm using wasabi at the moment. $6.99 TB/month. No fees for egress or API requests
you’ve convinced me to go another route. i’m going to just buy external 2TB SSD. the price is about $100. so for $400 i have a pool of 8TB of storage. totally readable using a modern interface.
I would prefer traditional disks instead of SSD due to SSD effectively using a capacitor for storing data that can leak if not powered on for a long period of time. But if it works for you, sounds like a cheaper way than tape.
@@pbolducNot a problem if the backups are being regularly refreshed.
Isn't there a way to limit writing in case your system can't keep up?
Yes, that works if you think 5TB is a lot. These days that's 2 weeks worth of holiday videos if you have a 360 camera.
Not to mention that those tapes and tape drives are not sealed and sensitive to dust and need to be climate controlled to reach their full 30 year life span.
It might just make sense to instead of 2 or 3 LTO-5 tapes, use a single 5TB sealed SMR HDD instead for like 100 bucks each that you replace every 3 to 5 years. And they're gonna be a lot cheaper in a few years when you migrate your data. And you don't need an uncommon, sold used drive that you can read them back, instead you can plug them in everywhere.
I would trust that more than used enterprise tape drives. Tape drives make sense for big enterprise imho and large companies where they run in dust controlled, climate controlled datacenters and you have professional tape cleaning machines that cost 5kUSD and warranty and economies of scale.
I find it hard to justify LTO financially, time wise, or data safety wise for home, small or medium business. Just my 2 cents.
I got a good deal on a lto5 tape changer and drive, she's a beast, haven't used it yet....in 5 or so years....one day
I'd like a tape changer for my next setup, but probably not soon
you're forgetting the cheap off-site backup that is the trunk of your car!
Only if you don't work from home
that assumes you have an attached garage,
i guess theres always the buried in the backyard approach
Mailing tapes back and forth sounds awfully a lot like a monthly cost :D
I get the appeal, but wouldn't SSDs be as viable, or dare I say even better, a solution?
I'm currently seeing a Samsung T7 2TB ("rugged") external SSD for $99 on Amazon, so $300 would get you 6TB of storage: less money for twice the storage of a couple of LTO-5 tapes. $1800 would get you 36TB today, but probably much more when you actually need the additional storage, as prices of SSDs keep going down. And I don't know if an internal SSD is a resistant as a tape, but a Lexar 2TB SSD is just $60 now, so you could get 6 drives/10TB for $360 (what you spent) and 30 drives/60TB for $1800, likely to go down. Not to mention the speed, the smaller size and the (probably much) lower weight.
What am I missing?
I've been looking at those ~$100 no-name SATA SSDs and thinking of building a Ceph cluster out of them. Tape definitely is a last level archival medium, not really for working data or even immediate backups.
When you put your data on a public storage site, your data is available, at least, to the people at that site. If the site gets broken into, your data is essentially stolen and may show up anywhere. I don’t even trust encryption completely for an online storage site.
Appreciate the video. You should take into account the hours of managing all these tapes. 2 tapes is not much, but if you have to manage 50 thats a considerable amount of time. The human error aspect also plays a role. If you forget to backup the tape or you ship the wrong one youre also fucked.
Is HP has the largest market share for tape devices?
You forgot the postage on mailing a tape each way every month in your cost comparison.
In my case I’m physically carrying them and I’m not making an extra trip, so it’s not an extra cost. But it definitely could be something to consider.
I need something for about 24TB worth of data. What should I use for backup?
Depends! What format is it (VM disks, files, ...?)
18:46 Was that a CAT? what is his name why has he/she not been feutred!?
It’s actually a different cat, this one is Danny the associate producer. The lead producer is Sherlock and he’s been featured many times before.
@@apalrdsadventures Way to briefly it seems! ;) Like you videos they are well reaserched and have your own spin not some other channel's repeat. Props!
@@apalrdsadventures ha ha ha ... always one to bring humor to your commentary ... Again, that's why I love your channel, @Apalrds.
I've got a LTO-3 drive and loads of tapes. Any way to still use it with Proxmox?
Not officially, although it might work even though it's unsupported. But you still need to get it to talk scsi / sas to the host.
Where did you find LTO-5 that cheap? I checked eBay and all of the ones in operational condition were a couple hundred more than what you found.
Can you use it as a regular storage drive or only for backup
Eh not really. It can only do sequential access. Some video professionals use it to store projects when they are done with them or to transfer them to other people.
My concern with using tape for backup is if the drive develops a fault it could then go on to destroy any tape you insert, leading to data loss. Using hard drives this is much less likely to happen. Plus tape really isn't much cheaper than hard drive. A brand new 4TB SMR drive is $55 which is less per TB than your $30 tapes, plus it doesn't need an expensive drive which could be hard to get if needed.
I picked up 150 LTO-4 tapes for £60
It can be very cheap.
@@deangray4502great deal!!
LTO drives are very robust and are designed for industrial use so they rarely fail and if they fail it's more firmware related stuff. My biggest issue with LTO tapes was finding cheap software for archiving purposes (with a catalog to browse content that is on tapes). Usually you only get backup functionality...
'tar' is the classic solution for that.. not great, but at least for me I tend to have tens of GB per project and could just write out the start and end project numbers on the label.
@@djole02yep. I’ve literally backed up thousands of systems on lto and not seen a single failure. Lto tapes a guaranteed for 25 years if stored correctly.
Still have a soft spot for backupexec from all those years back. Never let me down.
What about passing trough a whole disk to a vm and run windows with back blaze on it ? It’s 60 bucks a year if a not wrong. 😁
I was looking at the business pricing, not personal, but I also need to be able to get the data from the second level backup (Proxmox Backup Server) so it needs to integrate with that somehow. The options are the PBS chunk format or a tape or tape-like massive file.
@@apalrdsadventures samba file share wouldn’t do the trick? I know it very cumbersome but it might be a good offsite backup 😎 anyhow I really like your content and how much detail and effort you out on your videos. Keep up the excellent work.
I will probably try to afford 1 copy on hard drive, 1 copy on solid state storage. The third copy will probably on solid state storage because hard drive is sensitive to vibration, on off while testing motherboard bios, power failure, size. Consumer PC case have too little 3.5 inch bays and their mounting is not sturdy, they leave a lot of room for vibration and movement. I do not felt comfortable to use them. specialize case for hard drives are too expensive and I still not sure about their mounting.
Oh. I also forgot to mention about the SMR "plague". CMR drive now is not worth the cost for the speed and size compare to SSD.
Is it possible rewrite data on same tape or use a new tape for new data?
You can add new backups to the end of an existing tape if there is space, rewrite the tape, or use a new tape.
@@apalrdsadventures Also, Can i rewrite on existing data on tape? How many can i rewrite?
It's more like a CD, you usually write new data to the end until you erase the entire tape and start over
Tape drives are cool. I’ve played around with HP DDS drives and changers in the past. Having said that, for this application I would probably go with nvme drives, which can be had for about $60 for 2TB. You also mentioned shipping them offsite, so a padded mailer and some cardboard would be sufficient for thin nvme drives. I think nvme drives would last longer than tapes too, so replacement costs would be lower.
Not as fun as tapes though!
Tape is cool...still :) But setting up 2 Synology NAS devices does the same thing and the data can be live within minutes during a failure. But tape is still cool :)
How much the drive cost?
It really is a shame they have not made some consumer tape Backup system that used media like VHS Tapes.
And drives that only cost about $500.
I remember some apples software I bought way back in the day that would use my mini DV tapes and my digital eight tapes to store actual computer data on them but that was way back in 2003.
Is a lot cheaper than M-Discs, however all the money you save on the tapes you put it up front on the drive
It still seems like an expensive solution. Wouldn't it be better to have the remote site power down the hard drives when not in use and power up periodically for backing up?
Depends on the quantity of your data and the speed of your connection. On a good day of filming I can generate 10G of originals (which do get backed up, proxies do not), and the fastest plan I can get at my location is Comcast 1000/35. I only have ~500/25, and the 25 is a real killer for offsite anything, especially if I want to rate-limit the uploads so the connection is still usable to normal browsing.
So offsite over the internet is feasible but not fantastic for me to begin with, and mailing tapes (or driving them to places I'm already going) is easy enough.
I’m still using 1/4” tapes and dvds.
So clearly you need this www.heroinewarrior.com/heroine2200/heroine2200.php
@@apalrdsadventures That is awesome. I need this but for LTO. I know about the Dell rackmount changers but they're physically not very suitable for vertical space use.