New Home Backup Server (Dell T640 with 18 20TB Disks)

Поділитися
Вставка
  • Опубліковано 28 вер 2024

КОМЕНТАРІ • 83

  • @ScottDotDot
    @ScottDotDot  3 місяці тому +2

    A couple of things I didn't get to in the video if anyone was curious: Noise was about 55dBA @ ~0.5m from the front of the chassis with the disks under full load (array consistency check). At idle it was 45dBA at the same measurement location. (Though one catch is that it was in a small and otherwise empty/echo-y closet, which probably made it measure louder than it would have been in some kind of proper acoustic chamber. Mainly I was concerned with relative measurements, so it served well for that.)
    It drew about 105W at idle without any HDDs, and ~235W at idle with all 18 HDDs installed. Under full disk load it was almost bang on 300W most of the time.
    To put the sound in perspective, the previous setup was about 66dbA. That was mostly due to the HP D2600 DAS that was perched on top of the old server. (I had spec'd the server itself to be super quiet. Didn't measure it independently, but it was much quieter than the Dell for sure.) But one important thing with sound levels: The D2600 fans ran faster and were of a much higher pitch and buzzier than those in the T640. So even with the disks working hard, the T640 "felt" far quieter than just that 11dB difference. It's more like a low "woosh", and not distracting or uncomfortable to be around... though of course that's fully down to personal preference/tolerance.
    I should be posting a follow-up soon, so you can see and hear what I mean. And here it is now: ua-cam.com/video/Secv5P01k2M/v-deo.html

  • @gatekeeper88
    @gatekeeper88 3 місяці тому +9

    Thanks for the vid but please don't gatekeep seller's, it's a really bad look for others who would like to do a project like this or similar, it's really poor form.

  • @StringerNews1
    @StringerNews1 3 місяці тому +17

    As for RSYNC backups, I used to do this as a service to small business clients. I'd typically have individual computers do an incremental backup every hour, so no more than an hour's work could be lost. But instead of having daily backups that rolled over every 30 days, I saved them daily, then weekly, then monthly and finally annually. That way, there were multiple pipelines of backups, with different sample rates and different rates of expiry. If nobody noticed a ransomware attack all week, someone would the following Monday, and would have the entire previous week's daily backups to restore from. After going unnoticed for more than a week, the granularity would drop from daily to weekly and so on. The longer it goes unnoticed, the greater the granularity, but that's better than dropping from all to nothing in one day.

  • @arbyyyyh
    @arbyyyyh 3 місяці тому +4

    I’d definitely call those caddies. The things for CD’s back in the day we’d call cassettes.

  • @Solkre82
    @Solkre82 3 місяці тому +3

    Wooo love tapes. I manage a LTO8 library that just touched 7PB.

  • @CoreyPL
    @CoreyPL Місяць тому

    Since you are serious about your backups and already have a capable server, then I would definitely split the array at the cost of usable space. Used system, used drives - it's not that hard to have an error on the array and every time it happens it will resilver or scrub. On ZFS maybe 2 RAIDZ2 vdevs or 4 RAIDZ1?

  • @primeral
    @primeral 5 днів тому

    Just found your vid, this 3.5" x18 is the configuration i want to upgrade my R730 with. It's interesting that the 14g T640 use the 13g drive caddies/sleds. This is great because the 13g caddies are a lot cheaper.
    I got my R730 idling at 120W with 8x 3.5" drives. I couldn't find it in the comments, have you tested what the T640 idles at with 18 HDDs?

  • @daven6634
    @daven6634 Місяць тому

    I'm always curious about what data people are storing on these servers. All I can think of is video formats, like raw editing videos or like movie storage. What else would you do with it?

  • @armstrongskyview2810
    @armstrongskyview2810 3 місяці тому

    Awesome video, can this host a WordPress site?

  • @ikkuranus
    @ikkuranus 3 місяці тому

    How about 4x raid z1 with 2 hot spares? Is it even possible to use 1 or more hot spares to cover multiple vdevs?

    • @leexgx
      @leexgx 3 місяці тому

      Your on a good chance that you could have a read error in 2-3 days, z2 minimum
      Hotspare can cover all Vdevs (z2 before considering Hotspare) in his setup as it's only a backup node, 1 large z3 vdev or 2xz2 Vdevs
      Also the way he has his backup setup (taring up the changed files) when could just use btrfs on top of md raid and run a snapshot before starting backup pull rsync

  • @StringerNews1
    @StringerNews1 3 місяці тому +1

    "But you can't use RAID for backup" is what all the Internet ninnies say.

    • @nadtz
      @nadtz 3 місяці тому +1

      If you are doing 3-2-1 backups it's fine, what people generally mean when they say that is raid should not be your only backup in case of catastrophic failure which is 100% true.

    • @StringerNews1
      @StringerNews1 3 місяці тому

      @@nadtz regardless, one reason why we use RAID is specifically because a RAID array can survive hardware failures. That makes RAID _more_ suitable for backups, not less.

    • @nadtz
      @nadtz 3 місяці тому

      @@StringerNews1 Obviously, but the point is it should not be your only backup because it would be a single point of failure. Software raid makes things a lot easier now a days but I've personally experienced hardware raid failures that were either unrecoverable or took an extremely long time to rebuild and our clients were always happy we had planned for that contingency. I treat my home network the same way.

    • @StringerNews1
      @StringerNews1 3 місяці тому

      @@nadtz 1. that's not true, and 2. your long-winded narrative doesn't resemble what I was referring to. Your tactic of finding fault with things I _didn't_ say is what's weak.

    • @nadtz
      @nadtz 3 місяці тому +1

      @@StringerNews1 1. Yes it is, there are enough posts on the internet from people who didn't have a backup of a NAS that failed or had a ransomeware attack or deleted something by mistake that I don't even know why you would say it's not true. Feel free to look it up for yourself.
      2. Because you are mischarecterizing why people say 'RAID is not a backup'. It's not because you can't use raid as part of a backup strategy, it's because RAID only protects against disk failures and that's not the only possible way to lose data.

  • @punch3n3ergy37
    @punch3n3ergy37 3 місяці тому

    Nice video!
    Could you maybe link the ebay reseller?

    • @davelb54
      @davelb54 3 місяці тому

      Ditto on this request. A reliable ebay reseller of enterprise large capacity drives is golden!

    • @ScottDotDot
      @ScottDotDot  3 місяці тому +1

      It was serverpartdeals. So far so good with the drives.. I know it took me forever to get to ur comment (sorry about that!) but at least I can now say the 18 in the system have been running OK for over a month now (and the other 2 passed some basic testing). But I don't want to 100% recommend them either, because for all I know the drives could all be duds that crap out after 3 months. But I suppose that's the risk with any refurbs (or new drives come to think of it..)

  • @Thewickedjon
    @Thewickedjon 3 місяці тому +6

    what type of business are you in, that you require so much storage / hardware?
    is this youtube channel THAT BIG??
    Ddo you host services for others? or do you just like big expensive server hardware? :D (like me)
    I'd love to know

    • @ScottDotDot
      @ScottDotDot  3 місяці тому +3

      Well it's not that the channel is big, but the files are big. For some godforsaken reason I record in 2160p ProRes 422 HQ (~220Mbps). Also I may or may not rip my own blurays that I purchased with my own money and duplicated for personal use and archival purposes, and those may or may not be accessible through a Plex server. Plus I'm just generally a data hoarder.
      Quite a lot of space for full VM disk snapshots of servers as well, which are saved for around 15-30 copies over time.
      But yeah, most definitely I'm into big expensive server hardware lol. I mean, hyper expensive when new.. not so much by the time I get to it on ebay.

  • @videofeed99
    @videofeed99 2 місяці тому +2

    You got my subscription after: "... mouth hole on my face..." 17m57s --- Priceless 😂

  • @gordoncreAtive
    @gordoncreAtive 3 місяці тому +7

    I think I got the same drives of the same seller - two of them were faulty. One DOA, the other one got bad sectors after my burn in test. I propose you test them. I recorded all the SMART data, put them into a ZFS mirror, filled them to the brink with F3, verified the data, scrubbed the disks and then did a long SMART test. I think that's the most thorough and viable approach.

    • @stephenxs8354
      @stephenxs8354 3 місяці тому

      How is this not worse than using a default 4 pass badblocks R/W.

    • @gordoncreAtive
      @gordoncreAtive 3 місяці тому +2

      @@stephenxs8354 badblocks can't deal with 18TB drives.

    • @ScottDotDot
      @ScottDotDot  3 місяці тому +2

      Oof, I don't like to hear that. The seller was serverpartdeals. I didn't want to call them out in the video or seem like I was shilling for them, but I guess it's not exactly top secret. Now you're making me doubt myself; I've found that a full array build will usually sus out any immediately bad disks, but half of it was just me not wanting to spend X days running 400TB worth of data through them and then waiting for the array to build afterwards. 😕 FWIW the array has been through about 150TB of writes and a consistency check since the video and so far no [apparent] problems and no SMART issues.

    • @stephenxs8354
      @stephenxs8354 3 місяці тому +1

      @@gordoncreAtive Did you try with block size of 4096 or bigger? My 14tb worked fine with -b 4096.

  • @TheChadXperience909
    @TheChadXperience909 3 місяці тому +1

    Split it up into 2 x Raid 6 (z2) 8-wide HDD arrays, with a hot spare each. A single one of those arrays should just saturate a 10gbps nic.

  • @JasonsLabVideos
    @JasonsLabVideos 3 місяці тому +3

    Good video sir! The only thing i wish Dell would enable is Bifrication so we can use those multiple m.2 pcie x16 cards..

    • @ScottDotDot
      @ScottDotDot  3 місяці тому +1

      Thanks! Interesting you mention that.. For this project I bought a PCIe card which supports two NVMe drives in sleds that are externally accessible (also from ICY DOCK by coincidence -- ToughArmor MB842MP-B). Was considering using them for my most essential files initially. It said it required bifurcation, so I checked the T640 BIOS and there is an option to turn bifurcation on. That being said, I didn't end up testing it so I can't say 100% it would've worked. But I was kinda surprised the option was even there.

    • @JasonsLabVideos
      @JasonsLabVideos 3 місяці тому

      @@ScottDotDot My Lenovo St550 with Dual 4114's and 256gigs ram stuff with goodies ! Love it but Bifrication isn't an option. You can buy cards that bypass the bifrication but they are 4x the price :(

    • @dash8brj
      @dash8brj 3 місяці тому +1

      My R730XD supports this on the x16 slot. I have it populated with a 4 m.2 PCI-E card with 8Tb of NVME drives installed.

  • @chrismay2298
    @chrismay2298 14 днів тому

    Ah yes, another "mouth hole" user. "Eye hole" is also a favorite of mine. Nice work. Getting a T630 soon.

  • @hburke7799
    @hburke7799 3 місяці тому +2

    for the T630/T640 I like using the MB998IP-B (icydock 8 bay 2.5'') and the D P/N 010WY power cable.
    the 18 bay chassis is a little short on 5.25 space, and comes with a one sata power cable. using the mutli-head power cable saves you needing a power splitter. and the 8-bay icydock has two 8643 mini-SAS connections which are FAR more convenient than using 6 or 8 (especially locking) SATA connectors in such a tight space.
    the T630 has SAS connectors on the motherboard. though the T640 has oculink, but can be adapted to SAS 8643 fine. (or pcie if you are exceedingly daring)

    • @ScottDotDot
      @ScottDotDot  3 місяці тому +1

      I appreciate the heads up on that. I hadn't really looked through IcyDock's product line, but yeah, a 8643 connectors would be wayy nicer for cable management. Definitely going to keep that in mind for future systems.

  • @oldshield
    @oldshield 4 години тому

    who is the person on ebay you got the drives thru?

  • @debugin1227
    @debugin1227 3 місяці тому

    your show so salute, tip your fedora or what ever you like .. just keep up the interesting stuff! and thanks

  • @rok9526
    @rok9526 3 місяці тому +1

    Hi Scott, excellent video, thanks so much for sharing :)
    Two comments from my side:
    - Regarding snapshots, you could use btrfs with built-in snapshots (like on Synology). This way you get free data deduplication and still keep the snapshots principle going, to protect against malware. Data would only be "duplicated" in case of being overwritten by malware.
    - Not sure if it's an overkill, but I do new HDD burn-in with badblocks -wsv You can choose how many runs etc. Takes quite a long time, but gives good confidence imo.
    My two cents. Thanks again for the great video!

  • @shephusted2714
    @shephusted2714 3 місяці тому +1

    connected with gigabit and then you are going to move it? instead run a couple cables and make it 10g bonded but other wise plenty of space, nice setup

  • @FINPJENN
    @FINPJENN 3 місяці тому +1

    How many watts does it pull with all drives connected?

  • @bak1necWWE
    @bak1necWWE 3 місяці тому +3

    Hi. why you use hba mode with software raid rather than the raid that was on dell bios?

    • @ScottDotDot
      @ScottDotDot  3 місяці тому +4

      Oh yeah, I should've mentioned that! For the initial array creation, it was so that I could address the disks individually to check SMART stats. But long-term I prefer software RAID for portability. If this system dies and I need to recover the data, I could connect all the disks to any old Linux system. Some older RAID cards and cards from other brands wouldn't necessarily be able to read the metadata on the drives, so I'd need a PERC H740p (or compatible) in the replacement/recovery system. I'd definitely go hardware RAID for performance, but portability is more important to me in this case.

    • @Sebyllis7350k
      @Sebyllis7350k 3 місяці тому +1

      hardware raid is long dead at this point.

    • @RomanShein1978
      @RomanShein1978 3 місяці тому +1

      Hardware raid is dead.

    • @incandescentwithrage
      @incandescentwithrage 3 місяці тому

      Lots of comments that hardware RAID is dead.
      I'd argue hardware RAID is dead for flash.
      Spinners with the 8GB cache of the H740p is preferable to MD RAID6, IMO.
      H740p is a rebadged LSI Megaraid and you can import the array with a HBA & mdadm if you ever need to.
      Smart data and even controller configuration is available from within the OS with perccli
      Patrol reads and consistently checks can be run on a schedule.
      Pretty good as HW RAID goes.

    • @ChristopherGoggans
      @ChristopherGoggans Місяць тому +1

      ​@@incandescentwithrageI'd say hardware raid is dead even with spinning rust. BTRFS and ZFS offer COW and checksumming, so that your data survives bit-rot and other corruption issues, as well as offering compression and deduplication.

  • @mlitzy
    @mlitzy 3 місяці тому +2

    Have you considered FreeBSD with ZFS?

    • @ScottDotDot
      @ScottDotDot  3 місяці тому

      Yup, I use that combo on my main file server. I don't have a 100% solid reason for not using it on this backup system, other than ZFS gets complainy at (iirc) 85% array capacity and I kinda like the idea of using incompatible OSes for data duplication. (Probably pointless, but it's under the vague premise that maybe I'd get a malware infection that targets BSD but not Linux or vice versa... that's crazy though, right?)

    • @mlitzy
      @mlitzy 3 місяці тому

      @@ScottDotDot I did see on one of the new CanBSD videos it was mentioned that some businesses didn't want a single point of failure so they added BSD systems away from a fully linux environment. So you are in good company going with both.

  • @TheTrulyInsane
    @TheTrulyInsane 3 місяці тому +1

    I usually just batch print all of the labels at the same time, saves all the cutoffs/waste

  • @rdsii64
    @rdsii64 3 місяці тому +1

    This is going to sound strange, but my back up "server" runs desktop windows. Why? because the back blaze unlimited service is less than $10.00 a month but that service doesn't support server OS's or linux. Right now I have about 50 terabytes of stuff in my media server ( growing very quickly) and its all backed up off sight for less than $10.00 a month.

    • @Sebyllis7350k
      @Sebyllis7350k 3 місяці тому

      if you have thoroughly tested this solution, there's nothing wrong using Windows as a server. updates and other Windows shenanigans should obviously be taken care of, but other than that, Windows is the most popular OS by far after all.

    • @KS-wr8ub
      @KS-wr8ub 3 місяці тому

      Did you ever try rebuilding from BackBlaze Personal? I've heard very mixed results when you have that much data.

    • @ikkuranus
      @ikkuranus 3 місяці тому

      It might not natively support linux but there is a docker with backblaze personal backup which makes use of WINE and vnc to manage it.

    • @rdsii64
      @rdsii64 3 місяці тому

      @@KS-wr8ub I have never tried to restore my complete server but I have had to restore single drives and it worked fine. The tier I pay for gives me a year of version history. If I had to restore the complete server I suppose I could do it one drive at a time. It certainly isn't ideal but so far its the only service I can afford with this much data (and growing).

  • @jeffreylaw4027
    @jeffreylaw4027 23 дні тому

    what's the $$ for the build?

  • @magog6852
    @magog6852 2 місяці тому

    power consumption = terrible.

  • @jaimeduncan6167
    @jaimeduncan6167 3 місяці тому

    Many people do not take the rebuild time risk into account. I have explained it many times, and often people are not even paying attention. People nowadays are more and more used to social media information overload and they just feel and react. It's funny because in the hard sciences, engineering and business, we used to look funny to the social science students deciding what is true because it "resonated with them" and today it's mainstream. Flash-based SSDs have a pretty bad write speed (out of cache) but can still perform many more IOPS. They are still expensive, but they already 15TB in the mainstream.

  • @BrianThomas
    @BrianThomas 3 місяці тому

    Dude, im a little new to the channel here. What the heck are you doing that youll need 400 TB of drive space? Man oh man that's a ton of storage.

  • @Yandarval
    @Yandarval 3 місяці тому

    The huge empty slot bay cover will be the basis for stuff like card readers to be added with a bezel change. I have an HP server with a similar feature. if you don't need the PCIE lanes or the cores. Taking out the second CPU will reduce your power bill.
    You are correct about the CPU 2 expansion slots. You need a second CPU to supply the PCIE lanes for them. A similar deal if you want to use all the RAM slots. A second CPU to use the slots for it.

  • @tonyc523
    @tonyc523 3 місяці тому

    You forgot power draw numbers, with and without drives. Also, dont salute w ur left hand. Nice video. Thanks.

  • @StenIsaksson
    @StenIsaksson 3 місяці тому

    The 6 bay drive cage looks like the one I have. I set the fan speed to low (just to get SOME airflow) and you can't really hear the fans.
    Very low noise. You can hear them if it set to High

  • @jmustafa1973
    @jmustafa1973 3 місяці тому

    Could you do video on the scripts you use to do the backup. I use rsync for remote backup, but nothing fancy like you have.

  • @lenano
    @lenano 3 місяці тому

    You might wanna watch out with that sata power splitter for the boot drives, it looks like it's made using those molded sata plugs that are prone to shorting out and catching fire

  • @kopiluwak_kayubakar
    @kopiluwak_kayubakar 3 місяці тому

    this not for home, this suited for small business. from my experience this kind of storage server is hot and very noisy almost like enterprise server class. you should has dedicated room with good air circulation and insulation.

    • @ScottDotDot
      @ScottDotDot  3 місяці тому +1

      I just posted a pinned comment with some followup on the power/noise. 100% that server is targeted at the small biz market, but as servers go it's -relatively- quiet and not very power hungry. The noise really comes down to personal preference; Some people could live with it happily, but it would drive others crazy. ~235W power consumption, so not a crazy amount of heat.

  • @MasterJediSean
    @MasterJediSean 3 місяці тому

    I believe that "Sleds" are just a plastic sliding tray with no electrical function like the 12v Hot Swap tray or Caddie. I've heard them referred to as either a Caddie or a SAS Drive Tray. but sleds are just a piece of plastic, which are in most desktop cases.

    • @BigBenAdv
      @BigBenAdv 3 місяці тому +1

      Depending on the system and HBA, some caddies (trays/ sleds) could also incorporate a SAS interposer module. This allows SATA drives to be installed in a system that otherwise would only work with SAS drives (Dell Powervault for example).
      It basically enables more SAS specific features (e.g. delay spin-up and some SAS specific diagnostics) and allows for dual SAS controller redundancies for the most part.
      Most Dell PERC cards will also flag a warning if they don't see a Dell FRU firmware on the drive when they don't see T10 (SAS) support on the drives.
      On more modern servers, they'll offer Value/ Nearline/ Midline SAS which are basically SATA drives with a SAS type controller that doesn't require the use of interposers.

  • @MrSamadolfo
    @MrSamadolfo 3 місяці тому

    😏👍 Very Nice, now fill it up 😊

  • @bobkoss280
    @bobkoss280 3 місяці тому

    New subscriber. Need a homelab tour.

  • @chenxuewen
    @chenxuewen 3 місяці тому

    二手还是很划算的

    • @billzhou2481
      @billzhou2481 13 днів тому

      国内18盘的T630/T640准系统贵的离谱

  • @dleer_defi
    @dleer_defi 3 місяці тому

    What a build!

  • @Koop1337
    @Koop1337 3 місяці тому +1

    As soon as you dropped a F bomb I thought "This guy is from the North East" immediately. Instant subscribe.

  • @OfficialMikeJ
    @OfficialMikeJ 3 місяці тому

    The constant swearing when I am trying to teach my kid about computers made me very angry having to tell her to go do something else. She doesn't need to hear this kind of language when being taught about computers and servers. very disappointed you kept throwing out the F bombs. Not acceptable by me. Not impressed at all. no need to be swearing.

    • @magog6852
      @magog6852 2 місяці тому

      teach your kid how to be a man, unlike his father

  • @youtubegarbage4u
    @youtubegarbage4u 19 днів тому

    what type of raid, filesystem and volume manager you using on top of it? i believe you mentioned ubuntu OS.