The Perfect JBOD SAS expander | for building your own DAS JBOD disk enclosure
Вставка
- Опубліковано 7 жов 2021
- In this video I'm going to show you guys what I think just might be the perfect JBOD SAS expander board! This is a board from a Dell enterprise storage system with Dell P/N 6TDVN or 5R10N. This is a SAS-2 expander board with 24-internal SAS lanes (in 6x SFF8087 ports) and 12-external SAS lanes (in 3x SFF8088 ports). We'll take a close look at this board and talk about it's features, as well the problem with powering on this board. I will also demonstrate a proof of concept build on how to use this board for building your own JBOD DAS disk enclosure chassis.
If you'd like to get one of these Dell JBOD SAS expanders, check it out here: ebay.to/3MCBd19
Video index:
0:14 - some back story
3:59 - a close up look at the Dell JBOD SAS expander board
6:43 - the problem with this Dell JBOD SAS expander
8:07 - Dell and their shananigans. The story about my old PowerEdge 1800
10:04 - a solution to the problem; customer power adapter cable
10:58 - describing the proof of concept (PoC) demonstration setup
15:19 - powering on the PoC and verifying that it works!
23:19 - wrapping it up
23:58 - server hardware drama pr0n; just having a little fun ;-P
If you'd like to support this channel, please consider shopping at my eBay store: ebay.to/2ZKBFDM
eBay Partner Affiliate disclosure:
The eBay links in this video description are eBay partner affiliate links. By using these links to shop on eBay, you support my channel, at no additional cost to you. Even if you do not buy from the ART OF SERVER eBay store, any purchases you make on eBay via these links, will help support my channel. Please consider using them for your eBay shopping. Thank you for all your support! :-) - Наука та технологія
![Lp. Последняя Реальность #107 РОДНОЙ ДОМ [Финал] • Майнкрафт](http://i.ytimg.com/vi/IK3QKzKUlHM/mqdefault.jpg)
Fresh cup of coffee and a new Art of Server video, what a great way to start a day :)
Thanks Unkyjoe! :-)
I'm an hdd engineer and basically set stuff like this up for a living. I'm surprised there's any videos on SAS expanders out there. I see all the secret enterprise hardware the big companies use. Too bad they don't sell their proprietary enclosures and hardware. Some of it is really really good.
So glad to see you back at it.
Thank you Tom! :-)
I keep buying from you for a good reason! Thanks for not gouging us.
Thank you for supporting me!!! :-)
Thanks so much for taking the time to explain all the whys and hows. I wished I had that sort of detailed information available, back in the day
Also, I really appreciate you taking to time to assemble and sell the power connectors in your store.
Thanks for the kind words. I appreciate your support! :-)
i always watch your videos. you’ve made my job so much easier
Glad I've been able to help! Thanks for your support! :-)
Thanks for your time and effort.
Thanks!
The Adaptec 82885T look excellent as well for people who want the 12GBPS SAS3 capability. Affordable as well!
Cool! Thanks for sharing!
I've been buying these for 20-40 bucks before they get expensive, but I've had some odd results with these. I have at least one which doesn't allow use of all ports for drives, sometimes the port works as the HBA connection, other times they don't seem to function at all. So despite having enough ports for my 24 drives, I had to connect 20 drives and run one cable from the second HBA port. I'm sure it's just a firmware issue, but I haven't gotten any leads on what firmware it can take or if there are updates at all for it.
The cards seem to be a controller for a large Dell rack JBOD, and someone ended up with them outside the chassis to resell. They absolutely need a fan (I just ziptie 40mm Noctua's). Even with normal case fans on them they get crazy hot (server cases probably have enough airflow if they are in the path).
The controller chip shouldn't care which ports are used for drives and which are used for HBAs (up to 2 HBA links). Again, why I have some that seem to care is probably firmware related.
I made up a few SATA power -> Expander cables (and a couple Molex like yours). Pinout on the expander end is:
1 - GND
2 - GND
3 - 12V
4 - 5V
If anyone is comfortable making their own cables.
As a bonus, you can get SFF-8088 to SFF-8087 or SATA cables to make use of 2 of those external ports to bring the drive count up to 32 per card.
Thanks for sharing your experience and knowledge about these! The ones I've tested so far have not experienced the type of problem you described. I wonder what would cause that? On the HBA side, you can configure the ports to be in either initiator or target mode, but I don't know of anything similar for SAS expanders...
Thanks for watching!
Thanks for posting this info. I picked one up from a different seller as Art of the Server is sold out. So I will be making my own power adapter. Question for you if you happen to see this. If you plug in two SAS cables from the HBA into this card it's supposed to get better bandwidth. Does that work with SATA drives as well? Or only SAS drives? Thanks!
I really appreciate your work and explanations and your store.
Thank you. I appreciate your support! :-)
Thanks for another great video!!
Thanks for watching!
Just found your UA-cam from your eBay account, content looks awesome, got yourself another avid subscriber :)
Welcome aboard! And thanks for subbing! Hope you enjoy the rest of my content. :-)
Great video and detailed explanation. I was thinking on building a DIY 48DRIVES enclosure and this expander is missing key. I've got few Supermicro 12bay backplanes from cse-826 chassis i built that didn't need those parts. This will be a hell of a project. Hope everything goes smoothly. Cheers
Good luck with the project!
Thanks for the info on the 4-pin power connector. This is crazy. Dell should have used Molex for that.
Totally agree about Dell. I really do think they do this stuff to mess with people who try to stray and use non-Dell parts. Thanks for watching!
As always, full of excellent info.
Thanks for watching!
@@ArtofServer been watching for awhile, trying to decide how to build, reconfigure Proliant servers. I picked up several for practically nothing last year. Your vids are extremely informative.
Glad they have been helpful! :-)
Another great video. I looked up the Supermicro CB2 JBOD power board but unfortunately too expensive (thanks to that annoying chia stuff). But I remembered that I have several older mATX motherboards lying around that I can use instead to power the expander and is pretty much 'free' to me
Cool! Yeah, some older mobo are great substitute!
Well, you could've used a pcie x16 to usb adapter for gpu mining, iirc it does deliver 12v and 5v
Maxtor.....my old nemesis we meet again.!
I actually miss the days when there were more brands of drive manufacturers.... anyone remember Quantum, Fujitsu, Conner, etc.? Or that most of the computer manufacturers also made HDDs like DEC, HP, IBM, etc. We are now left with only Western Digital, Toshiba, and Seagate. :-(
@@ArtofServer I have all those except a Conner in my junk box....many of them still work most all are ide interfaces
I've been getting into this subject for work over the last 6 months and gotta say, maybe my standards are out of wack but pricing seems pretty reasonable right now, maybe the thing you mentioned collapsed already. They're not cheap or anything as I'm sure you know, but they're cheap enough that I have to remind myself not to buy anything for home use. I do not need this at home, but I can afford it and it's fun.
During the peak of the Chia mining boom, one of my suppliers where I buy the IBM 46M0997 SAS expander from increased their price to $700 per item!!! Things have calmed down since then, but there was a time when these type of items were much cheaper. I suspect prices on non-essentials may continue to decline as the cost of food and energy are increasing.
Ah, that's why my server caught on fire. Lol thanks dell. I learnt to just get a 4pin to molex instead, another issue I had with these boards is that, bic doesn't have screw holes that lines up with anything..have to punch stand off screws in chassis.
The power extended thing you have is really cool, I want one of those
oh no! you server caught fire?
yeah, definitely need to make custom mounting standoffs for these.
@@ArtofServer It started smoking, I attached a normal 12V cpu header in it.. but i did some research and learnt that it will take 5v from molex later.. it was too late but meh, lessons learnt. Also, love you youtube channel man. Im a developer but love networking, just started home labbing and both your youtube videos and your ebay store has been a really good resource for me, do you happen to have a discord server or something btw?
Top form. Great work to dissect this obscure enterprise stuff and put it to good use; I wonder how many of these have ended up in the trash.
Thanks! I know of a few that got fried. :-(
Thanks for watching!
as usual a great video, super clear, super informative... thank you man.
Thanks for your kind words!
Thank you for another great video! I have two questions. Firstly, is it possible to connect the JBOD to two servers via the two SFF8088 ports? Secondly, do you offer the CSE-PTJBOD-CB2 JBOD power card?
It is possible to connect 2 servers to the same JBOD enclosure. However, you need to have software to manage which drivers are controlled by which server or you will see the 2 systems clobber each other's data writes. This mechanism is sometimes used for fail-over scenarios where one server goes offline, the other assumes control of the storage and comes online to continue operations.
Sorry, I do not currently offer that JBOD controller board. :-(
Would of loved to buy one of your sas expanders. it’s a shame im from 🇬🇧 you got me into this 😂😂
Thanks! At least the server hobby isn't as bad to your health as alcohol, tobacco, or sugars! :-)
You guys need to stop your politicians from implementing silly import laws. Due to the new safety labeling requirements, I'm not allowed to sell to the UK anymore! :-(
Thx for video and your quick reply on an email I sent previously.
No problem. Hope I was able to help. Thanks for watching!
I have an old WD Sentinal DS5100 that I would like to use just for the power and box. How hard would it be to convert it into a JBOD but do not want to use the MB as it requires marvel drivers to be installed before you can see the HDD. Would this be a task worth pursuing?
great video thumb up
Thank you! Cheers!
Great videos. I am very new to the world of JBOD and SAS ect ect. If I had an old desktop running unraid, is there a setup where I can purchase a pci expansion card to attach a commercial jbod enclosure and have it work with unraid? Thanks.
Hi! welcome to the channel! :-) Yes, you would need an external SAS controller to connect to the JBOD SAS enclosure. Watch this video to learn more about the different HBA SAS controller options: ua-cam.com/video/hTbKzQZk21w/v-deo.html
Just order a 9206-16 from you. Thanks for the videos! don't they make a PCIe SFF8088 to SFF8088 pass though
Thanks for supporting my store! I don't know what you mean by "PCIe SFF8088 to SFF8088 passthrough?"
I've got 4 Dell LGA 2011 boards I could never get to energize with an ATX pwr supply. I think they may still be ok. Any suggestions as to what to use or a work around to power these boards?
I don't know those specific motherboards. In the past, Dell would swap 2 pins, a 5V and 12V line. If you figure out the pinout, you can swap those same pins on a normal ATX power supply. Just be careful because that ATX power supply will fry a normal motherboard if you repurpose it later.
Great stuff! Thank you! Im curious as to where you found the lsiutil app?
Also Im connecting this expander to a supermicro BPN-SAS3-826EL1-N4 Backplane and I can see any of the drives...got any ideas? Thank You!!
lsiutil is an old LSI program. the source code and pre-compiled binaries are available if you search online. If you are having issues with drives showing up, I recommend you watch this video: ua-cam.com/video/1dCd6IepB5s/v-deo.html
@@ArtofServer Thanks for your response! I used the Dell Expander with other machines and no problems what so ever. Thought the BPN-SAS3-826EL1-N4 might have a configuration error or something. I have 2 of the dell expanders and haven't had any issues with them at until I tried wth this backplane. Thanks!
This! This, in SGI / Rackable Systems S3012 chassis!
I'm not familiar with that chassis. But when I think of rackable systems, I can only think of SGI's final 🪦... :-(
I miss the old IRIX systems...
@@ArtofServer It's a 3U 12 drive chassis. For example, it's used in Arima NM46X. Best thing is that those SGI/RS boxes use backplanes that have these 2x2 power connectors, and as a result those power supplies have the neccessary nonnectors.
So before I make a very expensive mistake.... if we wanted to replicate this, is the jbod controller board mandatory or could we just wire a SPST switch to an atx psu jumper for the jbod psu?
you could do that. just understand you need a way to drive the chassis cooling system too, not just power on drives.
Would a simple SPST switch on the green line on the power supply suffice?
Of course you can always rig something to accomplish something similar. Just take into account you also need to manage chassis cooling too.
Any chance that 3rd external 8088 can take four more drives? Making it a 28 total drives?
I believe that's possible.
Hi, if you power down the main server that has the HBA, would it power down the JBOD chassis and its drives?
When the power goes out, would powering on the main server power on the JBOD chassis?
No, the JBOD enclosure has it's own power system. If you want to synchronize the power states of the host server and the enclosure, you need to devise some mechanism to do that. Some JBOD controller boards have an IPMI network interface where you can send commands to change power state. Alternatively, you could also use a network controlled PDU where you can send commands to power on/off certain outlets. I'm sure there are other solutions too.
SAS is pretty cool. It reminds me of FireWire for some reason
Lol... For some reason, in all my years around tech, I never got to use FireWire.
So i can use this Dell board and Connect to hba board to my computer/server and will work? Im just getting started with this machines im trying to learn
Yes, you can connect this SAS expander to a HBA card and basically have more ports for storage drives.
Is there an HBA with internal and external ports? I have an mitx and it only has 1 PCIe slot, but I would like to add a DIY JBOD like this
Yeah, look at cards like the 9212-4i4e, 9217-4i4e, or 9300-4i4e as examples.
This is probably a dumb question, but if you have an HBA with only internal connectors, how do you connect your expander? Is the only choice another HBA?
You can connect it to one of the internal ports as well.
I notice you sell 2 different versions of this expander. What is the difference between the 2? 5R10N vs the N4C2D
They have different Dell P/N, but from what I can tell they are basically the same.
Do you have a link to the power controller? I was not able to find it in the other videos of the playlist...
Which one are you asking about? I'm not sure what you mean?
Maybe the same part I'm wondering about. It's the jbod controller I believe. At 13:50 you talk about it. Do you sell those?
@@janilom81 yes, exactly!
@@ArtofServer The other circuit board in all the video, whats that, are you doing a Dell om us.
Is it possible to insert a combination of SFF8088 to SFF8087 cables + SAS PCI bracket for an aesthetic reason between the HBA JOBD expander and PCI slot of the rack?
I'm not sure what you mean?
@@ArtofServer Since the SAS cable outward passes via the vacant I/O shield, so I just wonder whether it is possible to make the looks to be clean by inserting anything without negative effects 😅 longer cables, more connection points, etc. Anyway, thank you for replying.
lsiutil not found, did it get replaced? I'm on Ubuntu 22.04.3 LTS
you need to install the program. it does not come standard with Linux distributions.
My apologies if I missed it but how can you do fan speed control with this setup? Anyway to do it based on hard drives thermals
Add a programmable pwm fan controller.
@@ArtofServer any recommendations for a unraid build?
Hi Just wanted to ask? I have a Dell R515 with a dead motherboard + one that works fine could i do this to the one that dead for more storage space??.
Maybe. I don't recall whether the r515 has an independent PDB or onboard PDB.
@@ArtofServerHi it has a separate board behind the power supply's same as the R510 ??.
Great 👍 then it might be able to turn that into a disk enclosure.
Would this work
With any sas card?
I don't know about "any", but I've been testing these with LSI based SAS controllers and they've all seem to work so far.
Where does one get one of those JBOD controller boards?
After many people have asked, I will be selling those in my store shortly.
@@ArtofServer it seems like a critical component. I’m at the point I need to build an external enclosure (soon)
Would love one of these but you dont ship to the uk and i cant find one here
It's not that I don't ship to UK, it's because UK has new product safety laws that are preventing import to UK on certain electronics products. See my community post about this issue.
@ArtofServer just checked the good old UK making it hard for us again
I’m building a plex server using amd /win 10 and 24 bay with back plane , but I can not get a card to display the hdds when loaded 🤦♂️🙏 help
If you're using a LSI based HBA controller, try my troubleshooting guide here: ua-cam.com/video/TC4DovIcGLU/v-deo.html
@@ArtofServer much appreciated for replying and link sharing, I am checking out the video, the issue is I’m new to all this and was just hoping for a quick fix , I just want to access my drives , I have found many videos explaining there raid card and setup options etc but my setup is completely different/ options menus os making this very difficult to get passed , god bless 🙏 you
Have a dell precision 5600 and 5610.. would like to use 4 to 6 drives for NAS.... have a LSI board that seems to be working but I only have (that I know of) 2 power cables for 2 drives and 1 for CD... which I think with some work can make into drive #3...
Would you have info on how I get power for the 1 to 3 more drives? If this would be a consultation just let me now an e-mail I can use to communicate with you.
Sorry, i'm not really familiar with the T5600 or T5610.
Is there anything like this for SAS3?
There are SAS-3 expanders, but I don't know of one exactly like this. It's possible...
@@ArtofServer Just bought an aec-82885t, which can be powered by a molex plug and are affordable.
So you can use 2x 8088's / 8 SAS lanes for 24x HDDs? Sorry storage noob here :)
yes.
@ArtofServer tyvm I read up a little on expanders as well. You can for sure expect an order from me in the future
Appreciate you and your content!
Dose this setup work with regular SATA drives as well?
Yes
@@ArtofServer thanks. Unfortunately the first video I saw after this one was a guy using the same card for a Chia jbod expansion chassis. Really sad…
Oh well... :-(
@@mattiaippolito1625 The good news is that Chia is sort of dying, the fact it eats SSDs makes it not worth the money investment it currently has so that market is fully crashing which is great!
Great for those of us who wants to make a proper NAS.
I got my SAS expander for 65 USD after shipping, I thought that was a-ok price
Lenovo 03X3834 LSI 6Gbps 16 Port PCI-E SAS Expander Card 9240-8i 9261-8i 11407-1
With inflation going on, these non-essentials are starting to come back to normal pre-chia prices! :-)
1M might be too small of the block size of dd, it could be simply choking the CPU instead of capping the drives
You think 1M blocks are too small? really? I mean, a lot of people are still running 4k block benchmarks... that's 1/256 times smaller...
@@ArtofServer Well, If macbook shows pitiful results when copying to a USB 3.1 flashdrive with 1M and something more adequate with 32M, then that's the realworld performance that matters anyway
How do you source the specs for the power / plugs etc as you had to do on this one?
It's easy to make an adapter molex cable... yet knowing the difference is there is the essential part.
Dell doing it different is a deliberate dick move. They are aware of the standards and still choose not to use proprietary connectors.
I would be cautious even suggesting it saves them money to use the generic connector when they do given their economies of scale.
You said you would have them. HaHaHa yes I get it. Anymore coming?
Long story, short, I won't be getting many of these any time soon.
Nice meme 7:27
Lol. Thanks for watching!
Skip to 11 mins for the start of topic.
Thanks for the tip
10:53 "Go to my store. I have them in my store" - I went to your store. You did not have them in your store. 😞
I just had 1 in stock, but it got sold.
Don't get so miffed about the power connector! Why would they use anything but the cheapest, easiest to source, off the shelf components? That's why they use them! Someone would get fired if production were shut down because they ran out of some other connector.... like maybe during a global supply shortage during a pandemic! No tier 1 enterprise hardware manufacturer is ever going to factor someone trying to use other than factory parts for a specified application into their business model, so this is really a non-issue on their side.
raidz v. draid thoughts?
Not sure how that relates to this vid, but I think draid is awesome.