I Took a Risk on This Server - Can I Get it Fully Functional?
Вставка
- Опубліковано 2 кві 2024
- Hello there and welcome to my video. In today's video, I'm going to be messing around with, and eventually fixing this Dell R720 server I bought for cheap. This server has some issues with a subsystem that's not on desktop PCs, so figuring this one out was a little tricky!
Music, in order:
Beyond - Patrick Patrikios
Hanging Out - Bruno E.
Island Dream - Chris Haugen
Sunday Rain - Cheel
Sunny Days - Anno Domini Beats
I fixed the exact same iDRAC issue with UART about two years ago on an DELL R620. It still works to this day :)
The clips of the RAID controller breaking is an unfortunate issue with the plastic DELL used. But it's not a big issue, as the RAID Controller still sits firmly on it's quite large connector
The SATA SSD not being recognized in the OS is most likely due to the RAID controller. You need to enter the RAID Controller management BIOS and create a RAID0 for your SSD. You could also (probably) flash the RAID Controller into IT-Mode, so that all drives are passed through. Though iirc booting from these drives then doesn't work. So it's most useful if you want to use the drives in something like TrueNAS for Storage.
how did you fix the issue? Usually it's failing NAND. I have 10+ of the R620 motherboards here broken down for parts because of the iDRAC :) or did you do the "replace with sd card" thingy?
@@callmefoxie2950 iirc, I put the firmware.d7 file onto the SD Card like DELL said in their Help guide. But the flash didn't start automatically. I had to connect a USB TTL module to the UART pins on the motherboard. In the serial terminal I then had to press enter (and maybe type Y or YES) to start the firmware flash. Also, if possible, use a fairly small SD Card. As I also remember it didn't work on the first try with a 32GB SD Card. I used a 1GB Kingston SD Card in the end.
@@callmefoxie2950 Also be aware that the iDRAC Enterprise Licence will be lost after the flash. I called DELL and they kindly sent me the Licence file via Email. (Some others also had luck by using the DELL help forum to get their Licence back)
Oh right. This doesn't work for most people as the NAND is actually bad and requires a change. There should be a way how to change the medium to SD card but eh good luck selling server with that for any profit margin :)
There's some great info in this little comment thread that could be helpful for people in the future with this issue, thanks for sharing!
Anyone else feel the masculine urge to build a maxed out server rack with terabytes of RAM and petabytes of storage despite not need such stuff? No? Maybe it is just me... Either way, nice video!
I got an uktra masculine urge to build a server out of old computers, and I ain't talking about core i5 from 2nd gen intel, I mean like pentium 3 cpu running like alpine Linux or something just for it to save my half a gb wallpaper collection and maybe some vids
@@nicecarphill I feel that!
Thanks! Yeah, I definitely want to build a stupidly powerful server, even though I certainly don't need one! The urge to put 768GB of RAM in this one is real...
Yeah man, that Tim the tool man Taylor urge is definitely there, "Grrrrr... More power! grunts Hoo hoo hoo!
You are not alone.
Ram blanks are to protect from dust in the slots , which I have had plenty of issues with HP servers in dusty environments, however I like your reasoning with the blanks acting as flow stators , that’s good logic 👍👍👍
How about RTFM? iDRAC is encrypted! By design. The I-button is for 'identification' of the system in the rack, this LED you turn on and off FROM iDRAC when working correctly. You have never accessed the UART, as a simple ISP router? Or a SBC? Very similar to those scenarios to reflash the iDRAC. Almost every question you ask... is in the manual. Some Dell servers require DIMMs in pairs, and the mainboard senses if the slots are empty. So depending on the model, such as Dell 2900R for example. Most Dell 2U servers have 3 PS units and the server can run on 2 of 3, so the 3rd is a spare redundant in case of failure of either of the primary PS units. The noise from FANs spin up is NORMAL, until the temperature sensors validate the FANs are functionally normally, then the FANs will back off. This can be validated via iDRAC or BIOS. You need to find out in the BIOS, to see which RAID card is present. You cannot save configuration completely and consistently until the onboard battery is functional, as well. Many Dell servers also have an internal USB bport, so you could actually boot ESXi from a 8GB USB thumb drive since ESXi is RAM resident after initial load.
Those are just dust-cover's for the DIMM-slots.
Those fans can blow air pretty fast across the dimm-slots (which also means the slots can get pretty dusty).
Congrats on the new server. I've had one just like it running for a few years. The front SAS backplane will happily accept either SAS or SATA drives. I've had mine running consumer Samsung SSD's. the issue you're facing is that the server has the H710 raid card. this card doesn't support passthrough non raid drives. Two options here, either make a single drive raid 0 using the raid configuration utility you see on boot. or get an H310 raid card. this card still supports making raid arrays. but also will allow you to just plug in any old drive and get a more conventional hot swap functionality.
I remember sometimes you have to cover a lane on the SATA connector with tape for the drive to work - some kind of incompatibility.
Good work on the system board replacement, the small cables going to the front panel are notorious for getting damaged when the lid goes back on.
DIMM blanks are for airflow but do help keep dust from settling in the sockets.
The fibre channel SFPs will be outputting invisible laser light, so if you're not using them you should unplug the SFPs or put dust covers in them, or just remove the cards and put backplate blanks in the slots. The light will damage your eyes if you look directly into the SFP.
Yes, a corrupt iDRAC will almost always end up with a system board replacement if the server is under Dell support.
iButton and LCD are run by the Lifecycle controller and iDRAC so not surprising that they weren't working.
Thanks for the warning on the SFP's! I wasn't entirely sure whether or not they'd be outputting any light when there weren't cables plugged in, so thanks for informing me that they will.
To use the front 2.5in bays you first plug the drives in (ideally with a caddy) and then access the Dell raid configuration utility (it's CTRL+R on my Dells). From there you create virtual drives through the raid controller. Once you're done, still in the raid config utility you select the virtual drive the server is supposed to boot from and you're done.
Also, you may need to load the driver for the RAID controller, as Windows 10 may not include it in the setup.
Yes, when using raid you need to create the partition. I have r610 and t320 and before I had ibm x3550 and I couldnt figurate that out so I give up and sold the server now I older and have more patient and use the server as saas.
Came to the comments to say this, as I have deployed several similar dell servers for clients, and I sometimes forget to do this too. I Just picked up a R730XD which I will be setting up in my home lab to replace my HP DL360 gen7. I'm switching from ESXi to Proxmox for my new server, and will be running as my NAS, Docker host, as well as Jellyfin. I have plans to throw a P100 and a couple of NVMe drives to make a self hosted game streaming VM as well.
Interesting video, being a new tech myself I expect to see this sometime in the future. Loveto read all of the helpful advice from experienced techs in the comments.
Hey dude, the issue with the iDRAC is a very well known issue with 12th gen Dell servers.
Basically, during the 12th gen cycle, they changed how the iDRAC, Lifecycle Controller and BIOS firmwares worked. Before some specific version, all three were split into different firmwares that needed to be updated together.
However, at some point, the iDRAC and Lifecycle Controller firmwares were merged into one and thus, only 2 firmwares remained afterwards.
Problem is: if you don't go through the exact route of firmwares for both/all 3, you end up bricking the DRAC. There's a specific version that has to be installed first to combine the firmwares, and then you can normally update it.
Unfortunately, I've been there and did that exact mistake and it cost me a T620. That being said, I learned and got a R620 working fully after carefully doing the updates in the right order. (There's a Dell community post somewhere with the exact order to update.)
So yeah, that board was bricked. Luckily you got one for cheap, mine for the T620 was around 700$.
Yikes, having to spend $700 bucks on a new board doesn't sound fun, I'm sorry that happened to you. I had found a little bit of information regarding the iDRAC/LCC firmwares being merged into one at one point and causing issues, but your comment put it really clearly for me, thanks! I might just leave this board and its firmware alone at this point haha, no need to fix what isn't broken (at least for my basic use cases) and I'd rather not risk having to buy a new board. (I'm pretty sure I'd find a way to mess up the firmware flashes lol)
Thank you for an interesting video. Buying the new motherboard was a great step. You were able to get your hands into it. And it worked. I remember buying one in which only one socket as working, the second was with bad pins. I had to replace it. And it was a wonderful journey. The way you removed the old motherboard tells me you have read about it. Great 👍
Like it, very interesting first dive into rack servers, look forward to follow up.
The issue with the drive not being found on the server is that it has a RAID controller and you have to build a hardware raid array before anything will be seen. If you want to install Proxmox or TrueNAS on this server, you will need to flash the raid card to IT mode and also remove the battery for the raid controller.
Even if your Dell server appears dead as a stone, you can get the Service Tag and plug that into Dell's support site. It should at least tell you how the server was configured when Dell shipped it.
Lifecycle Controller is still hideous. Hopefully you don't have to reboot often because if you do, that stuff has to re-initialize EVERY reboot, even if it's warm. Obviously no one from Sun got a job at Dell (Suns have "diag-switch?" and "diag-level" settings and only switch to MAX if OBP detects a problem. It's comical that an E3000 from the 1990s can POST way faster than these Dells!)
Also, if you ever use SAS with this, the PERC RAID will probably give you trouble. Be sure to get drives that are verified to work on the Dell servers, or you'll find they're "failed" or otherwise unsupported, and you won't even be able to configure them in.
Well that display on the front working was great sign. I may have looked around at the component level for obvious problems like on board fuses and foreign objects before getting the new board. Probably made the right move.
I’m excited to see what you do with this
Great video!!! I have 7 rackservers they are all interesting/sometimes infuriating to work on but in the end so fun!
Great video! Interesting
looks like youtube actually recommended me something that i’m actually interested in ! you earned a new sub !
Thanks for the sub! I'm glad to hear you enjoyed the video :D
Hello, it’s already been said and it’s correct, you need to enter in the Perc BIOS with CTRL+R and configure a RAID but to be able to do it the boot must be set in BIOS mode, if it is in UEFI mode you can do the same thing through the LCC. You can also through the LCC deploy one of the server supported operating systems and one of the options in the deployment wizard is to create a RAID, be careful with this option though as if you have data on the HDD/SSD you are using , information will be overwritten. By the way, as a disclaimer, I work 4 Dell
Hey dude. The reason why your ssd doesn't show up, is that the raid controller doesn't pass the entire drive info to the OS like an HBA would. You'd probably need to create a virtual drive under the raid config or try and turn the raid card into a HBA. Awesome video BTW!
Thanks for the info!
lol I said the same thing in my original comment then noticed this one, so I deleted mine and am upvoting yours!
@@arcomador oh shoot. Don't delete yours! It's okay.
The BIOS is a mirror of iDRAC from what I remember. I also remember xmodem'ing a lot of firmwares to chips
Its funny I have worked with hardware for 10 15 years and I always enjoy people buying a server then learning on the oddities that is the enterprise server world
Ahh the iDRAC classic. You def can fix it using the UART. I'd say give it a go in the future.
The RAM Blanks is for consistent Airflow and to reduce Air turbulences.
In order for your server to see your SATA ssd you have to create a virtual drive in the raid controller. While booting you should see a option rom that you ctrl+r to configure, or while in system setup (f2) go to devices and configure from there.
Thanks for the video, good job learning, you paid a lot more for this stuff than it's worth, but it's probably worth it in education value :)
Yay ur channel growing again!!!!
As far as I understand, the RAM covers are to prevent dust build up in the slots not used. There is probably an airflow benefit though.
Really good video, looking forward to the next on this.
As a side note, perhaps its worth looking into the dell 7810 workstation, i have just one and its amazing but id love a better mind to assist in showing how to add raid function for 4-6 drives. Dont worry, they are not crazy expensiveabout 800 usd converted from my 1500 sgd. Take care and keep it coming.
Thanks for the suggestion, I'll absolutely look into it!
@@CompHwTipsAndTutorials Thank you for taking the time to read and reply
i love my t7810 i have dual e5 2699p v4s . yes those cpus lol and a 1300 watt psu to feed those two 200 plus watt 3ghz base cpus with22 cores and 44 threads with a turbo of 3.8 ghz all core combined they draw 400 watt lol
@amdintelxsniperx dam that sounds awesome, any idea if I could put those into my 7810 and if so, would you happen to have a link to buy them on ebay please
@@armstrongskyview2810 yes you could but they will run hotter so grab some ptm 7950 pads and maybe look at a 1300 watt psu. Unless you have the 825 watt . Also eBay is your best bet . The cpus are over 200 bucks now I got lucky with mine as I traded some gear . Also if you do plan on going 1300 watt psu look into the modified power distribution board as it will allow 3090 4090 gpus
FYI: Red/orange tabs are removable when system is running, Blue tabs item must be removed when powered off.
That's cool, I was aware about Dell's blue plastic indicators on things that can usually be pressed on/slid to remove, but the orange/blue difference to make clear what can be hot swapped and what can't is something I hadn't realized yet. Thanks for sharing!
@@CompHwTipsAndTutorialsthe underside of the cover actually shows that too
Definitely try finding the raid controller driver for win10 when you try installing it again.
For your ssd you have to go into the bios of your raid card, because you have to configure a pool, that the server than can access
Former poweredge R720 owner here. and yes, I have been through a idrac death too, like you. the initial mobo is toast/unusable due to dead idrac...even if you don't use it its so critical to having a happy server. the reason why the BIOS options wouldn't save? 100% due to dead idrac. the fan calibration/power thing the replacement mobo did happens on configuration changes as it calibrates fan speeds ran by, you guessed it idrac. and those modules you showed? those do not have idrac on it, those are LAN enablement cards...basically to put idrac on its own Ethernet port...that's all they do
Idrac7 and up (R720 is 7) idrac is 100% on the mobo and the model does not matter. T620, R820...
I'm pretty sure the magic version is flash idrac to 1.9 then to 2.0..then you can jump it to the latest version and it won't brick; I recall I think it was 2.0.20 that was the version where it combines lifecycle and idrac. if you jump to latest...idrac dies and you are maybe trying UART method or replacing mobo.
older servers like the R710..those are not embedded and can be ran without a idrac card.
like others have said, these dell RAID cards require a drive to be in...RAID just inserting a drive will cause it to be unrecognized if you enter the PERC setup and pick RAID 0 then a single drive shows up fine...or can be flashed into IT mode, removes RAID and then it basically works like a SATA card...just with SAS capabilities (there is a guide that adds booting support it can be done ive done it) . the plastic clips on the RAID card are very fragile, mine snapped like a twig too lol
Thanks for sharing all that! You've given me some great information that I really appreciate
It's not very intuitive, but the reason you can not see the drive you're putting in the front SAS bay is most likely because you haven't configured it in the RAID controller BIOS. I didn't see you show this screen on the new mobo, but look at 11:55 in the video. When you see this come up you need to hit "C" to get into the configuration and wipe any old configs that are in there. I would recommend then setting each drive as stand alone if possible and marking each drive as bookable. Afterward, you will be able to set that drive as a boot device in the UEFI BIOS. You may even want to consider flashing the RAID controller to IT Mode which is absolutely necessary if you want the drive fully exposed to the OS.
was also gonna mention this
Thats most probably the reason why it didn’t work.
Thanks for the info! I'll give that a shot in the future
"Dell system updater" or dsu is what you need in this case find yourself a usb bootable live linux os install dsu ran and update everything and voila %90 its just going to be okay, about the ssd you need to configure it at raid 0 you can press ctrl + r during the "LSI PERC" firmware initialize. I still use r720 or similar devices in enterprise places they are pretty rock solid and cheap only downside is its not a electric friendly device which where the cheapness is coming from, with the bill of the datacenter you can get R740 or 730, still there are tricks you can use L series cpus and such, anyways good luck with your toy!
I have an old Dell server but I haven't gotten around to playing with it yet. I mean, there is nothing practical I could do with it - it has 8 Core 2 cores and burns about 750W. I used to work in aviation and I have heard small gas turbine engines run that aren't as loud.
It is far less powerful than my inexpensive modern gaming PC. It is likely the Sun Ultra 60 I also have waiting for me to get some time will actually be usable for something. A server for vintage games maybe. It has a Pentium MMX system on a card. I think it would be fun to have a vintage game streaming capability running on original hardware, though I might need a modern SBC or at least a fairly sophisticated microcontroller to make it work. It isn't like it was designed for that. It is powerful enough, surprisingly, to run modern FreeBSD or Linux. Shame it has taken so long for the world to take RISC CPUs seriously. Intel is running scared; they just decided to license their architecture for the first time in a long time. They are afraid of ARM and RISC-V. I think x86 is all but dead, regardless.
My only experience with enterprise hardware was an internship where, among other things, I helped roll out a brand new PII based network, lol. Things have changed a bit since then.
When starting the server, press ctrl+r when Poweredge expandable radid perc 710 is displayed and delete the old setting and the disk should be displayed after the next restart.
It's been mentioned before, but you need to configure the installed SAS/SATA drives in the raid configuration utility before you can use them. However, do note that not all solid state drives will work with this machine regardless because of firmware limitations on the raid card. I have two of these in operation, and some of the cheap solid state drives(such as silicon power) do not work.
Interesting, I might go out and buy some proper SAS drives that are known to work on this server then, as I'd rather not have to deal with a billion compatibility issues lol
@@CompHwTipsAndTutorialsSamsung 870s for the most part work with the H710p, as well as some Spark SSDs. Even then though, firmware on the drives can create problems because they aren't all using the same version.
Basically, when you install the incompatible drives they are seen initially. After configuring them, the controller is unable to see or interact with them, and claims they're either missing or failed. After rebooting the system, they're either offline completely or marked missing.
SSDs are much more power efficient than the mechanical disks, but with these machines it's always a hit or miss. The mechanical SAS drives can be had for very cheap on the used market, at the cost of increased power consumption and heat production.
Interesting, I might have to give an 870 a shot then. Do you know of any SAS SSDs that are known to work with the H710p? Or does this server predate those kinds of drives too?
@@CompHwTipsAndTutorialsUnfortunately, I don't. I've never used SAS solid state drives because they're often quite expensive. According to research though, the Samsung SAS SSDs as well as Kingston's mixed use drives are known to work. You can probably find a list online somewhere.
I saw you pull out the SD cards from inside the server. Did you make sure to put those back? One of those is most likely connected directly to the idrac. I have a r810 and my idrac has an SD card slot directly on it.
Those two SD Cards are only for OS boot like ESXI. Also on the R710/R810 the iDRAC works differently than on his newer R720. The SD Card on your iDRAC is used for vFLASH. it's used to install an OS on your Server for example. On the R720 the vFLASH slot is the one in the front he used to try to flash the firmware image with. So an SD Card is not necessary for iDRAC to work
Yeah, as retropctech2407 said, these SD cards aren't linked to the iDRAC in this server
@@retropctech2407 ahh okay gotcha. thanks for the correction! I knew there was a difference between idrac6 and 7, especially regarding licensing, but didn't know there was a big difference hardware wise
On some systems, you may have to configure the drives that you install using the LifeCycle controller. Worse, the jargon can be confusing even between slightly different Dell systems resulting in wiped drives when you really didn't intend to wipe them.
Dracs having issues is not totally uncommon; yours at least booted; I have one that is in production use, out of support, that hangs every 3 months or so during Windows Updates restarts; the DRAC works until it does not; and then the power plugs need to be pulled to get it functional again. Out of 8 servers like this, two had oddball issues; this is one of three that are still in use. Can't really work on it to even troubleshoot because the DRAC logs are useless ( no logs when it happens ) and I can't take it down but during the 4 hours or so that is slated for Windows Updates; so even if I could troubleshoot it, I could not get the parts and test it till the next month, but Windows Updates are priority over hardware maintenance ( until it finally fails completely ). It is slated for permanent removal in the next 3 months. 25% had significant issues; so, not really uncommon.
Yeah, from what I was able to gather these failing DRACs aren't a rare occurrence
I have a r720xd. You can flash the raid controller to be an HBA and it will work better with things like truenas or unraid. Its simple enough to do. Shoot me a dm if you need help with this beastie.
ExDatacenter guy here - Servers usually ramp up their fans for a short time on booting. I think this is to try an blow out dust.
I suspect it is more of a calibration thing because I have seen some really dusty servers that would debunk the dust theory. 😉
A fair few desktop computers seem to do this as well, though I've always heard that it's for the sake of testing the fans and making sure that they're spinning properly. I'm not 100% sure though, it could be for any number of reasons haha
You should flash your Raid card to IT mode that might fix your issue. How ever if you enter the raid set up on boot you will probably then see your drive as you would have to create a virtual image first to pass that along to your OS. But do you self a favor and just flash it to IT Mode. You remove the battery's then flash them I was able to do it so I am sure you could. Art of Server has a few good guides on this.
Thanks for the tip! I'll likely end up flashing the controller to IT mode soon.
I have a Dell R720XD and its populated with SATA drives. If SATA drives won't work with your R720 it seems that you still have a problem to find.
It seems that I just didn't know how to work with the RAID controller. I haven't gotten around to messing with it more since making this video, but I think I might just need to configure some things to get it all functional.
You need to get into the raid card to enable your ssd.
what you're working if you didn't work on IT and related field like software engineer or anything else ?
I don't really reveal any details of my personal life on my channel, but I will say that I am hoping to go down a path in professional IT in the near-ish future.
Just Send To Calbretro😂😉
Also the ram clips have likely just fallen off not broken off (had it happen to me they just clip back on)
I had that thought too, initially. However, in my case, the clips actually were snapped which is something I've not run into before! It's a shame that they broke like that, though they seemed in rough shape when I received the server. Maybe the previous owner was unreasonably rough with it or something?
i bought a r720 , off eBay, last year and plugged it in to my surge protector ( was new ) and the servers power supply's went pop, wasted £230 luckily only had to pay for shipping to get a refund. found out later from the seller that the power supply might have been damaged in shipping, but it killed the motherboard and 8 ram sticks :(
Oh man, that's no fun. Hopefully the return shipping wasn't exceptionally expensive
The idrac is not a part of the motherboard at all its a module that comes off its the small thing with a heat sink and 2 blue tabs on both sides
That's actually the RAID controller card, though it is located close to where the iDRAC is.
It is alarming that people are reporting iDrac deaths in the R720 servers. Those modules at 8:53 aren't iDrac modules they are just breakout dedicated interface ports for the iDrac. I wish iDrac was a separate module on the newer systems though because I scrapped a bunch of R420 and R320 with the same iDrac issues you are seeing on this R720. The problem from my understanding is the storage chips for iDrac are failing in the 12th gen servers causing these issues. Luckily it isn't needed for the server to work as far as I can tell. Mostly you lose remote management capabilities and the fans run at 100%.
Brittle motherboard plastics on the 12th gen Dell servers aren't too uncommon depending on their operating environment.
I'm no expert by any means but have a lot of experience abusing these R720. If you run into any issues I might have an answer. You've basically seen the worst of it so far. The only other hurdle would be powered GPU support. Also they don't support PCIe bifurcation limiting your options when it comes to NVMe drives.
Ah I see, I thought those iDRAC modules seemed a little bit simplistic and small. Makes sense that they aren't the full iDRAC, considering they didn't look like they consisted of much. Glad to hear that hopefully my experience with this server should be uphill from here, minus powered GPU support, as you said. Hopefully I won't run into any extreme issues when working with that!
I learned Windows NT on an old 486 x4 Proliant someone gave me for free.
Hook me up, I used to work with R710 & R720 a few years ago, ok?
That's cool that you've had opportunities to work with these servers!
Very old. Power hungry. Next.
Fair opinion to have, I'm still playing with my server though.
Low cost is the biggest reason for buying these old servers.
First
SAS drives do work with SATA controllers. You just need SAS-to-SATA cables
...👏🎉🎯💯👍💣💥❗
✨⚡⌨️🖱️🎮⚡✨
Also 4 the algorithm.