Dell PowerEdge R510 | How to add 3x GPUs!
Вставка
- Опубліковано 24 вер 2020
- In this video, I'm going to show how to install 3x GPUs into a Dell PowerEdge R510 server! The R510 with 12x3.5" backplane is one of the best storage servers from Dell's 11th generation PowerEdge servers. Naturally, many people use this model to build their NAS or PLEX servers. And if you're using PLEX, you'll want to use a GPU for transcoding.
If you're building a R510, and want some of the items I used in this video, you can find them in my eBay store:
1. Dell R510 PCIe riser with open slots: ebay.to/3eCu6HP
2. Nvidia Quadro P400 GPU: ebay.to/2OtkWmc
3. Dell H200 flashed with LSI IT mode firmware: ebay.to/2N6PCZT
If you'd like to support this channel, please consider shopping at my eBay store: ebay.to/2ZKBFDM
eBay Partner Affiliate disclosure:
The eBay links in this video description are eBay partner affiliate links. By using these links to shop on eBay, you support my channel, at no additional cost to you. Even if you do not buy from the ART OF SERVER eBay store, any purchases you make on eBay via these links, will help support my channel. Please consider using them for your eBay shopping. Thank you for all your support! :-) - Наука та технологія
![Dell PowerEdge Tips | How to mount SATA HDDs and avoid this common mistake! [11G,12G,13G PowerEdge]](http://i.ytimg.com/vi/Y9PBMaS9KpI/mqdefault.jpg)
![Dell PowerEdge Tips | How to mount SATA HDDs and avoid this common mistake! [11G,12G,13G PowerEdge]](/img/tr.png)
I tried doing this mod without the drill press and tried a by hand late night mod - well, pcie plane has a few undocumented holes in it now and This channels owner is saving my bacon with his last modified plane via UA-cam.
You sir are an amazing human being and saving me from the wrath of my family who don’t have Plex as a result of my hubris. Kudos sir. Kudos.
Thanks for your kind words! I appreciate your support 🙏
One of the most thorough, easy-to-follow, clear and concise videos I've watched in a long time, and I don't even have an r510 with which to try this!! At present, I have plex / Emby running on a 310 tower but am in need of extra bays for more storage.
I have found an r510 going for buttons and am (understandably) curious, but the clarity in which you put this solution over, coupled with the helpful hints along the way and your step back at the end to point out the issue with the lower card, has helped me to decide that the r510 poses a good solution to my growing media server dilemma.
Now going to investigate other videos you've posted - if for no other reason than to enjoy the clarity you provide.
Thank you! 🙂
Glad you found this video helpful! Thanks for your compliments and I hope you'll enjoy some of my other content as well! Thank you for watching!
Looks great for Plex + some VMs. On your channel I can see every "crazy" idea I have with this servers before buy one of them. Thank you so much! Please come to open an EU ebay franchise for your shop :P
Thank you for watching! There will be a series of videos covering the R510 in the future! I wish I could open local operations in the EU, but not there yet. Still need to grow more locally first...
I did this for my R720 with a Quadro K2000 card for my Plex/Emby setup and it works great, now to replace my two remaining 710's. Except mine had x16 slots in it so no need to fiddle with the slots.
Yeah, R720 is nice and already has the x16 slots. The R720XD evolved from the success of this R510/12-bay model... If you don't need a lot of local storage, checkout the R620 too... they are super quiet! (quieter than R720)
oh Yes, I really enjoyed this Video and it was very useful for me. Thank you very very much for your videos.
You're very welcome!
Perfect I'm needing to go from a x3550 M4 to something simple and all in one box and this ticks all of my needs
Cool! Glad this gave you some ideas... thanks for watching!
De-fan the p2000 will fit 3
Fantastic ! I'd been putting off deciding what to do with the 510, thinking just 'storage fileserver' and there you come along with this ! So then I'm on to eBay for 2 Quadro P600's and other stuff (the x16 converter mentioned below, and a bid on a K2200), it's just a shame the US carrier charges are so unreasonable to the UK.
Just one query though - have you considered whether AMD based GPU's would work better with the AMD CPU's in say the R715/815 ?
Perfect server, i have similar r515 on opteron cpu, excelent for truenas with additional 10gb lan
yeah, i love the 12-bay format in 2U for storage servers!
I really enjoy it.
Thank you! Glad you enjoyed it!
This machine is actually my most favorite storage server- only using h200 with default Dell firmware and Windows server storage spaces pool for iscsi for other VM host machines :-) I would never thought about gpu, but definitelly some PCIe storage options.
Indeed, it's one of my favorites too!
I would recommend using the LSI firmware on the H200 though... Dell's firmware crippled the queue depth on the SAS2008 chipset; the hardware is much more capable than Dell wanted it to be for a low end card and so they artificially crippled it's performance.
@@ArtofServer❤
I was thinking of getting an r510 but I have been hearing that the pci riser only supplies about 25 watts of power which I think would eliminate a bunch of different desktop cards that might otherwise just work off of pci power. Can anyone confirm this is the case? I would love to use a spare single fan 1050ti I have but I am worried now about power delivery.
nice work, i ended up getting a riser cable, x8 to x16 and coverting my P400 to low profile bracket and pluggin that in. But i can ask your thoughts on amount of power per PCIe port. Is there enough to power 3 cards? i read somewhere that each PCIe port is about 25w of power and the P400 is a 30w card.
Hey, that's a nice idea!
The 3 GPUs seem to work okay, but I did not put a heavy load on them. The PCIe spec has power limits based on the width of the PCIe bus, and there's also a "high power device" mode that allows GPU to pull up to 75W from a x16 slot. I don't recall the spec right now as it's past midnight and I'm about to fall asleep.. :-) But a search should find for you what the PCIe specs say about power.
I just bought R520 and I need to put a video card for Plex. What do you recommend me. I’m very new to this. I bought the server for 2K Canadian, 38TB, 192GB RAM, 2Xeon 24.. 12 core. Was that a good purchase or soups I sell and built a new server. I’m only using Plex
OMG perfect timing, i just got an R510 and have a NVidia P600 on the way for it!!!!!!!
Which one did you get? the 12-bay model? It's one of my favorite storage servers.... It's like I read your mind, huh? :-D
@@ArtofServer yes the 12 bay model... but i guess you already knew that since you can read my mind..
@@ArtofServer BTW I dont plan to cut my PCIe slots yet, i'm trying this first www.ebay.ca/itm/NEW-PCI-E-Express-8X-to-16X-Adapter-Riser-Card-Durable-Without-Extended-Cable-L/253569027613?hash=item3b09e44e1d:g:BL4AAOSw~O1a1vmn since the P600 is half height card.
@@SpazHeadSlotVideos Hey, I'd be really interested to hear how you get on with that - I've never seen one of those until just now, but it seems like a genius solution to getting round the shortage of x16 slots all over the 11th gen range ! Especially as the riser board mentioned in @Art of Server's R710 series costs over £100 (GBP) when you can get one...
@@rufusmurphy9990 thanks, hoping the adapter height will be just perfect so that i can use the half height bracket, if not will have to bend the full size bracket, or just remove it.
Thanks for your videos! I keep meaning to ask, what color scheme/theme are you using in your CentOS terminal? Would you mind sharing? I just like the way it looks and I have not found a comparable one through searching.
Hey thanks! I'm using MATE terminal, and I just picked the colors out of the standard palette. The text foreground color is just a shade of green called 'palette entry 11'. The background is just black and I turn on transparency.
Could also 3D print a PCIe cover of that bottom card.
I just got a 1050ti for my R520. Excited to try this out. I am concerned with the system using the 1050ti over the integrated gpu. Should I need to change any settings in the bios? or will installed the the card and it's drivers be enough?
turns out I didn't have to do any modding on the riser. 8x card and the riser slots are 8x. :D
Do you think drilling into PCBs will be an issue? I think a manufacturer had an issue with this and ended up recalling everything due to fire. They had to fit a piece onto a PCB, but instead of remanufacturing them, they just drilled a hole on the other side and shipped it. The drilling exposed metal and allowed a short to happen to cause a fire. Not sure if it was ASUS or NZXT. It was on Gamers Nexus channel and they explained how it happened. It may have been for a cooler or psu.
No, definitely do not drill into the PCB unless you really know what you're doing. There's usually a ground layer in there and you don't want to expose or make contact with that. My modification only cuts open the plastic of the PCIe slot.
Cheers! AOS I just picked up an 8 bay poweredge T620 and put a gtx 960 in it and it works amazing. I used a sata to 8 pin and it works great. However I would like to replace both my sons gaming PCs with VMs run off this machine. I would like to put two 1660 ti cards in it however I'm not comfortable pulling that kind of juice off the sata power. They make a gpu enablement kit however I have not found a kit with the cables and board needed and want to make sure I get the correct part. Have you had any experience with this? Am I going to run into any issues with the pascal cards? I have 2x 2680 v2 in my cart and 128gb ram. Any suggestions would be great!
I've never done something like that so not really sure. Don't even know the power draw on the 1660TI and whether you'd have enough in that T620.
Hello Nice video. R720xd are more easy in this case I think and have pci 16x
Indeed, very true. The 12th gen servers were a big improvement over the 11th gen.
I have a question regarding the R510. Would it be possible to add the LCD screen from the 8 drive model to a 4 drive model of the poweredge r510?
Sorry, no idea.
I guess I'm struggling with using 2 x 30W cards and 1 x 75W card in three slots rated for 25W max. You've shown it's physically and electrically possible to do this BUT, what are the long term ramifications to the server by running these slots over their power rating. I could *perhaps* see running the P2000 by itself as 25W x 3=75W, so at some base level the entire riser card (and the slot *it* plugs into) should be able to handle 75W. Any thoughts to the long term viability of this?
The P400 cards are very low power. The P2000 for video transcoding use doesn't use more than 25W that I know of and it probably can't request high power PCIe mode either. So, I think the riser should be able to handle 25Wx3.
Will this machine take Server 2019?
If I added 3 video cards to this server, could I dedicate one of the cards to a specific VM, one to another VM, one to Plex, and have the onboard shared over the other VMs? I'm completely new to this but recently bought this same server and H200 IT Mode. My goal is to have a storage server that can also run about 6 or so VMs.
You know, I've never thought about passing the onboard Matrox to a VM... not sure if there are issues doing that. That aside, yes, I believe you can pass those other GPUs to separate VMs, or dedicate them to the host system, etc. I believe R510/12-bay does support VT-d and other required features to do PCI passthrough.
Does the PCIE slot supply enough power for the Quadro P2000? I thought the PCIE slot was limited to like 75W of power?
Well, since the P2000 is working, I would say yes, although I have not pushed it to the power limits. I believe the riser can supply up to 50W.
Hi - i wonder if you could answer a question for me.....if you were to fit a gpu - when your logged into iDrac can you still see graphics...the reason i ask is that ive got an ML360 tower with a p2000 in there for plex work and with the nvidia card in there i dont get any remote desktop inside ilo....i think ilo only works with the on board card - i can see bios stuff but when its passed over to the graphics card to start windows i loose my graphics at that point....nice if the dell would do that....cheers.
there might be a setting in the system setting menu to choose which gpu to use for console output. check to see if you can manually force the setting instead of "auto" which might not get you the results you're looking for.
@@ArtofServer thanks... Good point but... The question was directed at the dell as I may upgrade to something a bit more meaty. l
I’m in the market for one of these for my father in law to upgrade his UnRaid build. I know the unit I’m looking at is an H700 card and I will
Need the H200 and I do plan to add a P400 for him. But my question is, how much RAM should I get? I plan to also setup Pi-Hole, Radarr, and Sonarr for him but it’s mainly for Plex
If you're not on an extremely tight budget, considering that DDR3 ECC RDIMMs are so affordable right now, I would consider maxing out the RAM (I think 128GB officially, 256GB unofficially) and just not worry about upgrading later on.
If you're on a tight budget, well, then you need to add up all the RAM requirements of every application you plan to use and I would at about 20% on top of that and round the result to the nearest multiple of 16 or 32.
Great video! Please make a video of like a R730 with dual GPU and usb cards for virtual machines :)
Do you have a R730 with USB cards and dual GPU?
@@ArtofServer Not at the moment but got some really itchy fingers on buying one :) Would make a really nice multi purpose machine, NAS, vm host for gaming but from what i can read there problem with the usb part of things. Planing on having 1 server 2 gaming machine at home but need some pcie usb cards to passthrough.
Could go with a supermicro whitebox but whats the fun in that and also R730 is a great machine and only 2U. Maybe you had a R730 laying around and could try out, break some myth and so :)
@@linusgrip6400 I don't have a R730 around, but I do have some pretty awesome Haswell/Broadwell generation hardware. There will be some videos on those things in the future...
@@ArtofServer Looking forward to that :) But from what i read no poweredge R-machine play nice with pcie usb card so if you're doing some videos and have a card laying around could give it a try. As always awesome video!
That would be very boring as the R730 is designed for two dual height GPU's or four single height. You can even fit them with the latest A100 cards, and they just work fine. Got quite a few at work, they are going to be part of a Kubernetes cluster to run a JuypterHub system. The only interesting part is that on stock Dell power cable (9H6FV) the plugs for the GPU itself don't fit the latest cards from NVIDIA, but I just cut them off and attached the right ones, thanks a bunch NVIDIA. You also assuming you are using a proper GPU card need Dell part 320-0867-001 which is an extension bracket that holds the end of the card securely, and helpfully NVIDIA have changed the position of the mounting holes on at least the A100 from memory, I think it just works with the V100/P100 but I could be wrong. I ended up making a jig to drill a whole bunch of brackets. The documentation also says to blank off the half height slots with solid covers to maintain airflow over the GPU's. I note that in the video he is fitting cards with fans in a server, generally a bad idea.
Here's the goods on R730 with a random spare P100 for testing
[root@kube01 ~]# dmidecode |grep PowerEdge
Product Name: PowerEdge R730
SKU Number: SKU=NotProvided;ModelName=PowerEdge R730
[root@kube01 ~]# lspci | grep NVIDIA
04:00.0 3D controller: NVIDIA Corporation GP100GL [Tesla P100 PCIe 16GB] (rev a1)
I have a 1U PowerEdge which just sits on the rails. Do you have rails that you recommend to simplify sliding the server out?
Get the appropriate Dell "Rapid Rail" kit for your particular model. Some "short" servers may not have such an option though.
Can you use multiple GPUs to transcode for increasing streaming or improving transcoding? Or can you only use one GPU to transcode?
I think that's software dependent. If the transcoding software is aware of multiple GPUs and can use them, then it shouldn't be a problem. If the software is naive and assumes only the 1st GPU, then it cannot.
This may be well out of date but I am wondering if H700 perc card can be used in this r510 to accommodate 6 TB STA drives? Thanks for any information.
Yeah, of course, you can use a H700 "integrated" in the integrated slot. Not sure what you mean by "accommodate 6TB STA drives?" You don't need the H700 to use 6TB SATA drives...
@@ArtofServer Sorry I wasn't clear, I just wanted to make sure the r510 can use 6 TB HDDs. The real reason Im looking at the 510 is my 2950 finally died.
on the H200 card what this power pin cable is needed for that you unplugged?
Supposed to be for HDD activity LED...
will this just work for quadro cards? or could I use an nvidia with 6gb or more for video transcoding?
Any GPU should work, as long as it fits, and the power requirements are met.
How noisy does it get? I’m curious because the more amps pulled, more fan spin you get.
Great question! Subjectively, it was no louder than the R710 I had before .. so you can see the R710 video to get a comparison. I do plan to make a series of videos on this R510 in the future, and will have one that covers noise, power consumption, and temperatures.
Can you let us know how well the transcoding works with one, two and the three GPU's installed?
when I have the time, I'll check that out. plex uses ffmpeg for transcoding, and i'm not sure if that supports multi-GPU? anyone know? I think things like hashcat support multi-GPU acceleration... not sure about ffmpeg, and specifically the ffmpeg plex uses.
@@ArtofServer Have you had a chance to check the transcoding with Plex? thanks
I just got my r510. What would you say is the most bang for the buck gpu as of today?
That really depends on what you want to do with it. I really like the P400 for video transcoding when all you need is a light to moderate GPU load.
how loud is that R510 running 3 high power draw cards ? every time I add a non Dell card in any of my Dell boxes, the fans crank up to 100% and never quiet down
The P400+P2000 are not really high powered cards, so this situation is probably different than yours. But with those cards installed, the server was no louder than the R710 I had previously; and you can see the details from the R710 video series.
The 12th gen PowerEdge servers are a lot more sensitive with fan control. And honestly, I'm convinced the algorithm is bi-polar...
no canned air?
HI there@
! i wonder if someone can help n me on what i m trying to accomplish . I had an r520 that i used for farming chia plots, now chia has a new option to compress and decompress online the plots an save space using an gpu. I ve a 3070TI card that does not fit on the r520 enclosure so buying an external gpu enclosure to fit the card and adding a thunderbold pci express will fit my needs ? I read the hack that is posted with r510 but 3070ti does not fit. Thanks
I think you'd be better off just getting a GPU that will fit. Modern GPUs exceed most PCIe form factor dimensions and they are just not going to fit without modifying the cooler of the GPU.
Just bought R520, did you finish your setup. I’m only using mine for Plex and need to buy a video card. What did you do
I have the same issue. Which tools have u used?
Sorry, not sure what you're talking about or asking?
A tip for beginners, Never work with metal hardware above the server at anytime when power is on. Good habbit, don't do it at any time as old Murphy and his law states that a bloody screw will fall in and magic smoke can leak out of components.
As for opening slots, an old Soldering iron tip will allow you to melt platic and be a lot safer with regards to snagging a brass pin.
That's generally a good tip to avoid accidentally dropping metal on a live motherboard. I've done it a few times myself! LOL Although, most modern server motherboards will immediately shutdown if it detects any kind of short so I've survived those accidents.
I've tried several methods for opening PCIe slots. The melting plastic method(s) just don't give very clean results. This method has given me the best results so far and I've improved upon it by using a 1.8mm milling bit now instead of 2.0mm. I have never snagged a pin with the milling bit, but rather when I'm cleaning up the slot with the hobby knife is where I can get in trouble.
Thanks for watching and sharing your tips! :-)
@@ArtofServer Some systems will self-protect but others, nah, smoke is quick to release.... lol
Melting plastic is only half, a sharp knife like you use and suggest still works best for cleaning up. Die grinders and cutting discs, yeah, had a few clients who had brought there machines to me for these mods and ended up needing new boards or risers.
Can anybody explain to me the purpose of 3GPU on my R510? What is needed for? Plex works good with onboard VGA. Thank you!
PLEX cannot accelerate transcoding with R510 onboard VGA. Installing GPUs in servers is mostly meant for GPGPU applications, like PLEX transcoding being 1 possibility. Also, some people also pass the GPU to desktop VMs if they want better graphics performance in a VM, such as light gaming.
@@ArtofServer❤
0:25 🤤drooling about storage 🙂
LOL. Welcome to the channel! :-)
What about performance limitations running at x8 or x4 instead of x16?
Good question, but depends on what you're doing. GamerNexus has shown that x8 vs x16 makes almost no difference for gaming. If you're doing just video transcoding, the x4 is enough, as you're not going to push more than 2GB/s of video over even a 10Gb NIC. I'm sure some GPGPU applications might push the bandwidth limits, but a lot of use cases don't require the full PCIe bandwidth of x16.
Awesome. It'd be just for plex hardware transcoding. Planning on p400 in x8 slot and 10gig nic in upper x4 slot. Already have h200 in storage slot
@@DarylClose did you get this going with R510 and Plex Transcoding with a P400?
Can I send you a GP347 riser board to help me with slot opening?
Isn't the GP347 already a x16 slot?
Hmm, don't those get a bit toasty if you use them? :)
Not really.
Will this riser board work with the 8 Bay r510? Been looking for 12 Bay ones to buy but that'll break my bank. Thanks in advance
That's a great question. I don't know for sure... but I suspect it will. The R510 PCIe slots are not different between the 8-bay vs 12-bay model as far as I know (but I don't always know what I'm talking about!) ...
@@ArtofServer that's no problem, I guess I'll know soon enough, I saw there's only one more board available from you, will you be modding more?
@Shaji Solaman yes, eventually I will make more if there is demand for them.
@@ArtofServer thank you very much
What do you think about Dell PowerEdge R710?
ua-cam.com/play/PL28eVGz5vFQ9hUBeeg5bxj-s0IeitoGIb.html
Can this riser card work on the R610
No, the R610 is a 1U server and has a different riser board. However, I might be able to modify the R610 riser for GPU too... is that what you're looking to do? Install a GPU in R610 server?
Yes
A by 16 gpu not by 8 gpu
Because I have a pci 8 to pci 16 conversion cable that plugs in and appears to work but nothing displays on the monitor
@Brad Wiebe I don't think it works like that. I think your GPU is working fine, but that doesn't mean the system will use it for video output since it already has onboard VGA video. Boot into the system BIOS, and see if there's an option to output to add-on video card instead of the onboard VGA. If there's no such option, then it may not be possible. In the use cases I demonstrate, it's mostly to use the GPU for GPGPU stuff like PLEX transcoding, etc.
welcome to 2021, where nobody can afford even 1 GPU XD
idk... hasn't it been that way since like 2019, then 2020, now 2021? ;-P