so facebook is the main reason Cooper Lake wasn't cancelled altogether? Also, Facebook rushing this out and going with cooper lake instead of waiting for later process node makes the Ice Lake SP situation seem a bit dire
@@ServeTheHomeVideo yuuuuup ! www.anandtech.com/show/15631/intels-cooper-lake-plans-the-chip-that-wasnt-meant-to-exist-dies-for-you Cooper Lake was to be canned and the only reason it didn't completely end up in the trash is Facebook wanted it (for some reason ?)
Is Cooper Lake the architecture that was going to include that 56 Core, 400W pressure cooker? Does that thing actually exist, or is Intel still planning to make it if it doesn't?
No consumer can typically buy any Xeon until it becomes obsolete for the enterprise and is put on the used market, or is one of the lowest of the low models.
Look for qualifying sample (QS) parts on eBay. They're final pre-release parts that are used by OEMs to validate their systems, but they're identical to retail parts aside from not having production fuses blown internally, which sometimes even gives you unlocked clocks and multipliers. They show up as genuine Intel parts and work just the same, including features like AMT and VROC. I'm running a pair of Xeon 8276L QS processors in my workstation and they were almost one tenth the sticker price of retail parts.
Guys day Patrick and STH team, I enjoy your content as always. I was considering purchasing Mikrotik CRS328 poe+ switch, can you please do a review/online review? I just bought quad nic for a pfsense build with my unraid server, would it be possible to use mikrotik router os with pfsense? Or I will I be limited to use switch os only? If I can, can you give some guidance please. Thanks in advance. Regards Shane from Trinidad.
I use 328 switches in production. poe and swtch features work great but Vlan on routeros just plain sucks. but you can run routeros or switchos on 328 series with ease. whenever i need vlans, i go for switchos. i try not to use CRS series as routers. We have RB4011 for routing, that is a beast.
@@PatrickKennedy2 Thanks Patrick, much appreciated. I'll hold strong until your review before i pull the trigger. Provided you get the switch in a timely manner you have an idea when you might publish the review? (rough estimate, i won't hold you to it.lol) Thanks in advance.
That's the unit of measurement that server racks use, and so do many form factors in computers. Its unfortunate, but that's they way it is. I would like for things to be metric, but you can't call a standard what it isn't.
Electronics is designed using combination of inches, mils and mm... it's been that way for few decades soon and it's not going away. Every SMD passive out there is defined in inches or mm and it depends what you get from what manufacturer, most SMD chip packages are defined by mm, but the inch equivalents are there because that's just how it is... yeah... conventions are hard to break.
Don’t see there need to use the 21” racks as it makes for extra cost in special hardware mounts and as you don’t gain as much going width bigger it’s better to go depth bigger. Also makes for better cost overall . Selling such non-standard hardware size reduces the money you get in the recycler cash offering.
As far as I understand, it's better from a hyperconverged perspective. The chassis are wide enough for the CPUs and memory to live next to each other rather than offset, leaving more room in the rear for add ins of various types.
Probably have an existing relationship with Intel when it comes to manufacturing server platforms, and it's likely that their existing in-house expertise is also heavily geared towards Intel equipment.
I would bet money the intel Facebook conversation went something like this "Facebook : We are EOL on our servers waiting on your processors that are "coming soon" we can no longer wait If you don't produce processors for us were going EPIC INTEL: !@#@#$ .... what can we scrape together.... we lose 80% of the die but we can't afford to lose another customer ..... well recan an older processor tell them it's new and they will never know "insert evil laugh Facebook looks at contract and facepalms
There is a good dive into azure by Mark Russinovich showing all this good stuff. They are designing air, liquid and submersion solutions for cooling crazy 1.2kW GPUs too
i wonder why facebook didnt choose AMD over intel since the AMD server systems blow intel away on every metric it cant be that facebook chose intel simply for the optane memory support? i would be surpised if they did because i would rather have way more performace for much cheaper in a smaller server space then over priced intel shit anyday.
Facebook is almost an exclusively Intel shop. They are one of the only cloud providers not listed in NVIDIA's A100 release. They got Intel to buy Habana labs.
It would have made more sense for Facebook to have several design teams where each have systems, software, and network engineers. A Facebook exec could say to the head of each team: I want (at least) this much performance, at this power density, in this form factor, in this quantity, and by this date. Each team would have to work with their respective CPU vendors (Intel, AMD, Arm, RiscV, etc.) to manufacture and test the systems. Each of those CPU companies would fall all over themselves to get the design win and supply Facebook for X number of years.
@Brad Viviviyal It's actually about software licensing costs. Intel has spent a lot of "marketing" dollars keeping everyone else away from the data centre space but that is changing. AMD EPYC servers can't use DCPMM but they can use Optane Nvme SSD's. I suppose when you only buy slow Intel servers, DCPMM is a godsend. "AMD kiddies", well that made me realize you probably have no argument, but when you mentioned Intel GPU then I knew for sure. The "data centre" is not a monolithic segment and neither is the solution space. From a risk management perspective putting all your eggs in a single basket is, well, risky. Even if it's an Intel basket. Nvidia's DGX A100 AI servers must not have needed Optane Memory since they are using EPYC. I guess Nvidia doesn't know what's happening in the data centre either. As far as Facebook goes, they probably don't pay much for hardware, it's all in long Intel support contracts.
Im guessing its because cooper lake has some good ai features baked in like bfloat16 support. I expect cooper lake to be used exclusively for ai researching purposes, not mainstream server workloads
And that why more and more people are leaving facebook? I mean FB was facing serious privacy issues adding the most insecure CPUs into the mix does not seem to solve any of that... but at least they can blame their next problem on Intel... 🤫
@@cortexauth4094 Most? Known or unknown? It is a server dude.... it should aim to have the least vulnerabilities, so it should start by having the less vulnerable CPU. Servers is where security is most important. Who cares if someone hacks 1 personal computer, but if they hack a server is like hacking millions of computers at once. Also "far better hardware protection" does not make the vulnerabilities go away, they already had that in place without the vulnerabilities in mind.
I wish both intel and Facebook all the good luck, I think the market is ready for the Epyc servers.
so facebook is the main reason Cooper Lake wasn't cancelled altogether?
Also, Facebook rushing this out and going with cooper lake instead of waiting for later process node makes the Ice Lake SP situation seem a bit dire
Some are of the opinion they can simply stop at "Facebook is the main reason for Cooper Lake."
@@ServeTheHomeVideo yuuuuup ! www.anandtech.com/show/15631/intels-cooper-lake-plans-the-chip-that-wasnt-meant-to-exist-dies-for-you
Cooper Lake was to be canned and the only reason it didn't completely end up in the trash is Facebook wanted it (for some reason ?)
@@ServeTheHomeVideo Made them an offer they couldn't refuse... because nobody else wanted them.
More like Intel made an offer they couldn't refuse
They called it Cooper Lake because of all the coper we’ll need to cool their aging architecture?
Brilliant comment
@Brad Viviviyal so you admit that intel is overpriced
Interesting that they found a supplier for the BMC, that used to be something they wanted to produce themselves as part of OCP
awesome content as usual!
Hey! It's STH's beardy brother! I knew I'd find you somewhere!
Are m.2 accelerators only ssds or does Facebook have something else they put in there
Is Cooper Lake the architecture that was going to include that 56 Core, 400W pressure cooker? Does that thing actually exist, or is Intel still planning to make it if it doesn't?
Any hope of more Epyc OCP servers in the future?
Wait, the boot drive is managed by the PCH and not the CPU?
No consumer can typically buy any Xeon until it becomes obsolete for the enterprise and is put on the used market, or is one of the lowest of the low models.
That's the point, Xeons aren't consumer platforms.
Look for qualifying sample (QS) parts on eBay. They're final pre-release parts that are used by OEMs to validate their systems, but they're identical to retail parts aside from not having production fuses blown internally, which sometimes even gives you unlocked clocks and multipliers. They show up as genuine Intel parts and work just the same, including features like AMT and VROC. I'm running a pair of Xeon 8276L QS processors in my workstation and they were almost one tenth the sticker price of retail parts.
@@gsuberland Not exactly a real answer to the problem that Intel just doesn't sell decent Xeons to consumers.
@@timramich I'm not sure I'm following you here. Are you talking about cost or availability? And what do you mean by "sell to consumers"?
@@gsuberland There is no retail method for buying decent Xeons. Go buy an engineering sample and you're SOL if something happens to it.
Guys day Patrick and STH team, I enjoy your content as always. I was considering purchasing Mikrotik CRS328 poe+ switch, can you please do a review/online review? I just bought quad nic for a pfsense build with my unraid server, would it be possible to use mikrotik router os with pfsense? Or I will I be limited to use switch os only? If I can, can you give some guidance please. Thanks in advance. Regards Shane from Trinidad.
On the todo list but we do not have one yet.
Just looked up the switch ETA 5/27.
I use 328 switches in production. poe and swtch features work great but Vlan on routeros just plain sucks. but you can run routeros or switchos on 328 series with ease. whenever i need vlans, i go for switchos. i try not to use CRS series as routers. We have RB4011 for routing, that is a beast.
@@PatrickKennedy2 Thanks Patrick, much appreciated. I'll hold strong until your review before i pull the trigger. Provided you get the switch in a timely manner you have an idea when you might publish the review? (rough estimate, i won't hold you to it.lol) Thanks in advance.
@@janisvaskevics93 Thanks Janis, your feedback is GREATLY appreciated, Take care.
"inches"? The 18th century called and wants it's units of measurement back. Use standard units of measurement, it is 2020.
That's the unit of measurement that server racks use, and so do many form factors in computers. Its unfortunate, but that's they way it is. I would like for things to be metric, but you can't call a standard what it isn't.
Height measured in 'U', housed in a 19" rack that is inside a cabinet that's measured in millimeters .... don't ya just luv standardization!
Electronics is designed using combination of inches, mils and mm... it's been that way for few decades soon and it's not going away. Every SMD passive out there is defined in inches or mm and it depends what you get from what manufacturer, most SMD chip packages are defined by mm, but the inch equivalents are there because that's just how it is... yeah... conventions are hard to break.
Inches is a standardised unit of measurement.
Don’t see there need to use the 21” racks as it makes for extra cost in special hardware mounts and as you don’t gain as much going width bigger it’s better to go depth bigger. Also makes for better cost overall . Selling such non-standard hardware size reduces the money you get in the recycler cash offering.
As far as I understand, it's better from a hyperconverged perspective. The chassis are wide enough for the CPUs and memory to live next to each other rather than offset, leaving more room in the rear for add ins of various types.
Oh hell yeah!
Why aren't they using Epyc Rome CPUs?
Probably have an existing relationship with Intel when it comes to manufacturing server platforms, and it's likely that their existing in-house expertise is also heavily geared towards Intel equipment.
Cooper lake has some niche ai processing features. Its probs gonna be used by facebooks ai researchers rather than for typical server workloads
"Bribes" in the silicon valley jargon
Contracts.
Because they want to use Intel's persistent memory. It's not compatible with AMD.
21" Racks eh?
#WideBoys
I would bet money the intel Facebook conversation went something like this "Facebook : We are EOL on our servers waiting on your processors that are "coming soon" we can no longer wait If you don't produce processors for us were going EPIC INTEL: !@#@#$ .... what can we scrape together.... we lose 80% of the die but we can't afford to lose another customer ..... well recan an older processor tell them it's new and they will never know "insert evil laugh Facebook looks at contract and facepalms
Ya, I'd imagine Intel is if not giving these away, not far off, just to keep the business.
Do we know what hardware do others use ? Google, Azure, AWS etc ??? Would love to know ...
There is a good dive into azure by Mark Russinovich showing all this good stuff. They are designing air, liquid and submersion solutions for cooling crazy 1.2kW GPUs too
@@Wahinies Where ? I'd love to read it. Tx
Very very very great explanation.
Big LIKE
Big SUBSCRIBE.
Scaleable - Like a Fish.
And it's smelling about like one now too.
what is their obsession with lakes
It rains a lot in Oregon where Intel has a big facility. Lots of rain means lots of lakes.
i wonder why facebook didnt choose AMD over intel since the AMD server systems blow intel away on every metric it cant be that facebook chose intel simply for the optane memory support? i would be surpised if they did because i would rather have way more performace for much cheaper in a smaller server space then over priced intel shit anyday.
Facebook is almost an exclusively Intel shop. They are one of the only cloud providers not listed in NVIDIA's A100 release. They got Intel to buy Habana labs.
@@ServeTheHomeVideo I suppose you can do anything you want if you don't have to pay taxes. Self imposed vendor lock-in, interesting.
It would have made more sense for Facebook to have several design teams where each have systems, software, and network engineers. A Facebook exec could say to the head of each team: I want (at least) this much performance, at this power density, in this form factor, in this quantity, and by this date. Each team would have to work with their respective CPU vendors (Intel, AMD, Arm, RiscV, etc.) to manufacture and test the systems. Each of those CPU companies would fall all over themselves to get the design win and supply Facebook for X number of years.
@Brad Viviviyal It's actually about software licensing costs. Intel has spent a lot of "marketing" dollars keeping everyone else away from the data centre space but that is changing. AMD EPYC servers can't use DCPMM but they can use Optane Nvme SSD's. I suppose when you only buy slow Intel servers, DCPMM is a godsend. "AMD kiddies", well that made me realize you probably have no argument, but when you mentioned Intel GPU then I knew for sure. The "data centre" is not a monolithic segment and neither is the solution space. From a risk management perspective putting all your eggs in a single basket is, well, risky. Even if it's an Intel basket.
Nvidia's DGX A100 AI servers must not have needed Optane Memory since they are using EPYC. I guess Nvidia doesn't know what's happening in the data centre either.
As far as Facebook goes, they probably don't pay much for hardware, it's all in long Intel support contracts.
Im guessing its because cooper lake has some good ai features baked in like bfloat16 support. I expect cooper lake to be used exclusively for ai researching purposes, not mainstream server workloads
What happened to James Damore???? I thought FACEBOOK was supposed to be neutral but went totally left.
It's quite baffling for them to choose aging architecture... oh well, it's their shame.
Facebook.... SENSORS!?!? *gasp*
Just kidding. No one is surprised.
And that why more and more people are leaving facebook? I mean FB was facing serious privacy issues adding the most insecure CPUs into the mix does not seem to solve any of that... but at least they can blame their next problem on Intel... 🤫
most hardware vulnerabilities are mitigated on server because of far better hardware protection. Stop senseless fanboying
@@cortexauth4094 Most? Known or unknown? It is a server dude.... it should aim to have the least vulnerabilities, so it should start by having the less vulnerable CPU. Servers is where security is most important. Who cares if someone hacks 1 personal computer, but if they hack a server is like hacking millions of computers at once. Also "far better hardware protection" does not make the vulnerabilities go away, they already had that in place without the vulnerabilities in mind.
hope Facebook gets hacked using Intel CPUs
It's really only a problem on shared systems, Facebook doesn't share their servers with anyone
@@GW2_Live it's a problem with VMs a lot of times as well
That's very mature