Managing 8 Rome CPUs in 1U: Cray’s Shasta Direct Liquid Coolingby Ian Cutress on November 19, 2018 5:00 PM EST
- Posted in
- Trade Shows
- Enterprise CPUs
- Supercomputing 18
*After posting this news, we have recieved additional information from Cray that states that the top plate is not being used for CPUs, but for the Slingshot interconnect. As a result, the system supports only 8 CPUs in 1U. The text below has been changed to reflect this new information.
The Supercomputing show was a hive of activity, with lots of whispers surrounding the next generation of x86 CPUs, such as AMD’s Rome platform and Intel’s Cascade Lake platform. One of the big announcements at the start of the show is that Cray has developed an AMD Rome based version of its Shasta supercomputer deployments.
Cray had a sizeable presence at the show, given its position as one of the most prominent supercomputer designers in the industry. At its booth, we saw one of the compute blades for the Shasta design:
It’s a bit difficult to see what is going on here, but there are what looks like eight processors in a 1U system. The processors are not facing in the traditional way for a server, in line with airflow, because the system is not air cooled.
The cooling plate and pipes connect to separate reservoirs that also supply additional cooling to the memory. Of course in this situation we’re not in a dense memory configuration, given the cooling required.
This display was being shown off in the CoolIT booth. CoolIT is a well-known specialist of liquid cooling for both the consumer and enterprise worlds, and they had a few interesting points on their implementation for Shasta.
Firstly, the eight CPU design with split memory cooling was obvious, but CoolIT stated that these processor cooling plates were designed to cool a CPU below and the custom Cray Slingshot interconnect above. We were told that with this design, the full cooling apparatus could go up to eight CPUs, several interconnect nodes, and up to 4kW.
The cooling loops are split between the front and the rear, recombining again near the front. This means that four cooling plates and memory will be pumping through in each loop. There is a small question about how the memory all fits in, physically, but this is certainly impressive.
CoolIT is claiming that its custom solution for Cray’s Shasta platform helps double the compute density over an aircooled system, offers 20% better rack-level efficiency, and up to 30% better facility-wide energy efficiency.
Shasta is set to be deployed at NERSC as part of its upcoming ‘Perlmutter’ supercomputer, slated for delivery in late 2020.
*This article was updated on Nov-20 with new information.
Post Your CommentPlease log in or sign up to comment.
View All Comments
HStewart - Monday, November 19, 2018 - linkI am curious how many Chipsets the systems has - it looks like to me it possibly have 4 or 8 chipsets on the same board.
SaturnusDK - Tuesday, November 20, 2018 - linkEPYC CPUs do not require chipsets at all.
HStewart - Tuesday, November 20, 2018 - linkSo the question is it seed as 8 desktop computers or one.
SaturnusDK - Tuesday, November 20, 2018 - linkNo matter how many of these blades you have they can be seen as one computer.
Duncan Macdonald - Monday, November 19, 2018 - linkEPYC has a SOC inside the package and does not require a traditional chipset. From a look at the pictures the visible PCB's are low density - possibly power supply (the bigger PCBs) and I/O (the smaller ones). Remember as part of a supercomputer, very few of the CPUs will have traditional I/O (disks etc).
HurleyBird - Monday, November 19, 2018 - linkQuad channel memory?
HurleyBird - Monday, November 19, 2018 - linkNevermind, optical illusion!
HStewart - Monday, November 19, 2018 - linkWhen I search for Cray Shasta - I see another CPU then these. So I not sure this is real information.
Ej24 - Tuesday, November 20, 2018 - linkShasta is just a model of available systems from Cray. Sort of like how Ford makes several different models of cars. Shasta systems can be configured with different cpu's and other options just like a Ford Focus can be configured with different engines, transmissions, etc. Rome cpu's won't be available until early 2019 that's probably why they're not officially listed as an available configuration yet.
HStewart - Tuesday, November 20, 2018 - linkOk that is what I thought - it just a different model of line - different vendors based on customer requirements and preference. Of course 2019 will have other models also.