afds 2012 keynote phillip pokorny
TRANSCRIPT
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
1/22
HPC COMPUTING,FROM THE INSIDE OUT
An Insiders Perspective Philip Pokorny, CTOPenguin Computing, Inc.
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
2/22
2 | HPC Computing, from the Inside Out | June 14, 2012
PENGUIN COMPUTING HOW DID WE GET HERE?
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
3/22
3 | HPC Computing, from the Inside Out | June 14, 2012
BEGINNING AT THE END Worlds First APU Cluster
Delivered September 2011
Collaboration among AMD, Sandia National Lab,Penguin Computing
104 Altus 2A00 nodesA8-3850 APU, 16GB of memory, SSD scratchdrive, Qlogic Infiniband
Complete solution with:Management nodes (boot, login,
storage gateways)Networking (Ethernet and Infiniband)Six racksPower distribution with per outlet switchingSerial console monitoring
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
4/22
4 | HPC Computing, from the Inside Out | June 14, 2012
Founded in 1998 by Sam Ockman
Focus on systems running Linux
Focus on reliable systems
Customized systems
First Linux 1U rackmount server
Short depth 1U
2U Storage/Virtual Tape server with sixIDE drives
Customer specified filesystem layout
Customer specified RAID configurations
HISTORY OF PENGUIN COMPUTING
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
5/225 | HPC Computing, from the Inside Out | June 14, 2012
MEMORABLE ADVERTISING CAMPAIGNS
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
6/226 | HPC Computing, from the Inside Out | June 14, 2012
AMD USHERS IN THE 64-BIT PC ERA
AMD Opteron Processor AMD64 architecture64-bit registers (and more of them)
Orthogonal instruction setIntegrated memory controllerHyperTransport Interconnect
Linux64-bit ready and tested
Altus 1000eDual socket systemFan control and monitoringTemperature sensor driver(written and contributed to kernel)Penguin Remote Control
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
7/227 | HPC Computing, from the Inside Out | June 14, 2012
FROM SERVERS TO CLUSTERS
Large orders for Altus 1000eBeowulf style clusters
Scyld SoftwareDon BeckerMerged with Penguin Computing in 2003
Complete clusters with all the accessoriesRacks, UPS, keyboard & monitorNetworks, cabling, management
Factory integrated and testedSoftware installed and configuredTested as a complete clusterShipped in a crateReady to plug into network and power on-site
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
8/228 | HPC Computing, from the Inside Out | June 14, 2012
FROM THE INSIDE OUT
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
9/229 | HPC Computing, from the Inside Out | June 14, 2012
Dual Socket AMD Opteron Processor Memory Configurations
CPU CORES AND MEMORY SIZES OVER TIME
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
10/22
10 | HPC Computing, from the Inside Out | June 14, 2012
Dual Socket AMD Opteron Processor Memory Configurations
MEMORY PER CORE OVER TIME
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
11/22
11 | HPC Computing, from the Inside Out | June 14, 2012
Dual Socket AMD Opteron Processor Memory Configurations
WHAT'S THE TREND?
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
12/22
12 | HPC Computing, from the Inside Out | June 14, 2012
Dual Socket AMD Opteron Processor Memory Configurations
LONG-TERM TRENDS IN CPU AND MEMORY
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
13/22
13 | HPC Computing, from the Inside Out | June 14, 2012
MEMORY BANDWIDTH OVER TIME
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
14/22
14 | HPC Computing, from the Inside Out | June 14, 2012
THOUGHTS ON THE FUTURE
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
15/22
15 | HPC Computing, from the Inside Out | June 14, 2012
STORAGE PERFORMANCE
7200 RPM drives from 2002 to 2012
80 GB to 3000 GB
40 MB/sec to 160 MB/sec
Lots more space, not much more performance
RAID
Less than perfect scaling
Reliability level impacts performance
Motherboard SATA ports limit number of drives
Solid State Drives
Limited writes
May need ATA TRIM for better results
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
16/22
16 | HPC Computing, from the Inside Out | June 14, 2012
STORAGE PERFORMANCE
SATA really? Compatible with 1985 MFM drive controller!
Assumes compatibility at register level
Butquirks to every controller One drive per physical portHot swap added onOops, forgot the drive activity signal
SASHigher performance drives (15,000 RPM)Flexible number of drivers per controllerCan host SATA drivesMultiple ports for redundancy and performance
Ask for it
IS THAT A GPU IN YOUR SOCKET?
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
17/22
17 | HPC Computing, from the Inside Out | June 14, 2012
IS THAT A GPU IN YOUR SOCKET?
Mixed number of CPU/GPU
Socket compatible FPGAs Spansion flash controller and NOR flash DIMMs
Just need the hardware
POWER EFFICIENCY AND IDEAS FOR IMPROVEMENT
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
18/22
18 | HPC Computing, from the Inside Out | June 14, 2012
POWER EFFICIENCY AND IDEAS FOR IMPROVEMENT
CPUs well-covered elsewhere
MotherboardsMore efficient voltage converters (VRM)Better VRM tuningGenerate voltages only where needed
Power supplyHigher efficiency
Beware of hidden power distribution boards
Single rail (one voltage, not five or more)
Common infrastructure12.5V distributionShared fansMatch power supplies to server load
WHERE DOES THE POWER GO?
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
19/22
19 | HPC Computing, from the Inside Out | June 14, 2012
WHERE DOES THE POWER GO?
You cant control what you cant measure
Need to compare multiple systems
Need high-speed capture for some analysis
Diagnose problems
Spot trends
CPU versus motherboard versus anything else
Monitor each voltage rail individuallyCPU, motherboard, fans, hard drives
High speed (1000 samples per second or faster)
Correlate samples to code execution
COMING SOON
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
20/22
20 | HPC Computing, from the Inside Out | June 14, 2012
COMING SOON
A glimpse of what we might learn
CPU power draw during make j4
Different phases of compilation
Clearly different power levels
Larger files can load CPU 100% forlong periods of time
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
21/22
21 | HPC Computing, from the Inside Out | June 14, 2012
THANK YOU QUESTIONS?
-
7/31/2019 AFDS 2012 Keynote Phillip Pokorny
22/22