openpower foundation technical overview - ibm · • numerous disruptive forces are impacting these...
TRANSCRIPT
© 2014 IBM Corporation
LinuxCon Japan 2014
OpenPOWER Technical Overview
Jeff ScheelChief EngineerLinux on Power
May 21, 2014
© 2014 IBM Corporation2
Agenda
1. OpenPOWER Foundation Overview
2. OpenPOWER Hardware Technologies
3. OpenPOWER System Software Architecture
4 © OpenPOWER Foundation 2014
Industry trends
• The number of companies designing & building servers is increasing
– Traditionally there have been few companies designing systems: HP, IBM, SUN, Dell, etc.
– Today there are many more: Google, Microsoft, Facebook, Rackspace, Huawei, Sugon, Inspur, etc.
– A fairly mature ecosystem including the Taiwanese ODMs is a key enabler of this trend
• Numerous disruptive forces are impacting these custom system designs and driving designers to consider new ways of innovating
– Ability to handle rapid growth in Big Data & Analytics based solutions– Choice and Innovation– CPU SOC integration drive need for chip development
• These trends create a need for a server targeted “chip-system-software” ecosystem
– IBM has technology and a software stack ready to meet these needs– IBM recognizes the need to work with partners to create this ecosystem– IBM recognizes the need for choice and options in processor sourcing
5 © OpenPOWER Foundation 2014
OpenPOWER Foundation Structure
OpenPOWER is an industry foundation based on the POWER architecture, enabling an Open community for development and opportunity for member differentiation and growth
6 © OpenPOWER Foundation 2014
The goal of the OpenPOWER Foundation is to create an open ecosystem, using the POWER Architecture to share expertise, investment, and
server-class intellectual property to serve the evolving needs of customers.
OpenPOWER drives industry innovation
– Opening the architecture to give the industry the ability to innovate across the full Hardware and Software stack
• Simplify system design with alternative architecture
• Includes SOC design, Bus Specifications, Reference Designs, FW OS and Open Source Hypervisor
• Little Endian Linux to ease the migration of software to POWER
– Driving an expansion of enterprise class Hardware and Software stack for the data center
– Building a complete ecosystem to provide customers with the flexibility to build servers best suited to the Power architecture
7 © OpenPOWER Foundation 2014
Building collaboration and innovation at all levels
Welcoming new members in all areas of the ecosystem100+ inquiries and numerous active dialogues underway
Boards/Systems
I/O, Storage, Acceleration
Chip/SOC
System/Software/Services
8 © OpenPOWER Foundation 2014
Proposed Work Groups and Projects
Work Group Projects Participants
System Software(Open Source)
•Linux LE•KVM•Firmware
–OpenPOWER FW interface
PublicPublicPublic
•POWER LE ABI Public
Application Software(Open Source)
•System Operating Environment– OpenPOWER Software ecosystem
enablement
Public
•Toolchain Public
Open Server Development Platform
•Power 8 Developer Board Member
•POWER 8 Reference Design Member
Hardware Architecture
•OpenPOWER profile of architecture – Power8 ISA Book 1, 2, 3
Member
•Coherent Accelerator Interface Architecture (CAIA)
Member
Compliance •Compliance Member
9 © OpenPOWER Foundation 2014
OpenPOWER Innovation Roadmap
Domain Project ParticipantsChip design ISA Standard Products/ SOCs IBM, Suzhou Power Corp
Systems CM/ODM/OEM models Tyan, Servergy, Inspur, ZTE, ChuangHe,IBM, Hitachi
Memory Processor - Memory Interface optimizations
Hynix, Micron, Samsung
I/O Network Optimization- RDMA Performance- tightly coupled NIC-processor
Storage Optimization
Mellanox
Fusion I/O
Acceleration Tightly coupled custom FPGA accelerators
Enterprise & HPC Workloads
Xilinx, Altera
NVIDIA
Software Cloud Optimized Firmware Stack & LE Linux
Google, Canonical , IBM
10 © OpenPOWER Foundation 2014
OpenPOWER System Software Overview
• Draft charter proposes working group to be “stewards, incubators, and drivers (when feasible) of system software enablement for OpenPOWER Foundation hardware”
• Open to any members of the foundation
• Key workgroup products include:– OpenPOWER Foundation web presence– Creation of new open source communities– Complete open source software stack from firmware to system management software– Documentation of key community interfaces including LE ABI, firmware specifications,
and virtualization APIs
• Jointly works with HW Architecture and Compliance workgroups to:
– Define and manage the POWER ISA– Ensure compliance to software interfaces– Implement/exploit new technology such as CAPI
12 © OpenPOWER Foundation 2014
POWER8 Processor
Caches • 512 KB SRAM L2 / core• 96 MB eDRAM shared L3• Up to 128 MB eDRAM L4
(off-chip)
Cores • 12 cores (SMT8)• 8 dispatch, 10 issue,
16 exec pipe• 2X internal data
flows/queues• Enhanced prefetching• 64K data cache,
32K instruction cache
Accelerators• Crypto & memory expansion• Transactional Memory • VMM assist • Data Move / VM Mobility Energy Management
• On-chip Power Management Micro-controller• Integrated Per-core VRM• Critical Path Monitors
Technology•22nm SOI, eDRAM, 15 ML 650mm2
Memory• Up to 230 GB/s
sustained bandwidth
Bus Interfaces• Durable open memory
attach interface• Integrated PCIe Gen3• SMP Interconnect• CAPI (Coherent
Accelerator Processor Interface)
ComputerWorld: To make the chip faster, IBM has turned to a more advanced manufacturing process, increased the clock speed and added more cache memory, but perhaps the biggest change heralded by the Power8 cannot be found in the specifications. After years of restricting Power processors to its servers, IBM is throwing open the gates and will be licensing Power8 to third-party chip and component makers. The Register: the Power8 is so clearly engineered for midrange and enterprise systems for running applications on a giant shared memory space, backed by lots of cores and threads. Power8 does not belong in a smartphone unless you want one the size of a shoebox that weighs 20 pounds. But it most certainly does belong in a badass server, and Power8 is by far one of the most elegant chips that Big Blue has ever created, based on the initial specs. PCWorld: With Power8, IBM has more than doubled the sustained memory bandwidth from the Power7 and Power7+, to 230 GB/s, as well as I/O speed, to 48 GB/s. Put another way, Watson’s ability to look up and respond to information has more than doubled as well.
Microprocessor report: Called Power8, the new chip delivers impressive numbers, doubling the performance of its already powerful predecessor, Power7+. Oracle currently leads in server-processor performance, but IBM’s new chip will crush those records. The Power8 specs are mind boggling.
Source: Hotchips presentation
13 © OpenPOWER Foundation 2014
POWER8 CAPI
CustomHardware
Application
POWER8
CAPP
Coherence Bus
PSL
FPGA or ASIC
Customizable HardwareApplication Accelerator • Specific system SW, middleware, or user application• Written to durable interface provided by PSL
POWER8
PCIe Gen 3Transport for encapsulated messages
Processor Service Layer (PSL)• Present robust, durable interfaces to applications• Offload complexity / content from CAPP
Virtual Addressing• Accelerator can work with same memory addresses that the
processors use• Pointers de-referenced same as the host application• Removes OS & device driver overhead
Hardware Managed Cache Coherence• Enables the accelerator to participate in “Locks” as a normal thread
Lowers Latency over IO communication model
Coherent Accelerator Processor Interface (CAPI)
© 2014 IBM Corporation14
Coherent Accelerator Processor Interface (CAPI) Overview
CAPP PCIe
POWER8 Processor
Typical I/O Model Flow
Flow with a Coherent Model
Shared Mem. Notify Accelerator
AccelerationShared Memory
Completion
DD CallCopy or PinSource Data
MMIO NotifyAccelerator
AccelerationPoll / Int
CompletionCopy or UnpinResult Data
Ret. From DDCompletion
FPGA
Fu
nctio
n n
Fu
nctio
n 0
Fu
nctio
n 1
Fu
nctio
n 2
CAPI
IBM Supplied POWER Service Layer
© 2014 IBM Corporation15
Example: Innovative “In-Memory” NoSQL/KVS Integrated Solution - via POWER8 CAPI-attached Flash
WWW
10Gb Uplink
POWER8 Server
Flash Array w/ up
to 40TB
Differentiated NoSQL(POWER8 + CAPI Flash)
Infrastructure Attributes- 192 threads in 4U Server drawer
- 40 TB of memory based Flash per 4U Drawer- Shared Memory & Cache for dynamic tuning
- Elimination of I/O and Network Overhead- Cluster solution in a box
5X Cost Reduction with
equivalent performance
WWW
500GB Cache Node500GB
Cache Node500GB Cache Node500GB
Cache Node500GB Cache Node500GB
Cache Node
Backup Node
Load Balancer
Today’s NoSQLin memory (x86)
10Gb Uplink
Infrastructure Requirements- Large Distributed (Scale out)
- Large Memory per node- Networking Bandwidth Needs
- Load Balancing
Power CAPI-attached Flash model for NoSQL offers dramatic (24:1) density advantage
© 2014 IBM Corporation16
Power Systems S822LPower Systems
S812L• 1-socket, 2U• Linux Only
• 2-socket, 2U• Linux Only
• 2-socket, 2U• All Operating Systems
Power Systems S822
Power Systems S814
• 1-socket, 4U• All Operating Systems
Power Systems S824
• 2-socket, 4U• All Operating
SystemsPower Systems S824L
• 2-socket, 4U• Linux Only• SOD
1 & 2 Sockets
New IBM Power Systems based on POWER8
© 2014 IBM Corporation17
Non-IBM POWER8 products
http://www.enterprisetech.com/2014/04/28/inside-google-tyan-power8-server-boards/
The Tyan reference (ATX) board, SP010, measures 12” by 9.6”➢ one single-chip module (SCM)➢ four DDR3 memory slots➢ four 6 Gb/sec SATA peripheral connectors➢ two USB 3.0 ports➢ two Gigabit Ethernet network interfaces➢ keyboard and video➢ intended for developers
The Google reference board➢ two single-chip module (SCM)➢ four modified SATA ports➢ Google use only
19 © OpenPOWER Foundation 2014
OpenPOWER Proposed Ecosystem Enablement
XCATXCAT
CloudSoftware
OperatingSystem / KVM
Standard OperatingEnvironment
(System Mgmt)
So
ftw
are
Power Open Source Software Stack Components
ExistingOpen
Source Software
Communities
Firmware
Hardware
New OSS Community
OpenPOWERTechnology
OpenPOWERFirmware
CAPP
PC
Ie
POWER8
CAPI over PCIe
“Standard POWER Products” – 2014
Har
dw
a re
“Custom POWER SoC” – Future
Customizable
Framework to Integrate System IP on Chip
Industry IP License Model
Multiple Options to Design with POWER Technology Within OpenPOWER
© 2014 IBM Corporation20
KVM and OpenStack enable “defacto” APIs for integration
Power8 Platform
OpenPOWER FW
Qemu
VM1Linux
VM2Linux
LibvirtAPI & virsh CLI
Linux Kernel
Hos
t O
S
ConsoleShell CLI
Linux UserspaceOpenstack
compute nodecomponents
Openstackcontrollerservices
xCatChef
PuppetCustom scripts
ProjectKimchi
BrowserOr
Client
IPMIServiceProcessor
KVM
VM3Linux
Note: oVirt has been enabled in version 3.4 for Linux on Power.
Standard interfaces
© 2014 IBM Corporation21
Integrating POWER into a datacenter
H/W Management- Custom scripting- Heavily Automated- Open technologies
Platform Management- Custom management- Cloud based
In general– Bring base components in parity with KVM on x86 – “KVM should be KVM”– Co-exists peacefully with other end-points, whenever makes sense.– Enables hybrid clouds with common management environments
Manage
Provision
Deploy
Procure
xCatPuppetChefCustom Scripts
PXEIPMI
OpenStackCustom Scripts
22 © OpenPOWER Foundation 2014
OpenPOWER Enables Hybrid Cloud
• Linux – to provide commonality for:– Operating system management
– Operating system feature
– Application programming model
• Little Endian – to provide source code and data commonality
• KVM – to provide virtualization management and feature commonality
• Firmware interfaces – to provide platform management commonality
• Software written in interpreted languages (Javascript, PHP, Perl, Python, Ruby, Java, etc.)
– Generally, no work is required.
• Software written in compiled languages (C/C++, Fortran, etc.)
– Generally, this requires just a simple recompile for POWER.
• Rarely, dependencies on specific behaviors can require source code modification:
– Multi-threaded applications that don’t use standard synchronization models and depend upon specific memory ordering behavior (unusual)
– Applications that depend upon specific memory page sizes (rare)
OpenPOWER software commonality Migrating software to OpenPOWER
© 2014 IBM Corporation23
POWER8 supports little endian data access, like x86
● POWER8 processors support execution in both big endian (BE) and little endian mode (LE)
● Today's POWER operating systems run the processors in BE mode – Linux, AIX, and IBM i
● Most compiled open source software is designed (defacto) to run in little endian mode.
● Linux on Power has chosen to exploit little endian (LE) processor mode based on OpenPOWER partner feedback.
– Eases the migration of applications from Linux on x86.– Enables simple data migration from Linux on x86.– Simplifies data sharing (interoperability) with Linux on x86.– Improves Power I/O offerings with modern I/O adapters and
devices, e.g. GPUs.
● Creating LE OS for Linux on Power means creating a whole new operating system “platform” (ppc64le).
● LE distributions for Linux on Power does NOT mean x86 applications magically run: applications must still be compiled for Power.
BE and LE are simply different ways of ordering of how data is stored. Important in 1980.Not as important in 2013.Market has moved.
© 2014 IBM Corporation25
Where to find more information? http://openpowerfoundation.org/
© 2014 IBM Corporation26
Q & A
Summary:
1.The OpenPOWER Foundation provides a framework for open innovation on hardware and software
2.The system software structure embraces many key open source technologies – KVM, OpenStack, and other
3.Opportunities abound for hardware innovation with the POWER8 processor as a foundation