field programmable gate arrays: power in future hpc systems eric stahlberg ohio supercomputer center...
TRANSCRIPT
Field Programmable Gate Arrays:Field Programmable Gate Arrays:Power in Future HPC SystemsPower in Future HPC Systems
Eric StahlbergOhio Supercomputer Center
July 12, 2005
July 12, 2005 NCSA RSSI 2005
Discussion RoadmapDiscussion Roadmap
• Setting the context• Why FPGAs now?• Challenges to adoption• OpenFPGA as a bridge
July 12, 2005 NCSA RSSI 2005
Why Are People Here?Why Are People Here?
• FPGAs are cool• I want to learn valuable skills• I have a good idea I want to share• I have something to offer as a product or service• I want to learn about other developments in RC
• Where’s the free food?
July 12, 2005 NCSA RSSI 2005
About The Ohio Supercomputer CenterAbout The Ohio Supercomputer Center
• Located in Columbus, Ohio
• The Ohio Supercomputer Center was established in 1987 to position Ohio universities and industries at the forefront of computationally intensive research, development, engineering and networking.
• Delivering high performance computing and high capacity optical network for production and research
• Strategic aims OSC provides a reliable high-performance computing and communications infrastructure for a diverse,
statewide/regional community including education, academic research, industry, and state government.
OSC strives to be in the forefront of computational research in order to act as a key enabler for the state’s aspirations in advanced technology, information systems, and advanced industries, and,
OSC acts as a catalytic partner of Ohio universities and industries to enable
• Supporting High Performance Computing Requirements of- Bioinformatics- Advanced Manufacturing- Agriculture- Data-centric applications- Materials- Modeling, Testing and Instrumentation
July 12, 2005 NCSA RSSI 2005
Production Computing EnvironmentsProduction Computing Environments
SGI Altix – Linux IPF Shared Memory System
.20 TF64 GB32 900 MHz Intel Itanium 2
SUN COE with Time logic boards for specialized bionformatics apps
.04 TF48 GB48 900 MHz UltraSPARCIII
CRAY X1 AC .20 TF64 GB16 800 MHz vector processors
.23 TF
1.92 TF
.31 TF
2.5 TF
1.1 TF
Peak Performance
One of 8 sites around the country for computational grid testing and deployment
52 GB52 2.2 GHz Pentium 4 Xeon
Alliance Grid Testbed (AGT) Cluster
Cluster Ohio -- Distributed computational grid at 15 institutions around the state
736 GB192 550 MHz Pentium III Xeon, 144 733 MHz Itanium, 256 1.4 GHz AMD Athlons
BALE Visualization cluster with NVIDIA Quadro4 900 XGL graphics board / node
100 GB100 1.53 GHz Athlons
IA-32 Distributed parallel (128 nodes with Infiniband interconnect) and serial applications
1024 GB512 2.4 GHz Intel Pentium 4 Xeon
IA-64 Distributed parallel and serial applications
752 GB300 900 MHz Intel Itanium2
Special PurposeTotal MemoryProcessorsSystem
July 12, 2005 NCSA RSSI 2005
FPGAs Have Been Around A WhileFPGAs Have Been Around A While• FPGA computing is well established in some areas
MAPLD RECON ACM/IEEE
• Exploited by programmed logic designers in lieu of an ASIC approach General Fixed logic Reconfigurable – an option to change without physical replacement
• Utilized in many devices Vehicle systems Defense systems (radars, missiles, aircraft) Medical systems Communications hardware
…but only more recently appearing in general purpose information systems
July 12, 2005 NCSA RSSI 2005
Early Use of FPGAs in Production Systems: Early Use of FPGAs in Production Systems: TimeLogic DeCypherTimeLogic DeCypher
• FPGA for production bioinformatics Field Programmable Gate Array Deployed 2002 in SunFire 6800 enterprise systems High throughput for searching/scanning data
• Available Algorithms Tera-BLAST (N,P,X, TN, TX) Smith-Waterman HMM searches and queries Profile scan Custom target building
• The environment was closed to custom development
July 12, 2005 NCSA RSSI 2005
A Surge in FPGA Interest:A Surge in FPGA Interest:The Economic End of Moore’s Law?The Economic End of Moore’s Law?
• Clock-rate gains have slowed Power constraints Yield constraints Cost constraints
• Core limits: ‘Core-allary’: The number of
cores will double every 18 months
At what cost? For how long? Then what?
• FPGAs capabilities are still rising Gate counts Clock rates
High clock rate is a cost, not a benefit;
it drives up costs of everything else
in the machine.
-- eWeek
July 12, 2005 NCSA RSSI 2005
Businesses Challenged by Businesses Challenged by Increasing Data VolumesIncreasing Data Volumes
• Increasing amounts of data generated By research applications and instruments By transactions and transportation By sensors and cameras
• Need to store and secure more information for longer times Sarbanes-Oxley controls Protection of intellectual property HIPAA protection of health care information
• Data volumes are soaring! Example: Genbank volume doubles every 18 months Walmart preparing to add TBs regularly for RFID
• Raising needs to rapidly search, index, manage and relate data
July 12, 2005 NCSA RSSI 2005
Challenged by Power Demands for SpeedupChallenged by Power Demands for Speedup• General purpose parallel clusters eat power
For example: One Itanium II cpu requires 100 watts of power 50x speedup for a 99% efficient parallel program requires 99
processors Power demand to 9.9 kWatts! Even with more cores, power demands will increase
• The cost of power is ever increasing Energy prices rise (ala Enron) Cost of power-outages has increasingly enormous business impact Backup power must be able to maintain critical business operations
which are increasingly data intensive
• Bottom line: Power is money…out the door
July 12, 2005 NCSA RSSI 2005
Data Centers Constrained by Data Centers Constrained by Space and BudgetSpace and Budget
• Large systems need space… A hypothetical 50 node 1U system requires est. 20 square
feet of floor space for air flow One rack (dual core) delivers 100 CPU units Speedup potential of only 50 for 99% parallel app
• …for cooling systems Need for ambient air cooling increases with CPU power Multiple cores = more cooling per node
• Bottom line: Building-size refrigerators (data centers) can be very expensive to build or retrofit To build: $67M to build data center consuming 15 MW power To operate: $12M+ per year at 40% capacity
July 12, 2005 NCSA RSSI 2005
Adoption Dilemma for FPGAAdoption Dilemma for FPGA• The Application Catch-22
Few valued applications to drive market Small market for skilled developers Searching for the veritable ‘Killer App’
• Cost Benefit Equation Imbalance Cost of adoption is high
• Hardware introduction price• Application development costs• Application and system support costs• Risk of single vendor solutions
July 12, 2005 NCSA RSSI 2005
Business Challenges At HandBusiness Challenges At Hand
• Key Challenge: Absence of meaningful applications Potential is not convincing Tendency to adopt other technology solutions (clusters of cores) Cost of proven applications is high
• Potential Solutions: Educate on proven capabilities Develop the full ROI picture for FPGA solutions Deliver more applications, less expensively
July 12, 2005 NCSA RSSI 2005
Business Challenges At HandBusiness Challenges At Hand
• Key Challenge: Small pool of capable developers Limits choices for implementation Brings long-term supportability in question Cost of skills and support is high
• Potential Solutions: Enhance experience and education of developers Ease supportability of applications Wage and price controls
July 12, 2005 NCSA RSSI 2005
Complimentary Perspectives?Complimentary Perspectives?Programming vs. Hardware DesignProgramming vs. Hardware Design
EncapsulationGeneralityMaintainability
Efficient implementationImmutable definitionLow-level specification
Reliable execution
Reusability
July 12, 2005 NCSA RSSI 2005
Technical Challenges At HandTechnical Challenges At Hand
• Key Challenge: Lack of portability of invested effort FPGA algorithm implementation requires special skills Vendor solutions are generally unique
• Potential Solutions: lower amount of effort required to deliver Improve portability of elements at every level
July 12, 2005 NCSA RSSI 2005
Technical Challenges At HandTechnical Challenges At Hand
• Key Challenge: A lack of affordable, common tools Expensive tools harder to justify De facto lock-in once purchased Need to maximize return on tool investment
• Potential Solutions: Cheap tools --- in due course, when sustainable Improved portability of solutions to better maximize ROI
July 12, 2005 NCSA RSSI 2005
Technical Challenges At Hand Technical Challenges At Hand • Key Challenge: Long development times
Detailed specification may be required Range of input conditions may require multiple
independent options for execution Even Place and Route can be time-consuming
• Potential Solutions: Better development tools Mutual education – application to implementation Improved reusability of elements Adopt allowable algorithmic short-cuts
July 12, 2005 NCSA RSSI 2005
A Familiar Set of ChallengesA Familiar Set of Challenges
• Absence of meaningful applications• Small pool of capable developers • Long development times• A lack of affordable, common tools• Lack of portability of invested effort
July 12, 2005 NCSA RSSI 2005
Emerging Attractiveness of FPGA Emerging Attractiveness of FPGA Solutions Solutions
• Capability for parallel performance boost Example: Latest Virtex 4 has 6 million logic gates Demonstrated applications moving beyond illustrative examples Utilizing increasingly complex algorithms and functional units
• Easier development Many options are emerging for programming Not just VHDL anymore
• Validation More vendors buying in
• Easier introduction No remodeling to add power or cooling Single integrated solutions are emerging
• Bonus: Upside potential for increased clock speeds
July 12, 2005 NCSA RSSI 2005
encryption
simulation
annotationimage recognition
data filtering
FPGA
comparative informatics
FPGA Opportunities at HandFPGA Opportunities at Hand
July 12, 2005 NCSA RSSI 2005
OpenFPGA: Motivating ObservationsOpenFPGA: Motivating Observations
• The use of Field Programmable Gate Array technology was missing a collaborative organization focused on fostering the use of FPGA technology in high level applications.
• The results of several months of early discussions and activities involving FPGA hardware vendors, academic, government and commercial application developers, technology integrators and computer centers clarified several common needs in the FPGA application development area.
• Addressing these needs, remains inherent to successfully foster the broader and more general integration of FPGA technology in higher level and general purpose applications.
July 12, 2005 NCSA RSSI 2005
Why OpenFPGA?Why OpenFPGA?
• Build on past successes of OpenMP and MPI to collectively foster easier application parallel solutions
• Diverse options for approaching FPGA application development and integration
• Combine efforts early key to fostering mutual evaluation and adoption of best practices
• Sooner or later, standards of interoperability needed to extend reuse and portability among FPGA applications
July 12, 2005 NCSA RSSI 2005
OpenFPGA MissionOpenFPGA Mission
“The mission of OpenFPGA.org is to promote the use of Field Programmable Gate Arrays in high level and enterprise applications by collaboratively defining, developing and sharing critical information, technologies and best practices.”
The mission is realized through pursuit of the following objective areas Innovation and Evaluation – sharing tools and best practices Standardization – collectively defining common implementations Education – developing skilled professionals Promotion – advancing successful application uses of FPGAs Communication and Collaboration – fostering communication Participation – expanding involvement
July 12, 2005 NCSA RSSI 2005
Technically, Where Does Technically, Where Does OpenFPGA Fit?OpenFPGA Fit?
• Connect Apps to FPGAs• Use common approaches
and APIs to enable mobility
• Improve supportability• Start small and focused• Build to future capabilities
Multi-vendor OS/FPGA
Environments
Apps
FPGA FPGA FPGA
Apps
open
fpga
July 12, 2005 NCSA RSSI 2005
Where Does OpenFPGA Fit?Where Does OpenFPGA Fit?• Technical possibilities
Open-source area for applications Open-source tool options Baseline standards for communication and interoperability Benchmarks
• Education Technical Public perception
• Collective progress Sharing results Debating new ideas and approaches
July 12, 2005 NCSA RSSI 2005
OpenFPGA So FarOpenFPGA So Far
• OSC took the initiative in late 2004 to form OpenFPGA following Demonstrated potential of FPGA technology in the Time Logic System Hearing common challenges expressed during several prior months of
discussion with FPGA vendors, tool providers, developers and end-users Hosting a very successful FPGA Workshop in October 2004
• Effort officially initiated by OSC in February 2005 with letters of invitation to form an exploratory OpenFPGA Steering Group 16 initial respondents (presently 19 member organizations) Cross-cutting across multiple interests, international, academic, commercial,
government
• First announced February 2005 at Manchester England Reconfigurable Computing workshop
• Co-host for RSSI held at NCSA July 2005
July 12, 2005 NCSA RSSI 2005
OpenFPGA Steering GroupOpenFPGA Steering Group
• Cray, Inc• GE Global Research• Koan Corporation• Mitrion• Nallatech• NASA• NCI-ABCC• NCSA• NIST• OSC
• Sandia National Lab• SGI• SRC• Starbridge Systems, Inc.• University of Cincinnati• University of Manchester• University of South
Carolina• University of Toledo• Zuse Institute Berlin
July 12, 2005 NCSA RSSI 2005
Current OpenFPGA ParticipationCurrent OpenFPGA Participation• More than 160 registrations representing academic,
government and commercial entities from over 11 countries
• Active in efforts ranging over Numerical simulation HPC Benchmarking and evaluation Biological modeling Clusters of FPGAs Signal processing FPGA programmability Bioinformatics algorithm acceleration Systems of linear equations High level languages Data mining Pattern matching, statistics, realtime datastream
July 12, 2005 NCSA RSSI 2005
OpenFPGA Status ReportOpenFPGA Status Report• Past
Formed steering group with regular bi-weekly call-in meetings Established mailing list to share information Created web presence at www.openfpga.org Surveyed user community for needs and interests Established forums for community discussion in objective areas
• Present First face-to-face steering group meeting on 7/15 (this week) Completion of organizational charter is top priority Identifying specific key efforts to pursue worldwide Opening participation
• Future Broadening participation BOF at SC 2005 Source to help define and evaluate standards for FPGA apps
July 12, 2005 NCSA RSSI 2005
Details on Current EffortsDetails on Current Efforts
• Finalize organizational charter to characterize the organization Develop a sustaining support model Constitute Advisory Board and Board of Directors per charter
• Prioritize developmental activities from across many potential areas Shared development tools and facilities Shared validation facilities Application specific standards Benchmark standards Application development and delivery
• Define, initiate and execute projects in common areas of collaboration, targeting community availability and impact
• Generally, demonstrate the capability of FPGA computation
July 12, 2005 NCSA RSSI 2005
Fostering Development and Use Of FPGA-based ApplicationsSign up at www.openfpga.org today.
July 12, 2005 NCSA RSSI 2005
Bridging to SolutionsBridging to Solutions
Valued Applications
Cheap Faster Solutions