STRATEGIC PLAN (December 13, 2010)

2010-2020

“In my opinion, the largest supercomputers at any time, including the first exaflops, should not be thought of as computers. They are strategic scientific instruments that happen to be built from computer technology. Their usage patterns and scientific impact are closer to major research facilities such as CERN, ITER, or Hubble.” (Andrew Jones. Vice President, HPC Business, NAG Ltd.)

1 | P a g e

_____________________________________________________________________________________

Table of Contents ___________________________________________________________________

1. Executive Summary........................................................................................................................42. Authorization..................................................................................................................................63. Vision and Mission and Objectives..................................................................................................7

3.1 Vision...................................................................................................................................................73.2 Mission.................................................................................................................................................7

4. Strategic Goal..................................................................................................................................74.1 Strategies to Achieve the Goal...............................................................................................................7

4.1.1 Clarify the Governance Model............................................................................................................7 4.1.2 Bring HC?GDP Ration to the Mean for Industrialized Countries..................................................8

4.1.3 Provide Up-To-Date HPC Facilities.................................................................................................8 4.1.4 Develop Highly Qualified Personnel.............................................................................................9

5. The Value Proposition for Stakeholders.........................................................................................9 5.1 The Stakeholders.................................................................................................................................9 5.2 The Value Proposition....................................................................................................................10

6. The Current Context......................................................................................................................107. Organizational History…………………………………………………………………………………………………………………128. International Trends.....................................................................................................................149. Strategic Issues..............................................................................................................................16

9 .1 HPC Resources that meet the needs of Canadian Researchers..........................................................169.1.1 Desktop/Small Lab Computing..........................................................................................................18

9.1.2 Mid-Level Computing.................................................................................................................19 9.1.3 Tier 1...........................................................................................................................................19 9.1.4 Highly Qualified Personnel.........................................................................................................20 9.1.5 Technology and Timelines........................................................................................................21

9.2 Data, Data, Data................................................................................................................................22 9.3 Programs and Outreach....................................................................................................................23

10. Long-Range Budget Forecast........................................................................................................2410.1 Consequences of Underfunding.........................................................................................................24

11. Performance Assessment Framework..........................................................................................2511.1 Objectives........................................................................................................................................25

11.1.1 National HPC Platform............................................................................................................25 11.1.2 Support for Researchers.......................................................................................................25

12. Conclusion...................................................................................................................................28

2 | P a g e

APPENDICESAPPENDIX 1: International Review Panel Recommendations...................................................................29APPENDIX 2: What Researcher Stakeholders Are Saying.........................................................................33APPENDIX 3: Stakeholder Comments on Digital Economy Consultation Submission...............................37APPENDIX 4: HPC/GDP.............................................................................................................................39APPENDIX 5: Strengths, Weaknesses, Opportunities and Threats...........................................................40APPENDIX 6: S&T Priorities Supported by Compute Canada....................................................................42

3 | P a g e

1. Executive Summary

High Performance Computing (HPC) is redefining the way that research is done. As a consequence, research in all disciplines is delivering new knowledge and innovation, resulting in wealth creation, social advantage and well-being for all Canadians. Compute Canada’s strategic plan is designed to build on the significant successes achieved to date and to address the opportunities and challenges that lie ahead.

Compute Canada’s HPC infrastructure provides a national platform that enables Canadian researchers to compete on an international scale, attracts top talent to Canadian universities and broadens the scope of research. This national platform has become an integral part of Canada's digital economy as demonstrated by the rapidly growing use of Compute Canada’s hardware, software and people resources and evidenced by a growth of 30% in the number of researchers supported in the past year alone.

High Performance Computing is a critical component of digital infrastructure, essential to creating new knowledge and generating the knowledge that results in innovation. Canada’s “supercomputing” platform is part of the digital infrastructure that includes repositories of complex data sets, a wide range of network-accessible research equipment, digital devices and distributed sensors, high-speed networks, and the related tools and services as well as rapidly increasing computing capacity.

As part of the implementation of this strategic plan, Compute Canada and its digital infrastructure partners will define a coherent and integrated vision for this digital infrastructure. In particular, Compute Canada is working with CANARIE to align goals and strategies, integrating our infrastructures, our programs and our skills to deliver a world-class research environment.

Without a national platform HPC resources will be isolated at the individual institutions that can afford them. Canadian researchers will lose the ability to undertake a broad range of scientific research and will lose the opportunity to work on science’s greatest challenges and collaborate on international grand challenges. Canada's ability to do much of the cutting-edge research that drives the digital age will be sacrificed.

To ensure a sustainable national platform, Compute Canada will rationalize the number of high performance computing centres in Canada and merge several existing consortia into regional divisions. We will take advantage of economies of scale while ensuring that the local support needs of the Canadian research community are met. This rationalized HPC landscape will make room for a new Canadian Tier-I centre that is competitive on the world stage and that forms part of a coherent HPC ecosystem. This will ensure that Canadian researchers can leverage the capabilities of a rapidly advancing technology.

With a series of other concrete proposals on infrastructure, management and the evolution of user support, this plan addresses the recommendations made by the Mid-Term International Review Panel (see Appendix 1 for details and www.computecanada.org for the full report ) and identifies a clear way forward for Canadian HPC. It is designed to improve service to our stakeholders and to broaden our user community by reaching out to non-traditional research segments, including the humanities, medicine and business interests.

4 | P a g e

GOALS IMPLEMENTATION STRATEGIES

Implement and maintain a national platform that contributes to and supports the very best science that depends on HPC

Incorporate and staff Compute Canada; establish 4 regional divisions and consolidate data centres; broaden the membership of the Board of Directors to include government research labs, provincial representatives and the private sector and the Chair of the International Advisory Panel

Establish a Tier 1 facility and map the path to exascale computing over the next decade

Bring the Canadian HPC/GDP ratio to the mean for industrialized countries within a five year period by aggressively seeking additional funding from traditional and non-traditional sources and developing a plan to achieve exascale computing in a timeframe that meets the needs of Canadian researchers

Support HPC equipment and researchers using HPC and train HQP1 to harness HPC to perform world class research and improve competitiveness

Work with universities to develop HPC support expertise and train researchers to use HPC effectively and efficiently

Undertake HPC research to ensure that both software and scientific applications are ready for advances in computer hardware

Build on the broad base of current Compute Canada users, extended to Canadian participation in the G8 Funding Agencies exascale initiative and work with NSERC, SSHRC and CIHR to develop an on-going program for HPC professionals

Serve Canada’s existing and planned Canadian and international major science initiatives;

Formalize relationships with the Canadian Astronomy Data Centre, SNOLAB, the Canadian Light Source (CLS), NEPTUNE/VENUS, TRIUMF, ATLAS, CBRAIN, Genome Canada, the Square Kilometre Array (SKA), and other large science facilities and ensure resources are available to meet their needs

Establish virtual HPC centres of excellence initially for (i) SMEs (ii) medical research and (iii) humanities and social sciences.

Establish HPC internships in Small and Medium-sized Enterprises (SMEs) in conjunction with MITACS and encourage SMEs to work with academic researchers; develop a pilot project with the medical research community; develop a strategic accelerator plan, including workshops and seminars for the humanities and social sciences.

1 Highly Qualified Personnel (HQP): such personnel must be maintained and developed both as staff within Compute Canada and users within the research community.

5 | P a g e

2. Board of Directors’ Authorization

____________________________________ ________________________________

Andrew Woodsworth, Chair Ted Hewitt, Vice-President (Research & International Relations), University of

Western Ontario

____________________________________ ________________________________

John Hepburn, Vice-President Research, R. Paul Young, Vice-President, University of British Columbia Research, University of Toronto

____________________________________ ________________________________

Joseph Hubert, Vice-rector, à la recherché, Steven Liss, Vice-President Université de Montréal Queen’s University

____________________________________ ________________________________

Christopher Loomis, Vice-President, Research Lorne Babiuk, Vice-President Memorial University of Newfoundland (Research) University of Alberta

____________________________________ ________________________________

Rosie Goldstein, Vice-President Research and Alan EvansInternational Relations, McGill University Researcher Representative

____________________________________ ________________________________Jonathan Schaeffer Susan Baldwin, Executive DirectorResearcher Representative Compute Canada

6 | P a g e

3. Vision and Mission

3.1 VISION: To advance research, support and accelerate innovation and excellence, develop highly qualified personnel, and enable competitive advantage, economic prosperity and well-being for all Canadians through the effective use of high performance computing.

3.2 MISSION: To create a world-class sustained national platform of shared high performance computing and data resources and personnel, accessible by researchers in all disciplines independent of resource or researcher location and to promote high performance computing nationally and internationally.

4. Strategic Goal

Compute Canada will contribute to and support world-leading scientific research that depends on High Performance Computing, by

Implementing and maintaining a world-class national HPC platform; Including a Tier 1 facility and preparing for exascale computing; Developing the Highly Qualified Personnel required to support state-of-the-art HPC

equipment and world-class research using HPC and training Highly Qualified Personnel capable of harnessing state-of-the-art HPC equipment to perform world class research and improve the competitiveness of Canadian Industry ;

Undertaking HPC research to ensure that both software and scientific applications can adapt to and take advantage of advances in computer hardware;

Serving existing and planned Canadian and international major science initiatives; Establishing virtual HPC centres of excellence initially for (i) SMEs (ii) medical research

and (iii) humanities and social sciences; and Developing a strategic communications plan to ensure that Canada’s accomplishments

in HPC-driven research and innovation are better appreciated by all of our stakeholders.

4.1 Strategies to achieve the Goal

4.1.1 Clarify the governance model and staff Compute Canada appropriately Incorporate Compute Canada and become membership-based at the level of

the institution (university, government labs, provincial governments, private sector);

Establish four regional divisions: Compute West, Compute Ontario, Calcul Québec and Compute Atlantic;

Re-examine the composition of the Board of Directors. For example, voting Board members to include VPRs as designated by the regional divisions, 2 Compute Canada researchers, 2 representatives of government research labs, 2

7 | P a g e

provincial representatives, 2 private sector representatives (1 vendor,1 user) , chair of the International Advisory Panel;

Regional VPR Board members must sit on the regional Board or management committee to ensure national decisions are implemented locally;

Develop the management team, strengthen the role of the role of the Executive Director with respect to operational decisions and add the staff necessary to implement the defined strategies;

Consolidate data centres (machine rooms) while ensuring that support for researchers remains locally available;

National rationalization of resources including equipment, user support personnel and funds. For example, new infrastructure funds, budgets for operational funds will be prepared by each region and submitted to Compute Canada for determination of the regional distribution; and

RFPs for equipment acquisition will be based on an equipment acquisition plan and developed by a lead institution in conjunction with a national committee. The RFP will be issued by an institution on behalf of Compute Canada.

4.1.2 Bring the Canadian HPC/GDP ratio to the mean for industrialized countries within a five year period by aggressively seek additional funding, from both traditional and non-traditional sources in order to serve a rapidly growing user base and accommodate technology advances

Request funding from CFI and provincial governments to upgrade current facilities and broaden the base of funding for both equipment and operations by requesting funding from NSERC, SSHRC, and CIHR;

Seek funding for a Tier 1 facility; Undertake a gap analysis of the skills required to support the research

community with respect to the three tiers of computation, the requirements to prepare the HPC users for advanced applications as we approach exascale capability, and need for additional support personnel to fill expertise gaps such as data management, storage and archiving and software development for exascale; and

Examine the potential for a service fee structure as part of a sustainable business model.

4.1.3. Provide researchers with up-to-date facilities that are acquired and managed cost-effectively Principles for determining the location of new equipment will include: support

expertise; projected costs for power, past performance, space, personnel, renovation, etc.; contribution of institutions; agreement to adhere to defined service standards ;

Development of an equipment acquisition plan that targets the general purpose clusters required by the majority of researchers, the equipment required by “big

8 | P a g e

science” initiatives, and the equipment and support personnel that will be required to position Canadian researchers as world-leaders in their fields; and

Timelines for acquisition and implementation will be staggered over a 4 year period in order to take advantage of advances in technology, economic value, changing or unanticipated researcher needs, and the requirement to support new “big science” initiatives.

4.1.4 Develop the Highly Qualified Personnel necessary for success Decouple funding for personnel from funding for equipment to have the ability

to apportion funds as necessary; Fill the gaps in service support (i.e. for non-traditional users, data management,

software specialists, new areas of economic benefit, medical research); Train researchers to develop efficient applications and utilization of HPC

resources, including those in the non-traditional disciplines and the private sector;

Train support staff to support researchers and in particular to optimize software;

Train staff to offer new services and to provide the required change of code for petascale and exascale; and

Enter a MITACS Accelerate agreement to develop an HPC internship program.

5. The Value Proposition for Compute Canada Stakeholders

5.1 The Stakeholders

Compute Canada’s stakeholders include academia, federal and provincial governments, funding agencies, the private sector, government-funded science initiatives, national and provincial high-speed networking organizations and the Canadian taxpayer. Some of these stakeholders, including astrophysicists, the sub-atomic physics and climate science communities and the humanities research community, have explicitly referenced the need for high performance computing and stated the desire to work with Compute Canada to ensure a dynamic HPC environment for Canadian researchers (see Appendix 2). Compute Canada’s submission to Industry Canada’s Digital Economy Strategy Consultation received the most votes, demonstrating a broad base of support from stakeholders. A representative sampling of comments is included in Appendix 3.

In the past, the traditional core areas of research requiring HPC facilities have been physics, chemistry and engineering. While they still predominate, these areas themselves now include specializations such as bio-chemistry and bio-physics and reflect the continuing multi-disciplinary trend in research. Compute Canada resources are being used to fight infectious bacteria, diagnose Alzheimer’s disease, track ocean circulation variability and predict future climate evolution, map the “Big Bang”, develop everything from fuel efficient cars to safe plastic, and analyze markets for more useful risk indicators.

9 | P a g e

World-class scientists are being attracted to do their research in Canada through the Canada Research Chair Program and the Canada Excellence Research Program. A majority of these researchers and the teams of graduate students and post-doctoral fellows working with and learning from them, whatever their fields, require HPC facilities and support from Compute Canada’s highly qualified personnel. Indeed, major science initiatives, funded by the federal government, such as ATLAS, the SNOLAB, VENUS/NEPTUNE, astronomical observatories and the Canadian Light Source, among others, require supercomputing and are relying on the use and continuing enhancement of Compute Canada facilities for their research.

Computers have re-invented everything we do. High Performance Computing is re-inventing how research is conducted and accelerating the timeframe for that research. Some businesses are already using HPC to advantage. The movie industry uses HPC for animation and rendering of special effects; retailers regularly use data mining; and credit card companies use HPC for fraud detection. Simulations to design chemicals that help the immune system fight bacteria now take three months on one of Compute Canada’s new high performance computers rather than the more than ten years it would have taken without it. The Canadian academic environment and Compute Canada are working with the private sector through investment in R&D to realize such order of magnitude impacts on the efficiency of the Canadian economy.

The Canadian taxpayer is a direct beneficiary of the research that requires HPC through improvements in the diagnosis, prevention and cure of disease; better understanding of pandemics so the general population can be better protected; green energy; understanding the impact of climate change and defining how to stabilize and improve our environment; food quality and safety throughout the food chain; and almost every aspect of health and well-being.

5.2 The Value Proposition

Compute Canada presents a value proposition that is central to economic prosperity, to an international reputation for leadership in science and research, and to the social and cultural lives of Canadians.

The value proposition includes: Support for excellence in scientific research that requires HPC equipment and

expertise; Acceleration of research and innovation through the use of advanced HPC that

enhances collaboration both nationally and internationally; Certainty that major science initiatives can be assured of the availability of HPC

resources; Inclusion of medical research, humanities, social science and arts and design

within the support system;

10 | P a g e

Increasing the return on investment for federal and provincial investments in R&D;

Outreach activities and programs such as HPC skills training; An HPC internship program in coordination with MITACS2; Collaborative research between academia and Small and Medium-sized

Enterprises; A cost-recovery program for SME access to HPC equipment and consulting

services; Consultation with the research community to determine the need and timing

for data storage and archiving; Increased private sector investment in R&D; Access to resources across the country, independent of the location of the

researcher; Naturally forming centres of expertise in manufacturing, development of green

technologies, new products, emerging industries; New world-class initiatives with Canadian leadership; and Repatriation of world-class scientists

Canada has a productivity gap relative to other G8 countries. The role of technology in realizing a reduction in that gap cannot be overstated. High performance computing is one of the necessary technologies that will contribute to that reduction and an improvement in productivity and economic performance.

6. The Current Context

The research enterprise now includes computation as a third distinct methodology, the other two being the traditional areas of laboratory experiment and theoretical analysis. Numerical simulation and modeling advance the understanding of complex phenomena and contribute to innovation in a wide variety of sectors from automobile to aerospace to pharmacology to animation to disease prevention and control. Tasks that can take months or years using “normal” computers can be completed in days or even minutes with high performance computing; these systems also allow investigations of a scope that would otherwise be simply impossible.

A component of the digital infrastructure that is critically important for high performance computing is networking. Canada’s national advanced network for research, industry and education, CANARIE, and the regional ORANs must continue to evolve and be capable of meeting the technological requirements of advanced computing and the corresponding expanding needs of the HPC community. Computing power is increasing exponentially. Data management, storage and accessibility requirements are expanding. The potential weak link is networking. As we implement petascale computing (the current

2 MITACS is an organization that brings together academia, industry and the public sector through research and training initiatives to develop cutting edge tools vital to the knowledge-based economy.

11 | P a g e

Tier 1 level) and plan for exascale computing, it is imperative that Canada’s national network focus on the continuing development of an advanced network that can deliver the required capability. There are also a number of “last mile” issues that must be addressed by the provincial Optical Regional Advanced Networks (ORANs) and universities within provincial boundaries in order that researchers can truly take advantage of the national shared computing platform and collaborate nationally and internationally in the knowledge and scientific challenges that lie ahead. Compute Canada will continue to work with the network providers at the national, provincial and campus levels to ensure that networking is an accelerating factor for research and does not become an inhibiting factor. A common digital infrastructure vision will help ensure that the critical components are mutually reinforcing and provide research, economic and social advantages.

As the world generates ever more data and as more and more data that was generated in the past is digitized, the only way to understand and extract its value is with high performance computing.

7. Organizational History

Seven regional HPC consortia were formed beginning in 1998 in response to the creation of CFI and its first call for proposals. In 2005, the C3.ca Association Inc., an organization that brought coherence and visibility to high performance computing and supported the development of the regional consortia, published its Long Range Plan for High Performance Computing in Canada: Engines of Discovery: The 21st

Century Revolution. In 2006, the Canada Foundation for Innovation created the National Platform Fund program in part as a response to the Long Range Plan (LRP). In 2007, the Long Range Plan for HPC in Canada was re-examined in light of the significant accomplishments of the past decade and the promise of the future. The Long Range Plan remains a valuable position paper that highlights the use, benefits and contributions of HPC in Canada. It defined the HPC community and united it with a common vision and sense of significance. It led to the establishment of Compute Canada as a national platform and a national representative for HPC.

This strategic plan builds upon the LRP and presents objectives and strategies for the implementation of the vision it presented. The LRP recognized that Canada has established a strong international position with respect to mid-range HPC facilities and that, as a result, many outstanding researchers have been attracted to Canadian universities and helped to grow Canadian industry. It stated:

“If we are to keep these people and gain the full benefits of the investment, this HPC infrastructure must be sustained at a competitive level. If we are also to address the grand-challenge problems, it will also be necessary to establish a high-end computing facility. … With this investment [$76M in 2006, $87M in 2009 and $97M in 2012] and these facilities Canadian researchers can be leaders in discovery and innovation, and Canadian industry can be internationally competitive in a new and lucrative range of fields.”

12 | P a g e

Since its inception, Compute Canada has made decisions in the national interest of all Canadian researchers while providing the flexibility to ensure the most effective and efficient implementation of these decisions at the regional level. The creation of a Compute Canada Data Base has ensured a single point of entry to Compute Canada and its constituent parts. A National Resource Allocation Committee has been established and two national calls for proposals issued. There is a single point of submission for all allocation requests regardless of where the researcher is located or the machine requested and there is effective coordination between the national and regional levels to ensure optimum service and value to researchers. In terms of interacting with and understanding the needs of the researcher community, town hall meetings were held across the country thereby providing input to this strategic plan; a hands-on workshop for humanities researchers assisted proposed projects to get up and running and effectively using HPC resources; and an HPC for health research workshop was held leading to discussions centered on the utilization of shared resources. Compute Canada provided strong support to NSERC to ensure Canadian researchers could participate in the G8 funding agencies exascale competition. Compute Canada and MITACS have signed a Memorandum of Understanding that will lead to establishment of HPC internships, providing advantages to graduate and post-graduate researchers as well as to Canada’s business community. And the Board of Directors has expanded to include two researcher representatives. Compute Canada continues to gain visibility nationally and is internationally recognized as representing Canadian HPC.

The community of researchers that requires HPC is growing rapidly and Compute Canada must ensure that the HPC resources it provides track the research needs. Demand arises both from the growing user base and from the requirements of researchers to remain competitive as the technology advances and system sizes and capabilities increase. As new machines become available, experience has shown that the additional capacity will be fully utilized almost immediately. The existing demand is rapidly outstripping the current supply.

In 2008-2009, the number of researchers using Compute Canada resources grew by almost 30% over the previous year. With the addition of new machines in early 2010 and early 2011, more researchers with increasingly complex applications are expected to request access to Compute Canada resources. There are also researchers in Canada at present whose work would be significantly advanced if Compute Canada had a Tier 1 system in place.

Compute Canada is supporting Canada’s major science initiatives, including ATLAS, NEPTUNE, Ocean Networks, the Canadian Light Source, SNOLAB, and astronomical observatories, among others. Compute Canada also supports private sector initiatives such as the Cancer Biomarker Network.

The HPC platform and the highly qualified personnel who support it are an integral and essential component of Canadian research infrastructure.

13 | P a g e

8. International TrendsCanada, as part of the global community, faces many research challenges such as those related to climate change, pandemics, and energy and water shortages. Canadian researchers depend upon the availability of competitive high performance computing facilities in order to participate in addressing these “grand challenges” as well as collaborating with other researchers in Canada and around the world on these and many other economic, social, and medical challenges.

On a global basis, there is a clear trend of increasing investment in high performance computing by both national and state governments, and by the European Union (EU). According to the IDC Special Study, July 7, 2010, (D2 Interim Report: Development of a Supercomputing Strategy in Europe): “the supercomputer segment is in a high-growth mode, even with the current recession – it grew 25% in 2009.” Appendix 4 shows the HPC investment per GDP for 40 countries. Canada ranks 24 th. And it is not just the advanced or developed nations that are investing. Cyprus, Malaysia, and Bulgaria are all investing more in HPC/GDP than Canada is.

Even in such difficult financial times, countries such as Australia, New Zealand, U.K. (Wales) are making significant investments in high performance computing infrastructure and personnel. There is a common understanding among the most productive nations that HPC is a fundamental pre-requisite for economic security and performance, competitive advantage and productivity growth.

Old technology cannot compete with new technology – and the field of HPC moves very quickly. Researchers using old technology are at a competitive disadvantage with the result that the innovations that might result from that research cannot keep pace with the number and speed of innovations by researchers in other countries. Canada, through the Canada Excellence Research Chairs Program (CERC) program, has been very successful in attracting the best and the brightest researchers from other countries. It is imperative that we provide the best infrastructure to support their work and reap the resulting benefits in innovation, excellence, advantage and reputation that these researchers are expected to bring to Canada.

One of the recommendations in the IDC Recommendations Report: For EU HPC Leadership In 2020 states “It is recommended that the EU and the nations make HPC a higher priority and step up to either the “Full leadership level” or at least the “Funding to reach major goals level” scenario level: This would require a net new investment reaching 600 million euros a year within five years”.

According to the Oak Ridge National Laboratory the predicted performance of leadership computing platforms will grow from under a petaflop in 2005, and one petaflop in 2010, to 20 petaflops in approximately 2012, 100 petaflops by 2015 and 1 exaflop (1000 petaflops) in approximately 2018. Japan is targeting 10 petaflops for 2011.

PRACE (Partnership for Advanced Computing in Europe) is a pan-European research infrastructure for high performance computing that is funded by partner countries and the European Union’s 7 th Framework Programme. PRACE “provides Europe with world-class systems for world-class science and strengthens Europe’s scientific and industrial competitiveness.” PRACE will install its second Tier-0

14 | P a g e

[defined as one of the few largest systems in the world] system in France before the end of 2010 and three more Tier-0 systems will be deployed by Italy, Spain, and Germany. Each of those countries has committed 100 million euros for PRACE resources and remain committed in spite of serious economic difficulties. All the related procurements will be made by the end of 2013. Each system will provide several petaflops of computing power. PRACE is targeting exaflop computing power by 2019.

In June 2010 the Governor of Massachusetts (population approximately 6.5 million) announced a project to “create a world-class high performance computing center that will provide an infrastructure for research computing in life sciences, clean energy and green computing; help establish a collaborative research agenda; and catalyze the development of an innovation district in downtown Holyoke.”

Wales (population approximately 3 million) is investing $60 million in regional HPC capability (including equipment, distribution networks, training). “Cutting-edge computing facilities will be available for use by businesses working independently or in collaboration with academics and will establish Wales as a key international centre for specialist computational research.” (Wales, Assembly Government, July 2010)

Exascale computing is on the horizon and initiatives are underway to examine the software and middleware needs. NSERC sought the advice of Compute Canada regarding whether or not to participate in the initiative by the G-8 funding agencies “Interdisciplinary Programme on Application Software toward Exascale Computing for Global Scale Issues”. Compute Canada strongly supported Canadian participation. This is critical in order to position both Canadian researchers and the analysts who support them for the future of high performance computing and, therefore, the future of research. That being said, Compute Canada should be providing a current Tier 1 facility now as there are significant numbers of researchers who need that capability to be competitive as well as to ensure that those researchers who will require exascale capability when it becomes available will be part of a community that has the skills to be able to use it.

The software and methodologies required in order to ensure that researchers can use an exascale machine are significantly different from those used today. This, too, represents a significant research challenge. It is essential to ensure that Canadian academic researchers – and by extension the private sector - are positioned to take advantage of such technological advances. It is imperative that Compute Canada work with international teams to develop new approaches and prepare both academic and private sector researchers for early adoption of this pivotal technology. By investing in the ability to leverage HPC across the spectrum of capability, Canada’s private sector will be well positioned to compete in the digital economy – an economy that will rely heavily on high performance computing.

Compute Canada must be able to position its research community to be part of an international community of research using state-of-the-art HPC facilities. While we do not need to duplicate the models of other countries, we need a credible presence to allow our researchers to be fully engaged participants in the global HPC community and, therefore, plausible partners in joint undertakings. At present investments in HPC in Canada are between 35%-50% of the levels of our G8 counterparts.

15 | P a g e

Canada’s partners in the G8, which have experienced significantly more severe reversals of their economic fortunes than has Canada, appear to be stepping up their commitment to HPC rather than diminishing it. This is a clear indication that high performance computing is understood to provide an important impetus to innovation and, hence, is a key component of recovery and growth.

9. Strategic Issues

The identification of the strategic issues is the result of the analysis of Compute Canada’s strengths, weaknesses, opportunities and threats (SWOT).

Compute Canada’s strengths centre on the policies, mechanisms and operations that are driving the implementation of a national HPC platform and its consolidation and integration and ensuring substantial infrastructure and researcher access to machines and expertise across the country.

Compute Canada’s weaknesses relate to the need for a robust and sustainable business model and a strengthened governance structure. As stated in the Mid-Term Review submitted to CFI: “Compute Canada’s current financial model is arguably its greatest weakness and poses a significant challenge going forward”.

Compute Canada’s opportunities include outreach to many different research communities including the private sector in order to extend the use of HPC for innovation and economic development and ensuring we are prepared with the people and the expertise to assist them.

The threats facing Compute Canada come from a variety of sources ranging from the lack of recognition generally of the strategic importance of HPC to research, innovation and the economy to the increasing cost of power to not knowing if or when new funding for HPC resources will become available.

The complete SWOT listing may be found in Appendix 5.

9.1 HPC Resources that meet the needs of Canadian researchers

Compute Canada is a national platform and as such must support researchers in many different disciplines who have different requirements and varying skill levels with respect to HPC, and projects that range in scope from an individual project to multi-country projects requiring international agreements that stipulate the HPC commitment. The use of high-performance computing across many research disciplines is evidence of its necessity as a scientific tool. A national platform is composed of highly qualified personnel, facilities, and software.

Discipline-specific facilities such as a telescope, a particle accelerator or a neutrino observatory become rallying points for their communities of interest. It is much harder for infrastructure platforms that serve many disciplines to create that same sense of community. As an example, there was little interest in the

16 | P a g e

internet among the Canadian research community twenty years ago. Now, the internet, like high performance computing, is fully acknowledged as an indispensable research tool.

Compute Canada is a key component of Canadian digital research infrastructure, perhaps the key, unifying component. The national HPC platform, that is relied on by Canadian researchers and their international partners, can be sustained only if there is predictable and sustained funding.

There are several types of user support expertise and infrastructure required within the Compute Canada user community. Although the facilities and expertise required to support the diversity of research groups vary significantly, both user support personnel and the computing systems themselves may be effectively shared. This sharing of resources is critical to their efficient and effective use as well as to maximizing their value to all stakeholders including the research community and those who provide funding for those resources.

Compute Canada's computational capability and storage facilities have grown from a modest beginning to the current offerings that make up the national platform. Total capability will expand slightly in the first half of 2011 as the final components of the NPF (National Platform Fund, a Canada Foundation for Innovation program) are deployed. After 2011 the older, pre-NPF (National Platform Fund, a Canada Foundation for Innovation Program) hardware will be decommissioned reducing Compute Canada's total capabilities significantly. Compute Canada will begin decommissioning the NPF funded hardware in 2013 with all those facilities becoming obsolete by 2016.

Figure 1 : The Computing Ecosystem

In order to serve its broad base of constituencies and to provide pathways for researchers to develop expertise to use more sophisticated systems as demanded by their research programs, Compute Canada must provide a full ecosystem of capabilities and expertise. High Performance Computing requires the availability of many different capabilities from desktop/small lab computing, to mid-range computing to the very largest systems as well as new architectures such as GPGPU and cloud computing. In Figure 1

17 | P a g e

we note that the collectivity of Compute Canada’s mid-range capability may be considered an incipient “cloud”.

Computing is itself a tool that continues to evolve. Compute Canada will continue to support all levels of capability from the mid-range to Tier 1 and will introduce the required new architectures for the benefit of the research community. This support encompasses not only the machines but also the infrastructure and user support staff necessary to meet the needs of both individual researchers and research groups at the national and international levels. The Tier 1 and mid-range levels would include both user and infrastructure support. At the desktop/small lab computing level, Compute Canada provides support to researchers primarily in the form of advice to enable them to move into the higher levels of the ecosystem in order to expand the objectives and capabilities of their research.

A computing ecosystem must be built for performance and must optimize the range of capabilities and capacities of the research community it serves. The computing ecosystem proposed by Compute Canada has three levels as identified in Figure 1.

CFI funds a portion (approximately 40%) of infrastructure and some part of the operational needs; however, Compute Canada, in order to meet the needs of researchers capable of developing a strong HPC capacity and capability for Canada needs to offer much more than simple access to infrastructure. Compute Canada must design and offer programs that enable researchers to make effective use of the infrastructure and to assist and collaborate with researchers in order to elevate their research output and enable them compete on the world stage. We must also engage in outreach to private sector SMEs and to government science departments in collaboration with university-based scientists. This is the opportunity that can be exploited through the provision of the full computing ecosystem.

9.1.1 Desktop/Small Lab Computing

The basic level of the computing ecosystem is composed of local computing facilities, typically desktop or small lab systems that are used to develop and test applications to be ported to the next level and to perform modest visualization tasks and analyses. Acquiring and managing such systems is the responsibility of the researchers themselves. This level also includes clusters that can be quite substantial, although typically far from the Top500 category. One of the most important services Compute Canada offers to researchers at this level is user support.

To advance their research or remain competitive in their fields the majority of these researchers will need to take advantage of the mid-range level of the computing ecosystem either for specific projects or, increasingly, as their new mode of computing. Compute Canada works with researchers at the desktop/small lab level to accelerate their progress to mid-range in support of excellence and innovation. Some researchers within the humanities and social science communities are currently working at this “staging” level. These researchers have recognized the need to move to the mid-range

18 | P a g e

and have approached Compute Canada for assistance in making that transition. Compute Canada will continue to offer hands-on workshops and seminars as well as one-on-one support to enable this.

9.1.2 Mid-Range Computing

The second, or mid-range level, is composed of a variety of systems that typically rank on the Top500 on installation and that are the main production nodes of the ecosystem. Compute Canada’s mid-range systems will be consolidated to ensure a viable mid-range capability and will be located at approximately ten sites in Canada. These facilities are intended to satisfy the needs of the majority of researchers. This category constitutes the backbone of the HPC ecosystem.

Compute Canada has internationally competitive mid-range facilities that are supporting a wide variety of researchers in very diverse fields. The majority of the researchers who use Compute Canada’s facilities use mid-range equipment. These researchers are undertaking significant work and making significant contributions to the base of knowledge across a wide range of disciplines. These are the researchers who are positioned to join international collaborative projects based on the world-class expertise they have demonstrated while working in the mid-range. These researchers are addressing priorities such as environmental science and technologies, natural resources and energy, health and related life sciences and technologies, and communications and information technology.

It is imperative that these facilities are maintained and renewed in order for our researchers to remain competitive over time and to continue to contribute to the Canadian economy, culture and society and the well-being of its citizens.

Mid-range systems which will be distributed geographically, will be equally accessible to all Canadian researchers and the mid range will include as a component cloud computing. Cloud computing plays a valuable role in enabling Compute Canada to support the diverse and non-traditional user communities which are part of its mandate. Cloud computing testbeds are currently operating on Compute Canada hardware in support of astronomy, medical and other research computing. Compute Canada has specialized expertise in this area and will continue to provide leadership in this rapidly-developing computing methodology.

9.1.3 Tier 1

The highest level in the computing ecosystem consists of world class Tier 1 facilities. At this time, such a facility would be a petascale computer, although this definition evolves with the general progression of computing power. A Tier 1 facility would typically be ranked in the top 20 facilities worldwide and would be designed to allow very large-scale, parallel computations at the leading edge of research in many fields. Applications on such systems currently run on several thousand processing cores or more and this requires special architectures, especially a fast interconnect to be effective research tools. Given the speed of technological advance, current Tier 1 systems will be mid level in three years. Given the

19 | P a g e

challenges in using such systems, a Tier 1 system is necessary to prepare HPC users to move to the next generation of mid-range facilities: as computer power increases, users worldwide must follow and adapt to new technologies in order to be fully productive when new mid-range systems are installed en masse . Access to a Tier 1 facility provides researchers with a four to five year advantage which will dramatically increase their competitiveness.

Canadian researchers require a Tier 1 facility now. This could be achieved by significantly upgrading an existing Compute Canada facility which has demonstrated the capability of managing high performance systems or it could be an entirely new facility. Tier 1 is an important step in preparing both researchers and support analysts for the exascale equipment that is expected in the 2018 timeframe and which is widely believed to present very significant challenges for the HPC community as well as unprecedented opportunities for new results and breakthroughs.

Appendix 6 provides examples of research objectives that can be met with existing capabilities and the unprecedented new knowledge and advantages that could be achieved with Tier 1 capability. Examples are provided for each of the four strategic priorities identified in the Government’s Science and Technology Strategy: environmental science and technologies, natural resources and energy, health and related life sciences and technologies, and communications and information technology.

There are economic and social advantages that result from the knowledge gained by participating in research at this scale. The knowledge results in innovation within a wide variety of sectors. Canadian researchers must have the HPC facilities necessary to earn a place at the table where such challenges are addressed. A good example is the Canadian team that assisted in designing, building and operating the CERN facility in Switzerland and was involved in the first set of results from the Large Hadron Collider. Without HPC and associated networking capabilities “at home” those researchers could not have participated and Canada would not be in a position to benefit from and capitalize on the “spin-off” knowledge that is inevitable from an undertaking of this magnitude.

HPC is dynamic and what is Tier 1 today will be mid-range in 2-3 years and will be almost obsolete with the arrival of exascale. Canada needs an internationally competitive Tier-1 facility and today that is Petascale. With a Tier 1 facility, Canada’s researchers will accelerate scientific discovery, enable researchers to conduct transformational science, and enable innovation throughout all levels of the economy.

9.1.4 Highly Qualified Personnel

The key “product” academia provides to businesses is Highly Qualified Personnel. The development of highly qualified personnel constitutes the most effective form of knowledge transfer. Computationally trained individuals are a very valuable output of the academic environment. These individuals go into businesses and enable those businesses to take advantage of HPC for reducing the time for innovation and the time to get a product to market. We must continue to develop these

20 | P a g e

talented and creative people in order to position our private sector to be early adopters of new computing power and technologies and realize the resulting economic advantages and wealth creation.

As computing architectures evolve to provide enhanced performance, the associated software, both system software and applications software, must be re-written to accommodate the new features. This highly specialized software will continue to need to be re-written and a new generation of software tools must be developed in order to keep pace with the architectural advances. This is both a challenge and an opportunity for Canada.

Individual scientists and researchers are usually not specialists in the development of the software they need to perform their research. Given the growing complexity of data sets and the exponential growth in the volume of data available to researchers it is imperative that we increase the numbers of highly qualified HPC personnel available to support the research community. For this reason, HPC specialists are an essential component of the scientific research that is dependent upon HPC. These individuals are part of the competitive advantage we provide. At present, we do not have sufficient numbers – in either academia or business.

“The investments required for scientific and economic success include many areas in addition to the cost of the computers. The most critical area is human expertise, including the scientists and researchers as well as the experts in using the supercomputers. There is a growing worldwide shortage of HPC talent due to an aging workforce and a scarcity of new graduates in various HPC fields.” (IDC 2010)

9.1.5 Technology and Timelines

The rapid and continuous progress in computer technology is a powerful driver of the HPC environment and community. This dramatic growth enables researchers to tackle problem today that were unimaginable a few years ago. It has also resulted in a number of strategic challenges that Compute Canada must meet in order to leverage these trends effectively. This section gives an overview of two key challenges: power consumption and software development.

The scale of modern massively-parallel systems presents extreme challenges in terms of the infrastructure required to support them, particularly power and cooling. Although the per-processor power requirement is now largely static, the overall power consumption continues to rise as the number of processors per system increases. Predictions for the first exascale system are that it will consume 1GW of power although it is believed that real power consumption will have to be under 100MW for it to be feasible. To address this, Compute Canada will consider the cost of power as an important criterion in determining the future locations of data centres

21 | P a g e

The combination of the sheer scale of large systems together with their increasingly heterogeneous nature makes writing efficient software for them an extreme challenge. It is generally recognized that current architectures have far outstripped the availability of software that can take advantage of their capabilities. As an example, when an application program must take advantage not only of all cores in a node but of thousands of cores across nodes, the challenge can be extreme. These trends highlight the need for skilled technical support, including programmers, to ensure that Canadian researchers are able to take advantage of contemporary and future systems. It will be increasingly challenging for individual programmers to develop applications for such systems and the possibility of writing a one-off program for a specific short-term need for such a machine is becoming remote. Compute Canada will actively encourage and enable the use of these systems – at both the mid-range and Tier 1 levels — through training and programming support. The development of a culture of software is a long term investment and one in which we must begin to invest immediately. These activities are labour intensive and will require additional staff.

9.2 Data, Data, Data

A final important factor is the explosion in data. In the past data volumes have been driven primarily by larger models running on larger computers; however, the volume of data from other sources is now beginning to dominate. Large experiments and instruments such as ATLAS and the SKA are generating or will generate petabytes to 100s of petabytes of data. In addition, many other activities are generating data: gene sequencing, medical imaging, sensor networks, satellite data to name just a few. The sources of digital data are growing at a rate that is greater than the rate at which hardware storage capacity is growing. The provision of data storage and the management and archiving of large datasets must become a key focus of Compute Canada. Some of these issues are described in the following sections.

Data storage, management and archiving are continuing to grow in importance for HPC datacentres, driven by the current "data explosion." Data and the information derived from it are core components of science and as a result accessing, handling, sharing and combining that data and information will become more and more important. Data-intensive applications are quickly emerging as a significant new class of HPC workloads.

As a result of the digitization of the research enterprise a growing number of researchers must address the storage and processing of increasing amounts of digital data. This “data deluge” is increasing the number of both traditional and non-traditional researchers that require access to large storage and computing capacity of the type normally available only at HPC sites. They will need a very significant amount of storage, appropriate software and computing power to process and analyze the data sets obtained from experimental platforms or clinical trials.

The International Review Panel suggested that Compute Canada consider its role in the management of research data and use the data storage capabilities and expertise of the regional divisions to develop a national approach to managing large research data collections. In order to support the full requirements

22 | P a g e

of researchers and scientists, Compute Canada should be working with its partners to provide a highly integrated storage infrastructure that supports the computational facilities.

Data StorageSupport will be provided for storage of large (hundreds of Terabytes to Petabytes) data collections in network proximity to the compute resources needed to process these data.

Data ManagementFacilities and policies will be developed to support operational data management systems hosted at Compute Canada sites.

Data Archiving Facilities and policies will be developed to support long-term (10 years and longer, some data sets in perpetuity) archiving of large science and other datasets.

9.3 Programs and Outreach. Compute Canada will offer training programs that make effective use of infrastructure, bring a full range of disciplines into the use of HPC, encourage researchers to “upgrade” to take advantage of technological advances and prepare for a Tier 1 machine, offer science fairs, work with SMEs and the private sector generally.

SME’s are an important element within the Canadian economy, accounting for approximately 80% of businesses. In order for SME’s to become and stay competitive and to grow their global market share, they must access to HPC resources and training. The International Review Panel report recommended that Compute Canada develop a coordinated plan that would provide better support to commercial organizations. SME’s may be aware of the competitive advantages of HPC; however, they may lack the necessary skills to take advantage of the technology and apply it to their businesses. Even if they do have the skills, they may find it very difficult to access HPC equipment. Encouraging Canada’s small and medium sized enterprises to adopt and adapt HPC for competitive advantage and sustainability will give them leverage in the marketplace.

Much research today requires HPC as a tool. The benefits of the research conducted and the resulting achievements include dramatic reductions in the time to results ratio; undertaking research in new and little-understood areas; the ability to access and analyze more data and more complex data sets; using simulation rather than building physical models to construct airplane wings or crash-test cars. Compute Canada will develop a strategic communications plan to ensure that Canada’s accomplishments in HPC-driven research and innovation are better articulated to all of our stakeholders. In addition, the major science initiatives, as partners in research, will be asked to acknowledge their use of and reliance upon Compute Canada resources.

23 | P a g e

10.0 Long-Range Budget Forecast ($millions)

Note: Each column represents a three year period ($Millions)

Notes: * Assumes the regional divisions will pay the full cost of power when data centres are rationalized.* * This increase would bring Canada closer to the mean of the G8 countries in terms of investment in HPC.

To bring the Canadian HPC/GDP ratio to the mean for the industrialized countries within a five year period would require an investment in HPC equipment of $72 million per year or $360 million. An investment of this order would place Canada between the UK and Germany in terms of the HPC to GDP ratio. (It is worth stressing that this would not mean that we would have nearly as much HPC in absolute terms as the UK since our GDP is less – in fact we would have around 40% as much – but the availability per GDP$ – or researcher – would be much closer). This level of investment would place Canada between Taiwan and Saudi Arabia.

11.1 Consequences of Underfunding

There will be significant consequences of underfunding the national HPC platform and the support Compute Canada provides the Canadian research community. These consequences will directly impact Canada’s achievements in science and our international reputation, economic performance and the ability to innovate. These include:

Brain drain to those countries that are supporting HPC (scientists, researchers and the highly qualified personnel who support them);

24 | P a g e

Researchers will limiting their research due to the limitations of facilities; Difficulty in attracting or retaining Canada Research Chairs and Canada Excellence Research

Chairs; Competitive disadvantage in research and in industry and innovation will decline; Loss of ability to participate in the grand challenges (climate change/pandemics/energy/ATLAS)

that are international collaborations; Lost opportunities for researchers in disciplines such as the humanities and social sciences who

are beginning to develop and leverage HPC to their advantage; Movement of researchers to private or commercial clouds even if these do not fully meet their

needs (i.e. our researchers settle for second or third best); Decrease in R&D investment in academic research by the private sector; and The loss of scientists and researchers who rely on mid-level capabilities and support.

11.0 Performance Assessment Framework

The performance assessment framework consists of two components: The Logic Model and the Performance Measurement Model. The logic model in figure 2 will allow the Board of Directors to evaluate the performance of Compute Canada and its regional centres.

A performance assessment framework is outlined in figure 3.

11.1 Objectives

11.1.1 To establish and sustain an integrated national HPC platform that takes advantage of new technologies and software in order to provide the best possible resources for researchers by:

Designing, acquiring, maintaining and managing efficient and cost-effective HPC facilities Broadening the base of partners for whom Compute Canada hosts equipment and offers shared

resources Establishing a Tier 1 facility in Canada, maintaining mid-range facilities and ensuring support for

researchers who need routine access to HPC for research advantage Improving the support for data intensive computing and developing the capability for large scale

and long-term data management, storage and archiving

11.1.2 To provide expert support for researchers using HPC in order to enhance their efficiency and effectiveness and to accelerate the results of their research by:

Optimizing access to and utilization of HPC facilities by researchers irrespective of location Developing HQP to meet present needs and prepare for future requirements Offering a variety of programs that encourage and facilitate the use of HPC

25 | P a g e

Figure 2: Compute Canada LOGIC MODEL

26 | P a g e

Figure 3: Performance Measurement Model

OUTPUT PERFORMANCE INDICATOR DATA SOURCE COLLECTION FREQUENCY

National Platform

Efficient & Effective HPC Platform # researchers utilizing

Processor clock speed, peak flop rate, cache structure, cache latencies, main memory bandwidth and latency, cores per compute node, communication bandwidth and latency and I/O bandwidth

Compare CC cost to buyingIn the commercial cloud

Upgraded HPC Infrastructure1 machine in top 20 of

TOP500 TOP500 list semi-annually

machines replaced at 5 years

meets international standards

Improved capability for use of increase in non-trad users CCDB annual

collaboration

HPC Infrastructure increase in medical research CCDB annual

Improved capability for dynamic storage, sharing, manipulation & analysis of data

Researcher Support

Improved access to HPC infrastructure 100% of large projects receive NRAC annual

allocation requested

user satisfaction survey annual

Improved science quality

Improved access to HQP user satisfaction survey annual

75% of gap areas staffed

Increased research productivity

Increased national collaboration 80% of "big-science" served

27 | P a g e

Papers and citations

SSHRC/NSERC/CIHR/CANARIE

grants supported by HPC

# of MITACS placements

# of SMEs supported

Increased international collaboration Papers and citations

12. Conclusion

The success of Canadian research depends upon our researchers having access to the mandatory tools for research in the twenty-first century. In an era of massive data volumes and the need to understand complex phenomena, High Performance Computing has become an indispensable tool for the advancement of knowledge and the resulting innovation, wealth creation and well-being for Canadians. With the capabilities afforded by HPC resources, scientists and researchers are addressing issues of immediate significance for contemporary society. They position the Canadian government to make informed policy decisions; they position the medical community to keep Canadians healthy; they can position the financial community to understand and avoid economic meltdowns such as we have recently experienced. They make our world and our lives richer and safer.

Compute Canada will ensure that our stakeholders, including all Canadians, both understand and reap the benefits of the research that results from our provision of world-class systems HPC and human resources to Canadian scientists and researchers.

28 | P a g e

APPENDIX 1 : International Review Panel Corrective Measures and Key Recommendations

CORRECTIVE MEASURES

Based on its findings, the IRP recommended to CFI that the following corrective measures should be undertaken and implemented by Compute Canada to build on its current achievements and ensure the long-term success of the Compute Canada initiative. A strategic plan for Compute Canada that strengthens its national role in the Canadian research landscape. The plan should contain:

o The value propositions for organizations to participate in and contribute to Compute Canada. o A performance assessment framework and arrangements with the consortia that would allow the

Board of Compute Canada to regularly review the performance of Compute Canada in delivering on its plan.

A formal executive process to allow the Compute Canada Board to implement the strategic plan at both the national and regional level.

The IRP recommended that Compute Canada be asked to provide this plan to CFI by December 31, 2010.

Future CFI payments associated with project #12866 should be contingent upon the implementation of these corrective measures in a timely fashion.

KEY PANEL RECOMMENDATIONS

The key recommendation of the IRP is reflected in the corrective measures outlined above.

In addition, the IRP believes the following key activities should be undertaken by Compute Canada in the short term:

An overall lifetime cost-benefit analysis of the acquired systems to inform future infrastructure acquisitions.o Compute Canada should conduct a detailed cost-benefit analysis to determine which sites should

eventually receive additional infrastructure. This analysis should take into consideration the cost over the lifetime of the equipment (acquisition + on-going operation). It should also consider any other benefits of placing infrastructure in a given location (redundancy, contribution from an institution, etc.).

A gap analysis for the need of additional support personnel o While there is general agreement that additional support personal would be beneficial to users,

there is a need for a gap analysis of the skills required to support the user community. This analysis should clearly outline the costs and benefits of having more support staff.

Regarding the NRAC, Compute Canada should:o Explore arrangements to provide long-term allocations and support to the most meritorious

research groups. o Raise the awareness of the process and increase the number of applications by undertaking

29 | P a g e

promotional activities. A coordinated plan that would allow the consortium members to better support non-traditional

communities and commercial organizations.

1. Cost-Benefit AnalysisAll regional divisions are contributing to the cost of this analysis which is currently being drafted. The University of Alberta has agreed to act as the contractor on behalf of Compute Canada. Compute Canada will review all responses to the request for proposals which will be issued in early January. CPAC has developed the following principles for the location of equipment and personnel which will be used to guide the analysis:

1. Accountability: Centres will operate in close, ongoing coordination with CC and will adhere to agreed national standards and practices. Past performance will be crucial to determine wherenew resources will be located.

2. Objectivity: Compute Canada should set up an external review process to insure that theobjectives are met in a rational way.

As a consequence:Individual data centres should be reviewed by an external review panel.‐The review should be based on past performance and on a sound proposal for future‐

hosting. An example “application form” for individual data centres is included as anannex.

The full text of the document may be found on the Compute Canada website : (www.computecanada.org). It is anticipated that the work for this will take approximately 60 working days from the date the contract is issued and will involve site visits.

2. Gap AnalysisThis analysis is being undertaken internally and is intended to identify those areas in which new and/or additional user support skills and the corresponding staff are required to meet and anticipate the needs of researchers. Areas which will be reviewed include:

data management and archiving; the specialized skills required to support medical research within a shared national platform; support to the arts, humanities and social sciences in order to migrate those researchers from

the research computing level to the true high performance computing that will advance their research and ensure they are internationally competitive; and

the development of the skills, both maintenance and software, required in preparation for exascale computing and Compute Canada’s ability to assist researchers to move to exascale computing quickly.

3. NRAC30 | P a g e

The key recommendation as stated by the International Review Panel has been fully met.

As stated in the September 2010 Call for Proposals “The primary mission of the NRAC is to ensure that the best science that is received in the proposals is put on the appropriate machines.” The call indicates that allocations will be valid for one year; however, it also notes that “Under exceptional circumstances and for long term national and international projects an extended timeframe may be granted. Allocations of longer than one year will be subject to NRAC approval and will require the submission of an annual progress report.”

In order to ensure awareness of the process, Compute Canada promoted the fact that the call would be issued in advance of the release date. Notification was sent to everyone holding a Compute Canada Database (CCDB) account. In addition, the regional divisions distributed emails to a wider audience and included prospective applicants even if they are not currently in the CCDB.

4. Coordinated plan to support non-traditional communities and commercial organizations.

There are three elements to Compute Canada’s plan to support commercial organizations:

i. A Memorandum of Understanding has been signed with MITACS to identify a minimum of 10 internship opportunities. This will provide not only support to commercial organizations but will be designed to advance the potential for using HPC in their businesses and encouraging further investment by the private sector in R&D.

ii. Providing access by SMEs to a limited amount of Compute Canada high performance computing equipment and expertise.

iii. A Memorandum of Understanding with Rocky Mountain Supercomputing Centres in Montana to develop virtual clusters of expertise (wind power, precision agriculture, manufacturing) for SME’s in order to develop new products.

There are four primary elements to Compute Canada's plan to support non- traditional communities.

i. Development of a comprehensive support resource for identified communities. This would include: web resources that were widely advertized to target communities that detailed clearly the range of support available; dedicated support personnel, knowledgeable in these disciplines - ideally some of these staff would be new hires; and a clear path indicating how these resources, including the specific additional elements of the plan described below, can be accessed.

ii. Provision of alternate computing models, including support of portal interfaces to HPC and cloud computing, that are often more closely aligned with the needs and research methodologies of the non-traditional user communities.

iii. Targeted conferences and workshops. Conferences and seminars expose the community to examples of what has been achieved in these disciplines by providing successful examples and allow for exchange of ideas. Hands-on workshops, fully supported by technical staff, have been successful in moving projects from the ideas stage to a prototype within a few-day intensive session.

31 | P a g e

iv. Targeting a small number of key projects - selected through a competitive process - for accelerated support. This may typically involve programming support to develop applications for HPC platforms, data visualization etc.

All of these strategies have been used within various areas of Compute Canada already. This plan formalizes many of the successful activities. It is anticipated that the implementation of this plan will be that some significant aspects are undertaken as a specialization within one or two of the regional divisions.

32 | P a g e

Appendix 2: What Researcher Stakeholders Are Saying

1. CANADIAN ASTRONOMICAL COMPUTING, DATA AND NETWORK FACILITIES: A WHITE PAPER FOR THE 2010 LONG RANGE PLANL. Jonathan Dursi (CITA), David A. Bohlender (HIA/NRC), James Wadsley (McMaster University) and JJ Kavelaars (HIA/NRC)

ABSTRACTSignificant investment in new large, expensive astronomical observing facilities spanning a substantial portion of the electronic spectrum was a dominant theme of LRP2000 and continues to benecessary for Canadian astronomy to maintain its world position. These developments are generatingincreasingly large volumes of data. Such investments only make sense if they are balanced by stronginfrastructure support to ensure that data acquired with these facilities can be readily accessed andanalyzed by observers, and that theoreticians have the tools available to simulate and understandtheir context. This will require continuing investment in computational facilities to store and analyzethe data, networks to ensure useful access to the data and products by Canadian researchers, andpersonnel to help Canadian researchers make use of these tools.

In addition, large parallel simulations have become an essential tool for astrophysical theory, andCanadian Astronomy has world-leading simulators and developers who rely on world-class High Performance Computing facilities being maintained in Canada to do their research effectively.We recommend that Compute Canada be funded at $72M/yr to bring HPC funding per capita inline with G8 norms; that part of every Compute Canada technology renewal include a Top-20 classcomputing facility; NSERC and other funding agencies begin supporting software development as anintegral component of scientific research; that the stable funding for consortia be tripled, includinglocal access to technical analyst staff; and that the last mile bottleneck of campus networking lessthan 10 Gb/s be addressed where it is impacting researchers, with particular urgency for the current1 Gb/s connection at the CADC.

2. Canada’s Digital Environment for Research, Innovation and Education, a submission under the Digital Economy Strategy Consultation by Canadian Digital Media Network, Canadian Research Knowledge Network, Canadian University Council of CIOs, CANARIE Inc. and Compute Canada

The elements of the digital environment to support RIE[Research, Industry and Education] have been evolving and include: the preservation and management of huge repositories of data and rich digital content; ever-larger compute capacity; digital devices and distributed sensors; low-latency, high-bandwidth networks; middleware that integrates the infrastructure and supports its use; and the expertise required to manage and operate them. … The elements of Canada’s digital infrastructure for RIE have been evolving for several years, enabled by increasingly powerful computing and networking technology. Canada’s investments in these areas are part of a world-wide response to the growing reliance on ever-increasing volumes of shared research data. A parallel shift towards greater reliance on collaborative models is a response to the flood of data and the collective need to manage it in a cost-effective way. The combination of highly skilled personnel, collaborative models, and appropriate

33 | P a g e

supporting infrastructure were identified as key elements supporting Canada’s innovation system in the Science Technology, and Innovation Council’s 2008 State of the Nation Report.

3. Resolution concerning High-Performance Computing and the Humanities

Be it resolved that:

The Society for Digital Humanities / Société pour l'étude des médias interactifs (SDH/SEMI) recommends that the Society work with Compute/Calcul Canada (CCC) to develop the high-performance computing (HPC) facilities funded for Canadian researchers so that they can be meaningfully used by humanists in need of research computing support. To achieve meaningful engagement SDH/SEMI encourages CCC to include digital humanists in their planning and governance process in order that the next generation of CCC facilities, support, and training be appropriately extended to be truly inclusive. SDH/SEMI offers to work closely with CCC to make the case for support of humanities research.

Further SDH/SEMI recommends that universities and funding bodies like SSHRC respond to CCC engagement with programs capable of supporting research using CCC facilities adequately where that research meets standards of excellence.

Background. In the last two decades there has been an epochal shift in humanities research as the cultural record we study and care for is being digitized. The mass digitization of human histories, literatures, art, cinema and music has challenged our research practices deeply but also offer the opportunity for large-scale digital humanities. Compute/Calcul Canada could be a partner of the humanities as it changes the scale of its work and learns to articulate new types of questions. Also, given that the humanities represent a significant percentage of the professoriate, the serious engagement of CCC in supporting research across the arts and humanities would dramatically increase the pool of users of the CCC infrastructure. Further, the materials humanists work on interest all Canadians – after all, we study the arts, the stories, and the histories that Canadians care passionately about and use to define themselves as Canadians. Support for humanities research would therefore have secondary effects that benefit all Canadians. Imagine Compute/Calcul Canada playing a foundational role in Canada’s cultural knowledge; SDH/SEMI can!

Two HPC Consortia, SHARCnet and WestGrid, have reached out to the digital humanities community organizing special events and acting on recommendations. See the draft report from the recent Mind the Gap workshop for background and links, http://docs.google.com/View?id=dhbw7427_4hnbkr8cd . SDH/SEMI encourages CCC to take the recommendations coming out of such meetings seriously. We realize that many of the recommendations propose new services that might dilute the core mission of CCC, but we believe that that core mission can be extended and that extending CCC is preferable to developing parallel infrastructure. We believe CFI thinks so too. For all these reasons we encourage CCC to engage the humanities in a meaningful way and offer to assist.

34 | P a g e

4. ATLAS

15 October 2010

CERN Computing Resources Review Board SummaryWalter Davidson (NRC), Canadian C-RRB representativeRobert McPherson (UVic/IPP), Canadian National Contact Physicist to CERN/ATLAS

Worldwide LHC Computing Grid (WLCG) operation with first data: The initial use of the W-LCG has been extremely successful, exemplified by the number of physics results extracted by the LHC experiments in timely fashions from the earliest data. Included in the successes is the routine analysis of data at the Tier-2 centres by thousands of physicists using computing grid tools. Currently, over 100M computing jobs are completed in the WLCG each day, corresponding to 100,000 CPU-days used each day. Sites are learning from experience with first data which should lead to some economies in the future. The load on operations personnel at the centres is high, but just manageable by making strong use of collaborative problem solving using experts from different sites. It is clear that we cannot reduce the number of people running the centres. Detailed scrutiny of the computing use by the different experiments show that storage allocations are used efficiently, while some inefficiencies are present in the CPU use at the Tier-0 and Tier-1 centres. Gratifyingly, ATLAS stands out has having Tier-0 and Tier-1 CPU use efficiency nearly a factor of two better than the CMS experiment. CMS funding agencies particularly encouraged economies be found by increasing the utilization efficiency.

Resource planning for near and medium-term future: countries and funding agencies are committed to future WLCG support. Eg, in 2011, ATLAS in particular has full commitments from all funding agencies for our needed computing resources. All funding agencies encourage continued resource scrutiny seeking economies, including understanding the implications of the LHC running schedule which will likely result in resource needs peaks and valleys. While the schedule may result in modified purchase profiles, the total running time averaged over a few year period will be the same as that used in computing resource need projections so the total resources needed will not change significantly. All funding agencies are committed to maintaining their share of computing resources required to analyze the LHC data.

35 | P a g e

Summary for Canada: we should expect a continued need for computing resources for ATLAS in Canada. There will be some year-by-year fluctuation due to the LHC running schedule with planned shutdowns in 2012, 2016 and 2020 but with continuous operation (multi-year runs) through other years. Our international partners are maintaining their strong commitment to the WLCG, and maintaining our 5% share of ATLAS computing will require computing hardware resource funding at the level of our existing projections, as well as maintaining existing support for computing centre and grid operations at the Canadian sites. For the TRIUMF Tier-1 centre, solutions for both hardware renewal and centre operations after the CFI award completes in early 2012 are critical to continued Canadian exploitation of our LHC investment. For the university-based Tier-2 sites, continued allocation of computing hardware resources from Compute Canada and operational support from the universities, Compute Canada and NSERC funds is essential (emphasis added).

5. Ontario Cancer Biomarker Network (OCBN)

The close collaboration that the OCBN has established with Compute Canada enables OCBN to function seamlessly as a single entity across the multiple nodes of the organization, many of which are connected in a virtual manner. With the high performance computing support and infrastructure provided, theOCBN is able to execute in parallel multiple, large scale research programs, each of which has significant‐computational, storage, and workflow management needs. Without the infrastructure and support ofCompute Canada, the OCBN would be severely limited in its ability to carry out its business and scientific activities to the level required to be competitive nationally and internationally. Further, OCBN would be severely compromised in its ability to coordinate and support the activities of its multiple proteomics and genomics core laboratories without the expanded collaboration between Compute Canada and OCBN.

6. SNOLAB

“Researchers at SNOLAB institutions, including Queens and other institutions in Canada and internationally have been using Compute Canada resources for the detailed Monte Carlo simulation of detectors under development as well as extensive analysis of detectors in operation. The scientific objective of these detectors is the observation of dark matter particles created in the Big Bang that are known to make up about 25% of the Universe. Another objective is the observation of neutrino-less double beta decay that can provide information about neutrino mass and processes in the early universe that led to our matter-dominated universe. Other measurements will include neutrinos from the sun, the earth and distant supernovae. All of these measurements will benefit from the unique environment at SNOLAB that is the lowest radioactive location in the world. HPC is essential for all of this work as the complexity of the analysis and simulation work requires such capability.” (Dr. A. MacDonald, SNP Project Director)

7. NEPTUNE Canada and VENUS

36 | P a g e

"The NEPTUNE Canada and VENUS cabled ocean observatories are world-leading facilities in a new generation of ocean S&T which brings electrical power and the Internet to the ocean enabling continuous and concurrent measurement of a broad suite of biological, physical, chemical and geologic properties in ways hitherto not possible. The data streams, including scalar, video and acoustic data, generate short and longer term time-series which support the analysis of complex interactive ocean system processes. But herein lies a major high performance computing challenge, namely the management, integration and analysis of massive data sets of varied modalities over time. These challenges find expression in areas that include, but are not limited to, data visualization, massive data handling, adaptive modelling, systems modelling, and non-parametric analysis. At the same time these challenges are exciting opportunities for harnessing leading edge ocean science with the HPC capabilities provided through Compute Canada, all supported by the CANARIE network. As a result, Canada is especially well positioned to be an international leader in this area." (Dr. M. Taylor, President & CEO, Ocean Networks Canada)

37 | P a g e

APPENDIX 3: Stakeholder Comments on the Digital Economy Strategy Consultation Submission

“This is an excellent suggestion. HPC does change the way we think about research.All of a sudden we can do myriads of things at the same time unravelingfundamental principles hidden in large arrays of data. Access to HPC is criticalto so many areas of our life spanning from pure science such as astrophysics andparticle physics to weather modeling, genome/drug research to day-to-day banking.” (noskovsy)

“I am an academic psychologist who studies how human memory works. The work hasboth theoretical and practical implications. Applications include knowledgeretrieval & translation. Without HPC, the field is dead in Canada, and mystudents will continue to contribute to the US economy. With HPC, my studentscan stay in Canada.” (mewhort)

“HPC is a primary concern in my research. I carry out Finite Difference TimeDomain (FDTD) simulations of electromagnetic wavepropagation/scattering/radiation with anin-house developed software with unique capabilities. The range of applicationsis widespread, from design/analysis of antennas, photonics components,electromagnetic interference, metamaterial, etc. HPC has opened new possibilitiesin my research. I would not think of doing without it now.” (Jasmin E Roy)

“HPC as long been realized through many international initiatives as a means ofincreasing competitiveness. Canada needs to embrace and promote what has beendeveloped through Compute Canada. We have to provide appropriate access to thisCanadian infrastructure asset (cyberinfrastructure) to the research, innovationand educational community. That means access even to the SME innovation engine ofCanada which makes up the highest collective economic driver we have.” (TerryDalton)

“HPC is a key to success in any scientific research these days.Unfortunately, Canada is way behind all other developed countries in thiscompetition. I vote that it should be changed.” (sergeychelsky)

“Without HPC most of the problems we solve are of the nature of conceptuallyidealized scenarios only. HPC makes massive simulation possible, which allow usthe capacity to deal with real field problems.” (zhaoga)

“In genomics/proteomics/bioinformatics research HPC pervades at multiple levels -from storing and crunching large volumes of raw data generated from next-gensequencers and mass spectrometers, to enabling data integration and data mining

38 | P a g e

of these datasets, to providing security and privacy with respect to confidentialclinical data. This infrastructure is critical to retaining our highly qualifiedtalent right here in Canada, and in attracting international business andinvestment in our biotech and research sectors. The need for HPC – the hardware,the software, and the people – will only continue to grow, particularly in lightof the “grand challenge” of Personalized Medicine, which is as much acomputational problem than anything else, given the complexity of the molecularmechanisms of biological systems and diseases. We have an opportunity to be atthe forefront of this research, and investment in HPC will be critical to gettingthere.” (mdharsee)

“One aspect of computing that is changing the economics of product development isthe use of extensive simulation instead of prototyping ( The Boeing 787 neededonly 4 test wings instead of 18 for the 747). Simulation is becoming the keyfactor in getting a competitive advantage.” Extensive simulation requires large scale computing: that means large scale computing infrastructure BUT ALSO and moreimportantly top level programming specialists. One cannot do without BOTH.Because of these essential requirements, developing countries cannot compete withtheir cheap manpower on that innovation front, hence a clear advantage forcountries developing a computing literate workforce ( here I am not referring towindow users but advance programming specialists). This is one area where Canadacan best exploit its university trained students.” (jmpoutissou)

“Well supported HPC infrastructure and associated human support is important alsoif Canada is to develop its digital content industry (from animation to games toelectronic texts.) HPC facilities will be the backbone to large-scale archives,large-scale content mining, and multimedia content sharing.” (GeoffreyRockwell)

“Increased HPC investment, to support SME sector, can provide an immediateadvantage to Canada. If we look at one sector, renewable energy, and morespecifically wind, an SME with access to an HPC to assist in the selection of awind farm site, can make a site selection that generates 25%-50% more power thana site selected without access to HPC’s. The result is a competitive SME, HQPjob growth, and a greener economy.” (kmacneill)

39 | P a g e

APPENDIX 4: HPC/GDP

HPC/GDP (2005-2009 average)

0.030.050.060.07

0.140.170.180.190.200.200.210.210.22

0.250.280.28

0.320.370.380.39

0.530.540.540.560.56

0.590.61

0.630.640.65

0.670.690.72

0.760.79

1.241.261.27

1.502.34

2.46

0.00 0.50 1.00 1.50 2.00 2.50

TurkeyIndonesia

UAEMexico

BrazilSouth Africa

EgyptAustria

BelarusBelgium

DenmarkSingapore

IrelandLuxembourg

ItalyPoland

AustraliaCanada

Korea (S)RussiaCyprus

NetherlandsChinaJapanSpain

NorwaySaudi Arabia

TaiwanMalaysia

IndiaFranceFinland

SloveniaBulgaria

GermanyUK

SwedenSwitzerland

IsraelNewZealand

US

HPC/GDP (relative to mean of "G6")

G8 U G10 (excluding Canada)

Canada

Other advanced/developed nations

from comparator group (Fig 1)

Other HPC-invested nations

Fiducial factor of 2 above and below mean of "G6"

40 | P a g e

APPENDIX 5: Strengths, Weaknesses, Opportunities and Threats

Strengths Pan-Canadian national HPC platform being consolidated and integrated with substantial infrastructure and researcher access to machines and expertise across Canada HQP, solid base of HPC expertise Distribution of user support staff to allow close collaboration with research

teams Support for “big science” projects/international collaboration Enhances federal and provincial investments in research Enabler for innovation On-going examination of technological advances in equipment and software development to

ensure researchers needs can be met Excellent university research using HPC Acquisitions are cost-effective and represent good value for the funds available

(IAP Report) Contribution of committees/regional staff to Compute Canada Ability to coordinate activities at the national level Quality facilities and support provided within a very limited budget

Weaknesses The governance structure needs to be strengthened Insufficient staff and resources in Compute Canada’s central office Insufficient user support personal Lack of a Tier 1 facility Limited outreach to non-traditional research disciplines Few connections between Compute Canada and industry Low profile of Compute Canada Researchers/scientists have been diverted to the management of HPC facilities Dependence on “volunteers” to serve on committees reduces the time scientists have for

science Lack of service standards for data centres Lack of a robust business model

Opportunities

Industrial outreach through training courses, workshops and collaboration with academic researchers

41 | P a g e

Development of a national approach to managing large research data collections in conjunction with other national organizations

Increased use of HPC in the humanities and social sciences Creation of a shared environment that meets the needs of the medical research community Convince CFI to aggregate small HPC grants to expand and sustain the national platform Green HPC/energy efficiency Increase collaboration/partnership between research and industry, particularly support to SMEs

to demonstrate economic impact Implementation of a Tier 1 facility and readiness for the arrival of exascale computing; Create career progression opportunities for HQP and build Canadian expertise Implement a cloud computing test bed

Threats

Increasing cost of power Limited political support for or recognition of the strategic importance of HPC to research and

the economy Funding is significantly lower than that of our trading partners and Canada is falling behind in

providing HPC to support research Dependency upon multiple sources of funding creates several different sets of rules and

accountability, making integration difficult Lack of sustained, long-term funding Lack of funding for HPC research and limited resources to research HPC systems, hardware,

software, middleware and trends Loss of HQP/ brain drain to other countries where HPC is a priority and well-funded Funding for HPC infrastructure and personnel unknown after March 31, 2012 Lack of integrated federal and provincial funding for HPC HPC equipment is very expensive and has a short useful life. Almost all information technology

equipment grows obsolete before it stops working. The well-known Moore’s law states that computer performance, or “bang for the buck”, doubles every two years

Movement of researchers to private or commercial clouds if Compute Canada funding is insufficient to provide the resources they require

42 | P a g e

APPENDIX 6: S&T Priorities: Examples of Support by Compute Canada

Research Area: Environmental Science and TechnologiesResearch Objective The reduction of carbon emissions given its economic and societal

priority and significance in Canada through better understanding and optimization of fluidized bed catalytic gasification. Gasificationrefers to the conversion of solid or liquid carbonaceous material to aclean synthesis gas (syngas) product. Gasification can producehydrogen, which can be used for green energy sources such as fuelcells, while also producing a concentrated carbon dioxide streamsuitable for capture-and-storage sequestration. In a world with highfossil fuel prices and low emissions targets, it is expected thatenvironmentally friendly gasification of low-grade coals and otherbiomass or organic waste will become an increasingly importanttechnology in the production of green energy sources.

Objective with current resources Despite the astounding capabilities of modern computing, producinghigh-fidelity simulations of industrial-scale gasifiers remainsimpractical without specialized computing equipment. Even using relatively new equipment from Compute Canada and thestate-of-the-art US Department of Energy software MFIX, it typicallytakes 24 hours of real (clock) time to produce simulation of 50seconds on a model domain that is ten thousand times smaller than a real industrial gasifier. Furthermore the anticipated memory required for a realistic simulation is much larger than what is available. Performance numbers such as these make it impossible to take full advantage of the power of simulation for practical purposes.

Objective with Tier 1 Petascale computing will allow the production of realistic data that can be compared to experiment to further improve mathematical models. It will also provide a framework for the development of better simulation algorithms and ultimately perform process optimization.

Potential Application The transfer of knowledge to industrial partners.

43 | P a g e

Research Area: Environmental Science and TechnologiesResearch Objective The problem of global climate change and climate change projection

is a widely acknowledged "grand challenge problem". Projections of the ongoing impacts of the global warming process into the future for a century or more are required input to the formulation of the appropriate national and international policies that are needed to minimize harmful impacts on the planet’s life support systems.

Objective with current resources Present capabilities of computer systems in the fraction of a petaflop (1000 trillion calculations per second) range enable global projections based upon assumed Representative Concentration Pathways (RCP's) for greenhouse gas increase, the increase to be performed at a horizontal spatial resolution of approximately 100-200 km. This resolution is far to coarse to serve as a basis for regional environmental policy formulation. At present computing systems are able to produce useful finer scale projections only through the application of dynamical downscaling techniques. A further issue in climate change projection is the fact that existing models currently do not include a number of system components that are expected to be crucial on the century timescale over which useful projections are required. These include sub-surface hydrology required to capture the impacts of climate change on the depth of the water table and the additional climate response associated with the reaction of the great polar ice-sheets to the warming process. Both of these hydrological impacts are seriously compromised by inadequate spatial resolution of the models.

Objective with Tier 1 The availability of true Tier 1 capability will enable global scale models to be integrated at the high spatial resolution required to address these critical issues without the need to invoke dynamical downscaling techniques which are significantly error prone.

Potential Application National and international policy makers will have reliable information with which to make the decisions required to address issues related to climate change impacts and adaptation.

44 | P a g e

Research Area: Natural Resources and EnergyResearch Objective Associated with meso and microscale modeling of wind flows is the

application to wind turbine aerodynamic modeling (so power performance of individual turbines and windfarms- especially interactions of turbines with the wakes of other turbines in a particular site) and noise generation. Noise (both broadband and tonal) is one of the biggest hurdles for widespread application of large wind turbines. Being able to model the noise generation can lead to design changes that reduce turbine noise. Being able to model and understand the propagation of the noise from the turbine to the surrounding ground can lead to better regulation, limits and zoning.

Objective with current resources With current resources we can model turbine aerodynamics and power performance, because this needs only a fairly coarse resolution (still on the order of 100 million control volumes, and weeks with hundreds of processors). Multiple runs to evaluate changes and optimization are unfeasible. Modeling windfarms and wake interactions in real settings is possible only by crudely representing a turbine as a simple disk that extracts power from the local wind - all effects such as rotation in the wakes of the turbines in a windfarm are neglected. We are stretched to be able to do the very smallest computational aero-acoustics simulations of the simplest geometries (weeks with thousands of processors for a simple flow impinging on a flat plate). Computational aero-acoustics of turbine blades is beyond current resources.

Objective with Tier 1 With Tier 1 resource, we can do optimization of wind turbine bladesfor power production, and interacting turbines in a windfarm without gross simplifications. We would be able to do computationalaero-acoustics of turbine blades and gain understanding of noisegeneration mechanisms and how the noise propagates.

Potential Application Wind farms would be placed to maximize power output while ensuring the health and safety of people living near them.

45 | P a g e

Research Area: Natural Resources and EnergyResearch Objective Turbulence is a multi-spatial and time scale phenomenon that

pervades industrial and environmental air and water flows. It alone remains as the last unsolved problem in classical physics. In combination with combustion, the necessary predictive capability is severely limited because current models can only predict NOx, CO, and soot emission trends not levels, and cannot accurately predict turbulent burning rates as turbulence intensity increases, and are not able to predict combustion instabilities for realistic, technologically relevant configurations.

Objective with current resources Existing HPC facilities do not currently permit high-fidelity simulation of the entire complex geometry of full-scale practical combustor systems under realistic operating conditions (pressures, temperatures, and turbulence levels), including coupling with other engine and/or burner components without significant use of in many cases greatly simplified modeling. For this reason, quantitative predictions of unsteady, thermo-acoustic phenomena and combustion instabilities are currently not possible. Such capabilities are required for developing low-temperature, low-emissions burner configurations. Additionally, today's high-fidelity simulations cannot provide quantitative predictions of most emissions and are either too costly computationally or still not feasible to be used in formal design optimization procedures for new combustor designs.

Objective with Tier 1 Access to a Tier 1 facility would leverage existing Canadian research expertise and accelerate the advancement of numerical combustion science and enable Canada to be among the world leaders in the development of the next-generation, high-fuel-efficient, low-emissions combustion technology for conventional and emerging alternative fuels. In particular, Tier 1 facilities would permit the following:

DNS of larger laboratory-scale flames at higher more practical levels of turbulence with more sophisticated and accurate physical modeling, thereby performing simulations containing more of the relevant physics, results from which will yield a greater understanding and feed the development of significantly improved and more accurate LES and hybrid RANS/LES models.

LES and hybrid RANS/LES based simulations of full-scale, industrial, combustor configurations at realistic turbulence levels using more complete and more accurate sub-scale physics models. Use of finer computational mesh (billions of

46 | P a g e

nodes) to alleviate the high reliance on modelling of unresolved scales and provide quantitatively accurate predictions of emissions such as CO and soot.

Numerical simulations of full-combustor configurations would be able to incorporate a full description of the complex three-dimensional combustor, including coupling with other engine and/or burner components, and thereby enable realistic and accurate predictions of thermo-acoustic phenomena and the onset combustion instabilities.

The use of high-fidelity simulations in formal design optimization procedures for new combustor designs.

Potential Commercial Application In terms of fuel utilization, Canada has also become the international leader in design and manufacture of small gas turbine engines for aviation applications (Pratt & Whitney Canada) and has made significant inroads in the development of small and large gas turbine engines for industrial and power generation applications (Rolls-Royce Canada). Pratt & Whitney's operations and service network span the globe and its engines power the largest fleet of business and regional aircraft and helicopters. They employ 10,000 people worldwide including 7,000 in Canada. In 1997, Rolls-Royce Canada was designated the worldwide centre for design and manufacturing of all large Rolls-Royce aero-derivative industrial gas turbine engines for power generation and they employ a highly educated workforce of more than 1,000 people in Canada. To remain competitive, Canadian gas turbine engine manufacturers will need to develop new low emissions technology for future advanced engines. They will also require new combustion technology for alternative (non-fossil) fuels that will enable them to competitively enter new markets that are expected to arise from the introduction of more stringent environmental regulations. As an example of changing policy, the International Air Transport Association (IATA) recently outlined the aviation sector's commitment to environmental responsibility, which includes the use of 10 percent alternative fuels by 2017.

Research Area: Health and Related Life Sciences and Technologies

47 | P a g e

Research Objective The area of multiscale modeling of biological systems and processes is advancing rapidly. One can envisage spatial scales going from the picometer scale of electronic and atomic motion in biomolecules (nanometers) to the collective behavior of those molecules in a cell (micrometers) and beyond to the formation of tissues (millimeters) from cells, organs (centimeters) from tissues, organisms (up to meters) from organs and ecosystems (kilometers) that involve organisms interacting with each other and with an environment. Similarly large ranges of time scales apply, from femtosecond chemistry, to nanosecond molecular dynamics, to millisecond conformational changes of biomolecules, to the beating of a heart, and so on. Researchers are currently examining small portions of each of the space and time scales. The next great challenge is to combine these approaches into integrated multiscale models of biological phenomena.

Objective with current resources Let us take, as an example, the lower end of the spatial scale that runs from atoms to cells. Similar considerations will apply to research activities that are aimed at coarser levels of spatial resolution. With current Compute Canada resources leading researchers are working with costly quantum mechanical (QM) methods that have high accuracy but are limited to a maximum of 1000 atoms. On-the-fly molecular dynamics are typically limited to some tens of picoseconds, even less for the larger systems. In the world of atomistic Molecular Dynamics (MD), the quantum mechanical behaviour is replaced by a Molecular Mechanical (MM) force field which allows many aspects of the dynamics and statistical mechanics to be explored for much larger systems (100 000 atoms) for longer simulation times (the state-of-the-art is approaching a millisecond). Coarse-grained approaches, in which a number of atoms are frozen together into a combined particle whose external dynamics is then studied, will increase these limits to perhaps a million atoms and simulation times of the order of seconds. These classical MM simulations do not allow the possibility of treating chemical reactions so one of the current paradigms combines QM and MM approaches in so-called QM/MM methods. A typical example would follow a chemical reaction taking place in the central QM part of a protein with a model of around 100 atoms in the active site, embedded in an MM model of the surrounding protein and water of 100 000 atoms. Current efforts focus on determining appropriate methods for following the free energy of the reaction with good statistical sampling of the motion of the environment.

Objective with Tier 1 Having substantial allocations on a tier-1 (peta-scale) computer would allow many of these applications to be taken to the next level, towards a true integration of time and length scales. It would, for example, be possible to follow a chemical reaction with significant conformational changes and important fluctuations. Examples that

48 | P a g e

are currently being formulated and that would be ready for peta-scale facilities include the study of transcription by the RNA-polymerase enzyme, a drug-delivery model that involves siRNA in a lipid mixture with water/salt and nano-pores that are being considered for DNA sequencing devices. The proof of principle for simulations on the atoms-to-cells scale should be feasible on a peta-scale machine, preparing the ground for the exa-scale which will extend the models to tissues and organs.

Potential Application While most of the frontier work is of a fundamental nature, aimed at understanding the processes of life, there are already applications in drug design and in biomedical engineering devices, for example.

Research Area: Communications and Information Technology

49 | P a g e

Research Objective While silicon is the material backbone of current communications and information technology, advanced materials are constantly being developed and studied for better performance either in processing, switching or for displays. One can also envisage entirely new technologies based on molecular electronics or on quantum computers. In all cases, accurate ways of simulating material properties will accelerate development. Examples of materials under study include graphene, various organic conductors and even superconductors.

Objective with current resources Current resources allow first principles calculation of electronic properties of materials for elements containing valence electrons in the s or p shells in the first few rows of the periodic table. Commonly used programs include Wein2k, Abinit, Gaussian etc… By contrast, in the case where electron-electron interactions are important, either in narrow bandwidth organic materials or for elements containing d-shell electrons, as one often finds in unconventional superconductors, the difficulty of the calculation increases considerably. Only highly simplified models can be studied. Current resources can combine first-principles calculations with methods that take into account strong electron-electron interactions but only at a rudimentary level. For example, in the new pnictide superconductors, such methods can handle only the normal state, not the superconducting one.

Objective with Tier 1 To perform more realistic atomistic modeling and to take into account the effects of disorder and of nano-scale inhomogeneities that are important in practice, increases by orders of magnitude in computing power are necessary. As an example of what has already been achieved with about 50,000 cores, a new algorithm to enable 400+ TFlop/s sustained performance in simulations of disorder effects in high-Tc superconductors (G. Alvarez, et al. SC 2008, IEEE Press, Piscataway, NJ USA 2008) has won the Gordon Bell Prizes awarded by the Association for Computing Machinery in conjunction with the Institute of Electrical and Electronics Engineers each year at the Supercomputing Conference to recognize outstanding achievement in high-performance computing applications. High-temperature superconductors are considered grand-challenge problems but other new materials with interesting electronic properties pose the same challenge.

50 | P a g e

Potential Commercial Application Patents on materials that form the basis of new electronic devices for the communication infrastructure could lead the development of entirely new industries. Even software companies could emerge to develop and sell the successors of Wien2k, Gaussian and the like. For example, the company Atomistix which develops software solution for nanoscale electronic device modeling was founded in 2003 based on such research at McGill University (See http://www.quantumwise.com/ )

51 | P a g e