South Africa eScience / eResearch … the next steps?

PLAN-E

Third Plenary Meeting

20150904

9/4/2015 PLAN-E 1

Prof Colin Wright Special Advisor: Cyberinfrastructure

Department of Science and Technology cjwright@csir.co.za

9/4/2015 PLAN-E 2

What is the problem? a. Create a National Integrated Cyberinfrastructure System (NICIS) out of

3+ stand alone entities: i. Tier 1 national HPC Centre; NREN & SAGrid; Data Initiative ii. Promote / facilitate Tier 2 and Tier 3 Centre

b. Develop NICIS into an e-Infrastructure Commons c. Design and establish National eScience facilitating framework

i. Create a new National eScience Education intervention ii. Persuade NRF to consolidate their presently scattered e-Research

funding into a programme iii. Establish national eScience Initiative: federated hub and spokes

model d. Challenges

i. Seek integrated vertically and horizontally: build partnerships ii. Existing Building blocks iii. Governance: stakeholder involvement iv. DST funded…..other Govt Departments? v. Build an ecosystem

e. Build international collaborations i. SADC ii. Global partners

9/4/2015 PLAN-E 3

… and then PLAN-E appeared on the horizon!

• The principal goal of PLAN-E is to act in support of

Enhancing Science • Objective of PLAN-e is to bring together leading influential

e-Science centres across Europe to help coordinate ongoing innovation in scientific methods and exploitation of infrastructure

• Goals of PLAN-E cover all the topics that help promote the escience approach and strengthening the groups and centers conducting eScience.

9/4/2015 PLAN-E 4

… and then PLAN-E appeared on the horizon!

More particularly, PLAN-E: • *** Forum for exchanging knowledge and expertise in the

field in order to strengthen the EU eScience position; • *** Discusses common approaches to eScience; • Communicate eScience and results in all disciplines; • Represent the European eScience scene as embodied by

the PLAN-E community externally and internationally in addition to the individual representations from the participating members where applicable. In particular towards the EC in relation to future funding schemes;

• *** Propose evaluation criteria for the quality, impact and benefits of eScience activities;

9/4/2015 PLAN-E 5

More particularly, PLAN-E: • *** Supports actions towards data stewardship and

software availability and sustainability; • Endeavour to stimulate quality and quality ranking of

eScience publishing means; • *** Facilitates interaction between its members; • *** Provide eScience requirements towards improved e-

infrastructure provisioning and usage; • *** Communicate best eScience practices regarding the use

of e-infrastructures and ICT tools; • *** Strive for improvement of the skills-level of students

and researchers in eScience techniques and stimulate the upgrading of the status of eScience technologists and engineers.

Strategic Science Imperatives led DST into the arena

VLDB /

DIRISA

South African National e-Infrastructures

2006

9/4/2015 PLAN-E 6

VISION: take national leadership in the provision of a comprehensive Cyber-Infrastructure essential to 21st century advances for South Africa in research, education and innovation. MISSION: increase knowledge creation through provision of a national platform of essential Cyber-Infrastructure.

2012/3 Review: National Integrated Cyber-Infrastructure System

7 9/4/2015

NICIS PRINCIPLES • Joint planning and budgeting • Good governance • Visibility of CI services • Sustainability • Constructive stakeholder

engagement NICIS: TIER 1

PLAN-E

Advanced Services

Governance Strategy Advisory

Board (Stakeholders,

principals) Senior Management

User Inputs

Network Infrastructure

Computing Infrastructure

Research Data Infrastructure

LHC Astro

Biol

Med

Social

Econ Enviro

NICIS: SA National Integrated CyberInfrastructure System

Support

Policy

(International)

Cooperation

and Links

9/4/2015 PLAN-E 8

9/4/2015 PLAN-E 9

Networking Services: NREN o SANReN: advanced services development and backbone network

infrastructure architecture and infrastructure, o TENET: operating the network and providing production level services. Collectively, SANReN and TENET constitute NREN.

+

– Approximately 1 million users (reseachers, scientists, engineers, educators, students)

– 10 Gbps core network, 4 major metro networks, 204 sites with primarily 1 Gbps / 10 Gbps fibre connectivity; SEACOM and WACS connectivity

– All main campuses of SA universities; research councils e.g. CSIR and ARC; SKA core site; MeerKAT; CHPC; SANAP (SA Antarctic Research Station)

– Excess of 1500km dark fibre and 5000km managed bandwidth, over next 4 years backbone upgrade to 100Gbps and upwards

– Eduroam; Federated Identity Management platform (SAFIRE); Mconf web conferencing; DMZs; perfSONAR

– LightPath connectivity for HartRAO’s participation in eVLBI (4Gbps)

– Connectivity SA institutions participate in CERN LHC experiments

– Coordinate SAGrid community (Є EGI) and lead development of its services

– SANReN WACS support SA SKA bid.

SANReN and TENET: The SA NREN

9/4/2015 PLAN-E 10

Centre for High Performance Computing

Machines • 2015 upgrade • Tsessebe 61.5Tf • Blue Gene/P 11.5Tf • iQudu 2.5Tf • GPU cluster 16Tf • VLDB data storage

CHPC in essentially its current form take on the role of the Computing Services area, with some changes to its mandate.

• Compute-intensive, • Communication-intensive • Data-intensive • Sustainable, impactful, user sensitive • Visualisation, Cybersecurity

CHPC T1

T2 T3

• Cultivating African Links • NICIS: T1 • NRF & DHET: T2 & T3 • CERN site • SKA

9/4/2015 PLAN-E 11

Human Capital Development

• Annual Student Supercomputing Cluster Building Challenge: In 2013 the SA team (6 students selected from the national winners) won the ISC’13 international competition, and this title was successfully defended by the team of 2014. This was never achieved by any country before. 2015 too second place.

9/4/2015 PLAN-E 12

9/4/2015 13 PLAN-E

DIRISA be the Tier 1 organisation to advocate for and implement data initiatives across the research community. NICIS-DIRISA will work with the community to develop an ambitious proposal on data services to DST. The data services recommendations are to be implemented in their entirety.

Data Services: DIRISA

9/4/2015 14 PLAN-E

Big…!?

Research data lifecycle model adopted by the UK Data Archive

Not only Big—or Extreme—also heterogeneous long tail

9/4/2015 PLAN-E 15

Presently knitting these into a coherent e-Infrastructure • Recognise differences and share common tasks • Manage remit creep • Create an advanced services layer to provide the “commons” one stop shop • Engage meaningfully with and involve users • Recognise and acclaim that users have different needs:

o Commodity connectivity but also software defined networking o HPC, HTC, Grid and Cloud +…. o Extreme, Big, Long tailed / dark data

• The shortage of e-I skills is global phenomenon. • Collaborations between the national organisation and the universities, particularly in

the area of human skills and training. • Build cohort of data professionals to support research infrastructure development.

But what about the dearth of skills?

1. Co-develop and co-ordinate courses with HE sector

2. Formulate and establish e-Science / e-Research National Master

3. Develop eScience skills at multiple levels • Entry level researchers • Mid-career researchers • Specialist & General

4. Appropriate specialist and training skills are not all at same place—silos?

5. Collaboration 6. Other examples

o UK Doctoral Training Centres o Swedish model o …?

9/4/2015 PLAN-E 16

• Bioinformatics

• Climate change

• Southern Oceans

• LHC CERN experiments • Digital Sociology

• Health Care

• Virtual Research Org

• Genome

• Earth Observation

• TB & HIV research?

SA … Some Data Driven Research Challenges – local Data

Tsunami

9/4/2015 PLAN-E

e-VLBI

9/4/2015 PLAN-E 18

Further e-Infrastructure / eResearch developments

NICIS is the T1 component of this

Development of a South Africa Research Infrastructure Roadmap

13 RIs identified across the 6 thematic areas

9/4/2015 PLAN-E 19

Northern Cape/ Karoo

9/4/2015 PLAN-E 20

Northern Cape/ Karoo

Karoo Radio free zone

• KAT-7 • Meerkat

9/4/2015 PLAN-E 21

• Supercomputer in the desert—or in Cape Town?

• Extreme data. • How much data to ship

out? • Energy consumption? • Green ICT in the semi-

desert?

9/4/2015 PLAN-E 22

Provide national guidance and leadership to establish eScience Institute:

• 25 Universities: City and Rural, Research intensive and developing • Research Councils: some have embraced eScience whilst others have

not • NRF funded research publications and supporting data deposited open

access repository … but? • DST/ NRF SARChI: SA Research Chair Initiative • DST/NRF Centres of Excellence • Plethora of appropriate research challenges • Skills lack amongst academics and researchers • Skills shortage and where exist are thinly distributed • Citizen Science

Academia Science councils

: RI’s

Core Services

Networked resources

• National data integrative enabler supporting

– MTSF

– Research Strategy

– SARIR,…

• Overarching coordination & national strategy

– National (Tier1)

– Institutional (Tier2)

• Amalgamated, physically distributed cyber platform for data intensive research

– Data

– Networking

– Computing

– Crosscutting

– S&T

• Principles

9/4/2015 PLAN-E 23

NICIs T1 Platform for eScience

Computing Services (CHPC +)

Networking Services

(SANREN)

Data Services

(DIRISA +)

Materials & Manuf.

Energy Earth & Environment

Phy Sci & Eng.

Humans & Society

Health, Bio & Food

Compute & Data intensive research environments (SA_Grid, Cloud. FIM, SDN, perfSONAR, VREs, …)

Ph

ysic

al-S

ervi

ce

Sup

po

rt

Ap

plic

atio

n

Skills & Training / VREs / Collaboration (Comp Sci-s, Data Sc, Stats, NA, Visualisation, Soft Eng, ...) Sk

ills

Astro

NICIS ecosystem

9/4/2015 PLAN-E 24

UCT eResearch Centre

9/4/2015 PLAN-E 25

NICIS eResearch Activities Other partners

Support Innovation Spin-offs, industry

Support . [Open] Information & Data repositories HEIs, SC, Publishers

Support ? [Open] Research publications & outputs Researchers

Support -facilitate

~ Domain e-Research: Professional; Citizen Science DST, NRF, other Depts, Researchers

Develop ~ Research into e-Research methodologies: scientific workflows, tools

DST, NRF, Researchers

NICIS # Research into e-I infrastructures & tools NRF, HEIs, vendors

Strategic facilitator

. National e-Science Graduate programme

HEIs, MOOCs, vendors

Support ~ [Open] Raw data (as infrastructure) Researchers

Build # One stop e-Infrastructure Commons (T1 + T2 + T3) Partners

NICIS + TENET

* Middleware & services; Policies; Handles and PIDs, DMP good practice, data stewardship…….

Vendors

CHPC DIRISA

* Computers, storage T1 * T2 ~ T3, T4 ~ HEIs, SC, collaborations

SANREN * NREN Institutions

National e-Infrastructure Commons and e-Research stack

9/4/2015 PLAN-E 26

Strategy … not always intentional … has been • Develop eResearch in non-coherent pockets • Build e-Infrastructures—local then in >2006 national • NRF support SARChI, CoEs, graded researchers in eResearch • 2012/13 review recommends National Integrated CI System • NICIS beginning to come together

• T1 + T2 + T3,4 form an ecosystem • Separate CHPC, SANReN + TENET and DIRISA, but

• Single strategy, senior governance and national T1 funding line • Must be stakeholder sensitive but Users pay for O&M • CI commons cross cutting layer: Support Institutions, Researchers &

Academics; Single national call center etc • Other Govt Depts showing interest: DHET, DAE, DTPS now • e-Science / e-Research

• Build e-Research ecosystem • Work with NRF to develop e-Research funding track across Instruments • National e-Research Masters • National e-Research Institute … perhaps? • Encourage T2 & T3 e-Research Centers.

• Sustainability…in all its dimensions?

9/4/2015 PLAN-E 27