parallel session: data centres

Parallel session H: Data centres

Chair: Guy Sudron

Please switch your mobile phones to silent

19:30

No fire alarms scheduled. In the event of an alarm, please follow directions of NCC staff

Dinner (now full)Entrance via Goldsmith Street

16:30 - 17:30

Birds of a feather sessions

15:20 - 16:00

Lightning talks

Jisc shared datacentres

Guy Sudron

Shared datacentres update

»The Jisc portfolio

»The Jisc frameworks

»Current status

03/05/2023 Jisc shared datacentres

Jisc’s role»Procurement management and contract authority

»Investment to start up the datacentre› Procurement costs› Legal costs› Connection of the datacentre to the Janet core

»Ongoing promotion & awareness to drive uptake

»Anchor tenant group facilitator

»The datacentres as core to a broader ecosystem of services03/05/2023 Jisc shared datacentres

Shared datacentre (south)

»Provided by VIRTUS, located in Slough»Connected to Janet at 2 x 400Gb»Framework launched in September 2014


SDC(south) tenantsAnchor tenants Other tenants


Current status SDC(South)

3 Jisc datahalls, c240 racks = 2.4MW

60/40 split HPC vs Enterprise

Projects: eMedlab, MedBio03/05/2023 Jisc shared datacentres

Our new shared datacentre

»Conversations with universities based in the north of England who had visited Slough led to:

› Full OJEU procurement, led by Jisc

› Working with 4 anchor tenants

› Contract signed in September 2016


Shared datacentre (north)

»Available now

»Provided by aql from their new DC5 in Leeds

»Connected to Janet at 2x100G initially› Network live on 30th March


SDC(north) tenants


Anchor tenants Other tenants

The Jisc shared datacentre model

»Jisc frameworks› 5 + 5 +5› Full OJEU procurement

»Designed for education and research› Research and enterprise applications› Flexibility and scalability

»Datacentres connected to the Janet core03/05/2023 Jisc shared datacentres

The Jisc shared datacentre model (2)

»Commercial offering

› Very favourable rack pricing (regularly benchmarked)

› Contractual PUE

› Pass through price for power


Shared datacentre model (3)

»Flexible services

› Able to ‘mix and match’ racks of varying power density with no compromise in terms of power or cooling

› Racks can be supplied and installed by the provider or by the customer

› Smarthands; storage and assembly space; office space etc.

› Flexible networking – layer3, layer2 VLANs, dedicated wavelength03/05/2023 Jisc shared datacentres

Lessons learnt

»Perceived barrier of location away from campus not an issue, but does need support from senior management

»SDCs allow central IT to contribute to institution’s ‘green agenda’ and improve efficiency

»Don’t under estimate the need for re-architecting services

»Clear, transparent, and manageable costs»Network latency not an issue, but architecture and

services need to be planned03/05/2023 Jisc shared datacentres

Findings to date

Estates DC capacityRe developRelease for

T&L

ResearchSharedInstitutionAttract

fundingDatacentre Flexibility

Back-upPrimeHybridShared

servicesSavingsVolumeConnectivit

yProcuremen

tPower

Soft benefits recognised» Collaborative R&D» Shared resource» Shared learning» Best practice (HPC)

» Desire for on-campus research data

» Some have capacity

» Tech, people, RDM, storage shared services

» Build v buy» Latency/location

» Recognition that collaboration (incl health) is important

» Shared infra = best use of public monies

» VAT» ‘herd’ of tenants to max

savings» Power, build and plant – similar

costs

» Space NOT always at a premium


jisc.ac.uk

Contact me

Guy Sudron

[email protected] 822292 | 07941 657127

http://www.jisc.ac.uk/

Jisc Services and Their Part in Data Centre

Strategies SDC North

03/05/2023

Outline of Presentation

»Why The University of Liverpool supported the project»What we were seeking from SDC North»What did we learn from the process»Where does SDC North fit in with UoL’s overall strategy»What other services would we like to see in

conjunction with SDC North?

Jisc Services and Their Part in Data Centre Strategy

03/05/2023

Why UoL Supported the Project

»Recognised that the growth of IT services could not be met entirely with on prem. solutions

»The original SDC in Slough had been a success and delivered a service for sector tailored for their needs

»Important that we had input from as many northern universities as possible

»The Jisc/Janet brand is strong and would encourage use in the sector


03/05/2023

What We Were Seeking From The Project

»As part of a consortium it was clear that the group had more bargaining powers than a single entity

»Universities have very diverse requirements and need flexible contracts

»Having an offsite DC with solid Janet connections was a

major factor»As a member of N8 it was important that we helped to

support the requirements of HPC as part of the bid


03/05/2023

What Did We Learn From The Process

»We were aware of the amount of time needed to support this and it was quite a commitment

»Jisc’s procurement team are first class. The complexity of the procurement would have been too much for an individual university

»We were able to get all our requirements addressed. This was no mean achievement given the range of options

»Being able to have a contract for a single rack to a full HPC installation was impressive.


03/05/2023

Where Does SDC North Fit In With Uol Plans

»The UoL’s data centre strategy covers local, colocation and cloud services

»The colocation element is especially important. It is the one that requires a robust agreement to meet university needs.

»Primarily we wanted a small presence for BC. The agreement allows for this and we still benefit from reduced rates created by the larger tenant groups

»The SDC is the basis for other shared servicesJisc Services and Their Part in Data Centre Strategy

03/05/2023

What Other Services Would We Like To See

»The presence of Jisc in the DC allows the development of other shared services

»Potential for UoL are archiving services via the DC, connectivity to Cloud providers, a storage service etc

»Options to create collaboration with other Universities.»Access to data sets held locally at the DC seen as

advantageous.»Potential development of other HPC services


03/05/2023

What Other Services Would We Like To See

»Thank you for your attention – Any Questions?


jisc.ac.uk

03/05/2023

Steve Aldridge

University of Liverpool


http://www.jisc.ac.uk/

Data CentresProfessor Paul Jeffreys

The Institute of Cancer Research

03/05/2023All Aboard the Cloud Express

, Prof P.W.Jeffreys, ICR, Networkshop, April 2017

Data CentresICR’s Distributed Storage Platform

at the Jisc Shared Data CentrePaul W Jeffreys

12 April 2017


29

Our mission is to make the discoveries that defeat cancer.

Our mission is to make the discoveries

that defeat cancer


Presentation Outline

1. ICR / RM - very much part of the story

2. Existing ICR Research Infrastructure

3. ICR Research Data Storage Programme

4. RDS Service: solution, rationale, design

5. Overview of RDS Service


1. Institute of Cancer Research and

The Royal Marsden


ICR at a glance


Academic successes33

The ICR is ranked as the top academic research centre in the UK. cCame first in the Times Higher Education league table of university research quality compiled from the Research Excellence Framework (REF 2014)

Together with The Royal Marsden we are rated in the top four cancer centres globally measured by citation impact

Joint top of the Times Higher Education table for Innovation – based on worldwide citation of our research in patents. Joint top of the Times Higher Education table for Innovation – based on worldwide citation of our research in patents.


Our substantial and diverse funding34

2016• Total income £162m• HEFCE 12% based on research excellence• Grant income 38%• Legacies and donations 7%• Invention income from our discoveries 16%

Total incoming resources 2016

12%Higher Education FundingCouncil for England

38%Research grants

16%Royalty income

7%Legacies and donations

3%Investment and tuition fees

24%Sale of part of our future royalty stream


Royal Marsden Hospital35

The ICR’s unique partnership with The Royal Marsden and “bench-to-bedside and back” approach means we make discoveries and deliver clinical impact in a unique way

We have an outstanding record of research achievement dating back more than 100 years

Together we are rated in the top four comprehensive cancer centres worldwide for impact on cancer research and treatment


ICR and Royal Marsden Vision

Dynamically adapt treatment to individual patient over time and in response

Requires: data integration and Big Data analytics


Making the discoveriesOur strategy to defeat cancer

37

The ICR and The Royal Marsden have worked together on a joint strategy covering the next five years.

Our visionWe will overcome the challenges posed by cancer’s complexity, adaptability and evolution through scientific and clinical excellence, innovation and partnership


Making the discoveriesThe four pillars of our strategy

38

23

Unravelling cancer’s complexityWe will comprehend the full complexity of cancer by harnessing the power of new technologies and Big data

4

Innovative approachesWe will take on the challenge of cancer’s complexity, evolution and drug resistance through the discovery of innovative new approaches to cancer treatment

Smarter, kinder treatmentsWe will move a step closer to cure by bringing personalised treatments into the clinic and developing them for patients

Making it countWe will deliver better outcomes and improved quality of life for patients by establishing innovative treatments, diagnostics and strategies for prevention as part of routine healthcare

1


Two sites in London:Sutton and Chelsea

39

Chelsea

39

Sir Richard Doll Building Opened 2006

Centre for Cancer ImagingOpened 2015

Brookes Lawley Building Opened in 2003

Centre for Molecular PathologyOpened 2012

Chester Beatty LaboratoriesFulham Road

Sutton


ICR Research Divisions

Breast Cancer Research

Cancer Biology

Cancer Therapeutics

Clinical Studies

Molecular Pathology

Radiotherapy and Imaging

Structural Biology

Genetics and Epidemiology


The London Cancer HubA global centre for cancer innovation

41

We plan to deliver an exceptional environment for cancer research that enhances the discovery of new treatments and their development for patients.

We will provide state-of-the-art facilities, and be joined by a multitude of high-tech enterprises in a network of 10,000 researchers, clinical staff and support staff all operating from one site.

The London Cancer Hub aims to create a world-leading life-science campus specialising in cancer research, diagnosis, treatment, education and biotech innovation.


2. Existing ICR Research Infrastructure


Existing Research Infrastructure:End-of-life Data Storage Service• NGS – 90%

HPC Cluster• At Jisc Shared Data Centre: 1080 cores c. 38GFlops

• Similar capacity in Chelsea/Slough• NGS – 70%• 70% average use

Jisc Shared Data Centre:• Most of HPC resource will be based in Jisc data centre at Slough• No remaining space and power in London

Implementing new ICR Research Data Storage (RDS) Service


3. ICR RDS Programme


ICR RDS Programme


RDS Service User Requirements

Collected from researcher workshops:

• 6PiB* storage, expandable to growing research need• High level redundancy ensuring robust solution• Cost effective and competitively priced solution• Rapid access to all levels of data• Ability to ‘snapshot’ to protect against accidental loss of datasets• Ability for researchers to manage data transfers between Tiers• Staging Areas to guarantee access to storage from instruments

* PiB is a binary unit of storage equivalent to 1.13PB (the decimal version)


Research Data Growth

“Imaging instrumentation is currently in a period of rapid technological advance. It is therefore difficult to project the likely data acquisition volumes beyond a relatively short predictive horizon (for example 6 – 9 months)”


RDS Service Decisions

New type of leading, innovative, cost-effective service for ICR

Decisions made in partnership with researchers:

• Single service for research data under active use

• Service with two tiers (best and most cost-effective)• Choice of solution / supplier• Replication / resilience• Expandability / flexibility• Disk only• No “backup”• Single namespace; straight forward to use


4. RDS Service

solution, rationale, design


RDS Service Schedule24 Mar 2016: Invitation to Tender (ITT) published

29 April 2016: ITT exercise complete• Different types of solution offered• Best and final offers requested

31 May 2016: Preferred supplier chosen • OCF / DDN• GPFS and Object Store• Over three locations

12 Aug 2016: Contract signed with OCF/DDN

7 Dec 2016: Service delivered to users (Tier 1)


RDS Service – Location

Slough

ChelseaSutton

Geographically distributed system to increase level of protection for ICR data

Service located in Sutton and two different data halls in Slough

10Gbps private network between sites

Each site also has a 10Gbps JANET connection to the internet

Network traffic can route around the failure of any inter-site link


RDS Service Network


RDS Service – High Level Architecture


RDS Service – Conceptual Design


RDS Service - PrinciplesTiered and disk-only system:

Tier 1: mirrored, fast storage for instant access to files by large numbers of clients

Tier 2: less performant, but more scalable object storage for large volumes of data; less frequently used data will mostly be on Tier 2

Able to offer direct access to both tiers from desktops, servers and HPC

To the end-user it looks like one big “bucket” of storage – complexity is hidden


RDS Tier 1 Characteristics• Located at Jisc Shared Data Centre (Slough) and Sutton• Two Gridscaler 7ks + GPFS Tiebreaker

• Allows failover to surviving site without disruption• Stretched (Mirrored) GPFS parallel filesystem with single namespace• SSD cache• 8TB HDDs• MEDIAScaler provides clustered NFS and SMB• 4 NAS* gateway servers serving as GPFS nodes (2 Sutton, 2 Slough)• 2PiB usable capacity (really 4PiB)• No SPFs

* Growing to 8


RDS WOS Bridge Characteristics

• Based on HSM rules• In conjunction with user, over ridable rules • Selected files and directories will be migrated to the long-term storage

component• Bridge rules for each research team (in principle)

• WOS Bridge uses synchronization service to exchange name space and data changes between Tier 1 and Tier 2 objects that have been migrated to the long-term storage

• Migrated file data is protected and replicated according to the target WOS policy of the WOS Bridge rules


RDS Tier 2 Characteristics• Located at Jisc Shared Data Centre (Slough) in two data halls, and Sutton• Based on DDN Object Storage system• 3-site WOS cluster based on 12 x WOS9660 storage appliances

• 4 per site• 12 x WOS9660 appliances communicate over standard TCP/IP and expose a

single global namespace• Global ObjectAssure used (erasure coding across 3 sites)

• Hierarchical erasure-coding scheme encoding object• Expansion factor – 1.875 times• Designed for cost-effective resiliency• Allows a site offline + additional 2 disk failures in the remaining appliances

without impacting data availability

• 4.3PiB usable capacity (really 8PiB)


RDS Service – WOS Bridge Example

Example data flow:

1. Copy to Sutton T12. Mirror to Slough T13. File not used for a Month4. Create on object5. Stub files

File

Object

File FileStubStub


Current Use of Tier 1


5. Overview of RDS Service


RDS Service StatusRDS• Tier 1 - live 7 December 2016• Tier 2 - operating from March 2017• Bridge in process of being switched on, team by team• Daily snapshots being taken• Service has run without interruption

Decision to use Jisc Shared Data Centre• No remaining room nor power in London• Needed geographical separation• Used data halls 1 and 3 (for Global ObjectAssure)• Reliable, resilient, well managed data centre

Intention to grow RDS Service, add other elements of RDS programme


Thank Youand

[email protected]


Cancer Research Centre of ExcellenceAn ICR and Imperial partnership

12

• The ICR and Imperial College London have developed a strategic partnership through the creation of a virtual Cancer Research Centre of Excellence.

• The partnership will harness the complementary expertise of both partners to deliver an enhanced and synergistically world-leading programme of cancer research.

• Pooling resources and expertise, and working together in strategic partnership, will enhance the ICR’s basic and translational research.

ICR scientists will gain access to the extensive resources and expertise of a multi-faculty university in new areas, including:

• basic life sciences• medicine• physical sciences• bio-engineering

Thank you