hosting clarin services in the egi cloud · pdf fileubercloud 19/09/17 7 the egi service...

www.egi.eu EGI-Engage is co-funded by the Horizon 2020 Framework Programme

of the European Union under grant number 654142

support@egi.eu http://go.egi.eu/cloud

Hosting CLARIN services in the EGI cloud

Gergely Sipos EGI Foundation Boris Parak

CESNET Willem Elbers CLARING ERIC

2 19/09/17

EGI: Advanced Computing for Research

3 19/09/17

EGI Membership •  Majornationale-Infrastructures:22NGIs•  EIRO:CERN(EuropeanIntergovernmentalResearchOrganisation)•  Membershipfeessustainthefederationandcommunity

–  EGIFoundation– Collaboration&federation

4 19/09/17

EGI Federation Largest distributed compute e-Infrastructure of the world

300+HTCproviders23Cloudproviders

650kCPUcores

285PBonlinestorage

1.7Millionjobs/day

2.6BillionCPUhours/

year

240VirtualOrganisaKons

>48000users

EGI Foundation

(Amsterdam)

5 19/09/17

International Partnerships

AfricaandArabiaCouncilforScienKficandIndustrialResearch,SouthAfrica

IndiaCentreforDevelopmentofAdvancedComp.

ChinaInst.OfHEPChineseAcademyofSciences

La/nAmericaUniversidadeFederaldoRiodeJaneiro

UkraineUkrainianNaKonalGrid

USA

Canada

AsiaPacificRegionAcademiaSinicaatTaiwan

6 19/09/17

EGI serves researchers and innovators

ESFRIs,FETflagships

Sizeofindividualgroups

Mul;na;onalcommuni;es,(e.g.H2020projects)

‘Longtailofscience’

WLCGELICTAELIXIREPOSBBMRIINSTRUCTCLARINEISCAT_3DDARIAHLOFARLifeWatchICOSEMSOCORBELENVRIplus…

VREprojectsOpenDreamKitWeNMRDRIHMVERCEMuGAgINFRACMMSTLSGCSuperSitesExploitaKonEnvironmentalsci.neuGRID…

PeachNoteCEBAGalaxyeLabSemiconductordesignMain-beltcometsQuantumpysicsstudiesVirtualimaging(LS)BovinetuberculosisspreadConvergentevol.ingenomesGeographyevoluKonSeafloorseismicwaves3DlivermapswithMRIMetabolicratemodellingGenomealignmentTapewormsinfecKononfish…

Industry,SMEs

AgroknowCloudEOCloudSMEEcohydrosgnubilaSinergiseSixSqTEISSTerradueUbercloud…

7 19/09/17

The EGI Service Catalogue www.egi.eu/services

8 19/09/17

Introduction to EGI cloud

9 19/09/17

EGI Cloud •  Gridofclouds•  Uniformeduserinterfaces•  Harmonisedoperational

behaviour•  Basedonopenstandardsand

opentechnologies•  PrimarilyanIaaScloud,but

withcommunity-hostedPaaSandSaaSontop

•  InfrastructureàAccessANDTechnologystackàDeploy

10 19/09/17

The current infrastructure Today:•  23providersfrom14NGIs

•  15OpenStack•  7OpenNebula•  1Synnefo

11 19/09/17

Typical usage models

•  Computeanddataintensiveworkloads–  Batchandinteractive(e.g.Jupyternotebooks)withscalableand

customizedenvironments

•  ServiceHosting–  Long-runningservices(e.g.webserver,database,applicationserver)

•  Datasetsrepository–  Storeandmanagelargedatasets(inastoragevolume)

•  Disposableandtestingenvironments–  Hosttrainingenvironments,testapplications

12 19/09/17

Computing models

HTCGridservice(e.g.clusters;HTC)

Commercialcloudservice(e.g.Azure)

EGIcloudservice(federatedIaaScloud)

•  ForbatchcompuKng•  Forembarrassingly

parallelapplicaKons•  UsingmulKplesitesat

aKme

•  ForinteracKveandbatchcompuKng

•  ForservicehosKng•  FlexibleOSandapplicaKon

use•  Typicallyonesiteisused•  Pay-as-you-go•  Nousersupport(orcentral

paidservice)

•  ForinteracKveandbatchcompuKng

•  ForservicehosKng•  FlexibleOSandapplicaKon

use•  UsesingleormulKplesites

ataKme•  VirtualOrganisaKonbased

access•  Freeatthepointofuse(for

research)•  Withlocalusersupport

13 19/09/17

Main resource about EGI Cloud

EGI Federated Cloud User Guides: •  https://wiki.egi.eu/wiki/Federated_Cloud_user_support

14 19/09/17

CLARIN services in the EGI Cloud

15 19/09/17

Brokering resources for a community

Project/Communityrepresen;ngtheVO

Nego;ator Gridprovider

Cloudprovider

OperaKonLevel

Agreement

ServiceLevelAgreement

SaKsfacKonreview(every3/6/12months)

Storageprovider

Servicerequirements Condi;ons

Applic.provider

Performancereports

SupportTraining

Type,number,size,cost,availability,etc.

TriggertheprocesswithaservicerequestontheEGIwebsite

16 19/09/17

CLARIN cloud provider:

•  National grid & cloud infrastructure including storage

•  Multiple production IaaS clouds operated by CESNET

–  MetaCloud (OpenNebula-based) •  2,900 CPU, ≈ 30 TB RAM1

–  EGI Federated Cloud (also OpenNebula-based) •  ≈300 CPU + 1,600GB RAM

–  Full-fledged & certified FedCloud site –  Numerous testbed deployments (also OpenNebula-based) –  ownCloud personal cloud-based storage

17 19/09/17

CLARIN services in the CESNET cloud

1.  CLARIN component registry (http://clarin.eu/cmdi), the backbone of the Component-based MetaData Infrastructure.

2.  The CLARIN docker registry (http://docker.clarin.eu), which plays an important role in the CLARIN deployment workflow.

3.  The CLARIN maven repository (http://nexus.clarin.eu), intensively used during CLARIN software development projects.

4.  A grafana instance (http://metrics.clarin.eu), to visualize server status and metrics.

5.  Document archive for the CLARIN ERIC central office.

18 19/09/17

Future outlook – The European perspective European Open Science Cloud

“The European Open Science Cloud (EOSC) is a vision for a federated, globally accessible,

multidisciplinary environment where researchers, innovators, companies and citizens can publish,

find, use and reuse each other's data, tools, publications and other outputs for research,

innovation and educational purposes.”

Credits: Open Science Policy Platform EOSC wg

à EOSCpilotH2020project:2017.Jan1–2018Dec31.à EOSC-hubH2020project:2018.Jan1–2020Dec31.

19 19/09/17

Next Community Event Digital Infrastructures for Research (DI4R)

hlps://www.digitalinfrastructures.euCo-locatedwithEOSCpilot1stStakeholderEngagementEvent(28-29Nov)

CALLFORCONTRIBUTIONISNOWOPEN!

20 19/09/17

Summary

•  EGI Foundation: Cloud service broker •  CESNET: Cloud service provider •  CLARIN ERIC: Cloud customer •  CLARIN community: Cloud user