cern clouds and culture at gigaom london 2013

30

Upload: tim-bell

Post on 18-Dec-2014

336 views

Category:

Spiritual


1 download

DESCRIPTION

 

TRANSCRIPT

Page 1: CERN clouds and culture at GigaOm London 2013
Page 2: CERN clouds and culture at GigaOm London 2013

Tim [email protected]

@noggin143

19/09/2013 CERN Infrastructure Evolution 2

Page 3: CERN clouds and culture at GigaOm London 2013

19/09/2013 CERN Infrastructure Evolution 3

CERN was founded 1954: CERN was founded 1954: 12 European States12 European States “ “Science for Peace”Science for Peace”

Today: 20 Member StatesToday: 20 Member States

Member States:Member States: Austria, Belgium, Bulgaria, the Czech Republic, Denmark, Austria, Belgium, Bulgaria, the Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Italy, the Netherlands, Norway, Finland, France, Germany, Greece, Hungary, Italy, the Netherlands, Norway, Poland, Portugal, Slovakia, Spain, Sweden, Switzerland and Poland, Portugal, Slovakia, Spain, Sweden, Switzerland and the United Kingdom the United Kingdom

Candidate for Accession: Candidate for Accession: RomaniaRomania

Associate Members in Pre-Stage to Membership: Associate Members in Pre-Stage to Membership: Israel, SerbiaIsrael, Serbia

Applicant States for Membership or Associate Membership:Applicant States for Membership or Associate Membership:Brazil, Cyprus (awaiting ratification), Pakistan, Russia, Slovenia, Turkey, Ukraine Brazil, Cyprus (awaiting ratification), Pakistan, Russia, Slovenia, Turkey, Ukraine

Observers to Council:Observers to Council: India, Japan, Russia, Turkey, United States of America; India, Japan, Russia, Turkey, United States of America; European Commission and UNESCO European Commission and UNESCO

Member States:Member States: Austria, Belgium, Bulgaria, the Czech Republic, Denmark, Austria, Belgium, Bulgaria, the Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Italy, the Netherlands, Norway, Finland, France, Germany, Greece, Hungary, Italy, the Netherlands, Norway, Poland, Portugal, Slovakia, Spain, Sweden, Switzerland and Poland, Portugal, Slovakia, Spain, Sweden, Switzerland and the United Kingdom the United Kingdom

Candidate for Accession: Candidate for Accession: RomaniaRomania

Associate Members in Pre-Stage to Membership: Associate Members in Pre-Stage to Membership: Israel, SerbiaIsrael, Serbia

Applicant States for Membership or Associate Membership:Applicant States for Membership or Associate Membership:Brazil, Cyprus (awaiting ratification), Pakistan, Russia, Slovenia, Turkey, Ukraine Brazil, Cyprus (awaiting ratification), Pakistan, Russia, Slovenia, Turkey, Ukraine

Observers to Council:Observers to Council: India, Japan, Russia, Turkey, United States of America; India, Japan, Russia, Turkey, United States of America; European Commission and UNESCO European Commission and UNESCO

~ 2,300 staff~ 2,300 staff ~ 1,000 other paid personnel~ 1,000 other paid personnel > 11,000 users> 11,000 users

Budget (2013) ~1,000 MCHFBudget (2013) ~1,000 MCHF

~ 2,300 staff~ 2,300 staff ~ 1,000 other paid personnel~ 1,000 other paid personnel > 11,000 users> 11,000 users

Budget (2013) ~1,000 MCHFBudget (2013) ~1,000 MCHF

Page 4: CERN clouds and culture at GigaOm London 2013

What are the Origins of Mass ?

19/09/2013 CERN Infrastructure Evolution 4

Page 5: CERN clouds and culture at GigaOm London 2013

Matter/Anti Matter Symmetric?

19/09/2013 CERN Infrastructure Evolution 5

Page 6: CERN clouds and culture at GigaOm London 2013

Where is 95% of the Universe?

19/09/2013 CERN Infrastructure Evolution 6

Page 7: CERN clouds and culture at GigaOm London 2013

19/09/2013 CERN Infrastructure Evolution 7

Page 8: CERN clouds and culture at GigaOm London 2013

19/09/2013 CERN Infrastructure Evolution 8

Page 9: CERN clouds and culture at GigaOm London 2013

19/09/2013 CERN Infrastructure Evolution 9

Page 10: CERN clouds and culture at GigaOm London 2013

Collisions

19/09/2013 CERN Infrastructure Evolution 10

Page 11: CERN clouds and culture at GigaOm London 2013

A Big Data Challenge

19/09/2013 CERN Infrastructure Evolution 11

Page 12: CERN clouds and culture at GigaOm London 2013

The CERN Meyrin Data Centre

19/09/2013 CERN Infrastructure Evolution 12

Page 13: CERN clouds and culture at GigaOm London 2013

New Data Centre in Budapest

19/09/2013 CERN Infrastructure Evolution 13

Page 14: CERN clouds and culture at GigaOm London 2013

Good News, Bad News

19/09/2013 CERN Infrastructure Evolution 14

Page 15: CERN clouds and culture at GigaOm London 2013

We’re Not Special

19/09/2013 CERN Infrastructure Evolution 15

Page 16: CERN clouds and culture at GigaOm London 2013

19/09/2013

Bamboo

Koji, Mock

AIMS/PXEForeman

Yum repoPulp

Puppet-DB

mcollective, yum

JIRA

Lemon /Hadoop /

LogStash /Kibana

OpenStack Nova

Hardware database

Puppet

Active Directory /LDAP

CERN Infrastructure Evolution 16

Page 17: CERN clouds and culture at GigaOm London 2013

The Agile Experience

19/09/2013 CERN Infrastructure Evolution 17

Page 18: CERN clouds and culture at GigaOm London 2013

Cultural Barriers

19/09/2013 CERN Infrastructure Evolution 18

Page 19: CERN clouds and culture at GigaOm London 2013

Status• Toolchain implemented in 18 months with

enhancements and bug fixes submitted back to the community

• Now in production in 3 OpenStack clouds (over 50,000 cores in total) in Geneva and Budapest managed by Puppet

• Target is more than 300,000 cores by 2015 and 90% compute resources in the private cloud

19/09/2013 CERN Infrastructure Evolution 19

Page 20: CERN clouds and culture at GigaOm London 2013

Summary• Constraints on resources have led to major

technology transformations at CERN• Open source community participation helps

drive cultural change and motivates staff• CERN benefits and contributes back through

code and outreach

19/09/2013 CERN Infrastructure Evolution 20

Page 21: CERN clouds and culture at GigaOm London 2013

Questions ?

19/09/2013 CERN Infrastructure Evolution 21

Page 22: CERN clouds and culture at GigaOm London 2013

19/09/2013 CERN Infrastructure Evolution 22

Page 23: CERN clouds and culture at GigaOm London 2013

Service Models

19/09/2013 CERN Infrastructure Evolution 23

• Pets are given names like pussinboots.cern.ch • They are unique, lovingly hand raised and cared for• When they get ill, you nurse them back to health

• Cattle are given numbers like vm0042.cern.ch• They are almost identical to other cattle• When they get ill, you get another one

Page 24: CERN clouds and culture at GigaOm London 2013

19/09/2013 CERN Infrastructure Evolution 24

Page 25: CERN clouds and culture at GigaOm London 2013

19/09/2013 CERN Infrastructure Evolution 25

http://www.eucalyptus.com/blog/2013/04/02/cy13-q1-community-analysis-%E2%80%94-openstack-vs-opennebula-vs-eucalyptus-vs-cloudstack

Page 26: CERN clouds and culture at GigaOm London 2013

19/09/2013 CERN Infrastructure Evolution 26

Page 27: CERN clouds and culture at GigaOm London 2013

19/09/2013 CERN Infrastructure Evolution 27

Tier-1 (11 centres):•Permanent storage•Re-processing•Analysis

Tier-0 (CERN):•Data recording• Initial data reconstruction•Data distribution

Tier-2 (~200 centres):• Simulation• End-user analysis

• Data is recorded at CERN and Tier-1s and analysed in the Worldwide LHC Computing Grid

• In a normal day, the grid provides 100,000 CPU days executing over 2 million jobs

Page 28: CERN clouds and culture at GigaOm London 2013

19/09/2013 CERN Infrastructure Evolution 28

Page 29: CERN clouds and culture at GigaOm London 2013

19/09/2013 CERN Infrastructure Evolution 29

Microsoft Active Directory

CERN DB on Demand

CERN Network Database

Account mgmt system

Horizon

Keystone

NetworkCompute

Glance

Scheduler

Cinder

Nova

Block Storage Provider

Page 30: CERN clouds and culture at GigaOm London 2013

Training for Newcomers

19/09/2013 CERN Infrastructure Evolution 30

Buy the book rather than guru mentoring