2

Frdric Hemmer CERN IT Department Head 21 March 2013

21 March 2013 Large Scale Scientific Data - Frdric Hemmer

7000 tons, 150 million sensors generating data 40 millions times per second

i.e. a petabyte/s

The ATLAS experiment

The LHC Computing Grid - Frdric Hemmer 4 March 2013

!"

A collision at LHC

The LHC Computing Grid - Frdric Hemmer 5

Ian.Bird@cern.ch #"

The Data Acquisition

The LHC Computing Grid - Frdric Hemmer 6

$%&'"(")*"+,-./"0123%4%5678"9%'4*":)44"'&1674*'315678"";*6')%4*'%?3567"

$@&"AB+"+6C:357

CERN IT Department CH-1211 Genve 23

Switzerland www.cern.ch/it

$@&"AB+">)*)"+@)II&7

0"E%4*'%?3*&E"16C:357)*)"F"9'GEG'%1"B&CC&'"

!"#$%&%'()*%)+,%'(-.%

Tier-0 (CERN): Data recording Initial data reconstruction Data distribution

Tier-1 (11 centres): Permanent storage Re-processing Analysis

Tier-2 (~130 centres): Simulation End-user analysis

^"

-&I%&4"67"" Z_.8"D,0.$8"`;FAB+.&*" .-,.4"="6*@&'"7)567)I"="

%7*&'7)567)I":'6T%E&'4"

"/#%01*'2345+6%

a("A)'

#7809%:)*)%'35**1+%;)*)"')*&4"%71'&)4&E""PFb"Dcd4"%7:3*"Wa!"Dcd4"63*:3*"

40179 40210 40238 40269 40299 40330 40360 40391 40422 40452 40483 40513 40544 40575 40603 40634 40664 40695 40725 40756 40787 40817 40848 40878 40909 40940 40969 41000 41030 41061 41091 41122 41153 41183 41214 41244

D,W^D

>/

d>^

D^

>,

Jun-6-2008 Jun-13-2008 Jun-20-2008 Jun-27-2008 Jul-4-2008 Jul-11-2008 Jul-18-2008 Jul-25-2008 Aug-1-2008 Aug-8-2008 Aug-15-2008 Aug-22-2008 Aug-29-2008 Sep-5-2008 Sep-12-2008 Sep-19-2008 Sep-26-2008 Oct-3-2008 Oct-10-2008 Oct-17-2008 Oct-24-2008 Oct-31-2008 Nov-7-2008 Nov-14-2008 Nov-21-2008 Nov-28-2008 Dec-5-2008 Dec-12-2008 Dec-19-2008 Dec-26-2008 Jan-2-2009 Jan-9-2009 Jan-16-2009 Jan-23-2009 Jan-30-2009 Feb-6-2009 Feb-13-2009 Feb-20-2009 Feb-27-2009 Mar-6-2009 Mar-13-2009 Mar-20-2009 Mar-27-2009 Apr-3-2009 Apr-10-2009 Apr-17-2009 Apr-24-2009 May-1-2009 May-8-2009 May-15-2009 May-22-2009 May-29-2009 Jun-5-2009 Jun-12-2009 Jun-19-2009 Jun-26-2009 Jul-3-2009 Jul-10-2009 Jul-17-2009 Jul-24-2009 Jul-31-2009 Aug-7-2009 Aug-14-2009 Aug-21-2009 Aug-28-2009 Sep-4-2009 Sep-11-2009 Sep-18-2009 Sep-25-2009 Oct-2-2009 Oct-9-2009 Oct-16-2009 Oct-23-2009 Oct-30-2009 Nov-6-2009 Nov-13-2009 Nov-20-2009 Nov-27-2009 Dec-4-2009 Dec-11-2009 Dec-18-2009 Dec-25-2009 Jan-1-2010 Jan-8-2010 Jan-15-2010 Jan-22-2010 Jan-29-2010 Feb-5-2010 Feb-12-2010 Feb-19-2010 Feb-26-2010 Mar-5-2010 Mar-12-2010 Mar-19-2010 Mar-26-2010 Apr-2-2010 Apr-9-2010 Apr-16-2010 Apr-23-2010 Apr-30-2010 May-7-2010 May-14-2010 May-21-2010 May-28-2010 Jun-4-2010 Jun-11-2010 Jun-18-2010 Jun-25-2010 Jul-2-2010 Jul-9-2010 Jul-16-2010 Jul-23-2010 Jul-30-2010 Aug-6-2010 Aug-13-2010 Aug-20-2010 Aug-27-2010 Sep-3-2010 Sep-10-2010 Sep-17-2010 Sep-24-2010 Oct-1-2010 Oct-8-2010 Oct-15-2010 Oct-22-2010 Oct-29-2010 Nov-5-2010 Nov-12-2010 Nov-19-2010 Nov-26-2010 Dec-3-2010 Dec-10-2010 Dec-17-2010 Dec-24-2010 Dec-31-2010 Jan-7-2011 Jan-14-2011 Jan-21-2011 Jan-28-2011 Feb-4-2011 Feb-11-2011 Feb-18-2011 Feb-25-2011 Mar-4-2011 Mar-11-2011 Mar-18-2011 Mar-25-2011 Apr-1-2011 Apr-8-2011 Apr-15-2011 Apr-22-2011 Apr-29-2011 May-6-2011 May-13-2011 May-20-2011 May-27-2011 Jun-3-2011 Jun-10-2011 Jun-17-2011 Jun-24-2011 Jul-1-2011 Jul-8-2011 Jul-15-2011 Jul-22-2011 Jul-29-2011 Aug-5-2011 Aug-12-2011 Aug-19-2011 Aug-26-2011 Sep-2-2011 Sep-9-2011 Sep-16-2011 Sep-23-2011 Sep-30-2011 Oct-7-2011 Oct-14-2011 Oct-21-2011 Oct-28-2011 Nov-4-2011 Nov-11-2011 Nov-18-2011 Nov-25-2011 Dec-2-2011 Dec-9-2011 Dec-16-2011 Dec-23-2011 Dec-30-2011 Jan-6-2012 Jan-13-2012 Jan-20-2012 Jan-27-2012 Feb-3-2012 Feb-10-2012 Feb-17-2012 Feb-24-2012 Mar-2-2012 Mar-9-2012 Mar-16-2012 Mar-23-2012 Mar-30-2012 Apr-6-2012 Apr-13-2012 Apr-20-2012 Apr-27-2012 May-4-2012 May-11-2012 May-18-2012 May-25-2012 Jun-1-2012 Jun-8-2012 Jun-15-2012 Jun-22-2012 Jun-29-2012 Jul-6-2012 Jul-13-2012 Jul-20-2012 Jul-27-2012 Aug-3-2012 Aug-10-2012 Aug-17-2012 Aug-24-2012 Aug-31-2012 Sep-7-2012 Sep-14-2012 Sep-21-2012 Sep-28-2012 Oct-5-2012 Oct-12-2012 Oct-19-2012 Oct-26-2012 Nov-2-2012 Nov-9-2012 Nov-16-2012 Nov-23-2012 Nov-30-2012 Dec-7-2012 Dec-14-2012 Dec-21-2012 Dec-28-2012 Jan-4-2012

40909 40940 40969 41000 41030 41061 41091 41122 41153 41183 41214 41244

>/D^d>^D^KDW^^>,EEEdK&h^Z

1RVWRSIRUWKHFRPSXWLQJ

$FWLYLW\RQ-DQXDU\5XQQLQJ-REV7UDQVIHUUDWHa*L%V

&(517DSH$UFKLYH5HDFKLQJ3%

*ULG&38:RUNORDGV+(36SHF0RQWKa&38FRUHVLQXVHDWDQ\SRLQWLQWLPH

'DWDWREHUHSURFHVVHG 6LPXODWLRQIRU+LJKHU(QHUJ\ $QDO\VLVFRQWLQXLQJ

WREHUHSDFNHGRQQHZPHGLD

Connectivity (100 Gbps)

21 March 2013 Large Scale Scientific Data - Frdric Hemmer 13

CERN IT Department CH-1211 Genve 23

Switzerland www.cern.ch/it

$@&"+,-.">)*)"+&7*'&"%7".3C?&'4"

>)*)"+&7*'&"Z:&')5674"O$%&'"(V" LbNH"6:&')*6'"43::6'*")7E";M4*&C"0EC%7%4*')567"4&'T%1&4"*6"43::6'*"LbNH"

6:&')567"6K")II"e$"4&'T%1&4f" B)'EJ)'&"%74*)II)567"="'&5'&C&7*"

WH8((("@)'EJ)'&"C6T&C&7*4dM&)'g"WaS(("E%4["K)%I3'&4dM&)'" Y)7)%0[0" VC4E"3"

T+?!6%0[!!;%\/0! 2V3!

B+:/!9+;.;%6*/0! F#333!

B+:/!0&'.0! #V333!

>+.+!'(!B+:/!U8%DX! V#!

^B!8'?/;!9'(0KH:_'(! "F#V!`Q!

B'.+&!8'?/;!9'(0KH:_'(! EZR3!`Q!

A)'

Scaling CERN Data Center(s) to anticipated Physics needs

21 March 2013 Large Scale Scientific Data - Frdric Hemmer 15

CERN Data Center dates back to the 70s Upgraded in 2005 to support LHC (2.9 MW) Still optimizing the current facility (cooling automation,

temperatures, infrastructure)

Exploitation of 100 KW of remote facility down town Understanding costs, remote dynamic management,

ensure business continuity

Exploitation of a remote Data center in Hungary Max. 2.7 MW (N+1 redundancy)

- Business continuity 100 Gbps connections

Consolidation for accommodating 450 KW of critical IT loads (increasing total IT Load to 3.5 MW)

Key Messages Distributing Petabytes of scientific Data 24x7 is possible

- Networks have be by far more reliable than anticipated - and affordable

Important to build resilient systems - Things are going to fail - Took 10+ years to build the WLCG

Worldwide Federated Identities are key - Scientists move, change organizations, but still want to access their

data Data Preservation is still an issue

- Not only at the bit level even that is difficult - Requires permanent efforts, even after funding for experiments have

stopped - Now becoming a requirement of funding agencies - Our duty for further generations

21 March 2013 Large Scale Scientific Data - Frdric Hemmer 16

21 March 2013 Large Scale Scientific Data - Frdric

Hemmer 17