the lch grid - high performance computing in high energy particle physics
DESCRIPTION
Luke Kreczko of CERN, presented at Bristol ITMegaMeet 2013 CERN hosts many experiments and accelerators. One of them, the Large Hadron Collider (LHC), is the world's largest particle accelerator. Its experiments produce enormous amounts of data which is analysed using the LHC Grid. I will describe the Grid framework developed for the CMS experiment that produces around 20PB of data every year and processes it in regular intervals.TRANSCRIPT
![Page 1: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/1.jpg)
The LHC Grid High Performance Computing in
High Energy Particle Physics
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 1
![Page 2: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/2.jpg)
$ whoami
• Lukasz Kreczko – ParGcle Physicist • Graduated in Physics from University of Hamburg in 2009
• 2009 – 2013 PhD in ParGcle Physics at the University of Bristol
• Currently CompuGng Research Assistant at the University of Bristol
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 2
![Page 3: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/3.jpg)
The experiment: a big digital camera
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 3
![Page 4: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/4.jpg)
The experiment: a big digital camera
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 4
40 million “pictures” per second
Each “picture” around 1 MB!
![Page 5: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/5.jpg)
The data: a structured mess
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 5
![Page 6: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/6.jpg)
The data: a much nicer picture
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 6
Muon:
pT=71.5GeV/c
η=‐0.82
MissingET:
22.3GeV
Jet:
pT=89.0GeV/c
η=2.14
Jet:
pT=85.3GeV/c
η=2.02
Jet:
pT=90.5GeV/c
η=‐1.40
Run:163583
Event: 26579562
Jet:
pT=84.1GeV/c
η=‐2.24
m(F)=1.2TeV/c2_
![Page 7: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/7.jpg)
The goal: extend our knowledge
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 7
Muon:
pT=71.5GeV/c
η=‐0.82
MissingET:
22.3GeV
Jet:
pT=89.0GeV/c
η=2.14
Jet:
pT=85.3GeV/c
η=2.02
Jet:
pT=90.5GeV/c
η=‐1.40
Run:163583
Event: 26579562
Jet:
pT=84.1GeV/c
η=‐2.24
m(F)=1.2TeV/c2_
Billions of + simulaGon
(GeV)!!m110 120 130 140 150S
/(S+B
) Wei
ghte
d E
vent
s / 1
.5 G
eV
0
500
1000
1500
DataS+B FitB Fit Component
"1±"2±
-1 = 8 TeV, L = 5.3 fbs-1 = 7 TeV, L = 5.1 fbsCMS
(GeV)!!m120 130
Eve
nts
/ 1.5
GeV
1000
1500Unweighted
![Page 8: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/8.jpg)
The goal: extend our knowledge
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 8
Muon:
pT=71.5GeV/c
η=‐0.82
MissingET:
22.3GeV
Jet:
pT=89.0GeV/c
η=2.14
Jet:
pT=85.3GeV/c
η=2.02
Jet:
pT=90.5GeV/c
η=‐1.40
Run:163583
Event: 26579562
Jet:
pT=84.1GeV/c
η=‐2.24
m(F)=1.2TeV/c2_
Billions of + simulaGon
(GeV)!!m110 120 130 140 150S
/(S+B
) Wei
ghte
d E
vent
s / 1
.5 G
eV
0
500
1000
1500
DataS+B FitB Fit Component
"1±"2±
-1 = 8 TeV, L = 5.3 fbs-1 = 7 TeV, L = 5.1 fbsCMS
(GeV)!!m120 130
Eve
nts
/ 1.5
GeV
1000
1500Unweighted
That’s the famous Higgs boson
![Page 9: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/9.jpg)
Analysing all data
• CMS records 10 000 Terabytes of data every year (around 70 years of full HD movies)
• + same amount of simulaGon
• To analyse this on a single computer would
take 64,000 years!
9
![Page 10: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/10.jpg)
Analysing all data
• CMS records 10 000 Terabytes of data every year (around 70 years of full HD movies)
• + same amount of simulaGon
• To analyse this on a single computer would
take 64,000 years!
10
![Page 11: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/11.jpg)
The compuGng model: the decision
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 11
![Page 12: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/12.jpg)
The compuGng model
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 12
![Page 13: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/13.jpg)
The compuGng model
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 13
Bristol is one of the T2 centres
![Page 14: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/14.jpg)
The computer centres
![Page 15: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/15.jpg)
All over the world
![Page 16: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/16.jpg)
All over the world
On a normal day, the grid provides 100,000 CPU days execuGng 1 million jobs
![Page 17: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/17.jpg)
PhEDEx: A Bristol invenGon
• short for “Physics Experiment Data Export”
• Sodware for data placement and the file transfer system
• Development started in Bristol
• One of the main developers (now at Cloudant) is giving a talk later today “Your Database to the Cloud, an Intro to Cloudant NoSQL”
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 17
![Page 18: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/18.jpg)
PhEDEx: The components
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 18
Transfer agent
Transfer management database
Management agent
Tools to manage transfers
Local agents
Web monitoring
![Page 19: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/19.jpg)
PhEDEx: data monitoring
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 19
150 PB
70 PB
hjp://www.flickr.com/photos/paulrossman/6555010435/sizes/l/in/photostream/
![Page 20: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/20.jpg)
Current and future topics
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 20
Things changed: Network is genng cheaper as well
![Page 21: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/21.jpg)
Current and future topics
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 21
Things changed: Network is genng cheaper as well
We are moving into the cloud
![Page 22: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/22.jpg)
Current and future topics
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 22
Things changed: Network is genng cheaper as well
We are moving into the cloud
Any data anywhere!
![Page 23: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/23.jpg)
Other current and future topics
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 23
Protocol buffers CouchDB
Hadoop Provisioning of compuGng
![Page 24: The LCH Grid - High Performance Computing in High Energy Particle Physics](https://reader034.vdocuments.us/reader034/viewer/2022052505/555a2e80d8b42ab3088b502d/html5/thumbnails/24.jpg)
Summary & Outlook
• The LHC grid is a world-‐wide distributed compuGng network that makes discoveries possible
• The grid is changing to adopt to current technologies
• In two years Gme the LHC will increase its energy to 14 TeV – more data, more discoveries to come!
Saturday, 1 June 13 Lukasz Kreczko -‐ Bristol IT MegaMeet 24