![Page 1: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/1.jpg)
LHC-CMS Tier2 facility at TIFR
Kajari Mazumdar Department of High Energy Physics
Tata Institute of Fundamental Research Mumbai, India.
http://indiacms.res.inT2-IN-TIFR
NKN Workshop, IIT, Bombay November 1, 2012
![Page 2: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/2.jpg)
e-Science and e-Research
Large collaborative research is made possible by sharing resources across the
globe via internet (data, computation, people’s expertise...).
– Crosses organisational , national and international boundaries.
CERN-LHC project is an excellent example of large scale colalboration dealing
with huge amount of data.
• WWW was born in CERN to satisfy the needs of previous generation HEP
experiments.
• LHC has been a driving force for GRID computing technology which evolved
naturally from internet .
High Energy Physics(HEP) is often compute as well as data intensive.
•Experiments performed by several thousand scientists over several years.
![Page 3: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/3.jpg)
The GRID Computing Goal • Science without borders. • Provide Resources and Services to store/serve O(10) PB data/year
• Provide access to all interesting physics events to O(4000) collaborators
• Minimize constraints due to user localisation and resource variety • Decentralize control and costs of computing infrastructure Solution through LHC Computing GRID Much faster delivery of physics
![Page 4: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/4.jpg)
To begin at the end:
• The operations of the LHC machine and the experiments have been a great success.
• Fantastic output, often only days after the data is taken about 200 scientific publications per experiment in 3 years.
LHC Computing Grid is the backbone of the success story
Amount of data written by LHC experiments during last 6 months: 19 Petabyte. Expected volume of data by end of 1st phase of LHC operation: 30 PB. Total transfer rate across globe: ~ 10 Gbps
• Today >140 sites • ~250k CPU cores • ~100 PB disk
Internal speed of computers have become comparable to the speed of large distance network.
![Page 5: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/5.jpg)
Large Hadron Collider (LHC)
• 27 km circumference • at 1.9 K • at 10-13 Torr • at 50-175 m below surface • > 10K magnets 4 big experiments: >10K scientists, students, engineers. Operational since 2009, Q4.
Largest ever scientific project 20 years to plan, build 20 years to work with
![Page 6: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/6.jpg)
Big Bang
Today
~300‘000 years
Experiments in Astrophysics & Cosmology
WMAP (2001) COBE(1989)
LHC: ~ 10-12 seconds (p-p) ~ 10-6 seconds (Pb-Pb)
![Page 7: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/7.jpg)
What happens in LHC experiment
Proton-Proton 1380 bunch/beam Protons/bunch 2. 1011
Beam energy 4 TeV / proton Luminosity 7.6*1033 /cm 2/s Crossing rate 20 MHz Total event rate 108 Hz
2012
2 major multipurpose experiments at LHC • A Toroidal LHC Apparatus (ATLAS) • Compact Muon Solenoid (CMS)
• 10 Million electronic channels per experiment . • Proton bunches collide every 50 nano sec. • Higgs production <1 per sec.
![Page 8: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/8.jpg)
Example of a modern detector 3170 scientists and engineers including ~800 students 169 institutes in 39 countries
India
![Page 9: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/9.jpg)
Data rates @ CMS as foreseen for design parameters
Today’s data collection rate ~ 850 Hz
![Page 10: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/10.jpg)
![Page 11: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/11.jpg)
Tier 0
Tier 1 National centres
Tier 2 Regional groups in a continent/nation
Different Universities, Institutes in a country
Individual scientist’s PC,laptop, ..
Experimental site
CERN computer centre, Geneva
ASIA (Taiwan)
India China Korea Taiwan
France Italy Germany USA
TIFR BARC Panjab
Univ.
Indiacms T2_IN_TIFR
Layered Structure of CMS GRID connecting computers across globe
Delhi Univ.
Pakistan
Online data recording
Several Petabytes/sec.
10 Gbps
1-2.5 Gbps
CERN
![Page 12: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/12.jpg)
CMS in Total: 1 Tier-0 at CERN 7 Tier-1s on 3 continents 50 Tier-2s on 4 continents
CMS T2 in India : one of the 5 in Asia-Pacific region Today : 7 collaborating institutes in CMS: ~ 30 scientists + 40 students 2.1% of signing authors in publication, Contributing to computing resource of CMS ~ 3%
![Page 13: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/13.jpg)
Distributed Analysis is not a wish, it is a necessity tools have to be reliable!
•First job of the offline system is to process and monitor data at T0. •T0: 1 M jobs/day + test-jobs. traffic : 4Gbps input, > 13 Gbps served •CERN Tier 0 moves ~ 1 PB data per day, automated subscription • T1 processes data further several times a year, coordinates with T2s • T2s are the real workhorses of the system with growing roles Hosts specific data streams for analysis Gets main data from T1s, recently more communications among T2s Typically 100k analysis jobs/day/experiment Site readiness is at high level 24X7 availability lot of invisible effort! Storage in T2 partitioned into central and local space.
Quick description of LHC grid tiers
![Page 14: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/14.jpg)
Middleware: - Storage Elements - Computing Elements - Workload Management - Local File Catalog - Information System - Virtual Organisation management - Inter-operability among different GRIDs
Resources - CPU : 3000 (7000) - Disk : 700 TB (950 TB) - Nework (WAN) : 1.5 - 2 Gbit/s
T2-IN-TIFR, Mumbai • Contributing to the computing efforts of CMS experiment credits earned against mandatory service jobs.
End-user does not have to bother about the resources
A large amount of invisible human effort is essential
![Page 15: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/15.jpg)
1.5/2 Gbps to CERN peered to GEANT
2.5 Gbps NKN +TEIN3
TIFR-INDIACMS T2
1 Gbps to VECC RRCAT, IPR
Network connections
VECC/SINP: INDIA
![Page 16: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/16.jpg)
Monitoring the performance of a CMS T2 site
• Availability and reliability : fraction of time all functional tests in a site are successful
• Data transfers number of active links with sustained rates • Job Robot (simulates small analysis jobs/user jobs) for failures, the logfile describes the nature. the investigations carried out locally + central experts.
![Page 17: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/17.jpg)
~1.5× 109 events produced in 2011
![Page 18: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/18.jpg)
Global status
![Page 19: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/19.jpg)
Data Transfers from/to TIFR
•Total data volume at present ~ 250 TB
•Total transfers during last 6 months ~ 70 TB
• TIFR hosting i) centrally managed data (simulated, custodial) ii) collision data skims
• Current CMS total CPU pledge at T2s : 18k jobs slots • Nominal Analysis pledge : 50% • Slot utilization during Summer/Fall 09 was reasonable but need to go into sustained analysis mode
upload
download
![Page 20: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/20.jpg)
August 15-18, 2011 Maximum: 1.5 Gbps Avg. : 1Gbps
![Page 21: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/21.jpg)
Low latency transfers
![Page 22: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/22.jpg)
Physics datasets for analysis
•Distribution of data to participating centres all over the world •Huge datasets (~ few TB) cannot be transferred to each user the analysis jobs go to “where the data is” Results come to user!
![Page 23: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/23.jpg)
The KEY of GRID Computing: NETWORK
Need a service for monitoring networks.
![Page 24: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/24.jpg)
• CMS Tier2 center at TIFR is in working condition and satisfies all CMS guidelines and requirements.
• Making what we have today more sustainable is a challenge
Conclusion
Sincere thanks to ERNET, EU-India grid and NKN for the support towards our effort in puttng India in CMS-Grid map.
Need to invest MORE in networks across the country and make full use of the distributed system for a remote participation to be satisfactory!
![Page 25: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/25.jpg)
Backup
![Page 26: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/26.jpg)
LHC is meant to resolve some of the most puzzling issues in physics: • Nature of elementary particles and interactions shortly after Big Bang how many interactions when the universe was much hotter which elementary particles existed with what properties? • we recreate conditions of very early universe at LHC. • Origin of the mass mass patterns among different particles in today’s universe why photon is massless, while carriers of weak interaction are massive? if the symmetry is broken spontaneously, what is the signature? existence of the God Particle? • The Higgs boson yet to be discovered, but coming soon, stay tuned! LHC is at the threshold of discovery likely to change the way we are used to think of Nature!
Motivation of LHC experiments
![Page 27: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/27.jpg)
• Nature of dark matter: we know only 4% of the constituent of the universe • A good 25% of the rest is massive enough to dictate the motion of galaxies non-luminous, and hence “dark” LHC can tell us the nature of this dark matter! LHC will also shed light on: • why there is only matter and no antimatter today. • properties of the 4th state of matter: Quark-Gluon-Plasma which existed 1 pico sec. after the big bang, before formation of neutrons and protons. ….
Grand menu from LHC
All this is possible because LHC is essentially a microscope AND a telescope as well!
observed
Expected from visible Distribution of matter
![Page 28: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/28.jpg)
•Versatile experiments, equipped with very specialized detectors. ~107 electronic channels per experiment, ready every 25 ns to collect information of debris from violent collisions. Reconstruct 20K charged tracks in a single event (lead-lead collisions at LHC)
Challenges
10 vertices in a single proton-proton collision, to be discriminated from the interesting process
• Event size related to flux/intensity • 1.5 Billion events recorded in 2010 • > 2B events, much more complicated, to be recorded during 2011 resource utilization to be prioritized by carefully throwing not-so-interesting collisions.
Charged tracks from heavy ion collision vertex
![Page 29: High Energy Physics and Networkingmazumdar/talks/IIT-NKN.pdfDepartment of High Energy Physics . Tata Institute of Fundamental Research . Mumbai, India. ... High Energy Physics(HEP)](https://reader036.vdocuments.us/reader036/viewer/2022071423/611da8e705c9270ca878491c/html5/thumbnails/29.jpg)
Enter a New Era in Fundamental Science
Exploration of a new energy frontier in p-p and Pb-Pb collisions
LHC ring: 27 km circumference
CMS
ALICE
LHCb
ATLAS
Discovered Higgs boson. Stay tuned!