us-cms t2 centers us-cms tier 2 report patricia mcbride fermilab gdb meeting august 31, 2007 triumf...
DESCRIPTION
US-CMS T2 Centers August 31, 2007US-CMS T2 Centers3 Current US CMS T2 Resources US CMS T2 Site CPU (kSI2K) Disk (TB)WAN (Gb/s) Caltech Florida MIT Nebraska Purdue UCSD WisconsinTRANSCRIPT
US-CMS T2 Centers
US-CMS Tier 2 Report
Patricia McBrideFermilabGDB MeetingAugust 31, 2007Triumf - Vancouver
August 31, 2007 US-CMS T2 Centers 2
US-CMS T2 Centers
US Tier 2s for CMS
• There are 7 CMS Tier-2 centers in the US.
• The US-Tier 2s are hosted at universities and are OSG sites.
• Funding through the US-CMS Software and Computing Project.
• L2 coordinator for US-CMS T2s: Ken Bloom, U. of Nebraska
August 31, 2007 US-CMS T2 Centers 3
US-CMS T2 Centers
Current US CMS T2 Resources
US CMS T2 Site
CPU (kSI2K)
Disk (TB) WAN (Gb/s)
Caltech 586 60 10Florida 519 104 10MIT 474 157 1Nebraska 650 105 10Purdue 743 184 10UCSD 318 98 10Wisconsin 547 110 10
August 31, 2007 US-CMS T2 Centers 4
US-CMS T2 Centers
US-CMS Tier2s
• In 2008 we expect all US T2 sites to be operating with 1M SI2K of CPU, 200TB disk and 10Gbit/s WAN - nearly double the current resource levels.
Note: In CMS about half the T2 resources are dedicated to simulation and the other half to analysis activities.
Some details• All US T2 sites use SL4 with Condor (or PBS),• Disk pools are managed by dCache with SRM as an
interface for remote data transfers.• Gratia provides the accounting services.• CMSSW software and CMS services such as
PheDEX are installed at the sites.
August 31, 2007 US-CMS T2 Centers 5
US-CMS T2 Centers
US -Tier 2 coordination and operations
• T2 sites are operated by the university groups.– Close connection to the CMS local user community– Local management of the facilities, purchases– Coordination through US CMS S&C project
• Each T2 site has 2 FTEs for operations:– This includes 1 FTE for operations of the storage system.
• Good communication and planning has paid off– The T2’s have a close collaboration with Fermilab and the FNAL
T1 group.– There is a US T2 meeting every 2 weeks. – There seems to be good coordination among the sites.
• T2s provide help for US-Tier3 sites, other OSG sites (Brazil, for example).
• The sites have often done early deployment and been testers for upgrades to software and services for CMS and OSG.
August 31, 2007 US-CMS T2 Centers 6
US-CMS T2 Centers
US-CMS T2 in CSA06
• US-CMS T2 sites were leaders during CSA06 in number of jobs hosted. Note the high success rates for these jobs.
August 31, 2007 US-CMS T2 Centers 7
US-CMS T2 Centers
Monte Carlo production for CSA07
• text
US Tier 2 sites are making a major contribution to CMS MC event production: 66M/196M events in the recent months.
Expectation for 2007: 50M events per month at all T2 sites. This has been achieved.
August 31, 2007 US-CMS T2 Centers 8
US-CMS T2 Centers
Commissioning• CMS Computing Model calls for analysis dataset transfers from
Tier-1 to non-regional Tier-2s.– The CMS commissioning team has set up a program of work to
debug these links and bring them to production quality.• Metric for a commissioned T1<->T2 link: 4 out of 5 days above 300
GB/day; in excess of 1.7TB for 5 days. • Links will be “decommissioned” or removed from production after 7
days with <300 GB/day.• Only commissioned links will be used in production for CSA07.
– This is a work in progress and it has been vacation season. Stay tuned.
• The seven US-CMS Tier 2 sites are commissioned and are used in production.– All US-Tier-2 links for transfers to/from the FNAL-Tier-1 have been
commissioned. – These sites were used extensively in CSA06 and in pre-CSA07 MC
production.
August 31, 2007 US-CMS T2 Centers 9
US-CMS T2 Centers
CMS Site Availability TestingHourly monitoring of sites with CMS SAM tests has improved site availability for CMS. CMS SAM “Green” correlates to high job success rate at site.US T2 sites normally have good availability rankings >80%.
80%
Includes OSG/CMS sites in Brazil
August 31, 2007 US-CMS T2 Centers 10
US-CMS T2 Centers
Data Transfers to US-CMS T2 CentersSnapshot of daily transfers
August 31, 2007 US-CMS T2 Centers 11
US-CMS T2 Centers
Data Transfers rates to T2s
Average daily transfer rate to US T2s sites is ~80 MB/sec.
These rates are dominated by transfers from Fermilab.
2007 Target: 20-200 MB/s depending on the link
August 31, 2007 US-CMS T2 Centers 12
US-CMS T2 Centers
Transfers: Source to all US-Tier2Most transfers to US T2 sites are from FNAL, but CMS has started to commission links to the T2 sites from other (non-regional) T1 sites.
All transfers to US T2 Source: Non-regional T1s
August 31, 2007 US-CMS T2 Centers 13
US-CMS T2 Centers
User analysis at the Tier-2s
• The number of jobs hosted at the seven US CMS T2s is routinely more than 1k/day.
• We expect the number of jobs to increase during CSA07 when more MC data sets will be hosted at T2 sites for the analysis. (Users needed!)
August 31, 2007 US-CMS T2 Centers 14
US-CMS T2 Centers
Data hosting/data managers
• Each T2 site will have a data manager who decides which datasets are hosted at the site. – Users make requests which are
honored if space is available.
• T2 disk is regarded as cache; datasets will not reside permanently at a T2.
• The T2 system is designed for frequent replacement of data sets determined by the needs of the local users or the physics groups.
Data managers at US Tier 2 sites.
Coordinating the placement of skim datasets at the US-T2 will be tested in CSA07.
August 31, 2007 US-CMS T2 Centers 15
US-CMS T2 Centers
Summary
• The US Tier 2 sites for CMS have been working well.• In early 2008, we expect all sites to be operating at
the full capacity required for startup.• For more details on US CMS T2 computing, see the
CHEP poster by Kenneth Bloom “US CMS Tier-2 Computing”.
• US-Tier 2s facilities are well organized and there are communication paths between sites to the Tier-1 at Fermilab.– We followed this example and created a new position in the
CMS computing org: the T2 liaison. Ken Bloom and Giuseppe Bagliesi are now the liaisons to the T2 sites around the globe.