lhc cms detector upgrade project 401-04-04 calorimeter trigger sridhara dasu, university of...
Post on 14-Dec-2015
216 Views
Preview:
TRANSCRIPT
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 1
401-04-04 Calorimeter Trigger
Sridhara Dasu, University of Wisconsin
DOE CD1 Review
26 August 2013
Sridhara Dasu, 26 August 2013
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 2
LHC Calendar 2012: Discovered the Higgs 2013-2014: Make upgrades to handle 14 TeV + 2 x 1034 cm–2 s–1
2015-2017: Discover new high energy physics phenomena using data at 14 TeV + study the Higgs + EWSB mechanism
2018-2019: Make upgrades to handle 14 TeV + 5 x 1034 cm–2 s–1
2020-2022: Program is dependent on what we find by 2018
CMS Plans 2013: Finalize trigger designs and prototyping 2014: Construct the trigger modules 2015: Early operation with pre-production modules (Stage-1) 2016-2017: Operations with fully upgraded trigger (Stage-2)
Rapid plan to ensure success in 2015-2017 run! Plans backed up by thorough prototyping that is already underway
The Grand Plan for Trigger Upgrade
Sridhara Dasu, 26 August 2013
LHC CMSDetectorUpgrade
Project
FPGA & Telecom technology advances Ubiquitous 10 Gbps links FPGAs with 72 rx/tx links with built in serdes
o Xilinx Vertex-7 platformo Gigantic coreso DSP units, BRAM and Embedded processors
Fiber optic componentso Avago mini-pods with 12x
Ripe time to upgrade from VME to uTCA
LHC long shutdown 2013-2014 Sufficient time for copper optical transition for inputs Ensures that upgraded electronics is ready for operation in 2016 Partial upgrade operation from 2015 itself
Sridhara Dasu, 26 August 2013DOE CD1 Review - US CMS Upgrade - Trigger 3
Technical Opportunity
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 4
Calorimeter Trigger Layers 1 & 2
Sridhara Dasu, 26 August 2013
US:
UK:
Vienna:
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 5
Algorithms
Algorithm layers First layer builds local clusters of energy deposits
o Determines energy and cluster type (EM or Hadronic) Second layer builds triggerable objects from clusters
o Electron/Photon, Tau, Jet, MET, …
Mapping to architectures Traditional architecture
o Algorithm layer-1 maps to physical layer-1 electronics– Key feature: nearest-neighbor sharing– Sufficient to build clusters locally
o Algorithm layer-2 maps to physical layer-2 electronics– All clusters available in one FPGA
Time multiplexing architectureo All primitive data passes through one FPGAo Potential for clever on-the-fly processing, if feasible/necessary
Sridhara Dasu, 26 August 2013
LHC CMSDetectorUpgrade
ProjectCurrent & Upgraded Calo Trigger
Sridhara Dasu, 26 August 2013DOE CD1 Review - US CMS Upgrade - Trigger 6
HCALEnergy
ECALEnergy
Regional Calo Trigger
Global Calo Trigger
EMcandidates
Regionenergies
HFEnergy
HCALEnergy
Layer 1Calo Trigger
Layer 2 Calo TriggerC
urre
nt L
1 T
rigge
r S
yste
m
Upg
rade
L1
Trig
ger
Sys
tem
HCAL Optical SplittersRisk mitigation
by operating current and
adiabatic stages of upgrade CT concurrently
CopperOptical
US
UK
US
UK
oSLB
oRM
oRSCCTP
MP7
HTR TCC mHTR mHTR
LHC CMSDetectorUpgrade
Project
Sridhara Dasu, 26 August 2013DOE CD1 Review - US CMS Upgrade - Trigger 7
Calorimeter Trigger ComponentsType Name Purpose
TCC Trigger Concentrator Card Existing ECAL trigger primitives (TP) (France)
oSLB Optical Serial Link Board Transmit ECAL TPs to RCT & CTP (Portugal / France)
mHTR HCAL Trigger & DAQ Board Create and transmit HCAL TPs
RCT Regional Calorimeter Trigger Existing trigger system
oRM Optical Receiver Modules Receive ECAL TPs on RCT(Portugal / France for Stage-1)
oRSC Optical Regional Summary Card Convert & transmit RCT output (US DOE NP for Stage-1)
CTP Compact Trigger Processor Large FPGA w/ optical & backplane
CIOX Compact IO cross-point switch Intra-crate sharing switch
MP7 Multipurpose Processor Large FPGA w/ only optical links (UK)
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 8
Algorithm Studies
Stage-1 uses clusters from RCT+HF processed by new mTCA cards Adiabatic improvements to calorimeter trigger dataflow Algorithm stage takes advantage of larger FPGA resources Performance of the “revised” algorithms is very good
o eGammaTau studied by FNAL/Wisconsin o eGamma and muon isolation studied by UCSD o Muon isolation and b-jet tagging with muons studied by Ohio o Heavy Ion jet cleanup studied by MIT o Improved jet finding with PU subtraction by UIC
Further improvements using full ECAL granularity in Stage-1o Improved eGammaTau isolation & position (FNAL, Wisconsin)o Pile-up handling with lower LSB for ECAL (FNAL, Wisconsin)o Improved jet position and PU subtraction (Davis)
Sridhara Dasu, 26 August 2013
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 9
Electron / Photon Trigger
Improved control on isolation Factor of 2-3 reduction in rate with small loss in efficiency
Sridhara Dasu, 26 August 2013
LHC CMSDetectorUpgrade
ProjectTau Trigger
Big improvement in efficiency with ~10X rate reduction!
Sridhara Dasu, 26 August 2013DOE CD1 Review - US CMS Upgrade - Trigger 10
Current tau trigger has large and negative PU dependence
LHC CMSDetectorUpgrade
ProjectJet Triggers
Sridhara Dasu, 26 August 2013DOE CD1 Review - US CMS Upgrade - Trigger 11
Single jet
Single jet thresholds similar to current Multi-jet trigger thresholds better in upgrade (PU sub.) Small improvement in efficiency turn-on
Quad jetHT = S PTJets
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 12
Process(x2 improvement highlighted)
1.1 x 1034 cm–2 s–1 2.2 x 1034 cm–2 s–1
Current Upgrade Current Upgrade
W(en),H(bb) 57.7% 87.0% 37.5% 71.5%
W(mn),H(bb) 95.9% 100% 69.6% 97.9%
VBF H( ( )tt mt ) 42.6% 51.3% 19.4% 48.4%
VBF H( ( )tt et ) 24.4% 44.3% 14.0% 39.0%
VBF H( ( )tt tt ) 17.2% 53.7% 14.9% 50.1%
H(WW(eenn)) 91.4% 97.8% 74.2% 95.3%
H(WW(mmnn)) 99.9% 99.9% 89.3% 99.9%
H(WW(emnn)) 97.6% 99.4% 86.9% 99.3%
H(WW(menn)) 99.6% 99.5% 90.7% 99.7%
StopbWce, jets (600 – 450 GeV) 55.8% 68.2% 50.3% 64.8%
StopbWc ,m jets (600 – 450 GeV) 78.1% 81.6% 76.4% 84.5%
RPV Stopjets (200 GeV) 70.1% 99.9% 43.6% 99.9%
RPV Stopjets (300 GeV) 93.7% 99.9% 79.7% 99.9%
Physics Performance Summary
Ave
rage
Im
prov
emen
t: 1
7% (
Low
Lum
i) &
40%
(H
igh
Lum
i)
Sridhara Dasu, 26 August 2013
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 13
Prototypes for all major cards exist & have passed tests
Upgrade is based upon common μTCA hardware platform also used by HCAL & other CMS systems
Upgrade trigger is replacing existing trigger hardware built by the same team → interfaces well understood
Same team wrote software & firmware for existing trigger system → scope & requirements well understood
Requirements are frozen in approved Trigger TDR
Sridhara Dasu, 26 August 2013
Technical Feasibility
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 14
Wesley Smith CMS Trigger Project Manager 1994-2007, Trigger Coordinator 2007 – 2012 US CMS L2 Trigger Project Manager 1998 – present
Sridhara Dasu US CMS L3 Manager for Calorimeter Trigger, 1998 – present Author of original & upgrade cal. trig. Algorithms 1994 – present
Pam Klabbers – Operations & Testing CMS RCT Operations Manager (more than a decade on RCT project) CMS Deputy Trigger Technical Coordinator
Tom Gorski – Hardware & Firmware Over a decade of engineering on the CMS Calorimeter Trigger
Mathias Blake – Firmware New to project -- 5 years Applications Engineer at Xilinx, 2 years FPGA Engineer at DRS
Technologies, 3 years FPGA Design Engineer at Dolby Labs
Jes Tikalsky and Evan Friis – Software Providing prototype testing software (Friis has experience with current RCT)
Sridhara Dasu, 26 August 2013
Calo Trigger Team Experience
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 15
Calorimeter Trigger WBS Detail
Sridhara Dasu, 26 August 2013
←MuonTrigger
M&S paid byCMS-France
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 16
Upgraded HCAL Readout & Trigger Electronics (μHTR) is after split that sends data to old electronics (HTR)
μHTR supplies TP to upgrade trigger, HTR continues to send to present trigger
ECAL Trigger Concentrator Cards (TCC) are not upgraded, instead their mezzanines with trigger serial links are replaced (Lisbon):
Optical Serial Link Board oSLB replaces single copper link to current Regional Calorimeter Trigger (RCT – U. Wisc.) with two optical links (WBS 401.04.04.06)
One optical link to optical Receiver Module (oRM) on RCT (WBS 401.04.04.05)o Replaces copper Receiver Module
One optical link to input of Upgraded Calorimeter Trigger: CTP7 Card
574 oSLB’s & oRM’s with Fibers installed in 2014 French Contribution to Calorimeter Trigger (no US Upgrade Funds)
Presently oSLB & oRM in final prototype stage oSLB:
oRM:
Sridhara Dasu, 26 August 2013
HCAL & ECAL Trigger Primitives
OSLB Output / oRM Input: 4x channels @ 1.2 Gbps 4.8 Gbps
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 17
Main processor card Virtex-7 690T
for processing ZYNQ
for TCP/IP + linux 60 10G optical
Input links 36 10G optical
Output links Function: Find ET
clusters & transmit to Layer-2
• Card Count = 46• 36 total +
8 spares + 2 test setups
Sridhara Dasu, 26 August 2013
401.04.04.02: CTP Card Concept
Virtex-7 VX690T FPGA
ZYNQXC7Z03
0EPP
1.5V
S
up
ply
2.5V
S
up
ply
3.3V
S
up
ply
1V
30A
S
up
ply CXP Module
12Tx + 12 Rx
CXP Module12Tx + 12Rx
CXP Module12Tx + 12 Rx
12XRx
12XRx
(CTP-6 BG View)
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 18
CTP: Virtex-6 Prototype Exists
Sridhara Dasu, 26 August 2013
Back End FPGAXC6VHX250T/XC6VHX380T
Front End FPGAXC6VHX250T/XC6VHX380T
Avago AFBR-810B Tx Module
4X Avago AFBR-820B Rx Module
MMC Circuitry
JTAG/USB ConsoleInterface Mezzanine
Power Modules
Dual SDRAM for dedicated DAQ and TCP/IP buffering
12x Multi Gig Backplane Connections
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 19
CTP7 is simpler than CTP6 prototype Engineering effort for in-house design based on CTP6 work Procurement of PCBs is based on recent CTP6 purchase Good estimate of cost and schedule from prototype manufacture
Quotes available for all parts Virtex-7 690T is the major cost driver – with recent quote
o 3 690Ts donated by Xilinxo 5 690Ts purchased from Xilinxo Total of 8 FPGAs in-hand for prototypes
Optical modules based on recent purchase Miscellaneous parts also have quotes or estimated from
prototype purchases (power modules)
Sridhara Dasu, 26 August 2013
CTP: Basis of Estimate - Hardware
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 20
Firmware and software efforts estimated from prototype testing + prior RCT experience
Firmware algorithms for testing provide the basis for the core firmware, which is the most complicatedo Tom Gorski and Mathias Blake
Trigger algorithm firmware work is based on prior experience managing GCT project (UK)o Wesley Smith was L1 project manager overseeing GCT
Software effort is based on current RCTo Sophistication of embedded Linux / TCP and shorter timeline for
implementation requires professional software effort– Jes Tikalsky
o Bulk of testing software from post-doc and graduate student– Pam Klabbers, Evan Friis and graduate students will develop software
Board and Firmware/Software done by the same group (Wisconsin) results in efficiencies
Sridhara Dasu, 26 August 2013
CTP: Basis of Estimate - Effort
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 21
Function: For inter-board connections
Input/Output: 48x 4.8-6.4 Gbps
Prototype exists
Fully tested
Firmware is minimal and exists
CIOX cost well known
Ready for productionSridhara Dasu, 26 August 2013
401.04.04.03: Crosspoint IO Card– U. Wisconsin
Controller (MMC and link mgmt)
4X Avago AFBR-79EQDZ QSFP+ ModulePositions
4x4
Lan
e B
idir
ecti
on
al M
ult
i G
ig
Bac
kpla
ne
Co
nn
ecti
on
s
Backplane Rx/Tx
Redriver ICs (top and
bottom sides)
(2/crate x 3 crates = 6 + spares)
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 22Sridhara Dasu, 26 August 2013
401.04.04.04: CTP Infrastructure:Vadatech VT894 Crate Test Setup
BU
AM
C13
Vad
atec
h
MC
H UW
CT
P-6
UW
CT
P-6
TTC Downlink
UW
Au
x
(Final system: 3 crates w/ 12 CTP7 ea. + 2 test setups + spare = 6)
U. Wisconsin designed backplane with dense card interconnectsmanufactured & installed in commercial Vadatech VT892 Crate available in Vadatech Catalog
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 23
The dominant risks are: Availability of ALL trigger primitive inputs from ECAL,
HCAL in optical format Validation of large optical link plant in limited time Fully validated trigger algorithms in firmware Control and operations software
Risk mitigation strategy: Continue to provide fully operational current trigger
system in parallel with upgrade commissioning Partial operation of the upgrade systems provide
tangible benefits from 2015 onwards
Sridhara Dasu, 26 August 2013
CTP: Risk and Mitigation Strategy
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 24
Goal (Original): Provide readout of original RCT descoped
during construction project Present readout through GCT input buffer
not workable with trigger evolution Uses connection to a single CTP6 prototype
or CTP7 card for DAQ readout
optical Receiver Summary Card (oRSC) paid by DOE Nuclear
VME Slave Interface Card Fits in current RCT Crates (1 per crate) 18 Cards in 2015 System Receives RCT Jet Sum Card Output
to GCT on Copper “SCSI” Data Cables Provides direct optical input to GCT
o Bypassing old optical conversion cards Planned use for Heavy Ion Triggers Prototype under test
Multiple optical outputs provide: Inputs for upgrade calorimeter trigger &
parallel operation of old & new trigger wherever ECAL & HCAL electronics notavailable in 2015
Sridhara Dasu, 26 August 2013
Calorimeter Trigger Risk MitigationoRSC Function:
Conversion to opticaloRSC Input:
RCT egt and jet clusters
80 MHz Parallel ECLoRSC Output:
6x copies at 6.4 Gbps optical
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 25
Input/Output verified – low bit error rate – clear eyes
Sridhara Dasu, 26 August 2013
Prototypes Status (Optical Links)C
TP
6 to
CT
P6
(48x
6.4
Gbp
s lin
ks)
oRS
C t
o C
TP
6 (1
1x 6
.4 G
bps
links
)
oRSC to MP7(4x 6.4 Gbps links)
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 26
12-links tested at 6.4 Gbps – no errors
Sridhara Dasu, 26 August 2013
HCAL TPG to CTP Integration Test
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 27
Running Linux on CTP6 FPGA – PetaLinux
Startup screen via USB console
Root login, pinging controller PC at 192.168.1.2
Running on CTP-6 back end FPGA (`VHX250T)
Important step towards CTP-7 ZYNQ-based Linux
Sridhara Dasu, 26 August 2013
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 28
mHTR to CTP6 integration test at Madison (done) Wisconsin/Minnesota : 12 links operated successfully
oRSC to CTP6 demonstrator at Madison (done) Test 2-3 oRSCs and interface before shipment to CERN
oRSC to CTP6 platform at Prevessin (done) 2 oRSCs emulate RCT + 1 CTP6 for RCT readout and algorithm tests Full platform for system operations and control software development
oRSC - MP7 Integration test at Prevessin (done) Wisconsin/Imperial July 2013 Milestone
CTP6 - oRSC-oRM/RCT - oSLB/TCC Integration Test at Prevessin Wisconsin/Lisbon/LLR October 2013 Milestone
CTP6 - oRSC-oRM/RCT - oSLB/TCC -uHTR Integration Test at 904 Wisconsin/Lisbon/LLR/Minnesota November 2013 Milestone
CTP7 - MP7 Integration test at Prevessin Wisconsin/Imperial March 2014 Milestone
Sridhara Dasu, 26 August 2013
Demonstrators & Critical Integration Tests
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 29
2013 M&O + R&D Activity : oRSC construction, installation Firmware development for RCT readout and testing using CTP6 Lisbon / France : oSLB and oRM construction, installation Minnesota / Wisconsin : mHTR to CTP6 testing
2014 Ensure fully working legacy RCT system with oRSC + CTP6 RO CTP7, CIOX, … construction, validation Firmware development for CTP7 testing and 2015 operation
2015 Commission and operate Stage-1 system Firmware development and integration of Stage-2 system
2016 Commission and operate Stage-2 system
Sridhara Dasu, 26 August 2013
Adiabatic Upgrade Timeline
LHC CMSDetectorUpgrade
Project
DOE CD1 Review - US CMS Upgrade - Trigger 30
Launch Production (Buy parts, fabricate PCBs): 1QFY2014
Deliver Stage-1 Calorimeter Trigger (4 CTP boards + 1 crate + oRSCs): 4QFY2014
Initial operation at CERN (Firmware for Stage-1 + HF clusters): 2QFY2015
Deliver Stage-2 Calorimeter Trigger(All CTPs, CIOx, 3 crates): 2QFY2015
Test High Luminosity Operation of Calorimeter Trigger (All Firmware for Stage-2): 4QFY2016
Sridhara Dasu, 26 August 2013
CTP: Production, Installation, etc
top related