the read out driver for the atlas muon precision …thei/hef/atlas/pub/lecc2002-prest-mrod.pdf11...
TRANSCRIPT
11 September 2002 LECC 2002, Colmar 1
The MRODThe Read Out Driver for the
ATLAS Muon Precision Chambers
Marcello Barisonzi, Henk Boterenbrood,Peter Jansweijer, Gerard Kieft, Jos Vermeulen
NIKHEF, AmsterdamAdriaan König,Thei Wijnen
NIKHEF and Univ. of Nijmegen, Nijmegen
11 September 2002 LECC 2002, Colmar 2
Contents
• MROD System Overview• MROD-0 Feasibility Study• MROD-1 Prototype• First Results• Conclusions & Outlook
11 September 2002 LECC 2002, Colmar 3
ATLAS MDT Muon Detector
~ 300.000 Drift Tubes
~ 1200 MDT Chambers
~ 200 Towers of 6 Chambers
11 September 2002 LECC 2002, Colmar 4
System OverviewChamber Tower
24 ch. TDC
CSM18 x
24 ch. TDC
CSM-Link(GOL)*
6 x MROD 160 MBytes/sS-Link to ROB
24 ch. TDC
CSM18 x
24 ch. TDC
CSM-Link(GOL)
*) http://proj-gol.web.cern.ch/proj-gol
11 September 2002 LECC 2002, Colmar 5
CSM Functionality
Serial to Parallel&
Clock Domain Separator
40 Mbit/sData/Clockfrom TDC
18 x
Serial to Parallel&
Clock Domain Separator
40 Mbit/sData/Clockfrom TDC
Separator
(GOL)
≈1 Gbit/s
1 Start bit32 Data bits
1 Parity bit2 Stop bits
36 bits @ 25 ns = 900 ns
1 Separator word (S)18 TDC data words19 words in 900 ns 85 MB/s
S1
18
CSM
11 September 2002 LECC 2002, Colmar 6
TDC0, word 1
TDC2, word 4
TDC3, word 2
TDC0, word 1
TDC1, word 3
TDC2, word 5
TDC3, word 3
TDC3, word 0TDC2, word 0TDC1, word 0
TDC1, word 1
TDC1, word 2
TDC2, word 1
TDC2, word 2
TDC2, word 3
TDC3, word 1
Build events in a partitioned memoryfrom TDC data fragmentstim
e
(tdc 1) 000…000
Separator word
Separator word
Separator word MROD FunctionalityCheck (do not store)
Skip (do not store)
(tdc 0) 000…000
TDC0, word 0
11 September 2002 LECC 2002, Colmar 7
MROD Form Factor• 9 U VME board (single slot), 6 CSM Inputs, 1 S-Link Output• Optionally 2 extra CSM Inputs with “extension” board to
accommodate some special towers with > 6 chambers• CSM Input Interfaces integrated on main board
• 1 MROD Crate (Sub rack) contains:12 MRODs (12 η Segments)Up to 4 MROD extension boards1 Crate Master with Ethernet Interface (ROD Crate DAQ)1 TIM: TTC-Rx Interface Module (incl. ROD Busy)
• @ 192 towers: 192/12 = 16 MROD Crates (1 per ϕ Sector)
11 September 2002 LECC 2002, Colmar 8
MROD Crate
ROD CrateDAQ
MROD MROD… total ...12 x
6 CSMs 6 CSMs
TIM (TTCrxInterface)
ROD BusyROB ROB
DAQ / DCS
VME-bus
One MROD Crate services 12 towers (one full ϕ sector). In total 16 crates will be required for all MDT chambers. Some MRODs may have 7 or 8 input links via “slave” MROD input cards.
From TTC system
“TIM-bus”
Network
11 September 2002 LECC 2002, Colmar 9
MROD Throughput
MRODMROD-in
MROD-out
Average 5 hits per TDC + header + trailer = 7 words/eventPer tower of 6 chambers max. 88 TDCs * 7 ≅ 600 words/event (= 2.4 kB/event)Worst case est.: @ 100 kHz L1A rate 240 MB/s per MRODCalculation based on actual tower layout (J.Chapman): max. rate < 60 MB/s per MROD
CSM-Link
S-Link
CSM-Link
CSM-Link
CSM-Link
CSM-Link
CSM-Link
MROD-in
MROD-in
11 September 2002 LECC 2002, Colmar 10
MROD-0 Feasibility Study
MROD-in
MROD-in
MROD-in
MROD-in
MROD-in
MROD-in
MROD
MROD-out
CSM
CSM
CSM
ROB
CSM
CSM
CSM
11 September 2002 LECC 2002, Colmar 12
MROD-0 Prototype
MROD-out
SHASLINK
MROD-in
MCRUSHsorted
TDC-dataover
SHARC Link
11 September 2002 LECC 2002, Colmar 13
MROD-0 Input Channel
1 MBZBT
Memory
SHARC
FPGA
Data FIFO
TetrisRegister
InputOutput
FIFO Control
Control/StatusError signaling
6 Sh
arc
l inks
@ 4
0 M
B/ s
eac
h
FIFO
Length FIFO
MCRUSH
11 September 2002 LECC 2002, Colmar 14
MROD-0 Output Channel
SHaSLINK
PCI 9054
PCI b
us
SHARC
Altera10K10A
S-Link max.@ 160 MB/s
6 SHARC Links@ 40 MB/s each
11 September 2002 LECC 2002, Colmar 15
MROD-0 Emulation Hardware
MROD-out
SHASLINK
1
0MROD-in
(3x)
MCRUSH
0
2
3
MROD-0
fragment lengths
sortedTDC-data
sorted +merged
TDC-data
optionally double/tripleMROD-in output thus
simulating 2 or 3 MROD-insevent fragment lengths
via SHARC-link simulatesfuture MROD-1 functionality
1 3
S-Link
Module typexxxxx
SHARC-links
ROBSIM
CRUSH +SHASLINK
SHASLINK dataTDC-
CSMSIM0
11 September 2002 LECC 2002, Colmar 16
MRODOUT ROBSIM CSMSIM MRODIN
MROD
CSM-simulator performance
0.0
50.0
100.0
150.0
200.0
250.0
300.0
350.0
400.0
450.0
0 5 10 15 20 25 30 35
Words/TDC
Even
t rat
e [k
Hz]
11 September 2002 LECC 2002, Colmar 17
MRODIN (1x) + MRODOUT + ROBSIM, 18 TDCs
0,0
20,0
40,0
60,0
80,0
100,0
120,0
140,0
160,0
180,0
0 5 10 15 20 25 30 35Words/TDC
Eventrate
[kHz]
MRODIN MRODIN+MRODOUT MRODIN+MRODOUT+ROBIN
MRODOUT MRODIN
MRODMROD-0PerformanceStudy Results
CSMSIM ROBSIM
11 September 2002 LECC 2002, Colmar 18
MRODIN (1x) + MRODOUT + ROBSIM, 6 TDCs
0,0
20,0
40,0
60,0
80,0
100,0
120,0
140,0
160,0
180,0
0 5 10 15 20 25Words/TDC
Eventrate
[kHz]
MRODIN MRODIN+MRODOUT MRODIN+MRODOUT+ROBIN
MRODOUT MRODIN
MRODMROD-0PerformanceStudy Results
CSMSIM ROBSIM
11 September 2002 LECC 2002, Colmar 19
MRODIN (2x) + MRODOUT + ROBSIM, 6 TDCs
0,0
20,0
40,0
60,0
80,0
100,0
120,0
140,0
160,0
180,0
0 5 10 15 20 25Words/TDC
Event rate
[kHz]
MRODIN MRODIN+MRODOUT MRODIN+MRODOUT+ROBIN
MRODOUT MRODIN
MRODMROD-0PerformanceStudy Results
CSMSIM ROBSIM
11 September 2002 LECC 2002, Colmar 20
MRODIN (3x) + MRODOUT + ROBSIM, 6 TDCs
0,0
20,0
40,0
60,0
80,0
100,0
120,0
140,0
160,0
180,0
0 5 10 15 20 25Words/TDC
Event rate
[kHz]
MRODIN MRODIN+MRODOUT MRODIN+MRODOUT+ROBIN
MRODOUT MRODIN
MRODMROD-0PerformanceStudy Results
CSMSIM ROBSIM
11 September 2002 LECC 2002, Colmar 21
MROD-0 Performance Analysis• Measured event rate for single output SHARC-I @ 40
MHz with 2 and 3 emulated CSM inputs: maximumevent rates of 70 and 50 kHz respectively are measured.
• MROD-1 will use the SHARC-II @ 80 MHz: both the processing speed and the bandwidth increase proportionately → event rate ≈ 100 kHz ?
• → use 2 SHARC-II processors for MROD-out.
11 September 2002 LECC 2002, Colmar 22
Memory
SHARC
FPGA
SHARC(2x)
MemoryFPGA
3x (in total)
VME64x
TTCInterface
Memory
SHARC
FPGA
MemoryFPGA
Sharc Links
MROD-1Prototype
11 September 2002 LECC 2002, Colmar 24
The ADSP-21060 and the ADSP-21160 SHARCs
• 40 MHz / 80 → 100 MHz CPU (SIMD mode)
• 512 KB / 512 KB internal memory
• 6 x 40 / 80 → 100 MB/s links. Throughput of all links simultaneously is 160 / 480 → 600 (?) MB/s, without disturbing the CPU.
• No handshaking on links, but hardware XON-XOFF protocol,
• 10 / 14 DMA channels
• Support for bus arbitration: at max. 6 SHARCs can be connected to a commonbus without glue logic. Each SHARC can access the internal memories of each otherSHARC. The SHARCs also provide support for a so-called host interface, which canact as an additional master on the common bus.
• Fast interrupt servicing due to the presence of shadow registers
• Two 40 Mbit/s / 40 → 50 Mbit/s (at max.) synchronous serial ports
• Can be booted via link 4
11 September 2002 LECC 2002, Colmar 25
MROD-1 Form Factor• 9 U VME boards, 2 units wide
• 1 S-Link output on daughter board
• 6 GOL inputs on daughter boards
• SHARC II (ADSP21160) DSPs:(3 for input, 2 for output processing)
• Altera APEX FPGAs, 200k gates• TIM bus over special P3 back plane• VME64x interface
GOLdaughterboards
Input
Input
S-Linkdaughterboard
Input
Output
Motherboard
11 September 2002 LECC 2002, Colmar 28
MROD-1 Prototype Status• Fully functional MROD-1 modules exist:• 7 MROD-in and 3 MROD-out boards have all
been extensively tested, also in conjunction with simulated data via the input link.
• Software to boot the SHARCs via the VME bus and a run-time environment providing file and terminal I/O via a server program under both the LynxOS and Linux operating systems on the VME processor is available.
11 September 2002 LECC 2002, Colmar 29
MROD-1 Prototype Status (Contd.)
• Current MROD-1 SHARC software is largely ported from the MROD-0 prototype. Further development of this code is required.
• This process is hampered by a compiler which is not fully reliable, in particular when it comes to optimization which in turn is very important to obtain good performance.
• Transition to SHARC-II DSPs has not always been completely straightforward: work-aroundshave been implemented. See example below ….
11 September 2002 LECC 2002, Colmar 31
First MROD-1 Results
• Preliminary single channel tests with a CSM simulator show a sustainable 125 kHz event rate with 18 simulated TDCs with five 32-bit words per event.
• Equivalent of 45 MBytes/sec.• Rate of 125 kHz appears to be limited by the
input (i.e. the CSM simulator) rather than by the MROD-1 itself. This is being investigated.
11 September 2002 LECC 2002, Colmar 32
Conclusions & Outlook• The first MROD-1 test results are in line with the
MROD-0 results and the SHARC-I to SHARC-II extrapolation.
• Results are encouraging. Improvements still possible. More tests need and will be done.
• Development of a GOL receiver card to connect to the CSM is well underway.
• System integration tests with both the CSM and the DAQ-1 and ROD CRATE DAQ follow soon.
• Application in NIKHEF local Cosmic Ray Test Stand.