firmware development and testing of the atlas ibl back-of-crate card

1 of 1406.06.2014

Marius Wensing, M. Sc.Firmware development and testing of the ATLAS IBL BOC cardTIPP2014, Amsterdam

Firmware development and testing of the ATLAS IBL Back-Of-Crate card

Marius Wensing on behalf of the ATLAS IBL DAQ group

TIPP2014, Amsterdam, 2nd-6th of June 2014

2 of 1406.06.2014


ATLAS Pixel Detector

ATLAS is one of the four big experiments at the LHC at CERN.

Pixel Detector is the innermost detector with 80 mio. pixels in 3 barrel-layers and 3 disks per side.

Currently it is being updated with a new innermost layer (Insertable B-Layer) with additional 12 mio. pixels.

Pictures: CERN

3 of 1406.06.2014


Insertable B-Layer

Bandwidth permodule increasesby a factor of 2

IBL stave:

–12 planar double-chip modules

–8 3D single-chip modules

→ 32 front-end chips per stave New readout hardware

to cope with higher bandwidth

Pixel B-Layer IBL

Pixel size [µm x µm] 50 x 400 50 x 250

Number of pixels per module

46080 53760

Distance to interaction point in z [mm]

50.5 33

Readout bandwidth per module [Mbit/s]

160 320

→ Details in Cécile Lapoire’s IBL overview talk

4 of 1406.06.2014


IBL readout system

IBL readout isbased on thecurrent Pixeldetector readout.

Parts of the Off-Detector-system:–ROD: Read-Out-Driver (→ details in next talk by Shaw-Pin Chen)

–BOC: Back-Of-Crate card

–TIM: Timing, Trigger, Control Interface Module

–SBC: VME Single-Board-Computer

5 of 1406.06.2014


ATLAS IBL BOC card

BOC card is responsible for signal processing of data to and from the detector as well as monitoring the signalquality and providing the link to thehigher level readout.

Components:

–1 BOC Control FPGA (BCF, Spartan-6)

–2 BOC Main FPGAs (BMF, Spartan-6)

for signal processing

–Gigabit Ethernet, VME

–Optics: SNAP12, QSFP Optical interfaces:

–Downlink: 16 channels (BPM, 40 Mbit/s)

–Uplink: 32 channels (8b10b, 160 Mbit/s)

–Higher level readout: 8x 2 Gbit/s SLINK

6 of 1406.06.2014


Signal processing

Downlink (to the detector):

Uplink (from the detector):

Requirements:• Sending clock phase is unknown• 160 Mbit/s data stream• 8b10b encoded data

Requirements:• BPM encoding of clock and data

into single data stream• Adjustment of the detector timing in

steps of ~100 ps

7 of 1406.06.2014


Implementation of the Fine Delay block

Adjustment of the detector timing is needed to compensate the effect of fibre lengths and distances to the interaction point.

Requirements:

–Delay in steps of around 100 ps

–Duty cycle should not exceed (50±2)%

to ensure proper optoboard operation. Different approaches were

taken into account:

–External delay chips

–Propagation delays in-

side the FPGA

– IODELAY2 primitive

Picture: Xilinx

8 of 1406.06.2014


Implementation of Fine Delay with IODELAY2

Spartan-6 has different configuration options for the IODELAY primitive:

–Fixed output delay

–Fixed/variable input delay Variable output delay requires

workaround:

–Variable input delay of unused I/O

will be routed back to the output.

–High distortion of the duty cycle.

–Additional I/O ports are needed.

Picture: Xilinx

9 of 1406.06.2014


Partial reconfiguration approach

Fixed output delay configuration can be changed by accessing the internal configuration memory over the ICAP interface.

FPGAs have the possibility ofpartial reconfiguration.

Idea: After the synthesis differentconfiguration files will be generatedby changing the delay settingper channel.

A differential (partial) configurationis generated and put into the FPGA‘sconfiguration memory.

Possibility to change the delayduring operation of the system.

10 of 1406.06.2014


Results from the delay test

Partial reconfiguration of the IODELAY2 gives excellent results in term of linearity and duty-cycle.

Mean: 34.93 ps / SettingRMS: 0.7 ps / Setting

11 of 1406.06.2014


Front-End Emulator

Front-End Emulator was implemented into the BOC firmware.

Emulator can be used for debugging the full readout chain during ATLAS milestone runs, even when the detector is not yet available for operation.

Features:

–Global register read/write

–Configurable Chip ID

–Manual/Random hit injection

–Configurable number of random hits per trigger

12 of 1406.06.2014


Front-End Emulator: Hit occupancy measurement

→ Emulator provides uniform distribution of hits!

13 of 1406.06.2014


Summary

The upgrade of the pixel detector needs new FPGA-based readout electronics processing 32 channels at a total data rate of 5 Gbit/s.

Timing of the detector is controlled by FPGA internal resources.

New method of adapting the fine delay using partial reconfiguration shows a very homogenious distribution of the delay per setting.

Front-end emulator makes debugging of the full readout chain much easier.

Production test ensures the good quality of delivered cards.

14 of 1406.06.2014


Thank you for your attention!

15 of 1406.06.2014


BACKUP SLIDES

16 of 1406.06.2014


IBL BOC Block diagram

Det

ecto

rD

etec

tor

ROS

ROS

VM

E b

ackp

lane

17 of 1406.06.2014


BOC production test overview

After reception from the manufacturer a test procedure is performed on all cards:

1. Optical inspection

2. Electrical test and FPGA programming

3. Optical tests (loopback, signal integrity)

4. Coarse and fine delay calibration

5. Backplane tests & SLINK test Software utility for automatic testing has been developed.

So many time-consuming tests can be run without user interaction.

Test results will be documented in a database.

firmware development and testing of the atlas ibl back-of-crate card

Documents

atlas ibl boc cardtipp2014

ibl stave

atlas ibl daq grouptipp2014

detector timing

innermost detector

readout bandwidth

crate cardmarius wensing

boc control fpga bcf