fib & ffo - hep.ucsb.edu

Marcel Stanitzki Yale UniversitySilicon Workshop 2006 Santa Barbara

FIB & FFOBrothers in crime

Marcel StanitzkiYale University


The Big Picture

SVX DAQ is big,

but not really complicated !

Focus of this talk


Lovely Acronyms

● FFO: FIB FanOut● FIB: Fiber Interface Board● FTM: FIB Transition Module● VME bus: VERSAModule Eurocard bus


The FFO

● Basically a Signal/Command Distributor (1 SRC -> n FIBS)

● Necessary as a local Clock source for a FIB crate (in absence of an SRC)

● The FFO has a lot of diagnostics features to report back to the SRC or via VME

FFO

Commands/Clock from SRC

ClocksCommands

ClocksCommands

J3 Backplane


The FFO (Rev B)

● 9 U VME Board● 1 GLINK Receiver & 1 RS-485

port● Custom J3 Backplane (CDF FIB)● Firmware cannot be upgraded via

VME● Also used for the Resonance

Protection System (See Ghostbuster Talk)


The FFO in SVXDAQ

Diagnostics and FIFO control (You'll never use those)

FFO Operation Mode

Controls for Remote Reset and Resonance

protection

GLINK Status

State MachineInfo

Board RevisionFirmware version


FFO failures● Very stable board, very few problems so far● Main problem with FFO's is the GLINK Sync with the SRC

– 1 Board was swapped in 2005 due to various GLINK sync losses– Only swap so far (as I recall)

● Some Problems with the READOUT RETURN line addition in early 2004 (fixed in firmware)

● Discovery of a logic race between SRC/FFO in Autumn 2004– Needs very specific conditions in order for it to happen– If it happens, it is spectacular, entire detector might trip.– Protected by software


The FIB - what does it do ?

● One FIB board controls 2 wedges (= 10 HDI's)● Receives commands from SRC, translates it into SVX3 D

command sequences and then send them to the HDIs● Provides FrontEnd (132 ns) and BackEnd (19 ns) Clock to

the Chips● Receives Data from the DOIMs, does some basic

processing and sends it to the VRB via GLINKS


The real thing

● 9U VME Board● 17 FPGAs (Altera)● Custom J3 Backplane

(CDF-FIB)● 4 GLINK Transceivers● Firmware can be

updated via SVXDAQ


FIB box diagram

VME Int erface

J3

GLINK Controller

GLINK 0

GLINK 1

GLINK 2

GLINK 3

FIB Command Sequencer

& Chip Clock Generator

VME

Con tro l

01234

01234

SRC Commands & Clock via FFO

Pipeline Controller

Chip SequencesFE/BE clockData Wedge A & B

Data to VRB

FIB Configuration via VME

e.g. SVXDAQ or Runcontrol


FIB Command Sequencer

● Commands are queued in Command FIFO

● Simple message turns into complicated control sequence

● FIFO can be stopped, skipped, current sequence can be interrupted – Timing matters!

● Static RAM and Sequencer can be overwritten (via VME)


FIB PipelineData

● A FIB has 10 Pipelines● Pipeline Processor inserts FIB headers/End of record● Can perform Pedestal Subtraction (Calc/Ped RAM). We

do not use this● UnGray Data (SVX3D writes data in Gray code)


FIB Data Stream

The FIB writes “The self describing data” (S.NAHN)

e0002cfc 8319003e 013e023a 033f043d 0b3b0c29 0d3a0e39 133a1438 153a163a 25392628 2738283b 2a392b38 2c3a8219 a1194d39 4e394f39 553b5628 573ea019 7f01c1c1

● FIB Header: 2 bytes ● Bunch Crossing (BX)/ BackEnd state(BE) : 2 bytes

– BX/BE from SRC, central for all 10 HDI's● Chip ID/CAPID: 2 bytes per Chip● End of Record (EOR) :2 bytes

For 1 HDI :


The FIB Header

● W:– e or SVX f for ISL

– f for L00

● X:– bulkhead 0-5

(SVX/ISL)

– 8 for L00 West, a for East

● Y:– Wedge 0-b for SVX

– Wedge 0-5 for ISL/L00

● Z– Layer 0-4 (SVX/ISL)

– Layer 0-3 (L00)

Format is WXYZ

WXY can be set freely via VME (just a name)


FIB EOR

● C1C1 : Data stream ok● C2C2 :

– No data – Data stream corrupt (e.g not last chip last channel)– Max number of BackEnd clocks exceeded– 00F3 8 identical bytes in a row

● C4C4 : Fill Characters● C8C8 :

– Event abort (e.g. SRC L2R)

FIB terminates data stream from a HDI with CXCX


GLINK

● Commercial Protocol (originated by HP)● Serial Protocol, 1.5 Gigabit/s● Transmitted via Fibers (Finisar Drivers 850 nm

Wavelength)● max 20 Bit per GLINK● CDF's funny encoding

– 4 GLINKs 80 Bits = 10 HDIs with 8 bit each– 1 GLINK e.g. HDI 0/1 + 4 Bits of HDI 4

● GLINK needs synchronisation


What is D0 ?

● GLINK Data stream must to be continuous

● DOIM Data stream is not● For every gap, 0xD0 are

inserted (Fill bytes)● This avoids Data UnderRun● The VRB removes them (see

Tuula's Talk)

CHIP Data Data

DOIM Data

GLINK Data

D0D0 D0D0


The Blinking Lights ...VME Bus access

Voltages (3 LEDs, must be on)

RESET

Voltages (all on) VME Bus access

FFO Mode (RUN) Clock

(SRC GLINK(LOCK)FIFOS

(OFF) RESET

FFO Return cableto SRC

GLINK INPUT from SRC

GLIN

K O

UT

to VR

BThe Blinking lights tell it all !

(Yale Mantra)


The FIB in SVXDAQ

Set FIB header(WXY)

Channel enables

GLINK ControlsDigitize/Readout

Clocks

Set Last CHIP ID

HDI <-> PipeMatching

Clock controls

MicroSequencer Controls (Don't touch)

PCB VersionECO LevelSerial No.


The FTM

● Connects the HDI's to the FIB

● Uses the CDF FIB J3 Backplane (non Bus part)

● 2 Control Cables (A/B) for 2 Wedges using LVDS

● 10 Optical Receivers (aka RX) for the DOIMs from 10 Ladders (see Sasha's Talk)

Wedge B

Wedge A

Wedge B

Wedge A

Data

CommandsClock

FTMFIB


BackEnd Kludges

● Dead OBDV in one channel● BackEnd Kludge Fix

– Take clock from Command line– Add delay (adjustable, infamous

potentiometer)– Use this to clock in the incoming data

● Requires special Pipe Firmware● And Requires “Last Chip Override” set in

the Chip Editor


FIB failure modes

● Pipeline Failures – Bit errors in one HDI

● Command Clock Failure– Two Wedges affected (Trips, corrupted data, C2C2)

● C2C2 for one ladder– Pipeline dead– May be RX problem, check FTM

● Stuck VME Bus– Fuses blown, swap the board (Usually one FPGA died)


A FIB VME Crate

● 1 VME Crate CPU● Up to 12 FIB's● 1 FFO● FIB J3 from Slot

8-20● FIB's only work

there !● J3 is fragile,

Be careful !

A FIB Crate contains :1 2 3 4 5 6 7 8 910 11 12 13 14 15 16 17 18 19 20 21

MVME 2401

FIB

FIB

FIB

FIB

FIB

FIB

FIB

FIB

FIB

FIB

FIB

FIB

FF0

FIB J3

VME SLOT

VME Crate Controller

9U


Summary

● Important to know what the FIB and FFO can do !– Helps Diagnosing problems– Stop putting all the blame on the DAQ

● The manuals cover quite an amount of detail● Many “odd” things have a not always obvious reason for

being that way● If you don't know, you should ask people ...


Backup Slides


FFO Race● In order for this thing to

happen, you'll need to have several things coming together– The SRC processes non-fatal

errors

– The FFO return cables (RS-485) are connected

– The FFO has to report an error on these error lines

– for the crates to reboot, you need to have system reset enabled on the FFO.

SRC

FFOError reporting

Clear the error

SRC is too fast for FFO


FFO Race Fix

● Make sure SRC Process Not Fatal Errors is Disabled (happens on each CONFIG)

● Error line 0-2 on all FFO's are disabled (3 is used for resonance protection and not affected)


FFO GLINK Blues

● If a FFO looses the Glink sync, an entire FIB crate is affected

● Resyncing the GLINK– Do a SRC Reset Glink -> Sync

Glink– do a VME reset of the FIB

crate– Powercycle the crate


Gray Code

● Alternative Binary Coding System● Only 1 Bit changes at

incrementing a number● Originally invented to solve

electromechanical problems● For Chips

– Less Power consumption– More Robust ADC

Decimal Binary Gray0 000 0001 001 0012 010 0113 011 0104 100 1105 101 1116 110 1017 111 100


Timing● After L1A, FIB performs DIGITIZE (fixed

time/event)– 2.26 µs setup/cleanup + 4.57 µs ADC ramp

● then does READOUT (occupancy driven)– 1.3 µs setup/cleanup + 19 ns/Byte


Command Cable Signals

● FE_CLK FrontEnd Clock (132 ns)● BE_CLK BackEnd Clock (19 ns)● C_CLK Command Clock (38 ns)● C Command 6 bits● C_DATA Command DATA● L1A Level 1 Accept● PIPE_RD2


Downloading FIB Firmware

● 2 kinds of “Firmware“– FIB Sequences ( MS RAM menu)– FIB Firmware (JAM menu)

● Checking Firmware Version– Sequences : Checksum– JAM : Use Verify


FIB special value

● For Data, the values 0xEF to FF are reserved● Most prominent is 0xF3

– Data from Chip is 0x00 (after DPS)

– FIB replaces this with 0xF3● Other error codes are 0xF1,0xF2 ● See CDF/DOC/CDF/PUBLIC/4152


Super/Duper FIB

● Replacing chips on the input fifo to increase the tolerance of varying duty cycles on OBDV coming from the SVXII chips

● Add the "ISL Mod" which increases the width of FE Clock HIGH, adding some breathing space for edges which must fall inside a FE Clock

● This upgrades the FIB to ECO level 5


SRC->FIB commands to know

● 0x00 No operation● 0x06 Digitize● 0x05 Readout● 0x0C KeepAlive● 0x02 Preamp Reset Start (XQT=1)● 0x04 Preamp Reset Stop (XQT=1)

fib & ffo - hep.ucsb.edu

Documents