epics collaboration meeting fall 2012 @ pal october 22 ~ 26, 2012 lcls-i/lcls-ii timing system low...
TRANSCRIPT
EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
LCLS-I/LCLS-II Timing SystemLow LevelKukhee Kim for LCLS Timing TeamICD Software, SLAC National Accelerator Laboratory
October 22, 2012
2EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
Event System is a most complicated system!
3EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
Accelerator complex for current and future(LCLS-I, LCLS-II and …)
LCLS
LCLS-II
End oflinac
LCLSLCLS-II
Bypass line
|Sector 0
|Sector 10
|Sector 20
|Sector 30
FEL Injector Test Facility
3 km SLAC LINAC
LTU
Undulators
Interleaved Accelerator Operation
LCLS-III ? LCLS-IOne gun, one destination
120 Hz
LCLS-IIOne gun, two destinations,
plus shared tunnel, plus multibunch?
120 Hz
Separate but synchronous timing systems
360 Hz = 6 x 60 Hz power line timeslots
Independent rate control to each destination
MPS-aware timing
TS4, and TS1
TS5 and TS2
TS6, and TS3
4EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
360Hz – constraint: What is the timeslots?
LCLS-I LCLS-ILCLS-II LCLS-IIFACET FACET
5EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
Legacy Timing System
• SLC timing system- The Linac is a Pulsed Machine (get a packet of beam per pulse) runs at
a max of 360Hz (120Hz)- Three Main Timing Signals:
• Hardware- 476MHz Master Accelerator Clock (runs down 2mile Heliax Main Drive Line cable)- 360Hz Fiducial Trigger / encoded onto the 476MHz master clock
• Software – MPG (Master Pattern Generator)- 128-Bit PNET (Pattern Network) Digital Broadcast (contains time slot, trigger setup,
beam type & rate information, beam destination)- Provide the timing pattern 3 fiducial prior
• Advantage of the Legacy system- Physicists & Operator are used to the old system- Provide event patterns 3 fiducial prior
• give a time margin to slow device, or pre-trigger up to 2 fiducial ahead
6EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
Old architecture for the LCLS-I Timing System
DEV
~ Linac main drive line
Sync/Div
SLCMPG
PNET
119 MHz 360 Hz
SLCevents
LCLSevents
PNET
mP
PDU
EVR
IOC
TTL-NIMconvert.
DigitizerLLRFBPMsToroidsCamerasWire ScannerSLC klystrons
TTL
SLC Trigs
FAN
IOC
Low Level RF
EPICS Network
Precision<10 ps
fiber distribution
LCLS Timing System components are in RED
*MicroResearch
*
*
EVG
LCLS MasterOscillator
476MHzLinac
Master Osc
System is based around the EVent Generator and EVent Receiver
FIDO PDURaw 360 Hz LCLS Timeslot Trigger
7EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
LCLS-I Timing System
• LCLS Timing System
• Use MRF EVG and EVRs• Generate event code at 360 Hz based on:
- PNET pattern input (beam code + 128 bits pattern that define beam path
and other conditions, scheduled information)- Receive MPS*) information (real-time information), and queue it up into
the pattern pipeline (pipeline bypass for quasi real-time)- Add LCLS information (MPS, BPM calibration, diagnostics and ….)
• Extend the pattern to 192 bits
- Hardwired BCS fault
• Send out timing pattern, including EPICS timestamp with encoded
pulse ID on the timing fiber• Manage user-defined Beam Synchronous Acquisition (BSA) MPS*) Please, see the talk for LCLS Machine Protection System (M. Boyes)
in hardware solution session (16:30 – 16:50 Oct 22, 2012)
8EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
How to work the EVG
• PNET message broadcast from the old timing system (following a fiducial,
360 Hz rate)
• PNET receiver in EVG IOC generates IRQ
• EVG software takes the PNET data and uses it to assign the proper event
codes (the PNET data pipelined 3 fiducial ahead)
• Event code setup in EVG one cycle head
• Next fiducial is sent: 360 Hz fiducial signal received by EVG
• The EVG begins to send out the event codes in its Sequence RAM
• The event codes get sent across the timing fiber links to the EVRs
• Inside the EVR, its mapping RAM is set to map a specific HW trigger to an
event code
• when the Event Code matches the same value in the mapping RAM, a hit is
generated and after a programmed delay, a HW trigger is output form the
EVR to the device
9EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
How to work the EVR
• Set trigger delays, pulse width, and enable/disable via user
requests
• Set event code per trigger
• Receive timing pattern ~8.3 msec (3 fiducials prior) before
corresponding pulse. Provides EPICS timestamp to record
processing
• Perform BSA based on tags which are set by EVG in the
timing pattern
• Process pre-defined records when specific event codes are
received
10EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
Timeline for the EVR
F0
0
Fiducial
Time (usec)
B0
Acq Trigger
1023
Beam Kly Standby
Record processing (event, interrupt)
Fiducial Event Received
Event Timestamp, pattern records, and BSA ready
Receive pattern for 3 pulses ahead
Hardware Triggers
~5000.3
~40100
Triggering Event Codes
Kly Accel
F1
Fiducial
…2778
110
Start End
11EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
Upgrade Timing system for both LCLS-I and LCLS-II
• Main Goals
• Remove the dependency with the SLC timing system• Independent EVG for each beam program
- LCLS-I, LCLS-II, FACET, XTA and future beam programs- Promise for independent operation- Relieves the pain from limited event number (255 limitation)
• Keep compatibility with old system, keep SLAC specific features (advantage of the
legacy system)- Beam code and timing pattern pipeline:
• For pattern aware operation
- BSA- Event Code Sequence for photon group
• Covers various platforms for EVR: RTEMS, linux, and linuxRT- LCLS-I, and LCLS-II both are going to use various hardware platforms
• VME, PMC, Standard PC, uTCA, and COM-X• Need to support linux/linuxRT for new hardware platform
12EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
Timing System Upgrade Cont’d
• Upgrade
• Develop a new hardware called VMTG: VME based Master
Trigger Generator which provides 360 Hz IRQ to drive EVG
software instead of the PNET receiver (VMTG in EVG side)• Develop new software module PABIG: Pattern Bits Generator
which mimics the PNET• Develop a new software tool to design the event/timing
pattern visually: evGUI1)
• Work on the EVR software to support linux/linuxRT2) for the
new hardware platform: uTCA, and COM-XevGUI1) Please, see the talk for LCLS timing system for pattern design , evGUI and high level (M. Zelazny) in the timing system session (15:00 – 15:20 Oct 22, 2012)Support for linux/linuxRT 2) Please, see the talk for Real-Time Performance Issues on linux/linuxRT (K. Kim) in the low level application session (14:10 – 14:30 Oct 23, 2012)
13EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
New architecture for the timing system
Device
AnalogFront End
Device
AnalogFront End
EVR
DAQ
IOC
EVR
DAQ
IOC
Linac LLRF Phase Distribution~
LCLSLocal
Oscillator
SynchronizerDivider
FiducialDetector
EV
G
Fa
nout
IOC
Timing ReferenceDistribution
360 Hz
119 MHz 360 Hz
119 MHz 119 MHz Fiber Distribution
Timing Event System Fiber Distribution
EPICS CA Network
476 MHz
Device
AnalogFront End
Hardware trigger
signal
Beam SynchronousAcquisition
BSAData Buffers
To clocked devices
To LCLS-IEVG
EVR
DAQ
IOC
MasterOscillator
MPS Link Node MPS Network
LCLS-II CD-2 DOE Review / Aug 21 - 23, 2012
14EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
VMTG
Sector 0 (LI00) VMTG – Master Trigger Generator• Provides the synchronized 360Hz trigger to the fiducial generator• Input: 360Hz AC trigger train + 476 MHz MDL RF• Ouput: 360Hz synchronized trigger• Then, the fiducial generator makes synchronization with the Master
Oscillator
Sector 20 (IN20) VMTG with the EVG – Master Trigger
Generator• Provide 360 Hz interrupts with the Timeslot counter• Provide Timeslot Synchronization• Input: 60 Hz AC trigger (AC phase C) + Synchronized 360 Hz trigger• Output: 360 Hz interrupts + timeslot counters
15EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
Disabling PNET allows VMTG control
• PNET task needs to be delayed ~ 150 usec after the PNET IRQ to wait the MPS
message.
• PNET IRQ is using HW_TIMER to make the delay• The PNET ISR implement 150 usec delay on the HW_TIMER• The callback for the HW_TIMER releases signal to execute the PNET Task.
• VMTG has configurable IRQ delay
• Configure the VMTG IRQ delay as ~570 usec• The VMTG IRQ occurs 150 usec after the PNET IRQ
Fiducial
PNET
VMTG
Unknown delay
PNET IRQ
ISR implements the timer
Timer callback releases the signalto the PNET task
PNET task
VMTG IRQ
The ISR releases the signal to the PNET task
~150 usec
~ 570 usec
MPS message arrives somewhere before the signal.
16EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
Software Stacks for the new EVG application and the PABIG
PNETVMTGAcromag DIO EVG
BCS SBI PermitBCS Beam Permit 119MHz Clock 360Hz Fiducial TriggerPNET broadcasting
Asyn module VMTG module
EVGDrv/Dev
PABIGGenVarMasterBeamControlBits 0 to 8via CA
HW input bits in PABIG EVG Application
EPICSDB
EDEF reservation for the BSA
PNBN PVs from SoftIOC
SNL program
476MHz Clock360Hz Trigger
360Hz Fiducial Trigger
2nd NIC
udpComm
mpsCommPNET Task
MPS message(beam destination &Permit)
17EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
EVR Software Stacks/ Form factor & OS dependency
VME EVR Hardware PMC EVR Hardware
mrfCommon/mrfVme64
drvErMrf
devErMrf
erRecordBSA
EVR Processing Logic
Event Module for RTEMS/vxWorks
erapi
drvLinuxEvr
devErMrf
erRecordBSA
EVR Processing Logic
Event Module for linux/linuxRT
PMC EVR Hardware
Works with old register map Works with modular register map (new)
18EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
Summary
•Successfully upgraded timing system
• Standalone from the SLC system• Same code base for LCLS-I, LCLS-II and future projects• Backward compatibility, keep the advantages from the legacy system
•Our Team has done the following work:
• Develop new VMTG hardware: John Dusatko• Develop PABIG software: Till Straumann• Develop evGUI: Mike Zelazny and Partha Natampalli• EVG software integration,
EVR software support for linux/linuxRT and real-time performance
issues: Kukhee Kim, Till Straumann, Stephanie Allison
19EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
Related Talks
• LCLS Machine Protection System (M. Boyes)
in hardware solution session (16:30 – 16:50 Oct 22, 2012)
• LCLS timing system for pattern design , evGUI and high level (M. Zelazny)
in the timing system session (15:00 – 15:20 Oct 22, 2012)
• Real-Time Performance Issues on linux/linuxRT (K. Kim)
in the low level application session (14:10 – 14:30 Oct 23, 2012)
EPICS Collaboration Meeting Fall 2012 @ PALOctober 22 ~ 26, 2012
Thank You!