the use of labview fpga in accelerator...
TRANSCRIPT
The Use of LabVIEW
FPGA in Accelerator
Instrumentation.
Willem Blokland
Research Accelerator Division
Spallation Neutron Source
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Introduction
Spallation Neutron Source at Oak Ridge National Laboratory:
$1.4B facility using a 1 GeV proton beam to generate pulsed neutrons
Built by a collaboration of five national laboratories
Neutron scattering to study materials
2
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Introduction
The Beam Instrumentation Group at the Spallation Neutron Source is responsible for measuring various attributes of the particle beam, such as losses, electrical charge, and position.
Many instruments can and are implemented with a general OS (XPe). However, some measurements must be done consistenly and quickly (16ms) to protect the machine or may require very fast reaction times (10 µs)
We began using FPGA on NI Hardware as well as LabVIEW FPGA for various instruments:
Beam Accounting systems that use FPGA and LabVIEW but was directly programmed with the Xilinx tools (fast charge calculator and differential monitor)
A Beam Loss Monitor that was programmed in LabVIEW FPGA on cRIO
A new version of the Beam Accounting system, using FlexRIO and LabVIEW FPGA
3
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Spallation Neutron Source Beam Pulse
At SNS:
The beam pulse repeats at 60 Hz
Beam is chopped at ~1 MHz
RF is 402.5 Mhz and 805 MHz
4
mini-pulse
Chopper system makes gaps
16.6 ms
Cu
rren
t1ms
Accumulated in
the Ring
945 ns
Cu
rre
nt
1 ms macro-pulse
2.5 ns
Cu
rre
nt
Cu
rre
nt
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Accelerator Beam Instrumentation Features
5
400+ PCs
16+ VME systems
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Accelerator Beam Instrumentation Features
A beam instrument’s function can be one of the following:
Measurements to a studier: Is my accelerator setup better or worse?
• Only needed during studies
• Interactive (1s)
Measurements to an operator: Is the accelerator operating within specifications?
• 24/7, Needed during production
• Interactive (1s) and/or automatically monitored
Measurements for beam accounting: What are the statistics?
• 24/7, Needed during production
• Automatic (<16.6ms)
Measurements to the abort system: Is something going wrong?
• 24/7, Needed during production
• Automatic either with alarm or direct hardware connection to abort system (e.g. abort beam in less than 20µs)
6
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Example of Beam Instruments
SNS Wire Scanner (PC, LabVIEW)
Steps a wire through the beam
Tied to a max beam rep rate of 1Hz to protect wire
Only used when performing a study
Single failures allowed
Beam Position (PC, LabVIEW)
Measures the position of the beam pulse
Only needed at 1Hz (beam rep rate is 60Hz)
Digitizes at 40Mhz
Single failures allowed
Beam Loss Monitors (VME, VxWorks)
Measures beam losses to protect machine
Software must run at 60Hz
Analog hardware to abort beam within 20 µs
No failures allowed (exceptions can be made)
7
Fork
BPM Plates
Neutron detectors
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Accelerator Beam Instrumentation Types
8
Machine
Downtime
µs s
Now and
then
Always
Response time
Mostly
Consequences
Damage
Time lost
ms
Win, Mac, Linux,…
VxWorks,
LVRT, PLC, …
Circuits, FPGA,
DSP, …..
Uptime
Hardware
Real-Time
Non RT
standalone
Non RT
interactive
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
How Do We Implement An Instrument?
We are interested in the fastest and cheapest way to implement a reliable instrument
Small group can’t afford to know everything
• Budget can be used to buy time from contractor with needed expertise
• Work with other groups within division
Have to work with different parts from different vendors
• Real-time OS, Compilers, FPGA Board, Digitizer, Slot-zero controller, Backplane, Amplifiers
Maintenance
Solutions
If deployment is large enough and time is available, then learning curve is not as significant and in-house hardware can be developed
Buy turn-key system
Buy off-the-shelf parts and integrate to complete
9
Considerations
Cost
Available personnel
Development time
Reliability
Performance
Obsolescence
Maintenance
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Use of FPGA in Accelerator Instrumentation
These are unique instruments, one or two copies of each at most. We have used LabVIEW FPGA for various small projects, such as:
Laser position feedback loop
Pulse delay generator
Accelerator Instruments:
Pulse-by-Pulse Beam Charge Monitor (PBCM)Calculate the charge of a pulse and transmit digital value over optical fiber within 2-3 ms to Target Hall Instruments
Differential Beam Current Monitor (DBCM)Subtract the signals of two current transformers and determine if the difference is too large (5, 50, 500 µs sliding windows). Abort beam within 5 µs.
For these instruments LabVIEW FPGA was not available with a fast enough digitizer. We choose to use COTS parts (NI PCI-5122EX), but involve a contractor (Maverick Systems) to write the Xilinx FPGA code.
10
Beam pulse
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
PBCM Setup
11
PXI crate with 5122EX
PC with EPICS and PCI Timing card
To target
instruments
Toroid in tunnel
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
PBCM Singal Processing
12
The FPGA on PXI version ofdigitizer calculates independentlyfrom LabVIEW and OS the chargeand beam pulse edge.Processing of each channel includes:• Baseline correction,• FIR/IIR filter or no filter or test pattern,• Charge integration, and• Detection of beam pulse edge.
Fiber-optic Transmission includes:• Transmission of the charge and begin
of beam pulse over two digital outputs,and
• Electrical to optical conversion (Opticalconverter box by R. Riedel) to transmitvalue to the Target Group forcorrelation with neutronmeasurements.
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
PBCM Use For Commissioning
13
To target
instruments
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
DBCM Setup
14
PXI with EPICS and 5122EX card
Sliding window traces
Beam with 5% beam loss (zoomed)
Chopped Beam
Red trace is
differenceDifferential Beam Current Monitor
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Flow Chart Of DBCM Signal Processing
15
The FPGA on PXI version of
digitizer calculates
independently from LabVIEW
and OS the difference between
two current monitorsProcessing of each channel
includes:
• Baseline correction,
• FIR/IIR filter or no filter or test
pattern,
• Charge integration (5, 50, 50 µs)
Abort includes:
• Digital Out to Machine Protection
System
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Experiences with 5122EX (PBCM CBCM each)
Contractor
9-12 months (not full time)
Labor cost ~$85k
Learning Curve
Taken care of by contractor
Hardware
$17k
Specification
2 weeks (within group: no cost)
Integration to Control System
2-3 weeks (within group: no cost)
Upgrades
Any change would require outside help as expertise is not within group -> high cost and time overhead
16
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Beam Loss Monitor
One unique instrument for use in a test facility. Similar to a standard Beam Loss Monitor, but resources to modify existing systems were scarce. cRIO was identified as viable alternative.
RF Test Facility Beam Loss Monitor Specifications
Sample 12 channels at 100kHz
• Baseline correction, calibration, and integration
Alarm in 20usec when integrated beam loss is above threshold
Control and read back HV power supplies voltages
Send test pulse to test Photo Multiplier Tubes
17
Feature VME cRIO
Sample rate 100kS/s 100kS/s
Resolution 24 16
Alarm Analog FPGA
Analog Channels 16 12-16
Hardware Cost* $29k $12k-13k* CPU, Crate, Power supply,
ADC, DAC, DIO
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Beam Loss Monitor cRIO
18
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Beam Loss Monitor Diagram
19
Impressive:
Adding test feature required minimal code and development time
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Beam Loss Monitor cRIO
20
BLM IOC CPU with EPICS
BLM Amplifiers
BLM cRIO
HV Power supply
RF Cavity Video System PXI crate
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Beam Loss Monitor VME Rack
21
Analog front-end
VME Controller,
Digitizer, Digital IO,
Custom Timing Card
Custom Analog Card
Abort System Modules
Power Supply
HV modules
Analog Front-end
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Experiences with cRIO-based BLM
Contractor
3 months part-time (to get started)
Labor cost ~$7.5k, also covers LabVIEW RT setup
Learning Curve
Sample-by-sample processing versus standard
Setting up LabVIEW FPGA
Hardware
$12k.
Specification
1 week (within group: no cost)
Integration
1 week (within group: no cost)
Upgrades
Easily done for small changes ~1 day
22
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Comparison
BLM VME programming versus LabVIEW FPGA & RT
Sample-by-sample processing fitted requirements very well and got rid of issues such as
• Sequential processing bottlenecks: Waveform acquisition takes 2 ms, then the transfer to slot zero, then processing. This must fit within 16.6msec. Balancing act between digitizing speed, backplane data transfers and CPU processing. FPGA can process data sample-by-sample without data transfers.
• Separate hardware board to perform Abort functions. An analog board with integrator and analog threshold is used to implement an abort within 20 µs. FPGA can set alarm in each 10 µs cycle.
cRIO Hardware integration takes care of low-level drivers
• Our project requirements matched very well with cRIO capabilities and no FPGA hardware knowledge was required.
VME approach is standard in accelerators
• Custom timing decoding card and software is available
• Many groups (Controls) have experienced people
• Floating point math
• Multi-year project (multiple labs)
23
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
FlexRIO
FlexRIO is now available and the Differential Current Monitor can now be implemented with LabVIEW FPGA. The goal is to upgrade the DCM (change trigger behavior and simplify interface) and improve maintenance but stay with a 100Mhz single cycle loop (aka 1000x faster).
The implementer is a student in the Physics Bachelors Program with some LabVIEW background.
NI provided with a proof of principle example program bringing us 90% to completion (Of course, the last 10% is at least 90% of the work)
NI Support during development
24
Courtesy of Mariano Padilla
Integration Windows
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Experiences Using FlexRIO at 100MHz
Student
2.5 Man-Months in 4 months, about 0.5 MM to complete
Labor cost <$10k
Learning Curve (~60%)
Point-by-point processing versus standard
FlexRIO details
Hardware
$14k.
Specification
1 week (within group: no cost but time)
Integration
Estimated at 3 weeks (within group: no cost but time)
Upgrades
Easily done for small changes (~1 day)
25
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
FlexRIO Experience
26
Experiences with 100MHz single cycle loop:
Detailed knowledge of LabVIEW FPGA implementation is required:Number of pipeline delays
Fixed point math and exact conversion details
Detailed knowledge of FlexRIO hardware is required:Being able to estimate most likely utilization of FPGA resources
Knowing how DDR memory is interfaced: how many channels/timing
No Xilinx expert required
Iterate many times to complete project
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
FlexRIO Experience
27
When performance requirements fit well below the maximum performance of what LabVIEW FPGA and hardware supports, you can expect fast development with a good hiding of the underlying technology.
When challenging the performance, development becomes slower and more complex.
BUT
It is an integrated development environment by one vendor
Can cover at least up to 100Mhz
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Comparison
28
Xilinx cRIO FlexRIO
Processing Speed 100 MHz 100 kHz 100 MHz
Project Length 12 Months 3 Months 6 Months
(Learning)
Expertise VHDL
FPGA DSP
Hardware
FPGA DSP FPGA DSP
Hardware
Labor/Hardware
Cost ratio
5:1 1:1 1:1 (Student)
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX, 29
Future LabVIEW FPGA Based Diagnostics
Instrumentation
Systems with 100kS/s – 1MS/s, reaction within 100 µs, rep rate of 60 Hz:
BLM
Charge/Power Measurements
Systems with > 1MS/s, reaction within 10 µs, rep rate of 60 Hz:
Pulse-by-Pulse BCM
Long Term:
Wire Scanners
Harp
Image Analysis
Damper (1GHz)
OAK RIDGE NATIONAL LABORATORY
U. S. DEPARTMENT OF ENERGY
NI Week 2010, Aug 2nd 2010, Austin, TX,
Conclusion
30
LabVIEW FPGA has enabled us to cover a wider range of instrumentation
developed and supported within our group
LabVIEW FPGA FlexRIO
LabVIEW FPGA cRIO
Machine
Downtime
µs s
Now and
then
Always
Response time
Mostly
Consequences
Damage
Time lost
ms
Win, Mac, Linux,…
VxWorks,
LVRT, PLC, …
Circuits, FPGA,
DSP, …..
Uptime
Hardware
Real-Time
Non RT
standalone
Non RT
interactive