miguel correia control and data acquisition group ipfn-ist

Instituto de Plasmas e Fusão NuclearInstituto Superior TécnicoLisbon, Portugalhttp://www.ipfn.ist.utl.pt

M.Correia| Lisboa, May 24th 2010 | RT2010

ATCA/xTCA-based hardware for Control and ATCA/xTCA-based hardware for Control and Data Acquisition on Nuclear Fusion fast control Data Acquisition on Nuclear Fusion fast control

plant systemsplant systems

Miguel CorreiaControl and Data Acquisition GroupIPFN-IST

Author’s name | Place, Month xx, 2007 | EventM.Correia| Lisboa, May 24th 2010 | RT20102

CDAQ in FusionCDAQ in Fusion

Applications of CDAQ in modern Fusion plant systems•Equipment integrity and interlock •System state monitoring •Recording system control actions •Tokamak system operations control •Preparing plasma discharge•Ensuring plasma discharge quality•Record system performance•Human safety/security to enable control of the plant system


Requirements for fast CDAQRequirements for fast CDAQ

Example of requirements for tokamak vertical stabilization of plasma

Measurement resolution •ENOB = 16 bit (ideally 18 bit).

Sampling rate:•20 kSample/s , 200k Sample/s for micro-instabilities detection•200 kSample/s as normal operation mode•Dynamic 20/200kSample/s with data decimation, Event distribution network required

Loop cycle latency (general rule for all PCS diagnostics)•Maximum of ~10% of the characteristic time of the phenomenon to be controlled.

To implement a prototype of a Fast Controller for Vertical stabilization the system shall have 150 acquisition channels. (with redundancy)


Need for MIMO controlNeed for MIMO control

•Some actuators are driven by controllers using data from several different diagnostics.

•Other (secondary control) diagnostics serve as backup in case of failure of the primary.

•Many variables are shared between several controllers.

•MIMO required – full-mesh support between boards and low-latency deterministic links between systems for global variable sharing.



Challenges presented by modern Fusion experiments to the CDAQ subsystem•increasing number of interdependent parameters to be controlled •Increasingly faster loop-cycle response

Implications to the CDAQ•Massive processing power (parallel, multi-processing support) •High bandwidth for data-transfer•Advanced, intelligent, flexible timing & syncronization•Real-time multi-input-multi-output (MIMO) control

Steady-state operation for subsystems (including CDAQ) demands•High-availability•Failure prevention•Redundancy



Engineering point-of-view•Modularity•Expandability•Programmability•Compatibility with high-performing industry standards•Multi-purpose •Integrate further developed hardware•Adopting commercial-off-the-shelf (COTS) products


ATCAATCA

ATCA already successfully delivered valid solutions for MIMO CDAQ •Configurable network topologies•Supports various high-performance serial gigabit protocols•Multiple processor support

ATCA also provides important non-functional features, catering for the performance and security demands of such complex subsystems•hardware management•hot-swapping•fail-safe•large area form-factor and front-panel•large power dissipation capabilities •n+1 redundancy

Obstacles (hardware development)•Complexity – lenghty development•ATCA (as yet) still undefined for Physics/instrumentation applications


xTCA for PhysicsxTCA for Physics

xTCA – specification based on ATCA, specialized for Physics

Standardizes and facilitates hardware development for device operation in a Fusion control plant environment •Extensive, hot-swappable use of the Rear Transition Module (RTM) panel, for input-output (IO) HA

•Concise data port assignment•Improved signals for timing and synchronization(with compatibility between ATCA and AMC timing specs)Zone 3 (RTM) connectors specification


Hardware design overview Hardware design overview

•builds upon previous ATCA implementations (e.g. For JET and COMPASS)

•AMC carrier cards support

•Targeting compliance with xTCA specification

•Purpose: develop base prototypes for multiple aplications.

•Focus on MIMO control – high-bandwidth, low-latency communications networks (e.g. PCIe, SRIO, hardware assisted GbE)


Target boards and networksTarget boards and networks

3 types of boards 3 communications networks

•PCIe-hubs PCIe network (dual-star)•Timing & Synchronization-hubs T&S network (dual-star)•Node blades Node blade network (full-mesh)


PCIe networkPCIe network

PCIe network•PCIe-hub (redundant set)•Dual-star (slots 1 and 2) •Handles data from every PCIe endpoint, in every blade of the shelf•4 lanes (×4 PCIe Gen 1 or Gen 2)


PCIe data hubPCIe data hub

PEX 8696 (PCIe switch)• 96 PCIe Gen 2 (5 GT/s) lanes• up to 24 flexible ports• up to 8 upstream ports• on-chip NT port for dual-host

and fail-over applications• hot-plug support• 13 ×4 PCIe →ATCA fabric ch.

(PCIe network)• 4 ×4 PCIe → AMC • 1 ×16, ×8 PCIe → RTM• 1 ×4 PCIe → FPGA

Virtex-6 LXT FPGA• 128000 logic cells• up to 600 user IO ports • up to 20 6.6 Gb/s GTX transceivers• ATCA/AMC/ARTM clk switch• 5 ×1 SRIO → AMC/RTM• 1 ×4 PCIe → PEX8696


T&S networkT&S network

Timing & Synchronization network•T&S-hub (redundant set)•Dual-star (slots 3 and 4) •distributes a customized group of timing and synchronization signals to all Node-blades.

• LVDS clock/gate/trigger signals• Deterministic Gb timing link • 4 lanes (×1 SRIO / ×3 LVDS)


Timing & Synchronization hubTiming & Synchronization hub

PEX 8696• 2 ×4 PCIe →PCIe network• 4 ×4 PCIe → AMC • 1 ×16, ×8 PCIe → RTM• 1 ×4 PCIe → FPGA

Virtex-6 LXT FPGA• 128000 logic cells• up to 600 user IO ports • up to 20 6.6 Gb/s GTX

transceivers• ATCA/AMC/ARTM clock switch• 5 ×1 SRIO → AMC/RTM• 1 ×4 PCIe → PEX8696• 11 ×3 LVDS → T&S network• 11 ×1 SRIO → T&S network


Node network (full-mesh)Node network (full-mesh)

Node blade network•P2P between every Node card•Full-mesh (slots 5 to 14)•Each port has ×1 SRIO / ×3 LVDS•allows, in conjunction with the PCIe and T&S networks, to the CDAQ subsystem to achieve real-time MIMO connectivity to every endpoint.


Node-bladeNode-blade

PEX 8696• 2 ×4 PCIe →PCIe network• 4 ×4 PCIe → AMC • 1 ×16, ×8 PCIe → RTM• 1 ×4 PCIe → FPGA

Virtex-6 LXT FPGA• 128000 logic cells• up to 600 user IO ports • up to 20 6.6 Gb/s GTX

transceivers• ATCA/AMC/ARTM clock switch• 5 ×1 SRIO → AMC/RTM• 1 ×4 PCIe → PEX8696

• (9+2) ×3 LVDS → Node/T&S• (9+2) ×1 SRIO → Node/T&S


T&S-hub / Node-bladeT&S-hub / Node-blade

2 boards-types• one PCB• one BOM• same assembly process

• one DIFFERENCE: FIRMWARE

Virtex-6 LXT FPGA• ATCA/AMC/ARTM clock switch• 5 ×1 SRIO → AMC/RTM• 1 ×4 PCIe → PEX8696• 11 ×3 LVDS → T&S network • 11 ×1 SRIO → T&S network

(T&S-hub) or (Node blade)• (9+2) ×3 LVDS → Node/T&S• (9+2) ×1 SRIO → Node/T&S


Overall ATCA fabric network setupOverall ATCA fabric network setup

Resulting ATCA fabric interface configuration, denoting the implemented network topologies and respective fabric channel connections.


Examples of useExamples of use

IO

IO

Events

FPGA

Events

FPGA

Node

TRG

IO

IO

Clk GEN

Clk GEN

IO

PCIe

CPU

storage

REFCLK

REFCLK

T&S-hub

IO

storage

PCIe

CPU

Clk GEN

IO

PCIe

CPU

PCIe-hub PCIe-hub

PCIe

Over

cable

Clock

sync

IO

DSP

Clock

sync

T&S-hub


SummarySummary

•A hardware architecture solution for fast control plant systems was presented.

•It is based upon a development of the ATCA specification, the most promising architecture to substantially enhance the performance and capability of existing standard systems, which has already delivered very encouraging results.

• The hardware designed consists of 3 different prototypes.

•Quad-AMC carrier format provides flexible, modular, expandable CDAQ. xTCA compliancy provides diverse IO solutions (e.g. IO just on RTM for improved hot-swapping capabilities), clock and trigger signaling and parallel processing, making it suitable for real-time MIMO control.

•Being designed according to the currently available xTCA specifications, this hardware aims to offer a set of base prototypes which can perform highly on many future CDAQ applications, in a Fusion control plant environment.

miguel correia control and data acquisition group ipfn-ist

Documents