atlas dcs elmb prr, cern, march 2002fernando varela elmb networks can/canopen interoperability of...
DESCRIPTION
ATLAS DCS ELMB PRR, CERN, March 2002Fernando Varela Aims of the Test: TileCal Cooling and LV: 8 ELMBs Pixel Cooling: 7 ELMBs COMPASS 10 ELMBS Currently Pixel Cooling : 12 ELMBs MDT 10 ELMBs COMPASS 16 ELMBs Next Year 5000 ELMBs in ATLAS – 1200 MDT (test case, should cover all subdetectors) – 1000 RPC – 800 TGC ATLAS Powering Performance: tuning of parameters identifying bottlenecks 50 mm 66 mm Reliability Robustness Definition of recovery procedures Usage of the ELMB >> Give guidelines and recommendations to set up ELMB networks in ATLASTRANSCRIPT
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
ELMB NetworksELMB Networks
• CAN/CANopen Interoperability of the ELMB• Usage of the ELMB in ATLAS• ELMB Networks• Full Branch Test• Conclusions • Future work
Outline
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
Software Interface Communication LayerSofting CANopen OPC Server CAN-AC2-PCI CANopenBeckhoff TwinCat SoftPLC FC5120 CAN & CANopenVector CANalyzer CAN-AC2-PCI CAN & CANopen
Custom CANopen OPC Server NICAN-II CAN & CANopen
120 120
Beckhoff Wago
Compliance of the ELMBCompliance of the ELMB
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
Aims of the Test:
• TileCal Cooling and LV: 8 ELMBs• Pixel Cooling: 7 ELMBs• COMPASS 10 ELMBS
Currently
• Pixel Cooling : 12 ELMBs• MDT 10 ELMBs• COMPASS 16 ELMBs
Next Year
• 5000 ELMBs in ATLAS– 1200 MDT (test case, should cover all subdetectors)– 1000 RPC– 800 TGC
ATLAS
• Powering• Performance:
• tuning of parameters• identifying bottlenecks
50 mm
66
mm
• Reliability• Robustness• Definition of recovery procedures
Usage of the ELMBUsage of the ELMB
>> Give guidelines and recommendations to set up ELMB networks in ATLAS <<
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
• The maximum number of ELMBs per bus is 64 for addressing reasons (6 bits), i.e. 4096 Analog Inputs• However, the number of ELMBs per branch must be a compromise:
– Cost; a higher number of ELMB modules per bus => a lower number of buses and PCs– Affordable number of channels to be lost in case of a branch failure.– Performance and reliability of the readout system.– Powering of the CAN network (PS placed in USA15).
120 120
120 120Branch_1
Branch_2
ELMB NetworksELMB Networks
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
SCX
1U
SA15
Front-End SystemB
ack-End System
Local Area Network
Partition 3Supervisory only Partition m
UX
15
Partition 2
Common Infrastructure
Local Control Stations (LCS)
Partition 1
Expert Workstations
ServerOperation
Subdetector 1 Subdetector 2a Subdetector n
DetectorSub-system
SensorsActuators
Fieldbus 200m
ELMBElectronicRack
ELMBELMB
ELMB
Rack PC
Power Supply
CoolingInterlock
Box
Test
Rack PC
ELMB
Alarms
CFS
HVBarrel
Architecture of DCSArchitecture of DCS
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
ELMB Full Branch TestELMB Full Branch Test
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
ELMBELMB Full Branch Test
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
ELMBELMB Full Branch Test
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
Set UpSet Up
• Only one PS used.• Digital and CAN parts of the ELMB and NI-CAN card powered via the bus.• Analogue part powered from the digital (Power monitored by a scope). Normally powered from the FE equipment• Bus and Interface card power lines were de-coupled (Allows for independent reset of both elements).
120
0x3F 0x30
CANbus 200 m
16 V PS
LCS
120
D9 Connectors
Powering
Cable Resistance 37.5 Ohm / Km 7.5 Ohm in 200mVoltages
Currents (mA) VPS_CAN (V) 16CAN 16x20 320 Velmb_16_CAN (V) 11.05ADC 16x10 160 VPS_Dig+Ana (V) 16Analogue 16x15 240 Velmb_16_Dig+Ana (V) 8.26
GroundingDrop G_CAN (V) 2.63 Drop DP+AP (V) 3.88Drop P_CAN (V) 2.31 Drop DG+AG (V) 3.88Difference (V) 0.32 Difference (V) 0
Digital + Analogue Voltage drop during ADC activity (V) 1.44
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
• 16 x 8 = 128 digital input lines (Sync + Async), 16 x 2 x 8 = 256 digital output lines (Async)• Input and output lines interconnected• 16 x 64 = 1024 Analogue channels (Sync)• ELMBs reprogrammed to bypass filtering in OPC server and ensure maximum data volume transfer to PVSS-II (Worst possible case)
ELMB I/O FunctionalityELMB I/O Functionality
t
SYN
C_1
ELM
B_3
0
ELM
B_3
1
ELM
B_3
F
ELM
B_3
0_0
ELM
B_3
1_0
ELM
B_3
F_0
SYN
C_2
ELM
B_3
0
SYNC Interval
...
...
ELM
B_3
0_1
ELM
B_3
1_1
...
Digital Signals(Higher Priority)
Analogue Channels(Lower Priority)
ELM
B_3
F_1
ELM
B_3
F_63
Bus Cycle
...
If adc = 32.5 Hz and Bus Speed = 125 kbit/s• ~ 0.8 ms => Time difference between messages on the bus; ~ 30.8 ms => Time needed for ADC conversion
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
CAN Analyzer Diagnostic ToolCAN Analyzer Diagnostic Tool
SYNC
DigitalSignals
AnalogueChannels
Bus load
No. of CAN frames
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
Bus LoadBus Load
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
Sync Interval vs adc for 16 ELMBs @ 125 kbits/sTransfer Rate 100%, best case
0
5
10
15
20
25
30
35
40
0 5 10 15 20 25 30 35adc (Hz)
SYN
C In
terv
al (s
)
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
SYNC IntervalBus PeriodBus Period - Sync Interval
Time diff Bus Period - SYNC Interval (s)
PerformancePerformance
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
Data get lost at higher data rates when SYNC Interval -> bus cycle
Bottlenecks:• NI CAN interface card (buffer overflow) => Need for a new interface (cost and characteristics).• PVSS archiving (CPU consumption) => Better distribution of the tasks performed by SCADA.
Transfer Rate Bus - PVSS-II vs Sync Intervaladc=32.5 Hz, OPC Update Rate 1 s, 125 kbits/s
60
65
70
75
80
85
90
95
100
105
02468101214161820222426283032
Sync Interval (s)
Tran
sfer
Rat
e (%
)
Performance (II)Performance (II)
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
Comparison of the bus load peak at different bus speeds
0
5
10
15
20
25
30
0 5 10 15 20 25 30 35adc (Hz)
Bus
Loa
d Pe
ak (%
)
125 kbits/s250 kbits/s
Comparison at different Bus SpeedsComparison at different Bus Speeds
Results shown at 125 kbits/s were reproduced, i.e. 100 % transfer rate for a bus having 16 ELMBs and SYNC Interval equal to 4 s.
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
• 200 m long ELMB Branch with up to 16 nodes successfully operated.
• I/O channels of the order of magnitude of some subdetectors in ATLAS: 1024 AI, 256 DO, 128 DI
• Preliminary studies of the powering has been carried out.
• The test tried to reproduce the “worst possible case” => All ELMBs transmitting at the same time.
• The system has shown excellent performance:Transfer rate of 100% for
• messages of different priorities on the bus
• and of different transmission types.
• Maximum readout rate 4s at 32.5 Hz.
• Results are reproducible at 250 kbits/s => Less bus load (but also smaller bus length).
• Several issues were identified at higher adc when SYNC Bus Cycle
• Overflows in the read buffer of the NI-CAN interface
• Strange behavior of PVSS-II manager as a function of the OPC update rate.
• High CPU consumption by the PVSS archiving manager
However, the CANalyzer showed the correct operation of all ELMBs on the bus.
ConclusionsConclusions
Fernando Varela
ATLASDCS
ELMB PRR, CERN, March 2002
• Optimize powering of the bus => minimize voltage drop in the CANbus cable.
• Window-checking in the ELMB.
• Increase number of ELMBs up to 32 or 64.
• Several buses per PC.
• Test of network supervision and management -> Definition of error recovery procedures
• Long-term operability of the system in TCC2.
• Automatic power cycle of the bus needed due to SEEs (see below).
Future WorkFuture Work