PRIV
ATE
AND
CO
NFI
DEN
TIAL
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
Auth
or –
Org
anis
atio
nal A
bbre
viat
ion
–D
ate
[Mon
th D
D, Y
YYY]
–R
ev. x
.x
The adequacy of communication diagnostics for High Speed Rail
Nicholas DiSaiaJoe Greco<Title>October 14, 2018
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
Over thirty years of experience in Train Control applications with a focus on Unattended Transit Systems and Communication Based Train Control.
Led the wayside development of Bombardier’s Communication Based Train Control System in the 1990s and holds a patent for the positioning system for a moving block system.
Team member for the development of Bombardier’s Network Radio System. For 10 years managed the CBTC software development teams for both wayside and on-board
and is currently manager of Technical Solutions in Pittsburgh.
2
Bio: Joseph A. Greco, Manager Technical Solutions Bombardier Transportation, Rail Control Solutions, USA
Bio: Nicholas DiSaia, Manager Networks and Cyber SecurityBombardier Transportation, Rail Control Solutions, USA
13 years working in both Communications Engineering and Automatic Train Supervision. Responsible for project deliveries of the wired network and radio system. Manages R&D and product development efforts for Radios, Networks, and Communications
Software. Principal Software Engineer and architect of Bombardier’s Network Monitoring System
products.
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems
33
Safety and Cyber-Security
Communication Systems in High Speed Rail
Communication Infrastructure - Wired and Wireless
Industry diagnostics for communication Systems
Enhanced diagnostics for wired and wireless systems
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
Signaling System Components
Controlled Eurobalise
Signals Track circuits
TSR serverDispatching
Interlocking
RBC
Maintenance
CTC
Point machine
OCS
GSM-R Onboard Radio
ISDN Network
GSM-R Network
Mobile Switching
Centre
CTSC-2 TSR
CTSC-3 TSRTSR
TSR
CTC
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
WuGuang DPL ATP Onboard - System Components
Balise Transmission
Module
Odometry
DMI
Balise Antenna
Controlled Eurobalise
ATP Unit Communication & Encryption
CTSC-2 Control Unit
Pulse generator
Vital/Non vital Train Interface
GSM-ROnboard Radio
Doppler Radar Pick- Up Coil
Juridical Logger
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Supervisory Control and Data Acquisition Central Command will have the ability to control the following functions:
– Signaling Operations and Passenger Communication – Includes telephones, Closed Circuit Television (CCTV), Public Addressing, Passenger Information Systems, and Emergency Passenger Communications
6
Operator Control Room Signaling Equipment Room
Catalyst 3560 SERIES
SYST
MODE
SPEEDDUPLX
POE
STAT
RPS
1X
18X
17X
16X2X
15X 31X
32X 34X
33X 47X
48X
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 481 2 3 4 5 6 7 8 9 10
1
PoE-48
3
2 4
Catalyst 3560 SERIES
SYST
MODE
SPEEDDUPLX
POE
STAT
RPS
1X
18X
17X
16X2X
15X 31X
32X 34X
33X 47X
48X
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 481 2 3 4 5 6 7 8 9 10
1
PoE-48
3
2 4
Backbone DCSFor ATC, SCADA, ECS and Voice/Video
(as required)
Other Network(as required)
CISCO ASA 5520
POWER STATUS ACTIVE VPN FLASH
Adaptive Security Appliance
SERIESCISCO ASA 5520
POWER STATUS ACTIVE VPN FLASH
Adaptive Security Appliance
SERIES
CT/TMSServer
SCADA Server
Central LAN
InternetUsing VPN Tunnel
Internet Connection (such as DSL) used during system startup and commissioning for remote diagnostics.Disconnected during Revenue Service
Central Operator Training Simulator
Color Laser Printer
COTS LAN
ATS Trainee SCADA TraineeATS Trainer SCADA Trainer
P/O SCC DCS
P/O DCS
RATOATS Simulation Servers
CT/TMS Server
Color Laser Printer
Catalyst 3560 SERIES
SYST
MODE
SPEEDDUPLX
POE
STAT
RPS
1X
2X
PoE-24
1 2
12X
11X
11 121 2 3 4 5 6 7 8 9 10
14X
13X 23X
24X
13 14 15 16 17 18 19 20 21 22 23 24
CT/TMSServer
GSD Video Wall
ATS Supervisor Security MaintenanceSCADA Supervisor
ATS Operator GSD Video Wall Controllers
SCADA Operator x2x4
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Supervisory Control and Data AcquisitionSCADA – The SCADA system provides aggregation and processing of field device
signals and alarms as well as providing the ability to send control requests to field devices.
Standard SCADA functionality may be required such as alarm evaluation, alarm acknowledgement, data logging, visualization of data as well as functionality beyond that of a typical SCADA system such as the integration of Communications functionality.
Sub-system interfaces to the SCADA system:– Power Distribution System, Primary (Traction Power) and Secondary Power
Sources– Access Control/Intrusion Detection– Fare Collection– Fire Detection, Ventilation Systems– Signaling Alarms and Events
7
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Data Acquisition & Passenger Information Onboard the Train
8
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Wired and Wireless Data Transport Systems Summary of Communication System Services A. Signaling B. Central Control System C. CCTV D. Passenger Signs and Infotainment E. PA System, Telephone System F. Intrusion Detection G. Possible WiFi Access
To supply all the services listed above, a communication Network is defined
9
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems
1010
Safety and Cyber-Security
Communication Systems in High Speed Rail
Communication Infrastructure - Wired and Wireless
Industry diagnostics for communication Systems
Enhanced diagnostics for wired and wireless systems
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Wired and Wireless Data Transport Systems Key elements in the communication network design.
– What types of networks are required?• Signaling train to wayside for High Speed Rail
• GSM-R • Tetra• GPRS• LTE
• Wayside Infrastructure • Wired network with fiber along full alignment• Leased Lines• Wireless network between wayside objects
• Train to Wayside for non-signaling functions (for ex. Operational Radio & diagnostics)• GSM – voice• Tetra• LTE• Wireless Mobile Network
11
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Wired and Wireless Data Transport Systems Functional Operation
– Single Network Architecture with all services included– Separate networks for signaling and non-signaling functions
Performance– Size of wired network– Bandwidth of wireless data from train to wayside– Quality of Service
12
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Wired and Wireless Data Transport Systems
13
Layer 3: CSC Rail Control System(Advanced TMS)
Layer 2: EBI Screen(Basic TMS)
Layer 1: INTERFLO 450 or 550 (ERTMS Level 2/Regional)
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Wired and Wireless Data Transport Systems
14
The protocol architecture according to the existing ERTMS specifications
The protocol architecture according to the future ERTMS specification.
The future is now.
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Wired and Wireless Data Transport Systems Non-signaling Services
– Passenger Information Systems– Telephone– CCTV– Worker Protection Devices– Passenger Announcements– Operational Radio
15
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems
1616
Safety and Cyber-Security
Communication Systems in High Speed Rail
Communication Infrastructure - Wired and Wireless
Industry diagnostics for communication Systems
Enhanced diagnostics for wired and wireless systems
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Industry Diagnostics for Communications
17
Overview– Industry Diagnostics for Communication Systems
• Rail Industry Comparison• Communication Failures and Root-Causes
– Enhanced Diagnostics• Integrated NMS systems• Long-term Maintenance
– Safety and Cyber Security– Current State of Affairs– Normative Standards– Risk Tolerance, Avoidance, and Maintenance
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Industry Diagnostics for Communications
Rail Industry Commercial IT NetworksDesign Life 30+ years Variable –tends to follow market trends
and favors increases in capacity.
Network Flux Rare Frequent
Feature Set Narrow Wide
Product Changes Variable
Vendors unlikely to commit to project schedules
Rare
Change Management Structured baselines/releases of configuration.
Limit untested changes to live system.
Auto-discovery of new devices. Live updates of configuration.
Many cases changes are non-critical.
Network Commissioning Strict
Online, available, and built to spec -> Good
Relatively Lenient
Online & available -> Good
Licensing Fixed or preferably none Per node, per interface, per sensor, per feature, consumption based, annual service fees, etc.
May require internet access for validation. Move toward SaaS.
18
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Industry Diagnostics for Communications
What is a Communication Failure?– Depends on who is detecting the failure...
Root Causes– Radio System Issues
• AP failure vs Client radio failure• Interference/Jamming• Roaming problems
– RSSI Issues• Antenna alignment• Degradation of mechanical connections• Inline component failure• LoS Blockage• TX Power Amplifier and Low-Noise Amplifier
– Application Issues• Software stops sending messages• CRC issue
– Network Issues• Did a device fail?• Excessive errors on port/bad connections• Change of configuration or component• Routing & VLAN’ing
– Does the failure affect every train in an area or just a signal train? Limited to 1 vehicle or multiple vehicles?
19
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems
2020
Safety and Cyber-Security
Communication Systems in High Speed Rail
Communication Infrastructure - Wired and Wireless
Industry diagnostics for communication Systems
Enhanced diagnostics for wired and wireless systems
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Enhanced Diagnostics
Integrated Network Monitoring Systems
Device Level
CBTCLevel
• Pure NMS functions• Active Polling• Traps/events• Interfacing status to other subsystems
• ATC system protocols• Deep packet inspection of protocols• Real-time & playback modes
Integrated NMS
21
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Enhanced Diagnostics
Key Features:– SNMP (v1/v2c/v3) monitoring + other protocols– Real-time analysis and playback of log files– Online/Offline troubleshooting and diagnostics for ATC, VATC, and Comms.– Normalized live data management– Redundant operation
• A/B pairing• Control Center Clustering
– User Security and Access Control– Analysis
• Statistics and charting tools• Visualization of the system• Alarm/Event filtering/sorting/preview• Heat Maps
– System commissioning tools
22
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Enhanced Diagnostics
23
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Enhanced Diagnostics
24
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Enhanced Diagnostics
Maintenance and Health Checks:
- Integration with higher-level systems for predictive and preventative maintenance• Dispatch the right people at the right time
- Remote Diagnostics• On-Demand Supplier support
- Health Checks• RF/TWC System• Network • Cyber Security • Onsite vs Remote Health Checks
25
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems
2626
Safety and Cyber-Security
Communication Systems in High Speed Rail
Communication Infrastructure - Wired and Wireless
Industry diagnostics for communication Systems
Enhanced diagnostics for wired and wireless systems
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Safety & Security
27Source: https://ics-cert.us-cert.gov/sites/default/files/Annual_Reports/Year_in_Review_FY2015_Final_S508C.pdf
23% of ICS Incidents have happened in the
transportation Sector (USA)
Transportation Sector – ICS-CERT 2015
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Safety & Security
28
REASONS
• Today Industrial Systems are monitored and controlled by IT technologies operating in open networks
• Establishing standard protocols make a wider range of devices vulnerable
• Coexistence of Legacy and New systems
• Utilization of commercial of-the-shelf-products, but lack of awareness to establish the necessary security tools and software updates
• Wireless and cellular communication
Why Rail Infrastructures became more vulnerable
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Safety & Security
29
Attack Vectors and Intrusion Routes
6. Command & Control Centers
5. Rail Control & Signaling
(ERTMS & CBTC)
4. Wayside Wireless
Infrastructure (Track, Stations)
1. Train to wayside communication
2. TCMS network / car to car
3. Passenger WIFI AccessServices
7. Maintenance, Fleet
Management
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Safety & Security
30
Typical Standards- ISO 27001- APTA SS-CCS-RP001-10- APTA SS-CCS-RP-002-13- NIST 800 series- EN 50159- IEC 62443
Methods for Compliance- Physical Security Measures- Firewalls/NIDS/HIDS- Network segmentation, isolation, and ACLs- Policies and Procedures- Monitoring, reporting, and identification of problems- Security Server functions- Centralized authentication strategies- Password policies- Encrypted links
Source: APTA-SS-CCS-P-002-13: Securing Control and Communications Systems in Rail Transit EnvironmentsAPTA: American Public Transportation Association
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
High Speed Rail Train Control and Communication Systems – Safety & Security
31
Risk Tolerance and Continued Improvement
What is secure today will not be secure tomorrow!
- Threat and Vulnerability Assessment- Security Log- Principle of Least Privilege (PoLP)- Cyber Security Health Check & Pen Testing
© B
omba
rdie
r Inc
. or i
ts s
ubsi
diar
ies.
All
right
s re
serv
ed.
Thank You!
Questions?
32 CITYFLO, INTERFLO and EBI are trademarks of Bombardier Inc. or its subsidiaries.