teleprotectionover an ip/mpls network - etsi an ip/mpls network technical validation ... nd of 3 ms,...
TRANSCRIPT
Teleprotection over an IP/MPLS
Network
Technical validationTechnical validation
ir. Lieven LEVRAU
Alcatel-Lucent IP Division
April 4th, 2011
• IP/MPLS-based Utility Networks
• Challenges for IP/MPLS in teleprotection
• Other design considerations
• Conclusion
© Alcatel-Lucent 2011 All Rights Reserved
IP/MPLS-based Utility Networks1
All Rights Reserved © Alcatel-Lucent © 2011All Rights Reserved © Alcatel-Lucent © 2011
IP/MPLS-based Utility Networks1
IP/MPLS-Based Utility Network
5620 SAM SCADAOmni PCX
NOC
Power Generation
7750 SR
Corporate HQ / NOC
IED
CollaborationServers
Internet
IP/MPLSNetwork
7750 SR
LAN
OS6855
7705 SAR
All Rights Reserved © Alcatel-Lucent © 20114 | MPLS Network for Teleprotection | Sep 2009
NOC
Transmission
7705 SAR
CollaborationTool
Servers
SubstationSubstation
Network
VideoSurveillance
7705 SAR
TPR
LANRTU
7750 SR
VideoSurveillance
TDMLANTPR
IED
7705 SAR
7750 SR
OS6855
7705 SAR
How IP/MPLS Address the challenges of the Utility Networks?
Reliability
Traffic Isolation
Multiple Legacy technologies
High Availability, Fast Reroute, end-to-end,…
IP/MPLS based - L2-VPNs, VLL, L3-VPNs, PWE
FR, ATM, TDM, Serial, analog voice, synchronous, …
Traffic isolation, Rtng protocols authentication, L2/L3
All Rights Reserved © Alcatel-Lucent © 2011
Security
Unified management
Quality of Service `
Traffic isolation, Rtng protocols authentication, L2/L3
Encryption + dynamic secure Key distribution, DoS
Guaranteed bandwidth, low latency and jitter
Enhanced QoS on different network layers
End-to-end secure management of network and
services
Challenges for IP/MPLS in 2
All Rights Reserved © Alcatel-Lucent © 2011All Rights Reserved © Alcatel-Lucent © 2011
Challenges for IP/MPLS in
teleprotection2
Using MPLS Network for Teleprotection
∆t
kV
7750 SR
IP/MPLS
TPR
7705 SAR 7710 SR
E1/T1
Substation
TPR
7705 SAR
E&MRS-232
E1/T1
Substation
G.703G.703
Ethernet
E&MRS-232
Ethernet
MUXMUX
All Rights Reserved © Alcatel-Lucent © 20117 | MPLS Network for Teleprotection | Sep 2009
Teleprotection relay (TPR) signals must be transferred reliably and fast with low latency
End to end delay = telecom network latency + teleprotection equipment delay
� TDM over MPLS for legacy support (requires integration of legacy interface in MPLS node
to limit and control end to end delay)
� VPLS for IEC 61850 (requires high reliable Layer 2 transport)
Main Challenges for Teleprotection
End to End Delay
Jitter
Asymmetry
Depending on vendor equipmentMay include packetisation/depacketisation
Variation of delay in certain circumstances such as
traffic burst on network
Delay variations between transmit and receive
All Rights Reserved © Alcatel-Lucent © 2011
Resiliency
Denial of Services
Synchronization
Quality of Service
The impact of a failure in the network shouldn’t be noticed by application
TDM applications need end to end synchronization
How DoS/DDoS can affect applications
Guaranteed bandwidth, low latency and jitter
Enhanced QoS on different network layers
End-to-End TDM Transport Model (only left-to-right direction shown)
Packet SwitchedNetwork(PSN)
DS1LIU
DS1LIU
DS1 Data
Sig
PacketizationGigE GigE Data
Sig
Jitter Buffer
PacketizationNetwork
• Fixed delayPlayout
• TDM PW packets are
DS1
AccessCircuit
AccessCircuit
TDM Packets moving in this direction
All Rights Reserved © Alcatel-Lucent © 20119 | MPLS Network for Teleprotection | Sep 2009
• As TDM traffic from the Access Circuit (AC) is received, it is packetized and transmitted into the PSN
• Two modes of operation:
• CESoPSN (RFC5086) for structured nxDS0/64k channels
• SAToP (RFC4553) for unstructured T1
• Fixed delay
• Packet transfer delay based on link speeds and distances from end to end
• Variable delay
• the number of and type of switches
• queuing point in the switches
• QoS is key to ensure effective service delivery
• TDM PW packets are received from the PSN and stored into its associated configurable jitter buffer
• Play-out of the TDM data back into the AC when it’s at least 50% full
Using MPLS Network for Teleprotection
∆t
kV
7750 SR
IP/MPLS
TPR
7705 SAR 7750 SR
E1/T1
Substation
TPR
7705 SAR
E&MRS-232
E1/T1
Substation
G.703G.703
Ethernet
E&MRS-232
Ethernet
MUXMUX
Latency
All Rights Reserved © Alcatel-Lucent © 2011
TPR relay signals must be transferred in < ∆t
∆T = 1 cycle at 50Hz or 60Hz = 20ms or 16ms
End to end delay = telecom network latency
+ teleprotection equipment time
10 | Teleprotection over IP/MPLS - Validation - March2011
latency = packetization delay + network
delay + jitter buffer delay
The total end-to-end latency is calculated by summing the packetization delay (PD), network delay (ND) and jitter buffer delay (JBD) as shown here:
Total Latency = PD + ND + JBD
– e.g. PD of 2 ms (16 T1 frames/packet), ND of 3 ms, JBD of 4 ms
– Total Latency = 2 + 3 + 4 = 9 ms
Using MPLS Network for TeleprotectionLatency
∆t
kV
7750 SR
IP/MPLS
TPR
7705 SAR 7750 SR
E1/T1
Substation
TPR
7705 SAR
E&MRS-232
E1/T1
Substation
G.703G.703
Ethernet
E&MRS-232
Ethernet
MUXMUX
All Rights Reserved © Alcatel-Lucent © 2011
TPR relay signals must be transferred in < ∆t
∆T = 1 cycle at 50Hz or 60Hz = 20ms or 16ms
End to end delay = telecom network latency + teleprotection equipment time
Telecom network latency = packetization delay + network delay + jitter buffer delay
11 | Teleprotection over IP/MPLS - Validation - March2011
Calculation of Latency for TeleprotectionLatency
Latency is mainly at the edge where low speeds are present (Serial / E1 / 100 FX)
Latency in the core depends on number of nodes but mainly transmission delays
Each node adds a maximum of :
� 150µs (eqpt latency)
All Rights Reserved © Alcatel-Lucent © 2011
� 150µs (eqpt latency)
� 10µs (transmission of 1500 Bytes over a Gigabit link)
� 3µs / km : speed of light transmission over fiber
Example of a connexion between 2 TPRs :
� Over 1000km – traversing 10 nodes : 4ms
12 | Teleprotection over IP/MPLS - Validation - March2011
Jitter
� Jitter in normal operations may come from the equipments themselves
� This is minimal (less than 1% of router latency) and compensated by jitter buffer.
� Jitter happens in non normal conditions such as congestion.
� This can be solved by applying the correct QoS parameters to the node.
� The implementation should allow total control of the bandwidth required per application, minimizing jitter.`
All Rights Reserved © Alcatel-Lucent © 201113 | Teleprotection over IP/MPLS - Validation - March2011
TPR
Rate Limit TPR to 100Kbps
Rate Limit CCTV to 6Mbps
Ingress
Make Sure that no application can go higher than expected bandwidth. Via Rate limit per SAP
Egress
Ensure that TPR application always gets the priority
TPR gets to EF class
CCTV to BE Class
Per SAP policing Per interface queuing
H-QoS
Impact of failure
� Today’s teleprotection applications were developed towards SDH/PDH 50ms
failover time.
� The impact of a failure (node or link) can have huge impact in case the failure
exceeds the 50ms.
� MPLS FRR technology allows 50ms failover time in any failure scenario.
� These 50ms apply only in the core of MPLS network, but not in the case of the
All Rights Reserved © Alcatel-Lucent © 2011
� These 50ms apply only in the core of MPLS network, but not in the case of the
Access switch failing.
14 | Teleprotection over IP/MPLS - Validation - March2011
Conclusion on network failure impact
� Failover in the backbone can be limited to 50ms with FRR
� FRR has to be implemented in the first node to minimize failure risks.
� All ALU Service routers support FRR.
� Integrating TDM in MPLS (as in 7705 SAR) brings the FRR to the application and
limits the failure risks (compared to a 2 box solution)
� Providing extended rapid failover scenarios may be key in many applications.
All Rights Reserved © Alcatel-Lucent © 2011
� Providing extended rapid failover scenarios may be key in many applications.
� Support very rapid convergence time even in case of non direct connectivity
(BFD), for example with Microwave links
� Implementing MC-LAG, G.8032 and BGP-MH for IEC 61850 greatly enhances
total availability of the applications.
3
All Rights Reserved © Alcatel-Lucent © 2011All Rights Reserved © Alcatel-Lucent © 2011
Other design considerations for Teleprotection
16 | Teleprotection over IP/MPLS - Validation - March2011
3
Packet flow asymmetry
LSPs are unidirectional
Topology changes or mis-configuration may result in different path being used in both directions.
Only trouble shooting tools may discover that.
5620 CPAM allows this detection and raises alarms.Bypass Green and
All Rights Reserved © Alcatel-Lucent © 201117 | Teleprotection over IP/MPLS - Validation - March2011
Active
Path
Bypass
Logical
Link
actual
path
Tracking of LSP status and history
Red LSP
follow
different
Path
Tracking of LSP Path for a given service
Synchronization
Information need to be delivered with time precision from an application standpoint.
� Many TDM applications require clock synchronization
(as SDH)
� Some applications require Time of Day type of
synchronization
All Rights Reserved © Alcatel-Lucent © 2011
Packet networks can deliver clocking through different techniques :
18 | Teleprotection over IP/MPLS - Validation - March2011
Summary4
All Rights Reserved © Alcatel-Lucent © 2011All Rights Reserved © Alcatel-Lucent © 2011
Summary4
Conclusion
Teleprotection is the most stringent application that
can be transported over networks due to :
� Low delays requirements, very low jitter requirements
� Impact of a failure in the application
Alcatel-Lucent demonstrates that its IP/MPLS can be used as
the next generation network for such applications with :
All Rights Reserved © Alcatel-Lucent © 2011
the next generation network for such applications with :
� Native TDM integration (incl. interfaces, synchronization, …)
� Very high resistance to potential failures in network
� Several management tools to anticipate, control and trouble
shoot network
20 | Teleprotection over IP/MPLS - Validation - March2011
IP/MPLS is the foundation to prepare the Smart Grid data explosion, and manage the transition of existing mission critical applications