-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
1/50
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
2/50
2
Agenda
Background: Substations Past and Present Ethernet Refresh Switching Refresh Sources of Latency in a Switched Ethernet IEC 61850 Core Components, Architecture &
Communication Model IEEE 802.1Q VLANs User-Priority / Classes of Service / IEEE 802.1P Example
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
3/50
3
Substation Network - Past
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
4/50
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
5/50
5
Typical Substation DiagramSub
Transmission
Line No. 1
SubTransmission
Line No. 2
PowerTransformers
Tie Breaker
Feeder Breakers
IED 4TX 1
IED 1LN 1 IED 3
TB 1
IED 2LN 2
IED 5TX 1
IED 6FDR 1
IED 7FDR 2
IED 8FDR 3
IED 9FDR 4
IED 10FDR 5
IED 11FDR 6
IED 12TB 2
IED 14BUS 2
IED 13BUS 1
Wiring requirements: CTs Total 22 x 4c cables
2 x Transmission Lines DistanceProtection
2 x HV Busbar Protection 2 x Bus Tie Busbar Protection 2 x 2 Step Down Transformers 6 x Feeder Protection 6 x LV Busbar Protection 2 x LV Tie Breaker
VTs Total 8 x 2c cables; 4 x 4c Cables 2 Single Phase Transmission Lines 2 x Three Phase HV Bus 2 x Three Phase LV Bus 6 x Single Phase LV Feeders
Circuit Breakers Control Total 14 x12c cables 2 x Transmission Lines
1 x HV Bus Tie 2 x HV Step Down Transformers 2 x LV Step Down Transformers 6 x LV Feeders 1 x LV Tie Breaker
Power Transformer Control 2 x 12cCables
Communication cables between relaysand RTU or PLC Total 14 cablesTotal: At least 50 multi-core cables fromthe HV equipment to the Control Building
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
6/50
6
Substation Network Future
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
7/50
7
Configuring the LAN for IEC 61850
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
8/50
8
Ethernet Refresh
Ethernet Initially: shared media (collisions) MAC (Media Access Control) protocol based-on CSMA/CD
(Carrier Sense Multiple Access with Collision Detection) Contention-based
Set of devices that contend for access to a segment arein the same collision domain Bus network Hub Network
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
9/50
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
10/50
10
Where Are We?
Communication between IEDs is necessary Ethernet has many advantages over serial
communications
Shared medium means less predictability: collisions
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
11/50
11
Switches Help
Switches helped by breaking-up collision domains Node connected directly to a switch, doesnt contend for access
to media No collisions, (more predictable)
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
12/50
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
13/50
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
14/50
14
Switching Refresher
1 Broadcast domain per switch (~ per VLAN) Broadcast frames are flooded out all ports (in the same VLAN) Multicast frames are flooded out all ports (in the same VLAN)
Unicast frames: flooded (if the destination MAC address is not in the MAC Data
Base) forwarded to only one port (if the destination MAC address is in
the MAC Data Base)
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
15/50
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
16/50
16
Switched Ethernet
For some applications, the network must guarantee acertain maximum latency or the application may notbehave (or fail outright)
Switched Ethernet Networks have many sources oflatency: Store & Forward, Switch Fabric, Wireline, Queuing
All of these latencies (except for queuing) aredeterministic The effects of frame queuing can also be calculated providing
one knows the nature of all sources of traffic on the network
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
17/50
17
Who Cares?
The benefits and drawbacks of Ethernet technology areimportant
IEC Chose Ethernet as the underlying technology for
61850
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
18/50
18
IEC 61850 Primer
Culmination of: Comprehensive EPRI-Project UCA 2.0 & European Experience (IEC 60870-5-101, -103, -104)
Framework for SS communication, high-level descriptionof SS automation Standards required:
Interoperability & Integration: data representation, how devicesshould look / behave to network applications
Intuitive device/data modeling/naming: Hierarchical andstructured
Fast and convenient communication Lower cost: install, configure, maintain, & also connect legacy
devices
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
19/50
19
Core Components
Object Model (info available from primary equipment &SS automation functions)
Inter-IED Message/communication format
Environmental robustness Configuration language
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
20/50
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
21/50
21
IEC61850 Substation Architecture
I E C 6 1 8 5 0
P r o c e s s
B u s
I E C 6 1 8 5 0
S t a t i o n
B u s
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
22/50
22
IEC61850 Substation Architecture
I E C 6 1 8 5 0
S t a t i o n
B u s
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
23/50
23
Ethernet PacketStation Bus
Status, Trips, Close,GOOSE / GSSE
A, V, Hz, W, Wh,Var, Varh, SOEIED Block Diagram
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
24/50
29
IEC61850 Substation Architecture
I E C 6 1 8 5 0
P r o c e s s
B u s
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
25/50
30
Ethernet PacketProcess Bus
Merging Unit Block Diagram
I/OsProcess Bus data is used byother IEDs in the networkto perform real timeprotection & control mission
critical actions
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
26/50
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
27/50
32
Communication Model
GOOSE messages Used for fast transmission of substation events, such as
commands, alarms, indications A GOOSE message sent by an IED can be received several
receiversExamples: Tripping of switchgear Providing position status of interlocking
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
28/50
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
29/50
34
Answer: Use VLANs
Question: If queuing / buffering is important to operation of
Ethernet > How to minimize any adverse effect?
Answer: Identify the relative priority of frames Increase available bandwidth, reduce cost, improve
security Using standard mechanism: IEEE (802.1P/Q > 802.1D-2004)
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
30/50
35
VLANs - Basic Facts
VLAN: a logical group of ports on a single device or across multiple switches Used construct separate virtual networks ( independent
domains that share cabling and switch infrastructure)
IEEE 802.1Q defines how to carry multiple VLANs overthe same physical link (between switches)
Broadcasts are contained to a single VLAN
Since devices in the same VLAN are in the samebroadcast domain, they can communicate without arouter (L3 device)
A Layer 3 device is required to communicate betweenVLANs
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
31/50
36
Cost Effective Solution
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
32/50
37
Relay Meter
Red, Green, andBlue VLANs are
separate networksthat share the
sameinfrastructure.
SCADASurveillance
Billing
Trunk portscarry traffic
from allVLANs onsame fiber
MPEG
IEEE 802.1Q VLANs in the Substations
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
33/50
38
Frame formatTagged v. Untagged
StandardFrame
Dest. Src. Length / Type Data6 bytes 6 bytes 2 bytes Variable
Dest. Src. Length / Type Data6 bytes 6 bytes 2 bytes Variable
TPID TCI
Priority CFI VID
2 bytes
3 bits 1 bit 12 bits
2 bytes
TaggedFrame
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
34/50
39
Edge port vs. Trunk port
R e d
B l u e
G r e e n
G r e e n
R e d
B l u e
G r e e n
R e d
B l u e
Trunk Link(Carries all 3 VLANs)
Trunk: A VLAN Trunk carries traffic for multiple VLANs overthe same physical link A VLAN tag identifies the VLAN membership of theframes on the trunk
Trunk ports send tagged frames
Edge:A VLAN Edge port carries traffic for a single VLANEdge ports send untagged frames
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
35/50
40
IEEE 802.1P Prioritization
Class of Service (CoS)
High priority traffic gets tobypass queue
Multiple egress trafficqueues: higher prioritytraffic sent first
Reduces jitter and latencyfor time-sensitive traffic(voice / GOOSE)
Shares tag header with802.1Q VLAN
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
36/50
41
GOOSE, Priority, and VLANs Different traffic flows in the
61850 substation networkmerit separate VLANs: Substation LAN management
(e.g. switches, routers,modems, etc.)
SCADA/Engineering Access GOOSE Messages Process bus (IEC 61850-9-2
sampled values) Synchrophasors
Protection A vs. Protection B Video surveillance and access
control
GOOSE should have the highest Class of Service (CoS) as per IEEE 802.1PGOOSE is faster than conventional I/O
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
37/50
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
38/50
43
Where Are We?
Communication between IEDs is necessary Ethernet has many advantages over serial
communications Shared medium means less predictability: collisions
Switches improve Ethernet performance by eliminatingcollisions
Queuing latency minimized with Classes of Service(CoS/802.1P)
CoS (priority) information carried in VLAN tag
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
39/50
44
IEC61850 Standard Multicast Addressing
The multicast (destination) addresses use following structure: First 3 Bytes assigned by IEEE: 01-0C-CD. 4 th Byte shall be: 01 for GOOSE, 02 for GSSE, and 04 for multicast
sampled values.
Last 2 Bytes to be used as individual addresses assigned by the rangedefined in Table B.1.
Source: IEC61850 Standard 61850-8-1First edition 2004-05Communication networks and systems in substations Part 8-1: Specific Communication ServiceMapping (SCSM) Mappings to MMS (ISO 9506-1 and ISO 9506-2) and to ISO/IEC 8802-3
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
40/50
45
Remember:
61850-enabled IEDs use messages to communicate GOOSE messages:
Multicast delivery (flooded by switches) So, all nodes see message (regardless of whether its required)
Want to convey priority to switch Not necessarily set VID Priority Frame :
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
41/50
46
GOOSE and Network Performance
Generic Object Oriented Substation Event (GOOSE): Very powerful IED to IED messaging mechanism Allows efficient real time exchange of any state or analog
parameter.
IEC 61850- 5 Type 1A Trip total transfer time at 3ms Transfer time = Application to Application and includes
IED times + Network delays. Worst case of total network delay is
100 s at 100MBps links speeds 10 s at 1Gbps
Small networks or for less demanding applications thedelay introduced by switches is tolerable
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
42/50
47
GOOSE and Network Performance
For larger networks of say twenty switches with 100Mbps links, a delay of 2 ms could impact the protectionscheme
GOOSE messages are priority tagged Highest Class of Service (CoS IEEE 802.1P) in the network GOOSE frames are placed in the front of the store and forward
queue Frames already being sent are not interrupted
Recommendation: Consider Gigabit links for networks running highly demanding
IEC 61850 GOOSE and Sampled Values applications
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
43/50
48
VLAN Advanced - Cost Effective SolutionSub Transmission
Line No. 1Sub Transmission
Line No. 2
PowerTransformers
Tie Breaker
Feeder Breakers
IED 4TX 1
IED1LN 1 IED 3
TB 1
IED 2LN 2
IED5TX 1
IED 6FDR 1
IED 7FDR 2
IED 8FDR 3
IED 9FDR 4
IED 10FDR 5
IED 11FDR 6
IED 12TB 2
IED 14BUS 2
IED 13
BUS 1 VLAN 10 Transmission +HV Bus Protection IEDs
VLAN 20 Distribution + LVBust Protection IEDs
VLAN 30 Bus BarProtection IEDs
VLAN 40 TransformerProtection IEDs
VLAN 50 VideoSurveillance
VLAN 1 Engineering /Operations / SCADA
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
44/50
49
VLAN Advanced - Cost Effective Solution
Sub TransmissionLine No. 1
Sub TransmissionLine No. 2
PowerTransformers
Tie Breaker
Feeder Breakers
IED 4TX 1
IED 1LN 1 IED 3
TB 1
IED 2LN 2
IED 5TX 1
IED6FDR 1
IED 7FDR 2
IED 8FDR 3
IED 9FDR 4
IED 10FDR 5
IED 11FDR 6
IED 12TB 2
IED 14BUS 2
IED 13BUS 1
VLAN 10 Transmission +HV Bus Protection IEDs
VLAN 20 Distribution + LVBust Protection IEDs
VLAN 30 Bus BarProtection IEDs
VLAN 40 TransformerProtection IEDs
VLAN 50 Video
SurveillanceVLAN 1 Engineering /Operations / SCADA
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
45/50
50
Example
IED1 LN1
IED2 LN2
IED3 TB1
IED4 TX1
IED5 TX2
IED6 FDR1
IED7 FDR2
IED8 FDR3
IED9 FDR4
IED10 FDR5
IED11 FDR6
IED12 TB2
IED13 BUS1
IED14 BUS2
SCADA
Engineering /Operations
VideoSurveillance
VLAN 10 Transmission +HV Bus Protection IEDs
VLAN 20 Distribution + LVBust Protection IEDs
VLAN 30 Bus BarProtection IEDs
VLAN 40 TransformerProtection IEDs
VLAN 50 VideoSurveillance
VLAN 1 Engineering /Operations / SCADA
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
46/50
51
IED1 LN1
IED2 LN2
IED3 TB1
IED4 TX1
IED5 TX2
IED6 FDR1
IED7 FDR2
IED8 FDR3
IED9 FDR4
IED10 FDR5
IED11 FDR6
IED12 TB2
IED13 BUS1
IED14 BUS2
SCADA
Engineering /Operations
VideoSurveillance
Network Behaviour VLAN 10 Transmission +HV Bus Protection IEDs
VLAN 20 Distribution + LVBust Protection IEDs
VLAN 30 Bus BarProtection IEDs
VLAN 40 TransformerProtection IEDs
VLAN 50 VideoSurveillance
VLAN 1 Engineering /Operations / SCADA
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
47/50
52
IED1 LN1
IED2 LN2
IED3 TB1
IED4 TX1
IED5 TX2
IED6 FDR1
IED7 FDR2
IED8 FDR3
IED9 FDR4
IED10 FDR5
IED11 FDR6
IED12 TB2
IED13 BUS1
IED14 BUS2
SCADA
Engineering /Operations
VideoSurveillance
Network Behaviour VLAN 10 Transmission +HV Bus Protection IEDs
VLAN 20 Distribution + LVBust Protection IEDs
VLAN 30 Bus BarProtection IEDs
VLAN 40 TransformerProtection IEDs
VLAN 50 VideoSurveillance
VLAN 1 Engineering /Operations / SCADA
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
48/50
53
Where Are We?
Communication between IEDs is necessary
Ethernet has many advantages over serial communications Shared medium means less predictability: collisions
Switches improve Ethernet performance by eliminating collisions Queuing latency minimized with Classes of Service (CoS/802.1P)
CoS (priority) information carried in VLAN tag VLAN configuration on a link must match (IED Switch) Trunk ports send traffic for multiple VLANs, each* identified by a
tag Edge ports send traffic for a single VLAN, and typically send
untagged frames
-
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
49/50
54
Thank You!Questions?
Ren MidenceRuggedCom Inc.
Vice President, MarketingPhone: +1 905 266 11 39Mobile: +1 647 504 58 15
E-mail: [email protected]
mailto:[email protected]:[email protected] -
8/11/2019 16h30 - PALESTRA - Aplicaes de Comunicao Em Redes - Ren Midance
50/50
References
IEC 61850 -Communication Networks and Systems inSubstations:An Overview of Computer Science , JianqingZhangand Carl A. Gunter, Illinois Security Lab
RuggedCom: The Communications Backbone for IEC61850, IEC 61850 Seminar, derived from UCA UsersGroup Meeting CIGRE 2006
DistribuTECH 2009 - Setting your network for IEC61850Traffic , Rene Midence