cisco connected utilities - utility technologyutilitytechnology.org/conference/2013... · cisco...

Cisco Connected Utilities Future Network Trends

Mike Kopczynski

October 17, 2013

© 2011 Cisco and/or its affiliates. All rights reserved. 2

Agenda

Operational Network Requirements Communications Network Trends Emerging Standards Impacts of Distributed Intelligence Securing the network


Definitions and Acronyms Switch:

DR:

Disaster Recovery Demand Response

4 © 2011 Cisco and/or its affiliates. All rights reserved.

Operations Improve System Reliability Improve Grid Inter operability Integrate renewable generation Manage aging infrastructure

Field Workers Aging Workforce Solutions Workforce mobility solutions Enterprise Social Networking

Customers Improve customer services Conservation and sustainability Customer enablement & participation Home Energy Management & EV

Legal Regulatory Compliance Governance Policy Security

Business/Financial Reduce operational expenditure Defer Capital Expense Integrate renewable generation Increase energy efficiency

Growth Control Cost

Customer

Productivity

Global Industry Transformation What Are Utilities Doing?

Workforce


Regulatory Compliance

Federal Energy Regulator Commission (FERC) Issued Notice of Proposed Rulemaking on April 18, 2013

North American Electric Reliability Corporation (NERC) Critical Infrastructure Protection Reliability Standards (CIP Version 5)

Considerations • Bulk Electric System have High, Medium, and Low impact sites • Electronic Security Perimeters • Access Control • Remote Monitoring • Non-routable protocol exclusion has been eliminated


communication networks


Key Trends Convergence Scalability Packet switched networks Open Standards IPv6

Low Latency networks


Converged Network Serves Energy Ecosystem

Demand/Capacity/Energy Management Metering, Billing, Pricing, Conservation Subscription Management, Customer Interaction Business Workflow, Enterprise Resource Planning B

usin

ess

S

ervi

ces

IP Network Monitoring/Troubleshooting Fault, Event Management Image/Configuration Management Network Performance Management Identity, Access Control, Trusted Systems

Adv. Distribution Management Systems Outage Management Systems Asset Management, GIS Demand Response Meter Data Management

IP N

etw

ork

Mea

sure

/ C

ontro

l

Service

Energy

IP Network Control

SCADA IEC 61850 1613 NERC-CIP DLMS/COSEM C12.22


Convergence of Power Grid Communications and Computing

Similar to what has already happened with consumer mobile wireless devices

Computing integrated with network communications

Unify grid level elements with control and data centers

Supports centralized, distributed, and hybrid intelligence models

Could be extended beyond utility assets as needed

http://www.cisco.com/web/strategy/docs/energy/control_architecture.pdf


Converged Management of Communication Network and Power Grid Devices

Grid devices are increasingly intelligent: embedded processors communications interfaces

Grid device management increasingly resembles communication device management

As numbers increase, managing them manually just not feasible from a scale standpoint

Some differences will continue to exist, but the value of a converged management tool set is compelling Probably need multiple user interfaces for differing users – also role-based access control and ID management Potential NERC CIP compliance issues


Mobility and the Utility Field Force

Converged networks can support voice, data, video

Enable central system and maintenance document management with field access

Real time access to grid data by field workers Example: meter ping from handheld device GIS in the field Connect OMS/DMS outage nested root cause analysis to field crews during outage restoration

Field Force Collaboration Data/document sharing Voice and video to/from field


N-Way and Cross-Tier Communications

Cross-tier control automatically implies cross-tier communication

Deep situational awareness has the same effect – see WAMS

Smarter grid devices imply increasing peer-to-peer communication


The Great Debate

Circuit Packet X TDM IP X

Switching Routing X

The answer to the question does not come from the technical strengths or weacknesses of the technologies, but on how well they supports the user service demands and the business needs.

© 2011 Cisco and/or its affiliates. All rights reserved. 14 14

Market drivers (External): “Smart Grid” funding has accelerated the deployment of new Use

Cases for the Network Legacy TDM and Serial Devices reaching EoL, multiple industries

transitioning to standards-based, IP-enabled Devices Legacy public “leased line” services going away – Frame Relay,

DDS, TDM? – SPs Migrating to IP/MPLS and Carrier Ethernet themselves

Infrastructure drivers (Internal): Flexible data rates and statistical multiplexing for efficiency Distributed N-Way communications vs. Centralized P-to-P and P-to-

MP Multi-service transport over a wide variety of Layer 1 technologies –

Fiber, Copper, Wireless, Power Line Carrier, etc... Consolidating networks onto common infrastructure to minimize

OPEX (Circuit and Management costs)

Why is Utility Transport Moving Toward Packet?


Communication Networks Implications

N-way flow of information -> IP

Number of devices involved -> IPv6

Security, QoS

Low latency Teleprotection: < 4 msec System control: < 1 cycle ( < 16 msec in North America) FAN level: (DA) < 2 cycles ( 32 msec in North America) Intra-substation: < 1 msec WAMS: < 50 msec and decreasing over time

Meter networks and MDMS are not adequate for distribution automation in the future


Reliance Upon Advanced Open Standard Protocols

Driven by need to take maximum advantage of communication networks

Extensive capabilities opened up via IPv6

Example: NASPInet MPLS and IP Multicast PIM/SSM GDOI/GETVPN Scaleability Management


standards


Several Significant Standards Sets Exist and Will Drive System Design/Interoperability

Tier Standard(s)

System Control: Transmission/Substations WAMS

IEC 61850 family*

Applications: EMS/DMS/GIS, etc

IEC Common Information Model (CIM)*

Communications Networks: Convergence at all levels

IPv6 protocol suite and packet switching

*Expect these to be harmonized; will see these and IEC 60870 at the distribution level

There are a great many standards being suggested for utilities, but a small set is emerging as crucial.


FAN Protocols Are Emerging Grid Device Protocols: Modbus, DNP3, IEC 60870-5-101, 104; IEC

61850 extensions – problematic for legacy devices

Network Protocols: extensive IP protocol set, including IPv4, IPv6, PIM-SSM, OSPF, MSDP, MLD, GRE, VRF, IGMPv2 , PTP (IEEE 1588)

FAN Endpoint Protocol Stack (AMI, DR, etc):

IEEE 802.15.4g (FSK, DSSS, OFDM)

IEEE 802.15.4e FHSS

6lowpan

IEEE P1901-2 (G3-based PLC)

IPv4 / IPv6

2G / 3G / LTE Cellular WiMAX Ethernet

TCP/UDP

SEP2.0/Web Services/EXI

IEEE 802.15.4 2.4GHz DSSS

IEEE 802.15.4e

RPL

SNMP/HTTPS/CoAP

IEC 61968 CIM C12.22

DLMS COSEM IEC 60870 DNP IEC 61850

PHY

/ MAC

Fu

nctio

nalit

y N

etw

ork

Func

tiona

lity

App.

La

yer

Com

m. N

etw

ork

Laye

r

802.1x / EAP-TLS based Access Control Solution

MODBUS

Oth

er

Com

m.

Med

ia


Device Standards

Extended Operating Temperature

No Fans - Convection cooling

Relative humidity

Input voltage

Surge withstand

RF Suseptability

Electrostatic

Shock and Vibration

IEEE 1613


distributed intelligence


The Centralized vs. Distributed Argument Distributed approach has been around for a long time

Autonomous substation concept

Some distributed solutions have existed for years Peer to peer FISR Standalone IVVR

Existing control system solutions have tended to reinforce centralized architecture

Many distribution engineers appreciate the distributed point of view, since it matches the distribution infrastructure


Compelling Values of Distributed Intelligence Low Latency Response

A distributed intelligence architecture can provide the ability to process data and provide it to the end device without a round trip back to a control center.

Scalability no single choke point for data acquisition or processing; analytics at the lower levels of a hierarchical distributed system can be processed and passed on to higher levels in the hierarchy. Such an arrangement can keep the data volumes at each level roughly constant by transforming large volumes of low level data into smaller volumes of data containing the relevant information. This also helps with managing the bursty asynchronous event message data that smart grids can generate (example: last gasp messages from meters during a momentary).

The scalability issue is not simply one of communication bottlenecking however – it is also (and perhaps more importantly) an issue of data persistence management, and a matter of processing capacity. Systems that use a central SCADA for data collection become both memory-bound and cpu-bound in a full scale smart grid environment, as do other data collection engines

Robustness Local autonomous operation

Fragmentation

Graceful system performance and functional degradation in the face of failures

Incremental rollout

Flexible platform for new apps


Problems Arising From Distributed Intelligence Device/system/application management – smart devices residing in

substations, on poles, in underground structures represent significant cost to visit. Even more so that with a PC network, it is impractical to send a person out to any of these devices to install a patch, reset a processor, or upgrade an application. Remote administration of smart devices on a power grid is necessary. This also implies remote monitoring of not just the devices themselves, but the databases and applications, along with the means to reset, patch, and upgrade remotely.

Harder to design, commission, and diagnose – distributed intelligence systems can inherently involve a larger number of interfaces and interactions than centralized systems, making design, test, and installation more complex than with centralized systems.

More complex communications architectures required – distributed intelligence may involve more peer-to-peer interaction than with centralized systems, so that the communication network must support the associated peer-to-peer communication. The resultant networks are more complicated than for a simple star, but the good news is that IP is ideal to provide the necessary flexibility.


Network Security


Cyber Security 2013+


Pervasive Architecture-Based Secure IP SolutionsImplemented Through Solutions

DEFEND

Defend Grid Operations

Securing the End-to-End Electric Power Supply Chain

Threat Defense

EXTEND PREVENT COMPLY

Achieve Regulatory Compliance

Prevent Loss of Critical Assets

Secure Utility Connectivity

Secure Mobile Workforce

Physical and Data Loss Prevention

Governance, Risk and Compliance

Mike Kopczynski [email protected]


Why MPLS to the Edge? OPEX forcing Infrastructure convergence for better utilization of fiber and

microwave assets, driving the need for virtualization MPLS enables this vitalization with advanced L2 and L3 VPN technologies.

Legacy TDM, archaic interfaces (Serial, E&M), and industry specific interfaces (C37.94 in energy) will persist for many years

MPLS supports the transport of this traffic with pseudowire based Circuit Emulation

Packet solutions involving mixed packet transport technologies (like MPLS core with Ethernet or IP Edge) are operationally complex due multiple control planes, OAM translation etc.

End-to-End MPLS across the transport infrastructure, lowers time to deploy OT & IT services by separating transport from service operations, and simplifying the operational process with single touch point service enablement and contiguous OAM and PM

Some industry specific use cases (like current differential teleprotection) require symmetric forward & return paths for time synchronized measurements

MPLS traffic engineering enables explicitly routed paths through a ECMP network

cisco connected utilities - utility technologyutilitytechnology.org/conference/2013... · cisco...

Documents