DATA MANAGEMENT & ANALYTICSFOR UTILITIES 2014In-depth brie�ng

AuthorStephen Witt

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES | 2www.smartgridupdate.com

DisclaimerThe information and opinions in this document were prepared by Smart Grid Update (FC Business Intelligence) and its partners. FC Business Intelligence has no obligation to tell you when opinions or information in this document change. Smart Grid Update makes every effort to use reliable, comprehensive information, but we make no representation that it is accurate or complete. In no event shall Smart Grid Update (FC Business Intelligence) and its partners be liable for any damages, losses, expenses, loss of data, loss of opportunity or profit caused by the use of the material or contents of this document.

No part of this document may be distributed, resold, copied or adapted without Smart Grid Update's prior written permission.

© FC Business Intelligence Ltd ® 2014

AuthorStephen Witt

Antanina KapchonavaResearch manager

About Smart Grid Update

Smart Grid Update is a research-driven news, market analysis, online networking portal and conference producer. Our work focuses on three core areas of the smart energy technology sector:

■ Providing business intelligence in all areas of smart energy.

■ Building smart energy communities for all key players to enable the exchange and sharing of ideas.

■ Producing conferences that assemble senior management, decision makers, and innovators to produce results for the smart energy initiative.

DATA

MA

NA

GEM

ENT FO

R UTILITIES

DATA MANAGEMENT & ANALYTICSFOR UTILITIES 2014In-depth brie�ng

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES | 3www.smartgridupdate.com

Acknowledgments

Smart Grid Update wishes to thank the following people and organizations for their help in compiling this briefing:

ACK

NO

WLED

GM

ENTS

John D. McDonaldDirector, Technical Strategy and Policy DevelopmentGE Digital Energy

James HorstmanManager, Enterprise ArchitectureSouthern California Edison

Cheri WarrenVice President Asset ManagementNational Grid

Samuel HarrelIndustry Director, Utilities North AmericaOracle

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES | 4www.smartgridupdate.com

INTRO

DU

CTIO

N

Introduction

In the real-time and evolving world of the utility industry, the implementation and integration of smart technology and its corresponding data management and analytics is no longer a question of why, but of when and how.

Industry executives clearly understand the paradigm shift this new technology brings in generating, maintaining and distributing electric, gas and water, as well as rede�ning the industry’s relationship with consumers. Conversely, executives must weigh the costs of this smart technology deployment and rollout against a return on investments (ROI) while balancing regulatory uncertainty and new cyber security concerns.

Nevertheless the exponential deployment of Advanced Metering Infrastructure (AMI) and intelligent supervisory control and data acquisitions (SCADA) systems requires immediate decision making on which vendors to utilize as many customized systems are not supported by original vendors. This involves high risk associated with breaking open and integrating the applications or systems for modi�cation.

The AMI, SCADA and related system rollout also challenges the utility industry’s traditional organizational siloed functioning practices. Smart grid solutions increasingly blend utility operating technology (OT) and utility information technology (IT) as well as a re-evaluation of roles in multiple departments.

As the industry passes from the initial technology adoption phase into the mass implementation phase, utilities are increasingly focusing on strategic interdependence and synergies of the big data coming from new information sources.

This requires new models of data management including the movement away from siloed storage and access amid new cyber security concerns. It also calls for a renewed focus on analytics to breakdown big data into descriptive, predictive and prescriptive subsets.

This brie�ng addresses the evolving role that big data analytics plays in the utility industry including its use of vendors in providing soft grid systems, and how the application of data analytics o�ers new solutions to better drive both e�ciency and reliability.

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES | 5www.smartgridupdate.com

1. Big Data Sources

MA

RKET O

VERV

IEW

One of the keys to long-term return on investment (ROI) on smart grid deployment is managing the big data assets associated with the smart grid roll out. This data comes from an increasing number of new and enhanced internal enabling technologies including:

■ Advanced Metering Infrastructure (AMI)

■ Meter Data Management Systems (MDMS)

■ Outage Management Systems (OMS)

■ Distribution Management Systems (DMS)

■ Enterprise Asset Management Systems (EAS)

Big data assets are also coming from third-party sources such as o�-grid data sets coming from various sources such as pricing details for demand response and forecasting information for renewable energy. Other big data comes from additional software and service vendors for asset management of additional smart devices and associated operating systems.

Source: Oracle Analytics Presentation 2013

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES | 6www.smartgridupdate.com

MA

RKET O

VERV

IEW

The rise of big data poses the challenge of where to put the data and what to do with it. The answer is in the creation of a dedicated data center, of which there are several options, all of which have their pros and cons.

Some of these options include: ■ Data center facilities – These facilities are dedicated, custom-built housing and are usually customer-owned or leased from telecommunications or IT service providers. While these facilities usually have some form of contingencies for disaster recovery, they often increase the levels of capital expenditures.

■ Storage arrays – These provide data storage functionality and the current main vendors are EMC Corporation, NetApp and IBM.

■ Server platforms – This is computing hardware needed for data, and the Main vendors are HP, IBM and Dell.

■ Storage area network equipment – This connects server, storage and external network resources, and the main vendors are Brocade, Cisco and QLogic.

■ Database systems – This is software systems for data management and analysis. The main vendors are Oracle, IBM and Microsoft.

■ Virtualization systems – These systems allow more e�cient use of discrete storage and computing resources. The main vendors are VMware, Citrix and Microsoft.

Corresponding to the storage of big data are the analytics options in interpreting the data. Some of these options include:

■ Develop systems in house – The bene�t to this is the system is highly tailored to the utility’s need. On the downside there is often a lack of in-house development skills and resources to create a fully-�edged a data analytics system from scratch.

■ Rely on OT vendor system – The advantage of relying on OT vendors for data analytics is that these companies are well versed in the issues facing utilities and can therefore provide products that are highly relevant to the industry. On the downside the analytics and IT integration capabilities might not align well with IT vendors.

■ Rely on IT vendor systems – The advantages are a good integration with IT systems with good design and support. On the downside there may be an increased cost for unnecessary features and a lack of alignment with the operational side.

■ Rely on point products or pure-play vendor system. This produces close alignment with operational processes and good integration with OT, but there’s a potential lack of IT system integration and lifetime support concerns.

■ Rely on third-party data analytics service provider system. There is a reduced cost and improved analytics, but it is an evolving concept with a limited track record and data sharing issues.

Along with assuring a ROI in smart grid deployment through big data management and analytics systems, this new digital paradigm also requires creating a new organizational foundation for data and analytics initiatives through a workplace culture change.

This starts with de�ning data ownership and responsibility. According to the Data Governance Institute, data doesn’t belong to individuals, but is an asset to the enterprise. As such some organizations assign “owners” to data making individuals or preferably teams responsible for data subsets utilized for operational e�ciency.

Organizational culture also entails the reduction of departmental segregation and the need to balance out disconnects between operational and IT departments.

“This must be done by the utility organization itself,” says John D. McDonald, Director, Technical Strategy and Policy Development at GE Digital Energy. “One suggestion is for the operational department to provide several of its people to the IT department to work for one to two years. Likewise, IT does the same to operations. In this way, each department will have a better understanding of the other department, and have close friendships in the other department.”

McDonald also recommends the utility should have a knowledge management system in place to capture this knowledge from workforce retirees to0 allow for incorporation of better standard engineering designs.

In the past, utilities tended to focus business practices on a single functional area, but in today’s integrated systems, these processes cross several boundaries to create synergies. Thus, these synergies challenge the utility’s traditionally siloed functional organization as the line between operations technology (OT) and utility information technology (IT) blurs. McDonald recommends utility management discuss the need to work holistically across siloed groups. This includes having the CEO “strongly encourage” or mandate holistic work. “Make sure the project managers for the Smart Grid projects have the responsibility and multi-discipline authority (across siloed groups). With the implementation of new technology there will be revision of existing business processes and changes in organizational structure and new skill sets needed.”

Also crucial to organizational change is the utility company’s ability to assess what data to collect. In order to do this management must �rst identify the business groups (asset management, power quality, maintenance, etc.) within the utility. Second, each business group must identify their data needs – both operational and non-operational data. This includes attributes of each data point – such as how often the data point is needed and frequency of update. Then operational data levels of importance are assigned to di�erent scan groups (di�erent scan rates).

Maximizing the value of smart grid systems involves bringing disparate data under a uni�ed management approach. Operations mangers must determine what data to collect, how to manage it, who should have access to speci�c data and how that data is used and stored.

This kind of organizational change demands job redesign, training and new skill development.

But just as legacy roles and responsibilities have to be modi�ed, so do legacy systems. This presents its own integration challenges. Many legacy systems were not

designed to manage signi�cant data loads or speci�c attributes that are needed to gain desired operational bene�ts, while at the same time maintaining ongoing reliable operations.

Many customized systems are no longer supported by the original vendors. This makes for high risk associated with breaking open the applications or systems for modi�cation.

Southern California Edison Manager of Enterprise Architecture James Horstman said this has not been a major issue for the utility. “Up until a few years ago the majority of our business applications were internally developed and supported. While we have gone more to a COTS (commercial o�-the-shelf ) approach the vendors we have worked with are all supported by the vendors,” he said.

Horstman did allow there were a few instances of smaller applications losing support but those were typically replaced without signi�cant risks coming to bear. “Typically our contracts with vendors include escrow provisions for providing the software to us in the event of non support. I can only think of one instance where that actually resulted in us taking over the software and it was not a critical application,” he said.

Another utility executive said his company looks at this challenge on a case-to-case basis. “You accept the fact and either change process to match the new version or accept the current version for all its issues,” he said.

In total, changing organizational culture into a data driven utility is crucial for utilizing the big data analytics to make a utility run smoother which translates into a larger ROI.

According to GTM research the cumulative spending power on utility data analytics is expected to top $18 billion over the next six years with an annual spend of $3.8 billion globally by 2020. The report also forecasts U.S. utilities alone will spend nearly $100 per home in grid operations and consumer-related analytics over the next six years.

The biggest growth market for vendors in utilities is on analytics software (soft grid) that supply systems that allow utilities to track, visualize and predict everything from grid operations to energy consumption. This market includes the biggest names in IT, who are positioning themselves as premiere vendors, but several seminal start-up and smaller companies are also o�ering custom speci�c analytic solutions.

In selecting a vendor, the best place to start, according to McDonald, is in evaluating the utility’s needs. “Identify the missing technology components needed for an integrated solution, and consider companies who have this technology. Frequently, these are start-up companies,” he said.

Other evaluation methods for selecting vendors include:

■ Get an overview of the latest big data solutions and plug in how they would work for the utility.

■ Assess the cost and risk factors associated with smart grid and big data investments and match them with proposed vendor solutions

■ Put together an RFP (Request for proposals) to select vendors to address the scope of the utilities’ needs and have vendors respond with solution plans and costs including possible cost overruns.

Utilities are unique in that the nature of harnessing, storing, supplying and distributing electric, gas and water is critical to the infrastructure of the entire globe.

Therefore, the use of analytics demands levels of reliability and safety far greater than typical personal, business and industrial requirements.

The three analytic subsets often used within the utility industry are:

■ Descriptive analytics that gauge current performance■ Predictive analytics that tell utilities what’s about to go wrong ■ Prescriptive analytics that point to problem prevention.

Analytics in utilities are used to improve reliability in business operations by preventing re-work, improving outage statistics like System Average Interruption Frequency Index (SAIFI), replacing overloaded transformers before they cause outages and identifying equipment malfunctions and cost trends.

Some �elds where analytics improve reliability include:

■ Predictive analytics are utilized for the maintenance and modernization of an aging infrastructure and for improved visibility across automated systems

■ Descriptive analytics leverage real time data for smart control to enable situational awareness in operations

■ Prescriptive analytics identify gaps in existing assets and establish sound asset management practices and programs.

Analytics are also used to drive e�ciency by mining data for common indicators of problems like slow meters, high usage, or failing feeders. This includes using sophisticated algorithms to identify, for instance, the causes of load ine�ciencies or missed opportunities for conservation program outreach. Analytics also creates e�ciencies by:

■ Utilizing predictive customer analytics for successful rollout of demand/supply programs

■ Predicting demand and supply, which reduces outages by geographic area with higher e�ciency

■ Hold down energy costs and o�er enhanced services through smart integrated control systems

■ Boost consumer engagement through new communications initiatives related to Smart Grid programs

■ Leverage data to improve relationship with customers through direct marketing programs and new tailored services

National Grid Vice President of Asset Management Cheri Warren said the introduction and availability of both real time and detailed operational performance information introduces opportunities to understand the state of the utility’s system and its performance at a lower level of the network than they have previously had available.

“National Grid will leverage this information in several ways. Through more detailed short term outage information, the company can better identify potential infrastructure issues or better understand customer experience and causes. By collecting this big data for multiple periods or seasons, long term asset performance will greatly improve as the data will inform the performance, health, network design, asset policy and peak needs of the network and identify investments that will enhance the resiliency and reliability,” Warren said.

Warren added that National Grid has been exploring integrating other operational and customer data from other source systems into a common "data fusion" database. “Bringing together information such as customer relationship management, asset performance, work management and smart grid data will produce new insights to help drive improved asset management policies, asset planning practices and customer experience,” she said.

Warren said smart grid technology allows utilities to gather information about system performance at a �ner level of accuracy. “Using communication and location capabilities National Grid will be better positioned to ensure the accuracy of asset registers and gather very detailed network operation measurements. Using these and other pieces of information, National Grid will be able to enhance our network modeling tools and techniques, moving away from estimates for data models, and allowing us to more precisely identify asset underperformance issues in existing assets to further focus investments. Making smarter investment decisions will help drive increased reliability, performance and power quality.”

Protecting critical assets and ensuring compliance is long a part of standard operating procedure for utilities. However, new challenges have arisen with the new game-changing smart technology initiatives and heightened public awareness of privacy concerns. And as data threats multiply, so do compliance requirements. Ever changing legislative and regulatory activity related to data protection is at an all time high from both federal and state governments as well as from public utility commissions about the privacy implications of smart meters.

Some key elements of e�ective privacy are:

■ Understand your company’s compliance and culture

■ Align and train management and sta� on security practices

■ Know your data, where it is, and what must be protected

■ Ensure third parties comply with your privacy policies

■ Understand your threats and controls

■ Test and update controls regularly

■ Be prepared to respond to incidents

Additionally, utilities must keep up with compliance standards relating to security as set by the North American Electric Reliability Corporation (NERC), which acts at the direction of the Federal Energy Regulatory Commission (FERC) - a federal organization overseeing interstate transportation and marketing of energy.

NERC is responsible for establishing reliability standards for the Bulk Electric System (aka the power grid), which has become an increasingly tempting target for cyber terrorism and nation level threats. NERC's Critical Infrastructure Protection (CIP) Standards identify the minimum cyber controls and protections which power suppliers and generators must address or face signi�cant penalties and �nes.

The NERC CIP compliance standards are organized into nine (9) key areas:

■ Sabotage

■ Critical Asset Identi�cation

■ Security Management Controls

■ Personnel and Training

■ Electronic Security Perimeter

■ Physical Security Protection

■ Systems Security Management

■ Incident Reporting and Response Planning

■ Recovery Plans

Smart technology and its associated big data management and analytics have transformed the industry with many avenues to ensure a growing ROI. This new technology has streamlined and improved techniques for distributing safe and clean energy for the global marketplace.

This technology has also created a growing cyber terrorism risk for enemies that understand all too well that the world depends and functions on utility infrastructure. It has also created challenges for the utility industry to safeguard privacy while at the same time continuing its mission of providing safe electricity, gas and water.

The complexity of delivering services on a mass scale has made utilities slow to change in keeping pace with new technologies, but both consumers in the marketplace and government, through compliance mandates and regulations, dictate that smart grids are necessary for sustained energy needs. In facing these challenges, utilities realize it means not only deploying and integrating smart technology, but changing organizational structure.

Data analytics has brought a new precision to understanding and improving operational e�ciency and reliability. Whether developed in-house or increasingly through soft grid vendors o�ering various solutions, data analytics has shifted organizational culture as it continues to create synergies and bring disparate data under a uni�ed management approach.

The future of data analytics continues to evolve as utility companies explore new uses such as developing a common “data fusion” database integrating other operational and customer data from other source systems into a common system.

The role of data analytics in the utility industry is now moving towards similar roles it plays in the telecommunications and �nance sectors. Due to the industry’s key role in global infrastructure it is critical to continue developing new uses and strategies for this cutting-edge technology.

2. Data Management

The rise of big data poses the challenge of where to put the data and what to do with it. The answer is in the creation of a dedicated data center, of which there are several options, all of which have their pros and cons.

Some of these options include: ■ Data center facilities – These facilities are dedicated, custom-built housing and are usually customer-owned or leased from telecommunications or IT service providers. While these facilities usually have some form of contingencies for disaster recovery, they often increase the levels of capital expenditures.

■ Storage arrays – These provide data storage functionality and the current main vendors are EMC Corporation, NetApp and IBM.

■ Server platforms – This is computing hardware needed for data, and the Main vendors are HP, IBM and Dell.

■ Storage area network equipment – This connects server, storage and external network resources, and the main vendors are Brocade, Cisco and QLogic.

■ Database systems – This is software systems for data management and analysis. The main vendors are Oracle, IBM and Microsoft.

■ Virtualization systems – These systems allow more e�cient use of discrete storage and computing resources. The main vendors are VMware, Citrix and Microsoft.

Corresponding to the storage of big data are the analytics options in interpreting the data. Some of these options include:

■ Develop systems in house – The bene�t to this is the system is highly tailored to the utility’s need. On the downside there is often a lack of in-house development skills and resources to create a fully-�edged a data analytics system from scratch.

■ Rely on OT vendor system – The advantage of relying on OT vendors for data analytics is that these companies are well versed in the issues facing utilities and can therefore provide products that are highly relevant to the industry. On the downside the analytics and IT integration capabilities might not align well with IT vendors.

■ Rely on IT vendor systems – The advantages are a good integration with IT systems with good design and support. On the downside there may be an increased cost for unnecessary features and a lack of alignment with the operational side.

■ Rely on point products or pure-play vendor system. This produces close alignment with operational processes and good integration with OT, but there’s a potential lack of IT system integration and lifetime support concerns.

■ Rely on third-party data analytics service provider system. There is a reduced cost and improved analytics, but it is an evolving concept with a limited track record and data sharing issues.

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES | 7www.smartgridupdate.com

MA

RKET O

VERV

IEW

Along with assuring a ROI in smart grid deployment through big data management and analytics systems, this new digital paradigm also requires creating a new organizational foundation for data and analytics initiatives through a workplace culture change.

This starts with de�ning data ownership and responsibility. According to the Data Governance Institute, data doesn’t belong to individuals, but is an asset to the enterprise. As such some organizations assign “owners” to data making individuals or preferably teams responsible for data subsets utilized for operational e�ciency.

Organizational culture also entails the reduction of departmental segregation and the need to balance out disconnects between operational and IT departments.

“This must be done by the utility organization itself,” says John D. McDonald, Director, Technical Strategy and Policy Development at GE Digital Energy. “One suggestion is for the operational department to provide several of its people to the IT department to work for one to two years. Likewise, IT does the same to operations. In this way, each department will have a better understanding of the other department, and have close friendships in the other department.”

McDonald also recommends the utility should have a knowledge management system in place to capture this knowledge from workforce retirees to0 allow for incorporation of better standard engineering designs.

In the past, utilities tended to focus business practices on a single functional area, but in today’s integrated systems, these processes cross several boundaries to create synergies. Thus, these synergies challenge the utility’s traditionally siloed functional organization as the line between operations technology (OT) and utility information technology (IT) blurs. McDonald recommends utility management discuss the need to work holistically across siloed groups. This includes having the CEO “strongly encourage” or mandate holistic work. “Make sure the project managers for the Smart Grid projects have the responsibility and multi-discipline authority (across siloed groups). With the implementation of new technology there will be revision of existing business processes and changes in organizational structure and new skill sets needed.”

Also crucial to organizational change is the utility company’s ability to assess what data to collect. In order to do this management must �rst identify the business groups (asset management, power quality, maintenance, etc.) within the utility. Second, each business group must identify their data needs – both operational and non-operational data. This includes attributes of each data point – such as how often the data point is needed and frequency of update. Then operational data levels of importance are assigned to di�erent scan groups (di�erent scan rates).

Maximizing the value of smart grid systems involves bringing disparate data under a uni�ed management approach. Operations mangers must determine what data to collect, how to manage it, who should have access to speci�c data and how that data is used and stored.

This kind of organizational change demands job redesign, training and new skill development.

But just as legacy roles and responsibilities have to be modi�ed, so do legacy systems. This presents its own integration challenges. Many legacy systems were not

designed to manage signi�cant data loads or speci�c attributes that are needed to gain desired operational bene�ts, while at the same time maintaining ongoing reliable operations.

Many customized systems are no longer supported by the original vendors. This makes for high risk associated with breaking open the applications or systems for modi�cation.

Southern California Edison Manager of Enterprise Architecture James Horstman said this has not been a major issue for the utility. “Up until a few years ago the majority of our business applications were internally developed and supported. While we have gone more to a COTS (commercial o�-the-shelf ) approach the vendors we have worked with are all supported by the vendors,” he said.

Horstman did allow there were a few instances of smaller applications losing support but those were typically replaced without signi�cant risks coming to bear. “Typically our contracts with vendors include escrow provisions for providing the software to us in the event of non support. I can only think of one instance where that actually resulted in us taking over the software and it was not a critical application,” he said.

Another utility executive said his company looks at this challenge on a case-to-case basis. “You accept the fact and either change process to match the new version or accept the current version for all its issues,” he said.

In total, changing organizational culture into a data driven utility is crucial for utilizing the big data analytics to make a utility run smoother which translates into a larger ROI.

According to GTM research the cumulative spending power on utility data analytics is expected to top $18 billion over the next six years with an annual spend of $3.8 billion globally by 2020. The report also forecasts U.S. utilities alone will spend nearly $100 per home in grid operations and consumer-related analytics over the next six years.

The biggest growth market for vendors in utilities is on analytics software (soft grid) that supply systems that allow utilities to track, visualize and predict everything from grid operations to energy consumption. This market includes the biggest names in IT, who are positioning themselves as premiere vendors, but several seminal start-up and smaller companies are also o�ering custom speci�c analytic solutions.

In selecting a vendor, the best place to start, according to McDonald, is in evaluating the utility’s needs. “Identify the missing technology components needed for an integrated solution, and consider companies who have this technology. Frequently, these are start-up companies,” he said.

Other evaluation methods for selecting vendors include:

■ Get an overview of the latest big data solutions and plug in how they would work for the utility.

■ Assess the cost and risk factors associated with smart grid and big data investments and match them with proposed vendor solutions

■ Put together an RFP (Request for proposals) to select vendors to address the scope of the utilities’ needs and have vendors respond with solution plans and costs including possible cost overruns.

Utilities are unique in that the nature of harnessing, storing, supplying and distributing electric, gas and water is critical to the infrastructure of the entire globe.

Therefore, the use of analytics demands levels of reliability and safety far greater than typical personal, business and industrial requirements.

The three analytic subsets often used within the utility industry are:

■ Descriptive analytics that gauge current performance■ Predictive analytics that tell utilities what’s about to go wrong ■ Prescriptive analytics that point to problem prevention.

Analytics in utilities are used to improve reliability in business operations by preventing re-work, improving outage statistics like System Average Interruption Frequency Index (SAIFI), replacing overloaded transformers before they cause outages and identifying equipment malfunctions and cost trends.

Some �elds where analytics improve reliability include:

■ Predictive analytics are utilized for the maintenance and modernization of an aging infrastructure and for improved visibility across automated systems

■ Descriptive analytics leverage real time data for smart control to enable situational awareness in operations

■ Prescriptive analytics identify gaps in existing assets and establish sound asset management practices and programs.

Analytics are also used to drive e�ciency by mining data for common indicators of problems like slow meters, high usage, or failing feeders. This includes using sophisticated algorithms to identify, for instance, the causes of load ine�ciencies or missed opportunities for conservation program outreach. Analytics also creates e�ciencies by:

■ Utilizing predictive customer analytics for successful rollout of demand/supply programs

■ Predicting demand and supply, which reduces outages by geographic area with higher e�ciency

■ Hold down energy costs and o�er enhanced services through smart integrated control systems

■ Boost consumer engagement through new communications initiatives related to Smart Grid programs

■ Leverage data to improve relationship with customers through direct marketing programs and new tailored services

National Grid Vice President of Asset Management Cheri Warren said the introduction and availability of both real time and detailed operational performance information introduces opportunities to understand the state of the utility’s system and its performance at a lower level of the network than they have previously had available.

“National Grid will leverage this information in several ways. Through more detailed short term outage information, the company can better identify potential infrastructure issues or better understand customer experience and causes. By collecting this big data for multiple periods or seasons, long term asset performance will greatly improve as the data will inform the performance, health, network design, asset policy and peak needs of the network and identify investments that will enhance the resiliency and reliability,” Warren said.

Warren added that National Grid has been exploring integrating other operational and customer data from other source systems into a common "data fusion" database. “Bringing together information such as customer relationship management, asset performance, work management and smart grid data will produce new insights to help drive improved asset management policies, asset planning practices and customer experience,” she said.

Warren said smart grid technology allows utilities to gather information about system performance at a �ner level of accuracy. “Using communication and location capabilities National Grid will be better positioned to ensure the accuracy of asset registers and gather very detailed network operation measurements. Using these and other pieces of information, National Grid will be able to enhance our network modeling tools and techniques, moving away from estimates for data models, and allowing us to more precisely identify asset underperformance issues in existing assets to further focus investments. Making smarter investment decisions will help drive increased reliability, performance and power quality.”

Protecting critical assets and ensuring compliance is long a part of standard operating procedure for utilities. However, new challenges have arisen with the new game-changing smart technology initiatives and heightened public awareness of privacy concerns. And as data threats multiply, so do compliance requirements. Ever changing legislative and regulatory activity related to data protection is at an all time high from both federal and state governments as well as from public utility commissions about the privacy implications of smart meters.

Some key elements of e�ective privacy are:

■ Understand your company’s compliance and culture

■ Align and train management and sta� on security practices

■ Know your data, where it is, and what must be protected

■ Ensure third parties comply with your privacy policies

■ Understand your threats and controls

■ Test and update controls regularly

■ Be prepared to respond to incidents

Additionally, utilities must keep up with compliance standards relating to security as set by the North American Electric Reliability Corporation (NERC), which acts at the direction of the Federal Energy Regulatory Commission (FERC) - a federal organization overseeing interstate transportation and marketing of energy.

NERC is responsible for establishing reliability standards for the Bulk Electric System (aka the power grid), which has become an increasingly tempting target for cyber terrorism and nation level threats. NERC's Critical Infrastructure Protection (CIP) Standards identify the minimum cyber controls and protections which power suppliers and generators must address or face signi�cant penalties and �nes.

The NERC CIP compliance standards are organized into nine (9) key areas:

■ Sabotage

■ Critical Asset Identi�cation

■ Security Management Controls

■ Personnel and Training

■ Electronic Security Perimeter

■ Physical Security Protection

■ Systems Security Management

■ Incident Reporting and Response Planning

■ Recovery Plans

Smart technology and its associated big data management and analytics have transformed the industry with many avenues to ensure a growing ROI. This new technology has streamlined and improved techniques for distributing safe and clean energy for the global marketplace.

This technology has also created a growing cyber terrorism risk for enemies that understand all too well that the world depends and functions on utility infrastructure. It has also created challenges for the utility industry to safeguard privacy while at the same time continuing its mission of providing safe electricity, gas and water.

The complexity of delivering services on a mass scale has made utilities slow to change in keeping pace with new technologies, but both consumers in the marketplace and government, through compliance mandates and regulations, dictate that smart grids are necessary for sustained energy needs. In facing these challenges, utilities realize it means not only deploying and integrating smart technology, but changing organizational structure.

Data analytics has brought a new precision to understanding and improving operational e�ciency and reliability. Whether developed in-house or increasingly through soft grid vendors o�ering various solutions, data analytics has shifted organizational culture as it continues to create synergies and bring disparate data under a uni�ed management approach.

The future of data analytics continues to evolve as utility companies explore new uses such as developing a common “data fusion” database integrating other operational and customer data from other source systems into a common system.

The role of data analytics in the utility industry is now moving towards similar roles it plays in the telecommunications and �nance sectors. Due to the industry’s key role in global infrastructure it is critical to continue developing new uses and strategies for this cutting-edge technology.

3. Changing Organizational Culture To Accommodate a Data Driven Utility

The rise of big data poses the challenge of where to put the data and what to do with it. The answer is in the creation of a dedicated data center, of which there are several options, all of which have their pros and cons.

Some of these options include: ■ Data center facilities – These facilities are dedicated, custom-built housing and are usually customer-owned or leased from telecommunications or IT service providers. While these facilities usually have some form of contingencies for disaster recovery, they often increase the levels of capital expenditures.

■ Storage arrays – These provide data storage functionality and the current main vendors are EMC Corporation, NetApp and IBM.

■ Server platforms – This is computing hardware needed for data, and the Main vendors are HP, IBM and Dell.

■ Storage area network equipment – This connects server, storage and external network resources, and the main vendors are Brocade, Cisco and QLogic.

■ Database systems – This is software systems for data management and analysis. The main vendors are Oracle, IBM and Microsoft.

■ Virtualization systems – These systems allow more e�cient use of discrete storage and computing resources. The main vendors are VMware, Citrix and Microsoft.

Corresponding to the storage of big data are the analytics options in interpreting the data. Some of these options include:

■ Develop systems in house – The bene�t to this is the system is highly tailored to the utility’s need. On the downside there is often a lack of in-house development skills and resources to create a fully-�edged a data analytics system from scratch.

■ Rely on OT vendor system – The advantage of relying on OT vendors for data analytics is that these companies are well versed in the issues facing utilities and can therefore provide products that are highly relevant to the industry. On the downside the analytics and IT integration capabilities might not align well with IT vendors.

■ Rely on IT vendor systems – The advantages are a good integration with IT systems with good design and support. On the downside there may be an increased cost for unnecessary features and a lack of alignment with the operational side.

■ Rely on point products or pure-play vendor system. This produces close alignment with operational processes and good integration with OT, but there’s a potential lack of IT system integration and lifetime support concerns.

■ Rely on third-party data analytics service provider system. There is a reduced cost and improved analytics, but it is an evolving concept with a limited track record and data sharing issues.

Along with assuring a ROI in smart grid deployment through big data management and analytics systems, this new digital paradigm also requires creating a new organizational foundation for data and analytics initiatives through a workplace culture change.

This starts with de�ning data ownership and responsibility. According to the Data Governance Institute, data doesn’t belong to individuals, but is an asset to the enterprise. As such some organizations assign “owners” to data making individuals or preferably teams responsible for data subsets utilized for operational e�ciency.

Organizational culture also entails the reduction of departmental segregation and the need to balance out disconnects between operational and IT departments.

“This must be done by the utility organization itself,” says John D. McDonald, Director, Technical Strategy and Policy Development at GE Digital Energy. “One suggestion is for the operational department to provide several of its people to the IT department to work for one to two years. Likewise, IT does the same to operations. In this way, each department will have a better understanding of the other department, and have close friendships in the other department.”

McDonald also recommends the utility should have a knowledge management system in place to capture this knowledge from workforce retirees to0 allow for incorporation of better standard engineering designs.

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES | 8www.smartgridupdate.com

MA

RKET O

VERV

IEW

In the past, utilities tended to focus business practices on a single functional area, but in today’s integrated systems, these processes cross several boundaries to create synergies. Thus, these synergies challenge the utility’s traditionally siloed functional organization as the line between operations technology (OT) and utility information technology (IT) blurs. McDonald recommends utility management discuss the need to work holistically across siloed groups. This includes having the CEO “strongly encourage” or mandate holistic work. “Make sure the project managers for the Smart Grid projects have the responsibility and multi-discipline authority (across siloed groups). With the implementation of new technology there will be revision of existing business processes and changes in organizational structure and new skill sets needed.”

Also crucial to organizational change is the utility company’s ability to assess what data to collect. In order to do this management must �rst identify the business groups (asset management, power quality, maintenance, etc.) within the utility. Second, each business group must identify their data needs – both operational and non-operational data. This includes attributes of each data point – such as how often the data point is needed and frequency of update. Then operational data levels of importance are assigned to di�erent scan groups (di�erent scan rates).

Maximizing the value of smart grid systems involves bringing disparate data under a uni�ed management approach. Operations mangers must determine what data to collect, how to manage it, who should have access to speci�c data and how that data is used and stored.

This kind of organizational change demands job redesign, training and new skill development.

But just as legacy roles and responsibilities have to be modi�ed, so do legacy systems. This presents its own integration challenges. Many legacy systems were not

designed to manage signi�cant data loads or speci�c attributes that are needed to gain desired operational bene�ts, while at the same time maintaining ongoing reliable operations.

Many customized systems are no longer supported by the original vendors. This makes for high risk associated with breaking open the applications or systems for modi�cation.

Southern California Edison Manager of Enterprise Architecture James Horstman said this has not been a major issue for the utility. “Up until a few years ago the majority of our business applications were internally developed and supported. While we have gone more to a COTS (commercial o�-the-shelf ) approach the vendors we have worked with are all supported by the vendors,” he said.

Horstman did allow there were a few instances of smaller applications losing support but those were typically replaced without signi�cant risks coming to bear. “Typically our contracts with vendors include escrow provisions for providing the software to us in the event of non support. I can only think of one instance where that actually resulted in us taking over the software and it was not a critical application,” he said.

Another utility executive said his company looks at this challenge on a case-to-case basis. “You accept the fact and either change process to match the new version or accept the current version for all its issues,” he said.

In total, changing organizational culture into a data driven utility is crucial for utilizing the big data analytics to make a utility run smoother which translates into a larger ROI.

According to GTM research the cumulative spending power on utility data analytics is expected to top $18 billion over the next six years with an annual spend of $3.8 billion globally by 2020. The report also forecasts U.S. utilities alone will spend nearly $100 per home in grid operations and consumer-related analytics over the next six years.

The biggest growth market for vendors in utilities is on analytics software (soft grid) that supply systems that allow utilities to track, visualize and predict everything from grid operations to energy consumption. This market includes the biggest names in IT, who are positioning themselves as premiere vendors, but several seminal start-up and smaller companies are also o�ering custom speci�c analytic solutions.

In selecting a vendor, the best place to start, according to McDonald, is in evaluating the utility’s needs. “Identify the missing technology components needed for an integrated solution, and consider companies who have this technology. Frequently, these are start-up companies,” he said.

Other evaluation methods for selecting vendors include:

■ Get an overview of the latest big data solutions and plug in how they would work for the utility.

■ Assess the cost and risk factors associated with smart grid and big data investments and match them with proposed vendor solutions

■ Put together an RFP (Request for proposals) to select vendors to address the scope of the utilities’ needs and have vendors respond with solution plans and costs including possible cost overruns.

Utilities are unique in that the nature of harnessing, storing, supplying and distributing electric, gas and water is critical to the infrastructure of the entire globe.

Therefore, the use of analytics demands levels of reliability and safety far greater than typical personal, business and industrial requirements.

The three analytic subsets often used within the utility industry are:

■ Descriptive analytics that gauge current performance■ Predictive analytics that tell utilities what’s about to go wrong ■ Prescriptive analytics that point to problem prevention.

Analytics in utilities are used to improve reliability in business operations by preventing re-work, improving outage statistics like System Average Interruption Frequency Index (SAIFI), replacing overloaded transformers before they cause outages and identifying equipment malfunctions and cost trends.

Some �elds where analytics improve reliability include:

■ Predictive analytics are utilized for the maintenance and modernization of an aging infrastructure and for improved visibility across automated systems

■ Descriptive analytics leverage real time data for smart control to enable situational awareness in operations

■ Prescriptive analytics identify gaps in existing assets and establish sound asset management practices and programs.

Analytics are also used to drive e�ciency by mining data for common indicators of problems like slow meters, high usage, or failing feeders. This includes using sophisticated algorithms to identify, for instance, the causes of load ine�ciencies or missed opportunities for conservation program outreach. Analytics also creates e�ciencies by:

■ Utilizing predictive customer analytics for successful rollout of demand/supply programs

■ Predicting demand and supply, which reduces outages by geographic area with higher e�ciency

■ Hold down energy costs and o�er enhanced services through smart integrated control systems

■ Boost consumer engagement through new communications initiatives related to Smart Grid programs

■ Leverage data to improve relationship with customers through direct marketing programs and new tailored services

National Grid Vice President of Asset Management Cheri Warren said the introduction and availability of both real time and detailed operational performance information introduces opportunities to understand the state of the utility’s system and its performance at a lower level of the network than they have previously had available.

“National Grid will leverage this information in several ways. Through more detailed short term outage information, the company can better identify potential infrastructure issues or better understand customer experience and causes. By collecting this big data for multiple periods or seasons, long term asset performance will greatly improve as the data will inform the performance, health, network design, asset policy and peak needs of the network and identify investments that will enhance the resiliency and reliability,” Warren said.

Warren added that National Grid has been exploring integrating other operational and customer data from other source systems into a common "data fusion" database. “Bringing together information such as customer relationship management, asset performance, work management and smart grid data will produce new insights to help drive improved asset management policies, asset planning practices and customer experience,” she said.

Warren said smart grid technology allows utilities to gather information about system performance at a �ner level of accuracy. “Using communication and location capabilities National Grid will be better positioned to ensure the accuracy of asset registers and gather very detailed network operation measurements. Using these and other pieces of information, National Grid will be able to enhance our network modeling tools and techniques, moving away from estimates for data models, and allowing us to more precisely identify asset underperformance issues in existing assets to further focus investments. Making smarter investment decisions will help drive increased reliability, performance and power quality.”

Protecting critical assets and ensuring compliance is long a part of standard operating procedure for utilities. However, new challenges have arisen with the new game-changing smart technology initiatives and heightened public awareness of privacy concerns. And as data threats multiply, so do compliance requirements. Ever changing legislative and regulatory activity related to data protection is at an all time high from both federal and state governments as well as from public utility commissions about the privacy implications of smart meters.

Some key elements of e�ective privacy are:

■ Understand your company’s compliance and culture

■ Align and train management and sta� on security practices

■ Know your data, where it is, and what must be protected

■ Ensure third parties comply with your privacy policies

■ Understand your threats and controls

■ Test and update controls regularly

■ Be prepared to respond to incidents

Additionally, utilities must keep up with compliance standards relating to security as set by the North American Electric Reliability Corporation (NERC), which acts at the direction of the Federal Energy Regulatory Commission (FERC) - a federal organization overseeing interstate transportation and marketing of energy.

NERC is responsible for establishing reliability standards for the Bulk Electric System (aka the power grid), which has become an increasingly tempting target for cyber terrorism and nation level threats. NERC's Critical Infrastructure Protection (CIP) Standards identify the minimum cyber controls and protections which power suppliers and generators must address or face signi�cant penalties and �nes.

The NERC CIP compliance standards are organized into nine (9) key areas:

■ Sabotage

■ Critical Asset Identi�cation

■ Security Management Controls

■ Personnel and Training

■ Electronic Security Perimeter

■ Physical Security Protection

■ Systems Security Management

■ Incident Reporting and Response Planning

■ Recovery Plans

Smart technology and its associated big data management and analytics have transformed the industry with many avenues to ensure a growing ROI. This new technology has streamlined and improved techniques for distributing safe and clean energy for the global marketplace.

This technology has also created a growing cyber terrorism risk for enemies that understand all too well that the world depends and functions on utility infrastructure. It has also created challenges for the utility industry to safeguard privacy while at the same time continuing its mission of providing safe electricity, gas and water.

The complexity of delivering services on a mass scale has made utilities slow to change in keeping pace with new technologies, but both consumers in the marketplace and government, through compliance mandates and regulations, dictate that smart grids are necessary for sustained energy needs. In facing these challenges, utilities realize it means not only deploying and integrating smart technology, but changing organizational structure.

Data analytics has brought a new precision to understanding and improving operational e�ciency and reliability. Whether developed in-house or increasingly through soft grid vendors o�ering various solutions, data analytics has shifted organizational culture as it continues to create synergies and bring disparate data under a uni�ed management approach.

The future of data analytics continues to evolve as utility companies explore new uses such as developing a common “data fusion” database integrating other operational and customer data from other source systems into a common system.

The role of data analytics in the utility industry is now moving towards similar roles it plays in the telecommunications and �nance sectors. Due to the industry’s key role in global infrastructure it is critical to continue developing new uses and strategies for this cutting-edge technology.

4. Breaking down organizational silos

The rise of big data poses the challenge of where to put the data and what to do with it. The answer is in the creation of a dedicated data center, of which there are several options, all of which have their pros and cons.

Some of these options include: ■ Data center facilities – These facilities are dedicated, custom-built housing and are usually customer-owned or leased from telecommunications or IT service providers. While these facilities usually have some form of contingencies for disaster recovery, they often increase the levels of capital expenditures.

■ Storage arrays – These provide data storage functionality and the current main vendors are EMC Corporation, NetApp and IBM.

■ Server platforms – This is computing hardware needed for data, and the Main vendors are HP, IBM and Dell.

■ Storage area network equipment – This connects server, storage and external network resources, and the main vendors are Brocade, Cisco and QLogic.

■ Database systems – This is software systems for data management and analysis. The main vendors are Oracle, IBM and Microsoft.

■ Virtualization systems – These systems allow more e�cient use of discrete storage and computing resources. The main vendors are VMware, Citrix and Microsoft.

Corresponding to the storage of big data are the analytics options in interpreting the data. Some of these options include:

■ Develop systems in house – The bene�t to this is the system is highly tailored to the utility’s need. On the downside there is often a lack of in-house development skills and resources to create a fully-�edged a data analytics system from scratch.

■ Rely on OT vendor system – The advantage of relying on OT vendors for data analytics is that these companies are well versed in the issues facing utilities and can therefore provide products that are highly relevant to the industry. On the downside the analytics and IT integration capabilities might not align well with IT vendors.

■ Rely on IT vendor systems – The advantages are a good integration with IT systems with good design and support. On the downside there may be an increased cost for unnecessary features and a lack of alignment with the operational side.

■ Rely on point products or pure-play vendor system. This produces close alignment with operational processes and good integration with OT, but there’s a potential lack of IT system integration and lifetime support concerns.

■ Rely on third-party data analytics service provider system. There is a reduced cost and improved analytics, but it is an evolving concept with a limited track record and data sharing issues.

Along with assuring a ROI in smart grid deployment through big data management and analytics systems, this new digital paradigm also requires creating a new organizational foundation for data and analytics initiatives through a workplace culture change.

This starts with de�ning data ownership and responsibility. According to the Data Governance Institute, data doesn’t belong to individuals, but is an asset to the enterprise. As such some organizations assign “owners” to data making individuals or preferably teams responsible for data subsets utilized for operational e�ciency.

Organizational culture also entails the reduction of departmental segregation and the need to balance out disconnects between operational and IT departments.

“This must be done by the utility organization itself,” says John D. McDonald, Director, Technical Strategy and Policy Development at GE Digital Energy. “One suggestion is for the operational department to provide several of its people to the IT department to work for one to two years. Likewise, IT does the same to operations. In this way, each department will have a better understanding of the other department, and have close friendships in the other department.”

McDonald also recommends the utility should have a knowledge management system in place to capture this knowledge from workforce retirees to0 allow for incorporation of better standard engineering designs.

In the past, utilities tended to focus business practices on a single functional area, but in today’s integrated systems, these processes cross several boundaries to create synergies. Thus, these synergies challenge the utility’s traditionally siloed functional organization as the line between operations technology (OT) and utility information technology (IT) blurs. McDonald recommends utility management discuss the need to work holistically across siloed groups. This includes having the CEO “strongly encourage” or mandate holistic work. “Make sure the project managers for the Smart Grid projects have the responsibility and multi-discipline authority (across siloed groups). With the implementation of new technology there will be revision of existing business processes and changes in organizational structure and new skill sets needed.”

Also crucial to organizational change is the utility company’s ability to assess what data to collect. In order to do this management must �rst identify the business groups (asset management, power quality, maintenance, etc.) within the utility. Second, each business group must identify their data needs – both operational and non-operational data. This includes attributes of each data point – such as how often the data point is needed and frequency of update. Then operational data levels of importance are assigned to di�erent scan groups (di�erent scan rates).

Maximizing the value of smart grid systems involves bringing disparate data under a uni�ed management approach. Operations mangers must determine what data to collect, how to manage it, who should have access to speci�c data and how that data is used and stored.

This kind of organizational change demands job redesign, training and new skill development.

But just as legacy roles and responsibilities have to be modi�ed, so do legacy systems. This presents its own integration challenges. Many legacy systems were not

designed to manage signi�cant data loads or speci�c attributes that are needed to gain desired operational bene�ts, while at the same time maintaining ongoing reliable operations.

Many customized systems are no longer supported by the original vendors. This makes for high risk associated with breaking open the applications or systems for modi�cation.

Southern California Edison Manager of Enterprise Architecture James Horstman said this has not been a major issue for the utility. “Up until a few years ago the majority of our business applications were internally developed and supported. While we have gone more to a COTS (commercial o�-the-shelf ) approach the vendors we have worked with are all supported by the vendors,” he said.

Horstman did allow there were a few instances of smaller applications losing support but those were typically replaced without signi�cant risks coming to bear. “Typically our contracts with vendors include escrow provisions for providing the software to us in the event of non support. I can only think of one instance where that actually resulted in us taking over the software and it was not a critical application,” he said.

Another utility executive said his company looks at this challenge on a case-to-case basis. “You accept the fact and either change process to match the new version or accept the current version for all its issues,” he said.

In total, changing organizational culture into a data driven utility is crucial for utilizing the big data analytics to make a utility run smoother which translates into a larger ROI.

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES | 9www.smartgridupdate.com

MA

RKET O

VERV

IEW

According to GTM research the cumulative spending power on utility data analytics is expected to top $18 billion over the next six years with an annual spend of $3.8 billion globally by 2020. The report also forecasts U.S. utilities alone will spend nearly $100 per home in grid operations and consumer-related analytics over the next six years.

The biggest growth market for vendors in utilities is on analytics software (soft grid) that supply systems that allow utilities to track, visualize and predict everything from grid operations to energy consumption. This market includes the biggest names in IT, who are positioning themselves as premiere vendors, but several seminal start-up and smaller companies are also o�ering custom speci�c analytic solutions.

In selecting a vendor, the best place to start, according to McDonald, is in evaluating the utility’s needs. “Identify the missing technology components needed for an integrated solution, and consider companies who have this technology. Frequently, these are start-up companies,” he said.

Other evaluation methods for selecting vendors include:

■ Get an overview of the latest big data solutions and plug in how they would work for the utility.

■ Assess the cost and risk factors associated with smart grid and big data investments and match them with proposed vendor solutions

■ Put together an RFP (Request for proposals) to select vendors to address the scope of the utilities’ needs and have vendors respond with solution plans and costs including possible cost overruns.

Utilities are unique in that the nature of harnessing, storing, supplying and distributing electric, gas and water is critical to the infrastructure of the entire globe.

Therefore, the use of analytics demands levels of reliability and safety far greater than typical personal, business and industrial requirements.

The three analytic subsets often used within the utility industry are:

■ Descriptive analytics that gauge current performance■ Predictive analytics that tell utilities what’s about to go wrong ■ Prescriptive analytics that point to problem prevention.

Analytics in utilities are used to improve reliability in business operations by preventing re-work, improving outage statistics like System Average Interruption Frequency Index (SAIFI), replacing overloaded transformers before they cause outages and identifying equipment malfunctions and cost trends.

Some �elds where analytics improve reliability include:

■ Predictive analytics are utilized for the maintenance and modernization of an aging infrastructure and for improved visibility across automated systems

■ Descriptive analytics leverage real time data for smart control to enable situational awareness in operations

■ Prescriptive analytics identify gaps in existing assets and establish sound asset management practices and programs.

Analytics are also used to drive e�ciency by mining data for common indicators of problems like slow meters, high usage, or failing feeders. This includes using sophisticated algorithms to identify, for instance, the causes of load ine�ciencies or missed opportunities for conservation program outreach. Analytics also creates e�ciencies by:

■ Utilizing predictive customer analytics for successful rollout of demand/supply programs

■ Predicting demand and supply, which reduces outages by geographic area with higher e�ciency

■ Hold down energy costs and o�er enhanced services through smart integrated control systems

■ Boost consumer engagement through new communications initiatives related to Smart Grid programs

■ Leverage data to improve relationship with customers through direct marketing programs and new tailored services

National Grid Vice President of Asset Management Cheri Warren said the introduction and availability of both real time and detailed operational performance information introduces opportunities to understand the state of the utility’s system and its performance at a lower level of the network than they have previously had available.

“National Grid will leverage this information in several ways. Through more detailed short term outage information, the company can better identify potential infrastructure issues or better understand customer experience and causes. By collecting this big data for multiple periods or seasons, long term asset performance will greatly improve as the data will inform the performance, health, network design, asset policy and peak needs of the network and identify investments that will enhance the resiliency and reliability,” Warren said.

Warren added that National Grid has been exploring integrating other operational and customer data from other source systems into a common "data fusion" database. “Bringing together information such as customer relationship management, asset performance, work management and smart grid data will produce new insights to help drive improved asset management policies, asset planning practices and customer experience,” she said.

Warren said smart grid technology allows utilities to gather information about system performance at a �ner level of accuracy. “Using communication and location capabilities National Grid will be better positioned to ensure the accuracy of asset registers and gather very detailed network operation measurements. Using these and other pieces of information, National Grid will be able to enhance our network modeling tools and techniques, moving away from estimates for data models, and allowing us to more precisely identify asset underperformance issues in existing assets to further focus investments. Making smarter investment decisions will help drive increased reliability, performance and power quality.”

Protecting critical assets and ensuring compliance is long a part of standard operating procedure for utilities. However, new challenges have arisen with the new game-changing smart technology initiatives and heightened public awareness of privacy concerns. And as data threats multiply, so do compliance requirements. Ever changing legislative and regulatory activity related to data protection is at an all time high from both federal and state governments as well as from public utility commissions about the privacy implications of smart meters.

Some key elements of e�ective privacy are:

■ Understand your company’s compliance and culture

■ Align and train management and sta� on security practices

■ Know your data, where it is, and what must be protected

■ Ensure third parties comply with your privacy policies

■ Understand your threats and controls

■ Test and update controls regularly

■ Be prepared to respond to incidents

Additionally, utilities must keep up with compliance standards relating to security as set by the North American Electric Reliability Corporation (NERC), which acts at the direction of the Federal Energy Regulatory Commission (FERC) - a federal organization overseeing interstate transportation and marketing of energy.

NERC is responsible for establishing reliability standards for the Bulk Electric System (aka the power grid), which has become an increasingly tempting target for cyber terrorism and nation level threats. NERC's Critical Infrastructure Protection (CIP) Standards identify the minimum cyber controls and protections which power suppliers and generators must address or face signi�cant penalties and �nes.

The NERC CIP compliance standards are organized into nine (9) key areas:

■ Sabotage

■ Critical Asset Identi�cation

■ Security Management Controls

■ Personnel and Training

■ Electronic Security Perimeter

■ Physical Security Protection

■ Systems Security Management

■ Incident Reporting and Response Planning

■ Recovery Plans

Smart technology and its associated big data management and analytics have transformed the industry with many avenues to ensure a growing ROI. This new technology has streamlined and improved techniques for distributing safe and clean energy for the global marketplace.

This technology has also created a growing cyber terrorism risk for enemies that understand all too well that the world depends and functions on utility infrastructure. It has also created challenges for the utility industry to safeguard privacy while at the same time continuing its mission of providing safe electricity, gas and water.

The complexity of delivering services on a mass scale has made utilities slow to change in keeping pace with new technologies, but both consumers in the marketplace and government, through compliance mandates and regulations, dictate that smart grids are necessary for sustained energy needs. In facing these challenges, utilities realize it means not only deploying and integrating smart technology, but changing organizational structure.

Data analytics has brought a new precision to understanding and improving operational e�ciency and reliability. Whether developed in-house or increasingly through soft grid vendors o�ering various solutions, data analytics has shifted organizational culture as it continues to create synergies and bring disparate data under a uni�ed management approach.

The future of data analytics continues to evolve as utility companies explore new uses such as developing a common “data fusion” database integrating other operational and customer data from other source systems into a common system.

The role of data analytics in the utility industry is now moving towards similar roles it plays in the telecommunications and �nance sectors. Due to the industry’s key role in global infrastructure it is critical to continue developing new uses and strategies for this cutting-edge technology.

5. Evaluating Vendors: The best bang for your buck

FIGURE: Global Utility Analytics Spending, 2012-2020Source: The Soft Grid 2013-2020: Big Data & Utility Analytics for Smart Grid

FIGURE: Leading Vendors in Soft GridSource: The Soft Grid 2013-2020: Big Data & Utility Analytics for Smart Grid

The rise of big data poses the challenge of where to put the data and what to do with it. The answer is in the creation of a dedicated data center, of which there are several options, all of which have their pros and cons.

Some of these options include: ■ Data center facilities – These facilities are dedicated, custom-built housing and are usually customer-owned or leased from telecommunications or IT service providers. While these facilities usually have some form of contingencies for disaster recovery, they often increase the levels of capital expenditures.

■ Storage arrays – These provide data storage functionality and the current main vendors are EMC Corporation, NetApp and IBM.

■ Server platforms – This is computing hardware needed for data, and the Main vendors are HP, IBM and Dell.

■ Storage area network equipment – This connects server, storage and external network resources, and the main vendors are Brocade, Cisco and QLogic.

■ Database systems – This is software systems for data management and analysis. The main vendors are Oracle, IBM and Microsoft.

■ Virtualization systems – These systems allow more e�cient use of discrete storage and computing resources. The main vendors are VMware, Citrix and Microsoft.

Corresponding to the storage of big data are the analytics options in interpreting the data. Some of these options include:

■ Develop systems in house – The bene�t to this is the system is highly tailored to the utility’s need. On the downside there is often a lack of in-house development skills and resources to create a fully-�edged a data analytics system from scratch.

■ Rely on OT vendor system – The advantage of relying on OT vendors for data analytics is that these companies are well versed in the issues facing utilities and can therefore provide products that are highly relevant to the industry. On the downside the analytics and IT integration capabilities might not align well with IT vendors.

■ Rely on IT vendor systems – The advantages are a good integration with IT systems with good design and support. On the downside there may be an increased cost for unnecessary features and a lack of alignment with the operational side.

■ Rely on point products or pure-play vendor system. This produces close alignment with operational processes and good integration with OT, but there’s a potential lack of IT system integration and lifetime support concerns.

■ Rely on third-party data analytics service provider system. There is a reduced cost and improved analytics, but it is an evolving concept with a limited track record and data sharing issues.

Along with assuring a ROI in smart grid deployment through big data management and analytics systems, this new digital paradigm also requires creating a new organizational foundation for data and analytics initiatives through a workplace culture change.

This starts with de�ning data ownership and responsibility. According to the Data Governance Institute, data doesn’t belong to individuals, but is an asset to the enterprise. As such some organizations assign “owners” to data making individuals or preferably teams responsible for data subsets utilized for operational e�ciency.

Organizational culture also entails the reduction of departmental segregation and the need to balance out disconnects between operational and IT departments.

“This must be done by the utility organization itself,” says John D. McDonald, Director, Technical Strategy and Policy Development at GE Digital Energy. “One suggestion is for the operational department to provide several of its people to the IT department to work for one to two years. Likewise, IT does the same to operations. In this way, each department will have a better understanding of the other department, and have close friendships in the other department.”

McDonald also recommends the utility should have a knowledge management system in place to capture this knowledge from workforce retirees to0 allow for incorporation of better standard engineering designs.

In the past, utilities tended to focus business practices on a single functional area, but in today’s integrated systems, these processes cross several boundaries to create synergies. Thus, these synergies challenge the utility’s traditionally siloed functional organization as the line between operations technology (OT) and utility information technology (IT) blurs. McDonald recommends utility management discuss the need to work holistically across siloed groups. This includes having the CEO “strongly encourage” or mandate holistic work. “Make sure the project managers for the Smart Grid projects have the responsibility and multi-discipline authority (across siloed groups). With the implementation of new technology there will be revision of existing business processes and changes in organizational structure and new skill sets needed.”

Also crucial to organizational change is the utility company’s ability to assess what data to collect. In order to do this management must �rst identify the business groups (asset management, power quality, maintenance, etc.) within the utility. Second, each business group must identify their data needs – both operational and non-operational data. This includes attributes of each data point – such as how often the data point is needed and frequency of update. Then operational data levels of importance are assigned to di�erent scan groups (di�erent scan rates).

Maximizing the value of smart grid systems involves bringing disparate data under a uni�ed management approach. Operations mangers must determine what data to collect, how to manage it, who should have access to speci�c data and how that data is used and stored.

This kind of organizational change demands job redesign, training and new skill development.

But just as legacy roles and responsibilities have to be modi�ed, so do legacy systems. This presents its own integration challenges. Many legacy systems were not

designed to manage signi�cant data loads or speci�c attributes that are needed to gain desired operational bene�ts, while at the same time maintaining ongoing reliable operations.

Many customized systems are no longer supported by the original vendors. This makes for high risk associated with breaking open the applications or systems for modi�cation.

Southern California Edison Manager of Enterprise Architecture James Horstman said this has not been a major issue for the utility. “Up until a few years ago the majority of our business applications were internally developed and supported. While we have gone more to a COTS (commercial o�-the-shelf ) approach the vendors we have worked with are all supported by the vendors,” he said.

Horstman did allow there were a few instances of smaller applications losing support but those were typically replaced without signi�cant risks coming to bear. “Typically our contracts with vendors include escrow provisions for providing the software to us in the event of non support. I can only think of one instance where that actually resulted in us taking over the software and it was not a critical application,” he said.

Another utility executive said his company looks at this challenge on a case-to-case basis. “You accept the fact and either change process to match the new version or accept the current version for all its issues,” he said.

In total, changing organizational culture into a data driven utility is crucial for utilizing the big data analytics to make a utility run smoother which translates into a larger ROI.

According to GTM research the cumulative spending power on utility data analytics is expected to top $18 billion over the next six years with an annual spend of $3.8 billion globally by 2020. The report also forecasts U.S. utilities alone will spend nearly $100 per home in grid operations and consumer-related analytics over the next six years.

The biggest growth market for vendors in utilities is on analytics software (soft grid) that supply systems that allow utilities to track, visualize and predict everything from grid operations to energy consumption. This market includes the biggest names in IT, who are positioning themselves as premiere vendors, but several seminal start-up and smaller companies are also o�ering custom speci�c analytic solutions.

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES | 10www.smartgridupdate.com

MA

RKET O

VERV

IEW

In selecting a vendor, the best place to start, according to McDonald, is in evaluating the utility’s needs. “Identify the missing technology components needed for an integrated solution, and consider companies who have this technology. Frequently, these are start-up companies,” he said.

Other evaluation methods for selecting vendors include:

■ Get an overview of the latest big data solutions and plug in how they would work for the utility.

■ Assess the cost and risk factors associated with smart grid and big data investments and match them with proposed vendor solutions

■ Put together an RFP (Request for proposals) to select vendors to address the scope of the utilities’ needs and have vendors respond with solution plans and costs including possible cost overruns.

Utilities are unique in that the nature of harnessing, storing, supplying and distributing electric, gas and water is critical to the infrastructure of the entire globe.

Therefore, the use of analytics demands levels of reliability and safety far greater than typical personal, business and industrial requirements.

The three analytic subsets often used within the utility industry are:

■ Descriptive analytics that gauge current performance■ Predictive analytics that tell utilities what’s about to go wrong ■ Prescriptive analytics that point to problem prevention.

Analytics in utilities are used to improve reliability in business operations by preventing re-work, improving outage statistics like System Average Interruption Frequency Index (SAIFI), replacing overloaded transformers before they cause outages and identifying equipment malfunctions and cost trends.

Some �elds where analytics improve reliability include:

■ Predictive analytics are utilized for the maintenance and modernization of an aging infrastructure and for improved visibility across automated systems

■ Descriptive analytics leverage real time data for smart control to enable situational awareness in operations

■ Prescriptive analytics identify gaps in existing assets and establish sound asset management practices and programs.

Analytics are also used to drive e�ciency by mining data for common indicators of problems like slow meters, high usage, or failing feeders. This includes using sophisticated algorithms to identify, for instance, the causes of load ine�ciencies or missed opportunities for conservation program outreach. Analytics also creates e�ciencies by:

■ Utilizing predictive customer analytics for successful rollout of demand/supply programs

■ Predicting demand and supply, which reduces outages by geographic area with higher e�ciency

■ Hold down energy costs and o�er enhanced services through smart integrated control systems

■ Boost consumer engagement through new communications initiatives related to Smart Grid programs

■ Leverage data to improve relationship with customers through direct marketing programs and new tailored services

National Grid Vice President of Asset Management Cheri Warren said the introduction and availability of both real time and detailed operational performance information introduces opportunities to understand the state of the utility’s system and its performance at a lower level of the network than they have previously had available.

“National Grid will leverage this information in several ways. Through more detailed short term outage information, the company can better identify potential infrastructure issues or better understand customer experience and causes. By collecting this big data for multiple periods or seasons, long term asset performance will greatly improve as the data will inform the performance, health, network design, asset policy and peak needs of the network and identify investments that will enhance the resiliency and reliability,” Warren said.

Warren added that National Grid has been exploring integrating other operational and customer data from other source systems into a common "data fusion" database. “Bringing together information such as customer relationship management, asset performance, work management and smart grid data will produce new insights to help drive improved asset management policies, asset planning practices and customer experience,” she said.

Warren said smart grid technology allows utilities to gather information about system performance at a �ner level of accuracy. “Using communication and location capabilities National Grid will be better positioned to ensure the accuracy of asset registers and gather very detailed network operation measurements. Using these and other pieces of information, National Grid will be able to enhance our network modeling tools and techniques, moving away from estimates for data models, and allowing us to more precisely identify asset underperformance issues in existing assets to further focus investments. Making smarter investment decisions will help drive increased reliability, performance and power quality.”

Protecting critical assets and ensuring compliance is long a part of standard operating procedure for utilities. However, new challenges have arisen with the new game-changing smart technology initiatives and heightened public awareness of privacy concerns. And as data threats multiply, so do compliance requirements. Ever changing legislative and regulatory activity related to data protection is at an all time high from both federal and state governments as well as from public utility commissions about the privacy implications of smart meters.

Some key elements of e�ective privacy are:

■ Understand your company’s compliance and culture

■ Align and train management and sta� on security practices

■ Know your data, where it is, and what must be protected

■ Ensure third parties comply with your privacy policies

■ Understand your threats and controls

■ Test and update controls regularly

■ Be prepared to respond to incidents

Additionally, utilities must keep up with compliance standards relating to security as set by the North American Electric Reliability Corporation (NERC), which acts at the direction of the Federal Energy Regulatory Commission (FERC) - a federal organization overseeing interstate transportation and marketing of energy.

NERC is responsible for establishing reliability standards for the Bulk Electric System (aka the power grid), which has become an increasingly tempting target for cyber terrorism and nation level threats. NERC's Critical Infrastructure Protection (CIP) Standards identify the minimum cyber controls and protections which power suppliers and generators must address or face signi�cant penalties and �nes.

The NERC CIP compliance standards are organized into nine (9) key areas:

■ Sabotage

■ Critical Asset Identi�cation

■ Security Management Controls

■ Personnel and Training

■ Electronic Security Perimeter

■ Physical Security Protection

■ Systems Security Management

■ Incident Reporting and Response Planning

■ Recovery Plans

Smart technology and its associated big data management and analytics have transformed the industry with many avenues to ensure a growing ROI. This new technology has streamlined and improved techniques for distributing safe and clean energy for the global marketplace.

This technology has also created a growing cyber terrorism risk for enemies that understand all too well that the world depends and functions on utility infrastructure. It has also created challenges for the utility industry to safeguard privacy while at the same time continuing its mission of providing safe electricity, gas and water.

The complexity of delivering services on a mass scale has made utilities slow to change in keeping pace with new technologies, but both consumers in the marketplace and government, through compliance mandates and regulations, dictate that smart grids are necessary for sustained energy needs. In facing these challenges, utilities realize it means not only deploying and integrating smart technology, but changing organizational structure.

Data analytics has brought a new precision to understanding and improving operational e�ciency and reliability. Whether developed in-house or increasingly through soft grid vendors o�ering various solutions, data analytics has shifted organizational culture as it continues to create synergies and bring disparate data under a uni�ed management approach.

The future of data analytics continues to evolve as utility companies explore new uses such as developing a common “data fusion” database integrating other operational and customer data from other source systems into a common system.

The role of data analytics in the utility industry is now moving towards similar roles it plays in the telecommunications and �nance sectors. Due to the industry’s key role in global infrastructure it is critical to continue developing new uses and strategies for this cutting-edge technology.

6. Selecting a vendor

The rise of big data poses the challenge of where to put the data and what to do with it. The answer is in the creation of a dedicated data center, of which there are several options, all of which have their pros and cons.

Some of these options include: ■ Data center facilities – These facilities are dedicated, custom-built housing and are usually customer-owned or leased from telecommunications or IT service providers. While these facilities usually have some form of contingencies for disaster recovery, they often increase the levels of capital expenditures.

■ Storage arrays – These provide data storage functionality and the current main vendors are EMC Corporation, NetApp and IBM.

■ Server platforms – This is computing hardware needed for data, and the Main vendors are HP, IBM and Dell.

■ Storage area network equipment – This connects server, storage and external network resources, and the main vendors are Brocade, Cisco and QLogic.

■ Database systems – This is software systems for data management and analysis. The main vendors are Oracle, IBM and Microsoft.

■ Virtualization systems – These systems allow more e�cient use of discrete storage and computing resources. The main vendors are VMware, Citrix and Microsoft.

Corresponding to the storage of big data are the analytics options in interpreting the data. Some of these options include:

■ Develop systems in house – The bene�t to this is the system is highly tailored to the utility’s need. On the downside there is often a lack of in-house development skills and resources to create a fully-�edged a data analytics system from scratch.

■ Rely on OT vendor system – The advantage of relying on OT vendors for data analytics is that these companies are well versed in the issues facing utilities and can therefore provide products that are highly relevant to the industry. On the downside the analytics and IT integration capabilities might not align well with IT vendors.

■ Rely on IT vendor systems – The advantages are a good integration with IT systems with good design and support. On the downside there may be an increased cost for unnecessary features and a lack of alignment with the operational side.

■ Rely on point products or pure-play vendor system. This produces close alignment with operational processes and good integration with OT, but there’s a potential lack of IT system integration and lifetime support concerns.

■ Rely on third-party data analytics service provider system. There is a reduced cost and improved analytics, but it is an evolving concept with a limited track record and data sharing issues.

Along with assuring a ROI in smart grid deployment through big data management and analytics systems, this new digital paradigm also requires creating a new organizational foundation for data and analytics initiatives through a workplace culture change.

This starts with de�ning data ownership and responsibility. According to the Data Governance Institute, data doesn’t belong to individuals, but is an asset to the enterprise. As such some organizations assign “owners” to data making individuals or preferably teams responsible for data subsets utilized for operational e�ciency.

Organizational culture also entails the reduction of departmental segregation and the need to balance out disconnects between operational and IT departments.

“This must be done by the utility organization itself,” says John D. McDonald, Director, Technical Strategy and Policy Development at GE Digital Energy. “One suggestion is for the operational department to provide several of its people to the IT department to work for one to two years. Likewise, IT does the same to operations. In this way, each department will have a better understanding of the other department, and have close friendships in the other department.”

McDonald also recommends the utility should have a knowledge management system in place to capture this knowledge from workforce retirees to0 allow for incorporation of better standard engineering designs.

In the past, utilities tended to focus business practices on a single functional area, but in today’s integrated systems, these processes cross several boundaries to create synergies. Thus, these synergies challenge the utility’s traditionally siloed functional organization as the line between operations technology (OT) and utility information technology (IT) blurs. McDonald recommends utility management discuss the need to work holistically across siloed groups. This includes having the CEO “strongly encourage” or mandate holistic work. “Make sure the project managers for the Smart Grid projects have the responsibility and multi-discipline authority (across siloed groups). With the implementation of new technology there will be revision of existing business processes and changes in organizational structure and new skill sets needed.”

Also crucial to organizational change is the utility company’s ability to assess what data to collect. In order to do this management must �rst identify the business groups (asset management, power quality, maintenance, etc.) within the utility. Second, each business group must identify their data needs – both operational and non-operational data. This includes attributes of each data point – such as how often the data point is needed and frequency of update. Then operational data levels of importance are assigned to di�erent scan groups (di�erent scan rates).

Maximizing the value of smart grid systems involves bringing disparate data under a uni�ed management approach. Operations mangers must determine what data to collect, how to manage it, who should have access to speci�c data and how that data is used and stored.

This kind of organizational change demands job redesign, training and new skill development.

But just as legacy roles and responsibilities have to be modi�ed, so do legacy systems. This presents its own integration challenges. Many legacy systems were not

designed to manage signi�cant data loads or speci�c attributes that are needed to gain desired operational bene�ts, while at the same time maintaining ongoing reliable operations.

Many customized systems are no longer supported by the original vendors. This makes for high risk associated with breaking open the applications or systems for modi�cation.

Southern California Edison Manager of Enterprise Architecture James Horstman said this has not been a major issue for the utility. “Up until a few years ago the majority of our business applications were internally developed and supported. While we have gone more to a COTS (commercial o�-the-shelf ) approach the vendors we have worked with are all supported by the vendors,” he said.

Horstman did allow there were a few instances of smaller applications losing support but those were typically replaced without signi�cant risks coming to bear. “Typically our contracts with vendors include escrow provisions for providing the software to us in the event of non support. I can only think of one instance where that actually resulted in us taking over the software and it was not a critical application,” he said.

Another utility executive said his company looks at this challenge on a case-to-case basis. “You accept the fact and either change process to match the new version or accept the current version for all its issues,” he said.

In total, changing organizational culture into a data driven utility is crucial for utilizing the big data analytics to make a utility run smoother which translates into a larger ROI.

According to GTM research the cumulative spending power on utility data analytics is expected to top $18 billion over the next six years with an annual spend of $3.8 billion globally by 2020. The report also forecasts U.S. utilities alone will spend nearly $100 per home in grid operations and consumer-related analytics over the next six years.

The biggest growth market for vendors in utilities is on analytics software (soft grid) that supply systems that allow utilities to track, visualize and predict everything from grid operations to energy consumption. This market includes the biggest names in IT, who are positioning themselves as premiere vendors, but several seminal start-up and smaller companies are also o�ering custom speci�c analytic solutions.

In selecting a vendor, the best place to start, according to McDonald, is in evaluating the utility’s needs. “Identify the missing technology components needed for an integrated solution, and consider companies who have this technology. Frequently, these are start-up companies,” he said.

Other evaluation methods for selecting vendors include:

■ Get an overview of the latest big data solutions and plug in how they would work for the utility.

■ Assess the cost and risk factors associated with smart grid and big data investments and match them with proposed vendor solutions

■ Put together an RFP (Request for proposals) to select vendors to address the scope of the utilities’ needs and have vendors respond with solution plans and costs including possible cost overruns.

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES | 11www.smartgridupdate.com

MA

RKET O

VERV

IEW

Utilities are unique in that the nature of harnessing, storing, supplying and distributing electric, gas and water is critical to the infrastructure of the entire globe.

Therefore, the use of analytics demands levels of reliability and safety far greater than typical personal, business and industrial requirements.

The three analytic subsets often used within the utility industry are:

■ Descriptive analytics that gauge current performance■ Predictive analytics that tell utilities what’s about to go wrong ■ Prescriptive analytics that point to problem prevention.

Analytics in utilities are used to improve reliability in business operations by preventing re-work, improving outage statistics like System Average Interruption Frequency Index (SAIFI), replacing overloaded transformers before they cause outages and identifying equipment malfunctions and cost trends.

Some �elds where analytics improve reliability include:

■ Predictive analytics are utilized for the maintenance and modernization of an aging infrastructure and for improved visibility across automated systems

■ Descriptive analytics leverage real time data for smart control to enable situational awareness in operations

■ Prescriptive analytics identify gaps in existing assets and establish sound asset management practices and programs.

Analytics are also used to drive e�ciency by mining data for common indicators of problems like slow meters, high usage, or failing feeders. This includes using sophisticated algorithms to identify, for instance, the causes of load ine�ciencies or missed opportunities for conservation program outreach. Analytics also creates e�ciencies by:

■ Utilizing predictive customer analytics for successful rollout of demand/supply programs

■ Predicting demand and supply, which reduces outages by geographic area with higher e�ciency

■ Hold down energy costs and o�er enhanced services through smart integrated control systems

■ Boost consumer engagement through new communications initiatives related to Smart Grid programs

■ Leverage data to improve relationship with customers through direct marketing programs and new tailored services

National Grid Vice President of Asset Management Cheri Warren said the introduction and availability of both real time and detailed operational performance information introduces opportunities to understand the state of the utility’s system and its performance at a lower level of the network than they have previously had available.

“National Grid will leverage this information in several ways. Through more detailed short term outage information, the company can better identify potential infrastructure issues or better understand customer experience and causes. By collecting this big data for multiple periods or seasons, long term asset performance will greatly improve as the data will inform the performance, health, network design, asset policy and peak needs of the network and identify investments that will enhance the resiliency and reliability,” Warren said.

Warren added that National Grid has been exploring integrating other operational and customer data from other source systems into a common "data fusion" database. “Bringing together information such as customer relationship management, asset performance, work management and smart grid data will produce new insights to help drive improved asset management policies, asset planning practices and customer experience,” she said.

Warren said smart grid technology allows utilities to gather information about system performance at a �ner level of accuracy. “Using communication and location capabilities National Grid will be better positioned to ensure the accuracy of asset registers and gather very detailed network operation measurements. Using these and other pieces of information, National Grid will be able to enhance our network modeling tools and techniques, moving away from estimates for data models, and allowing us to more precisely identify asset underperformance issues in existing assets to further focus investments. Making smarter investment decisions will help drive increased reliability, performance and power quality.”

Protecting critical assets and ensuring compliance is long a part of standard operating procedure for utilities. However, new challenges have arisen with the new game-changing smart technology initiatives and heightened public awareness of privacy concerns. And as data threats multiply, so do compliance requirements. Ever changing legislative and regulatory activity related to data protection is at an all time high from both federal and state governments as well as from public utility commissions about the privacy implications of smart meters.

Some key elements of e�ective privacy are:

■ Understand your company’s compliance and culture

■ Align and train management and sta� on security practices

■ Know your data, where it is, and what must be protected

■ Ensure third parties comply with your privacy policies

■ Understand your threats and controls

■ Test and update controls regularly

■ Be prepared to respond to incidents

Additionally, utilities must keep up with compliance standards relating to security as set by the North American Electric Reliability Corporation (NERC), which acts at the direction of the Federal Energy Regulatory Commission (FERC) - a federal organization overseeing interstate transportation and marketing of energy.

NERC is responsible for establishing reliability standards for the Bulk Electric System (aka the power grid), which has become an increasingly tempting target for cyber terrorism and nation level threats. NERC's Critical Infrastructure Protection (CIP) Standards identify the minimum cyber controls and protections which power suppliers and generators must address or face signi�cant penalties and �nes.

The NERC CIP compliance standards are organized into nine (9) key areas:

■ Sabotage

■ Critical Asset Identi�cation

■ Security Management Controls

■ Personnel and Training

■ Electronic Security Perimeter

■ Physical Security Protection

■ Systems Security Management

■ Incident Reporting and Response Planning

■ Recovery Plans

Smart technology and its associated big data management and analytics have transformed the industry with many avenues to ensure a growing ROI. This new technology has streamlined and improved techniques for distributing safe and clean energy for the global marketplace.

This technology has also created a growing cyber terrorism risk for enemies that understand all too well that the world depends and functions on utility infrastructure. It has also created challenges for the utility industry to safeguard privacy while at the same time continuing its mission of providing safe electricity, gas and water.

The complexity of delivering services on a mass scale has made utilities slow to change in keeping pace with new technologies, but both consumers in the marketplace and government, through compliance mandates and regulations, dictate that smart grids are necessary for sustained energy needs. In facing these challenges, utilities realize it means not only deploying and integrating smart technology, but changing organizational structure.

Data analytics has brought a new precision to understanding and improving operational e�ciency and reliability. Whether developed in-house or increasingly through soft grid vendors o�ering various solutions, data analytics has shifted organizational culture as it continues to create synergies and bring disparate data under a uni�ed management approach.

The future of data analytics continues to evolve as utility companies explore new uses such as developing a common “data fusion” database integrating other operational and customer data from other source systems into a common system.

The role of data analytics in the utility industry is now moving towards similar roles it plays in the telecommunications and �nance sectors. Due to the industry’s key role in global infrastructure it is critical to continue developing new uses and strategies for this cutting-edge technology.

7. Analytic Applications

Source: Oracle Analytics Presentation 2013

8. Improve Reliability

The rise of big data poses the challenge of where to put the data and what to do with it. The answer is in the creation of a dedicated data center, of which there are several options, all of which have their pros and cons.

Some of these options include: ■ Data center facilities – These facilities are dedicated, custom-built housing and are usually customer-owned or leased from telecommunications or IT service providers. While these facilities usually have some form of contingencies for disaster recovery, they often increase the levels of capital expenditures.

■ Storage arrays – These provide data storage functionality and the current main vendors are EMC Corporation, NetApp and IBM.

■ Server platforms – This is computing hardware needed for data, and the Main vendors are HP, IBM and Dell.

■ Storage area network equipment – This connects server, storage and external network resources, and the main vendors are Brocade, Cisco and QLogic.

■ Database systems – This is software systems for data management and analysis. The main vendors are Oracle, IBM and Microsoft.

■ Virtualization systems – These systems allow more e�cient use of discrete storage and computing resources. The main vendors are VMware, Citrix and Microsoft.

Corresponding to the storage of big data are the analytics options in interpreting the data. Some of these options include:

■ Develop systems in house – The bene�t to this is the system is highly tailored to the utility’s need. On the downside there is often a lack of in-house development skills and resources to create a fully-�edged a data analytics system from scratch.

■ Rely on OT vendor system – The advantage of relying on OT vendors for data analytics is that these companies are well versed in the issues facing utilities and can therefore provide products that are highly relevant to the industry. On the downside the analytics and IT integration capabilities might not align well with IT vendors.

■ Rely on IT vendor systems – The advantages are a good integration with IT systems with good design and support. On the downside there may be an increased cost for unnecessary features and a lack of alignment with the operational side.

■ Rely on point products or pure-play vendor system. This produces close alignment with operational processes and good integration with OT, but there’s a potential lack of IT system integration and lifetime support concerns.

■ Rely on third-party data analytics service provider system. There is a reduced cost and improved analytics, but it is an evolving concept with a limited track record and data sharing issues.

Along with assuring a ROI in smart grid deployment through big data management and analytics systems, this new digital paradigm also requires creating a new organizational foundation for data and analytics initiatives through a workplace culture change.

This starts with de�ning data ownership and responsibility. According to the Data Governance Institute, data doesn’t belong to individuals, but is an asset to the enterprise. As such some organizations assign “owners” to data making individuals or preferably teams responsible for data subsets utilized for operational e�ciency.

Organizational culture also entails the reduction of departmental segregation and the need to balance out disconnects between operational and IT departments.

“This must be done by the utility organization itself,” says John D. McDonald, Director, Technical Strategy and Policy Development at GE Digital Energy. “One suggestion is for the operational department to provide several of its people to the IT department to work for one to two years. Likewise, IT does the same to operations. In this way, each department will have a better understanding of the other department, and have close friendships in the other department.”

McDonald also recommends the utility should have a knowledge management system in place to capture this knowledge from workforce retirees to0 allow for incorporation of better standard engineering designs.

In the past, utilities tended to focus business practices on a single functional area, but in today’s integrated systems, these processes cross several boundaries to create synergies. Thus, these synergies challenge the utility’s traditionally siloed functional organization as the line between operations technology (OT) and utility information technology (IT) blurs. McDonald recommends utility management discuss the need to work holistically across siloed groups. This includes having the CEO “strongly encourage” or mandate holistic work. “Make sure the project managers for the Smart Grid projects have the responsibility and multi-discipline authority (across siloed groups). With the implementation of new technology there will be revision of existing business processes and changes in organizational structure and new skill sets needed.”

Also crucial to organizational change is the utility company’s ability to assess what data to collect. In order to do this management must �rst identify the business groups (asset management, power quality, maintenance, etc.) within the utility. Second, each business group must identify their data needs – both operational and non-operational data. This includes attributes of each data point – such as how often the data point is needed and frequency of update. Then operational data levels of importance are assigned to di�erent scan groups (di�erent scan rates).

Maximizing the value of smart grid systems involves bringing disparate data under a uni�ed management approach. Operations mangers must determine what data to collect, how to manage it, who should have access to speci�c data and how that data is used and stored.

This kind of organizational change demands job redesign, training and new skill development.

But just as legacy roles and responsibilities have to be modi�ed, so do legacy systems. This presents its own integration challenges. Many legacy systems were not

designed to manage signi�cant data loads or speci�c attributes that are needed to gain desired operational bene�ts, while at the same time maintaining ongoing reliable operations.

Many customized systems are no longer supported by the original vendors. This makes for high risk associated with breaking open the applications or systems for modi�cation.

Southern California Edison Manager of Enterprise Architecture James Horstman said this has not been a major issue for the utility. “Up until a few years ago the majority of our business applications were internally developed and supported. While we have gone more to a COTS (commercial o�-the-shelf ) approach the vendors we have worked with are all supported by the vendors,” he said.

Horstman did allow there were a few instances of smaller applications losing support but those were typically replaced without signi�cant risks coming to bear. “Typically our contracts with vendors include escrow provisions for providing the software to us in the event of non support. I can only think of one instance where that actually resulted in us taking over the software and it was not a critical application,” he said.

Another utility executive said his company looks at this challenge on a case-to-case basis. “You accept the fact and either change process to match the new version or accept the current version for all its issues,” he said.

In total, changing organizational culture into a data driven utility is crucial for utilizing the big data analytics to make a utility run smoother which translates into a larger ROI.

According to GTM research the cumulative spending power on utility data analytics is expected to top $18 billion over the next six years with an annual spend of $3.8 billion globally by 2020. The report also forecasts U.S. utilities alone will spend nearly $100 per home in grid operations and consumer-related analytics over the next six years.

The biggest growth market for vendors in utilities is on analytics software (soft grid) that supply systems that allow utilities to track, visualize and predict everything from grid operations to energy consumption. This market includes the biggest names in IT, who are positioning themselves as premiere vendors, but several seminal start-up and smaller companies are also o�ering custom speci�c analytic solutions.

In selecting a vendor, the best place to start, according to McDonald, is in evaluating the utility’s needs. “Identify the missing technology components needed for an integrated solution, and consider companies who have this technology. Frequently, these are start-up companies,” he said.

Other evaluation methods for selecting vendors include:

■ Get an overview of the latest big data solutions and plug in how they would work for the utility.

■ Assess the cost and risk factors associated with smart grid and big data investments and match them with proposed vendor solutions

■ Put together an RFP (Request for proposals) to select vendors to address the scope of the utilities’ needs and have vendors respond with solution plans and costs including possible cost overruns.

Utilities are unique in that the nature of harnessing, storing, supplying and distributing electric, gas and water is critical to the infrastructure of the entire globe.

Therefore, the use of analytics demands levels of reliability and safety far greater than typical personal, business and industrial requirements.

The three analytic subsets often used within the utility industry are:

■ Descriptive analytics that gauge current performance■ Predictive analytics that tell utilities what’s about to go wrong ■ Prescriptive analytics that point to problem prevention.

Analytics in utilities are used to improve reliability in business operations by preventing re-work, improving outage statistics like System Average Interruption Frequency Index (SAIFI), replacing overloaded transformers before they cause outages and identifying equipment malfunctions and cost trends.

Some �elds where analytics improve reliability include:

■ Predictive analytics are utilized for the maintenance and modernization of an aging infrastructure and for improved visibility across automated systems

■ Descriptive analytics leverage real time data for smart control to enable situational awareness in operations

■ Prescriptive analytics identify gaps in existing assets and establish sound asset management practices and programs.

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES | 12www.smartgridupdate.com

MA

RKET O

VERV

IEW

Analytics are also used to drive e�ciency by mining data for common indicators of problems like slow meters, high usage, or failing feeders. This includes using sophisticated algorithms to identify, for instance, the causes of load ine�ciencies or missed opportunities for conservation program outreach. Analytics also creates e�ciencies by:

■ Utilizing predictive customer analytics for successful rollout of demand/supply programs

■ Predicting demand and supply, which reduces outages by geographic area with higher e�ciency

■ Hold down energy costs and o�er enhanced services through smart integrated control systems

■ Boost consumer engagement through new communications initiatives related to Smart Grid programs

■ Leverage data to improve relationship with customers through direct marketing programs and new tailored services

National Grid Vice President of Asset Management Cheri Warren said the introduction and availability of both real time and detailed operational performance information introduces opportunities to understand the state of the utility’s system and its performance at a lower level of the network than they have previously had available.

“National Grid will leverage this information in several ways. Through more detailed short term outage information, the company can better identify potential infrastructure issues or better understand customer experience and causes. By collecting this big data for multiple periods or seasons, long term asset performance will greatly improve as the data will inform the performance, health, network design, asset policy and peak needs of the network and identify investments that will enhance the resiliency and reliability,” Warren said.

Warren added that National Grid has been exploring integrating other operational and customer data from other source systems into a common "data fusion" database. “Bringing together information such as customer relationship management, asset performance, work management and smart grid data will produce new insights to help drive improved asset management policies, asset planning practices and customer experience,” she said.

Warren said smart grid technology allows utilities to gather information about system performance at a �ner level of accuracy. “Using communication and location capabilities National Grid will be better positioned to ensure the accuracy of asset registers and gather very detailed network operation measurements. Using these and other pieces of information, National Grid will be able to enhance our network modeling tools and techniques, moving away from estimates for data models, and allowing us to more precisely identify asset underperformance issues in existing assets to further focus investments. Making smarter investment decisions will help drive increased reliability, performance and power quality.”

Protecting critical assets and ensuring compliance is long a part of standard operating procedure for utilities. However, new challenges have arisen with the new game-changing smart technology initiatives and heightened public awareness of privacy concerns. And as data threats multiply, so do compliance requirements. Ever changing legislative and regulatory activity related to data protection is at an all time high from both federal and state governments as well as from public utility commissions about the privacy implications of smart meters.

Some key elements of e�ective privacy are:

■ Understand your company’s compliance and culture

■ Align and train management and sta� on security practices

■ Know your data, where it is, and what must be protected

■ Ensure third parties comply with your privacy policies

■ Understand your threats and controls

■ Test and update controls regularly

■ Be prepared to respond to incidents

Additionally, utilities must keep up with compliance standards relating to security as set by the North American Electric Reliability Corporation (NERC), which acts at the direction of the Federal Energy Regulatory Commission (FERC) - a federal organization overseeing interstate transportation and marketing of energy.

NERC is responsible for establishing reliability standards for the Bulk Electric System (aka the power grid), which has become an increasingly tempting target for cyber terrorism and nation level threats. NERC's Critical Infrastructure Protection (CIP) Standards identify the minimum cyber controls and protections which power suppliers and generators must address or face signi�cant penalties and �nes.

The NERC CIP compliance standards are organized into nine (9) key areas:

■ Sabotage

■ Critical Asset Identi�cation

■ Security Management Controls

■ Personnel and Training

■ Electronic Security Perimeter

■ Physical Security Protection

■ Systems Security Management

■ Incident Reporting and Response Planning

■ Recovery Plans

Smart technology and its associated big data management and analytics have transformed the industry with many avenues to ensure a growing ROI. This new technology has streamlined and improved techniques for distributing safe and clean energy for the global marketplace.

This technology has also created a growing cyber terrorism risk for enemies that understand all too well that the world depends and functions on utility infrastructure. It has also created challenges for the utility industry to safeguard privacy while at the same time continuing its mission of providing safe electricity, gas and water.

The complexity of delivering services on a mass scale has made utilities slow to change in keeping pace with new technologies, but both consumers in the marketplace and government, through compliance mandates and regulations, dictate that smart grids are necessary for sustained energy needs. In facing these challenges, utilities realize it means not only deploying and integrating smart technology, but changing organizational structure.

Data analytics has brought a new precision to understanding and improving operational e�ciency and reliability. Whether developed in-house or increasingly through soft grid vendors o�ering various solutions, data analytics has shifted organizational culture as it continues to create synergies and bring disparate data under a uni�ed management approach.

The future of data analytics continues to evolve as utility companies explore new uses such as developing a common “data fusion” database integrating other operational and customer data from other source systems into a common system.

The role of data analytics in the utility industry is now moving towards similar roles it plays in the telecommunications and �nance sectors. Due to the industry’s key role in global infrastructure it is critical to continue developing new uses and strategies for this cutting-edge technology.

9. Drive Efficiency

The rise of big data poses the challenge of where to put the data and what to do with it. The answer is in the creation of a dedicated data center, of which there are several options, all of which have their pros and cons.

Some of these options include: ■ Data center facilities – These facilities are dedicated, custom-built housing and are usually customer-owned or leased from telecommunications or IT service providers. While these facilities usually have some form of contingencies for disaster recovery, they often increase the levels of capital expenditures.

■ Storage arrays – These provide data storage functionality and the current main vendors are EMC Corporation, NetApp and IBM.

■ Server platforms – This is computing hardware needed for data, and the Main vendors are HP, IBM and Dell.

■ Storage area network equipment – This connects server, storage and external network resources, and the main vendors are Brocade, Cisco and QLogic.

■ Database systems – This is software systems for data management and analysis. The main vendors are Oracle, IBM and Microsoft.

■ Virtualization systems – These systems allow more e�cient use of discrete storage and computing resources. The main vendors are VMware, Citrix and Microsoft.

Corresponding to the storage of big data are the analytics options in interpreting the data. Some of these options include:

■ Develop systems in house – The bene�t to this is the system is highly tailored to the utility’s need. On the downside there is often a lack of in-house development skills and resources to create a fully-�edged a data analytics system from scratch.

■ Rely on OT vendor system – The advantage of relying on OT vendors for data analytics is that these companies are well versed in the issues facing utilities and can therefore provide products that are highly relevant to the industry. On the downside the analytics and IT integration capabilities might not align well with IT vendors.

■ Rely on IT vendor systems – The advantages are a good integration with IT systems with good design and support. On the downside there may be an increased cost for unnecessary features and a lack of alignment with the operational side.

■ Rely on point products or pure-play vendor system. This produces close alignment with operational processes and good integration with OT, but there’s a potential lack of IT system integration and lifetime support concerns.

■ Rely on third-party data analytics service provider system. There is a reduced cost and improved analytics, but it is an evolving concept with a limited track record and data sharing issues.

Along with assuring a ROI in smart grid deployment through big data management and analytics systems, this new digital paradigm also requires creating a new organizational foundation for data and analytics initiatives through a workplace culture change.

This starts with de�ning data ownership and responsibility. According to the Data Governance Institute, data doesn’t belong to individuals, but is an asset to the enterprise. As such some organizations assign “owners” to data making individuals or preferably teams responsible for data subsets utilized for operational e�ciency.

Organizational culture also entails the reduction of departmental segregation and the need to balance out disconnects between operational and IT departments.

“This must be done by the utility organization itself,” says John D. McDonald, Director, Technical Strategy and Policy Development at GE Digital Energy. “One suggestion is for the operational department to provide several of its people to the IT department to work for one to two years. Likewise, IT does the same to operations. In this way, each department will have a better understanding of the other department, and have close friendships in the other department.”

McDonald also recommends the utility should have a knowledge management system in place to capture this knowledge from workforce retirees to0 allow for incorporation of better standard engineering designs.

In the past, utilities tended to focus business practices on a single functional area, but in today’s integrated systems, these processes cross several boundaries to create synergies. Thus, these synergies challenge the utility’s traditionally siloed functional organization as the line between operations technology (OT) and utility information technology (IT) blurs. McDonald recommends utility management discuss the need to work holistically across siloed groups. This includes having the CEO “strongly encourage” or mandate holistic work. “Make sure the project managers for the Smart Grid projects have the responsibility and multi-discipline authority (across siloed groups). With the implementation of new technology there will be revision of existing business processes and changes in organizational structure and new skill sets needed.”

Also crucial to organizational change is the utility company’s ability to assess what data to collect. In order to do this management must �rst identify the business groups (asset management, power quality, maintenance, etc.) within the utility. Second, each business group must identify their data needs – both operational and non-operational data. This includes attributes of each data point – such as how often the data point is needed and frequency of update. Then operational data levels of importance are assigned to di�erent scan groups (di�erent scan rates).

Maximizing the value of smart grid systems involves bringing disparate data under a uni�ed management approach. Operations mangers must determine what data to collect, how to manage it, who should have access to speci�c data and how that data is used and stored.

This kind of organizational change demands job redesign, training and new skill development.

But just as legacy roles and responsibilities have to be modi�ed, so do legacy systems. This presents its own integration challenges. Many legacy systems were not

designed to manage signi�cant data loads or speci�c attributes that are needed to gain desired operational bene�ts, while at the same time maintaining ongoing reliable operations.

Many customized systems are no longer supported by the original vendors. This makes for high risk associated with breaking open the applications or systems for modi�cation.

Southern California Edison Manager of Enterprise Architecture James Horstman said this has not been a major issue for the utility. “Up until a few years ago the majority of our business applications were internally developed and supported. While we have gone more to a COTS (commercial o�-the-shelf ) approach the vendors we have worked with are all supported by the vendors,” he said.

Horstman did allow there were a few instances of smaller applications losing support but those were typically replaced without signi�cant risks coming to bear. “Typically our contracts with vendors include escrow provisions for providing the software to us in the event of non support. I can only think of one instance where that actually resulted in us taking over the software and it was not a critical application,” he said.

Another utility executive said his company looks at this challenge on a case-to-case basis. “You accept the fact and either change process to match the new version or accept the current version for all its issues,” he said.

In total, changing organizational culture into a data driven utility is crucial for utilizing the big data analytics to make a utility run smoother which translates into a larger ROI.

According to GTM research the cumulative spending power on utility data analytics is expected to top $18 billion over the next six years with an annual spend of $3.8 billion globally by 2020. The report also forecasts U.S. utilities alone will spend nearly $100 per home in grid operations and consumer-related analytics over the next six years.

The biggest growth market for vendors in utilities is on analytics software (soft grid) that supply systems that allow utilities to track, visualize and predict everything from grid operations to energy consumption. This market includes the biggest names in IT, who are positioning themselves as premiere vendors, but several seminal start-up and smaller companies are also o�ering custom speci�c analytic solutions.

In selecting a vendor, the best place to start, according to McDonald, is in evaluating the utility’s needs. “Identify the missing technology components needed for an integrated solution, and consider companies who have this technology. Frequently, these are start-up companies,” he said.

Other evaluation methods for selecting vendors include:

■ Get an overview of the latest big data solutions and plug in how they would work for the utility.

■ Assess the cost and risk factors associated with smart grid and big data investments and match them with proposed vendor solutions

■ Put together an RFP (Request for proposals) to select vendors to address the scope of the utilities’ needs and have vendors respond with solution plans and costs including possible cost overruns.

Utilities are unique in that the nature of harnessing, storing, supplying and distributing electric, gas and water is critical to the infrastructure of the entire globe.

Therefore, the use of analytics demands levels of reliability and safety far greater than typical personal, business and industrial requirements.

The three analytic subsets often used within the utility industry are:

■ Descriptive analytics that gauge current performance■ Predictive analytics that tell utilities what’s about to go wrong ■ Prescriptive analytics that point to problem prevention.

Analytics in utilities are used to improve reliability in business operations by preventing re-work, improving outage statistics like System Average Interruption Frequency Index (SAIFI), replacing overloaded transformers before they cause outages and identifying equipment malfunctions and cost trends.

Some �elds where analytics improve reliability include:

■ Predictive analytics are utilized for the maintenance and modernization of an aging infrastructure and for improved visibility across automated systems

■ Descriptive analytics leverage real time data for smart control to enable situational awareness in operations

■ Prescriptive analytics identify gaps in existing assets and establish sound asset management practices and programs.

Analytics are also used to drive e�ciency by mining data for common indicators of problems like slow meters, high usage, or failing feeders. This includes using sophisticated algorithms to identify, for instance, the causes of load ine�ciencies or missed opportunities for conservation program outreach. Analytics also creates e�ciencies by:

■ Utilizing predictive customer analytics for successful rollout of demand/supply programs

■ Predicting demand and supply, which reduces outages by geographic area with higher e�ciency

■ Hold down energy costs and o�er enhanced services through smart integrated control systems

■ Boost consumer engagement through new communications initiatives related to Smart Grid programs

■ Leverage data to improve relationship with customers through direct marketing programs and new tailored services

National Grid Vice President of Asset Management Cheri Warren said the introduction and availability of both real time and detailed operational performance information introduces opportunities to understand the state of the utility’s system and its performance at a lower level of the network than they have previously had available.

“National Grid will leverage this information in several ways. Through more detailed short term outage information, the company can better identify potential infrastructure issues or better understand customer experience and causes. By collecting this big data for multiple periods or seasons, long term asset performance will greatly improve as the data will inform the performance, health, network design, asset policy and peak needs of the network and identify investments that will enhance the resiliency and reliability,” Warren said.

Warren added that National Grid has been exploring integrating other operational and customer data from other source systems into a common "data fusion" database. “Bringing together information such as customer relationship management, asset performance, work management and smart grid data will produce new insights to help drive improved asset management policies, asset planning practices and customer experience,” she said.

Warren said smart grid technology allows utilities to gather information about system performance at a �ner level of accuracy. “Using communication and location capabilities National Grid will be better positioned to ensure the accuracy of asset registers and gather very detailed network operation measurements. Using these and other pieces of information, National Grid will be able to enhance our network modeling tools and techniques, moving away from estimates for data models, and allowing us to more precisely identify asset underperformance issues in existing assets to further focus investments. Making smarter investment decisions will help drive increased reliability, performance and power quality.”

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES | 13www.smartgridupdate.com

MA

RKET O

VERV

IEW

Protecting critical assets and ensuring compliance is long a part of standard operating procedure for utilities. However, new challenges have arisen with the new game-changing smart technology initiatives and heightened public awareness of privacy concerns. And as data threats multiply, so do compliance requirements. Ever changing legislative and regulatory activity related to data protection is at an all time high from both federal and state governments as well as from public utility commissions about the privacy implications of smart meters.

Some key elements of e�ective privacy are:

■ Understand your company’s compliance and culture

■ Align and train management and sta� on security practices

■ Know your data, where it is, and what must be protected

■ Ensure third parties comply with your privacy policies

■ Understand your threats and controls

■ Test and update controls regularly

■ Be prepared to respond to incidents

Additionally, utilities must keep up with compliance standards relating to security as set by the North American Electric Reliability Corporation (NERC), which acts at the direction of the Federal Energy Regulatory Commission (FERC) - a federal organization overseeing interstate transportation and marketing of energy.

NERC is responsible for establishing reliability standards for the Bulk Electric System (aka the power grid), which has become an increasingly tempting target for cyber terrorism and nation level threats. NERC's Critical Infrastructure Protection (CIP) Standards identify the minimum cyber controls and protections which power suppliers and generators must address or face signi�cant penalties and �nes.

The NERC CIP compliance standards are organized into nine (9) key areas:

■ Sabotage

■ Critical Asset Identi�cation

■ Security Management Controls

■ Personnel and Training

■ Electronic Security Perimeter

■ Physical Security Protection

■ Systems Security Management

■ Incident Reporting and Response Planning

■ Recovery Plans

Smart technology and its associated big data management and analytics have transformed the industry with many avenues to ensure a growing ROI. This new technology has streamlined and improved techniques for distributing safe and clean energy for the global marketplace.

This technology has also created a growing cyber terrorism risk for enemies that understand all too well that the world depends and functions on utility infrastructure. It has also created challenges for the utility industry to safeguard privacy while at the same time continuing its mission of providing safe electricity, gas and water.

The complexity of delivering services on a mass scale has made utilities slow to change in keeping pace with new technologies, but both consumers in the marketplace and government, through compliance mandates and regulations, dictate that smart grids are necessary for sustained energy needs. In facing these challenges, utilities realize it means not only deploying and integrating smart technology, but changing organizational structure.

Data analytics has brought a new precision to understanding and improving operational e�ciency and reliability. Whether developed in-house or increasingly through soft grid vendors o�ering various solutions, data analytics has shifted organizational culture as it continues to create synergies and bring disparate data under a uni�ed management approach.

The future of data analytics continues to evolve as utility companies explore new uses such as developing a common “data fusion” database integrating other operational and customer data from other source systems into a common system.

The role of data analytics in the utility industry is now moving towards similar roles it plays in the telecommunications and �nance sectors. Due to the industry’s key role in global infrastructure it is critical to continue developing new uses and strategies for this cutting-edge technology.

10. Privacy, fraud protection and Cyber security risks

The rise of big data poses the challenge of where to put the data and what to do with it. The answer is in the creation of a dedicated data center, of which there are several options, all of which have their pros and cons.

Some of these options include: ■ Data center facilities – These facilities are dedicated, custom-built housing and are usually customer-owned or leased from telecommunications or IT service providers. While these facilities usually have some form of contingencies for disaster recovery, they often increase the levels of capital expenditures.

■ Storage arrays – These provide data storage functionality and the current main vendors are EMC Corporation, NetApp and IBM.

■ Server platforms – This is computing hardware needed for data, and the Main vendors are HP, IBM and Dell.

■ Storage area network equipment – This connects server, storage and external network resources, and the main vendors are Brocade, Cisco and QLogic.

■ Database systems – This is software systems for data management and analysis. The main vendors are Oracle, IBM and Microsoft.

■ Virtualization systems – These systems allow more e�cient use of discrete storage and computing resources. The main vendors are VMware, Citrix and Microsoft.

Corresponding to the storage of big data are the analytics options in interpreting the data. Some of these options include:

■ Develop systems in house – The bene�t to this is the system is highly tailored to the utility’s need. On the downside there is often a lack of in-house development skills and resources to create a fully-�edged a data analytics system from scratch.

■ Rely on OT vendor system – The advantage of relying on OT vendors for data analytics is that these companies are well versed in the issues facing utilities and can therefore provide products that are highly relevant to the industry. On the downside the analytics and IT integration capabilities might not align well with IT vendors.

■ Rely on IT vendor systems – The advantages are a good integration with IT systems with good design and support. On the downside there may be an increased cost for unnecessary features and a lack of alignment with the operational side.

■ Rely on point products or pure-play vendor system. This produces close alignment with operational processes and good integration with OT, but there’s a potential lack of IT system integration and lifetime support concerns.

■ Rely on third-party data analytics service provider system. There is a reduced cost and improved analytics, but it is an evolving concept with a limited track record and data sharing issues.

Along with assuring a ROI in smart grid deployment through big data management and analytics systems, this new digital paradigm also requires creating a new organizational foundation for data and analytics initiatives through a workplace culture change.

This starts with de�ning data ownership and responsibility. According to the Data Governance Institute, data doesn’t belong to individuals, but is an asset to the enterprise. As such some organizations assign “owners” to data making individuals or preferably teams responsible for data subsets utilized for operational e�ciency.

Organizational culture also entails the reduction of departmental segregation and the need to balance out disconnects between operational and IT departments.

“This must be done by the utility organization itself,” says John D. McDonald, Director, Technical Strategy and Policy Development at GE Digital Energy. “One suggestion is for the operational department to provide several of its people to the IT department to work for one to two years. Likewise, IT does the same to operations. In this way, each department will have a better understanding of the other department, and have close friendships in the other department.”

McDonald also recommends the utility should have a knowledge management system in place to capture this knowledge from workforce retirees to0 allow for incorporation of better standard engineering designs.

In the past, utilities tended to focus business practices on a single functional area, but in today’s integrated systems, these processes cross several boundaries to create synergies. Thus, these synergies challenge the utility’s traditionally siloed functional organization as the line between operations technology (OT) and utility information technology (IT) blurs. McDonald recommends utility management discuss the need to work holistically across siloed groups. This includes having the CEO “strongly encourage” or mandate holistic work. “Make sure the project managers for the Smart Grid projects have the responsibility and multi-discipline authority (across siloed groups). With the implementation of new technology there will be revision of existing business processes and changes in organizational structure and new skill sets needed.”

Also crucial to organizational change is the utility company’s ability to assess what data to collect. In order to do this management must �rst identify the business groups (asset management, power quality, maintenance, etc.) within the utility. Second, each business group must identify their data needs – both operational and non-operational data. This includes attributes of each data point – such as how often the data point is needed and frequency of update. Then operational data levels of importance are assigned to di�erent scan groups (di�erent scan rates).

Maximizing the value of smart grid systems involves bringing disparate data under a uni�ed management approach. Operations mangers must determine what data to collect, how to manage it, who should have access to speci�c data and how that data is used and stored.

This kind of organizational change demands job redesign, training and new skill development.

But just as legacy roles and responsibilities have to be modi�ed, so do legacy systems. This presents its own integration challenges. Many legacy systems were not

designed to manage signi�cant data loads or speci�c attributes that are needed to gain desired operational bene�ts, while at the same time maintaining ongoing reliable operations.

Many customized systems are no longer supported by the original vendors. This makes for high risk associated with breaking open the applications or systems for modi�cation.

Southern California Edison Manager of Enterprise Architecture James Horstman said this has not been a major issue for the utility. “Up until a few years ago the majority of our business applications were internally developed and supported. While we have gone more to a COTS (commercial o�-the-shelf ) approach the vendors we have worked with are all supported by the vendors,” he said.

Horstman did allow there were a few instances of smaller applications losing support but those were typically replaced without signi�cant risks coming to bear. “Typically our contracts with vendors include escrow provisions for providing the software to us in the event of non support. I can only think of one instance where that actually resulted in us taking over the software and it was not a critical application,” he said.

Another utility executive said his company looks at this challenge on a case-to-case basis. “You accept the fact and either change process to match the new version or accept the current version for all its issues,” he said.

In total, changing organizational culture into a data driven utility is crucial for utilizing the big data analytics to make a utility run smoother which translates into a larger ROI.

According to GTM research the cumulative spending power on utility data analytics is expected to top $18 billion over the next six years with an annual spend of $3.8 billion globally by 2020. The report also forecasts U.S. utilities alone will spend nearly $100 per home in grid operations and consumer-related analytics over the next six years.

The biggest growth market for vendors in utilities is on analytics software (soft grid) that supply systems that allow utilities to track, visualize and predict everything from grid operations to energy consumption. This market includes the biggest names in IT, who are positioning themselves as premiere vendors, but several seminal start-up and smaller companies are also o�ering custom speci�c analytic solutions.

In selecting a vendor, the best place to start, according to McDonald, is in evaluating the utility’s needs. “Identify the missing technology components needed for an integrated solution, and consider companies who have this technology. Frequently, these are start-up companies,” he said.

Other evaluation methods for selecting vendors include:

■ Get an overview of the latest big data solutions and plug in how they would work for the utility.

■ Assess the cost and risk factors associated with smart grid and big data investments and match them with proposed vendor solutions

■ Put together an RFP (Request for proposals) to select vendors to address the scope of the utilities’ needs and have vendors respond with solution plans and costs including possible cost overruns.

Utilities are unique in that the nature of harnessing, storing, supplying and distributing electric, gas and water is critical to the infrastructure of the entire globe.

Therefore, the use of analytics demands levels of reliability and safety far greater than typical personal, business and industrial requirements.

The three analytic subsets often used within the utility industry are:

■ Descriptive analytics that gauge current performance■ Predictive analytics that tell utilities what’s about to go wrong ■ Prescriptive analytics that point to problem prevention.

Analytics in utilities are used to improve reliability in business operations by preventing re-work, improving outage statistics like System Average Interruption Frequency Index (SAIFI), replacing overloaded transformers before they cause outages and identifying equipment malfunctions and cost trends.

Some �elds where analytics improve reliability include:

■ Predictive analytics are utilized for the maintenance and modernization of an aging infrastructure and for improved visibility across automated systems

■ Descriptive analytics leverage real time data for smart control to enable situational awareness in operations

■ Prescriptive analytics identify gaps in existing assets and establish sound asset management practices and programs.

Analytics are also used to drive e�ciency by mining data for common indicators of problems like slow meters, high usage, or failing feeders. This includes using sophisticated algorithms to identify, for instance, the causes of load ine�ciencies or missed opportunities for conservation program outreach. Analytics also creates e�ciencies by:

■ Utilizing predictive customer analytics for successful rollout of demand/supply programs

■ Predicting demand and supply, which reduces outages by geographic area with higher e�ciency

■ Hold down energy costs and o�er enhanced services through smart integrated control systems

■ Boost consumer engagement through new communications initiatives related to Smart Grid programs

■ Leverage data to improve relationship with customers through direct marketing programs and new tailored services

National Grid Vice President of Asset Management Cheri Warren said the introduction and availability of both real time and detailed operational performance information introduces opportunities to understand the state of the utility’s system and its performance at a lower level of the network than they have previously had available.

“National Grid will leverage this information in several ways. Through more detailed short term outage information, the company can better identify potential infrastructure issues or better understand customer experience and causes. By collecting this big data for multiple periods or seasons, long term asset performance will greatly improve as the data will inform the performance, health, network design, asset policy and peak needs of the network and identify investments that will enhance the resiliency and reliability,” Warren said.

Warren added that National Grid has been exploring integrating other operational and customer data from other source systems into a common "data fusion" database. “Bringing together information such as customer relationship management, asset performance, work management and smart grid data will produce new insights to help drive improved asset management policies, asset planning practices and customer experience,” she said.

Warren said smart grid technology allows utilities to gather information about system performance at a �ner level of accuracy. “Using communication and location capabilities National Grid will be better positioned to ensure the accuracy of asset registers and gather very detailed network operation measurements. Using these and other pieces of information, National Grid will be able to enhance our network modeling tools and techniques, moving away from estimates for data models, and allowing us to more precisely identify asset underperformance issues in existing assets to further focus investments. Making smarter investment decisions will help drive increased reliability, performance and power quality.”

Protecting critical assets and ensuring compliance is long a part of standard operating procedure for utilities. However, new challenges have arisen with the new game-changing smart technology initiatives and heightened public awareness of privacy concerns. And as data threats multiply, so do compliance requirements. Ever changing legislative and regulatory activity related to data protection is at an all time high from both federal and state governments as well as from public utility commissions about the privacy implications of smart meters.

Some key elements of e�ective privacy are:

■ Understand your company’s compliance and culture

■ Align and train management and sta� on security practices

■ Know your data, where it is, and what must be protected

■ Ensure third parties comply with your privacy policies

■ Understand your threats and controls

■ Test and update controls regularly

■ Be prepared to respond to incidents

Additionally, utilities must keep up with compliance standards relating to security as set by the North American Electric Reliability Corporation (NERC), which acts at the direction of the Federal Energy Regulatory Commission (FERC) - a federal organization overseeing interstate transportation and marketing of energy.

NERC is responsible for establishing reliability standards for the Bulk Electric System (aka the power grid), which has become an increasingly tempting target for cyber terrorism and nation level threats. NERC's Critical Infrastructure Protection (CIP) Standards identify the minimum cyber controls and protections which power suppliers and generators must address or face signi�cant penalties and �nes.

The NERC CIP compliance standards are organized into nine (9) key areas:

■ Sabotage

■ Critical Asset Identi�cation

■ Security Management Controls

■ Personnel and Training

■ Electronic Security Perimeter

■ Physical Security Protection

■ Systems Security Management

■ Incident Reporting and Response Planning

■ Recovery Plans

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES | 14www.smartgridupdate.com

MA

RKET O

VERV

IEW

Smart technology and its associated big data management and analytics have transformed the industry with many avenues to ensure a growing ROI. This new technology has streamlined and improved techniques for distributing safe and clean energy for the global marketplace.

This technology has also created a growing cyber terrorism risk for enemies that understand all too well that the world depends and functions on utility infrastructure. It has also created challenges for the utility industry to safeguard privacy while at the same time continuing its mission of providing safe electricity, gas and water.

The complexity of delivering services on a mass scale has made utilities slow to change in keeping pace with new technologies, but both consumers in the marketplace and government, through compliance mandates and regulations, dictate that smart grids are necessary for sustained energy needs. In facing these challenges, utilities realize it means not only deploying and integrating smart technology, but changing organizational structure.

Data analytics has brought a new precision to understanding and improving operational e�ciency and reliability. Whether developed in-house or increasingly through soft grid vendors o�ering various solutions, data analytics has shifted organizational culture as it continues to create synergies and bring disparate data under a uni�ed management approach.

The future of data analytics continues to evolve as utility companies explore new uses such as developing a common “data fusion” database integrating other operational and customer data from other source systems into a common system.

The role of data analytics in the utility industry is now moving towards similar roles it plays in the telecommunications and �nance sectors. Due to the industry’s key role in global infrastructure it is critical to continue developing new uses and strategies for this cutting-edge technology.

11. Concluding Remarks

DATA MANAGEMENT AND ANALYTICS FOR UTILITIES | 15www.smartgridupdate.com

MA

RKET O

VERV

IEW

http://www.datagovernance.com/gbg_assigning_data_ownership.html

http://www.zdnet.com/oracle-study-utilities-still-not-seizing-smart-grid-data-opportunity-7000018081/ July 23, 2013

https://www.greentechmedia.com/articles/read/Space-Time-Insight-Lands-Hydro-One-Launches-Predictive-Analytics

https://www.greentechmedia.com/articles/read/new-report-utility-data-analytics-market-to-hit-3.8-billion-annually-by-202

http://www.coal�re.com/NERC-CIP-Assessments

http://data-informed.com/utility-project-applies-predictive-analytics-to-slice-of-paci�c-northwest-power-grid/

http://www.ibm.com/expressadvantage/br/downloads/Asset_management_in_the_utilities_industry.pdf

http://www.nema.org/Policy/Energy/Smartgrid/Documents/NEMA-SGC-ROI.pdf

http://www.sas.com/news/analysts/Soft_Grid_2013_2020_Big_Data_Utility_Analytics_Smart_Grid.pdf

http://www.pwc.com/en_US/us/power-and-utilities/publications/assets/pwc-privacy-risks-data-protection-landscape.pdf

12. References