task force on

TASK FORCE ON

Measurement and Verification for Conservation Voltage Reduction

This report results from a collaborative effort among the CVR M&V TF members.

Task Force Committee: Shakawat (Shakil) Hossan, ComEd (Chair) Amin Khodaei, University of Denver (Vice-Chair) Wen Fan, ComEd (Secretary)

Task Force Members & Co-Authors:Ahmed Saber Operation Technology, Inc.Andrew Wilson Xcel EnergyMesut Baran North Carolina State UniversityEric Gupta Xcel EnergySepideh Fard ComEdMike Simms Duke EnergyVictor (Shengyang) He BGEEric Wedemeyer BGEHamid Abdurrahim BGEYuan Liao University of KentuckyZhaoyu Wang Iowa State UniversityZinn Morton PEPCODamien Tholomier VarentecDongbo Zhao Argonne National LaboratoryTu Anh Tran Reinhausen Manufacturing Inc. Farnoosh Rahmatian NuGrid PowerFei Ding National Renewable Energy LaboratoryAlfredo Contreras Southern California Edison

Acknowledgment of Special Contributions

The Task Force gratefully acknowledges the feedback of the Volt-var working group members and their support toestablish this Task Force/Study Team. The Task Force also acknowledges the tremendous support of theparticipating utilities: Commonwealth Edison Co., Duke Energy, Baltimore Gas and Electric (BG&E), SouthernCalifornia Edison, and Xcel Energy by providing data to study the need for a CVR M&V Standard orRecommended Practice. The Task Force is also thankful to professor Zhaoyu Wang of Iowa State University,Professor Yuan Liao of University of Kentucky, Ninalowo Sainab, Honghao Zheng, and Esa Aleksi Paaso ofCommonwealth Edison for their technical support and guidance while designing the survey and conductingthe study.

© IEEE 2021 The Institute of Electrical and Electronic Engineers, Inc.

2

KEYWORDS: Conservation voltage reduction, data quality, measurement and verification, volt-var optimization

TABLE OF CONTENTS

LIST OF FIGURES..........................................................................................................................................................4

LIST OF Tables..............................................................................................................................................................6

LIST OF ACRONYMS AND ABBREVIATIONS ............................................................................................................7

EXECUTIVE SUMMARY................................................................................................................................................8

1.1 Overview of CVR M&V.............................................................................................................................10

1.2 Utility Practices on M&V..........................................................................................................................10

1.3 Motivation of This Task Force.................................................................................................................10

1.4 Focus of the Survey and Study Conducted............................................................................................11

1.5 Intended Outcome of the Task Force.....................................................................................................11

2.1 Overview of Survey Parti cipants...........................................................................................................12

2.2 Type of Operati on and Data Quality....................................................................................................12

2.3 Uti lizati on of Data Points for M&V study...........................................................................................13

2.4 Uti lizati on of Methodologies for M&V analysis ................................................................................14

2.5 Commission Requirement......................................................................................................................15

2.6 Challenges in CVR Implementati on and Conducti ng M&V..............................................................15

2.7 Requirement of a Standard Procedure (Response from the Parti cipants) ..................................16

2.8 Key Takeaways of the Survey................................................................................................................16

2.9 Feedback for Study Design.....................................................................................................................17

3.1 Overview of the Data Received from the Uti liti es.............................................................................18

3.2 Data Cleaning Approaches.....................................................................................................................19

3.3 Data Quality Stati sti cs.............................................................................................................................20

3.3.1 Feeder Level.................................................................................................................................................20

3.3.2 Station Level.................................................................................................................................................20

3.4 Sample Methodologies Uti lized in the Study......................................................................................22

3.4.1 Temperature Correlated Comparison-based Methodology..........................................................................23

3.4.2 Linear Regression-Based Methodology........................................................................................................24

3

3.4.3 Calculation of energy baselines consumption...............................................................................................24

3.5 Scenarios....................................................................................................................................................25

3.6 Results........................................................................................................................................................32

3.6.1 Finding 1: Difference in Evaluation with Different Scenarios........................................................................32

3.6.2 Finding 2: Overlapping of Voltage Data in Different CVR Modes..................................................................34

3.6.3 Finding 3: Impact of Load Shift.....................................................................................................................36

3.6.4 Finding 4: Basis of Relatively High CVR Factor..............................................................................................36

3.6.5 Finding 5: Interruption in CVR Schedule.......................................................................................................39

3.6.6 Finding 6: Impact of Data Resolution in M&V...............................................................................................39

3.7 Summary....................................................................................................................................................40

4.1 Processes to be Streamlined..................................................................................................................41

4.2 Recommendation......................................................................................................................................41

REFERENCES................................................................................................................................................................42

4

LIST OF FIGURES

Fig. 2.1. Data quality issues faced by; a) Utilities, b) All participants....................................................................12

Fig. 2.2. Data Reconstruction (All participants)........................................................................................................12

Fig. 2.3. Different data resolutions used by participants........................................................................................13

Fig. 2.4. a) If participants utilized multiple resolutions; b) If participants observed discrepancies while using

multiple resolutions.....................................................................................................................................................13

Fig. 2.5. Discrepancies while using multiple methodologies..................................................................................14

Fig. 2.6. Challenges faced by different utilities........................................................................................................15

Fig. 3.1. Data quality statistics of feeders attached to the transformer 230 at U1 for the studied years of

2014, 2015, and 2016. (a) power (b) voltage............................................................................................................20

Fig. 3.2. Data quality statistics of feeders attached to the transformer 33 at U1 for the studied years of 2017

and 2018. (a) power (b) voltage.................................................................................................................................20

Fig. 3.3. Data quality statistics of U2 for the studied years of 2015 and 2017. (a) power (b) voltage ..............20

Fig. 3.4. Data quality statistics of U3 for the studied years 2013, 2014, and 2015. (a) power (b) voltage .......21

Fig. 3.5. Data quality statistics of U4 for the studied year 2020. (a) power (b) voltage ......................................21

Fig. 3.6. An example of clean and reconstructed power and voltage data against CVR status for feeder A1 in

scenario 1 – 24x7 CVR operation, original dataset..................................................................................................26

Fig. 3.7. An example of clean and reconstructed power and voltage data against CVR status for feeder E1 in

scenario 2 – CVR ON/OFF cycling, original dataset..................................................................................................27

Fig. 3.8. An example of clean and reconstructed power and voltage data against CVR status for feeder E1 in

scenario 3 – CVR ON/OFF cycling, modified dataset................................................................................................28

Fig. 3.9. An example of clean and reconstructed power and voltage data against CVR status for feeder A1 in

scenario 4 – 24x7 CVR operation, modified dataset................................................................................................29

Fig. 3.10. Voltage histogram with overlaps; a. U1; b. U2........................................................................................32

Fig. 3.11. Voltage histogram after cleaning overlap; a. U1; b. U2.........................................................................33

Fig. 3.12. Power consumption with load shift..........................................................................................................34

Fig. 3.13. Comparison between Temperature Vs a) mean power; b) mean voltage. ...........................................34

Fig. 3.14. (a) CVR OFF and ON Power Consumption; (b) CVR OFF and ON Operating Voltage; (c) Hourly

difference in Power and Voltage reduction..............................................................................................................35

Fig. 3.15. a) Voltage and status for feeder D1; b) Histogram to check voltage overlap. .....................................36

5

LIST OF Tables

Table 1.1. A summary of Utility practices on CVR M&V Methodologies..................................................................9

Table 2.1. Survey Participants.........................................................................................................................................11

Table 2.2. M&V methodologies used by the participants...............................................................................................13

Table 2.3. Utility Comments on updating CVR factor.....................................................................................................14

Table 3.1. Feeder and associated data characteristics....................................................................................................17

Table 3.2. Feeder and associated data quality statistics for certain studied years.........................................................19

Table 3.3. Data scenarios used in this study...................................................................................................................24

Table 3.4. An illustration of the lookup table.................................................................................................................24

Table 3.5. The priority chart for data reconstruction.....................................................................................................25

Table 3.6. Data scenarios with different data quality.....................................................................................................30

Table 3.7. CVR factor evaluation for cleaned data (24x7 operation)..............................................................................30

Table 3.8. CVR factor evaluation for reconstructed data (24x7 operation)....................................................................31

Table 3.9. Data scenarios with different data quality.....................................................................................................31

Table 3.10. CVR factor evaluation for cleaned data (cycling dataset).............................................................................31

Table 3.11. CVR factor evaluation for reconstructed data (cycling dataset)...................................................................32

Table 3.12. CVR factor differences with the cleaning of overlapped voltage.................................................................33

Table 3.13. Demonstration of load shift impact on analyses results..............................................................................33

Table 3.14. Summary of studied cases............................................................................................................................35

Table 3.15. CVR factor evaluation with interrupted CVR schedule.................................................................................37

Table 3.16. CVR factor evaluation with different data resolutions.................................................................................37

6

LIST OF ACRONYMS AND ABBREVIATIONS

7

AEP American Electric PowerAIC Ameren Illinois CompanyAMI Advanced Metering ANSI American National Standards

InstituteAPI Application Programming BCR Benefit-to-Cost RatioCDH Cooling Degree HoursComEd Commonwealth Edison CompanyCVR Conservation Voltage ReductionCVRf Conservation Voltage Reduction

FactorDA Distribution AutomationDER Distributed Energy ResourcesEKPC East Kentucky PowerGWP Glendale Water & PowerHDH Heating Degree HoursI&M Indiana Michigan Power IEEE Institute of Electrical and

Electronics EngineersIPC Idaho Power CompanyIPL Indianapolis Power & Light JTCM Joint Technical Committee KCP&L Kansas City Power and Light LTC Load Tap Changers

M&V Measurement and VerificationMW megawatt(s)NEEA Northwest Energy Efficiency Alliance NRECA Rural Electric Cooperative PECO Philadelphia Electric CompanyPEPCO Potomac Electric Power CompanyPES Power and Energy SocietyPG&E Pacific Gas and Electric CompanyPGE Portland General Electric CompanyPSE Puget Sound EnergyPSE&G Public Service Electric and Gas RCI Residential Commercial IndustrialRMSE Root Mean Square ErrorSCADA Supervisory Control and Data SCE Southern California EdisonSMUD Sacramento Municipal Utility DistrictTF Task ForceVVO Volt Var OptimizationVVWG Volt Var Working GroupWG Working Group

EXECUTIVE SUMMARY

The electric power grid is undergoing an unprecedented transformation into a more efficient, reliable, resilient,and clean infrastructure. This transformation is primarily supported by more comprehensive grid monitoringand control, enabled by advanced metering infrastructure, distribution automation, and phasor measurementunits, among other sensing and measurement technologies. This enhanced monitoring and control facilitatesmultiple viable practices, most notably Conservation Voltage Reduction (CVR) and Volt-Var Optimization (VVO).CVR, which has gained renewed interest in recent years, is a cost-effective way to deliver energy efficiencybenefits to customers. However, one of the most significant challenges in CVR application is the measurementand verification (M&V) of its effects. Once CVR is applied, there is no benchmark load consumptionmeasurement, which complicates assessing the CVR's real impact. This issue is complicated by the difficulty ofdistinguishing between changes in load and energy consumption caused by variations in temperature andcustomer usage patterns over time and those energy changes caused by the activity of regulating devices withand without CVR.

The IEEE CVR M&V Standard Taskforce was tasked by the IEEE Volt-Var Working Group in October 2020 to study the need for an industry-accepted standard for CVR M&V data collection and management. This standardaims to complement the group's existing work, including the IEEE P1885 Draft Guide for Assessing, Measuring, and Verifying Volt-Var Control Optimization on Distribution Systems; IEEE P1889 Guide for Evaluating and Testing the Electrical Performance of Energy Savings Devices; and IEEE P1459 IEEE Standard Definitions for the Measurement of Electric Power Quantities Under Sinusoidal, Nonsinusoidal, Balanced, or Unbalanced Conditions. The taskforce conducted a comprehensive survey and performed thorough studies using collected data from four electric utilities. This report provides an in-depth review of the survey results and simulations.

The major outcomes of this report are summarized below:

1. Most surveyed utility participants mention multiple data issues in their CVR programs;2. The discrepancy in data management causes significant divergence in calculating CVR impacts. In other

words, data issues significantly impact CVR calculations. Most notably, inadequate and tainted data canjeopardize the analysis regardless of the methodology used to derive the savings or CVR factor;

3. Utilities identify a lack of defined guidelines on selecting the methodology as a major challenge in CVRM&V;

4. Over 70% of all surveyed participants believe that established M&V procedures will be helpful indeveloping a CVR business case, maintaining the expected benefits, meeting regulatory requirements,and streamlining the data cleaning process for benefit estimation. Also, over 80% of all surveyedparticipants believe that an established M&V procedure will be helpful in selecting the methodologybased on data availability.

Based on the survey and studies, this report concludes that the industry needs a standardized datamanagement practice (including cleaning, reconstruction, and analysis) that includes at a minimum:

- Identification of cycling schedule disruption impact on benefit evaluation;- Standardize compression rates to achieve true values;- Detection of accurate CVR status;- Detecting the outliers;- Identification of load shifts;- Identification of discrepancies on data reconstruction;- Identification of true CVR factor range and system-level CVR factor;- Identification of data adequacy based on accurate CVR status and good numerical power and voltage

data;

8

- Methodology selection and assumption validation based on data availability.

The current state of the grid in CVR deployment, in which many utilities have rolled out CVR programs onhundreds of their feeders, further justifies that electric utilities are ready to adopt an industry-acceptedstandard to support their efforts. This is extensively elaborated on in this report.

9

10

INTRODUCTION

1.1 Overview of CVR M&V

Conservation Voltage Reduction (CVR) is commonly known as a subset of Volt-Var Optimization (VVO) and isconsidered a cost-effective measure for energy savings and peak demand reduction without the need of opting inby customers, unlike any other energy efficiency program. To activate CVR, the voltage across a distributionfeeder operates toward the lower regulated voltages by coordinating the utility equipment, including thesubstation transformer load tap changers (LTC), voltage regulators, and capacitor banks. According to theAmerican National Standards Institute (ANSI) standard C84.1, the voltage boundary across the distribution feedershould be between 114V to 126 V during normal operation [1]. Therefore, CVR operates effectively if the voltageband can be maintained at the lower half (114V-120V) of the C84.1 recommended voltage without causing anyharm to the utility assets and customer appliances [2]. As the voltage operates at the lower end, the voltage-sensitive loads at the customer terminal consume less than the normal operation, and achieve energy savings. Arecent benchmarking paper has discussed CVR efforts of 37 U.S. utilities [3].

To report energy savings to the public utility commissions or study cost-benefit-ratio to decide on further CVRdeployment, the energy savings benefits of CVR need to be quantified through measurement and verification(M&V). Quantification of benefits is a challenging task since the CVR ON and OFF measurements cannot beobtained simultaneously. As the changes in other confounding factors (e.g., temperature, season, and time of theday) profoundly impact load consumption, estimating the load deviation becomes an extremely complicatedeffort between CVR ON and OFF instants. On top of that, the quality of data being utilized in the analysis and theirinherent noise can further complicate the process.

Performance of CVR is measured by an index called CVR factor (CVRf), which is defined as the ratio of thepercentage change in energy consumption to the percentage change in voltage reduction. In the past, variousM&V methodologies have been developed to estimate the CVR performance. In general, they can be categorizedas comparison-based, regression-based, and simulation-based [2],[3]. Some descriptions of these methodologiesare provided in the data analysis section.

1.2 Utility Practices on M&V

CVR has been deployed in many utilities, either in pilot testing or as a large-scale program. Utilities employedvarious M&V methodologies to evaluate the CVRf and energy savings. Table 1.1 presents a summary of themethodology utilization based on the publicly available information [3].

Table 1.1. A summary of Utility practices on CVR M&V Methodologies

CVR M&V Methodologies Utilities

Comparison-basedCentral Lincoln People's Utility District, Choptank Electric Cooperative,

Dominion Energy, GWP, IPC, IPL, KCP&L, NEEA, NRECA, PGE, SMUD, BG&E

Regression-based AEP, AIC, Avista Utilities, ComEd, EKPC, I&M, PECO, PEPCO, PG&E, PSE,PSE&G, SCE, West Penn Power Company

Simulation-based Avista Utilities, XCEL Energy

Table 1.1, conveys that the comparison and regression-based methods are most popular among utilities. Thecomparison-based method is straightforward for design and implementation, and the regression-based methodhas physical meanings embedded in the model for easier understanding and analysis. These reasons mightexplain the popularity of comparison and regression methodologies.

1.3 Motivation of This Task Force

This task force (TF) was approved by the Volt-Var Working Group (VVWG) and formed to investigate the need for

11

having an established procedure for CVR M&V. To that effect, the TF has created a study team, includingcomplementary expertise from electric utilities, technology developers, and academia. The primary motivation ofthis TF is to find out the gaps in conducting M&V analysis across the industry, identifying the anomalies in dataand their sensitivities to different types of methodologies, and recommending a path forward to work towardfilling out the gaps. A proper process will help the utilities evaluate CVR implementation in a precise manner.

To fulfill the mission, since its formation in October 2020, the TF has met three times to discuss data requirementsfrom multiple utilities for the purpose of this study. The team has also conducted an industry-wide survey on thecurrent practices, problems, and need for having an established procedure. In the 2021 Power and Energy Society(PES) joint technical committee meeting (JTCM), the survey results were presented, and different data scenarioswere discussed to design the study. In the next subsection, the highlights of the survey and study conducted willbe discussed. The details of the survey responses and study results will be discussed in a following section of thisreport.

1.4 Focus of the Survey and Study Conducted

To investigate the need for an established procedure, the TF conducted a survey where a questionnaire wasdeveloped on different topics, including data sources and variables utilized in M&V; data quality issues and theirroot causes; implementation issues that can create data anomalies; type of M&V methodologies utilized by theindustry and their comparative output; reporting to public utility commissions; and ultimately, the need for anestablished procedure to conduct appropriate evaluations and receive the proper benefits. The surveyquestionnaire was comprehensive enough to extract the inherent anomalies on the data and operationalbarriers, which add additional complexities.

Based on the participants' feedback, the TF utilized the data received from four utilities to validate the concerns.The TF created different data scenarios to check the data sensitivity of different methodologies and theirimpacts on the evaluation. In addition, the TF investigated different issues related to incorrect CVR activation(ON/OFF) status detection and the interruption on activation schedules that impacts the M&V analysis. Theimpact of load shift on M&V analysis was also studied.

The TF also conducted a literature review of CVR factor values claimed by different utilities through pilot and/orprogram level studies and found no comprehensive analysis is currently available to define a boundary for trueCVR factor. Details about the survey and study analysis are discussed in Sections 3 and 4.

1.5 Intended Outcome of the Task Force

The cyber threat continues to evolve, with malicious actors increasing their attempts to manipulate physical assets across the system. Physical attacks, including the use of electromagnetic pulses, are also of concern.

The intended outcome of the TF is to prove the need for having an established procedure. To demonstrate theneed, the TF performed the following tasks:

a. Completed a literature review of current practices across the industry, cited in different places in thisreport depending on their relevancy;

b. Conducted a comprehensive survey to help with gap analysis; c. Conducted thorough studies using the actual utility data.

At the end of this report, the TF will provide concluding remarks on the need with proper justification and providea recommendation for the next endeavor.

12

SURVEY ANALYSIS

The TF conducted a survey within the VVWG. The survey was also circulated through Power Globe, an internetemail forum for persons having an interest in electric power engineering [4]. Survey topics included the overviewof the CVR programs run by different utilities; major drivers; data quality issues; utilization of data points forbenefit evaluation; utilization of different methodologies; the requirement for utilities to report on their benefitsto the commission; challenges faced on M&V analysis; and the opinion on having an established procedure. Inthis section, all these topics will be discussed based on the responses received from the participants.

2.1 Overview of Survey Parti cipants

A total of 30 individuals participated in the survey. Table 2.1 shows the percentage and the affiliation of theparticipants.

Table 2.2. Survey Participants

Affiliation Percentage (%)Utilities 33

Consulting 23Academia 20

Manufacturer/Vendor 17National Labs 3

Other- Research 3

Based on the program magnitude provided by the participant utilities, the study group created the three categories listed below:

More than 1000 feeders (4 utilities) Between 100 to 500 feeders (3 utilities) Less than 100 feeders (2 utilities) One utility responded that they had disabled CVR from their circuits

Utilities also provided the major drivers behind their CVR efforts:

Increase energy savings Reduce peak demand Improve operational efficiency (i.e., better controllability with the combination of hardware and

software) Regulatory requirements Increase line capacity

2.2 Type of Operati on and Data Quality

It was derived from the survey that different utilities run various CVR operations. Out of ten utilities, six run theirCVR operations on a 24x7 basis, two run CVR through on/off testing, and one utility has just started preparingone substation to activate CVR. Out of the six utilities conducting 24x7 operations, four of them initiallyconducted some on/off cycling tests. The utilities conducting on/off testing are on two and four days on/offcycling.

For M&V, utilities generally extract power (MW/Amps) and voltage data, with some also extracting temperatureand CVR status. Only one utility extracts Distribution Automation recloser status data. Out of the ten utilitiesthat participated, all of them extracted SCADA data, while three mentioned that they had extracted AMI data, aswell as for voltage, or power, or both.

13

Several data quality issues were listed in the survey related to the power, voltage, and CVR status, includingoutliers, non-numeric/missing data, repetitive data, interpolated data (i.e., estimated data), and irregularcycling. Based on the survey feedback, almost all participant utilities have multiple data quality issues. Theresponses from other participants were also aligned with the utilities. Figs. 2.1(a) and (b) below show theresponse on data quality issues. Utility participants have also experienced communication failures, poweroutages, incorrect meter/feeder to transformer mapping, and permanent/temporary load switching resulting indata anomalies.

a)

b)

Fig. 2.1. Data quality issues faced by; a) Utilities, b) All participants

Due to the impact of these data quality issues on M&V, they need to be eliminated and backfilled throughreconstruction for proper benefit evaluation.

In response to whether utilities reconstruct their anomalous data, out of 30 participants, only 22 responses werereceived. Seven of them mentioned that they went through a reconstruction process, and six mentioned thatthey did not, whereas the other nine did not leave any comment. Fig. 2.2 summarizes the responses onreconstruction.

Fig. 2.2. Data Reconstruction (All participants)

2.3 Uti lizati on of Data Points for M&V study

14

Questions on data resolution and their usage and impact on M&V analysis were also included in the survey.Below are responses from all participants who have responded to the questions about the utilization of datapoints on M&V analysis. Fig. 2.3 represents the resolutions used by different participants. It is evident that awide variety of resolutions were utilized in the analysis, starting from five minutes to an hourly interval. In Fig.2.4.(a), it shows that different individuals have experimented multiple resolution and majority of them havementioned that they haven’t observed any discrepancies while using multiple resolutions as depicted in Fig.2.4.(b).

Fig. 2.3. Different data resolutions used by participants

a)

b)

Fig. 2.4. a) If participants utilized multiple resolutions; b) If participants observed discrepancies while usingmultiple resolutions.

2.4 Uti lizati on of Methodologies for M&V analysis

In the survey, questions related to the utilization of the M&V methodologies were asked and Table 2.2 segmentsthe responses for the utilities and all participants.

Table 2.3. M&V methodologies used by the participants

Methodologies Type UtilitiesOnly

AllParticipants

Comparison between a circuit running on CVRand a similar circuit without CVR

ComparisonBased 2 9

Comparing the same circuit with on and off daysconsidering impacts of different factors such as

season, temperature, time of the day, on/offstatus, etc.

ComparisonBased 7 14

Establishing the analytical relationship among Regression 2 7

15

load, voltage, temperature, season, type of theday, time of the day, etc. through statistical

formulation (i.e., linear regression) BasedDeemed CVR factor (i.e., a constant CVR factor

estimated from a sample set of circuits andapplied to the entire circuit population to

calculate the savings)Deemed CVR

Factor 5 11Simulation (i.e., simulating the same circuit with

and without CVR to estimate the benefits)Simulation

Based 0 9N/A N/A 0 9

Two additional comments were also received regarding the type of methodologies to be used which are listedbelow and can be categorized under comparison and regression-based analysis:

1. Analytical relationship using machine learning techniques2. Sensitivity analysis (correlation between feeder's power changes vs. voltage changes)

Participants also provided their opinions on whether any discrepancies were observed while using multiplemethodologies, which is depicted in Fig. 2.5.

Fig. 2.5. Discrepancies while using multiple methodologies

Out of all the utility responses, only two utilities reflected that they do not consider line loss and savings at thecustomer end. Instead, they consider the savings at the feeder-head.

2.5 Commission Requirement

Several questions on how different utilities report on their benefit claims to the commission were included inthe survey. Below are the responses listed:

1. Only three utilities are not reporting their analysis to the commission; the rest are reporting their savingsto the commission,

2. Only one utility was using a penalty factor if CVR did not follow the schedules properly,3. Several utilities are required to update their CVR factors. Table 2.3 lists the comments

received from different utilities.

Table 2.4. Utility Comments on updating CVR factor

Utility CommentsUtility 1 Updating CVR factor every 1 yearUtility 2 Planning to update CVR factor every several years.Utility 3 Runs 24x7 so the only opportunity to evaluate CVRf is when the system goes downUtility 4 Updating CVR factor every year if it changes by a large enough amountUtility 5 Updating CVR factor every 2 yearsUtility 6 No update on CVR factorUtility 7 Planning to update the CVR factor soon (irregular update)

16

2.6 Challenges in CVR Implementati on and Conducti ng M&V

The TF also listed the questions regarding the challenges faced by different utilities on implementing CVR andconducting M&V. In Fig. 2.6 the challenges faced by different utilities are provided:

Fig. 2.6. Challenges faced by different utilities

Additional relevant comments received by the utilities are:

The amount of data to clean and analyze is a considerable challenge Choose a simplified M&V approach to avoid all the mentioned challenges Resource availability to support consistent operation and triage of issues Unexpected or prolonged transmission-related outages Running into conflicts with actively operating the system; Also conflicts with DA switching Maintaining power flow models

2.7 Requirement of a Standard Procedure (Response from the Parti cipants)

Finally, the TF asked the participants’ input on whether they support creating established procedures. Below arethe responses from all participants:

1. 73% of responders believe established procedures will be helpful in developing a business case onestimating the benefit-to-cost ratio (BCR) (Yes, a standard/Recommended Practice=22, No=1, Blank=7)

2. 73% of responders believe established procedures will be helpful in maintaining the expected BCR andregulatory requirement (Yes, a standard/Recommended Practice=22, No=1, Blank=7)

3. 73% of responders believe established procedures will be helpful in streamlining the data cleaningprocess for benefit estimation (Yes, a standard/Recommended Practice=22, No=2, Blank=6)

4. 83% of responders believe established procedures will be helpful in selecting the methodology basedon data availability (Yes, a standard/Recommended Practice=25, No=1, Blank=4)

2.8 Key Takeaways of the Survey

1. All participants have highlighted that there are issues with data quality. Data quality issues include non-numeric/missing data, interpolated, repetitive, and outliers. In addition, issues related to active CVRschedule and on/off testing have been pointed out. Due to not maintaining an active CVR schedule whilerunning 24x7 operations and/or irregular cycling (deviation from regular on/off testing), data becomesuncorrelated which makes the analysis erratic.

2. After conducting a cleaning process, a gap is created based on the number of missing points. To fill thosegaps, data needs to be reconstructed. Based on the responses received, significant discrepancies have beenobserved about whether the reconstruction process took place or not while conducting M&V. Without doing

17

any reconstruction or having any proper reconstruction processes in place, the accurate benefit cannot berealized since the missing data points are not accounted for.

3. Survey participants have utilized various methodologies, and discrepancies have been observed whileutilizing different methodologies on the same dataset.

4. The majority of the participants believe that there should be an established procedure to conduct M&V inorder to receive a correct BCR, maintain regulatory requirements, streamline the data treatmentapproaches, and select the CVRf calculation methodology.

5. Different utilities have various levels of oversight from the commission, including but not limited to, periodicupdates on the CVR factor, penalty factor on the savings based on CVR schedule, whether, and how a lossfactor should be utilized to estimate customer savings. These make the evaluation process inconsistent.

2.9 Feedback for Study Design

In the next section, the comprehensive analysis results based on the data received from four major U.S. utilitiesare provided. Further feedback from the survey was considered and utilized while the study was performed,listed below:

1. Different data scenarios will be created to see the impact of 24x7 CVR operation and on/off testing. Thescenarios will be created based on randomly created anomalous data and disruption in the CVR schedule.

2. The study will experiment with the impacts of inaccurate CVR on/off status data in M&V analysis.3. The study will experiment on how the reconstruction process makes any difference.4. The study will explore the impact of load shift either permanently or temporarily based on the data

availability. It will also analyze the change in customer behavior pattern if data is available.5. The study will measure the methodologies’ sensitivity if adequate data is not available due to not

maintaining an active CVR schedule while running 24x7 CVR operations.6. The study will also provide a literature review to understand the basis of CVR factor boundary.7. The study will experiment the impact of data resolution in M&V analysis.

18

DATA ANALYSIS

3.1 Overview of the Data Received from the Utilities

This subsection provides an overview of the utility data that was utilized in this TF study. The TF requested theparticipant utilities to share the time-series power, voltage, and CVR status data for at least three feeders with thefollowing characteristics:

2 years of raw data (power, voltage) – One-year pre-CVR– One year with CVR ON/OFF testing

Data with the maximum available resolution List the source of data

– AMI– SCADA

Time-series CVR status data Pre-CVR peak load for the feeders Temperature Data / Location of the feeders (i.e., ZIP code) Nominal Operating Voltage

The data was requested for heavily residential feeders since the implementation of CVR is largely realized in thesefeeders due to their mostly resistive load characteristics. In addition, the TF requested to be provided with datafrom feeders with either insignificant or no DER penetration to limit the complexity that DER brings to the study.The framing of the initial study without DER was agreed upon during the WG and TF discussions. Table 3.1describes the feeders and their associated data.

Different utilities provided SCADA data with various resolutions as they were available and ZIP codes to retrievethe temperature data based on the location. Temperature data were collected through an applicationprogramming interface (API) for the specific ZIP code. All the provided data reflect 24x7 CVR operation, except forfeeder E1, as shown in Table 3.1 below. For privacy purposes, utilities and their associated feeder names are notlisted.

Table 3.5. Feeder and associated data characteristics

Utility Transformer Bus FeederNominalkilovolts Phase

PreCVR-PeakLoad

Operation Type Resolution

DataSource

U1 230 6 A1 13.2 3 3.728 24x7 15m SCADA

U1 230 6 A2 13.2 3 5.891 24x7 15m SCADA

U1 230 6 A3 13.2 3 6.788 24x7 15m SCADA

U1 33 3 B1 13.2 3 7.170 24x7 15m SCADA

U1 33 3 B2 13.2 3 6.785 24x7 15m SCADA

U1 33 3 B3 13.2 3 6.827 24x7 15m SCADA

U1 33 3 B4 13.2 3 3.836 24x7 15m SCADA

U2 1 1 C1 16 3 12.346 24x7 60m SCADA

U2 1 1 C2 16 3 10.341 24x7 60m SCADA

U2 1 1 C3 16 3 10.042 24x7 60m SCADA

19

U3 1 1 D1 12.47 39.625

24x7 10m SCADA

U3 2 2 D2 12.47 3 12.541 24x7 10m SCADA

U3 3 3 D3 12.47 3 9.525 24x7 10m SCADA

U4 51 51 E1 12.5 1 5.398 ON/OFFcycling 30m SCADA

3.2 Data Cleaning Approaches

In general, the raw data includes a variety of anomalies. The following data anomaly matrices are captured andeliminated from the analysis:

Negative values Non-numeric/Missing values Zero values Repetitive values Interpolated values Outlier values

While the detection of negative, non-numeric/missing, and zero values is straightforward, a process needs to beput in place for the outlier, repetitive, and interpolated values.

Outlier detection:

- A simple threshold-based rule is applied to detect the outliers- MW Outliers

o Feeders w/o DER: eliminate data if above 110% of the peak or below 10% of the peakdemand

The peak demand of a feeder is determined as the maximum summer MWvalue that was not a result of switching

Voltage Outliers- Eliminate data above 1.10 p.u. or below 0.90 p.u. of the nominal voltage level. Nominal

voltage was provided by the utilities.

Detection of repetitive values:

If three subsequent values in a column are equal up to six decimal places, the first value is flagged asrepetitive

- Take the column of MW or voltage values under consideration- Compare the first-row value with the second-row value (up to six decimal places) and the

second-row value with third-row value (up to six decimal places) and so on till the end of thedatapoints available

- If all three subsequent values match, flag the first value as repetitive- Repeat the first three steps for each column or a new set of data per feeder

Detection of interpolated values:

If three subsequent values have the same slope (up to three decimal places), the first value is flagged asinterpolated

- Take the column of MW or voltage values under consideration- Calculate difference of second and first row as (second-row value) – (first-row value), and

third and second row (third-row value) – (second-row value), and so on till the end of the data

20

points available; Assume, a = (second-row value) – (first-row value), b = (third-row value) –(second-row value), and so on

- If a equals b (up to 3 decimal places), flag the first-row value. Similarly, if b equals c, flagsecond-row value and so on

- Repeat the first three steps for each column or a new set of data per feeder

3.3 Data Quality Statistics

3.3.1 Feeder Level

This subsection presents the data quality statistics at the feeder level for the received utility data that fall withinthe studied data years of M&V. In Table 3.2, the percentage of power and voltage data that have data anomalyissues are listed in the last column. It shows that all the feeders have experienced data anomaly issues, especiallyissues with repetition and interpolations. It is worth pointing out that feeders attached to the same transformercan have different data quality statistics since their power data is specific to the individual feeders.

Table 3.6. Feeder and associated data quality statistics for certain studied years

UtilityTrans-former

Bus FeederNominalkilovolts

DataSource

StudiedDataYear

Data Quality of Power and Voltage (%)

NegativeNon-

numeric/Missing

ZeroRepetitive Interpolated Outlier

U1 230 6 A1 13.2 SCADA[2014,2015,2016]

0.38 0.00 0.15 0.76 5.78 0.60

U1 230 6 A2 13.2 SCADA[2014,2015,2016]

0.38 0.00 0.15 0.76 3.10 0.57

U1 230 6 A3 13.2 SCADA[2014,2015,2016]

0.38 0.00 0.15 0.76 3.13 0.59

U1 33 3 B1 13.2 SCADA[2017,2018]

0.00 0.00 0.19 0.32 2.03 0.20

U1 33 3 B2 13.2 SCADA[2017,2018]

0.00 0.00 0.19 0.32 2.57 0.20

U1 33 3 B3 13.2 SCADA[2017,2018]

0.00 0.00 0.19 0.32 2.69 0.20

U1 33 3 B4 13.2 SCADA[2017,2018]

0.00 0.00 0.19 0.32 2.72 0.32

U2 1 1 C1 16 SCADA[2015,2017]

0.00 0.00 2.38 1.36 4.83 2.68

U2 1 1 C2 16 SCADA[2015,2017]

0.00 0.00 2.92 4.75 5.20 3.20

U2 1 1 C3 16 SCADA[2015,2017]

0.00 0.00 2.08 4.45 4.92 2.66

U3 1 1 D1 12.47 SCADA[2013,2014,2015]

0.12 0.00 0.00 4.18 5.33 0.42

U3 2 2 D2 12.47 SCADA[2013,2014,2015]

0.00 0.00 0.00 0.78 2.17 0.17

U3 3 3 D3 12.47 SCADA[2013,2014,2015]

0.12 0.00 0.00 2.76 3.88 0.12

U4 51 51 E1 12.5 SCADA [2020] 0.00 0.21 0.00 0.00 1.22 0.06

21

3.3.2 Station Level

The data quality statistics at the station level for each studied data year are presented in the plots below from Fig.3.1 to Fig. 3.5. Each station will have two plots to demonstrate the statistics of power and voltage data,separately. U1 has provided two substations, each consisting of three feeders. U2, U3, and U4 consist of three,three, and one feeder, respectively.

0

1

2

3

4

5

6

7

8

9

Non-Numeric Negative Zeros Outlier Repetitive Interpolation

% o

f Bad

Qua

lity

Dat

a

U1_A - P

2014 2015 2016

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5


% o

f Bad

Qua

lity

Dat

a

U1_A - V

2014 2015 2016

(a) (b)

Fig. 3.7. Data quality statistics of feeders attached to the transformer 230 at U1 for the studied yearsof 2014, 2015, and 2016. (a) power (b) voltage

00.5

11.5

22.5

33.5

44.5

5


% o

f Bad

Qua

lity

Dat

a

U1_B - P

2017 2018

0

0.5

1

1.5

2

2.5


% o

f Bad

Qua

lity

Dat

a

U1_B - V

2017 2018

(a) (b)

Fig. 3.8. Data quality statistics of feeders attached to the transformer 33 at U1 for the studied years of2017 and 2018. (a) power (b) voltage

0

5

10

15

20

25


% o

f Bad

Qua

lity

Dat

a

U2 - P

2015 2017

0

1

2

3

4

5

6


% o

f Bad

Qua

lity

Dat

a

U2 - V

2015 2017

(a) (b)

Fig. 3.9. Data quality statistics of U2 for the studied years of 2015 and 2017. (a) power (b) voltage

22

0

1

2

3

4

5

6

7


% o

f Bad

Qua

lity

Dat

a

U3 - P

2013 2014 2015

0

1

2

3

4

5

6


% o

f Bad

Qua

lity

Dat

a

U3 - V

2013 2014 2015

(a) (b)

Fig. 3.10. Data quality statistics of U3 for the studied years 2013, 2014, and 2015. (a) power (b)voltage

00.10.20.30.40.50.60.70.80.9

1


% o

f Bad

Qua

lity

Dat

a

U4 - P

2020

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8


% o

f Bad

Qua

lity

Dat

a

U4 - V

2020

(a) (b)

Fig. 3.11. Data quality statistics of U4 for the studied year 2020. (a) power (b) voltage

3.4 Sample Methodologies Utilized in the Study

In this study, the TF utilizes the comparison and regression-based methodologies discussed earlier in this report.These methodologies have been widely used by different utilities in pilot and program level studies [1], [2].Moreover, these are the methodologies which have been mostly cited in the survey conducted by this TF asshown in Section 3. Some descriptions of these methodologies are provided below and the details of themethodologies utilized in this study are discussed thereafter. However, it should be noted that this exercise is notto study which methodology is superior to others. Instead, the purpose of the study is to show how the datascenarios can impact the analysis regardless of the methodologies.

There are two comparison-based M&V methodologies available: one is correlated-weather and the other iscorrelated-feeder. The correlated-weather method conducts CVR ON (treatment group) and OFF (control group)testing on a single feeder to collect the power and voltage measurements for comparison. To find out the CVReffects resulted from CVR operation (i.e., different voltage levels), the treatment and control group should sharesimilar characteristics such as temperatures, time of a day, day of a week, etc.

The correlated-feeder method conducts CVR ON testing on one feeder (treatment group), and at the same time,compares its operation with another feeder (control group) where CVR is OFF. The feeders in the treatment andcontrol group should be geographically adjacent to each other so that the temperatures are close. In addition,these feeders should have the same characteristics such as customer (RCI) mix, load behaviors, circuit miles, andfeeder topologies.

Both comparison-based methods are straightforward to implement. Ideally, the correlated-weather method

23

requires the treatment and control group to have the same temperature data. However, the temperaturedifference always exists during different testing periods while the control group required by the correlated-feedermethod may not always exist, and it is a challenge to find the most appropriate ones. Besides, the feeders need torestrain themselves from load shifting during the test periods in both methods.

Regression-based method models the load and/or voltage as a function of the different predictors or explanatoryvariables using a multivariable linear regression. These characteristics can include, but are not limited to,temperature, season, type of a day, hour of a day, and the CVR status. Based on the captured power and/orvoltage measurements and predictors, the coefficients of the corresponding power and voltage functions can bedetermined. Next, the counterfactual power and voltage can be estimated based on these coefficients andcontrary explanatory variables (i.e., CVR status). Then, using the difference in energy consumption and voltagelevel, CVR effects are revealed.

The multivariable linear regression has an advantage considering the physical meanings are embedded in theregression model itself, making the model and analysis results easier to interpret and understand. However, theregression model can have estimation errors due to inaccurate CVR effects estimation. In addition, the nonlineareffect of load consumption may not be captured precisely in linear regression.

In the simulation-based method, a feeder is run through time-series simulation with and without voltagereduction. Next, both sets of data are extracted and compared to retrieve the energy savings and voltagereduction. The keys to the simulation-based method are to model the loads in a certain way to ensure the load tovoltage sensitivity and assign the specific time-series load factor.

In practice, it might be inefficient to build every component of a feeder in the model, and thus aggregated loadmodeling becomes attractive as an alternative approach. However, it is a challenge to develop an aggregatedmodel with the time dependency factors to account for the dynamic load behaviors.

3.4.1 Temperature Correlated Comparison-based Methodology

The procedures to conduct a comparison-based approach for CVR factor/Savings analysis are listed step by stepbelow:

1. The analysis is framed around available CVR ON data, specifically around CVR ON temperature mean. Toavoid potential skewing of CVR ON/OFF data points, all data points (during CVR ON and CVR OFFconditions) outside +/-1 standard deviations of the CVR ON temperature mean were eliminated.Therefore, keeping 67% of CVR ON temperature range and a similar percentage of CVR ON and OFF datawill balance the dataset. If the utilities are running 24x7 operations, the pre-CVR and post-CVR data areapplied for about two years. Otherwise, if the utility is conducting on/off testing, a year’s worth of data issufficient as the entire year observes the cycling.

2. Segment the voltage and power data for each hour to calculate hourly mean voltage reductionpercentage ΔV% and power reduction percentage ΔP%. ΔV is calculated as (CVR OFF voltage - CVR ONvoltage)/ CVR OFF voltage. ΔV% = ΔV*100. ΔP% and ΔP are calculated similarly.

3. Calculate hourly CVR factors using hourly ΔV% and ΔP% values

CVRf ¿ hr=∆P%¿hr

∆V %¿ hr

(1)

4. Repeat steps two and three with 1000 iterations. This step will ensure an equal sample size of CVR ONand OFF hourly dataset since during the hourly data segmentation, the CVR OFF dataset may containmore datapoints than the CVR ON dataset and vice versa. The balance in sample size is processedrandomly so that all datapoints can be part of the sample size. This process also helps to draw a

24

conclusion based on fundamentals of large trial size of central limit theorem. Final CVR factor and voltagereduction will be the mean of these iterations.

5. After the final CVR factor is calculated, the total savings are calculated as below:

E savings=Ebaseline×∆V ×CVRf

(2)

3.4.2 Linear Regression-Based Methodology

The following two functions are defined to represent power and voltage:

MWhit=α 1VOit+α2Hour t+α3Daytypet+α 4Seasont+α5CDH t+α6HDH t

(3)

V it=β1VOit+β2Hour t+β3Daytype t+β4 Seasont+β5CDH t+β6HDH t

(4 )

where, VO represents the CVR status; Hour, Daytype, and Season denote the associated indices of time of theday, type of the day (i.e., Weekday/Weekend), and season of the year (i.e., Summer, Spring, Fall, and Winter); andCDH and HDH correspond to the absolute value of degrees F above and below 65 degrees F, respectively;MW it and V itdesignate the feeder-specific power and voltage data; i and t represent the feeder and time index,respectively.

The CVR factor and savings can be calculated following the steps below:

1. Utilize (3) and (4) to estimate the coefficients2. Using α 1 and β1along with the mean CVR OFF energy (MWhCVRoff ) and voltage data (V CVRoff ), voltage

reduction (ΔV%) and power reduction (ΔP%) are calculated, respectively, using the following equations (5)and (6)

∆ P%=α 1

MWhCVRoff∗100

(5)

∆V %=β1

V CVRoff

∗100

(6)

3. Estimate the CVR factor using calculated ΔV% and ΔP% as below:

CVRf =∆ P%∆V %

(7)

Estimate the savings using the CVR factor and voltage reduction calculated above using equation (2). Theprocedure to calculate the energy baseline is listed in the next subsection.

3.4.3 Calculation of energy baselines consumption

In this study, energy baselines (Ebaseline) are calculated in two different ways. Below are the assumptions listed:

25

1. If the actual dataset is utilized without any reconstruction, only actual CVR OFF data are used to calculatethe energy baseline for the duration of one year. If there are any missing CVR off data within the year, theywill be filled with the average of the available CVR OFF data. For any CVR ON data, the counterfactual will befilled up using the same average value. Then, the data for the entire year will be summed up together toestimate the energy baseline of the year.

2. If the reconstructed dataset is utilized, there should not be any missing data present. However, there willstill be existing CVR ON data. In order to calculate the baseline, CVR ON data are converted to CVR OFFusing the following eq (8):

Ebaseline ,t=ECVRON , t

1−(CVR¿¿ f∗ΔV )¿(8)

where

Ebaseline ,t Calculated CVR Off energy when CVR is ON (activated)

ECVR ON ,t Measured energy when CVR is ON

ΔV Average voltage reduction ratio

CVRf CVR factor

3.5 Scenarios

Several different data scenarios were studied to investigate the impact of data anomalies and sensitivities of thesolutions on M&V methodologies, and several different data scenarios were studied. These are categorized inTable 3.3 below.

Table 3.7. Data scenarios used in this study

Scenarios # Operation Type Scenario Descriptions Data Type

Scenario 1 24x7 Original dataset Clean

Reconstructed

Scenario 2 ON/OFF cycling Original datasetClean

Reconstructed

Scenario 3 ON/OFF cycling Modified datasetClean

Reconstructed

Scenario 4 24x7 Modified dataset Clean

Reconstructed

Scenarios 1 and 2 are base scenarios for 24x7 and cycling datasets, respectively. Scenario 3 is the modifiedscenario of Scenario 2, and Scenario 4 is the modified scenario of scenario 1. Elaborately, a certain percentage ofpower and voltage data in Scenario 1 is randomly selected and treated as bad quality to create the dataset forscenario 4. Furthermore, based on the regular ON/OFF cycling data in Scenario 2, some randomly selected CVRON status is manipulated to OFF to emulate the irregular cycling. The power and voltage data associated with themanipulated CVR status are treated as bad quality and the resulted dataset is utilized as Scenario 3.

For all the scenarios, cleaning will take place on the actual/manipulated datasets using the aforementionedapproaches to eliminate the negative values, non-numeric/missing values, zeros, outliers, etc. The resulteddataset is called a cleaned dataset and the M&V analyses will be conducted based on it.

26

A second dataset is created by reconstructing the cleaned data. A reconstruction process is needed to recover thecleansed datapoints and make a complete dataset. This process uses a lookup table (shown in Table 3.4) and apriority chart (shown in Table 3.5). This reconstruction process is described in [5].

The lookup table is created based on the following information:

Season (Spring, Summer, Fall, Winter) Temperature (binned to the ceiling of nearest 5°F interval) Day type (Weekday, Weekend) Hour (hrs. are binned in 1-hr range) CVR Status (CVR ON and CVR OFF)

For the seasons motioned above, they are defined within the months as below:

Spring (March through May) Summer (June through August) Fall (September through November) Winter (December through February)

An example lookup table is presented in Table 3.4 below,

Table 3.8. An illustration of the lookup table

Season Temperature Day Type

Hour CVR status Voltage (p.u) Power (MW)

Summer 90 1 1 1 0.99 2.30

Winter 30 2 24 0 1.04 1.93

In the lookup table, single flags are considered for each day type. For example, any day from Monday to Fridaycan be considered flag one, Saturday and Sunday can be considered flag two.

If multiple values with the same characteristics are identified, then the average of those values will be placed intothe lookup table. For instance, if five MW data points are found within the same season, temperature band, daytype, hour, and CVR status, the average of these five values are used in the lookup table.

Once the lookup table is created, the cleansed datapoints are reconstructed from the lookup table based on apriority sequence. Such priority sequences are defined in Table 3.5.

Table 3.9. The priority chart for data reconstruction

Priority Components and Commonalities of Table1 Season, Temperature, Day Type, Hour, CVR Status2 Temperature, Day Type, Hour, CVR Status3 Temperature +/-5°F, Day Type, Hour, CVR Status4 Season, Day Type, Hour, CVR Status5 Temperature, Day Type, CVR Status6 Season, Day Type, CVR Status7 Temperature, CVR Status8 Season, CVR Status9 CVR Status, Day Type, Hour

10 CVR Status, Day Type11 CVR Status

27

The data reconstruction process works based on the common factors of the missing data and lookup table asbelow:

Missing data (either MW or V) will be obtained from the lookup table for the same season, temperature,day type, hour, and CVR status (Priority 1).

If seasonal data is not located, then lookup for the same temperature, day type, hour and CVR status(Priority 2).

If data is not found in the previous step, search within temperature +/- 5°F for the same day type, hour,and CVR status (Priority 3).

The priority sequence continues in order as stated in the following table until the data is retrieved.

In this study, the reconstructed data will also be used to observe any variation in the evaluation results.

To illustrate the studied data scenarios, feeder A1 is being utilized as an example for Scenarios 1 and 4, andfeeder E1 for Scenarios 3 and 4 as they correspond to 24x7 and on/off testing (cycling), respectively. Figs. 3.6 to3.9 represent the power, voltage, and CVR status data in Scenarios 1, 2, 3, and 4, respectively. The other feedersin Table 4.2 follow the same manipulation/cleaning/reconstruction process to A1 or E1, depending on their CVRoperation types. In these figures, both cleaned and reconstructed data are plotted for the demonstration ofdifferent data scenarios. Specifically, the reconstructed power/voltage data are plotted against the cleaned datafor better visual comparison.

In the results subsection, examples illustrate how these scenarios are utilized per utility, based on their operationtype.

28

Jul 2014 Jan 2015 Jul 2015 Jan2016 Jul 20160

1

2

3

4

5

Po

we

r (M

W)

OFF

ON

CV

R s

tatu

s

Scenario 1 - Cleansed Power

Cleansed power CVR status

(a)


1

2

3

4

5

Po

we

r (M

W)

OFF

ON

CV

R s

tatu

s

Scenario 1 - Reconstructed Power

Cleansed powerReconstructed power

CVR status

(b)

Jul 2014 Jan 2015 Jul 2015 Jan2016 Jul 20160.9

0.95

1

1.05

1.1

Vo

ltag

e (

p.u

.)

OFF

ON

CV

R s

tatu

s

Scenario 1 - Cleansed Voltage

Cleansed voltage CVR status

(c)


0.95

1

1.05

1.1

Vo

ltag

e (

p.u

.)

OFF

ON

CV

R s

tatu

s

Scenario 1 - Reconstructed Voltage

Cleansed voltageReconstructed voltage

CVR status

(d)

Fig. 3.12. An example of clean and reconstructed power and voltage data against CVR status for feederA1 in scenario 1 – 24x7 CVR operation, original dataset

(a) cleansed power against CVR status (b) reconstructed power against CVR status

(c) cleansed voltage against CVR status (d) reconstructed voltage against CVR status

29

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

2020

0

1

2

3

4

5

6

Po

we

r (M

W)

OFF

ON

CV

R s

tatu

s



(a)


2020

0

1

2

3

4

5

6

Po

we

r (M

W)

OFF

ON

CV

R s

tatu

s



CVR status

(b)


2020

0.9

0.95

1

1.05

1.1

Vo

ltag

e (

p.u

.)

OFF

ON

CV

R s

tatu

s



(c)


2020

0.9

0.95

1

1.05

1.1

Vo

ltag

e (

p.u

.)

OFF

ON

CV

R s

tatu

s



CVR status

(d)

Fig. 3.13. An example of clean and reconstructed power and voltage data against CVR status for feederE1 in scenario 2 – CVR ON/OFF cycling, original dataset



30

(a)


2020

0

1

2

3

4

5

6

Po

we

r (M

W)

OFF

ON

CV

R s

tatu

s



CVR status

(b)


2020

0.9

0.95

1

1.05

1.1

Vo

ltag

e (

p.u

.)

OFF

ON

CV

R s

tatu

s



(c)


2020

0.9

0.95

1

1.05

1.1

Vo

ltag

e (

p.u

.)

OFF

ON

CV

R s

tatu

s



CVR status

(d)

Fig. 3.14. An example of clean and reconstructed power and voltage data against CVR status for feederE1 in scenario 3 – CVR ON/OFF cycling, modified dataset




2020

0

1

2

3

4

5

6

Po

we

r (M

W)

OFF

ON

CV

R s

tatu

s



31

(a)

(b)


0.95

1

1.05

1.1

Vo

ltag

e (

p.u

.)

OFF

ON

CV

R s

tatu

s



(c)

(d)

Fig. 3.15. An example of clean and reconstructed power and voltage data against CVR status for feederA1 in scenario 4 – 24x7 CVR operation, modified dataset




1

2

3

4

5

Po

we

r (M

W)

OFF

ON

CV

R s

tatu

s



32

3.6 Results

In this section, specific results and associated data anomalies are discussed with the example feeders from eachutility. The results of the study are presented in multiple findings for further illustration.

3.6.1 Finding 1: Difference in Evaluation with Different Scenarios

In this finding, we will discuss how the evaluation results vary with different data scenarios. As presented earlier,four different scenarios are prepared to study the changes in evaluation results. Scenarios 1 and 2 are the basescenarios where no manipulation is conducted, and the data is used as it is after the proper cleaning. Scenarios 3and 4 resemble the irregular cycling and additional bad data quality scenarios, respectively.

Table 3.6 presents three feeders in stations U1, U2, and U3 to demonstrate the differences between the base(Scenario 1) and manipulated bad data scenario (Scenario 4). Scenarios 1 and 4 are applied to U1, U2, and U3 asthe data resembles 24x7 CVR operations.

Table 3.10. Data scenarios with different data quality

Utility Feeder CVR ON/OFF data ratio

Percentage of bad power data

Percentage of bad voltage data

Scenario

U1 A1 1.03 14.41 0.73 1U1 A1 1.03 31.41 25.49 4U2 C1 0.72 10.18 2.80 1U2 C1 0.72 30.14 27.07 4U3 D3 0.90 4.47 0.35 1U3 D3 0.90 27.02 25.09 4

It appears from Table 3.6 that Scenarios 1 and 4 are well defined to have visible changes on % of bad quality inboth voltage (%vbad) and power(%pbad). After eliminating these anomalous data points, all these cleaneddatasets are reconstructed. The reconstructed data following the algorithm discussed previously in the Scenariossubsection will not have any bad quality data compared to the cleaned data.

Tables 3.7 and 3.8 present the evaluation results for the cleaned and reconstructed datasets, respectively for U1,U2, and U3 using their 24x7 operation data.

Table 3.11. CVR factor evaluation for cleaned data (24x7 operation)

Utility Feeder CVRFactor

Ebaseline(MWh)

Esavings(MWh)

EstimatedVoltage

Reduction(Percent)

Scenario Methodology

U1 A1 0.85 16926.843 169.304 1.18 1 RegressionU1 A1 0.77 16758.661 153.242 1.18 4 RegressionU1 A1 1.85 16926.843 337.36 1.08 1 ComparisonU1 A1 1.66 16758.661 301.72 1.08 4 ComparisonU2 C1 2.82 37448.340 993.383 0.94 1 RegressionU2 C1 3.07 37455.885 1081.183 0.94 4 RegressionU2 C1 3.72 37448.340 1406.21 1.01 1 ComparisonU2 C1 3.17 37455.885 1193.63 1.00 4 Comparison

33

U3 D3 1.45 37874.49 1269.33 2.30 1 RegressionU3 D3 2.02 37810.11 1752.22 2.29 4 RegressionU3 D3 1.06 37874.49 1000.826 2.48 1 ComparisonU3 D3 1.28 37810.11 1204.424 2.47 4 Comparison

Table 3.12. CVR factor evaluation for reconstructed data (24x7 operation)


Ebaseline(MWh)

Esavings(MWh)

EstimatedVoltage

Reduction(Percent)


U1 A1 0.91 16469.630 177.699 1.18 1 RegressionU1 A1 0.82 16449.136 160.213 1.19 4 RegressionU1 A1 1.94 16545.36 345.13 1.07 1 ComparisonU1 A1 1.68 16512.39 300.14 1.08 4 ComparisonU2 C1 2.75 41287.348 1061.294 0.93 1 RegressionU2 C1 3.08 41253.539 1052.246 0.83 4 RegressionU2 C1 2.92 41404.93 1199.09 1.00 1 ComparisonU2 C1 3.42 41423.38 1249.62 0.88 4 ComparisonU3 D3 1.43 38735.73 1271.60 2.30 1 RegressionU3 D3 2.13 38681.16 1885.05 2.29 4 RegressionU3 D3 0.79 38322.75 757.75 2.48 1 ComparisonU3 D3 1.16 38049.30 1101.46 2.48 4 Comparison

In both tables, baselines are quite close within the cleaned and reconstructed datasets, separately. However, thebaselines may have some differences between clean and reconstructed analysis. For instance, feeder C1 hasbaselines of 37448.34 MWh and 41287.348MWh using cleaned and reconstructed dataset, respectively in theregression based approach for scenario 1. This is due to their own different assumptions of filling up the datausing average CVR OFF power data or calculating the counterfactuals (CVR OFF power when CVR is ON) usingmeasured CVR ON power data, estimated CVR factor, and voltage reduction as mentioned in the calculation ofenergy baselines consumption subsection. In addition, the savings vary along with the CVR factors as the datasetchanges, which demonstrates that the data quality impacts the analysis regardless of the methodologies.

Then, similar to 24x7 operational data, cycling data are studied. Only utility U4 has provided cycling data whichgoes through 4 days of ON/OFF testing. Therefore, Scenarios 2 (regular cycling) and 3 (irregular cycling) apply onlyto this utility’s dataset. Below is the summary of the scenarios in Table 3.9 for this utility.

Table 3.13. Data scenarios with different data quality

Utility Feeder CVR ON/OFFdata ratio

%pbad %vbad Scenario

U4 E1 0.76 1.01 1.98 2U4 E1 0.56 8.24 9.10 3

In the base dataset (Scenario 2), utility had 1.01% and 1.98% of anomalous power and voltage data, respectivelywith 76% of time as the ratio of CVR ON and OFF data during the 4 days ON/OFF cycling. The data is manipulatedto create irregular cycling (Scenario 3) pattern, which has made this down to 56% and corresponding CVR ON dataare eliminated. Tables 3.10 and 3.11 show the differences in evaluation results for the four scenarios usingcleaned and reconstructed data, respectively.

34

Table 3.14. CVR factor evaluation for cleaned data (cycling dataset)

Utility FeederCVR

FactorEbaseline

(MWh)Esavings(MWh)

EstimatedVoltage

Reduction(Percent) Scenario Methodology

U4 E1 0.723 21098.41 710.69 4.65 2 Regression

U4 E10.293 21098.41 291.72 4.71

2 ComparisonU4 E1 0.625 21098.41 615.01 4.65 3 Regression

U4 E10.05 21098.41 49.19 4.71

3 Comparison

Table 3.15. CVR factor evaluation for reconstructed data (cycling dataset)


Ebaseline(MWh)

Esavings(MWh)

EstimatedVoltage

Reduction(Percent)


U4 E10.723 21034.17

708.619 4.656 2 Regression

U4 E10.300 20855.22 294.891 4.709

2 ComparisonU4 E1 0.609 21021.66 596.686 4.654 3 Regression

U4 E10.060 20826.76 56.80 4.697

3 Comparison

Based on the results in Table 3.10 and Table 3.11, the same conclusion can be derived as previously observed for24x7 datasets. The CVR factor and savings vary as the cycling pattern becomes irregular and the CVR ON/OFF dataratio changes. These variations are observed across different methodologies and within the same methodologyusing different scenarios.

3.6.2 Finding 2: Overlapping of Voltage Data in Different CVR Modes

In both U1 and U2 original datasets, voltage data is found highly overlapped, which may jeopardize the analysis ofCVR factor/savings evaluation and the data reconstruction process as well. Fig. 3.10. (a) and (b) show thehistogram of CVR ON and OFF for U1 and U2. The figures also depict that a large number of data points(highlighted in the green circle) reside within CVR OFF and ON region, simultaneously. CVR generally does notoperate as OFF and ON with the same voltage level.

When CVR is OFF, it maintains a constant voltage level with a defined bandwidth. For example, suppose CVR OFFoperating voltage is 124V; in that case, it can reside within ±1.5V of 124 V. However, to operate effectively whenCVR is ON, it needs to go below the bandwidth of the CVR OFF operating voltage, or at least remain close to that.Fig. 3.10. (a) and (b) below portray the actual voltage data overlapping scenarios to a large extent for U1 and U2.At this point, it should be noted that overlapping of voltage may occur due to I) inaccurate CVR status detectionand/or II) while the CVR is not running, the transformer is left in manual mode operating at lower operatingvoltage. Fig. 3.10 refers to such cases as some CVR OFF voltage data points are very low and vice versa.

35

a. b.

Fig. 3.16. Voltage histogram with overlaps; a. U1; b. U2.

In such cases, there is no provision on how to clean the data accurately. For the purpose of the study, intuitiveguesses were made to set up a threshold for CVR ON and OFF for both U1 and U2, respectively, 1.02 and 1.06 p.u.to experiment with the impact and sensitivity. CVR ON and OFF data will be considered if the voltage is below andabove these thresholds, respectively. Based on this additional cleaning, the histograms are shown in Fig. 3.11:

a. b.

Fig. 3.17. Voltage histogram after cleaning overlap; a. U1; b. U2.

Table 3.12 below shows the differences with and without the overlap cleaning explained above using just theregression analysis:

Table 3.16. CVR factor differences with the cleaning of overlapped voltage.


Ebaseline(MWh)

Esavings(MWh)

EstimatedVoltage

Reduction(Percent)

MWRMSE

V RMSE Overlap

MWaR2

V aR2

U1 A1 0.85 16926.84 169.30 1.18 0.1744 0.0091 Yes 0.899 0.391U2 C1 2.82 37448.34 993.38 0.94 0.6750 0.0065 Yes 0.82 0.48U1 A1 1.50 16926.84 536.80 2.12 0.1691 0.0061 No 0.901 0.781U2 C1 2.76 37448.34 1490.54 1.44 0.6644 0.0051 No 0.80 0.73

Based on the comparison in Table 3.12, it appears I) adjusted R2(aR2) for voltage increases significantly, whichshows that the explanatory variables can explain the prediction better after cleaning the overlap; II) root meansquare error (RMSE) for both power and voltage decreases slightly which reflects the increment in modelaccuracy with this adjustment; III) Voltage reduction increases as expected, which drives the savings higher.

A noteworthy point for this exercise is that this additional cleaning can cause a large amount of voltage andcorresponding power data loss based on the intuitive threshold which may not generate the adequate CVRON/OFF data ratio as shown in Fig. 3.11 (a) and (b). In addition, removing those data points can create an

36

imbalance in the analysis for any particular methodology or model depending on its sensitivity as the otherconfounding factors (i.e., temperature, CDH, HDH, time of the day) are also being eliminated from the analysis forthe same indices.

3.6.3 Finding 3: Impact of Load Shift

The TF also analyzed some abnormal CVR factors due to a highly visible load shift. Load shift in this study refers toload change over time that is caused by factors other than CVR operation. This was found in U1 feeder A3. Theanalysis using the actual data for the base scenario is listed in Table 3.13.

Table 3.17. Demonstration of load shift impact on analyses results


Ebaseline(MWh)

Esavings(MWh)

Estimated VoltageReduction (Percent)

Methodology

U1 A3 -17.972 32018.290 -6808.615 1.18 RegressionU1 A3 -37.256 32018.290 -13632.70 1.14 Comparison

These two unreasonable CVR factors can be explained due to a load shift shown in Fig. 3.12. CVR deploymentstarted later in the year 2015, as indicated by the status. However, comparing the post CVR power data with thepre CVR period, it appears the load has been shifted significantly up during the CVR ON period. This could be dueto feeder reconfiguration or natural load increase. Therefore, although the voltage reduction is observed, powerreduction is not observed or may not be rightfully comparable with the pre CVR data, resulting in a very low CVRfactor.

Fig. 3.18. Power consumption with load shift .

3.6.4 Finding 4: Basis of Relatively High CVR Factor

In Tables 3.7 and 3.8, some of the CVR factors were relatively higher (i.e., greater than two). The TF investigatedthe feeder C1 for utility U2 and found that the mean power consumption difference was higher whereas themean voltage was not reduced for the same temperature bin (ceiled temperature to the nearest 5-degree F). Fig.

37

3.13 (a) and (b) demonstrate the difference in consumption and voltage reduction.

a) b)

Fig. 3.19. Comparison between Temperature Vs a) mean power; b) mean voltage .

The voltage difference may have been impacted by voltage overlap or incorrect CVR status detection as discussedin finding two. Furthermore, both power and voltage are coupled with CVR status data. Therefore, powerconsumption may have been impacted as well due to incorrect CVR status detection. In the temperature levelvariation, the high variation in power consumption is observed compared to voltage variation. Specifically, thepower consumption difference during the low-temperature period was much higher than a relatively highertemperature. For example, at 45 degrees F, the difference in mean power consumption between CVR OFF and ONis approximately 14% which directs that there was higher variability in power usage from the customer side orsome temporary load transfer during the low temperature (winter) period. At this temperature, the differencebetween mean CVR ON and OFF voltage is 0.02%, which can lead to extremely large CVR factors for particularperiods. The same data is visualized in the time domain as well as portrayed in Fig. 3.14. (a), (b), and (c).

(a) (b)

38

(c)

Fig. 3.20. (a) CVR OFF and ON Power Consumption; (b) CVR OFF and ON Operating Voltage; (c) Hourly difference inPower and Voltage reduction.

Similar to temperature level variation, time segmentation has also shown higher variability.

On the utilized methodologies, there were no imposed constraints on the level of power and voltage reduction tominimize discrepancies. This is due to not having any defined boundary on power reduction or CVR factor.

On the other hand, different CVR factors have been reported/claimed in the different studies/pilots/programsconducted by the utilities where some CVR factors were relatively higher [3]. Table 3.14 demonstrates a summaryof claimed CVR factors by different utilities:

Table 3.18. Summary of studied cases

Utility/Project Type Year CVR Factor AssessmentMethod

CVR Factor

AEP [6]-[11] Program 2014/2016/2019 Regression-based *Central Lincoln People'sUtility District [12]

Pilot 2013-2014 Comparison-based 0.43 (summer); 1.05 (winter)

EKPC [13] Test case 2019 Regression-based *

AIC [14]-[15] Pilot/Program 2012-2013/2017-2018/2018-2025

Regression-based 0.148-1.48

ComEd [16]-[18] Program 2018-2025 Regression-based/ConstantCVR factor

0.8

IPC [19]-[21] Program 2009-2016 Constant CVR factor/ Comparison-based

0.41-5.75 (residential); 0.19-2.89 (commercial)

PEPCO [22]-[23] Pilot 2012-2014/2018 Regression-based *West Penn PowerCompany [22], [24]

Study 2012-2014 Regression-based 0.86

IPL [22], [25] Program 2012-2013 Comparison-based 0.85 (2012); 0.75 (2013)PECO [26]-[28] Program 2009-2012/2013-2016 Regression-based 1.08PGE [20], [29] Pilot/Plan for program 2014/2018 Comparison-based *SMUD [30]-[31] Test/Plan for program 2010-2014/2017 Comparison-based *Duke Energy Ohio [32]-[34]

Pilot 2008-2016 Constant CVR factor 0.50-0.79

Xcel Energy [35]-[37] Pilot/Plan for program2011-2012/2015-2020/2019

Simulation-basedmethod/Statistical analysis

1.7 (2011); 2.7 (2012);0.8(2019); 0.78 (2020);0.77(2021)

Avista Utilities [38]-[39] Program/Plan forprogram

2013-2014/2019 Regression-based/Simulation-based

0.833-0.881

PG&E [40]-[41] Pilot/Plan for program 2013-2016 Regression-based 0.6-0.8

SCE [42]DemonstrationProject/Plan for program

2012-2015/2019 Regression-based 1.56

GWP [43]-[44] Pilot/Program 2014-2015/2015-2018 Comparison-based *PSE [45] Program 2015-2016 Regression-based 0.475Dominion Energy [46] Program 2009-2011 Comparison-based 0.92

I&M [47]-[49] Program 2014-2015/2019 Regression-based -1.13-11.38 (2014-2015); -0.43-4.48 (2018)

PSE&G [50] Plan for pilot 2018-2025 Regression-based *KCP&L [51]-[52] Demonstration Project 2015 Comparison-based 0.14-2.073 (overall 0.889)Choptank ElectricCooperative [53]

Program 2018 Comparison-based *

39

NRECA [54] Test 2012-2014 Comparison-based 1.04-1.05NEEA [19] Pilot 2006-2007 Comparison-based 0.17-1.12

* No CVR factor was found

Furthermore, in a recent large-scale evaluation on ComEd and Ameren systems, 246 feeders were studied toestimate the CVR factors [49]. In this report, the range of CVR factors was observed between -2 to 9, with mostfeeders lying between -1 to 3. Finally, a system-wide load weighted average CVR factor was reported to beutilized throughout the systems. However, to the best of this TF's knowledge, there is no report which has shownthe boundary of definite CVR factors at feeder or system level with valid assumption.

3.6.5 Finding 5: Interruption in CVR Schedule

In U3 feeder D1, the CVR schedule has been mainly disrupted. This is highlighted in Fig. 3.15 (a). It contains 24% ofCVR ON data compared to the CVR OFF data utilized in the analysis. The study also checked the voltage overlapfor this feeder. However, the overlap was very minimal as opposed to the example feeders shown in finding two.Fig. 3.15(b) shows the histogram of CVR OFF and ON voltage to demonstrate the minimal overlap. Therefore, it isshown that the voltage data were isolated by default and CVR OFF and ON status was calculated quite accurately.However, the CVR OFF and ON data ratio was not adequate to account for the confounding factors throughoutthe year.

a) b)

Fig. 3.21. a) Voltage and status for feeder D1; b) Histogram to check voltage overlap .

Since the characteristics of different methodologies are different, this interrupted CVR pattern can be sensitive todifferent methodologies and create divergent solutions due to the lack of adequate data. It is difficult to justifywhich methodology will provide better accuracy based on the sensitivity to the particular data situation due toinadequate data. Table 3.15 shows the CVR factor evaluation results for feeder D1. The original dataset is utilizedto obtain these results.

Table 3.19. CVR factor evaluation with interrupted CVR schedule.

Utility Feeder CVR Factor

Ebaseline(MWh)

Esavings (MWh) Estimated Voltage Reduction (Percent)

Methodology

U3 D1 1.36 40195.31 1360.21 2.49 RegressionU3 D1 3.94 40195.31 4196.60 2.65 Comparison

Furthermore, the power and voltage adjusted R2 from the regression model are 0.64 and 0.62, respectively whichis relatively low and indicates that predictors cannot explain the model precisely. On the other hand, thecomparison-based approach does not have right values to compare and therefore resulted in relatively higherCVR factor.

40

3.6.6 Finding 6: Impact of Data Resolution in M&V

In the survey feedback, the majority of the participants mentioned that data resolution has no impact on M&Vanalysis. The study investigated this using the feeders from U3 since they provided the highest granular data withten minutes resolution. The resampling was done for 30 minutes and 60 minutes resolution to observe anychange. Table 3.16 shows the analysis results using the regression methodology.

Table 3.20. CVR factor evaluation with different data resolutions

Feeder CVRFactor

Ebaseline(MWh)

Esavings(MWh)

EstimatedVoltage

Reduction(Percent)

VO ON/OFFratio

MW aR2 V aR2 Resolution

D1 1.358 40195.312 1360.212 2.491 0.242 0.645 0.623 10mD2 1.168 45573.646 1156.309 2.172 0.802 0.742 0.829 10mD3 1.456 37874.491 1269.333 2.302 0.906 0.670 0.802 10mD1 1.360 40180.766 1361.331 2.491 0.243 0.653 0.623 30mD2 1.200 45665.069 1188.968 2.170 0.804 0.750 0.830 30mD3 1.383 38378.951 1219.246 2.298 0.913 0.678 0.801 30mD1 1.342 40160.490 1342.409 2.491 0.243 0.666 0.622 60mD2 1.207 45616.307 1193.058 2.166 0.804 0.766 0.829 60mD3 1.386 38346.525 1223.998 2.303 0.913 0.692 0.803 60m

Based on Table 3.16, a very high deviation was not observed using different data resolutions for the feeders in U3.Feeders D2 and D3 had some deviations in 30- and 60-minute resolution compared to ten minutes. This may havehappened due to some data loss because of resampling. It should be noted that there are always chances toeliminate good data and retain anomalous data while resampling which may cause deviation in results if anyfeeder contains a large amount of anomalous data at different times.

3.7 Summary

In this section, we evaluated and verified the feedback we received from the survey. We analyzed the utility-provided datasets to evaluate the feedback. To summarize, observations are listed below:

1. Quality of datasets (i.e., anomalous voltage and/or power data, interruption in CVR operation, irregularcycling) impacts the analysis on CVR benefits evaluation (i.e., CVR factor, savings) regardless of theapplied methodology.

2. Differences are observed in evaluation results using both cleaned and reconstructed datasets.3. Detection of inaccurate CVR statuses can impact the CVR benefits analysis and any kind of data

reconstruction process.4. It is imperative to maintain the loading consistent to analyze the benefits accurately. Any temporary or

permanent load shift can jeopardize the analysis.5. There is no defined boundary on what would be considered an accurate CVR factor based on the pilot

and program level study from different utilities.6. Data resolution has minor impacts on M&V analysis as noted in the survey feedback.7. There is no validated assumption on how to impose constraints on the data distribution to filter out

extremely divergent data in any methodology.

41

CONCLUDING REMARKS

4.1 Processes to be Streamlined

Through the survey and study using several utilities’ data, it is clear that an established process would help theutilities evaluate and analyze the performance of CVR implementation properly. It can also be utilized as a one-stop detail documentation to serve as a reporting guideline to the commission on CVR M&V benefits. In the shedof this report, the following topics, at a minimum, should be discussed in the process of streamlining establisheddocumentation:

1. Approaches to archive true values should be discussed in order to avoid recording interpolated andrepetitive values. In addition, if interpolated and repetitive values still exist in the utilized datasets,approaches need to be determined to eliminate these anomalous data points.

2. Process(es) need to be set on how the outliers should be detected and eliminated.3. The distribution system often goes through network reconfiguration. Therefore, avoiding load shifting

may not always be possible. This problem can be tackled in different ways. One way is to identify the timeinstants of temporary load shift and eliminate those data before conducting M&V analysis. Another wayis to provide any constraints in the utilized methodologies to avoid any out of the boundary loaddeviation in both positive and negative directions. There is no defined range for power consumptionreduction due to CVR deployment that can be utilized for filtering out irrelevant values, so it needs to bediscussed thoroughly. Furthermore, in case of permanent load shift during data collection, alternativeapproaches need to be sought for measuring the benefits.

4. There is no defined approach for how the CVR status is detected. It is not guaranteed that sending asignal from any vendor software to the field devices will always activate CVR. There may be an equipmentfailure, communication issues, or any other field issues (e.g., network outages), which will restrict CVRactivation or intermittently interrupt the CVR schedule. Approaches for defining time-series CVR statusneed to be established to create more correlated data.

5. Since different methodologies have various sensitivities on the data, how the methodologies should beselected based on data sensitivity needs to be discussed; In addition, the metrics which define theprecision of the evaluation analysis need to be documented (e.g., data adequacy after anomalous dataelimination, CVR ON/OFF data ratio, error matrices).

6. If the CVR schedule is interrupted or unable to activate when it is supposed to be activated, it needs to behighlighted whether a penalty factor should be considered or not, and if considered, the calculationprocess of such factors should be documented.

7. If the SCADA data is utilized from the feeder-head, it is required to have formal documentation onwhether a loss factor should be considered, and if that is the case, how a loss factor should be calculatedto account for the customer-level savings.

8. Feeder level CVR factors show large discrepancies, as portrayed in the study and literature survey, due toa variety of issues. A practical range of CVR factors should be developed in order to filter out theirrelevant values. Furthermore, due to deviation on feeder level CVR factors, it should be determined ifutility-wide CVR factors for individual utilities can be developed for system-wide usage through anyprobabilistic function or weighted average.

9. As the loads are becoming energy efficient, a process should be in place on how often the CVR factorsshould be revised.

4.2 Recommendation

42

Based on the facts discussed throughout the report, it is evident that an industry-accepted guideline is requiredfor evaluating the performance of CVR deployment through M&V either in the form of a recommended practiceor a standard. The established document will help the utilities to evaluate the benefits consistently. In addition,it will be helpful for the public utility commissions to drive the CVR efforts smoothly. From the literature reviewand conducted survey, it is observed that utilities have carried out a considerable number of CVR efforts. Thisillustrates that the industry has matured and is ready for a standard.

REFERENCES

[1] ANSI, “ANSI Standard C84.1-2016 Electric Power Systems and Equipment Voltage Ratings (60 Hz).” 2016.[2] Z. Wang and J. Wang, "Review on Implementation and Assessment of Conservation Voltage Reduction,"

in IEEE Transactions on Power Systems, vol. 29, no. 3, pp. 1306-1315, May 2014[3] Z. S. Hossein et al., "Conservation Voltage Reduction and Volt-VAR Optimization: Measurement and

Verification Benchmarking," in IEEE Access, vol. 8, pp. 50755-50770, 2020[4] The Power Globe E-mail Forum., Available: http://www.ece.mtu.edu/faculty/ljbohman/peec/globe/[5] M. S. Hossan et al., "Deployment of Conservation Voltage Reduction in Sample Feeders: A Systemwide

Energy-Savings Analysis," in IEEE Industry Applications Magazine, vol. 27, no. 2, pp. 36-46, March-April2021

[6] S. DePasquale, (2015, August 11). “Why Volt/VAR Optimization is the most important technology you'venever heard of,” accessed 22 Oct. 2019. [Online]. Available: https://medium.com/@ScottDePasquale/why-volt-var-optimization-is-the-most-important-technology-you-ve-never-heard-of-fba3d6d4cde

[7] Tom Weaver, PE, (November 12, 2013), American Electric Power’s Experience with Applying Technology onthe Distribution System [PowerPoint slides], Available:https://www.engineering.pitt.edu/Sub-Sites/Conferences/EPIC/_Documents/2013-Conference-Documents/11-12-AM-GTC-Panel---Weaver-Weaver-(AEP)_pdf/

[8] Brian Schell, et al. (2014), Conservation and Optimization Via Volt/var Control [PowerPoint slides]. Available:http://mydocs.epri.com/docs/PublicMeetingMaterials/1028/Session_2_Conservation_and_Optimization_via_Volt-Var_Control.pdf

[9] American Electric Power (AEP) (2017), Smart Circuits, accessed 20 Oct. 2019,http://2017.aepsustainability.com/energy/technology/circuits.aspx

[10] Kentucky Power Company, a unit of American Electric Power. 2016. “Integrated Resource Planning Reportto the Kentucky Public Service Commission, Volume A – Public Version. (Case No. 2016–00413.),”available: https://psc.ky.gov/pscecf/2016-00413/jkrosquist%40aep.com/12202016110531/KPCO_2016_IRP__Volume_A___Public__Version.pdf

[11] American Electric Power (AEP) (2019), Modernizing Distribution, accessed 20 Oct. 2019,https://www.aepsustainability.com/energy/grid-modernization/distribution/

[12] New York State Public Service Commission. "Proceeding on Motion of the Commission in Regard toReforming the Energy Vision (CASE 14-M-0101): Order Adopting a Ratemaking and Utility Revenue ModelFramework." Report, New York, United States (2015). [Online]. Available:http://documents.dps.ny.gov/public/Common/ViewDoc.aspx?DocRefId=%7B3C9B9118-BBE7-4B3A-BF33-DDD2FA176D80%7D

[13] NRECA, “A Test Case for Conservation Voltage Optimization,” (2019) [PowerPoint slides], Available:https://www.techadvantage.org/wp-content/uploads/2018/10/03.13-0800-A-Test-Case-for-Conservation-Voltage-Optimization-Ward.pdf

43

[14] Ameren Illinois, “Conservation Voltage Reduction Pilot Project-ICC Docket 10-0568,” Nov. 2013. [Online].Available: https://s3.amazonaws.com/ilsag/Ameren_Illinois_CVR_Final_Report.pdf

[15] T. Rousan and L. Brown, “Voltage Optimization: Distribution Planning Considerations and Demand SavingsPotential for an Emerging IDSM Technology,” (2018), [PowerPoint slides], Available:https://www.peakload.org/assets/38thConf/Rousan&Brown-Voltage-Optimization.pdf

[16] Navigant Consulting, Inc. “ComEd CY2018-2021 Evaluation Plan,” Report, Feb. 2018. [Online]. Available:https://s3.amazonaws.com/ilsag/ComEd_CY2018-2021_Evaluation_Plan_2018-02-22.pdf

[17] ILLINOIS COMMERCE COMMISSION, “COMMONWEALTH EDISON COMPANY Annual energy efficiencyformula rate update and revenue requirement reconciliation pursuant to Section 8-103B of the PublicUtilities Act. (Docket No. 19-0580),” 2019. [Online]. Available:https://www.icc.illinois.gov/downloads/public/edocket/510471.pdf

[18] Navigant Consulting, Inc. “ComEd Voltage Optimization Program Impact Evaluation Report (EnergyEfficiency / Demand Response Plan: Program Year 2018 (CY2018) (1/1/2018-12/31/2018)),” Report, April30, 2019. [Online]. Available:https://s3.amazonaws.com/ilsag/ComEd_Voltage_Optimization_CY2018_Impact_Evaluation_Report_2019-04-30_Final.pdf

[19] P. Anderson, “Conservation Voltage Reduction (CVR),” (2016), [PowerPoint slides], Available:https://assets.fiercemarkets.net/public/sites/energy/reports/cvrreport.pdf

[20] PUBLIC UTILITY COMIVHSSION OF OREGON, “REGON PUBLIC UTILITY COMMISSION STAFF: Status Updatesof PGE's, PacifiCorp's and Idaho Power Company's respective Conservation Voltage Reduction Activities”,Report, August 31, 2015. [Online]. Available: http://oregonpuc.granicus.com/MetaViewer.php?view_id=1&clip_id=27&meta_id=1127

[21] Idaho Power Company, “SMART grid Report”, Report, September 30, 2016. [Online]. Available:https://docs.idahopower.com/pdfs/aboutus/companyinformation/smartgrid/2016OregonSmartGridReport.pdf

[22] S. Sergici, The Brattle Group,” Conservation Voltage Reduction Econometric Impact Analysis,” 2016,[PowerPoint slides], Available:https://brattlefiles.blob.core.windows.net/files/7290_aesp_2016_pepco_md_cvr_presentation_5-9-16.pdf

[23] Exelon Corporation, “Sustainability Report”, Report, 2018. [Online]. Available:https://www.exeloncorp.com/sustainability/Documents/dwnld_Exelon_CSR%20(1).pdf

[24] Light, Duquesne. “Final Annual report to the Pennsylvania Public Utility Commission, for Pennsylvania Act129 of 2008 Energy Efficiency and Conservation Program, Docket No. M-2009-2093218,” Report, 2013.[Online]. Available: http://www.oracle.com/us/industries/utilities/far-penn-puc-west-penn-py4-5225460.pdf

[25] Indiana Utility Regulatory Commission, “Indianapolis Power & Light Company-2014 Integrated ResourcePlan (Public Version),” Report, October 31, 2014. [Online]. Available:https://www.in.gov/iurc/files/IPL_2014_IRP_Report.pdf

[26] PECO, “PECO Energy Efficiency and Conservation Plan (Program Years 2009 – 2012)”, Report, July 1, 2009.[Online]. Available: http://www.puc.pa.gov/pcdocs/1047483.pdf

[27] F. Stern and N. DeDominicis, “Conservation Voltage Reduction: What Are the Savings?” InternationalEnergy Program Evaluation Conference, Chicago, IL, 2013.

[28] PECO, “PECO PROGRAM YEARS 2013-2015 ACT 129 - PHASE II ENERGY EFFICIENCY AND CONSERVATIONPLAN”, Report, November 1, 2012. [Online]. Available:https://www.peco.com/SiteCollectionDocuments/PECO%20Phase%20II%20PLAN%20v64_final.pdf

44

[29] PGE, “Integrated Resource Plan DRAFT”, Report, May 2019. [Online]. Available:https://www.portlandgeneral.com/-/media/public/our-company/energy-strategy/documents/2019-irp.pdf

[30] Sacramento Municipal Utility District, “Energy Research and Development Division FINAL PROJECTREPORT”, Report, SEPTEMBER 2014. [Online]. Available: http://www.energy.ca.gov/2014publications/CEC-500-2014-078/CEC-500-2014-078.pdf

[31] Sacramento Municipal Utility District, “Climate Readiness Assessment and Action Plan”, Report,NOVEMBER 2016. [Online]. Available:https://www.smud.org/-/media/Documents/Corporate/Environmental-Leadership/2016-Climate-Readiness-Action-Plan.ashx

[32] MetaVu, Inc., “Duke Energy Ohio Smart Grid Audit and Assessment”, Report, June 30, 2011. [Online].Available:https://www.smartgrid.gov/files/Duke_Energy_Ohio_Smart_Grid_Audit_Assessment_201104.pdf

[33] M. Simms, “Duke Energy Production Experience with CVR”, IEEE PES Working Group on Smart Distribution,2016, [PowerPoint slides], Available: http://grouper.ieee.org/groups/td/dist/da/doc/Duke%20Energy%20Production%20Experience%20with%20CVR.pdf

[34] EDF Environmental Defense Fund, “Energy Master Plan,” Report, 2018. [Online]. Available:https://www.nj.gov/emp/pdf/moderngrid/Mary%20Barber%20EDF%202019%20EMP%20Comments.pdf

[35] PUBLIC SERVICE COMPANY OF COLORADO, “BEFORE THE PUBLIC UTILITIES COMMISSION OF THE STATE OFCOLORADO, DIRECT TESTIMONY AND EXHIBITS OF KELLY BLOCH,” Report, 2013. [Online]. Available:https://www.xcelenergy.com/staticfiles/xe-responsive/Company/Rates%20&%20Regulations/Regulatory%20Filings/Direct-Testimony-Bloch-2013.pdf

[36] Advanced Energy Economy, “Advanced Energy Roadmap for Colorado”, Report, 2018. [Online]. Available:https://info.aee.net/hubfs/Colorado%20Policy%20Platform%20Final%20(3).pdf

[37] Xcel Energy, “Settlement Agreement”, 2019. [Online]. Available:https://www.dora.state.co.us/pls/efi/efi_p2_v2_demo.show_document?p_dms_document_id=863954&p_sessi on_id=

[38] Navigant Consulting, Inc., “Avista Utilities’ Conservation Voltage Reduction Program-Impact Evaluation”,Report, May 1, 2014. [Online]. Available:https://www.utc.wa.gov/_layouts/15/CasesPublicWebsite/GetDocument.ashx?docID=1005&year=2016&docketNumber=160228

[39] The Edison Foundation, “INNOVATIONS ACROSS THE GRID-Partnerships Transforming The Power Sector”,Report, DECEMBER 2013. [Online]. Available:https://www.edisonfoundation.net/iei/publications/Documents/IEI_InnovationsAcrosstheGrid_FINAL.pdf

[40] “PG&E to Use Dominion Voltage for Volt/VAR Optimization Pilot Program,” accessed 28 Oct. 2019,https://www.tdworld.com/smarter-grid/pge-use-dominion-voltage-voltvar-optimization-pilot-program

[41] California Energy Commission, “Voltage and Reactive Power Optimization Pilot Summary and PotentialDeployment Plans”, 2018, [PowerPoint slides], Available:https://efiling.energy.ca.gov/GetDocument.aspx?tn=223679&DocumentContentId=53835

[42] SCE, “SOUTHERN CALIFORNIA EDISON COMPANY’S (U-338-E) ANNUAL REPORT ON THE STATUS OF THEELECTRIC PROGRAM INVESTMENT CHARGE PROGRAM,” 2019. [Online]. Available:http://www3.sce.com/sscc/law/dis/dbattach5e.nsf/0/359A9F0FC59AD3FB882583AF0081D425/$FILE/A1704028%20et%20al-SCE%202018%20EPIC%20Annual%20Report.pdf

[43] GWP, “UTILITY MODERNIZATION,” Report, 2015. [Online]. Available:https://www.glendaleca.gov/home/showdocument?id=24879

[44] California Energy Commission, “GWP Conservation Voltage Reduction Program”, 2018, [PowerPointslides], Available: https://efiling.energy.ca.gov/GetDocument.aspx?tn=223680

45

[45] Puget Sound Energy, “Evaluation of the Energy Savings Impacts of PSE’s Conservation Voltage ReductionProgram,” Report, January 2018. [Online]. Available:https://conduitnw.org/Handlers/conduit/FileHandler.ashx?RID=4284

[46] Dominion Voltage Inc., “Utility Case Study: Volt/VAR Control at Dominion”, 2012, [PowerPoint slides],Available: https://www.michigan.gov/documents/energy/Powell_418130_7.pdf

[47] INDIANA MICHIGAN POWER COMPANY, “PRE-FILED VERIFIED DIRECT TESTIMONY OF JON C. WALTER(Cause No. 44841),” Report, 2016. [Online]. Available:https://www.indianamichiganpower.com/global/utilities/lib/docs/info/projects/IMDemandSideManagement/44841%20Jon%20C%20Walter%20%20Direct%20Testimony%20&%20Attachments%20Vol%20I.pdf

[48] Indiana Michigan Power Company, “INTEGRATED RESOURCE PLANNING REPORT TO THE INDIANA UTILITYREGULATORY COMMISSION”, Report, July 1, 2019. [Online]. Available:https://www.indianamichiganpower.com/global/utilities/lib/docs/info/projects/IMIntegratedResourcePlan/2018-19%20IM%20IRP.pdf

[49] ADM Associates Inc., “2018 Indiana EECO Program EM&V Report”, Report, May 2019. [Online]. Available:https://www.indianamichiganpower.com/global/utilities/lib/docs/info/projects/IMDemandSideManagement/2018IMIndianaEECOEMVReport.pdf

[50] STATE OF NEW JERSEY BOARD OF PUBLIC UTILITIES, “PUBLIC SERVICE ELECTRIC AND GAS COMPANYDIRECT TESTIMONY OF KAREN REIF VICE PRESIDENT RENEWABLES & ENERGY SOLUTIONS,” Report, 2018.[Online]. Available:https://nj.pseg.com/aboutpseg/regulatorypage/-/media/7F560FA03D37431186B0E78A11218E17.ashx

[51] U.S. Department of Energy, Electricity Delivery and Energy Reliability, “Voltage and Power OptimizationSaves Energy and Reduces Peak Power,” 2015. [Online]. Available:https://www.smartgrid.gov/document/Voltage-Power-Optimization-Saves-Energy-Reduces-Peak-Power.html

[52] KCP&L, “KCP&L GREEN IMPACT ZONE SMARTGRID DEMONSTRATION-FINAL TECHNICAL REPORT (DOE-KCP&L-0000221),” Report, 2015. [Online]. Available:https://www.smartgrid.gov/files/OE0000221_KCPL_FinalRep_2015_04.pdf

[53] Dominion Voltage Inc., “DVI Voltage Optimization Technology Helps Choptank Electric Cooperative ReduceEnergy Demand by 5% in 2018,” March 2019, accessed 25 Oct. 2019, https://www.prnewswire.com/news-releases/dvi-voltage-optimization-technology-helps-choptank-electric-cooperative-reduce-energy-demand-by-5-in-2018-300807690.html

[54] Cooperative Research Network and NRECA, “Costs and Benefits of Conservation Voltage Reduction CVRWarrants Careful Examination”, REPORT, MAY 31, 2014. [Online]. Available:https://www.smartgrid.gov/files/NRECA_DOE_Costs_Benefits_of_CVR_0.