promotion of af systems for weekly database loads · promotion of af systems for weekly database...
TRANSCRIPT
Aaron Rosenthal - Operations Engineer, ERCOT
September 14, 2016
Promotion of AF Systems for Weekly Database Loads
About ERCOT
2
Introduction – ERCOT
• Electric Reliability Council of Texas, Inc.
– ISO/RTO for the state of Texas
• Four primary responsibilities
– System reliability – planning and operations
– Wholesale market settlement for electricity production and delivery
– Retail switching process for customer choice
– Open access to transmission
3
ERCOT Quick Facts
• 90% of Texas load
– 24 million customers
– 75% of load is retail-choice
• 77,000+ MW expected generation
• Record peak load of 71,197 MW
– August 11, 2016
• 16,000+ MW of installed wind capacity
– Most of any state in the nation
• 288 MW of installed solar capacity
4
PI at ERCOT
• Started using PI in 2010
• 530,000+ PI tags
– ~303,000 from SCADA
– ~123,000 for Performance Equations
– ~61,000 for AF analyses
• 2TB total used archive space
– ~2GB daily archive size
5
6
Official Control Room PI Displays
Next-gen Displays
• Prototype for newrenewables desk
• Tracking wind curtailment
• Uses web technologies
– PI Web API
– AngularJS
– Highcharts
7
9
Building Asset Framework
9
Requirements Gathering Build Asset Model DTAP PromotionDevelop AF Library
Determine existing PI tag naming conventions for various asset types
Decide on key asset metadata for both analysis and display purposes
Plan element hierarchy structure for both analysis and navigation purposes
Transform CIM model data into AF XML file format
Define elements based on both template and category information for maximum rollup analysis capability
Develop, Test, Acceptance, Production
Compare previous model file and model configuration to current
Create delta files to import into target environments for reduced import times
Build element templates based on requirements
Ensure template configuration is decoupled from the server environment
Include table connection configuration to make data outside PI accessible
Building Asset Framework
Study Develop Test Acceptance Production
Building Asset Model
• Asset Framework model file built weekly using custom queries of CIM model (stored in Oracle database)
– Data transform of CIM model snapshot
– Formatted in AF XML schema format
• XML file defines element structure
– Parent-child relationships and element references
– Template, categories, and attribute values
10
Comparing Model Files
• Comparison of model files (previous and current) to produce difference file
– Results in significantly faster import times (minutes versus hours)
– Necessary to capture deleted or renamed assets
• Fundamental building block of XML file is the <AFElement> XML block
– Traces a unique element path in the element hierarchy
11
Model File Structure
Flat Structure Nested Structure
12
Model File Structure
• Can be combination of flat and nested structure
• Allows maximum flexibility for developers of CIM model data transform queries
13
Model File Merging
• Merging process combines XML nodes that have identical paths
• Facilitates difference file creation algorithm
– No need to account for multiple path locations
• Preserves original order of XML nodes
– Necessary to avoid broken element reference dependencies
• Prescribes to AF XML schema validation rules
14
Example: Model File Merging
Before Merge After Merge
15
Difference Detection
• Deleted elements
• Unchanged elements (not included)
• New elements
• Renamed elements
16
Previous Model
New Model
Difference File Creation
• File structure can include a combination of nested and flat element and/or attribute value structures
• Path to same element can be defined in multiple locations (developer flexibility)
• Alternative keys can be used for element rename detection, for example:
– attribute value or extended property storing unique ID from CIM model
17
Difference Detection Algorithm
• Build compare key for all XML nodes using <Name> child node as primary key
• Operates from “inside-out”: XML nodes processed in order of decreasing depth
18
Example: Compare Key
19
• Path strings are guaranteed unique thanks to AF structure• Note – Flat paths don’t need to be expanded due to merging process
Example: Difference Detection
Previous Model New Model
20
Deleted Elements/Attributes
• Element/attribute nodes missing from new model file indicate a deleted attribute or element
• Indicated by operation=“delete” attribute in XML
• Program must keep track of parent XML nodes in new model file with same path as parent node in old file
– Needs knowledge of where to insert delete operation
• Note – Element references can also be deleted
21
Example: Deleted Elements
Previous Model New Model
22
Special Case: Attribute Reset to Template
• Attribute nodes missing from new model which are part of an attribute template
– Indicate a “reset to template” operation
– Requires knowledge of AF library configuration
– Sets attribute default value, or data reference/configuration string (if available)
23
Rename Detection
• Only necessary for <AFElement> nodes
• Build rename keys to all <AFElement> nodes using a combination of alternative keys (e.g., attribute or extended property value) and <Name> nodes
• Build element path strings to all <AFElement> nodes
• Renames occur when rename keys match but element path strings do not
24
Example: Rename Detection
Previous: Path\To\Element New: Path\To\RenamedElement
25
Compare key: AFElement[Name=Path]/AFElement[TEID=12345]
Renamed Elements
• Functionally the same as a create-plus-delete operation
• Rename detection facilitates PI tag renaming
• Note – Would be superior if operation=“rename” attribute existed in the AF XML schema
– Feature would automatically re-evaluate PI tag configuration strings and perform a tag rename if necessary
26
Library File Comparisons
• Development (or “gold standard”) library export
– Compared against multiple environment exports (test, production, etc.)
• Compare key generation follows similar logic
– Traces ancestor XML nodes
– Use <Name> child element as primary key
• Note – Rename detection not necessary for library files
27
Library File Normalization
• Library files require some preliminary transformations
– Replace references to PI system and AF database in the “gold standard” library before comparing
– Remove ‘override’ extended properties
– Remove sensitive or environment specific information
28
Library Difference File Creation
• Create/update/delete detection follows same logic
• Some exceptions:
– AFTable
– AFTableConnection
– AFAnalysisRule
– AFTimeRule
29
Ongoing Challenges
• Analysis/notification instance statuses
– Changes in development currently not propagated
• PI tag renames for attribute templates
– Complicated by multi-part renames
– Requires interaction with AF SDK or PI builder
• Better version control for library changes
– Possible integration with Git/Bitbucket
• What to do with GUIDs?
30
Aaron Rosenthal
Operations Engineer II
Advanced Network Applications
ERCOT, Inc.
31
Thank You