Datasets

Steve Edburg

Mohamed Abukaram

2 Las Vegas 20192

Agenda

• Global Datasets Team

• Database Best Practices

• Recent Database Updates

• New Products

• Future Plans

3 Las Vegas 20193

Global Datasets Team

Who: • Market research and analysis group providing simulation ready data for all

software products

What: • Datasets in North America, Latin America, Europe, Australia, and New Zealand• New datasets in consideration for Middle East, Africa, Japan and other Asian

countries

Why: • Simulation ready datasets to save you time and improve your analyses

• Research, collect and implement publicly available data • Supplement with expert market research• Properly benchmarked and transparent solutions

• Extensive data documentation, benchmarking analyses, and support

4 Las Vegas 20194

• Growing in numbers, expertise, and automated updates• Local expertise from global offices

• Increased industry expertise

• Investing in process improvements and automation

• Using agile framework for database development

2017

•Three analysts

2018

•Six analysts

2019

•Nine analysts

Global Datasets Team

Data Poll

www.slido.com

Event code = #EMFC

The most importance aspect about data for me is:

6 Las Vegas 2019

Agenda

• Global Datasets Team

• Database Best Practices

• Recent Database Updates

• New Products

• Future Plans

7 Las Vegas 20197

Overall Update Process

Start

Gather &

Process

Data

Implement into

Master DB

End

Analyze in

PLEXOS/Aurora

for Zonal ST/LT

Good?y

n

Good?Benchmarking

Analyses

y

n

Documentation

8 Las Vegas 20198

Benchmarking Process

Start

Download Historical

Drivers

Process Evaluation Data

Process into Simulation Ready

Data

Download Historical

Evaluation Data

EndBenchmarkingAnalyses

DocumentationConduct

Simulations

Implement Data Enhancements

9 Las Vegas 20199

Example Benchmarking Results

10 Las Vegas 201910

Example Benchmarking Results

11 Las Vegas 201911

Transparency of Inputs

• Internal metadata tracking for major variables for increased transparency

• Mapping to ISO sources with IDs• WECC ADS

• ERCOT CDR

• NYISO Gold Book

• ISONE CELT

• PJM RPM

12 Las Vegas 201912

Infrastructure and Tooling

• Cloud infrastructure• Developing & testing datasets

in cloud environment

• Cloud tooling• Version management &

collaboration

13 Las Vegas 201913

Stakeholder Engagement

• Presales • Customer insights

• Priorities

• Engineering • Ease of use

• Well-founded implementation

• Software• Performance improvements

• Algorithm improvements

Data Pollwww.slido.com

Event code = #EMFC

Which regions are you most interested in?

15 Las Vegas 2019

Agenda

• Global Datasets Team

• Database Best Practices

• Recent Database Updates

• U.S. Zonal (WECC focus)

• General PLEXOS enhancements

• PLEXOS ERCOT Nodal

• New Products

• Future Plans

16 Las Vegas 201916

Self Service

• One-stop shop for datasets, software, support, and more

https://portal.energyexemplar.com/

17 Las Vegas 201917

WECC Dataset Update (Q1 2019)

• Updates to:• Demand

• Link losses

• Emissions

• Resources

• RPS

• Fuel forecasts

• Benchmarking

18 Las Vegas 201918

WECC Dataset Update (Q1 2019)

19 Las Vegas 2019

Agenda

• Global Datasets Team

• Database Best Practices

• Recent Database Updates

• U.S. Zonal (WECC focus)

• General PLEXOS enhancements

• PLEXOS ERCOT Nodal

• New Products

• Future Plans

20 Las Vegas 201920

General Data Enhancements (PLEXOS)

Nodal Generator Mapping

• Interconnection queues used

•Regional mapping applied in some cases

•Geolocations for nodes and generators used in mapping

Battery Class

•Generators modeled using battery class for enhance modelling

Fuel Restructure

•Restructured and categorized for ease of use

21

General Data Enhancements (PLEXOS)

BATTERIES BEFORE BATTERIES AFTER

22 Las Vegas 201922

General Data Enhancements (PLEXOS)

FUELS BEFORE

FUELS AFTER

Membership

Property

Membership

Property

1

2

1

23 Las Vegas 2019

Agenda

• Global Datasets Team

• Database Best Practices

• Recent Database Updates

• U.S. Zonal (WECC focus)

• General PLEXOS enhancements

• PLEXOS ERCOT Nodal

• New Products

• Future Plans

24 Las Vegas 201924

PLEXOS ERCOT Nodal

• Product• Development of 2019-2025 future years SSWG case within one dataset

• Features• Detailed generation modeling with long-term expansion plan provided

as a scenario• Base assumptions structured for client-specific scenarios• Expert market research and implementation of interfaces, hub

definitions, and contingencies• Expert analysis to verify power flow with PSS®E and benchmark to

historical observations

• Example Use Cases• Transmission modeling• Resource valuation• Market analysis

25 Las Vegas 201925

Inputs for the ERCOT nodal case

Topology

Steady State Working Group (SSWG)

2019 to 2025

Evaluated with PSSE and PLEXOS

Load Mapping

Load forecasts at Region

Load mapped to nodes via load participation factor

Generator Mapping

Comprehensive generator mapping

Geolocations used

26 Las Vegas 201926

Inputs for the ERCOT nodal case

Interfaces

Market research from ERCOT sources

Hubs

Market research from ERCOT sources

Contingencies

Historical binding constraints reports

Monitored elements provided

27 Las Vegas 201927

Results: Backcast Benchmarking

• Installed capacity & generation by fuel type

27

28 Las Vegas 201928

Results: Backcast Prices

• Houston hub – July 2018, hourly LMPs

28

Data Pollwww.slido.com

Event code = #EMFC

Preference of dataset update delivery?

Preference of dataset delivery frequency?

30 Las Vegas 2019

Agenda

• Global Datasets Team

• Database Best Practices

• Recent Database Updates

• New Products

• Aurora Regional Carveouts

• PLEXOS East Interconnect (EI) Nodal

• Future Plans

31 Las Vegas 201931

Aurora Regional Carveouts

What: • Divide US_Canada database into regions

Why:• Improve your experience

• Smaller database

• Faster load times

• Removes unneeded data

When:• Available in Q4 2019 https://www.eia.gov/todayinenergy/detail.php?id=27152

32 Las Vegas 201932

Aurora Regional Carve Outs

• Process aimed at reducing complexity in database to improve your experience

• Removes unused:

Tables

•RMTs

•Capacity Price

•Conditions

•Examples

Columns

•Demand

•Monthly

•Hourly

•Escalation

• TS Hourly

Records

• Time Series

•Resources

• Fuel

• Zone Def.

•Many others

33 Las Vegas 201933

Aurora Regional Carve Outs

Database Size (MB)

Records in

Resources

Table

Records in TS

Annual Table

Columns in TS

Hourly Table

Columns in

Demand

Tables

US_Canada 89 26,000+ 1,924 214 206

WECC 23 6,000+ 185 69 53

Database properties for full US & Canada DB compared to the WECC Carve out.

34 Las Vegas 201934

PLEXOS East Interconnect Nodal

• PJM Multi-Model Working Group (MMWG) annual base cases (2018 series)• 2019, 2020, 2023, 2028

• Features• All future years in one XML• Regional generator mapping• Generators added from queues• Benchmarked & documented

• Performance• Annual hourly simulation(8760) completes in under 10 hours!

35 Las Vegas 2019

Agenda

• Global Datasets Team

• Database Best Practices

• Recent Database Updates

• New Products

• Future Plans

36 Las Vegas 201936

Data Team Road Map

• Provide simulation-ready datasets for PLEXOS and Aurora

Short-term (months)

•Provide nodal solutions for both tools

•Improve quality of benchmarking

Medium-term (quarters)

•Up-to-date zonal and nodal solutions in parallel

•Improve processes for parallel updates

Long-term (quarters to year)

•Cloud data offering

•Automated updates for some properties

37 Las Vegas 201937

PLEXOS Dataset Release Plan

Q4 2019

• EI Nodal

•WECC zonal

Q1 2020

•Mexico

• Europe

Q2 2020

•WECC nodal

• ERCOT nodal

38 Las Vegas 201938

Aurora Dataset Release Plan

Q4 2019

•WECC zonal

Q1 2020

•Mexico

• Europe

Q2 2020

•WECC nodal

• ERCOT nodal

39 Las Vegas 201939

Delivering World Class Datasets

Team

•Growing in numbers and expertise

•More investments planned for 2020 & beyond

Process

• Process review and improvement plans made

• Investments in automation and cloud tooling continue

Quality

• Improving benchmarking and best practices

• Seeking your feedback during UGM

Next Generation

• Engaging with industry leaders on sub-hourly datasets

Thank you