PQDIFPQDIF:A Technical Overview

Prepared by:Erich Gunther, Bill Dabbs, and Rob Scott

Electrotek Concepts, Inc.

NEW!IMPROVED!

PQDIFWhat are the goals of PQDIF?

• Provide a platform-neutral way to interchange Power Quality data between software packages and instruments.

• Be standardized by IEEE to provide a stable target for third-party developers.

PQDIFWhat other PQ formats

are out there?• IEEE COMTRADE

– Excellent for waveform and simple variable trend data, but is not suitable for other types of PQ data such as min/max/avg RMS trends, harmonic spectra, probabilities, event logs, or lightning flash data.

• PQView® and other databases– Excellent for analysis and storage of massive

quantities of data, but not for an interchange format.

PQDIFThe Desired Characteristics

• Simple• Extensible

… but without breaking old apps• Compact

… preferably binary… optional compression

• Flexible

PQDIFThe Basic Design

• A PQDIF file is made up of a set of “records” which are logically related:– Data Source (generally an instrument)– Monitor Settings (optional)– Observations

• Within each record, there is a set of “elements” which define the contents of the record.

PQDIFThe Record Structure

• The “high-level” record structure is completely separated from the “low-level” element structure.– The overall structure of the file can be parsed

quickly without having to read the entire file.• This will also allow us to use different high-

level structures in the future (storing records directly in a database, for example).

PQDIFThe Element Structure

• The internal record structure is made up of a set of tagged data “elements.”

• Because each element is tagged, there is a separation between the “physical” and “logical” structure.

• This allows the logical structure to be defined separately from the physical structure.

PQDIFThe four aspects of the design

High-level Low-level

Physical Record headerstructure

Record bodystructure

Logical Implicit recordhierarchy

Standard tags andhierarchy definitions

PQDIFHigh-level

logical format• The records are strung

together in an implicit hierarchy, based on the record type tags.– The file starts with a

Container record, which “owns” any number of Data Source records.

– Each Data Source record owns Monitor Settings and Observation records.

Container record (only one)

Data source record 1

Monitor settings record 1 (optional)

Observation record 1

Observation record 2

Observation record n

…

Data source record 2

Data source record n

…

Monitor settings record n (optional)

PQDIFThe high-level physical format

• At the highest level, the file is made up a series of linked records.

• This linked nature allows manipulations to the record structure without having to rewrite the entire file.

Record 1

Record 2

…

Record 3

Record n

PQDIFThe high-level physical format:

Inserting a recordRecord 1

Record 2

…

Record 3

Record n

Record 2B

• We can insert a new record -- even if it is physically at the end of the file.

PQDIFThe high-level physical format:

Deleting recordsRecord 1

Record 2 (deleted)

…

Record 3

Record n

• We can also delete a record by adjusting the links to skip around it.

PQDIFThe high-level physical format:

Reordering recordsRecord 1

Record 2

…

Record 3

Record n

• Likewise, we can reorder existing records by adjusting the links.

PQDIFInternal record structure

• Each record is made up of a “header” and a “body.”– The header provides

the high-level information.

– The body contains the low-level information.

PQDIFLow-level physical format

• The body of each record is made up of a set of “elements” which contain data.

• There are three elements:– Scalar represents a single data

value – Vector represents an array of data values– Collection contains other elements

PQDIFThe low-level physical format:

Identifying elements• In order to be able to identify a specific

element, each one is given a “tag.”• This tag is actually a GUID*, which makes

extending the list of tags virtually foolproof.

* A “Globally Unique Identifier” is a 16-byte integer which is defined by a standard algorithm. Any computer in the world can generate a GUID and be reasonably assured that it is absolutely unique.

PQDIFThe low-level physical format:

Think of it as a file system ...• A tag is a file name.• A collection is a

subdirectory.• A scalar is a file with one

number in it.• A vector is a file with lots of

numbers (or a string) in it.• A subdirectory (collection)

can contain another subdirectory.

Contents:

7200.0

Contents:Count: 20: 1101: 235

Contents:

“PQNode 8010”

PQDIFThe low-level physical format:

Primitive data types

– Signed integer (1, 2 or 4 bytes)

– Unsigned integer (1, 2 or 4 bytes)

– Boolean (1, 2 or 4 bytes)– Real (4-byte single or 8-

byte double precision)

– Complex (8-byte single or 16-byte double precision)

– Character (1-byte ASCII or 2-byte Unicode)

– Date stamp (12 bytes)– GUID (16 bytes)

• A scalar or vector element can hold values of the following types:

PQDIFLow-level logical format

• The low-level physical format allows each element to have a tag.

• The logical format defines what the tags are, and what element type and primitive data type(s) are expected for each.

• Since collections can contain other collections, the logical format must also specify this hierarchy.

PQDIFAn example...

PQDIFBenefits of a tagged format

• The tagged element structure provides a great degree of flexibility and allows the contents to be defined logically independently of the actual physical definition.

PQDIFWork Completed

• Physical format finalized.• Version 1.5 logical format finalized and documented• Third generation COM library for manipulating PQDIF

1.5 complete (written in C++)• Simple PQDIF reading and writing code examples

suitable for embedded systems complete (written in C)• Command line tool to dump PQDIF structure and data.• Program to convert Square D DaDisp based waveform

files and PASS 8010 Transient files to PQDIF.• Examples of PQDIF viewers (one VB example and one

Microsoft MFC example with source).

PQDIFWork still to be done

• More tools and examples.– COMTRADE converter– PASS 8010 PQNode Converter - support more types.

• More field trials.– Salt River Project - PML translator– TVA - substation monitoring project– Carolina Power and Light

• Provide leadership, documentation and sample code and for IEEE standards process.

PQDIFConclusion

• PQDIF meets all of our goals.• If handled properly, PQDIF can become a

widely-accepted, stable format.• With appropriate software tools, it appears

to solve a number of technical issues for our projects (PQDS, for example).