keysight test automation platform (tap)€¦ · this document describes the programmatic interface...

Developer Guide

Keysight Test Automation

Platform (TAP)

Notices

DFARS/Restricted Rights Notice

If software is for use in the performance of a U.S. Government prime contract or subcontract,

Software is delivered and licensed as "Commercial computer software" as defined in DFAR

252.227-7014 (June 1995), or as a "commercial item" as defined in FAR 2.101(a) or as "Restricted

computer software" as defined in FAR 52.227-19 (June 1987) or any equivalent agency regulation

or contract clause. Use, duplication or disclosure of Software is subject to Keysight Technologies’

standard commercial license terms, and non-DOD Departments and Agencies of the U.S.

Government will receive no greater than Restricted Rights as defined in FAR 52.227-19(c)(1-2)

(June 1987). U.S. Government users will receive no greater than Limited Rights as defined in FAR

52.227-14 (June 1987) or DFAR 252.227-7015 (b) (2) (November 1995), as applicable in any

technical data.

Warranty

THE MATERIAL CONTAINED IN THIS DOCUMENT IS PROVIDED "AS IS," AND IS SUBJECT TO

BEING CHANGED, WITHOUT NOTICE, IN FUTURE EDITIONS. FURTHER, TO THE MAXIMUM

EXTENT PERMITTED BY APPLICABLE LAW, KEYSIGHT DISCLAIMS ALL WARRANTIES, EITHER

EXPRESS OR IMPLIED WITH REGARD TO THIS MANUAL AND ANY INFORMATION CONTAINED

HEREIN, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY

AND FITNESS FOR A PARTICULAR PURPOSE. KEYSIGHT SHALL NOT BE LIABLE FOR ERRORS OR

FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES IN CONNECTION WITH THE FURNISHING,

USE, OR PERFORMANCE OF THIS DOCUMENT OR ANY INFORMATION

Technology Licenses

The hardware and/or software described in this document are furnished under a license and may

be used or copied only in accordance with the terms of such license.

© Keysight Technologies, Inc. 2015, 2016, 2017

Keysight Technologies, Inc.

900 South Taft Avenue

Loveland, CO, 80537-6378 USA

Document History

Published June 15, 2017 for TAP version 7.4.

Table of Contents

Introduction ................................................................................................................................................ 5

Audience ............................................................................................................................................ 5

TAP SDK ............................................................................................................................................. 5

Prerequisites ...................................................................................................................................... 5

Suggested Resources ........................................................................................................................ 5

What is TAP? ............................................................................................................................................... 6

TAP Architecture ................................................................................................................................ 6

TAP Engine ......................................................................................................................................... 7

TAP Graphical User Interface ............................................................................................................ 7

TAP Command Line Interface ............................................................................................................ 7

TAP API ............................................................................................................................................... 8

Plugin Modules .................................................................................................................................. 8

Test Plans ........................................................................................................................................... 8

Getting Started in Visual Studio ................................................................................................................ 8

View the Visual Studio Example Projects .......................................................................................... 9

Create a Project with the TAP Plugin Template ............................................................................. 13

Plugin Development Basics ...................................................................................................................... 14

TAP Plugin Object Hierarchy ........................................................................................................... 14

Best Practices for Plugin Development .......................................................................................... 15

Developing Using Interfaces ........................................................................................................... 15

Test Step Plugin Development ................................................................................................................. 16

Default Implementation ................................................................................................................... 16

TestStep Hierarchy .......................................................................................................................... 17

Verdict .............................................................................................................................................. 19

Log Messages .................................................................................................................................. 20

Validation ......................................................................................................................................... 23

Publishing Results ........................................................................................................................... 24

Serialization...................................................................................................................................... 25

Inputs and Outputs .......................................................................................................................... 26

Result Listener Plugin Development ........................................................................................................ 27

Custom Result Listeners .................................................................................................................. 27

TAP SQL Database ........................................................................................................................... 28

DUT Plugin Development ......................................................................................................................... 30

Instrument Plugin Development .............................................................................................................. 30

SCPI Instruments ............................................................................................................................. 31

Using Attributes in Plugins ....................................................................................................................... 32

Attributes Used by TAP .................................................................................................................... 32

Attribute Details ............................................................................................................................... 33

External Parameters ........................................................................................................................ 37

Component Settings ................................................................................................................................. 38

Global and Bench Settings .............................................................................................................. 38

Reading and Writing Component Settings ..................................................................................... 40

Connection Management ................................................................................................................ 41

Plugin Packaging and Versioning ............................................................................................................ 41

Packaging ......................................................................................................................................... 41

Packaging Configuration File .......................................................................................................... 42

Command Line Use ......................................................................................................................... 43

Versioning ........................................................................................................................................ 44

Advanced Test Plan Development ........................................................................................................... 45

Test Plan Control Flow .................................................................................................................... 45

Manual Resource Connection ......................................................................................................... 48

Testing Multiple DUTs ..................................................................................................................... 49

Command Line Interface ................................................................................................................. 50

Range Notation ................................................................................................................................ 51

Appendix A: Files Included in the TAP Installation .................................................................................. 52

Introduction

This document describes the programmatic interface to Keysight's Test Automation Platform (TAP)

and shows how to get started using TAP for implementing test steps, instrument plugins, DUT

plugins and result listeners.

Audience

This document is written for C# programmers who are developing TAP plugins or integrating TAP

into their own applications. It is not a reference manual, but rather a document that describes the

principles behind TAP and how to use its most important features from a programmer’s

perspective. Other documents cover an introduction to the TAP GUIs.

TAP SDK

TAP includes a Software Development Kit (SDK). The SDK includes Microsoft Visual Studio

2013/2015 extensions that provide code templates for creating TAP plugins. To install it, select

the SDK option during TAP installation.

Prerequisites

Before reading this document, you should be familiar with using TAP from the TAP GUI. Refer to

the TAP Graphical User Interface Help documentation accessed from within TAP with the product

and via the Keysight websites.

Development requires the following software:

Visual Studio 2013 or 2015

TAP, including the SDK

TAP license

Suggested Resources

Keysight I/O libraries

What is TAP?

TAP is a software platform for fast and easy development and execution of automated test and/or

calibration algorithms. These algorithms typically control Keysight (and other vendors) instruments

and possibly customer-specific devices under test (DUTs). By leveraging the features of C#/.NET

and providing an extendable architecture, the TAP SDK minimizes the code needed to be written

by the programmer.

TAP offers a range of functionality and infrastructure for configuring, controlling and executing test

algorithms. TAP provides an API for implementing plugins in the form of test steps, instruments,

DUTs and more.

TAP consists of multiple executables, including:

TAP Engine (as a dll)

TAP Graphical User Interface (GUI)

TAP Command line interface (CLI)

Timing Analyzer

Result Viewer

Steps frequently have dependencies on the DUT and on instrument plugins. Development of

different plugins is discussed in the following section.

TAP Architecture

This graphic shows how the TAP Engine is central to the TAP architecture, and how plugins (all the

surrounding items) integrate with the engine.

TAP

Engine

DUT Plugins Custom plugins for devices under test

Test Steps

Basic steps

Flow Control steps

Custom steps

User Interfaces

Graphical User Interface

Command Line Interface

Web interface

Result Listeners

PostgreSQL Database

SQLite Database

CSV

Text Log

Instrument Plugins

Keysight Instruments

Non-Keysight Instruments

Custom Instruments

Tools

Timing Analyzer

Results Viewer

Test Plan Run Explorer

Plugin Package Manager

Included with TAP

User-developed Public API

TAP Engine

The TAP.Engine assembly is the core of TAP and is required for any TAP plugin. The most

important classes in the TAP.Engine are TestPlan, TestStep, Resource, DUT, Instrument,

PluginManager and ComponentSettings. The engine also makes an API available, which is used by

GUI, the CLI, and other programs. The TAP Engine can be controlled by either the CLI, GUI or API.

TAP Graphical User Interface

The TAP GUI is an application that runs on top of the TAP Engine.

The GUI facilitates test plan configuration and execution by providing the features of TAP in a well-

structured and simple interface. It interfaces with the TAP API and uses C# reflection to expose

configuration options to the user. The GUI provides access to test step, instrument and DUT

configuration, as well as information logged during test plan execution. Note that:

If a GUI is not needed or desired, test plans can be executed using the TAP Command Line

Interface (CLI).

It is possible to build custom GUIs to expose only the functionality necessary for test

execution or to integrate TAP into preexisting UIs. This can be useful for operators in a

production environment.

Users can specify one or more of the following command line arguments when starting the

GUI:

Command Description Example

Open Opens the specified test plan

file.

Keysight.Tap.Gui.exe --open testplan.tapplan

Add Adds a specific step to end of

the test plan.

Keysight.Tap.Gui.exe --add VerifyWcdmaMultiRx

Search Allows PluginManager to search

the specified folder for installed

plugins or available (but

uninstalled) plugins. Multiple

paths can be searched.

Keysight.Tap.Gui.exe --search C:\myPlugins

TAP Command Line Interface

The TAP CLI is a console program that executes a test plan and allows easy integration with other

programs. The CLI has options to configure the test plan execution, such as setting external

parameters, setting meta data, and executing TAP as a service. For more information on CLI, see

Command Line Interface on page 50.

TAP API

The TAP API allows software to configure and run a test plan. A help file for the C# API

(TapApiReference.chm) is available in TAP installation folder. Examples on how to use the TAP API

can be found in the TAP_PATH\SDK Examples folder.

Plugin Modules

An essential feature of TAP is a flexible architecture that lets users create plugins.

TAP plugins can be any combination of TestStep, Instrument, and DUT implementations. Other

TAP components such as Result Listeners and Component Settings can be included in plugin

packages, as well as non-TAP files (such as data files or README files).

By default, TAP comes with plugins covering basic operation, flow control and SQL database

storage. Writing plugins is a major subject of this document.

Plugins are managed by the Plugin Package Manager application, which by default searches for

assemblies in the same directory as the running executable (the GUI, CLI or API). Additional

directories to search can be specified for GUI, the CLI and the API. When using the API, use the

Plugins.DirectoriesToSearch function to retrieve the list of directories, and add any new directories

to the list.

Test Plans

A test plan is a sequence of test steps with some additional attached data. Test plans are created

via the TAP GUI. Developing test plans is a major subject in the TAP GUI Help (TapGuiHelp.chm),

accessed within the TAP GUI. Test plan files have the .TapPlan suffix, and are stored as xml files.

Getting Started in Visual Studio

TAP plugin development begins with the Keysight TAP Plugin template, a Visual Studio extension

installed with the TAP SDK. This template:

• Contains TAP plugin classes and gives you a head start to plugin development by setting

up the project and providing code skeletons.

• Copies the project’s output to the location specified by the TAP_PATH environment

variable, which enables TAP to load the plugins during development.

View the Visual Studio Example Projects

Before you create your own project, look at the projects and files in TAP_PATH\SDK Examples.

This folder provides code for example DUT, instrument and test step plugins. First-time TAP

developers should browse and build the projects, then use the TAP GUI to view the example DUTs,

instruments and test steps.

SDK Examples contains the following:

Folder Description

ExamplePlugin\ExamplePlugin.csproj Creates a plugin package that contains one DUT resource,

one Instrument resource, and one test step.

PluginDevelopment\PluginDevelopment.csproj Creates a plugin package that contains a number of test

steps, two DUT resources, four Instrument resources, and

two results listeners.

TestPlanExecution\BuildTestPlan.Api

\BuildTestPlan.Api.csproj

Shows how to build, save and execute a test plan using TAP

API.

TestPlanExecution\RunTestPlan.Api

\RunTestPlan.Api.csproj

Shows how to load and run a test plan using TAP API.

See the following sections:

• Build and View the Example Plugin Project (page 10) provides a quick overview on how to

build this plugin and see its contents.

• Contents of the Plugin Development Project (page 12) shows the many resources and test

steps in this project. Follow the process in the previous section to build and view the

project.

• Create a Project with the TAP Plugin Template (page 13) describes how to get started on

your own project.

Build and View the Example Plugin Project

The Example Plugin project creates a basic plugin package that contains a test step, a DUT resource,

and an instrument resource. Follow these steps to build the project and view the results in TAP:

1. In Visual Studio:

a. Open ExamplePlugin.csproj. In the Solution Explorer, notice the three .cs files. These will

create an instrument resource, a DUT resource, and a test step:

b. Select Debug > Start Debugging to build the solution and open it in TAP.

2. In TAP:

a. Click the + icon, and add the Measure Peak Amplitude step:

b. View the step settings. Notice the step requires a Generator instrument:

c. In the Resource bar, click DUTs Add New, then click the + button. Add the Low Pass Filter

DUT, then close the window:

d. In the Bench Settings window, notice the Filter DUT lets users specify an ID and a

Comment:

e. In Bench Settings, click the Instruments tab, then click the + button. Add the Generator

instrument, then close the window:

f. Notice the Generator instrument allows users to specify the Visa Address.

3. Close TAP. Do not save any files.

4. If you want to remove the examples, go to your TAP folder and delete:

• Example.Tap.Plugins.ExamplePlugin.dll

• Example.Tap.Plugins.ExamplePlugin.pdb

Do not delete files or folder included with TAP. See Appendix A: Files Included in the TAP

Installation (page 52) for a list.

Contents of the Plugin Development Project

Follow the same process to build and view the Plugin Development project (TAP_PATH\SDK

Examples\PluginDevelopment\PluginDevelopment.csproj), which contains many examples.

• Several test step categories that contain a number of steps:

• Two DUT resources:

• Four Instrument resources:

• Two Results Listeners:

To remove these examples, go to your TAP folder and delete

Example.Tap.Plugins.PluginDevelopment.dll and Example.Tap.Plugins.PluginDevelopment.pdb.

Create a Project with the TAP Plugin Template

The TAP Plugin template is a Visual Studio extension is installed with the SDK. To start a new

Visual studio project that will contain TAP plugin classes:

1. Select File > New > Project. Expand the Visual C# templates and select the Keysight TAP

Plugin template when you create your project

This template:

• Copies the project’s output to the location specified by the TAP_PATH environment

variable. This enables TAP to load the plugins during development. Note that TAP_PATH is

typically either:

C:\Program Files\Keysight\TAP (64-bit systems)

C:\Program Files (x86)\Keysight\TAP (32-bit systems)

• Includes a “packaging” step (see Plugin Packaging and Versioning on page 41 for details).

• Includes a TestStep class (Step.cs). If you are not creating a test step, delete this class.

2. To create a particular plugin or add another plugin to your project:

a. Select Project > Add New Item.

b. Enter Tap in the search field.

c. Select the appropriate plugin.

Plugin Development Basics

TAP Plugin Object Hierarchy

At the very root of the TAP plugin class hierarchy is the ValidatingObject class. Items in blue are

TAP base classes. The green rectangles are user extensions to those base classes.

Extending ValidatingObject are the ComponentSettings, Resource, and TestStep base classes. All

of these are abstract classes and hence cannot be instantiated. They are used as base classes for

other classes.

• ComponentSettings is used for storing settings throughout the system. It contains

properties to be set or lists of instruments, DUTs, and so on. ComponentSettings are

visible via the various Settings menus in the GUIs.

Resource is a base class for the DUT, Instrument and ResultListener classes. Resources are

often specified in the settings for a test step. Referencing a ‘Resource’ from a test step will

cause it to automatically be opened upon test plan execution.

TestStep does not inherit from Resource but inherits directly from ValidatingObject.

ComponentSettings Resource TestStep

Instrument

ScpiInstrument Custom

Dut Custom

Instrument

CustomScpi

Instrument

Custom

Database

Custom

TestSteptep DutSetting

s Custom

Settings

ValidatingObject

ResultListener DUT

Best Practices for Plugin Development

The following recommendations will help you get your project off to a good start and help ensure a

smooth development process.

• You can develop one or many plugins in one Visual Studio project. The organization is up

to the developer. Keysight recommends the following:

Encapsulate your logic. Keeping all instrument logic inside the instrument class makes

it possible to swap out instruments and not change TestSteps. For example, a TestStep

plugin knows to call MeasureVoltage, and the instrument plugin knows how to get that

measurement from its specific instrument.

You can put Instruments, DUTs, and TestSteps all in separate packages and create a

“plug-and-play” type of interaction for test developers. For example, you can create

test steps that make a measurement and plot a result. If done properly, the steps don’t

necessarily care which instrument gets the data or what type of device is being tested.

• The namespace of a plugin should follow the folder structure. For example, a folder

structure of MyCompany\Tap\Plugins\Category should use a namespace

MyCompany.Tap.Plugins.Category.

• Don’t introduce general settings unless absolutely necessary. Instead try to move general

settings to test steps (such as a parent step holding settings for a group of child steps) or

to DUT or Instrument settings.

For DUTs, Instruments, and Result Listeners, set the ShortName attribute, which is the

name that appears in the Resource Bar.

Use the Display attribute (with a minimum of name and description) on properties and

classes. This ensures good naming and tooltips in the GUI.

Use Rules for input validation to ensure valid data.

Developing Using Interfaces

Developers may choose to NOT extend the TAP base classes, but instead to implement the

required interfaces. You can inherit from as many interfaces as you want, but from only one class.

The TestStepInterface.cs file provides an example of implementing ITestStep.

Developers implementing IResource (required by virtually all other TAP plugin classes), must

ensure that the implementation of

IResource.Open sets IsConnected to true, and

IResource.Close sets IsConnected to false.

Test Step Plugin Development

A test step plugin is developed by extending the TestStep base class. As you develop test steps,

Keysight recommends the following:

• Isolate each test step into a single .cs file, with a name similar to the display name. This

makes the code easy to find, and focused on a single topic.

• Use Testplan.Sleep() for sleep statements. This makes it possible for the user to abort the

test plan during the sleep state.

Default Implementation

The default implementation of a TestStep (as generated when using the Visual Studio Item

Template for TestSteps) includes:

A region for Settings, which are configurable inputs displayed in Step Settings panel.

While initially empty, most test steps require the user to specify settings, which are very

likely referenced in the Run method.

A PrePlanRun method that may be overridden. The PrePlanRun method:

Is called after any required resources have been opened and prior to any calls to

TestStep.Run. (It is NOT called immediately before the test step runs.)

Should perform setup that is required for each test plan run, such as configuring

resources that are needed for test plan execution.

PrePlanRun methods are called sequentially in a flattened, top-to-bottom order of the

steps placement in the test plan.

• A Run method that must be included. In the absence of any flow control statements, the

required Run method is called in the order of the placement of test steps in the test plan.

The Run method:

Implements the primary functionality of the test step.

Typically leverages the test step's settings.

Often includes logic to control the DUTs and instruments, determine verdicts, publish

results, log messages, etc. Separate sections deal with many of these topics.

• A PostPlanRun method that may be overridden. PostPlanRun methods are called

sequentially in a flattened, bottom-to-top order of their placement in the test plan (the

reverse of the PrePlanRun order.) The PostPlanRun method:

Is used for “one-time” cleanup or shutdown.

Is called after the test plan has competed execution, and prior to any calls to close the

resources used in the test. (It is NOT called immediately after the test step runs).

Is always called if PrePlanRun for the test step was called (also in case of errors/abort).

This method can be used to clean up PrePlanRun actions.

The following code shows the template for a test step:

{ [Display("MyTestStep1", Group: "MyPlugin2", Description: "Insert a description here")] public class MyTestStep1 : TestStep { #region Settings // ToDo: Add property here for each parameter the end user is able to change #endregion public MyTestStep1() { // ToDo: Set default values for properties / settings. } public override void PrePlanRun() { base.PrePlanRun(); // ToDo: Optionally add any setup code this step must run before testplan start. } public override void Run() { // ToDo: Add test case code here. RunChildSteps(); //If step has child steps. UpgradeVerdict(Verdict.Pass); } public override void PostPlanRun() { // ToDo: Optionally add any cleanup code this step needs to run after the // entire testplan has finished. base.PostPlanRun(); } } }

To allow user configuration of a test step, developers must add appropriate properties to the

plugin code. These properties will very likely be visible and editable in the TAP GUI. Properties

typically include instrument and DUT references, instrument and DUT settings, timing and limit

information, etc. Defining these properties is a major part of plugin development.

The SDK provides many examples of test step development in the TAP_PATH\SDK

Examples\PluginDevelopment\TestSteps folder.

TestStep Hierarchy

Each TestStep object contains a list of TestSteps called Child Steps. A hierarchy of test steps can

be built with an endless number of levels. The child steps:

• Are run in sequence by calling the RunChildSteps method from the Parent step’s Run

method.

• Can also be run individually using the RunChildStep method to get even more fine-grained

control. This is an advanced topic. Running child steps should only be done when the

control flow is inside the parent step’s Run method.

Defining Relationships

Parent/Child relationships are defined by attributes associated with the Test Step class definition:

• From a Parent perspective, the AllowAnyChild and AllowChildrenOfType attributes define

the parent/child relationship.

• From a Child perspective, the AllowAsChildIn attribute defines the parent/child

relationship.

If multiple attributes apply (such as AllowChildrenOf Type and AllowAsChildIn), then both must

evaluate to true for the GUI to allow a child to be inserted under a parent.

It is common practice for a child to use properties from a parent. For example, a child might need

to reference a DUT or instrument defined in a parent. The GetParent method allows a child to

search for a parent of a particular type, and retrieve a setting. For example:

public override void PrePlanRun() { base.PrePlanRun(); // Find parents of a certain type, and get a resource reference from them. // Resources include things like Instruments and DUTs. _parentsDut = GetParent<ParentWithResources>().SomeDut; _parentsInstrument = GetParent<ParentWithResources>().SomeInstrument;

It is valuable to use interfaces instead of types in the AllowChildrenOfType and AllowAsChildIn

attributes. This more general approach allows any test step child that implements the appropriate

interface. For example:

[Display( Groups: new[] { "Examples", "Feature Library", "ParentChild" }, Name: "Child Only Parents With InterfaceB", Description: "Only allowed in parents with interface B")] // This will only allow children that implement this interface. [AllowAsChildIn(typeof(IInterfaceB))] public class ChildOnlyParentsWithInterfaceB : TestStep

It is possible to programmatically assign children in the Parents constructor, as shown below:

public ExampleParentTestStep() { Name = “Parent Step”; ChildTestSteps.Add(new ExampleChildTestStep { Name = “Child Step 1”}); ChildTestSteps.Add(new ExampleChildTestStep { Name = “Child Step 2” }); }

For examples of parent/child implementations, see:

TAP_PATH\SDK Examples\PluginDevelopment\TestSteps\ParentChild

Verdict

TAP allows steps to be structured in parent/child hierarchy. The Tap.Verdict enumeration defines a

“verdict” indicating the state and progress of each test step and test plan run. The following table

shows the available values for Tap.Verdict, in increasing order of severity.

Verdict Severity (lowest to highest)

Description

NotSet No verdict was set (the initial value)

Pass Step or plan passed

Inconclusive More information is needed to make a verdict or the results were close to the limits

Fail Results fail the limits

Aborted Test plan is aborted by the user

Error An error occurred; this could be instrument, DUT, software errors, etc.

Each TestStep has its own verdict property. The verdict can be set using the UpgradeVerdict

function as shown below (from SetVerdicts.cs):

[Display("Set Verdict", Groups: new[] { "Examples", "Plugin Development", "Step Execution" }, Description: "Shows how verdict of step is set using UpgradeVerdict")] public class SetVerdict : TestStep { #region Settings public Verdict MyVerdict { get; set; } public double LowerLimit { get; set; } public double UpperLimit { get; set; } #endregion public SetVerdict() { MyVerdict = Verdict.NotSet; LowerLimit = 0; UpperLimit = 5; } public override void Run() { UpgradeVerdict(MyVerdict); var result = 2.5; if (result > LowerLimit && result < UpperLimit) { UpgradeVerdict(Verdict.Pass); } else UpgradeVerdict(Verdict.Fail); } }

When appropriate, TestSteps should change their verdict (often from NotSet to one of the other

values) during the run of the test step.

The most severe step verdict becomes the TestPlan verdict. For example, if five test steps pass,

two are inconclusive and one fails, the test plan verdict is Fail.

Log Messages

Log messages provide useful insight to the process of writing and debugging the test step code

(as well as other plugin code). The TestStep base class has a predefined Log source, called Log.

Log messages are displayed in the TAP GUI Log panel and saved in the log file.

When creating log messages, Keysight recommends the following:

Ensure that your logged messages are using the correct log levels. Make use of debug

level for less relevant messages.

Ensure that time-consuming operations write a descriptive message to the log that

includes duration (to ensure that the operation will be clearly visible in the Timing

Analyzer.)

Note: Logs are NOT typically used for RESULTS, which are covered in a different section.

Four levels of log messages – Error, Warning, Information, and Debug – allow the messages to be

grouped in order of importance and usefulness. Log messages are shown in the GUI and CLI and

are stored in the sessions log file, named SessionLogs\SessionLog [DateTime].txt. (Debug

messages are enabled by verbose command line argument.

By default, log messages for each:

Run are stored in TAP_PATH\Results (configurable in the Results settings).

Session are stored in TAP_PATH\SessionLogs (configurable in the Engine settings).

Log messages for each run are also available to ResultListener plugins as the second parameter

on the ResultListeners’s OnTestPlanRunCompleted method, as shown here:

void OnTestPlanRunCompleted(TestPlanRun planRun, System.IO.Stream logStream);

Note: Users can create their own logs by creating an instance of TraceSource as shown in the

code below. The name used to create the source is shown in the log:

Log.Debug(“Info from Run”); private Keysight.Tap.TraceSource MyLog = Keysight.Tap.Log.CreateSource("MyLog"); MyLog.Info("Info from Run");

Timestamps and Timing Analysis

The log file contains a timestamp for all entries. This time reflects the time at which the logging

method was called. Additionally, it is possible to log time spans/durations of specific actions, such

as the time it takes to measure, set up, or send a group of commands.

To log duration, overloads of the Debug(), Info(), Warning() and Error() methods are provided.

These accept a TimeSpan or a Stopwatch instance as the first parameter, as shown in the following

code:

// The Log can accept a Stopwatch Object to be used for timing analysis Stopwatch sw1 = Stopwatch.StartNew(); TestPlan.Sleep(100); Log.Info(sw1, "Info from Run"); Stopwatch sw2 = Stopwatch.StartNew(); TestPlan.Sleep(200); Log.Error(sw2, "Error from step");

This will result in a log message containing the event text and a time duration tag enclosed in

square brackets.

12:27:32.883 MyLog Info from Run [ 100 ms ] 12:27:33.083 MyLog Error from step [ 201 ms ]

The time duration tags make it possible to do more advanced post timing analysis. The Timing

Analyzer tool visualizes the timing of all log messages with time stamps.

Exception Logging

Errors that occur during test step execution prevent the step from finishing. If an exception is

thrown in a test step run method, it will abort the execution of the test step. The exception will be

caught by the TestPlan and it will move on to executing the next step.

If a test step fails, the TestStep can abort the entire TestPlan by throwing a TestPlan.AbortException,

as shown here:

throw new TestPlan.AbortException("I forced abort", true);

A message is written to the log when a step throws an exception. The log message contains

information on source file and line number if debugging symbols (*.pdb files) are available and

Settings > Show Source Code Links is enabled.

TraceBar

The TraceBar is a utility class used to display log results and verdicts in the Log panel. If an upper

and lower limit is available, the TraceBar visually displays the one-dimensional high-low limit sets

in a log-friendly graphic:

Additionally, it handles the verdict of the results. If all the limits passed, the

TraceBar.CombinedVerdict is Pass; otherwise it is Fail. If the result passed to TraceBar is NaN, the

verdict will upgrade to Inconclusive. For an example see the code sample in LogMessages.cs file.

Validation

Developers customize validation by adding one or more Rules to the constructor of their object. A

rule has three parameters:

• A delegate to a function that contains the validation logic (may be an anonymous function

or a lambda expression)

• The message shown to the user when validation fails

• The list of properties to which this rule applies

See an example of the use of validation in RuleValidation.cs, as shown below:

[Display("RuleValidation Example", Groups: new[] { "Examples", "Feature Library", " Commonly Used" }, Description: "An example of how Validation works.")] // Also works for instruments, result listeners, DUTs...., since they all extend // ValidatingObject public class RuleValidation : TestStep { #region Settings [Display("Should Be True Property", Description: "Value should be true to pass validation.")] public bool ShouldBeTrueProp { get; set; } public int MyInt1 { get; set; } public int MyInt2 { get; set; } #endregion public RuleValidation() { // Validation occurs during the constructor. // When using the GUI, validation will occur upon editing. When using the engine // without the GUI, validation occurs upon loading the test plan. // Calls a function that returns a boolean Rules.Add(CheckShouldBeTrueFunc, "Must be true to run", "ShouldBeTrueProp"); // Calls an anonymous function that returns a boolean Rules.Add(() => MyInt1 + MyInt2 == 6, "MyInt1 + MyInt2 must == 6", "MyInt1", "MyInt2"); //Ensure all rules fail. ShouldBeTrueProp = false; MyInt1 = 2; MyInt2 = 2; } private bool CheckShouldBeTrueFunc() { return ShouldBeTrueProp; }

The setting as displayed in the GUI looks like this:

Publishing Results

Publishing results from a test step is a fundamental part of test step execution. The following

section discusses publishing results in detail. At a high level, publishing results usually involves a

single call, as shown in the following code snip from TAP_PATH\SDK Examples\ExamplePlugin

\MeasurePeakAmplituteTestStep.cs.

InputData = new double[] {0, 0, 5, 5, 5, 50}; ReadOnlyOutputData = new double[]{10, 10, 15, 15, 15, 150}; Results.PublishTable("Inputs vs. Moving Average", new List<string>() {"Input Values", "Output Values"}, InputData, ReadOnlyOutputData);

Basic Theory

Test step results are represented in a ResultTable object. A ResultTable consists of a name, one to

N columns, and one to M rows. Each test step typically publishes one uniquely named table. Less

frequently (but possible), a test step will publish N tables, with different names, row/column

definitions and values. The ResultTable is passed to each of the configured ResultsListeners for

individual handling.

ResultTable Details

This graphic shows the ResultTable definition.

ResultSource Object

A ResultSource object (named Results in the test step base class) and its Publish methods push

result tables to the configured ResultListeners.

There are three major considerations for publishing results:

What is the “shape” of your results? Is it a single name/value pair, a single “row” (or a set

of name/value pairs) of data, or N rows?

How fast do you want to store the results?

What table and column names do you wish to use?

The following ResultSource.Publish methods are available.

Method Name General Use Scope

Publish<T>(T result) For a type T, publishes all the public scalar properties as a

single row with N columns. The names of the properties

become the column names. The values become the row

values.

The table name will be the name of the type T, unless

overridden by the Display attribute.

Single Row

Publish<T>(string name, T

result)

Similar to the above, but assigns a unique name to the table

name.

Single Row

Publish(string name, List<string>

columnNames, params

IConvertible[] results)

Publishes a row of data with N column names, and N values.

The number of columnNames must match the size of the

Results array.

Single Row

PublishTable(string name,

List<string> columnNames,

params Array[] results)

Publishes N columns of data, each with M rows. The

columnNames parameter defines the

ResultTable.ColumnNames property. The results parameter

(an array), with N columns, and M rows, is used to populate

the N ResultColumn objects, each with an array of data.

The size of the columnNames property must match the

results array column count. PublishTable:

Can be called repeatedly to fill up a table

Is the fastest way to store data and should be used when

results are large

N Rows

For different approaches to publishing results, see the examples in:

• TAP_PATH\SDK\Examples\PluginDevelopment\TestSteps\PublishResults

Serialization

Default values of properties should be defined in the constructor. Upon saving a test plan, the test

plan’s Tap.Serializer adds each step’s public property to the test plan XML file. Upon loading a test

plan from a file, the Tap.Serializer first instantiates the class with values from the constructor and

then fills the property values from the values found in the test plan file.

Because the resource references are declared as properties:

Their value can be saved and loaded from XML files

The GUI will support setting the references in a user-friendly way

This convention applies for many different types.

Inputs and Outputs

Inputs and outputs are test step settings that transfer data between test steps during a test plan

run. This is useful in situations where one step depends on a result from another step, but could

also be used for flow control.

For examples, see:

• TAP_PATH\SDK Examples\PluginDevelopment\TestSteps\InputOutput

The generic Input class takes one type argument. The Input property references an Output of a

different step. If no Output is assigned to the Input, the value of the Input is null, and will result in

an error.

The Output attribute indicates a property is an output variable. Outputs can be connected to

Inputs. Every step has a Verdict property, which is automatically an output property. The Verdict

output can be connected to the If step, which has an Input<Verdict> property.

The following code (from GenerateOutput.cs) shows how to generate Output properties.

[Output] [Display("Output Value")] public double OutputValue { get; private set; }

The following code (from HandleInput.cs) shows how to use Input properties. You should use the

InputValue.Value to access the value contained in the input variable. See the red box below:

[Display("Handle Input", Groups: new[] { "Examples", "Feature Library", "InputOutput" }, Description: "Handles a double input value.")] public class HandleInput : TestStep { #region Settings [Display("Input Value")] // Properties defined using the Input generic class will accept values // from other (typically prior) test steps with properties that have been // marked with the Output attribute. public Input<double> InputValue { get; set; } #endregion public HandleInput() { InputValue = new Input<double>(); } public override void Run() { if (InputValue == null) throw new ArgumentException(); Log.Info("Input Value: " + InputValue.Value); UpgradeVerdict(Verdict.Pass); } }

Result Listener Plugin Development

TAP comes with a number of predefined result listeners, summarized in the following table.

Group Name Description

Action Notifier Runs a program or plays a sound based on the verdict of a test plan start and

end.

Database PostgreSQL Stores results into a PostgreSQL database.

SqLite Stores results into a SqLite database.

Text CSV Stores results into a CSV file. Supported delimiters are semicolon (to avoid

conflicts with commas in VISA addresses), comma and tab are also supported

Log Stores log messages (NOT results) into a log file, one for each test plan run.

TAP also supports custom results listeners.

Custom Result Listeners

A custom result listener stores the data in a "custom" way. For example, if TAP is to be deployed in

a manufacturing shop that has a preexisting data storage system, you could create a custom

result listener to interface with that system.

To create a custom result listener, extend the Tap.ResultListener class to a new public class. The

ResultListener class has virtual methods that are called during test plan execution. Implement only

those that are needed for the specific ResultListener implementation. The SDK includes an

ExampleResultListener that places a summary of any ResultTables into the log.

For examples, see:

• TAP_PATH\SDK Examples\PluginDevelopment\ResultListeners

When extending the ResultListener class, the following methods can be overwritten:

public override void OnTestPlanRunStart(TestPlanRun planRun)

public override void OnTestStepRunStart(TestStepRun stepRun)

public override void OnResultPublished(Guid stepRun, ResultTable result)

public override void OnTestStepRunCompleted(TestStepRun stepRun)

public override void OnTestPlanRunCompleted(TestPlanRun planRun, Stream logStream)

These methods are called by the test plan execution engine and are guaranteed to be called in a

certain order.

When the test plan starts and after all resources has been opened (including Result

Listeners), OnTestPlanRunStart is called with the TestPlanRun argument, containing

parameters for the plan run.

When the test plan finishes, the OnTestPlanRunCompleted method is called. The

TestPlanRun object contains information regarding the run, including the duration and

final verdict. LogStream contains the log file produced by the plan run.

Between the plan start and complete methods, OnTestStepRunStart and

OnTestStepRunCompleted are called for each step in the test plan. Between the two test

step methods, calls for each result from the test step, OnResultPublished is called, as a

consequence of calls to Results.Publish calls.

Note: Calls to OnResultPublished are done on a separate thread, and are NOT

synchronous with the test plan execution.

After OnTestPlanRunCompleted has finished, Close is called on all the ResultListeners.

The Result Listeners run on a separate thread, and do NOT run synchronously with the test plan.

That’s important - the design of this class is to handle results, but NOT specifically to use the

above methods to control external operation. The non-synchronous behavior was designed to

allow fastest throughput.

The MacroPath system:

Can be used to expand metadata into a path. For example, the ESN number might be

inserted directly into a file name or path. The system is generic enough to be used for

other things. See the MacroPath.cs file.

Supports the use of Windows environment variables. For example, the user folder can be

accesses via %UserProfile%. Upon macro expansion, this will expand to

C:\Users\[UserProfile]\. Any environment variable can be used, but some will expand into

invalid paths.

TAP SQL Database

Interacting with the predefined databases or interfacing to an unsupported database requires DB

knowledge.

When the user runs a test plan for the first time, the entire test plan file (XML) is automatically

compressed and saved together with the test plan name in the Attachment database table in the

following figure. At the same time, the given test plan run is registered in the TestRun table, with

time of execution and a reference to the test plan saved in the Attachment table. The log file of the

test plan run is stored in the same table.

The TestRun table gets a new entry for each executed step in the test plan. The PlanRunID table

points at the TestRun entry of the test plan run. The ParentRunID points to the entry of the parent

step’s run (if there is any). Associated with each TestRun, you can store multiple results series in

the ResultSeries table that again points to the ResultType table, which contains additional

information about the result data such as titles for the data columns/axis (x,y,z). The TestRun table

points via the TestRun2Params to a range of different parameters in the Params table. These

parameters are the public properties of the test step when the result series were generated.

Lastly, ResultSeries is pointed to by the Results, so that each row in ResultSeries can have many

results belonging to it.

The TAP database schema consists of 11 tables, shown in the following diagram. You should

explore the schema of the sample databases before attempting to write a new result listener

targeting a new database.

DUT Plugin Development

To develop a device under test (DUT) plugin, extend (or inherit from) the DUT class, which itself

extends the Resource class. The Open and Close methods MUST be implemented:

The Open method is called before the test plan starts, and must execute successfully. The

Open method should include any code necessary to configure the DUT prior to testing. All

open methods on all classes that extend Resource are called in parallel, and prior to any

use of the DUT in a test step.

The Close method is called after the test plan is done. The Close method should include

any code necessary to configure the DUT to a safe condition after testing. The Close

method will also be called if testing is halted early. All close methods are called in parallel,

and after any use of the DUT in a test step.

The DUT template generated by the Visual Studio class wizard includes minimal implementations

of these calls.

Developers should add appropriate properties and methods to the plugin code to allow:

• Configuration of the DUT during setup. The Dut base class already has defined string

properties for ID and Comment.

• Control of the DUT during the execution of test steps.

For examples of DUT plugin development, see:

• TAP_PATH\SDK Examples\PluginDevelopment\InstrumentsAndDuts

Instrument Plugin Development

Developing an instrument plugin is done by extending either the:

Instrument class (which extends Resource), or

ScpiInstrument base class (which extends Instrument)

Keysight recommends using ScpiInstrument over the Instrument class when possible.

Instrument plugins must implement the Open and Close methods:

• The Open method is called before the test plan starts, and must execute successfully. The

Open method should include any code necessary to configure the DUT prior to testing. All

open methods on all classes that extend Resource are called in parallel, and prior to any

use of the DUT in a test step.

The Close method is called after the test plan is done. The Close method should include

any code necessary to configure the DUT to a safe condition after testing. The Close

method will also be called if testing is halted early. All close methods are called in parallel,

and after any use of the DUT in a test step.

Developers should add appropriate properties to the plugin code to allow:

• Configuration of the instrument during setup. The Instrument base class has no predefined

properties. The ScpiInstrument base class has a string property that represents the

VisaAddress (see SCPI Instruments below).

• Control of the instrument during the execution of test steps.

Similar to DUTs, instruments must be preconfigured via the Bench menu choice, and tests will use

the first instrument found that matches the type they need.

For instrument plugin development examples, see the files in:

• TAP_PATH\SDK Examples\PluginDevelopment\InstrumentsAndDuts

SCPI Instruments

TAP provides a number of utilities for using SCPI instruments and SCPI in general. The

ScpiInstrument base class:

• Has properties and methods useful for controlling SCPI based instruments

• Includes a predefined VisaAddress property

• Requires Open and Close logic

Important methods and properties here include:

ScpiCommand, which sends a command

ScpiQuery, which sends the query and returns the results

VisaAddress, which specifies the Visa address of the instrument

The SCPI attribute is used to identify a method or enumeration value that can be handled by the

SCPI class.

The SCPI class is a utility class that formats SCPI commands and parses the results from SCPI

queries.

For an example, see:

• TAP_PATH\SDK Examples\PluginDevelopment\TestSteps\Attributes\ScpiAttributeExample.cs

The example below shows how the VisaAddress property for a SCPI instrument is automatically

populated with values retrieved from VISA:

Using Attributes in Plugins

Attributes are a standard part of C# and are used extensively throughout .NET. They have

constructors (just like classes) with different signatures, each with required and optional

parameters. For more information on attributes, refer to the MSDN C# documentation.

For TAP, type information is not enough to fully describe what is needed from a property or class.

For this reason, attributes are a convenient way to specify additional information. The TAP Engine,

the GUI and CLI use reflection (which allows interrogation of attributes) extensively. Some

attributes have already been shown in code samples in this document.

Attributes Used by TAP

TAP uses the following attributes.

Attribute name Description

AllowAnyChild Used on step class to allow children of any type to be added.

AllowAsChildIn Used on step to allow step to be inserted into a specific step type.

AllowChildrenOfType Used on step to allow any children of a specific type to be added.

AvailableValues Used to create ComboBoxes with dynamic content.

BenchSettings Used to indicate a setting is specific to a particular bench. If not set, then the setting

is considered global, and not related to a particular bench.

ColumnDisplayName Indicates a property could be displayed as a column in the test plan grid.

DirectoryPath Indicates a string property is a folder path.

Display Expresses how a property is shown and sorted. Can also be used to group properties.

EnabledIf Disables some controls under certain conditions.

FilePath Indicates a string property is a file path.

Flags Indicates the values of an enumeration represents a bitmask.

HandlesType Indicates a IPropGridControlProvider can handle a certain type. Used by advanced

programmer who are modifying GUI internals.

HelpLink Defines the help link for a particular class or property.

IgnoreSerializer Used on classes to ignore serialization. Useful for cases where a plugin

implementation contains non-serializable members or types.

MacroPath Indicates a setting should use MacroPath values, such as <Name> and %Temp%.

MetaData A property is result metadata and will be provided to result listeners. If a resource is

used with this attribute (and show Meta dialog is enabled), a dialog prompts the user.

Output Indicates a test step property is an output variable.

ResultListenerIgnore Indicates a property that should not be published to ResultListeners.

Scpi Identifies a method or enumeration value that can be handled by the SCPI class.

SettingsGroup Indicates that component settings belong to a settings group.

ShortName A short readable name for a resource (such as EXT for an E6607).

Unit Indicates a unit displayed with the setting values. Multiple options exist.

VisaAddress Indicates a property represents a VISA address. The editor will be populated with

addresses from all available instruments.

XmlIgnore Indicates that a property should not be serialized.

For attribute usage examples, see the files in:

• TAP_PATH\SDK Examples\PluginDevelopment\TestSteps\Attributes

Some of the commonly used attributes are described in the following sections.

Attribute Details

Display

The Display attribute is the most commonly used TAP attribute. This attribute:

Can be applied to class names (impacting the appearance in dialogs, such as the Add New

Step dialog), or to properties (impacting appearance in the Step Settings Panel).

Has the following signature in its constructor:

(string Name, string Description = "", string Group = null, double Order = 0D, bool Collapsed = false, string[] Groups = null)

Requires the Name parameter. All the other parameters are optional.

Supports a Group or Groups parameter to enable you to organize the presentation of items

in the GUI.

The parameters are ordered with the most frequently used parameters first. The following

examples show example code and the resulting GUI appearance:

// Defining the name and description. [Display("MyName", "MyDescription")] public string NameAndDescription { get; set; }

See the examples in TAP_PATH\Examples\PluginDevelopment\TestSteps\Attributes for different

uses of the Display attribute.

Display has the following parameters:

Attribute Required? Description

Name Required The name displayed in the GUI. If the Display attribute is not used, the

property name is used in the GUI.

Description Optional Text displayed in tools tips, dialogs and editors in the GUI

Group

Groups

Optional Specifies an item’s group. Use:

Group if the item is in a one-level hierarchy.

Groups if the item is in a hierarchy with two or more levels. The hierarchy

is specified by the left-to-right order of the string array.

Use either Group or Groups; do not use both. Groups is preferred.

Groups are ordered according to the average order value of their child items.

For test steps, the top-level group is always ordered alphabetically.

Syntax:

Groups: new[] { "Group" , "Subgroup" }

Order Optional Specifies the display order for an item. Note that:

Order is supported for settings and properties, such as test step settings,

DUT settings, and instrument settings.

Order does not support types: test steps, DUTs, instruments. These items

are ordered alphabetically, with groups appearing before ungrouped

items.

Order is of type double, and can be negative. Order’s behavior matches the

Microsoft behavior of the Display.Order attribute.

If order is not specified, a default value of -10,000 is assumed.

Items (ungrouped or within a group) are ranked:

So that items with lower order values precede those with higher values

Alphabetically if order values are equal or not specified

To avoid confusion, we recommend that you set the order value for:

Ungrouped items to negative values so that they appear at the top

Grouped items to a small range of values to avoid conflicts with other

items (potentially specified in base classes)

For example, if ItemA has order = 100, and ItemB has order = 50, ItemB is

ranked first.

EnabledIf

The EnabledIf attribute enables settings based on other settings or conditions. For an example, see

TAP_PATH\Examples\PluginDevelopment\TestSteps\Attributes\ EnabledIfAttributeExample.cs.

In the following code, BandwidthOverride is enabled when Radio Standard = GSM.

public class EnabledIfExample : TestStep { #region Settings // Radio Standard to set DUT to transmit. [Display("Radio Standard", Group: "DUT Setup", Order: 1)] public RadioStandard Standard { get; set; } // This setting is only used when Standard == LTE || Standard == HCDHA. [Display("Measurement Bandwidth", Group: "DUT Setup", Order: 2.1)] [EnabledIf("Standard", RadioStandard.Lte, RadioStandard.Wcdma)] public double Bandwidth { get; set; } // Only enabled when the Standard is set to GSM. [Display("Override Bandwidth", Group: "Advanced DUT Setup", Order: 3.1)] [EnabledIf("Standard", RadioStandard.Gsm, HideIfDisabled - true)] public bool BandwidthOverride { get; set; } // Only enabled when both Standard = GSM, and BandwidthOverride property is enabled. [Display("Override Bandwidth", Group: "Advanced DUT Setup", Order: 3.1)] [EnabledIf("Standard", RadioStandard.Gsm, HideIfDisabled - true)] [EnabledIf("BandwidthOverride", true, HideIfDisabled - true)] public double ActualBandwidth { get; set; } #endregion Settings

When Radio Standard is set to GSM in the step settings, both Override Bandwidth options are

then displayed:

Flags Attribute

The Flags attribute is a C# attribute used with enumerations. This attribute indicates than an

enumeration can be treated as a bit field (meaning elements can be combined by bitwise OR

operation). The enumeration constants must be defined in powers of two (for example 1, 2, 4, …).

Using the Flags attribute results in a multiple select GUI, as shown below:

FilePath and DirectoryPath Attributes

The FilePath and DirectoryPath attributes can be used on a string-type property to indicate the

string is a file or folder system path. When this attribute is present, the TAP GUI displays a browse

button allowing the user to choose a file or folder. These attributes can be used as follows:

[FilePath] public string MyFilePath { get; set }

This results in the following user control in the GUI:

A similar behavior is associated with the DirectoryPath attribute, but in place of a file browse

dialog, a directory browse dialog is available when the browse (‘…’) button is clicked.

MetaData Attribute

Metadata is a set of data that describes and gives information about other data. The Metadata

attribute marks a property as metadata.

TAP has the ability to prompt the user for metadata. Two requirements must be met:

The MetaData attribute is used and the promptUser parameter is true

The Allow Metadata Dialog property in engine settings is set to true

If both requirements are met, a dialog will appear on each test plan run to ask the user for the

appropriate values. An example of where metadata might be useful is when testing many DUTs in

a row and the serial number must be typed into the GUI manually.

Values captured as metadata are provided to all the result listeners, and can be used in the macro

system. See SimpleDut.cs for an example of the use of the MetaData attribute.

Unit Attribute

The Unit attribute specifies the units for a setting. The GUI displays the units after the value (with a

space separator). Compound units (watt-hours) should be hyphenated. Compound unit

abbreviations (WH) should not be abbreviated.

Optionally, displayed units can insert engineering prefixes.

See the UnitArrributeDemo.cs file for an extensive example.

XmlIgnore Attribute

The XmlIgnore attribute indicates a setting should not be serialized. If XmlIgnore is set for a

property, the property will not show up in the GUI. If you want to NOT serialize the setting AND

show it in the GUI, then use the Browsable(true) attribute, as shown below:

// Editable property not serialized to XML [Browsable(true)] [XmlIgnore] public double NotSerializedVisible { get; set; }

Properties that represent instrument settings (like the one below) should not be serialized as they

will result in run-time errors:

[XmlIgnore] public double Power { set; { ScpiCommand(":SOURce:POWer:LEVel:IMMediate:AMPLitude {0}", value) } get; { return ScpiQuery<double>(":SOURce:POWer:LEVel:IMMediate:AMPLitude?"); } }

External Parameters

Editable TAP step settings can be marked as External. The value of settings marked as External

can be set through the TAP GUI, through an external program (such as TAP CLI), or with an

external file. This gives the user the ability to set key parameters at run time, and (potentially) from

outside the TAP GUI. You can also use the API to set external parameter values with your own

program.

Component Settings

Global and Bench Settings

In addition to the settings defined for test steps, DUTs, and instruments, there are several “built in”

settings collections. These can be divided into two families: Global and Bench.

Global settings are shown under the Settings menu. At a minimum, you will find settings

for the Engine, GUI, and Results.

Bench settings are applicable to a particular “bench” configuration and are shown under

the Settings > Bench menu. Refer to the TAP GUI help for more information on the content

of the resulting dialogs.

Creating a New Global/Bench Settings Dialog

TAP developers can create their own settings dialogs under the Settings or Settings > Bench

menus. By default, the dialog appears under the Settings menu. If the class is decorated with the

[SettingsGroup] attribute, the dialog will appear under the Bench menu. Examples of both are

shown in the following sections.

Single Instance of Multiple Different Settings

If you want to create a dialog consisting of multiple settings, you should inherit from the

ComponentSettings class. See the ExampleSettings.cs file. The result looks like this:

List of Similar Settings

Suppose you are trying to list several objects that are slightly different but share a common base

class. This is similar to what is used in the DUT or Instrument settings dialog. To do so, you should

inherit from the ComponentSettingsList class. See the CustomBenchSettings.cs file. The results

(with several instances created), are shown below:

Reading and Writing Component Settings

The SettingsRetrieval.cs file demonstrates different approaches for reading component settings,

as shown in the code below:

[Display("Settings Retrieval", Groups: new[] { "Examples", "Feature Library", "Step Execution" }, Order: 10000, Description: "Shows how to retrieve settings.")] public class SettingsRetrieval : TestStep { public override void Run() { // These settings always exist Log.Info("Component Settings directory={0}", ComponentSettings.SettingsDirectoryRoot); Log.Info("Session log Path={0}", EngineSettings.Current.SessionLogPath); Log.Info("Result Listener Count={0}", ResultSettings.Current.Count); if (DutSettings.Current.Count > 0) { string s = DutSettings.GetDefaultOf<Dut>().Name; Log.Info("The first dut found has a name of {0}", s); } if (InstrumentSettings.Current.Count > 0) { string s = InstrumentSettings.GetDefaultOf<Instrument>().Name; Log.Info("The first instrument found has a name of {0}", s); } // An example of user defined settings, which show up as individual tabs // Default values will be used, if none exist. Log.Info("DifferentSettings as string={0}", ExampleSettings.Current.ToString()); // An example of similar settings which show up in a grid. // This is similar to the DUT or Instrument editors. // Only use the values if something exists. if (SimilarSettingsBaseList.Current.Count > 0) { Log.Info("SimilarSettingsBaseList Count={0}", SimilarSettingsBaseList.Current.Count); Log.Info("First similar setting as string={0}", SimilarSettingsBaseList.GetDefaultOf<SimilarSettingsBase>()); foreach (var similarSetting in SimilarSettingsBaseList.Current) { Log.Info("Type={0} Time={1} MyProperty={2}", similarSetting.GetType(), similarSetting.MyTime, similarSetting.MyProperty); } } } }

Connection Management

A TAP connection represents a physical connection between Instrument and/or DUT ports. A

physical connection is modeled in software by creating a class that extends the

Tap.Engine.Connection abstract base class.

A port is the endpoint of a connection. Ports are often defined on Instruments or DUTs to

represent physical connectors. An instrument or DUT can have an arbitrary number of ports.

TAP comes with a number of predefined connections. These include:

RfConnection, which extends the Tap.Connection class, consists of two ports (inherited

from the Tap.Connection), and adds the concept of CableLoss, which is a list of LossPoints.

An RfConnection represents a physical RF cable with known loss characteristics by using

the list of loss points to represent the cable loss at different frequencies.

DirectionalRfConnection, which extends the RfConnection class to include direction.

For more information, see the Bench Settings – Connections topic in the TAP GUI help, or contact

support.

Plugin Packaging and Versioning

Packaging

A TAP Package is a file that typically contains plugin DLLs and supporting files. Packages are used

to distribute TAP plugins, while providing support for correct versioning and dependency checking.

This section deals with the construction and use of TAP packages. The different programs and

processes are described below.

The TAP install includes Keysight.Tap.Package.exe, a console program that creates

packages. Frequently the content of the package is defined in a user-created package.xml

file.

The default TAP plugin project (release builds only) includes a special AfterBuild task that

calls the Keysight.Tap.Package.exe executable and references the package.xml file. Based

on the declarations in the package file, Package.exe will perform various processing, and

ultimately generate a TAP package. The resulting TAP packages have the .TapPlugin suffix.

Files with this suffix are renamed zip files, and if renamed, can be examined by WinZip.

The TAP install includes the Keysight.Tap.PluginPackageManager.Gui.exe program. The

PluginPackageManager provides a GUI that permits package downloading, displays an

inventory of the packages, and ultimately installs package files found into the TAP install

directory.

When run from Visual Studio, most of the operation of the packaging system is automatic and

invisible to operation. However, the developer may wish to modify the content and/or properties of

the package by editing the package.xml file. The following package.xml is found in TAP_PATH\

SDK Examples\ExamplePlugin:

<?xml version="1.0" encoding="UTF-8"?>  <Package Name="BasicPluginExample" xmlns="http://keysight.com/schemas/TAP/Package" InfoLink="" Configuration="" Version=""> <Description>Plugins from the example.</Description> <Files> <File Path="Example.Tap.Plugins.ExamplePlugin.dll" Obfuscate="false" UseVersion="true"></File> <File Path="SomeSampleData.txt"></File> </Files> </Package>

Packaging Configuration File

When running Tap.Package, the configuration is typically specified using an xml file. Below is an

example of this file (typically called package.xml).

<?xml version="1.0" encoding="utf-8" ?> <Package Name="MyPlugin" xmlns="http://keysight.com/schemas/TAP/Package" InfoLink="http://myplugin.com" Version="$(GitVersion)" Configuration="$(GitBranch)"> <Description> <Status>Development</Status> <Organisation>My Organisation</Organisation> <Contacts>[email protected]</Contacts> <Prerequisites>None</Prerequisites> <Hardware>None</Hardware> <Links>http://MyLink.com</Links> </Description> <Files> <File Path="TapPlugin.MyPlugin.dll" Obfuscate="true" UseVersion="true"></File> <File Path="waveform1.wfm" Obfuscate="false" UseVersion="false"></File> <File Path="waveform2.wfm"></File> </File> </Package>

With the above configuration file, Tap.Package produces a TapPlugin file containing an obfuscated

version of TapPlugin.MyPlugin.dll and two waveform files. Neither waveform file is obfuscated

(when not specifying the Obfuscate attribute, the default is false). Tap.Package tags the TapPlugin

file name and metadata with the version number from TapPlugin.MyPlugin.dll (because of the

UseVersion attribute). The resulting filename uses the format MyPlugin.1.0.0.0.TapPlugin.

The configuration file supports three optional attributes:

Attribute Description

InfoLink Link used by the More Information link inside the Plugin Package Manager.

Version Displayed in the dropdown in the Plugin Package Manager. If under GIT, the $(GitVersion)

macro is expanded. If no version is provided, the default value of 1.0 is used.

Configuration Any kind of information. If under GIT, the $(GitBranch) macro expands to the branch name.

The xml configuration file is preserved inside the package as metadata. Tap.Package adds some

additional information to the package. The metadata file is shown below. Note the dependency

information: Tap.Package adds dependency and version information to allow

Tap.PackageManager to determine whether all prerequisites have been met before installing the

package on the client.

<?xml version="1.0" encoding="utf-8" ?> <Package Name="MyPlugin" xmlns="http://keysight.com/schemas/TAP/Package" InfoLink="http://myplugin.com" Version="1.0.0.9b084a8" Configuration="Release" Date="03/02/2017 20:08:26" > <Description> <Status>Development</Status> <Organisation>My Organisation</Organisation> <Contacts>[email protected]</Contacts> <Prerequisites>None</Prerequisites> <Hardware>None</Hardware> <Links>http://MyLink.com</Links> </Description> <Dependencies> <PackageDependency Package="XSeries" Version="7.3.2.ab29327" /> <PackageDependency Package="Tap" Version="7.3.1.e2fd7c2d" /> </Dependencies> <Files> <File Path="TapPlugin.MyPlugin.dll" Obfuscate="true" UseVersion="true"> <Plugins> <Plugin>TapPlugin.MyPlugin.Step</Plugin> <Plugin>TapPlugin.MyPlugin.MyDut</Plugin> </Pluginsile> </File> <File Path="waveform1.wfm" Obfuscate="false" UseVersion="false"></File> <File Path="waveform2.wfm"></File> </Files> </Packages>

Command Line Use

You can run Tap.Package from the command line or from MSBUILD (directly in Visual Studio). If

you create a TAP project in Visual Studio using the SDK, the resulting project will already be set

up to generate a TapPlugin using Tap.Package (only when building in "Release" configuration).

Tap.Package can be called from the command line, as shown in the following example.

Tap.Package.exe example.dll example.wfm

This call creates a TapPlugin package containing example.dll and example.wfm. If the dll is a .NET

assembly, Tap.Package will obfuscate it. The output of the above command will be a file named

example.TapPlugin (the name of the first input file).

The behavior of Tap.Package can be customized using these arguments:

Argument Description

--no-obfuscation No files are obfuscated (much faster, good for testing).

--include <filePath>, -i <filePath> File to include in the package; files are not obfuscated.

--verbose, -v Writes output from DotFuscator to std out.

--xml-package <xmlFilePath> Gets options from an XML configuration file.

--set-version <CSharpFilePath> Replaces the version number in AssemblyInfo.cs files

with the current version as described by GIT (when under

Git control). No package is created when using this

option.

--set-configuration <CSharpFilePath> Replaces the configuration in AssemblyInfo.cs with the

current version as described by GIT (when under Git

control). No package is created when using this option.

--project-directory <CSharpProjectPath> Used in collaboration with the Configuration/Version

macros to get the right values from GIT (only valid when

under Git control)

--out <fileName>, -o <fileName> Set the file path for the generated package e.g. "--out

C:\MyPlugin.TapPlugin".

--help, -h Prints command line help, then exits.

Versioning

The TAP executables and TAP plugins are versioned independently and should use sematic

versioning. TAP versions are of the form X.Y.Z.W, where:

X is the major version number, incremented upon changes that break backward compatibility.

Y is the minor version number incremented upon each release.

Z is the build number, automatically incremented upon every set of code changes.

W is the GIT short commit hash.

A plugin that references a TAP assembly version 7 is compatible with any TAP version 7.y, but not

compatible with an earlier TAP version 6 or a future TAP version 8. The PackageManager will

check for version compatibility before installing plugins.

When a plugin uses the SDK project template, Tap.Package is already set up to version the

*.Tap.Plugins package using the version number in the AssemblyInfo.cs file. The project template

automatically updates the version number. If using GIT, this update queries the underlying GIT

repository to get the version number and can work in two modes:

Auto incrementing without GIT tags: If the GIT repository does not have any tags, only the

AssemblyInfo version’s last number is incremented. You can then manually increment the

preceding numbers according to the semantic versioning scheme. For example, you could

start with a version number like this in AssemblyInfo.cs:

[assembly: AssemblyInformationalVersion("0.1.8")]

When built in release mode, this last number (the Patch version, 8 in this case) would be

automatically updated to the commit number acquired from the GIT repository.

Auto incrementing with GIT tags: It is also possible to use GIT tags to manage the

versioning. In this approach, GIT stores the complete version number, and the version in

the AssemblyInfo.cs file is completely overwritten.

For example, if last commit is marked with “v1.0” GIT tag, when built in release mode,

AssemblyInfo.cs will be updated to "1.0.0". On subsequent commits the patch version (the third

number) will automatically increment (1.0.1, 1.0.2, etc.).

Advanced Test Plan Development

Test Plan Control Flow

To use TAP to its full potential, developers must understand the control of a running test plan.

Several aspects of TAP can change or influence the control flow, and this to a large degree

determines what the test plan does.

Important aspects include:

Test plan hierarchy

TestStep.PrePlanRun, TestStep.Run, TestStep.PostPlanRun methods

Result Listeners

Instruments and DUTs (resources)

Test steps modifying control flow

The following test plan uses test steps, DUTs and instruments defined in the Demonstration plugin:

The test plan relies on the following resources being available and configured with appropriate

times:

If we ran this test plan, we would see the following results in the Timing Analyzer:

• Resources opened (and closed) in parallel:

• PrePlanRun of test steps run in order and run as soon as all resources required by that

step are open:

• The Run methods of test steps are typically executed in test plan order, as shown below:

• A more advanced test plan may incorporate flow control statements to change execution

flow. For example, adding a Parallel test step as a parent to child test steps will make the

test steps run in parallel, as shown below in the red box. Note that the Resource.Open and

TestStep.PrePlanRun remain unchanged:

The test plan below illustrates how child test steps are handled:

The methods in the test steps execute in the following order. Note that the PostPlanRun methods

execute in reverse order of placement in the test plan:

Order Step Method

1 Parent1 PrePlanRun()

2 Child1 PrePlanRun()

3 Child2 PrePlanRun()

4 Parent2 PrePlanRun()

5 Parent1 Run()

6 Child1 Run()

7 Child2 Run()

8 Parent2 Run()

9 Parent2 PostPlanRun()

10 Child2 PostPlanRun()

11 Child1 PostPlanRun()

12 Parent1 PostPlanRun()

An even more advanced test plan would involve test steps that explicitly call the Run methods of

their child steps. The TAP timing analyzer is a useful tool for examining test plan execution.

Exceptions

Exceptions from user code are caught by the TAP Engine. This could break the control flow, but all

resources will always be closed. If the exception gets caught before PostPlanRun, the steps that

had PrePlanRun called will also get PostPlanRun called. When a step fails (by setting the Verdict

to Fail or Abort) or throws an exception, execution can be configured to continue (ignoring the

error) or to abort execution. These settings are configured in Engine settings, by configuring the

“Abort Run If” property.

Manual Resource Connection

You may want to avoid the time required to open resources at each test plan start. To do so, you

may manually open the resources by using the Connection button:

Resources opened manually remain open between test plan runs. This eliminates the time

required to open and close them for each test plan run.

Note: You must ensure that the resources can be safely used in this manner. For example, if a

Dut.Open configures the DUT for testing, you may be required to take the default behavior of

opening the resource on every run.

Testing Multiple DUTs

Test step hierarchies can be built and attributes set to allow certain steps to be child steps of

other steps. This hierarchal approach and the possibility of communicating with one or multiple

DUTs from a single test step allow for a variety of test flows.

The following figure illustrates four different approaches where both sequential and parallel

execution is used. The upper part of the illustration is the flow; the lower part is the test plan

execution showing the corresponding TX and RX test steps.

Flow Option 1 is a simple sequential test plan execution where TX (transmit) and RX

(receive) test steps are repeated once for each DUT. In a production environment, this is a

simple way to reduce the test/calibration time, because it lets the operator switch in a DUT

while the other DUT is being tested.

Options 2 to 4 all consist of the same two test steps: a TX step that controls a single DUT

and an RX step that allow control of multiple DUTs simultaneously. Usually several DUTs

can listen to an instrument transmitting a signal at the same time, but calibration

instruments (which receive) usually can’t analyze more than one TX signal at a time.

In Option 2, the TX test step is a child step of the RX step (via the use of the

AllowChildrenOfType attribute). Being a child means the TX step will be executed as a

part of the RX step execution. The flow in Option 2 starts by executing the RX test step

that brings DUT2 into RX test operation. Then the TX child step runs, bringing DUT1 into

TX mode. When the TX child step has finished, the RX test step continues for DUT1.

Options 3 and 4 reuse the functionality, but form even more optimized test plan flows.

Note: All of the above is highly DUT and instrument dependent.

Command Line Interface

The CLI takes a number of command arguments. The most important one is the test plan file the

user wants to run.

When a test plan is running, log messages inform the user about the progress. Log messages can

be color-coded by using the -c or --color argument. Another useful command argument is -v

(verbose), which displays all possible log messages. The following table describes CLI options.

Command Description

--quiet, -q Only show error and warning messages.

--verbose, -v Also show verbose/debug level messages.

--color, -c Color messages according to their level.

--help, -h Show this help.

--settings Specify a bench settings profile from which to load settings. The parameter

given here should correspond to the name of a subdirectory of

./Settings/Bench. If not specified, the settings from TAP GUI are used.

--wait, -w Waits for key press after finished test plan run

--version Prints TAP version number and exits

--no-logo Don't print the greeting/logo message containing version.

--search Add a directory to search for plugin dlls.

--service, -s Runs TAP using a service to optimize startup time. The service will continue to

run in the background.

--stop-service Stops any TAP services running in the background.

--start-service Starts TAP as a service.

--port, -p Used with --start-service or -s to specify the port to use for the service.

--metadata Adds metadata to test plan using the variable=value syntax. Metadata can be

added multiple times.

Example: to add the serial number for your DUT, use --metadata dutid=5.

--non-interactive Never wait for user input.

--external, -e Sets an external test plan parameter. Can be used multiple times. Use the

syntax variable=value. e.g. "-e delay=1.0",

--list-external-parameters Lists the available external test plan parameters.

The example below shows how to run myTestPlan.TapPlan with the metadata named dutid set to a

value 123, and color enabled.

Tap.Cli.exe -–metadata dutid=123 --color myTestPlan.TapPlan

Session Logs and Results logs containing log messages are not influenced by the CLI arguments.

Recommendations:

• When running the CLI from a Windows prompt, ensure the screen/window buffer width is

a minimum of 140 characters. This avoids wrapping of the log messages.

• To ensure maximum performance when running the CLI:

Disable verbose log messages, or

Select a console emulator different from cmd.exe. Alternatively, the output can also be

piped to a file.

Range Notation

An array of numbers can be defined by one or more comma-separated range notations. This can

be useful when specifying step settings or manually creating an xml test plan file. It also is relevant

to test plan serialization/deserialization. A range notation with a default step size of 1 is defined as

follows (note the colon delimiter):

StartValue: MaxValue

A range notation with a defined step size is defined as follows:

StartValue : StepSize : MaxValue

The logic will fill an array with values, starting with StartValue and incremented by step size, while

the value is less than or equal to the MaxValue. The following table shows the results produced by

various entries.

Base Type Entry Note Results

Double [ ] 1.23456:5 Default step size of 1 1.23456, 2.23456, 3.23456, 4.23456

Double [ ] 1.3:.3:4.2 Step size of .3 1.3, 1.6, 1.9, 2.2, 2.5, 2.8, 3.1, 3.4, 3.7, 4

Int [ ] 1:5 Default step size of 1 1, 2, 3, 4, 5

Int [ ] 1:2:5 Step size of 2 1, 3, 5

Int [ ] 1:5,10:3:17 Using multiple ranges,

separated by commas

1, 2, 3, 4, 5, 10, 13, 16

Int [ ] 1:-1:-10 Using negative step size 1, 0, -1, -2, -3, -4, -5, -6, -7, -8, -9, -10

Int [ ] 1e1:1e1:1e2 Scientific notation 10, 20, 30, 40, 50, 60, 70, 80, 90, 100

Int [ ] 1,2,3:6,2:5:23,1,1,1, A mixture of comma separated

items, and range notations

1, 2, 3, 4, 5, 6, 2, 7, 12, 17, 22, 1, 1, 1

Int [ ] 0x80:0xF0 Hex notation, default step size

of 1

128, 129, ……. 240

Int [ ] 0x80:0x10:0xF0 Hex notation with step of 16 128, 144, 160, 176, 192, 208, 224, 240

Int [ ] 0b0101010:

0b0101110

Binary notation, default step

size of 1

42, 43, 44, 45, 46

Int [ ] 0b0101010:

0b0000010:

0b0101110

Binary notation, step of 2 42, 44, 46

Appendix A: Files Included in the TAP Installation

The TAP installation defines a TAP_PATH environment variable, which holds the root directory for

the TAP installation. The default values for TAP_PATH are:

• C:\Program Files\Keysight\TAP (64-bit systems)

• C:\Program Files (x86)\Keysight\TAP (32-bit systems)

The most important TAP files and directories are described below. As you develop for TAP, you

may want to delete your working files from the TAP_PATH folder. Do not delete any of the

following files.

Name Description

Keysight.Tap.Cli.exe Console program with parameters; called from other programs

to run TAP test plans

Keysight.Tap.Engine.dll Key DLL underlying TAP

Keysight.Tap.Gui.Controls.dll Key DLL underlying the GUI

Keysight.Tap.Gui.exe TAP GUI

Keysight.Tap.Package.exe Utility used during plugin development to create packages

Keysight.Tap.PluginPackageManager.exe Console program with parameters for managing Plugin Packages

Keysight.Tap.PluginPackageManager.Gui.exe GUI-based Plugin Package Manager

Keysight.Tap.Plugins.BasicSteps.dll DLL that contains the implementation of the Basic and Flow

Control test steps

Keysight.Tap.Plugins.BasicSteps.Gui.dll DLL used to support GUI operations for basic steps.

Keysight.Tap.Plugins.ResultListener.Csv.dll DLL containing a result listener emitting CSV files.

Keysight.Tap.Plugins.ResultListener.SQLite.dll DLL containing a result listener creating and updating SqLite

databases

Keysight.Tap.ResultsViewer.exe GUI program for viewing results

Keysight.Tap.RunExplorer.exe GUI program for exploring the artifacts from different test plan

runs

Keysight.Tap.Sdk.vsix Visual Studio extension that adds templates for TAP plugin

development

Keysight.Tap.TimingAnalyzer.exe GUI program for analyzing the timing of a test plan execution

\Package Definitions Directory that stores package definitions

\Results Directory that stores artifacts from each run of a test plan

\SDK Examples Directory (with children) that holds examples related to

developing with TAP

See View the Visual Studio Example Projects on page 9 for more

information.

\SessionLogs Directory that stores the log files from TAP sessions

\Settings Directory with children that stores TAP settings

keysight test automation platform (tap)€¦ · this document describes the programmatic interface...

Documents