© 2005 by ibm corporation; made available under the epl v1.0 | february 28 th 2005 adopting the...

© 2005 by IBM Corporation; made available under the EPL v1.0 | February 28th 2005

Adopting the Eclipse™ Test and Performance Tools Platform (TPTP) project

Roadmap to a successful adoption of TPTP

2Adopting the Eclipse TPTP Project | © 2005 by IBM Corporation; made available under the EPL v1.0

Introductions

Joe ToomeyIBM Rational

Eclipse TPTP Project Committer

[email protected]

Serge Lucio (absent)

IBM Rational

Eclipse TPTP Trace & Profiling Project Lead

[email protected]


Agenda

Eclipse TPTP Overview Eclipse TPTP Demo Introduction to TPTP Adoption Adopting TPTP from the ground-up Migrating existing products to Hyades


Eclipse TPTP Overview

Test and Performance Tools Platform (TPTP) project is an Eclipse top-level project.

TPTP project was formed in August 2004 by promoting and expanding Hyades project. The former Hyades project was reorganized into four new projects.

Hyades project was formed in December 2002 under Eclipse Tools project.

For more on the project visit http://www.eclipse.org/tptp

Mission is “to build a generic, extensible, standards-based tool platform upon which software developers can create specialized, differentiated, and interoperable offerings for world class test and performance tools.”

Being developed by three Eclipse Strategic Developer members and a total of eight Eclipse member organizations.

IDE

Eclipse TPTP

Application

I*E

Into Production

Into Development


TPTP Platform Project: The Foundation

1) Reference UIs and perspectives Basic metaphors for interacting with target systems and

resources - includes both remote and local systems.

UI Frameworks and common navigators, viewers, editors, and wizards. It provides many extension points used by TPTP tools and vendor tools.

2) Standard data models, and assets repository EMF-based information model implementations for test,

trace, log and statistical data.

Framework for running rule based queries against data model instances and some simple queries. This is the primary mechanism that ties UI with data models

3) Common data collection and execution framework Execution environment that supports deployment,

launch, and control of test cases and applications

Data collection and control frameworks and agents.

Communication service which is used by the distributed data collection and control frameworks.

Eclipse Reference UIs

Execution Framework

XMI Assets


TPTP Testing Tools Project

Provides testing tools framework by extending the TPTP platform.

Includes testing tools such as JUnit based component testing tool, Web application performance testing tool, and a manual testing tool.

Common nomenclature and metaphors facilitate integration of disparate test types and environments.

Deployment and execution of tests on remote and distributed systems.


TPTP Tracing and Profiling Tools Project

Provides frameworks for tracing and profiling tools by extending the TPTP platform.

Includes profiling tools for both single-system and distributed Java applications.

Provides correlation service A JVMPI monitoring agent that collects

trace and profile data.Collects and analyzes heap and stack information

Anticipate JVMTI-based monitoring agent.

Anticipate additional language and protocol support.

A generic tool kit for probe insertion - can instrument byte code of Java applications.


TPTP Monitoring Tools Project

Provides frameworks for building monitoring tools by extending the TPTP platform.

Includes tools for monitoring application servers (JBoss, Jonas, and Websphere) and system performance.

Collects, analyzes, aggregates, and visualizes data captured in the log and statistical models.

A typical example is collection of CPU or memory utilization and viewing, aggregation, and analysis of that data.

Supports Common Base Event (CBE), provides services for mapping of custom log formats to CBE, and regular expression based log filtering.

Correlates data across multiple instances of log and statistical models; also across instances of trace and test history models

Enables symptom and pattern analysis.


Agenda



Who is TPTP for?

Primary audience:ISVs building products or solutions in the test & performance domain

Examples:

IBM Rational Application DeveloperiCandle Pathwai Performance AnalyzerScapa StressTest Express

Secondary audience:Organizations building homegrown solutions to address test & performance needs of their development teams


Eclipse TPTP Adoption

Motivators Reuse TPTP Components

Accelerate time-to-market Reduce development & maintenance cost

Integrate with other TPTP-based tools

Leverage complementary tools Leverage the ecosystem of vendors/partners

Adopting TPTP is an incremental process The architecture is flexible enough to adjust to existing tools or TPTP-

based tools

Different levels of integration with TPTP yield various levels of inter-operability between tools


ExecutionEnvironment

Application

User’s Desktop

Test

Trace

EMF Data Models

Log

Statistical

Runtim

eM

onitor / log

Trace Analysis

And Profiling

Test Creation

and Execution

Artifact

Managem

ent

Eclipse TPTP GUI

Standard Widgets andCore Plug-ins

Reference PerspectivesAnd Workflow

Target System

Data C

ollection

Data C

ollectionInterface

InjectionCorrelation

Log Collection

Trace Collection

System PerformanceMonitor

JVMPI Monitor

Data Loader

DistributedData Collection

Framework

Agent

Control Interface

Agent

Control Interface

DistributedControl

Framework

Testability Interface

Test Engine

Plugging into the TPTP Architecture

Eclipse Platform


Testing Tools Integration (stage 1)

This stage is generally a required step for any existing tool migration

New tools should target stage 2 Target:

Existing tests are imported

Tests Suites and Test Cases are not created from Eclipse

Other test assets (datapools, behaviors, etc.) do not use the TPTP platform data models

Tests can be executed from the test perspective

Results are logged in a test execution history Benefits:

Common user experience metaphor with other TPTP test tools

Tests may be scheduled from other TPTP test tools

Tests execution can be combined with collection of trace, profiling or monitoring data



This stage is an intermediate level Benefits are incrementally gained by adopting/sharing more of the TPTP platform

data models Target:

Existing tests may be imported (in case of a tool migration)

Test Suites and Test Cases are created and defined from Eclipse

Test behaviors, datapools, test configurations, etc. may use the TPTP platform data models and their associated editors

Editors may be extended/replaced Tests can be executed locally or remotely

Results are logged in a test execution history Benefits:

Same as stage 1

A single user interface (Eclipse) is leveraged for all testing activities

Other Eclipse-based tools can be leveraged by your tool

Assets relying on the TPTP data models can leverage resources produced by other TPTP-based tools



This stage represents a complete adoption of the TPTP platform Target:

Legacy tests can be imported (in case of tool migration)

All of the tests assets are defined from Eclipse

Tests behaviors, datapools and test configurations use the TPTP platform data models and extend existing editors (if applicable)

Tests can be executed locally or distributed, mixed or not with other test types, with test logs including all the details about actions and verifications

Benefits: Same as stage 2

Users can freely switch between tools to optimize their testing activity, e.g. users with different skills may collaborate by using different tools exposing the same test through different user experiences


Trace & Profiling, Monitoring Integration (stage 1)

This stage can be useful to validate the use of existing TPTP views to show your profiling, tracing, or monitoring data

Target: Runtime data (trace, logs, counters, and logs) are not collected from

Eclipse

Runtime data is imported into the TPTP data model

The TPTP Trace, Profiling and Monitoring views are used to analyze the runtime data

Benefits: About 12 different views can be leveraged to analyze the data

Runtime data by your tool can be correlated with other runtime data collected by Eclipse TPTP tools



This stage is the most common integration level to be an Eclipse-based solution

Target: Data collectors and model loaders are used to collect runtime data

Runtime data is stored using all or part of the Eclipse TPTP data models

Trace, Profiling, or Monitoring views might be extended or replaced to display collected runtime data

Benefits: Same as stage 1

User experience is driven from Eclipse

Runtime data can be visualized as the application/system is running



This stage represents a complete adoption of the TPTP platform

Target: Data collectors are integrated with the launch configuration

Runtime data is stored using the Eclipse TPTP data models

Eclipse TPTP views are extended to display the runtime data Benefits:

Same as stage 2

Runtime data can be correlated with other TPTP-based data collection

Runtime data can be collected while debugging or testing the application


Agenda



Project Overview

Project Context New Product focusing on Performance Testing

Extends the Eclipse TPTP Testing Tool Project to provide high scalability performance testing for different protocols

Expands the HTTP Testing tool provided by Eclipse TPTP Adoption Level: Stage 3

All the test assets rely on Eclipse TPTP platform data models

Testing assets (test suites, test cases, etc.) Runtime data collected (execution histories, statistical data, etc.)

A number of components are reused

HTTP Recorder, datapool editor, execution history viewer are leveraged as-is

Test Editors are replaced to provided higher productivity Additional execution views are provided for better analysis


Agenda



Project Overview

Project Context Migration of an existing product (Rational PurifyPlus for Java) focusing on

runtime analysis (memory leak detection, performance analysis, etc.) of java applications

Extends the Eclipse TPTP Trace & Profiling Tool Project to provide better data collection (finer grain) and deeper analysis

Adoption Level: Stage 2-3 Data collectors are integrated with the Eclipse TPTP

Data collectors are integrated with the launch configuration

Most of the data is relying on the Eclipse TPTP data models Some of the finer grain data cannot be captured with the existing Eclipse

TPTP data models A number of components are reused

Launch configuration and data collection UI All the analysis views All the distributed data collection components


The end

© 2005 by ibm corporation; made available under the epl v1.0 | february 28 th 2005 adopting the...

Documents