Dolphin: Regression Test System for Latitude

Tao Jiang www.linkedin.com/in/taojiang2000/

Yongjin Yan

Rohitasva Rohitasva

Objectives

• Infrastructure as a Service

• Latitude Regression Test System

2

Latitude System

3

Challenges in Latitude System Testing

4

First Step

Standardize Test Station (Dolphinize)

• Create private network for each test station

• Extensively Use PG system board

• Eliminate moving parts to reduce error

• Set it, and forget it!

5

Dolphin Test Station Configuration

6

Infrastructure as a Service

Dolphin delivers Testing Infrastructure as a Service (IaaS)

• Virtualized Test Station (Hardware Virtualization)

• A Controller (virtual test station manager)

– Manage test farm machines

– Testing Service Provider

• Device configuration and control

• Regression Testing

• On demand testing hardware resource

7

Dolphin: System Overview

8

What Dolphin does?

Dolphin is a collection of:

• Cloud Infrastructure: Testing Infrastructure

– Test Station on Demand

– Regression Testing

• Cloud Applications: Software as a service over Internet

– Dolphin Web Application: Software on Demand

• Automated Device Setup & Test Control

– Integrate with Panda Web Application

• Automated Account Enrollment

• Dolphin API: Client interface to Dolphin Cloud Service

– Get Virtual Test Station Instance via API

– Test can be run on any machine not just on test station

9

Device Setup using Dolphin

10

Test Control for Exploratory Testing

11

Demo: System BAT

• Run Real Device

• End to End

• Acceptance Testing Effort

Latitude 6.0: ~ 1 week

Latitude 7.0: ~ 1 day

Dragon 1.0: ~ 1 hour on a Dolphinized test station

12

Dolphin: Regression Test System

• Run test on different test station without manual intervention.

• Auto configure hardware per test requirement.

• Auto configure and update supporting software packages.

• Work order scheduling.

• Test result reporting.

13

Work Order Example

workorder = {

"Submitter": "fortyTwo\/g041145",

"Test": {

"Arguments": ["-t", "Test1_Supported_PGs", "-c", "Case1_Supported_PGs", "-x", "JP_1.0_JAGJP", "-i", "W172-400-0", "-p", "pgsim=false,enroll=false,wac=10057,+auto"],

"Name": "Case1_Supported_PGs\/W172-400-0\/6443\/JP",

"Protocol": "SDVT_DeviceData_Supported_Devices",

"Stream": "DVT_System_Dragon",

"TestScript": "SDVT_DeviceData_Supported_Devices.py"

},

"Environment" : { # environment to run against

"Type" : "REG", # Clock, Biz , When type is Clock but different region, regression system will do a deployment.

"Region" : "JP“ # region field is a required.

},

"Hardware": { # Hardware information in order to run the test.

"Setup" : “true", # True if the test script want s regression system setup hardware for it. False, test script will setup hardware by itself.

"Communicator": {

"ModelNumber": "6443",

"SerialNumber": "5077766",

"ServerConnectionType": "ethernet"

},

"PulseGenerator": {

"ModelNumber": "W172-400-0",

"SerialNumber": "5077766",

"PGFamily": "Ingenio",

"Type": "PGBoard"

}

},

# URL for test result notification

"NotifyUrl" : "http://yourTestManagementSite.com/regression/results/"

}

14

Implementation Strategy

• Dolphin Controller and Agent Service • .Net application written in C#.

• Windows Communication Foundation (WCF): for building service-oriented applications.

• C# Selenium 2.0 WebDriver for driving Dragon App.

• Expose functionalities as a set of web services.

• Automated device setup and account enrollment

• Simplify test script development

15

Apache Server as a Cluster Service

• Cluster N + 1 application servers

– N farm machines and 1 controller

– Base URL: http://dolphin/dolphinws

– Forward command to target automatically

• Deliver static page content

16

Dolphin Web Service API

• Schedule regression test by submitting work order

• Test station rental service

• Configure and Control devices

17

Technology Highlights

• C# .NET 4.0

• WCF: Windows Communication Foundation

• Selenium 2.0 and WebDriver (C# version)

• jQuery and jQuery UI

• Apache HTTP Server

18

Results

• Standard hardware configuration

• On demand web application

• Integrated with Panda for automatic enrollment

• Centralized management reduces the maintenance cost

• Flexibility

– Hardware on demand

– Language-Neutral Solution to automated test development

– Platform-Neutral Solution to test execution

19

Conclusions

• Dolphin continues Panda's successful adventure of cloud

computing (Software as a Service)

• Dolphin brings hardware resource into cloud

(Infrastructure as a Service)

• Greatly simplifies Latitude system-level end-to-end

testing

• Reduces cost

20

21

Thank You