Download - Dolphin: Regression Test System for Latitude
Dolphin: Regression Test System for Latitude
Tao Jiang www.linkedin.com/in/taojiang2000/
Yongjin Yan
Rohitasva Rohitasva
Objectives
• Infrastructure as a Service
• Latitude Regression Test System
2
Latitude System
3
Challenges in Latitude System Testing
4
First Step
Standardize Test Station (Dolphinize)
• Create private network for each test station
• Extensively Use PG system board
• Eliminate moving parts to reduce error
• Set it, and forget it!
5
Dolphin Test Station Configuration
6
Infrastructure as a Service
Dolphin delivers Testing Infrastructure as a Service (IaaS)
• Virtualized Test Station (Hardware Virtualization)
• A Controller (virtual test station manager)
– Manage test farm machines
– Testing Service Provider
• Device configuration and control
• Regression Testing
• On demand testing hardware resource
7
Dolphin: System Overview
8
What Dolphin does?
Dolphin is a collection of:
• Cloud Infrastructure: Testing Infrastructure
– Test Station on Demand
– Regression Testing
• Cloud Applications: Software as a service over Internet
– Dolphin Web Application: Software on Demand
• Automated Device Setup & Test Control
– Integrate with Panda Web Application
• Automated Account Enrollment
• Dolphin API: Client interface to Dolphin Cloud Service
– Get Virtual Test Station Instance via API
– Test can be run on any machine not just on test station
9
Device Setup using Dolphin
10
Test Control for Exploratory Testing
11
Demo: System BAT
• Run Real Device
• End to End
• Acceptance Testing Effort
Latitude 6.0: ~ 1 week
Latitude 7.0: ~ 1 day
Dragon 1.0: ~ 1 hour on a Dolphinized test station
12
Dolphin: Regression Test System
• Run test on different test station without manual intervention.
• Auto configure hardware per test requirement.
• Auto configure and update supporting software packages.
• Work order scheduling.
• Test result reporting.
13
Work Order Example
workorder = {
"Submitter": "fortyTwo\/g041145",
"Test": {
"Arguments": ["-t", "Test1_Supported_PGs", "-c", "Case1_Supported_PGs", "-x", "JP_1.0_JAGJP", "-i", "W172-400-0", "-p", "pgsim=false,enroll=false,wac=10057,+auto"],
"Name": "Case1_Supported_PGs\/W172-400-0\/6443\/JP",
"Protocol": "SDVT_DeviceData_Supported_Devices",
"Stream": "DVT_System_Dragon",
"TestScript": "SDVT_DeviceData_Supported_Devices.py"
},
"Environment" : { # environment to run against
"Type" : "REG", # Clock, Biz , When type is Clock but different region, regression system will do a deployment.
"Region" : "JP“ # region field is a required.
},
"Hardware": { # Hardware information in order to run the test.
"Setup" : “true", # True if the test script want s regression system setup hardware for it. False, test script will setup hardware by itself.
"Communicator": {
"ModelNumber": "6443",
"SerialNumber": "5077766",
"ServerConnectionType": "ethernet"
},
"PulseGenerator": {
"ModelNumber": "W172-400-0",
"SerialNumber": "5077766",
"PGFamily": "Ingenio",
"Type": "PGBoard"
}
},
# URL for test result notification
"NotifyUrl" : "http://yourTestManagementSite.com/regression/results/"
}
14
Implementation Strategy
• Dolphin Controller and Agent Service • .Net application written in C#.
• Windows Communication Foundation (WCF): for building service-oriented applications.
• C# Selenium 2.0 WebDriver for driving Dragon App.
• Expose functionalities as a set of web services.
• Automated device setup and account enrollment
• Simplify test script development
15
Apache Server as a Cluster Service
• Cluster N + 1 application servers
– N farm machines and 1 controller
– Base URL: http://dolphin/dolphinws
– Forward command to target automatically
• Deliver static page content
16
Dolphin Web Service API
• Schedule regression test by submitting work order
• Test station rental service
• Configure and Control devices
17
Technology Highlights
• C# .NET 4.0
• WCF: Windows Communication Foundation
• Selenium 2.0 and WebDriver (C# version)
• jQuery and jQuery UI
• Apache HTTP Server
18
Results
• Standard hardware configuration
• On demand web application
• Integrated with Panda for automatic enrollment
• Centralized management reduces the maintenance cost
• Flexibility
– Hardware on demand
– Language-Neutral Solution to automated test development
– Platform-Neutral Solution to test execution
19
Conclusions
• Dolphin continues Panda's successful adventure of cloud
computing (Software as a Service)
• Dolphin brings hardware resource into cloud
(Infrastructure as a Service)
• Greatly simplifies Latitude system-level end-to-end
testing
• Reduces cost
20
21
Thank You