software engineering research paper presentation ali ahmad formal approaches to software testing...
TRANSCRIPT
Software EngineeringResearch paper presentation
Ali Ahmad
•Formal Approaches to Software Testing
•Hierarchal GUI Test Case Generation Using Automated Planning
Formal Approaches to Software Testing
P. DasiewiczIEEE Canadian Conference on Electrical and Computer Engineering2002
Introduction Software Testing: checking and validating the correctness
of software Time consuming Difficult Error prune
Formal methods Specifying and verifying software systems using mathematical and
logical approaches Target: proof of correctness
Some verification techniques Model Checking State space exploration
Introduction (cont’d)
Current methods are not feasible Too much manual effort Models that are very complex for analysis
It is needed to overcome the following The state explosion problem Cost of creating models
Applicability: Design level Less complexity May reduce maintenance costs
Introduction (cont’d)
UML Simple, general purpose, visual modeling Specify, visualize, construct and document UML-statechart basis for dynamic behavioral
description
Objectives: Overview model checking to the validation and test
case generation (using UML statecharts and interaction diagrams)
Applicability of formal methods to component integration and interaction testing
Integration/Component Level Testing
Offutt Changes in component states due to change
events UMLTEST, integrated with Rational Rose
Highly effective test cases can be generated for system level specification
Test prefix: ensure that the system is @ a certain pre state before performing the test
Issues:• Interaction is not considered• Some states may never been entered
Incorporated UML collaboration diagrams
Integration/Component Level Testing (cont’d)
Yoon Test case generation based on UML sequence and
collaboration diagrams A node that represents both integration target and
message flow Testing technique
• Extract sequence diagram• If concurrency, extract collaboration diagram• Divide into ASF (Atomic System Function)• Extract nodes and message flows• Obtain test cases by applying test criteria (e.g. all-edge)
Limitations• May require a large number of sequence diagrams• Test case coverage (only normal or abnormal flow)
Integration/Component Level Testing (cont’d)
Kim Divide statecharts into extended finite state
machines Control flows are represented as paths Coverage criteria
• Path coverage• State coverage• Transition coverage
Test cases • breadth or depth first searches• can be generated using the data flow on the UML statechart• Determine whether class implements correct control and data
flow
System/Integration Testing Hartmann
Testing the interfaces among several components UML statecharts Mealy Finite State Machines
with restricted point to point synchronous communication model
Incremental composition and reduction algorithm Test cases: category partition method Limitations
• Interaction are modeled as synchronous communications• Event exchanges contain no parameters• No support for nested machines
Source Code Based Testing Major problem: State space explosion
problem Reducing the state space entails
eliminating irrelevant code segments Hatcliff
Program slicing techniques Used by Bandera for translating Java code to Jimple, e.g. SPIN,
SMV Generates a finite state model for the reduced code
Java Path Finder Integrates model checking, program analysis and Testing Goal: Apply formal methods at source level Initially, check for safety properties like deadlocks (LockTree)
Translation of Statecharts Promela: input modeling language for
SPIN C-like, extended with non-deterministic, and
loop guarded constructs Latella presents a translation from UML
statecharts into Promela, where statecharts are first converted to hierarchal automata. A proof of correctness is also given
Component Interaction Testing Major issue: Are the components developed
separately work properly together Formal methods of component interaction
Define testing requirements Automatically generate the test cases using model checking
Focus of author’s research: Component interaction of software systems Detect subtle interaction errors without duplicating the work @
unit level Underlying model: Labeled transition system ObjectState: modeling language to show the feasibility of
creating formal models, it relies on hierarchal FSM.
Hierarchal GUI Test Case Generation Using Automated Planning
Memon A., Pollack M., Lou Soffa M.IEEE Transactions on Software Engineering 2000
Introduction GUI testing is difficult
The interaction space is enormous Determining the coverage of test cases Regression testing is a major challenge
Automation is necessary for generating GUI test cases
PATHS: Planning Assisted Tester for grapHical user interface Systems Input: Possible goals for GUI user Generates sequences of events to satisfy the user goals, these
become test cases
Introduction (cont’d)
PATHS performs automated analysis of the hierarchal structure of the GUI to create hierarchal operators that will be used during plan generation
Contribution to Research Make use of an AI technique (Automated Planning) Exploits GUI structural features Makes Regression testing easier The test suite is portable Allow the reuse of operator definitions that commonly
appear across GUIs
Overview GUI consists of components like labels, buttons,
menus, and pop-up lists Example: Microsoft WordPad GUI has 2 types of windows
GUI windows Object windows,
doesn’t contain any window components
Overview (cont’d)
PATHS- Plan Generation Inputs
• Initial state• Goal state• A set of operators applied to a set of objects
• Preconditions and effects Solution to a Planning Problem: sequence of instantiated
operators which result in the goal state when executed in the initial state
2 Phases• Setup
• Create a hierarchal model of the GUI• Plan Generation
• Specify scenarios (initial and goal states)• PATHS generate a test suit for the scenarios
Overview (cont’d)
Role of Test Designer and PATHS during test generation
GUI Events and Planning Operators
Overview (cont’d)
Operator Event Mapping
Example operator: Edit_Cut
Abstract: Invokes a window that monopolizes the user interaction
Overview (cont’d)
High level plan
Expanded plan
Plan Generation
GUI test case generation can be modeled by hierarchal plans that doesn’t require conflict resolution
Planning GUI Test Cases Developing a representation for the GUI and its operators
Operator Derivation Process• Traverse the GUI and press on its components, the label is read off the
component label GUI events
• Menu open (File, Edit)• Unrestricted focus (Basic shapes in PowerPoint)• Restricted focus (Edit-Preferences in PowerPoint)• System-interaction (Cut and paste)
Planning Operators• System-Interaction• Abstract
Modeling Initial and Goal States and Generating Test Cases using algorithm
Generating multiple plans in PATHS Creating multiple linearizations of the partial plans Repeating the planning process, thus generating a different test case
Planning GUI Test Cases (Cont’d)
Planning GUI Test Cases (Cont’d)
Planning GUI Test Cases (Cont’d)
Planning GUI Test Cases (Cont’d)
Some Experimental Results
Multiple Tasks
Hierarchal vs. Single Layer
Questions