![Page 1: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/1.jpg)
Model-based Testing
Introduction and Hands-on session
Stefan EkmanDublin, 2011-01-27
![Page 2: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/2.jpg)
Model Based TestingWhat is Model-based Testing?
![Page 3: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/3.jpg)
What is MBT?
Wikipedia:
“Model-based testing is software testing in which test
cases are derived in whole or in part from a model that
describes some (usually functional) aspects of the system
under test (SUT).”
![Page 4: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/4.jpg)
Advantages
• Model-based testing is easy to understand from both the business and developer communities
• Model-based testing separates (business-) logic from testing code
• Model-based testing increases the test coverage with the same effort as “Classic” test automation
• Model-based testing is the fastest way to get use of automated test
![Page 5: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/5.jpg)
Advantages
• Model-based testing enables us to switch testing tool if needed or support multiple platforms using the same model
• Model-based testing focuses on requirement coverage, not how many test cases you executed last week etc.
• Model-based testing proved to positively affect the maintenance problem, the nemesis of all test automation.
![Page 6: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/6.jpg)
The ModelModelling your tests
![Page 7: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/7.jpg)
Tools / Environment
.GraphML
Model with yEd
GraphWalker
MBT engine
Selenium
Java or SOAP
QTP
Via SOAP
Hudson
Build system
![Page 8: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/8.jpg)
![Page 9: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/9.jpg)
Code behind
public void e_Nav_GoogleImages()
{
log.info("Edge: e_Nav_GoogleImages");
selenium.click(“link=Images");
selenium.waitForPageToLoad(30000);
}
![Page 10: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/10.jpg)
Modelling
What should the model represent?
I suggest that you start with the model representing the end usage/user functionality of the system. That will usually resemble your traditional functional test case approach. There are several other approaches for you to explore later in you new MBT career…
![Page 11: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/11.jpg)
Modelling
To what level do we model?
![Page 12: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/12.jpg)
Modelling
Who creates the model?
• Start out yourself to begin with• Present the model – show it to SME’s and
developers for input• If the system is non-existing, you probably would
consider user stories, product backlog items, use-cases or the equivalent requirement's information.
![Page 13: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/13.jpg)
Modelling
During this process you will collect the first benefits of working with MBT!
![Page 14: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/14.jpg)
GraphWalker
GraphWalker is a tool for generating offline and online test sequences from Finite State Machines and Extended Finite State Machines.
http://www.graphwalker.org/
• Lightweight MBT – No UML• No exit/stop points• Online• Event-driven
GraphWalker is an Open Source MBT engine created by Kristian Karl and are stable and used in large IT testing environment.
![Page 15: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/15.jpg)
GraphWalker - Stop Criteria's
• REACHED_EDGE• REACHED_VERTEX• EDGE_COVERAGE• VERTEX_COVERAGE• TEST_LENGTH• TEST_DURATION• REACHED_REQUIREMENT• REQUIEMENT_COVERAGE• NEVER
![Page 16: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/16.jpg)
GraphWalker – XML configuration
<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE MBTINIT SYSTEM "mbt_setup.dtd" ><MBTINIT EXECUTOR="java:com.paddypower.mbt.BetPlacementRegression" EXTENDED="true"
GUI=“true"> <MODEL PATH="model/BetPlacementRegression.graphml"/> <CLASS PATH="Java\PPautomation\bin"/> <CLASS PATH=".\servlet-2.3.jar"/> <CLASS PATH="selenium\selenium-java-client-driver.jar"/> <CLASS PATH="selenium\selenium-server-2.0a6.jar"/><SCRIPT>
PortNumber = 5555; DBconnectionString =
"jdbc:informix-sqli://10.10.10.10/db:informixserver=testdb1;user=user;password=12345";MAX_SELECTIONS = 10;
</SCRIPT> <GENERATOR TYPE="RANDOM"> <CONDITION TYPE="REACHED_VERTEX" VALUE="v_Done"/> </GENERATOR></MBTINIT>
![Page 17: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/17.jpg)
GraphWalker - The Walks
• RANDOM• A_STAR
Will try to generate the shortest possible test
sequence through a model with given stop
condition. The algorithm only works well on
smaller models.
• SHORTEST_NON_OPTIMIZED
![Page 18: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/18.jpg)
GraphWalker - Edge
• An edge is a transition. Could be a click on a button, a timeout, a server API call.
• An edge can have a label, but it is not mandatory.
• The label of an edge will map against a method, function or sub routine during test execution.
• As a rule of thumb, the best practice is to start the name with 'e_'. The reason for this, is that it is easier to recognize the function in the test execution tool, as an edge. For example:e_StartApplicatione_EnterBasicView
• The same name of an edge, can be re-used in the model.
![Page 19: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/19.jpg)
GraphWalker - Vertex
• A vertex is some kind of state that is possible to verify using some kind of oracle.
• A vertex must always have a label.
• The label of a vertex is mapped against a method, function or sub routine during test execution.
• As a rule of thumb, the best practice is to start the name with 'v_'. The reason for this, is that it is easier to recognize the function in the test execution tool, as a vertex. For example:v_ApplicationStartedv_BasicView
• The same name of a vertex, can be re-used in the model.
![Page 20: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/20.jpg)
GraphWalker - Labels
Label [cond. expr.] / Statement1;Statement2;
White spaces not allowed.
Follow the naming convention for the implementing programming
language.
Example:
e_StartApplication
Example:
e_EnterBasicView
![Page 21: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/21.jpg)
GraphWalker - Guards
Label [cond.expr.] / Statement1;Statement2;
Conditional expression that returns a Boolean value.
The guard indicates if the edge is accessible in the current state of the
machine. The language is either Java or JavaScript.
Example:
e_Evaluate [x>0 && y<=10]
Example:
e_Checkout [shoppingBasket.contains("Cerials")]
![Page 22: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/22.jpg)
GraphWalker - Statements
Label [cond.expr.] / Statement1;Statement2;
Any applicable Java or JavaScript statements
Example:
e_Calculate / x=5;y++;
Example:
e_AddCerials / shoppingBasket.add("Cerials");
![Page 23: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/23.jpg)
GraphWalker – Keyword Start
Used by Vertices only.
Each graph must have a start vertex, and that vertex holds this
keyword. There is only one vertex in a graph, holding this keyword.
The Start vertex can only have one out-edge.
If the graph is the main-graph, that edge must have a
label.
If the graph is a sub-graph, that edge cannot have a label.
![Page 24: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/24.jpg)
GraphWalker - REQTAG
Used by Vertices
Used by GraphWalker for keeping track of which
requirements are fulfilled during testing.
In the example above, if v_BookInfomation is passed, it
would mean the requirement with tag UC01 2.2.3 has
passed.
v_BookInfomationREQTAG=UC01 2.2.3
![Page 25: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/25.jpg)
GraphWalker - Weight
Used by Edges only.
Used by MBT during random walks during test sequence generation. It holds a real value between 0 and 1, and represents the probability that a specific edge should be chosen. A value of 0.05, would mean a 5% chance of that edge to be selected during a run.
Note: It only works with the RANDOM generator
v_BookInfomationREQTAG=UC01 2.2.3
e_NavAweight=0.25
e_NavB
![Page 26: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/26.jpg)
GraphWalker – Keyword Blocked
Used by both Vertices and Edges.
A vertex or an edge with the key word BLOCKED, will be
excluded from the model when walked.
v_BookInfomationBLOCKED
![Page 27: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/27.jpg)
Metrics
• Standard test metrics does not apply so well.• Test cases are not meaningful when running online
tests.• MBT tends to drive the requirement work.
![Page 28: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/28.jpg)
GraphWalker – QTP & SOAP
WebService("SoapServicesService").SetTOProperty "WSDL", "http://myComputerName:9090/mbt-services?WSDL"If ( not WebService("SoapServicesService").Reload() )Then Reporter.ReportEvent micFail, "MBT failure", "MBT encountered an error. See the MBT log files for information." ExitTestEnd If
Do until not WebService("SoapServicesService").HasNextStep() action = WebService("SoapServicesService").GetNextStep() If len(action) > 0 Then If not Eval( action ) Then Reporter.ReportEvent micFail, "Script failure", "The script encountered an error when trying to run function: " & action ExitTest End if End IfLoop
Reporter.ReportEvent micDone, "MBT Statistics", WebService("SoapServicesService").GetStatistics()
![Page 29: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/29.jpg)
5 min break..
I am not in the office at the moment. Please send any work to be translated.
![Page 30: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/30.jpg)
Hands on session
1. Setup the environment
2. Create a model
3. Generate code from model
4. Automate with selenium
5. Create XML configuration file for GraphWalker
6. Run the model!
![Page 31: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/31.jpg)
Files
1. Create directory MBT i.e C:\MBT
2. Create directory Models under in the MBT folder
3. Copy GraphWalker and Selenium jar files to the directoryselenium-java-client-driver.jarselenium-server.jargraphwalker-2.5.1-SNAPSHOT-standalone.jar
![Page 32: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/32.jpg)
Model
Visit www.brickor.com/MBT.htm with Firefox
Record with Selenium IDE by clicking on all elements and as well right clicking the result
Switch off recording.
Options > Format > JUnit4 (Java)
![Page 33: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/33.jpg)
Model
Start yEd to create the model
![Page 34: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/34.jpg)
Generate the Java Code
/** * This method implements the {EDGE_VERTEX} '{LABEL}' */public void {LABEL}(){ log.info( "{EDGE_VERTEX}: {LABEL}" ); throw new RuntimeException( "The {EDGE_VERTEX}: {LABEL} is
not implemented yet!" );}
java -jar graphwalker-2.5.1-SNAPSHOT-standalone.jar source -f Models/mbttest.graphml -t java.template.txt > javacode.txt
![Page 35: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/35.jpg)
Eclipse – Create the Java project
Create new Java Project; MbtTest
Create new Java Package; ie.mbt.test
Create new Class; MBTtest
Copy paste the contents of the generated Java code from the model into the created class.
![Page 36: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/36.jpg)
The java code
Include references to Selenium and GraphWalker in your project. Right click on your project and choose Properties. Under Java Build Path choose Libraries and then click Add External JARs.. Choose the two jar files:
graphwalker-2.5.1-SNAPSHOT-standalone.jarselenium-java-client-driver.jar
private HttpCommandProcessor proc = new HttpCommandProcessor("localhost",4444, "*chrome", “http://www.brickor.com/MBT.htm”);
private Selenium selenium = new DefaultSelenium(proc);
private Logger log = Util.setupLogger(MBTtest.class);// import org.apache.log4j.Logger;
// import org.graphwalker.Util;
![Page 37: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/37.jpg)
Writing the Java Code
In e_init we start the selenium object.
selenium.start();selenium.open("http://www.brickor.com/MBT.htm");
Delete the Throw exception!
Update all e_r11, e_r12 etc
selenium.click("Radio1_1");row1 = Integer.parseInt(selenium.getText(“r11"));
![Page 38: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/38.jpg)
Writing the Java Code
Create the utility GetResult
public boolean GetResult() {int rowTotal = row1 + row2 + row3;int calcTotal =
Integer.parseInt(selenium.getText("totalDiv"));
if (rowTotal==calcTotal)return true;
elsereturn false;
}
IMPORTANT!
Compile you class after any changes, CTRL + B
![Page 39: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/39.jpg)
GraphWalker XML
<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE MBTINIT SYSTEM "mbt_setup.dtd" ><MBTINIT EXECUTOR="java:ie.mbt.test.MBTtest" EXTENDED="true"> <MODEL PATH="models/MBTTest.graphml"/> <CLASS PATH="MbtTest\bin"/> <CLASS PATH="selenium-java-client-driver.jar"/> <CLASS PATH="selenium-server.jar"/> <GENERATOR TYPE="RANDOM"> <CONDITION TYPE="TEST_LENGTH" VALUE="1000"/> </GENERATOR></MBTINIT>
![Page 40: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/40.jpg)
Selenium Server
java -jar selenium-server.jar
Will start with default settings on port 4444
![Page 41: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/41.jpg)
Start the scripts..
Command line to start GraphWalker with the correct settings:
java -jar graphwalker-2.5.1-SNAPSHOT-standalone.jar gui xml -f MBTTest.xml
![Page 42: SoftTest Ireland: Model Based Testing - January 27th 2011](https://reader033.vdocuments.us/reader033/viewer/2022061201/54795c83b4af9f3c5e8b48b9/html5/thumbnails/42.jpg)
Questions?
More info at:
www.graphwalker.org
mbt.tigris.org
http://en.wikipedia.org/wiki/Model-based_testing
Email: [email protected]