![Page 1: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/1.jpg)
©Clare McLennan 2014
Building an Automation Ecosystem
![Page 2: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/2.jpg)
Nature
![Page 3: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/3.jpg)
Software
Build and deployment
system
Code control
Test data
Automated tests
Manual tests
![Page 4: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/4.jpg)
The big picture
Ecosystem
Environment
Action
![Page 5: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/5.jpg)
The big picture
Ecosystem
Environment
Action
Why? When? Who?
How?
Where? Test data?
![Page 6: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/6.jpg)
Otherwise...
Tests unreliable
Tests aren’t maintained Not enough resource
![Page 7: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/7.jpg)
Ecosystem Environment Action
Ecosystem
![Page 8: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/8.jpg)
What’s the goal?
To create an automated test suite
Why?
![Page 9: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/9.jpg)
• Reduce testing cost
• Deliver quicker
• Deliver more frequently
• Reduce regression defects
• Develop features faster
• Reduce repetitive testing
• Smoke test
What’s the REAL goal?
![Page 10: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/10.jpg)
Is it achievable?
![Page 11: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/11.jpg)
Is it achievable?
0 121 year
![Page 12: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/12.jpg)
Is it achievable?
0 121 year
![Page 13: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/13.jpg)
Justifying the build cost
• Reduce manual regression testing
• Respond to customer needs quicker (more releases)
• Develop new features quicker
– Find bugs faster = cheaper to fix
– Refactor code to remove technical debt
• Reduce staff turn over by making work more enjoyable
![Page 14: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/14.jpg)
Right people?
• Tests automation systems are complicated software systems
• They require
–Design
– Some programming
–Best built by a team
–Commitment
![Page 15: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/15.jpg)
Right people?
• Support from – Developers
– Management
• Support to – Change existing systems
• Bigger job – $$ to do it right
Don’t try this by yourself! Get help from your team
![Page 16: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/16.jpg)
Ecosystem Environment Action
Ecosystem Environment
![Page 17: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/17.jpg)
Environment
• Where will tests run?
• Where does the test data come from?
![Page 18: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/18.jpg)
Data setup
![Page 19: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/19.jpg)
Data setup is critical Reason 1: Tests need to
run multiple times
• Develop the test
• Debug failing test
Once is enough
Once is enough – I know what I’ve done
![Page 20: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/20.jpg)
Data setup is critical Reason 2: Small changes
can break automation
• Uniqueness
– Eg. Adding the same user
• Paging
– Eg. The eleventh user may appear on a new page
I adapt tests
as I go
![Page 21: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/21.jpg)
Just because you can doesn’t mean you should
![Page 22: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/22.jpg)
Data setup is critical
Reason 3: Time
Timely test data is needed – Graph last 7 days data
Time stamps
Large time delays – Cookies expire after 30 days
![Page 23: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/23.jpg)
Data setup is critical
Reason 4: Summary reports
• Any change in the system changes the values
– Eg. Total $ received from all clients
Looks about right!
![Page 24: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/24.jpg)
Options for data
• Recent copy of production
• Canned data aka Golden
Database
• Create test data on-the-fly
![Page 25: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/25.jpg)
Production Copy
Implemented
What is the data?
Lots and lots and lots of
maintenance
Overcomplicated tests
![Page 26: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/26.jpg)
Canned data
Test script has: Reload testing database Log on as “Clare McLennan”
Manually create new organisation Upload data files Store database
2
1
![Page 27: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/27.jpg)
Canned data
Known state
Tests and database must be kept in sync
Loading databases slow
Writing new tests is slow
![Page 28: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/28.jpg)
Canned data
Upgrade path for database schema (including dev versions)
Storage
Prevent tests interfering
Tools available • Transactions
• Snapshots
![Page 29: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/29.jpg)
Create test data
Test script includes: Create new organisation Load “doc-huts.kml” Load “nz-fire-stations.kml”
![Page 30: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/30.jpg)
Create test data
Quick to write new tests
Options include – Database insertions
– Existing processes (eg CSV)
GUI tests run faster when data not created through UI
![Page 31: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/31.jpg)
Small is beautiful
• Run faster
• Easier to Debug/ Maintain
![Page 32: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/32.jpg)
Test Time Change application so test can
specify time/date
No need to change data now!
An extensive, but low risk change to application
![Page 33: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/33.jpg)
Environment
• Individual machines or shared environment?
1 *
![Page 34: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/34.jpg)
Environment
• Correct set up
• Tests independent
• In control – Change time
– Summed reports
![Page 35: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/35.jpg)
Shared environment
• Correct set up
• Tests independent
• In control – Change time
– Summed reports
1
*
Separate organisations
?
Per customer?
![Page 36: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/36.jpg)
Own machine
• Correct set up
• Tests independent
• In control – Change time
– Summed reports
![Page 37: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/37.jpg)
Team effort
• Database upgrades
• Software + test environment installer
• Source control
• Data extraction (small subset)
• Implementing test time
• Create test data
• Copy data (for customer/organisation...)
• Remove/Discard (customer/organisation..)
![Page 38: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/38.jpg)
Ecosystem Environment Action
Environment Action
![Page 39: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/39.jpg)
Action
Tools
First tests
Write to be maintained
![Page 40: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/40.jpg)
Which to choose?
![Page 41: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/41.jpg)
Design
Before GUI starts set up data Create “TestOrg1” Load “doc-huts.kml” Load “nz-fire-stations.kml”
![Page 42: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/42.jpg)
Design
Select layer “doc-huts” Select layer “nz-fire-stations”
Click X
![Page 43: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/43.jpg)
Visual recognition
X
New paradigm
Great for testing parts of application
• Visuals
• Difficult controls
![Page 44: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/44.jpg)
Choosing a tool • Flexibility
– Setting up data & environment
– Driven by another tool
• Maintainability
• Can everyone have a copy?
• Manual testing
![Page 45: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/45.jpg)
GUI testing
Component Unit Testing
GUI
Integration
(Automated) Slow
Accessible
Fragile
![Page 46: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/46.jpg)
Business logic testing
Component Unit Testing
GUI
Integration
(Automated) Reliable
May need devs help
Fast
![Page 47: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/47.jpg)
Design
Test Specification
Glue
Api for data
set up
Gui
Mock Business
logic
![Page 48: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/48.jpg)
Doing things differently
Adapting tests
Thinking
Usability testing
Doing things exactly the same
Repeating tasks
Checking numbers
People aren’t computers
![Page 49: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/49.jpg)
Easy to automate
Difficult to do manually
Valuable (reduces risk)
Sweet spot
Target start place
![Page 50: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/50.jpg)
Computers
• Hard to train to do a testing task
• Very good at doing it over and over ...and over and over...and over and over...and over...
![Page 51: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/51.jpg)
Good targets
Reports
Tax calculations
Multiple ad impressions
End to end UAT test
![Page 52: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/52.jpg)
Maintainability
Source: http://gojko.net/2010/04/13/how-to-implement-ui-testing-without-shooting-yourself-in-the-foot-2/
![Page 53: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/53.jpg)
Garden timer example
![Page 54: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/54.jpg)
Example
Go http://testsystem /admin Type “password” in input[id =‘password’]) Click button[text()=‘OK’] Wait for div[class=‘timerstart’] Click div[text= ‘Add interval’] Type “09:00” in input[class=“time”][0] Type “00:15” in input[class=“time”][1] Click button[text()=‘Save’] Wait for div[class=‘time’][0] has ’09:00’ Wait for div[class=‘time][1]. has
Rule
Workflow
Technical Activity
![Page 55: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/55.jpg)
Go http://testsystem /admin Type “password” in input[id =‘password’]) Click button[text()=‘OK’] Wait for div[class=‘timerstart’] Click div[text= ‘Add interval’] Type “09:00” in input[class=“time”][0] Type “00:15” in input[class=“time”][1] Click button[text()=‘Save’] Wait for div[class=‘time’][0] has ’09:00’ Wait for div[class=‘time][1]. has
So what’s this test doing?
Example
And what if we change the log in?
![Page 56: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/56.jpg)
Log on as admin Add time interval (start = 9:00, duration = 0:15) Wait until 08:55 Check timer is off Wait until 09:00 Check timer is on
Rule
Workflow
Technical Activity
Example
![Page 57: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/57.jpg)
If time interval is start = 9:00, duration = 0:15
then Timer should be off at 08:55 Timer should be on at 09:00
Rule
Workflow
Technical Activity
Example
![Page 58: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/58.jpg)
Image
Target
a
Tests (as rules)
![Page 59: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/59.jpg)
a
![Page 60: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/60.jpg)
public class ShouldTimerBeOn {
...
public void setStartTime(String onTime) {
this.startTime = onTime;
}
...
public String timerStatus() {
return Timer.isWaterOn(checkTime)
? "On" : "Off";
}
...
![Page 61: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/61.jpg)
Rule
Workflow
Technical Activity
Specification by Example aka ATDD
Tests expressed in a human readable and maintainable form
Testing at GUI layer or calling code directly
Tests relevant even when technology changed
![Page 62: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/62.jpg)
Lots of options!
![Page 63: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/63.jpg)
Ecosystem Environment Action
Ecosystem Action
![Page 64: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/64.jpg)
Nature
![Page 65: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/65.jpg)
As a separate activity
Coders Testers
Automators
![Page 66: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/66.jpg)
Specification by Example Discuss feature
![Page 67: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/67.jpg)
a
![Page 68: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/68.jpg)
Feature really DONE
Automate specs = tests
All test pass
Write specs Write Code
Discuss feature
Specification by Example
![Page 69: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/69.jpg)
Team approach
Testers specify the tests
Tests reduce manual testing
Coders write (testing) code
Team members doing
what they do best
![Page 70: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/70.jpg)
As part of story • Coverage
– Rate of application development = rate of test development
• Team ownership
– Coders consider how changes will affect tests
• Knowledge spread
• Easier hiring
![Page 71: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/71.jpg)
0 12
All on board
![Page 72: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/72.jpg)
Ecosystem Environment Action
Ecosystem Big picture
![Page 73: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/73.jpg)
The big picture
Ecosystem
Environment
Action
![Page 74: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/74.jpg)
My Guiding Rules
• Tests run on debug environment
• Testers can run tests
• Different users can run tests simultaneously
• Tests run as part of DOD
• Test run < 1 hour
![Page 75: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/75.jpg)
Tools
FitNesse http://fitnesse.org/
Sikuli http://sikuli.org/
Selenium http://seleniumhq.org/
![Page 76: Building an Automation Ecosystem - ANZTBBuilding an Automation Ecosystem . Nature . Software Build and deployment system Code control Test data Automated tests Manual tests . The big](https://reader033.vdocuments.us/reader033/viewer/2022060311/5f0ad75f7e708231d42d9a7e/html5/thumbnails/76.jpg)
Learn more
http://concordion.org/Technique.html http://www.engineyard.com/blog/2009/15-expert-tips-for-using-cucumber/
http://specificationbyexample.com/ http://gojko.net/
Gojko Adzic
[email protected] http://crazy-it-adventures.blogspot.com
Clare McLennan