deep dive modern apps lifecycle with visual studio 2012: software testing with microsoft test...
DESCRIPTION
More info on techdays.beTRANSCRIPT
Software Testing with Microsoft Test Manager 2012 and Lab Management 2012
Brian KellerPrincipal Technical Evangelist
USS Yorktown, SmartShip
• crew member entered 0 in a data entry field, caused a “divide by 0” error that shut down propulsion
• dead in the water for 2hrs 45mins
Ariane 5 Flight 501• re-used code from Ariane 4, but took a
different flight path because of different booster design
• conversation from 64bit float to 16bit signed int caused overflow (exception handler was disabled for perf reasons)
• cost: > $370 million @ 1996 prices
Vasa• sank after sailing less than 1 nautical
mile out of Stockholm
• specifications were faulty, failing to take into account enough ballast to keep the ship upright
• test teams revealed that it was unstable, but launch was rushed, and QA was ignored
Training
Conditioning
Climbing
Training Phase
Cost of Failure
Development
Test
Release
Software Phase
Cost of Bugs
Generalist Specialist
Manual Testing Some scripting
Creates scripts to set up lab, create data
Strong scripting skills
Some coding skills
Strong coding
Develops automated testing procedures
Expert coding skills
Tester Segmentation
Black Box Testing
White Box Testing
API Testing
70% of testing happens heremajority of test tools target here
What usually happens when a testerfinds a bug?
Graphic by Jeff Atwood (CodingHorror.com)
Unified ALM Solution
Team Foundation
Server
Planning SCM Work Item Tracking
Testing and Lab
Management
Build Automation
Reporting and
Analytics
demoMicrosoft Test Manager 2012
(1) Get source (2) Compile projects
(3) Copy build to running environment
(4) Run deployment scripts for each machine
(5) Create environment snapshot
(6) Executeautomated tests
(7) Send testresults
(8) Publish results toTeam Foundation Server
Build – Deploy – Test
Team Build
Team Foundation Server
Test Controller
VMM Agent Test Agent Test Agent
VM Host Web Server (VM)
Database Server (VM)
Virtual Environment
VMM Agent Test Agent Test Agent
VM Host Web Server (VM)
Database Server (VM)
SCVMM ENVIRONMENT
Test Architecture
Test Agent Test Agent
Web Server Database Server
STANDARD ENVIRONMENT
Test Agent Test Agent
VM Template 1
VM Template 2
VM TEMPLATE LIBRARY
Team Foundation Server
System Center Virtual Machine Manager
(SCVMM)
Team Build
Test Controller
SCVMM vs. Standard EnvironmentsSCVMM Environment Standard Environment
Build-Deploy-Test
Manual Testing
Snapshot / Restore
Share Bug Snapshots
Network Fencing
3rd Party Virtualization
Physical Machines
demoLab Management
Lord Kelvin, 1824-1907
“…you cannot manage what you cannot measure.”