Module 02Test Specificationand Execution

Topics

During the Specification phase, the required tests and starting points are specified. The aim is to have as much as possible prepared, in order to be able to run the test as quickly as possible when the developers deliver the test object.

http://www.tmap.net/en/tmap/4-essentials/structured-test-process/acceptance-and-system/specification-phase

The Specification Phase

Tests in the top quadrants validate business needs.

Tests in the bottom quadrants validate the implementation.

Agile Testing Quadrants

Performance &

Load Tests

Security Tests

‘ility’ Tests

Tests in the right-hand quadrants help the product.

Tests in the left-hand quadrants help the team.

Agile Testing Quadrants

Performance &

Load Tests

Security Tests

‘ility’ Tests

To obtain insight into the quality of the test object through the execution of the agreed tests.

http://www.tmap.net/en/tmap/4-essentials/structured-test-process/acceptance-and-system/execution-phase

The Execution Phase

Test Types

Acceptance Tests

Business-facing

Critique Product

A test script that checksif a feature behaves as expected.

Defined during backlog grooming and early inthe sprint.

Requirements and bugs link

directly to related test cases

Integrated Storyboarding with

PowerPoint keeps stakeholders and

all team members in sync

Measurable acceptance

criteria, visible to all team

members, aligns with tests

Result

User Story Test Status Report

Demo

Creating a Test Plan withLogical Acceptance Tests

Unit Tests

Supporting

the team

Technology

facing

Isolate each part of the program and show that the individual parts are correct.

Created in the sprint, Executed continuously.

Unit testing is a method by which individual units of source code, sets of one or more computer program modules together

with associated control data, usage procedures, and operating procedures, are tested to determine if they are fit for use.

Intuitively, one can view a unit as the smallest testable part of an application.

Definition of DoneAll unit tests are written and have been successfully executed.

Code CoverageCode coverage records for each package are available and whatever the team has decided as the minimum benchmarkhas been achieved.

Unit Tests

Use the testing framework

you’re familiar with to write

your unit tests

Quickly run all unit tests, or

target specific tests or

groups of tests

Quickly examine results of each

test during the previous test run

with the Test Explorer

Developer-focusedTesting

Red Green Refactor

Pluggable testframeworks

Run tests after build

Unit Test Explorer

Shims can be used to fake almost

any method call, rendering your

legacy code testable

Create a “fake” reference to any .dll

referenced by your testing project

The Fakes Framework allows you

to shim SharePoint web parts,

improving testability

Explore and execute tests using

the same first-class Test Explorer

Manage the unit test

project directly alongside

your SharePoint projects

Use familiar SharePoint code

to create your unit tests

Group unit tests by project,

traits, duration, results and more

Code coverage analysis determines

the portion of your code covered by

unit tests, down to the method level

Results can be exported and

saved for future analysis

Code Coverage

Code Clone Analysis

Static Code Analysis

Code Reviews

Test Impact Analysis*

Developer-focused Code Quality Tools

Demo

Code Quality Tools

Functional Testing

Supporting

the team

Technology

facing

Validates thefunctionality quality characteristic, with the functional design as the test basis.

Specified and executed during the sprint.

Specified and executed during the sprintAll functionality created during a sprint is tested during the same sprint.

Test coverageLinked work items are used for test coverage.

Definition of DoneAll priority 1 test cases are executed.All high risk backlog items have a test coverage of 80%.

Test Case promotion to regression setsSome test cases will be part of regression testing efforts.

Functional Tests

Test Steps

Shared Steps

Test Data Parameters

Specify Test Cases

Organize testing in multiple

configurations

Playback a prior recording to save

time with frequently-repeated

setup steps and prerequisites

Bind data to controls, to

support multiple iterations

and future automation

Pause, save, close, return, and

pick up where you left off

Test Runner

Action Recording

Execute Test Cases

IntelliTrace Logs

System logs & configurationDetailed status and history

visible to all team members

Audio & Video recordings

Get an overview of the test run

with summary pie charts, or drill

into the details for each test.

Analyze code coverage for manual

tests to determine which specific

parts of your code were tested .

Associate manual test runs

with builds and environments.

Recommended Tests

Shows passed Tests withchanged source code

Test Impact Analysis

Exploratory Testing

Supporting

the product

Supports the

Implementation

Simultaneous testdesign and execution.

Tests working software.

Create rich text comments

with screenshots while

exploring

Exploratory sessions can

be recorded to increase

reproducibility

Explore your application to

uncover new defects

The Test Runner docks on the left

side of the screen and automatically

records your steps as you explore

Time spent duringexploratory testingCourtesy of Jon Bach

Test Setup

Bug Filing

Testing

The 3 Components of Testing

Comments and steps are

automatically added

The exploratory test session

is paused while you create a

bug report

File new bugs without ending the

exploratory test session

Attach the video recorded during

the exploratory test session for

maximum reproducibility

Quickly create an

automatically linked test

case for this bug

Add verification values to

indicate the expected result

Session is paused while creating

test cases

Test steps are added

automatically

Comment block reflects

the added bugs and tests

Continue exploratory testing

right from where you left off

Quick access to bugs and test

cases created during the session

Demo

Exploratory Testing

Performance and Load Testing

Supporting

the product

Supports the

Implementation

Performance Test To determine or validate speed, scalability, and/or stability.

Load TestTo verify application behavior under normal and peakload conditions.

Web Performance Test

Load Test

Demo

Web Performance TestLoad Test

Test Automation

Test Automation

Performance &

Load Tests

Security Tests

‘ility’ Tests

Automated

and ManualManual

AutomatedTools

Automated

Visual Studio Support

Performance &

Load Tests

Security Tests

‘ility’ Tests

Convert the test case to

code for enhanced

flexibility

Generate your Coded UI test from

an existing recording, or record a

new test case on the fly

Test Automation with CodedUI Tests

CodedUI tests interact

directly with your app’s UI

Tests can be run and monitored

on the local machine, or further

automated on a virtual machine

Cross-browser testing support

allows automated testing on

multiple browsers

Coded UI tests run as part of your

unit testing suite, enabling

developers to detect defects early

Even if the look & feel changes, the

automated test still executes properly

Coded UI tests are part of

source control for

maximum visibility

Demo

End-to-end Test Workflowwith Coded UI Automation