Tester TrainingKevyn Matijevich & Chris Scofield Input by Shahid Khan

Just Break It• Your job is to attempt to break the software.• Think like the user, do not follow the rules.• The requirement follows an order. Mix up the order, when

possible.• Test all browsers, mobile testing, Mac OS testing.• See tips and tricks for more ways to break it.

Types of Testing

White Box

Grey Box

Black Box

• Based on Functional Requirements

• AKA Functional Testing

• Tested by test engineers

• Based on Functional & Design Specification

• Code Based Testing• AKA Unit testing• Tested by Developers

• Based on User Requirements

• AKA User Acceptance Testing

• Tested by Users

Black Box Test Types

Check the integrity of database field

values.

User Interface (aka User Experience) tests how

Users interact and view the software.

Follow the specification step by step and then don’t follow

step by step. Think outside the box, like a user. Don’t JUST follow the test script

path.

Black Box Test Types – cont.

Bounds/Range Checking

Check to make sure a number is within a

certain range. Test the edges and the middle.

Data Validation

Checking that data is valid. UI data matches

DB entries

White & Black Box Test Types

Process intended to reveal flaws in the security mechanisms of an information system that protect

data and maintain functionality as intended.

Due to the logical limitations of security

testing, passing security testing is not an indication that no flaws exist or that the

system adequately satisfies the security

requirements.

White Box Test Types

Installation Testing

Assures that the system is installed correctly and working on the hardware and integrated

modules work together as expected.

Performed to detect defects in the interfaces and interactions

between integrated components or systems, including hardware.

In an object-oriented environment, this is usually at the class level, and the minimal unit tests include the

constructors and destructors. Written by developers as they work on code to ensure that the specific function is

working as expected.

Grey Box Test Types

A subset of all defined/planned tests covering the basic functionality of a component or system. Smoke testing is

conducted in order to ensure that the basic functions of the whole program work correctly, without the necessity to go

into further details. Daily build and smoke testing are common practices.

Tests conducted to verify that the defined user requirements work as the user intended. The tests are usually performed by

clients or end-users. Also referred to as User Acceptance Testing or UAT.

Performance Testing

LOAD/STABILITYTesting heavy loads, such as testing of a web site under a

range of loads to determine at what point the system's

response time degrades or fails.

STRESS

Deliberately intense or thorough

testing used to determine the

stability of a given system or

entity. It involves testing beyond

normal operational capacity,

often to a breaking point, in

order to observe the results.

Executed to determine how a system or sub-system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify

other quality attributes of the system, such as scalability, reliability and resource usage.

Rho’s Agile Methodology• Agile• For Rho, larger project testing is done in an Agile process,

meaning testing is concurrent with software development. As code changes are made, the code is tested manually (soon to be automated) to ensure the code changes meet the requirements that governed them.

Rho’s Other Methodologies

• Few projects at Rho operate under • One phase must complete prior to the

next beginning• Used to be the industry favored

methodology. • Too strict and holds development and

technical advances back

Idea

Analysis

Test

Design

Develop

• Most of Rho’s projects operate under• The planning phase must complete

prior to analysis and the UAT phase must not start until development completes

• Analysis, development, and design may overlap

Waterfall Modified Waterfall

Validation

• Documented evidence that provides a high degree of assurance that software continuously meets the needs of users.

• May be required by Rho when it meets the requirements set forth in Rho’s policies and FDA regulation 21 CFR Part 11

• Validation effort begins with planning, includes all testing types, ends with reporting.

• Short story: define and document user requirements and document testing of requirements.

Role in Validation

Test types commonly used during validation at Rho• Unit• Functional (Regression as well)• Installation (test and production environments)• User Acceptance

Test types commonly used during validation • Unit• Installation qualification (test and production environments)• Functional• User Acceptance• Operational qualification• Performance qualification

Getting Started

Issues completed by Developers Issues vetted by Testers

Test Cases created for each issue (include feature,

release version, functional area, etc.)

Testing executed and issues tracked accordingly

How to Test• A Test Script is compiled by the tester

and added to the Test Cases. This is a list of steps a tester is to complete to verify the validity of the code changes made

• Defects are notated in the Test Case, a new issue is opened for review by the developer if defect does not relate to the active Test Case

• Documentation is critical for testing. See Documenting Your Test card

Test Stack• Through your desktops Remote application, you can access

the following VMs:• RhoVM212- Win7, IE8, Chrome, FF• RhoVM213- Win7, IE10• RhoVM214- XP, IE8, Chrome, FF• RhoVM215- XP, IE7• RhoVM216- Win8, IE11• RhoVM217- Win8, IE10• RhoVM218- Win7, IE11• RhoVM219- Win7, IE9

• Chris Scofield, the Test Engineer is the main contact for gaining permissions to the Test Stack

Tips and Tricks• Multi-click • Data entry tips and tricks• Irregular characters (%, &, #, etc.)• Spacing • Single and double quotes• Cutting and pasting from other software

• Folder for test files (good and bad) Word docs, excel, etc.• Page layouts across browsers• Lorem ipsum website http://www.lipsum.com/

Document Your TestBe explicit in steps for recreationWrite down your inputsTrack your browser and version, operating system

Take screenshots of errors/unexpected outcomesWrite down your outputs

Projects might use JIRA, Crucible, and Zephyr to record test outcomes, others may use

spreadsheets, or a combination of several

Documenting UAT/IQT

A quality control review is

performed on the executed and

completed test script.

Tester follows test steps and documents on test script as detailed in

tester manual and on test

script

Tester is trained how to

document, provided a

Tester Manual to follow, and provided the Test Scripts

Validation Incident Tracking

• SharePoint is used to create and track incidents that occur during validation

• Test Incident Report forms are filled out with:• Status • Incident summary, criticality, priority, corrective action• Type of incident (e.g. tester error, documentation error, software defect)• Resolution Notes, Resolved by, Resolved on• Closed by, Closed on, Closed during software version

Jira Issue Tracking

Agile Development via KanBan board

Bug and issue tracking

Zephyr plugins for Test Case creation

Zephyr JIRA plugin that allows for seamless integration for total Test Management

and data tracking

Creation of Test Cases and Test

Scripts

Creation of test cycles

Test execution tracking and

notes

Regression Testing

• Testing performed after making functional improvement or repairs

• Used to determine if the change has regressed other aspects of the program and introduced new errors.

• Often accomplished through the construction, execution and analysis of product and system tests.

Rho Regression Testing• Rho currently uses a Risk Based approach to Regression

testing. This means that for every Release Version, we only Regression test the features that have code changes. If a feature was not touched, it is not retested. Issues are logged in Excel, sorted by Feature.

• Testing begins with an IQT, then tests are executed on the Test Server (For Agile projects only)

• Execution notes are kept by the testers, all failures are triaged by Project Lead

Communication• Who do you go to for what?• Validation questions

• Kevyn Matijevich• Cheryl Scott• Carrie Dundas-Lucca

• Testing questions• Chris Scofield• Shahid Khan

• Where do you go to for helpful tips?• Testers• Developers• This Training• Internets