effective software test case design approach
TRANSCRIPT
Effective Software Test Case Design Approach
Charles Carson, MSSWE, CSM, Senior Software Quality Assurance Manager
I know the product, I can test off the top of my head!
I don’t have time to write down test cases… I have a few test cases just because. I don’t
need to worry about them ever, and only I need to understand them.
My tests are just to “complicated” to document…
Why bother with how I design test cases….Is this YOU?
Consistency/Structure Proper test configuration Understanding/No ambiguity
◦ Exactly how is the test run Accurate representation of User
Story/Requirement traceability/validation Consistent test run results/indication of software
quality/conformance Better communication with stakeholders i.e.
Project owner, development, sales, customer etc. in verification of software quality (test results)
Why we need effective test cases….Short list
What it is◦ Input◦ Action/Event◦ Response
Why◦ Validation◦ Standardized approach to validation◦ No “ad-hoc” approach
We have different opinions of the what and why of a test case don’t we…
Verify our software conforms to specification◦ Detect non-conformance◦ Communicate non-conformance◦ Contain non-conformance
Test activity accountability◦ Test Planning◦ Test Design◦ Test Execution/Reporting◦ Metrics
QA roles and responsibilities….A reminder
Generic testing process
Little bit of software test theory….Blah, Blah, Blah…
Test cases spawn from a Requirement/User Story/Use Case of some sort. But, first there is the User Story as in this case, or a short description of something that the end user wants.
Example User Story“US001: As a System User, I want to view the number of critical alerts at any given time per network so that I can monitor critical alerts more readily from a webpage.”
Now to the “nitty gritty”! Test case origination….Sure, I know this?
Just visually check it to “validate” the User Story!◦ We call this “Check the box testing”
No test case or just posting the User Story No definitive steps No logical path No real verifiable results
Conclusion “Yep, works as designed…I guess?”
Just how in the world does the tester approach this?
Yes, the User Story seems broado Yeah, I know this stuff (tribal knowledge)o I can just ask the developer, he/she might know.o Blah, blah, blah…
So what is really effective???
So what is the “effective” approach to designing a test case when there is only a User Story to go on?
Programmer(Development)
Test (Quality Assurance)
Domain Expert (Product Owner and or System
Architect)
Software test theory again…Consider the interactions….
How do we leverage this…working together?◦ The User Story
Acceptance criteria The boundaries of the User Story
Confirm if the story is complete Will the story work
Gathering information as detailed in the interactions The Product Owner/System Architect
Ask questions the what and the why The Developer
The how
Yeah, we work together…
“US001: As a System User, I want to view the number of critical alerts at any given time per network so that I can monitor critical alerts more readily from a webpage.”
Sooo, let us revisit the User Story
Use Case acceptance criteria…From interaction with the Product Owner/System Architect/Developer ◦ What type of critical alerts can be viewed◦ What is the minimum and maximum count of
critical alerts that are viewable◦ How much time does it require for a critical alert
to update◦ How are alerts generated and components
involved◦ Are the critical alerts accurate e.g. alert counts◦ Are non-critical alerts viewable
Effective test case design starts here!
Sooo, the Requirement/User Story is clear and verifiable… What have we done here?
Conversation – Details behind the story come out through conversations with the Product Owner/System Architect/Developer
Confirmation – Acceptance tests confirm the story is finished and working as intended
Confidence - We are “done” with the Requirement/User Story
User story acceptance criteria◦ Get the team to think through how a feature or
piece of functionality will work from the user’s perspective
◦ Remove ambiguity from requirements◦ For QA, acceptance criteria form the tests that
will confirm that a feature or piece of functionality is working and complete
Interactions, collaboration, etc. etc.
Now lets create our tests… Scope of User Story
◦ First, developers breakdown the user stories into discrete development focused tasks that are necessary to achieve what the User Story is describing.
◦ Knowledge of the tasks functions/participation at code design phase
◦ Must be “testable”◦ Put yourself in the “shoes” of the person using the
feature
Get to the point…Test Cases!US001: As a System User, I want to view the number of critical alerts at any given
time per network so that I can monitor critical alerts more readily from a webpage.
Test Step Expected Results
Status User Story/SCR
Generate and verify that critical alerts are present in “system being monitored” log files.
Critical alerts are present in log file.
Pass US001
Verify log file counts on each system.
Log file counts should match number of physical alerts generated.
Pass US001
On the System Alerts page, verify “system being monitored” counts are present.
Monitored systems should be present on System alerts page. Each system should accurately display respected counts
Fail US001/SCR3245
Nice isn’t it… Back to the “short list” We now have in our test
cases;◦ Consistency/Structure◦ Proper test configuration◦ Understanding/No ambiguity
Exactly how is the test run◦ Accurate representation of User Story/Requirement
traceability/validation◦ Consistent test run results/indication of software
quality/conformance◦ Better communication with stakeholders i.e. Product
owner, development, sales, customer etc. in verification of software quality (test results)
