getting out of the testing game by bill matthews test architect manager technical tester...
TRANSCRIPT
Getting out of the Testing GameBy Bill Matthews
Test Architect • Manager • Technical Tester@Bill_Matthews
What is the Testing Game?
Testing is seen as an end in itself rather than a means to an end
Testing has become Nominalised (verb treated as a noun)
Testing is seen as a barrier that must be passed
Testing driven by process not outcomes
Testing is disconnected from its context
Those outside testing dictate test activities
The challenge
“Testing is taking too long”
“Testing doesn’t find all the bugs”
“It all goes fine until we hit testing…then it just falls apart”
“The PMs struggle to get good testing done”
“We use industry best practices but I’m sure they’ve changed since we last had a consultant in”
“Can you help us improve our testing process practices?”
The Challenge
Company was in a competitive market and so always had various offers in place.
Most offers took about 4 weeks to implement from idea to launch.
Frequently had to roll back changes because of errors.
Releases were slow, very early morning affairs
Rollbacks resulted in system outages => possible loss of revenues
Difficulties in getting a clear picture of what was happening during testing
Phased testing…for small changes
How the management viewed testing
How would you improve the situation?
Traditional approaches to test process improvement take an analytical approach.
Assumes current practices approximate some “best practice”
Measures against an idealised “best practice”
Suggests a prescription to bring current practices closer to the “best practice”
What if your processes are fundamentally broken and don’t fit your current needs?
Design Thinking (Exploring What If)
2 minute introduction to the Business Canvas Model
The Value of Testing
The Customers and Value Proposition segment is where the value of our testing activities can be realised.
Who are your customers and what problem do we solve for them?
Do we solve the “testing problem” for them?
If you are in the Testing Game…yes.
And our customers also think that’s the problem we solve for them
If not the “testing problem” – what problem do we solve?
How do our customers want to be engaged?
The Customer Relationship and Channels segments is where we understand how our customers want to interact with us.
So how do we deliver the information our customers value?
Those in the Testing Game think it’s all about Test Plans and Metrics
But have never really explored how their customers (really) want to work with them?
If you were a customer how would you want to be engaged?
Think about the user experience/journey when they engage with you – does it feel right and appropriate?
How do deliver something our customers value
Key Activities, Key Partner & Key Resource Segments
This is where the activities associated with Testing sit.
What are the key activities that we need to do (and do well) to deliver value?
Those in the Testing Game have a testing centric view so know all about these segments.
But customers just get what they are given – take it or leave it.
Wouldn’t it be better to have a customer centric view and adapt the How in line with the Need?
What we came up withKey ResoucesTestersTest LeadTest EnvironmentsTracking ToolsInvestigation Tools
RelationshipsBespoke (fits their needs)Dynamic (fits the evolving needs of the project)Trusted Advisor (tell it to them straight)Collaborative (same team - same goal)
Key ActivitiesTestingReportingDefect trackingTest PlanningTest Preparation
ChannelsSelf Service (defects, regular updates, dashboard)Meetings & workshops
Revenue StreamsCross Charged against project
Customer SegmentsProject SponsorsProject ManagersDevelopersChange BoardAuditors
Value PropositionProvide information about issues and threats that is timely, appropriate and accurate to support the decision making processes.Information that supports the mission and possible threats to the mission.Project Sponsor - trends, risksProject Managers - estimates, progress, coverage, risksDevelopers - defects, coverageChange Board - risksAuditor - Who knows?
Key PartnershipsInternalInfrastructure TeamComplianceExternalPerformance TestingSecurity Testing
Cost StructureSalaries (direct)Test Environments (cross charged)Training
The catalyst for change
What would be different if this is how we approached testing?
A flood of energised ideas
Not all of them practical but at least we were Exploring What If
What’s stopping us?
Only the Permission to change
Organic change
Some changes that happened:
The test team started to communicate…and it was infectious!
Collaboration - No longer a long chain of sequential processes
Abandoned the “one size fits all” processes and naturally adopted a more context driven approach
“Does that mean we don’t need to write test scripts?”
Test Plans were replaced with Kanbans
the PMs were wary of this at first but agreed to let us try and manage our own work.
Soon Kanbans were popping up all over the office (outside of the test team)
Focus was on the flow of information
PMs finally stopped telling the testers how to test…and we stopped having test phases based around testing levels.
The team seemed more energised
Overall time for small changes reduced to about 2 week
still not great but an improvement
So the Business Canvas Model is Magic then?
So all the change happened because of a single sheet of paper?
No really, the magic is with the people
It helped the team see a wider context and focus on what adds value
In most cases they already had ideas on how to improve their practices.
Gave them a framework to discuss what they do without really talking about testing
They were given permission to make changes
A little sad that they needed to be given permission