translating tester-speak into plain english: simple explanations for 8 testing buzzwords
TRANSCRIPT
@neotys
TRANSLATING TESTER-SPEAK INTO PLAIN ENGLISH
@neotys
Simple Explanations for 8 Testing Buzzwords
@neotys
Is the website going to be ready for Black Friday?
Well, we’re in the middle of crowdsourcing tests for
likely user scenarios. Sanity checks have gone well but we haven’t run a full automated regression yet. Plus, Jimmy is going through some exploratory
work now.
@neotys
@neotys
EVEN PEOPLE ON THE SAME DEV TEAM HAVE THEIR OWN BUZZWORDS
Dev QA
Ops
@neotys
Crowdsource Cloud
Provisioning Pluggable architecture
@neotys
BUT WHAT DOES ALL *THIS* MEAN?
Testing as a Servi
ce
Smoke Testing
Crowdsourced Testing
Continuous Integration
Test Automation
Sanity Check
Exploratory Testing
@neotys
WE’RE ABOUT TO BREAK IT DOWN FOR YOU
Testing as a Servi
ce
Smoke Testing
Crowdsourced Testing
Continuous Integration
Test Automation
Exploratory Testing
Sanity Check
@neotys
1. CROWDSOURCED TESTING
Some hold a load-testing party
Others ask fellow testers to “attack” your server en masse.
@neotys
@neotys
1. CROWDSOURCED TESTING
It’s not terribly predictable, and doesn’t scale very far, but for the right cases it can be extremely effective.
It’s also a great way to get a community aware of what you are building.
@neotys
@neotys
2. TESTING AS A SERVICE
Outsourced testing services, where a 3rd party takes your script and executes a bunch of testing on your behalf.
@neotys
@neotys
2. TESTING AS A SERVICE
Suitable for:
• Unique environments• Short-term bursts of specialized activity• 3rd-party verification (security, audits)
@neotys
@neotys
2. TESTING AS A SERVICE
TaaS providers may operate with heavy automation in a well-equipped lab, or they
may be crowdtesting experts.
@neotys
@neotys
3. SMOKE TESTING
Back in the day… this was pretty literal.
Today, smoke testing is a cursory, preliminary system check. A gatekeeper.
@neotys
@neotys
3. SMOKE TESTING
Smoke testing checks the basics:
• Does it start?• Does it run?• Can a user log in?• Can a user execute a transaction?• Can the administrator view the dashboard?
@neotys
@neotys
3. SMOKE TESTING
Because when you kick off a long series of tests and go home for the night, you
don’t want the system to crash 5 minutes after you leave.
@neotys
@neotys
4. SANITY CHECK
Smoke tests• Ensure the software is stable• Test core functionality• Are automated and repeated
Sanity checks• Validate new features or fixed bugs• Check that changes are actually there• Are ad-hoc and unscripted
@neotys
@neotys
4. SANITY CHECK
Sanity checks make sure you are testing what you think you are testing.
@neotys
@neotys
5. REGRESSION TESTING
Did we create more bugs than we fixed?
Does everything that worked yesterday still
work today?
@neotys
@neotys
“5. REGRESSION TESTING
When I think about regression testing, I think about any testing that involves the reuse of tests (manual or automated) or test ideas (regression charters for example — a regression test does not necessarily need to be the exact same test) to manage the risks of change. This could include testing for bug fixes, testing to make sure a bug fix didn’t break something else.
Mike Kelly http://michaeldkelly.com
@neotys
@neotys
Give a man a fish, feed him for a day.
Teach a man to fish, feed him for a lifetime.
6. TEST AUTOMATION
@neotys
@neotys
QA used to be all about manual testing & finding bugs.
Today, modern QA organizations are coders, who make computers do this work for them.
Test automation improves scale, quality, efficiency – you name it!
6. TEST AUTOMATION
@neotys
@neotys
Many people think test automation is limited to functional testing. Not true.
More and more, test automation applies to many other areas like load & performance testing – automatically validating that software behaves well under load and stress.
6. TEST AUTOMATION
@neotys
@neotys
7. CONTINUOUS INTEGRATION
This is one of the most profound changes to the software development
process to happen in past decade.
@neotys
@neotys
7. CONTINUOUS INTEGRATION
CI relies heavily on automated testing. Every time code is checked in, a large test suite is run to make sure nothing is broken.
CI ensures a consistent level of high quality software, avoiding large disruptions that break builds for days at a time.
@neotys
@neotys
7. CONTINUOUS INTEGRATION
CI is a change to the organization that requires a high degree of:– Collaboration– Communication– Transparency– Automation– Teamwork
That’s why strong teams love it!
@neotys
@neotys
8. EXPLORATORY TESTING
Many people think it’s ad-hoc testing, but actually it’s an approach that is concisely described as simultaneous learning.
While methodically test a product, the tester learns how it behaves, and identifies its strengths and weaknesses.
@neotys
@neotys
“8. EXPLORATORY TESTING
A style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his or her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run parallel throughout the project.
Cem Kaner
@neotys
@neotys
THAT WASN’T SO BAD. WAS IT?
Testing as a Servi
ce
Smoke Testing
Crowdsourced Testing
Continuous Integration
Test Automation
Sanity Check
Exploratory Testing
@neotys
http://www.neotys.com/webcast/Performance-Testing-Horror-Stories.html
PERFORMANCE TESTING
Horror Stories
SEE THE FULL WEBINAR
TALES FROM THE SCRIPT
As the former manager of a load and performance team, Brad Stoner retells a number of haunting stories from his days in the field that carry insightful lessons learned about load and performance testing.
Watch this webinar and learn how to prevent problems from coming back to haunt you!