massolution nyc 2013: pass
TRANSCRIPT
-
7/29/2019 Massolution NYC 2013: PASS
1/24
Building your Next Generation Customer Appswith Enterprise Crowdsourcing
Market facing applications areoften a customers first point ofcontact with the enterprise!
Improve results throughCrowdtesting, a customer-centric solution in anapplication-driven world!
1
-
7/29/2019 Massolution NYC 2013: PASS
2/24
Enterprise Crowdtesting Solutions
2
20+ years in Software QA Services
Established Crowdtesting Practise
Leading Crowdtesting Platform Solution
Professional Crowdtesters Communitycovering >70 countries
Delivery Models:
Managed Service
Self Service
Platform as a Service
Financially independent
PASS Group
Industry Coverage
E-Commerce
Media & Entertainment
Education & eLearning
Financial Services
Manufacturing / Logistics
Technology
Retail
Healthcare & Life Sciences
Web Apps
Mobile Apps
Application Types Desktop Apps Client Server Apps
Client References
-
7/29/2019 Massolution NYC 2013: PASS
3/24
How relevant is on-line interaction with your
customers to your business?
3
-
7/29/2019 Massolution NYC 2013: PASS
4/24
How can we avoid this kind of customer
experience?
4
-
7/29/2019 Massolution NYC 2013: PASS
5/24
Software quality assurance has matured as a
professional discipline during the last 15 years,
but:
5
why is software quality still so bad?
-
7/29/2019 Massolution NYC 2013: PASS
6/24
Some reasons:
6
Too many use case scenarios, can't be
covered by traditional testing approach
The variety of mobile devices, OS, browsers isconstantly growing!
=> Today's testing can only
focus mainly on criticalfunctions
-
7/29/2019 Massolution NYC 2013: PASS
7/24
Todays IT landscape presents more challenges
than ever!
7
TestingGoals
Speed
Qua
li
ty
MoreConsumer Driven
MoreCloud Based
ShorterProduct Cycles
MoreHW & SW Choices
More Users,More Places
MoreOpen Source
IT
Cost
Companies cannot keep up
with quality, cost, and speed
Testing accounts for as much
as
30% of SW development costs
Testing has never been more
critical to
reputation/operations.
- ComputerWeekly
-
7/29/2019 Massolution NYC 2013: PASS
8/24
Traditional Testing Approach
8
Always the same
heads, always the
same computers used
for tests, no fresh
ideas
Limited availabilityEmployed Tester
earns always same
salary no matter how
many bugs he finds
or does not find
-
7/29/2019 Massolution NYC 2013: PASS
9/24
Crowdsourcing + Testing =
9
CrowdtestingMany varying
heads, many
different devices /
computers, diverse
demographics
Availabilityguaranteed on-
demand
Crowdtesters
earn as much as
they find - nooverhead costs!
-
7/29/2019 Massolution NYC 2013: PASS
10/24
Principles of Crowdtesting
10
Thousands of testersavailable
They run the race
and deliverimmediate results
You choose the right
ones
You offer them
attractive bounties &
rewards
-
7/29/2019 Massolution NYC 2013: PASS
11/24
The Crowdtester Species
11
Software
Users
Professional
Testers
Employees &
Contractors
Subject Matter
Experts
-
7/29/2019 Massolution NYC 2013: PASS
12/24
Organization Forms of Crowdtesting
12
Testers
TestDesigners
TestManagers
ResultReviewers
Managed Crowdtesting
Supervisors
Self-Service / Supervised
Crowdtesting
Testers
Testdesigners
TestmanagersReview
ers
-
7/29/2019 Massolution NYC 2013: PASS
13/24
Crowd Capacity and Productivity
13
Crowdtesters are mainly part time
workers or freelancers
Average capacity harvested from a
crowdsourced team can be random, > always
invite more than needed!
Crowdtesters invest an average of
30-50% capacity during a test
cycle, if paid based on deliverables
But productivity (output) is 3 times
higher due to the competitive
nature of the process and due to
higher number of testers
-
7/29/2019 Massolution NYC 2013: PASS
14/24
Benefits
14
On-demand
Better
Cheaper
Faster
Scalable
Lead times of hours/ days
instead of weeks >=3 times faster delivery
2-3 times more defects and
great coverage
50%-70% lower costsTens of thousands of
testers,
hundreds of projects
Crowdtesters think and work indifferent ways than internal testers, thisis why crowdtesting finds bugs whichhaven't been detected duringconventional software testing.
Michael Palotas,Head Quality Engineering eBay Europe
-
7/29/2019 Massolution NYC 2013: PASS
15/24
Concerns
15
Private Crowd
Solutions
Accountability
Security
Confidentialit
y
-
7/29/2019 Massolution NYC 2013: PASS
16/24
Case Study Online Auction Application
Effort Details 16 Weeks Duration, Biweekly Updates
8 Test Cycles
88 Crowd Testers + 5 Crowd Champions
1 Internal Test Managers
3 Reviewers
Technology Coverage
OS: Linux, Mac, Windows (incl. Win8 beta)
Browsers: IE7 to 10, Firefox, Chrome, Safari
Regional Coverage 26 Countries
5 Languages
Platform
16
56% total cost reduction achieved 33% reduction of test cycle time 2748 tickets logged with > 70% acceptance 150+ new features & enhancement requests accepted by client
Key Highlights
-
7/29/2019 Massolution NYC 2013: PASS
17/24
Case Study: Web Shop
0
100
200
300
400
500
600
Total Active Testers
Total Tickets
17
-
7/29/2019 Massolution NYC 2013: PASS
18/24
Enterprises have the choice to
18
Outsource to
crowdtesting
service vendors
Build their own
crowdtesting
solution
License crowdtesting
platforms
-
7/29/2019 Massolution NYC 2013: PASS
19/24
Make sure your applications get crowdtested
19
if you want
happy
customers!
-
7/29/2019 Massolution NYC 2013: PASS
20/24
PASS GroupZrich Berlin Boston BelgradeBangalore Mumbai
www.pass.ch
www.passbrains.com
20
http://www.pass.ch/http://www.passbrains.com/http://www.passbrains.com/http://www.pass.ch/ -
7/29/2019 Massolution NYC 2013: PASS
21/24
Crowdtesting helps to ensure Compatibility
21
It works
for me!
Workson mydevice!
I founda majordefect!
Same
issuehere!
-
7/29/2019 Massolution NYC 2013: PASS
22/24
Crowdtesting Platform
22
Communication &Collaboration
Test Management &Delivery
Real-time Reporting
Real-time dashboards forprogress tracking
Automated reports & metrics
Forums & chat rooms tocommunicate, broadcast andshare project information
Cloud based test managementtools suite, accessible acrossorganization and easy to useinterface
Project Scheduling and projectstaffing from private and publiccommunity
Tester database supportingrapid identification of matchingskill set, infrastructure,demographics and ratings
Knowledge Center
On-demand knowledgenetworking, knowledgeprofiling and expert
identification Social knowledge sharing
Tester Profiles
Projects & Schedules
-
7/29/2019 Massolution NYC 2013: PASS
23/24
Crowdtesting Business Model
23
Bounties & Rewards ($$$)based on their deliverables
Bug ReportsUsability ReportsDesigned Test CasesExecuted Test Cases
Client pays:
Fixed price Output based Subscription
=
Crowdtesters receive: Client receives:
-
7/29/2019 Massolution NYC 2013: PASS
24/24
Selection is Key
24
EnterpriseCrowdtestingPlatform
Demographics
Skills
Devices, Configurations
Domain Background
Available Capacity
Performance Records
Projects
ResourceRequirements
Schedules
24