aqui
DESCRIPTION
TRANSCRIPT
testing, hosting and managing
business applications
August 8, 2006
Managing Quality in your ERP Project12 Mistakes to Avoid & Best Practices to Adopt
Managing Quality in your ERP Project12 Mistakes to Avoid & Best Practices to Adopt
Dan Downing, VP Testing Services
MENTORA GROUPAtlanta • Boston • DC
www.mentora.com
2
Objectives
Shine the spotlight on key Quality mistakes that ERP implementations should avoid
Things you will learn:
– The 12 Mistakes to Avoid – Their tell-tale signs and risks – Strategies for mitigating their impact – Tools to enable repeatability
3
ERP: Definition
Strict: • Enterprise Resource Planning
– Financials, Manufacturing, HR, Supply Chain, CRM…– e.g., Oracle 11i, SAP, PeopleSoft…
Looser:• Any business-critical packaged application that on
which you run a substantial part of your business– e.g., Integrated healthcare, student administration, hospital
management, customer service, brokerage & trading, staff recruitment & placement…
4
ERP Challenges
Technology Stack
Application Configurability
Enterprise Scope
Vendor
Dependency
• High-intrusion
• Expensive to implement
• Unique workflows, interfaces
• Often coupled with BPR
• Don’t control quality
• Conflicting maintenance cycles • Complex middleware
• Challenging to tune
5
Competing Stakeholders & Key Goals
ManagementHosting Provider
ERP Consultant
BusinessIT
ERP Vendor
QA
Competitive Advantage
Cost-savingsTime & Money
Personal goals
Budget
6
12 Quality Risks
Functional correctness
System performance
Testing infrastructure
Test cases
Who does testing?
How much time for testing?
Measuring quality
What tools?
Automated vs. manual
Sponsorship
Test dataEnough testing?
7
Mistake #1 Confusing product quality with implementation quality
-50,000
100,000150,000200,000250,000300,000350,000400,000
v.Xv.Y v.Z
Next ?
One Vendor's Defect Trend
Tell-tale signs• Industry analysts still bearish on ERP
software quality• Software is configured to your workflows,
interfaced to your surrounding systems, with your converted data
Risk• Configuration decisions trigger software
conflicts that yield incorrect results
Best Practices to Mitigate• End-end workflow functional testing• Upgrades: Run same tests on both
releases
Resources• Testing team, test case inventory,
automated functional testing tools
8
Mistake #2
Tell-tale signs• Hosting provider hosts no/few similar
environments (hw, patch level, disk subsystems)
• System configuration based on standard vendor recommendations vs. hard statistics
Risk• Under-configured environment• Poor peak load performance
Best Practices to Mitigate• Performance test peak transaction
volumes
Resources• Cross-functional performance team,
load testing tool, performance expert
Trusting your hosting provider’s environment assertions
DB Server CPU at 100%
9
Mistake #3 Failure to secure strong sponsorship for testing
Tell-tale signs• Quality not a corporate priority• No quality representation on leadership team• No Quality owner• No separate testing budget
Risk• Rocky go-live at best, major failure at worst
Best Practices to Mitigate• Make Quality an integral KPI for each major
activity of the implementation
Resources• Enroll VP of Development or Director of QA
to develop strategy
6
10
Mistake #4 Confusing conference room pilot with test case development
Tell-tale signs• The only test cases generated by conference
room pilot team• Test cases incomplete: coverage, input data,
expected results
Risk• Uneven quality due to inadequate test
coverage; testing the process, not the load
Best Practices to Mitigate• Make complete test cases a deliverable by
each function group
Resources• QA, business users, Test Management tool
or Excel
Nav Action Data Expected Actualaaaa bbbb ? ? ?cccc dddd ? ? ?eeee ffff ? ? ?
?
11
Mistake #5 Believing that testing can be done by the implementation team
Tell-tale signs• ERP consultant’s focus is on the software
supporting the business, not on the infrastructure supporting the load
• No separate testing plan• Business configuration team under-resourced,
over-stretched
Risk• Uncertain functional quality, poor performance,
no quality metrics for decision-making
Best Practices to Mitigate• Plan and staff Testing as a separate subproject
Resources• QA team, business users, testing consultant
12
Mistake #6 Failure to allocate enough time for testing
Tell-tale signs• Project plan shows testing not starting until
after configuration and development are complete
• Testing resourced by same people as other activities
Risk• Uncertain quality due to compressed testing
time, incomplete coverage, no quality metrics
Best Practices to Mitigate• Start test planning, tester training, testing
setup early in the project
Resources• QA team, business users, testing consultant
13
Mistake #7 Failure to make quality visible early-on
Tell-tale signs• Quality not a regular leadership topic• No/weak quality metrics for each phase• No compelling visuals of quality KPIs
Risk• No basis for management decisions based
on quality; uninformed go-live decision
Best Practices to Mitigate• Identify key quality metrics by phase; graphs
to summarize supporting detail (early: risk areas, functional gaps, enhancements; later: test coverage, testing progress, defects)
Resources• Test Management tool, Excel 0
50
100
150
200
250
300
350
400
450
17-Mar
18-Mar
19-Mar
22-Mar
23-Mar
24-Mar
25-Mar
26-Mar
29-Mar
30-Mar
31-Mar
1-Apr 2-Apr 5-Apr 6-Apr 7-Apr 8-Apr 9-Apr 13-Apr
16-Apr
WIP
PO
OM
MRP
INV
GL
BOM
AR
Sum of TestSteps
Finish_Date
Oracle_Module
14
Mistake #8 Failure to use the right tools to support testing
Tell-tale signs• Test plan assumes testing is manual• No budget for test tools, testing consultant• Few defined test cycles• Testing function but not load
Risk• Low test repeatability; error-prone; fewer test
cycles; incomplete test coverage
Best Practices to Mitigate• Strategy that combines automated and
manual testing; budget to support it
Resources• Functional, load & test management tools;
expert testing consultant to accelerate • Pre-defined scripts from value-add vendors
15
Mistake #9 Believing that automated testing replaces manual testing
Tell-tale signs• Test plan assumes testing all automated• Test plan compresses test time on
expectation that automated is faster
Risk• Ineffective and incomplete testing; test
automation black eye based on failed expectations; uncertain quality
Best Practices to Mitigate• Plan that blends selective automated testing,
whose coverage grows over time
Resources• Functional testing tools; expert testing
consultant to implement & transfer knowledge
16
Mistake #10
Tell-tale signs• Infrastructure plan has no/limited
dedicated testing environments• Insufficient database instances, disk
storage• Volatile, non-production test data
Risk• Frustrated test team, low testing
productivity; incomplete testing; uncertain quality
Best Practices to Mitigate• Plan for at least these environments:
pilot, user acceptance, QA-1 (current release), QA-2 (new release), training (2)
Resources• QA leadership, infrastructure team,
ERP/DB vendor
Failure to secure enough testing infrastructure
TestWorkstations
Test Instances
17
Mistake #11
Tell-tale signs• No plan for populating / copying /
refreshing test databases• Test cases not planned with
‘cascading data dependencies’ understood
Risk• Frustration; low testing productivity;
failure to meet timeline
Best Practices to Mitigate• Plan for disk-to-disk test DB backups;
sequence test cases with macro workflows & id data dependencies
Resources• QA leadership, data expert, Test
Management tool
Underestimation of effort to create reusable test data
Script:`Enter Requisition
Script:Engineering
Approval
Script:Finance Approval
Begin
Requisition to Purchase Order Process
Script: PurchaseOrder
Over$100K
???NO
Script:Materials Manager
Approval
YES
ReceiptType
Script: StandardReceipt
Script: DirectReceipt
Script: InspectionReceipt
Standard
Inspection
Direct
End of Requisition toPurchase Order
Process
Procure to Pay
Disk-to-disk DB refresh
18
Mistake #12 Believing that you’re done with testing once you go live
Tell-tale signs• No operational maintenance plan• No/low consideration for test reusability• Not a focus for any one stakeholder• Not today’s concern
Risk• Inability to keep up with patches, upgrades;
risk of introducing problems without proper testing
Best Practices to Mitigate• Growing, reusable automated test inventory
Resources• QA leadership, functional and load testing
tools
Then what?
19
Summary
• ERP quality is different– Scope, complexity, stakeholders, cost, competing goals– Quality is embedded in many activities– Don’t trust a stakeholder’s quality assertions
• Make it a leadership topic– Define quality KPIs for each phase– Graph them!
• Use tools that will enhance testing productivity– Planning, environments and disk are key– Automation is not a silver bullet– Repeatability yields ROI – you’ll leverage this after go-live– No sure way to test system performance without tools
• Watch for the tell-tales signs of these common mistakes– Leverage best practices, enroll experts to mitigate!
