cost & quality effective management of your test environments dennis janssen testframe research...
Post on 19-Dec-2015
213 views
TRANSCRIPT
![Page 1: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/1.jpg)
Cost & quality effective management of your test environments
Dennis JanssenTestFrame Research Centre, CMG
18 April 2023
![Page 2: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/2.jpg)
2
Agenda
•What can go wrong with a test environment and why does it happen so often?
•Situation in practice
•Technical solutions and results of its implementation
•Orginazational solutions and results of its implementation
![Page 3: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/3.jpg)
3
What can go wrong?
•Errors due to environment instead of “real” software failure
•False sense of security (testing against the wrong environment)
•Delays in testing (and therefore the whole project) due to environmental problems
•Loads o’ money spent!!!!
![Page 4: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/4.jpg)
4
Why does it go wrong?
• It’s very complex to build in todays complex architectures
• Most are project based, so…• … focus on 1 IT system instead of business processes
• … for the duration of the project
• … no core business for developers
• Testdata problems, no dedicated environments
• No maintenance
• De-centralized, no co-ordination
![Page 5: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/5.jpg)
5
The situation we found…
![Page 6: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/6.jpg)
6
The technical environment
• Tandem
• IBM OS/390
• IBM RS6000
• PC-Lan
• Web Architecture
• Siemens Nixdorf
• Philips
• DECC/ Alpha
• Sun Solaris
• Windows 95/ 98/ NT
• Websphere
• CICS
• MQ-Series
• Netweave
• Generic Service Layer
• To name but a few…
![Page 7: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/7.jpg)
7
The organisation
• Very de-centralized
• Every project operates almost alone, loads of hardware
• Lots of “shopping” needed to build a test environment (> 10 contacts is not unusual)
• Test environment = numbre 1 problem for developers and testers alike (survey)
• No “owner” of the test environment problem
• Test environment maintenance? Never heard of!!
![Page 8: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/8.jpg)
8
Problem Analysis
• Technical nature • Organisational nature
![Page 9: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/9.jpg)
9
Technical Solution
• Insourcing of key information systems in a central test environment
• Master – copy concept
• Develop distribution procedures to create copies (software and data)
• Create dedicated environments
• Version control & configuration management is a must (master & copies)
• Consistent testdata (not frozen)
• Synchronizing with third party test environments
![Page 10: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/10.jpg)
10
A
C
B
D
Master environment
BA
BA
C
Copy environments
C D
I
II
III
Basics of technical environment management
![Page 11: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/11.jpg)
11
Results of technical solution
• 90% less duration when building atest environment (quicker work)
• 65 % reduction in costs of building a testenvironment (less work)
• Dedicated testenviroments per project (1:1)
• Almost no bugs (down by 95%) during testing caused by a mistake in the testenvironment (less wasted time and frustration during test execution phase)
• Synchronized testdata (also with third parties) so end-to-end tests are possible
![Page 12: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/12.jpg)
12
Orginazational solution
• Seperate department for building AND maintaining test environments
• Testenvironments = core business
• Divided in tasks
• Generalists
• Start early in the project
• Single Point Of Contact (SPOC)
• Planning office
![Page 13: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/13.jpg)
13
Order 1
TechnicalCo-ordinator A
Order 2
TechnicalCo-ordinator B
Order 3
TechnicalCo-ordinator C
Order 4
TechnicalCo-ordinator D
•Internal co-ordination•Account Management
•Accounting/ reporting•Planning•Copy procedures•Distribution procedures
•Update Master•Update Copies
•Create copy environments•Make tailor made solutions•Support during test execution phase
![Page 14: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/14.jpg)
14
Results of organisational solution
• Always the right environment
• Always up to date testenvironments
• 95% of needed testenvironments is deliverd in time
• Support for the project is there when needed
• 50% less hardware needed because of explicite planning
• Less pressure on projecmanager, we take care of the environmental business (spoc)
![Page 15: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/15.jpg)
15
Implementation challenges
• Challenges when implementing:
• De-centalized nature of organization
• “Not invented here”
• How to counter challenges:• Communicate top down and
bottom up
• Make small steps
• Make quick steps
• Give added value
• Give them a “good nights sleep”!
![Page 16: Cost & quality effective management of your test environments Dennis Janssen TestFrame Research Centre, CMG 04 June 2015](https://reader031.vdocuments.us/reader031/viewer/2022032703/56649d2c5503460f94a01a5e/html5/thumbnails/16.jpg)
16
Summary
• Developing and maintaining test environments is complex work
• Challenges both in technique and organisation, solving only one is like solving nothing at all
• Create a dedicated unit to focus on test environments
• Be a SPOC for your customers (tread them as such!)
• Duration and cost will go down, quality will go up!!!!