smarter infrastructure testing: a service oriented approachict infrastructure deployment new ict...
Post on 17-Jul-2020
4 Views
Preview:
TRANSCRIPT
Smarter Infrastructure
Testing: A Service Oriented
Approach
Egbert Bouman
Maintain
Smart Infrastructure TestingA service-oriented approach
Egbert Bouman, Maintain B.V.
Ellen Zondag, Kadaster
The Netherlands
Complex ICT infrastructures
FELETFELET
CEDAR
GL
CEDAR
GL KASKAS DelfiDelfi
CEDAR
AR/AP
CEDAR
AR/AP
IDITIDIT
SVSSVS
Postcode
Tabel
Postcode
Tabel
MAREGMAREG
Document
Flow
Services
Document
Flow
Services
DOC1
Generate
DOC1
Generate
Meeting PointMeeting Point
TP
Admini-
stratie
TP
Admini-
stratie
CervisCervis
POR/
ADN
POR/
ADN
ADN
Postbus
ADN
Postbus
PMB Polis Mutatie Bericht
PPR Prolongatie Bericht
PBI Polis Interface Bericht
CetisCetis
= Interface
xxx = Interface identificatie
Legenda
AUDA
SCAN
(RDC)
AUDA
SCAN
(RDC)
BOA
ARBO
BOA
ARBO
SEOSSEOS
Proces
Mngmnt
Proces
Mngmnt
KvKKvK
ARBO
Totaal
ARBO
Totaal
FISHFISH
BCS
BOA -
ARBO
WN-
bestand
Klant Order
Schermen
Klant Order
Schermen
Content
Manage-
ment
Content
Manage-
ment
E-ABSE-ABS
Scan
Straat +
FAX +
Scan
Straat +
FAX +
Content
Manage
ment
Content
Manage
ment
Klant
Beeld
Klant
Beeld
Contact
Historie
Contact
Historie
TRAVONTRAVON
DOC1
Inter-
active
DOC1
Inter-
active
002 001
007
008
009
010
011
012
013
021
022
023
024
025
077
027
028
030
032
033
035
037 038039
040
041
042 043 045 046
052
053
057
058
059
060
xxx = Bestandsinterface
BARBAR
065
BETA
(AAB)
BETA
(AAB)
MultiFit
Desktop
MultiFit
Desktop
066
GIM ServerGIM Server069
GIM
Server
GIM
Server070
SLBSLB
074
RDW
Services
RDW
Services
026
078
101
WALVIS
UWV
WALVIS
UWV
080
081083
084
085050
LDAPLDAP087086
049
DDDDDD
Overvoer
Volmacht
089
090
091
Batch
Adapter
Batch
Adapter
Mutatie
Net
Mutatie
Net
PP
Offerte
tool
PP
Offerte
tool
092
Klant
Order
Scherm
en
Klant
Order
Scherm
en
GIM
Ser-
ver
GIM
Ser-
ver
Inter-
face
Man
ager
Inter-
face
Man
ager
= Applicatie076
003093
094
100
004
= Schermpresentatie097
Printstraat
+ FAX +
Printstraat
+ FAX +
098
UWV031
VARVARClearing-
huis
Regres
Clearing-
huis
Regres
FIOD -
TAPE
099
Dossier
Volg
Systeem
Dossier
Volg
Systeem
= Request-Reply
= Fire&Forget
= Batch
= R&R en F&F
en Batch of nog
niet bepaald
102
103
ICT Infrastructure Deployment
New ICT infrastructure components Such as: OS‟s, DBMS‟s, network, storage solutions
Make end user and business applications work
Must be seamlessly “fitted in” With uninterrupted support of business app‟s and processes
Maintaining agreed service levels
Smooth and fast Excessive human resource claims not acceptable
Neither a top heavy and lengthy process
A universal challenge!
Primary goals are about minimizing risks : Keep faulty components away from production environment
Ensure reliability, availability, supportability (RAS)
Check unharmed application functionality and availability
Secundary goals : Enable ICT governance
Embed into maintenance standards and procedures
Prepare and train maintenance staff
Testing Infrastructure Deployment
Testing plays a vital role in this game
Different from application testing
Different Quality aspects Less: functionality, usability
More: reliability, availability, supportability, scalability, …
Different Stakeholders Less: business stakeholders, end users
More: acceptance by maintenance organisation
Different Components Less: custom built
More: standard components, commodity, COTS, open source
Different Objectives Less: realising competitive edge
More: cost reduction, standardisation
Fairly different test scope and risks
State of the practice
Testing methods, models and techniques Optimized for application testing
Not for infrastructure testing
Dedicated test types (PAT, Exploitation test) a technician‟s party
Testing partly managed by outsourcing partners Technical management: often OK
Application management: maybe OK
Functional management: maybe not OK
Some help available from: ITIL, ASL
Not from the testing communityDedicated test methods hardly available
Dutch Land Register
In Dutch: “Het Kadaster” Since 1832
Main office in Apeldoorn (near Utrecht)
Semi-government, 2000 employees, 400 in IT
Highly complex ICT infrastructure: Geographical and administrative applications,
SAP, Windows, VMS, Unix, Citrix, Mainframe
Crucial data for past and current society
Kadaster online (www.kadaster.nl)
e-Europe Awards for eGovernment 2005
Kadaster ICT environment
Kadaster TSC
Professional „Test Support Centre‟ Foundations laid in 1993
Full fledged independent department since 1999
Valued partner Good track record
Sound methodology
Testing business applications
Professional staff 35 test professionals
Manager: Jan van der Horst
Kadaster infrastructure test issues
Major business disruptions
Too little, too late, too long
No adequate process, test types and milestones
Technical know-how hardly shared
Miscommunication between testers and …
Stakeholdership not addressed
Testing optimized for business app’s only
Ap
pli
cati
on
Infr
astr
uctu
re
The “black hole” before PPT
ST GFT AF
S
PP
T
Reali
sati
eR
ealisati
on
?
Develop a dedicated test approach… … in close co-operation with Kadaster TSC…
… to deliver a reliable quality report…
… concerning all applicable quality aspects…
… for new infrastructure components…
… in combination with business applications…
… in order to fit them into place more smoothly…
… in a shorter time…
… and without disruptions
And let‟s put it into place … … starting with a pilot project…
… in the Kadaster organisation
Commission to Maintain
Who is Maintain?
Maintain Independent testers
For 16 years
Top-200 companies
70+ consultants
Mainly in the Netherlands
Recently: Valori group founded
SmarTEST test approach Just enough, smarter testing
Flexibility and adaptability
Best practices
Eurostar sponsor
www.maintain.nl
The answer
A dedicated test process framework For fitting infrastructure components into place
Jointly developed by Kadaster and Maintain
SmarTEST based
We call it FitTEST
Four key principles
Key principles
1. Get to know your infrastructureand adopt a service-oriented reference model, i.e. TOGAF
2. Define service-specific best test practicesand make „how to test‟ guidelines for each service type
3. Think parallel rather than linearand adopt a non-phased test approach
4. Apply a SMART artifact hierarchyand optimize it for re-use and knowledge sharing
Standard software Custom build software
Communication Infrastructure
Native
Manage
ment T
ools
Security S
ervice
s
DB
MS
‘s
Oth
er
Service
s
Servers and Storage
Operating Systems
Mid
dlew
areM
anagem
ent
Gene
ric Manage
ment
Tools
Mainframe
Quality
aspects
Test
objects
Specific Fit Criteria
PRIMA risk matrix
Generic
GuidelinesService-specific, with tips and
tricks
Project
documentation:
reqts and specs
Test DesignTest scenarios, clusters, cases
FitTEST ManualOverall handbook Infrastructure
Testing
Direct
stakeholder
input
Validation,
completeness check
Generic Fit criteriaAll stakeholders
Quality ProcessKadaster Quality Handbook
Get to know your infrastructure
Situation at Kadaster Three different inventories available
No clear grouping or reference model
Need for structure and grip From Kadaster service philosophy viewpoint
Why not an existing standard?
Architecture Frameworks
Which one to adopt?
Architecture Frameworks
Popular Architecture Frameworks in Europe: Zachman Developed at IBM
TOGAF The Open Group Architecture Framework (Open Group)
IAF Integrated Architecture Framework (CGEY)
We selected TOGAF because: It‟s service-oriented, it‟s open and it‟s free
It provides a suitable Technical Reference Model
Further reading: www.opengroup.org/togaf
www.software.org/pub/architecture
(enumeration and discussion of many available frameworks)
TOGAF technical reference model
Highest level:
three layers
Two bottom layers
represent infrastructure
Top layer out of scope
Applications
Applicationsplatform
Communication infrastructure
API: Applicatie Platform Interface
CII: Communicatie Infrastructuur Interface
www.opengroup.org/togaf
The Kadaster infra-model
Standard software Custom built software
Communication Infrastructure
Native
M
anagem
ent T
ools
Security S
ervice
s
DB
MS
‘s
Oth
er S
ervice
s
Servers and Storage
Operating SystemsM
iddle
ware
M
anagem
ent
Generic M
anagem
ent
Tools Infra
Structure
Main frame
Business Applications
Benefits of TOGAF for testing
The benefits of this open, service-oriented reference
model (TOGAF) for testing:
Standard Test suites available Free, proven
Support for development of additional suites
Easy to fit into currently popular service approach: Enterprise Service Bus
Software Oriented Architectures
XML, SOAP testing
Key principles
1. Get to know your infrastructureand adopt a service-oriented reference model, i.e. TOGAF
2. Define service-specific best test practicesand make „how to test‟ guidelines for each service type
3. Think parallel rather than linearand adopt a non-phased test approach
4. Apply a SMART artifact hierarchyand optimize it for re-use and knowledge sharing
Standard software Custom build software
Communication Infrastructure
Native
Manage
ment T
ools
Security S
ervice
s
DB
MS
‘s
Oth
er
Service
s
Servers and Storage
Operating Systems
Mid
dlew
areM
anagem
ent
Gene
ric Manage
ment
Tools
Mainframe
Quality
aspects
Test
objects
Specific Fit Criteria
PRIMA risk matrix
Generic
GuidelinesService-specific, with tips and
tricks
Project
documentation:
reqts and specs
Test DesignTest scenarios, clusters, cases
FitTEST ManualOverall handbook Infrastructure
Testing
Direct
stakeholder
input
Validation,
completeness check
Generic Fit criteriaAll stakeholders
Quality ProcessKadaster Quality Handbook
Service-specific best test practices
Empower your skills They are there
Just find, document and deploy them to everyone
For each service: Make standard checklists and acceptance criteria (what to test)
Select and refine smart test techniques (how to test)
Add tips and tricks
Bundle in one handy guideline for each service group
Better in-company test support: „How to‟ reference manual and checklists per service group
Improved knowledge management and skills allocation
Bundled in “Test Guidelines”
Ownership and
continuous
improvement
Not obligatory But strongly
recommended
Based upon growing
experience
A typical guideline: OS’s
Introduction
Characteristics of OS‟s General
Open VMS
True64 UNIX
Windows
Test approach Component Integration Test
Application Integration Test
Management Integration Test
Applicable Test Techniques
Top-10 Quality Aspects
Hints for Test execution
Key principles
1. Get to know your infrastructureand adopt a service-oriented reference model, i.e. TOGAF
2. Define service-specific best test practicesand make „how to test‟ guidelines for each service type
3. Think parallel rather than linearand adopt a non-phased test approach
4. Apply a SMART artifact hierarchyand optimize it for re-use and knowledge sharing
Standard software Custom build software
Communication Infrastructure
Native
Manage
ment T
ools
Security S
ervice
s
DB
MS
‘s
Oth
er
Service
s
Servers and Storage
Operating Systems
Mid
dlew
areM
anagem
ent
Gene
ric Manage
ment
Tools
Mainframe
Quality
aspects
Test
objects
Specific Fit Criteria
PRIMA risk matrix
Generic
GuidelinesService-specific, with tips and
tricks
Project
documentation:
reqts and specs
Test DesignTest scenarios, clusters, cases
FitTEST ManualOverall handbook Infrastructure
Testing
Direct
stakeholder
input
Validation,
completeness check
Generic Fit criteriaAll stakeholders
Quality ProcessKadaster Quality Handbook
Test Types
Test types A well-known sequence: UT, IT, ST, AT
Test types for infrastructure testing: CT: Component Test
= intake and unit test
CIT: Component Integration Test
= does it fit into the existing technical infrastructure
AIT: Application Integration Test
= does it fit the business applications
MIT: Maintenance integration test
= does it fit into the maintenance organization
PPT: Pre Production Test
= last gateway before production, in production like environment
How to organize test types in time
CTCIT
AITMIT
PPTTiled
Parallel
CIT AIT MITSequentially CT PPT
CT PPT
Mix
Iterative
The Integration Circle or Fit Circle
Fit
Criteria
Intake
De
charge
To Production
Environment
CT
PPT
Ready
for fit
Key principles
1. Get to know your infrastructureand adopt a service-oriented reference model, i.e. TOGAF
2. Define service-specific best test practicesand make „how to test‟ guidelines for each service type
3. Think parallel rather than linearand adopt a non-phased test approach
4. Apply a SMART artifact hierarchyand optimize it for re-use and knowledge sharing
Standard software Custom build software
Communication Infrastructure
Native
Manage
ment T
ools
Security S
ervice
s
DB
MS
‘s
Oth
er
Service
s
Servers and Storage
Operating Systems
Mid
dlew
areM
anagem
ent
Gene
ric Manage
ment
Tools
Mainframe
Quality
aspects
Test
objects
Specific Fit Criteria
PRIMA risk matrix
Generic
GuidelinesService-specific, with tips and
tricks
Project
documentation:
reqts and specs
Test DesignTest scenarios, clusters, cases
FitTEST ManualOverall handbook Infrastructure
Testing
Direct
stakeholder
input
Validation,
completeness check
Generic Fit criteriaAll stakeholders
Quality ProcessKadaster Quality Handbook
The 5 principles of SMART testing
Strategic Connect to strategic business goals, critical
succes factors and KPI‟s
Men-oriented Plan and design your process for maximum
stakeholder participation and commitment
Adaptable Be prepared for change and opt for flexibility, just
enough process and documentation
Risk based No risk, no test
Transparant Create transparancy and report visually, based
upon fit and acceptance criteria
Basic Test Design Process
Quality
aspects
Test
objects
Specific Fit Criteria
PRIMA risk matrix
Generic Project or maintenance release
Workshop with
stakeholders
Comment
GuidelinesService-specific, with tips and
tricks
Project
documentation:
reqts and specs
Facilitated bij Test
Support Centre
Test DesignTest scenarios, clusters, cases
Risk-based selection ,
development and
(autom.) execution
FitTEST ManualOverall handbook Infrastructure
Testing
Direct
stakeholder
input
Validation,
completeness check
Generic Fit criteriaAll stakeholders
Quality ProcessKadaster Quality Handbook
Check Fit Crteria with
PRIMA matrix
Kadaster Pilot Project
Citrix Metaframe XP roll-out Complete re-arrangement of server farms
Integration with other management facilities (CA Unicenter TNG
framework, agents, …) and maintenance procedures
High risk for business application availability
All desktops affected
Successsfully tested On time, on schedule
Positive evaluation report
Management decision to
adopt FitTEST approach
Issues still being addressed
Technics Test environments don‟t meet the needs (yet)
So testing is still done too late on an unstable environment
Culture: Ongoing struggle to get technical testers to use the handbooks
The guidelines have to grow by using them
Improvement feedback loop still weak
More than (management) commitment
required
Achievements made
Process: Black-hole is filled, successful pilot
More „grip‟ to the testing process, improved TSC role
TOGAF based reference model adopted Maintained bij Architecture Management department
Handbooks available Overall “Handbook Infrastructure Testing”
Six guidelines for specific service-groups
Method incorporated into Kadaster practices Integration Test Circle incorporated
Growing use of acceptance criteria (not only infrastructure)
Kadaster improvements (being) achieved
So, now available to you:
A dedicated architectural reference model TOGAF, optimized for testing
Service-oriented, open, free
An effective process for infrastructure testing Fitting in components faster and more smoothly
By a combination of linear and non-linear process
Based upon the „integration test circle‟ concept
A dedicated design and documentation structure From risk analysis to test design
Optimized for re-use and knowledge sharing
Standard software Custom build software
Communication Infrastructure
Native
Manage
ment T
ools
Security S
ervice
s
DB
MS
‘s
Oth
er
Service
s
Servers and Storage
Operating Systems
Mid
dlew
areM
anagem
ent
Gene
ric Manage
ment
Tools
Mainframe
Quality
aspects
Test
objects
Specific Fit Criteria
PRIMA risk matrix
Generic
GuidelinesService-specific, with tips and
tricks
Project
documentation:
reqts and specs
Test DesignTest scenarios, clusters, cases
FitTEST ManualOverall handbook Infrastructure
Testing
Direct
stakeholder
input
Validation,
completeness check
Generic Fit criteriaAll stakeholders
Quality ProcessKadaster Quality Handbook
For the sake of less business disruptions
Thanks for your attention
Further questions, need help? Please feel free:
egbertbouman@maintain.nl
ellen.zondag@kadaster.nl
top related