service validation and testing - a ca service...
Post on 04-Feb-2018
220 Views
Preview:
TRANSCRIPT
TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING
Malcolm RyderCA SERVICES / ARCHITECT
MARCH 2010
Service Validation and Testing:A CA Service Management Process Map
Copyright © 2010 CA. All rights reserved. All trademarks, trade names, service marks and logos referenced herein belong to their respective companies. This document is for your infor-mational purposes only. To the extent permitted by applicable law, CA provides this document “As Is” without warranty of any kind, including, without limitation, any implied warranties of merchantability or fitness for a particular purpose, or non-infringement. In no event will CA be liable for any loss or damage, direct or indirect, from the use of this document including, without limitation, lost profits, business interruption, goodwill or lost data, even if CA is expressly advised of such damages.
Table of Contents
TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING 1
Executive SummaryChallenge
The Information Technology Infrastructure Library version 3 (ITIL® V3) process framework approaches IT Service Management (ITSM) in terms of the lifecycle of a service. The Service Lifecycle is an organization model providing insight into the way ITSM is structured, and embodies critical guidance for IT organizations seeking to improve service quality and align more closely with business goals to create value for the business and its customers.
However, ITIL V3 best practice guidelines across the five phases of the service lifecycle are complex and challenging to interpret, and they are not designed to provide definitive advice about implementing ITSM processes. Many IT organizations consequently undertake an ITIL journey without a firm idea of their goals and the path to achieve those goals.
One of the key ITIL management processes, Service Validation and Testing, presents its own special challenges. Participants in the service quality assurance effort must coordinate the priorities and extent of testing with the type and timing of business needs to be met; consequently, the measure of success in validation and testing extends beyond the technical aspect to the matter of business value.
Opportunity
The primary objective of the Service Validation and Testing process is to ensure that a service offering, as designed, deployed and supported, has the quality expected to meet customer expectations.This quality is a strategic process outcome. CA has developed a unique approach to representing the ITIL framework and its interdependent IT Service Management (ITSM) processes at a high level in the form of an easy-to-use subway map. This map is an ideal starting point for understanding and communicating about ITIL in support of successful program planning and implementation.
Benefits
Following the Validation and Testing map provides guidance to:
• Improve business process capabilities
• Effectively upgrade the IT infrastructure in timely alignment with business needs
• Foster the efficient delivery of multiple IT services
• Improve the quality of technical support
Executive Summary 1
SECTION 1:
Finding the Right Path to IT Service Excellence 3
SECTION 2:
Service Validation and Testing 5Adopt Best PracticesValidate & VerifyTracking Validations Versus RequirementsPerform TestsReport / Closure
SECTION 3:
Benefits 9A Key to Achieving IT Service Excellence
SECTION 4:
Conclusions 10
SECTION 5:
About the Author 10
2 TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING 3
Simplifying ITILThe ITIL V3 process framework focuses on the service lifecycle and the way that service management components are structured and linked. It embodies critical guidance for IT organizations that are seeking to improve service quality and align more closely with business goals.
But, the ITIL V3 best-practice guidelines across the five phases of the service lifecycle are complex and challenging to interpret. Moreover, they are not designed to provide definitive advice about implementing IT Service Management (ITSM) processes. Many IT organizations consequently undertake an ITIL journey without a firm idea of their goals and the path to achieve those goals.
CA has developed a unique approach to charting the ITIL journey through a visual representation of the ITIL framework and its interdependent ITSM processes modeled after an urban subway system. This three-part map (Figure A) presents an easy-to-navigate, high-level view of the ITIL terrain. IT executives, strategists and implementers can use these Service Management process maps along with the family of CA Service Management process map technology briefs that expand on them. The maps and technology briefs provide a common reference point for understanding and communicating about ITIL and help you with program planning and implementation.
How to Use the CA Service Management Process MapsCA’s Service Management process maps (Figure A) illustrate every process (or track), each activity (or station) and the key relationships that are relevant to navigating continuous IT service improvement. The ITIL quality cycle takes the form of a “circle” with each Plan-Do- Check-Act (P-D-C-A) step as a process integration point (junction) on the line. Junctions serve both as reference points when assessing process maturity, and as a means to consider the implications of implementing a process in isolation.
Strategic controls (Service Portfolio Management, Demand Management and Financial Management) are needed to reduce risk and optimize integration across the service lifecycle, as illustrated on the three points of the triangle centered in the P-D-C-A quality circle (seen more easily in Figure B). These strategic controls help in evaluating, prioritizing and assuring the appropriate levels of financial and human resources for existing and new services.
This paper is part of a series of Service Management Process Map technology briefs. Each brief explains how to navigate a particular ITIL process journey, reviewing each process activity that must be addressed in order to achieve process objectives. Along each journey careful attention is paid to how technology plays a critical role in both integrating ITIL processes and automating ITIL process activities.
SECTION 1: CHALLENGE Finding the Right Path to IT Service Excellence
Most organizations are highly sensitive to quality levels in IT deployments, and they already rely on continuously managed processes and practices that are expected to provide some assurance of the fitness of deployed components and systems for their purpose. For that matter, validation most often means verifying that component and systems specifications are appropriate to established norms for operational risk. Testing takes place under the watchful eye of domain and subject matter experts in the line of duty for resolving problems and executing changes.
Yet as we see in the process interactions shown on our subway map, the circuit from planned modifications to deployment releases is an incomplete journey. This is because builds and installations may or may not go according to expectations. But once utilization begins, quality requirements differ again, and acceptance of the deployments and releases depends on further steps defined more from a business user’s perspective than from a technical one.
Service Transition
Service Validation and Testing
Service Asset and ConfigurationManagement
Change Management
Transition Planning and Support
Release and DeploymentManagement
Key Intersections
Strategic Controls
Strategic Inputs
Continual ServiceImprovement
Prioritize
BuildSchedule
ImpactAnalysis
Execute(Emergency/Standard)
Schedule Change
CAB Review
Evaluation/Decision
RFCAnalysis
Categorize
Planning
Preparation
DevelopStrategy
Preparationand Planning
KnowledgeTransfer
Monitorand Report
ManageBuild/Release
FinancialManagement
DemandManagement
Service PortfolioManagement
Deploy
Verify
PerformTests
Validate &Verify
AdoptBest Practices
EnsureReleaseValue
Report/Closure
AssureQuality
Audit
StatusReports
Maintain AccurateService Configurations
CoordinateResources
BusinessResponsiveness
ConfigurationControl
IdentifyConfigurations
Manageand Plan
DO
CHECK
PLAN
ACT
TransitionPlanning and
Support
TransitionPlanning and
Support
Release andDeployment
Management
Release andDeployment
Management
ChangeManagement
ChangeManagement
Service Asset andConfigurationManagement
Service Asset andConfigurationManagement
ServiceValidation
and Testing
ServiceValidation
and Testing
FIGURE A
CA has developed three maps: Service Design, Service Transition and Service Operation since most ITSM discussions focus on these critical ITIL disciplines
Service Transition
Service Validation and Testing
Service Asset and ConfigurationManagement
Change Management
Transition Planning and Support
Release and DeploymentManagement
Key Intersections
Strategic Controls
Strategic Inputs
Continual ServiceImprovement
Prioritize
BuildSchedule
ImpactAnalysis
Execute(Emergency/Standard)
Schedule Change
CAB Review
Evaluation/Decision
RFCAnalysis
Categorize
Planning
Preparation
DevelopStrategy
Preparationand Planning
KnowledgeTransfer
Monitorand Report
ManageBuild/Release
FinancialManagement
DemandManagement
Service PortfolioManagement
Deploy
Verify
PerformTests
Validate &Verify
AdoptBest Practices
EnsureReleaseValue
Report/Closure
AssureQuality
Audit
StatusReports
Maintain AccurateService Configurations
CoordinateResources
BusinessResponsiveness
ConfigurationControl
IdentifyConfigurations
Manageand Plan
DO
CHECK
PLAN
ACT
TransitionPlanning and
Support
TransitionPlanning and
Support
Release andDeployment
Management
Release andDeployment
Management
ChangeManagement
ChangeManagement
Service Asset andConfigurationManagement
Service Asset andConfigurationManagement
ServiceValidation
and Testing
ServiceValidation
and Testing
FIGURE B
The Service Transition map represents a journey of developing and improving capabilities for the transition of new and modified services to production.
Service Design
Service Catalog Management
Service Level Management
IT Service Continuity Management
Capacity Management
Info. Security Management
Availability Management
Key Intersections
Strategic Controls
Strategic Inputs
Continual ServiceImprovement
Service LevelManagementService LevelManagement
IT Service Continuity
Management
IT Service Continuity
Management
CapacityManagement
CapacityManagement
Info. SecurityManagementInfo. SecurityManagement
AvailabilityManagementAvailability
Management
Service CatalogManagement
Service CatalogManagement
DemandManagement
Service PortfolioManagement
FinancialManagement
Reviewand Audit
ManageSecurity Incidents
OptimizeAvailability
Mitigate Risk
MonitorServices
Methods/Techniques
DocumentService
Definition
Build CatalogContents
BusinessServiceViews
TechnicalServiceViews
PublishLive Services
ReportAchievements
ManageIssues
CatalogOperational
Services
MonitorPerformance
Design SLAFramework
ReviseSLAs / OLAs
Meet BusinessRequirements
ServiceReview
CustomerSatisfaction
DetermineVulnerabilities
Assess Risk
Build
Test
MaintainBusinessServices
SpecifyContinuity
Requirements
MonitorDemand
Adjust and Tune
DeliverRequiredResources
Analyze
Test
ProactiveManagement
AnalyzePerformance
Build Plan
ForecastRequirements
Set SecurityControls
Assess &Classify Assets
MaintainPolicy
Model/Trend
DO
CHECK
PLAN
ACT
Service Operation
Problem Management
Incident Management
Event Management
Request Fulfillment
Knowledge Management
Access Management
Key Intersections
Strategic Controls
Strategic Inputs
Continual ServiceImprovement
InformedDecisions
Fulfillment
Approval(Financial, Compliance)
Capture Info.
Prevent andEliminateProblems
Record
Resolve
Monitor / Track
Service Request(Incl. Self-Service)
Access Request
Verify
ProblemControl
KnownErrors
Record
Review/Action
Select Response
Filter /Correlate
Detect(Incl. Fault Detection)
Raise Incident
Investigate
Diagnose
Escalate
Resolve/Recover
Work Around
RestoreService
DeliverStandardized
Services
Automateand Control
SecureService Access
ErrorControl
ExecutivePolicy
ProvideRights
Transfer/Disseminate
Store Info.
DemandManagement
Transformto Usable
Knowledge
FinancialManagement
Service PortfolioManagement
DO
CHECK
PLAN
ACT
ProblemManagement
ProblemManagement
KnowledgeManagement
KnowledgeManagement
AccessManagement
AccessManagement
RequestFulfillment
RequestFulfillment
IncidentManagement
IncidentManagement
EventManagement
EventManagement
4 TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING 5
Alignment and compliance with the business is the overall goal of validation and testing, mandating explicit awareness of business requirements and coordination with them. Key points in this coordination include:
• Cooperation of specific business and technical roles
• Documentation of requirements at different levels of service, and corresponding documentation of the plan for validations
• Tracking the validations versus the requirements
Another question that comes up is about how much testing is enough. Too much is difficult to justify, and too little is too risky. One of the main outcomes of the Service Validation and Testing process is an increase in the ability to recognize a return on the investment in testing, which encourages the inclusion of right-sized effective testing in the business-level planning and budgeting for IT Services development.
SECTION 2: OPPORTUNITY Service Validation and TestingThe Service Validation and Testing process is responsible for planning and coordinating tests to ensure that specifications for the service design are met and validated through delivery of the service and, including co-operation with the Release and Deployment process, to manage and limit risks that could result from insufficient utility and warranty of the service in operation. Service Validation and Testing takes the technical proficiencies and plans of quality assurance for IT and strategically places them in the larger context of service quality for the business.
The strategic issue is that the business must be able to grow and adapt in order to maintain its success and critical competencies in its market ecosystem of customers, partners and competitors. This makes both Capacity and Change strategic areas of risk, and consequently it makes investments in service deliveries and service modifications part of financial management as well as business risk management.
The ultimate goal of the Service Validation and Testing track (Figure C) is a full alignment of Service characteristics to the utility and warranty requirements of the business. The subway map for this journey plots the key stations along the course in this way:
• Adopt Best Practices
• Validate and Verify
• Perform Tests
• Closure and Reporting
FIGURE C
The Service Validation and Testing track represents management’s attention to how verification of service characteristics is proactively aligned with the requirements for utilization and warranty of an upcoming new deployment of service
6 TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING 7
Adopt Best PracticesThe key practices of Service Validation and Testing will be
• A model for identifying business alignment
• An integrated set of test plans
• Training, knowledge transfer, and communications
• Verification of results through pre-defined tracking and reporting
These practices are represented in the following table, which illustrates the incorporation of testing within a larger process framework (inputs, activities, and eventual outcomes) of identifying business value from quality assurance.
The inputs for this high-level process express two things: a proactive identification of the customer’s view of a service, and the perspectives of the testing organization as driven by that view. Explicit definitions and standardized formulations of the relevant factors across the process are reusable, which allows for direct comparisons of different testing occasions. The results of such comparisons create an awareness that supports prioritization of service deliveries; and they inform analyses for the improvement of quality management procedures.
In the table above, items such as Service Level Packages (SLPs), Test Plans, and Reports are typical artifacts of the ITIL management process and can be researched in detail within most substantive ITIL publications. A service management organization will want to reach a level of confidence that the various items exist and are clearly related to each other. This will call for assigning clear responsibilities for routine production and review of the items.
Validate & VerifyThe outstanding feature of Service Validation and Testing is its deliberate synchronization of test types with defined business requirements, agnostic to particular technologies and to the variations of hardware and software. The following illustration presents an overview of this synchronization.
Tracking Validations Versus Requirements
• Specs decompose from top down—business to service to supplier
• Tests build from bottom up—components to functions to business agreements
SPECIFICATIONS VALIDATION TESTS KEY ROLES
Business Requirements Alignment to Business Agreements
• Customer
• Compliance Manager
Service Requirements User Acceptance • Service Level Manager
• Test Manager
• Test Manager User
Service Solution Design Operational Readiness • Change Manager
• Security Manager
Service Release Requirements
Release Package • Release Manger
Development Components with Assembly • Technical and Subject Matter Experts
Qualified Supplier(s)
Perform TestsThe essence of manageable testing is the ability to define tests as repeatable, measurable procedures.
This introduces the aspect of Key Performance Indicators (KPIs). As noted in most literature on performance management, KPIs are metrics that address efficiency, effectiveness and cost issues traceable to the design of an executing process and related to the outputs of that process. Indicators identify output characteristics that correlate to desired outcomes, and the means of driving those indicators will be seen as Critical Success Factors (CSFs).
Accordingly, testing will feature standards, scripts (methods) and metrics that are defined and logically related to each other before test execution actually begins.
Further, as a matter of setting scope to testing as well as methodological compliance, policies will be established to govern the decisions about when testing must occur and what type must occur. As identified in the table above, key roles in the organization take on responsibility and authority as stakeholders in the specifications to be validated. In that light, cooperation amongst the roles is expected and directed by policies, while also being institutionalized as integrations of their respective management processes. At points of process integration, the processes may feed each other as participants in shared workflow. In the ITIL subway map of processes, this is notable especially where the Build and Release of a service package involves validation and testing.
QUALITY ASSURANCE
Inputs Validations Outcomes
Service Quality Policy• Value Expectations
Service Package• Description of Designed IT Service
Delivered Value• Available and Supportable
Service Level Package• Intended Utility and
Warranty
Utility Level(fit for purpose)
Warranty Level(fit for use)
Release for Targeted Line of Service
Requirements and Targets• Measures
Usability Compliance Certification
Test Strategy• Plans, Model
• Resources• Methods, Scripts
Specifications Buisness Needs Quality Rating and Results Report
8 TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING 9
SECTION 3: BENEFITS
ITIL v3 describes Service Design, Service Transition (deployment), and Service Operation as three major phases in the service lifecycle. Within Service Transition, the Service Validation and Testing effort is one of several high-level management processes that link the appropriate design of the service to the smooth operability of the service.
IT Services are the building blocks of business processes and business services. A successful delivery of a service will feature the right thing arriving at the right place at the right time for the right reason. Validation and Testing confirms that the service is fit to its target purpose and fit to its offered use. The business uses multiple services to support its own performance and growth. Executives own the service portfolio, set service goals and control the investment in the service. This executive demand and influence thus should sponsor the qualification of the service as a primary concern of financial and risk management for the business.
The executive view of a successful service implementation includes concerns about the control, compatibility and delivery of the service. These three factors translate, respectively, into management of the quality, risk, and value of the provided service, as deliberately generated from end-to-end throughout the service lifecycle .
The goal of Service Validation and Testing is to ensure that specifications for the service design are met and validated through delivery of the service and, including co-operation with the Release and Deployment process, to manage and limit risks that could result from insufficient utility and warranty of the service in operation.
A Key to Achieving IT Service ExcellenceAutomating ITSM through technology can help your organization reduce the amount ofresources required to achieve ITIL v3 best practices. This assists your IT department in improving the quality of its services while embarking upon a continuous IT service excellence program focused on fostering business growth.
As you reach the end point of the Validation and Testing journey outlined in the CA Service Transition process map, your organization should have a better handle on the steps needed to promote successful service deployments. Specifically, bringing deployment efforts in line with ITIL best practices can help you:
• Improve business process capabilities
• Effectively upgrade the IT infrastructure in timely alignment with business needs
• Foster the efficient delivery of multiple IT services
• Improve the quality of technical support
One of the most important aspects of these process integration points is the documentation that occurs and is used from validation and testing. For example, a test plan usually precedes release and deployment activities, and an update of service records in a configuration management database (CMDB) usually follows deployment but precedes a post-implementation review (PIR). This arrangement highlights the cooperation expected between managers of services, configurations, changes and releases.
Report / ClosureDocumentation of results is also a fundamental deliverable of the Service Validation and Testing effort, as testing itself is expected to be the source of evidence that the service is not ready for delivery and may go through another iteration of development or modification before it can succeed.
As part of proper management of testing itself, these cycles of tests need directives identifying and justifying whether the scope and depth of testing is to be different or not in the next cycle versus the prior one. Because test cycling directly affects the turnaround time between development and release, the management of the test iterations affects the ability of the organization to meet service level agreements with customers.
Test reporting explicitly establishes the reasons why something is tested and why it is deemed suitable for release or not. As material for decision support, the reporting affects resource allocations as well as readiness for compliance auditing.
10 TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING
This document shall not serve to (i) affect the rights and/or obligations of CA or its licensees under any existing or future license agreement or services agreement relating to any CA software product; or (ii) amend any product documentation or specifications for any CA software product. The development, release and timing of any features or functionality described in this document remain at CA’s sole discretion. Notwithstanding anything in this document to the contrary, upon the general availability of any future CA product release referenced in this document, CA may make such release available to new licensees in the form of a regularly scheduled major product release. Such releases may be made available to current licensees of the product who are active subscribers to CA maintenance and support, on a when and if-available basis. In the event of a conflict between the terms of this paragraph and any other information contained in this document, the terms of this paragraph shall govern.
Copyright © 2010 CA. All rights reserved. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Intel and Intel Itanium are trademarks or registered trademarks of Intel Corporation in the United States and other countries. Microsoft, SQL Server .Net , Visual Studio, Internet Explorer, Windows Vista and Windows are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. UNIX is a registered trademark of The Open Group in the United States and other countries. Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. ITIL® is a Registered Trade Mark, and a Registered Community Trade Mark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. Red Hat® and Red Hat® Enterprise Linux® are trade-marks or registered trademarks of Red Hat, Inc in the United States and/or other countries. SUSE is a registered trademark of Novell, Inc., in the United States and other countries. Eclipse is a trademark of the Eclipse Foundation, Inc. SPARC is a trademark or registered trademark of SPARC International, Inc. in the United States and other countries. Adobe and Dreamweaver are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States and/or other countries. Mozilla® and Firefox® are registered trademarks of The Mozilla Foundation in the United States and/or other countries. Solaris is a register trademark of Sun Microsystems in the United States and other countries. IBM, z/OS, z/Architecture, CICS, WebSphere. DB2, MQSeries, System p5 , System p6, AIX and DB2 Universal Database are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. All trademarks, trade names, service marks and logos referenced herein belong to their respective companies. THIS DOCUMENT IS FOR YOUR INFORMATIONAL PURPOSES ONLY. TO THE EXTENT PERMITTED BY APPLICABLE LAW, CA PROVIDES THIS DOCUMENT “AS IS” WITHOUT WARRANTY OF ANY KIND, INCLUDING, WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT WILL CA BE LIABLE FOR ANY LOSS OR DAMAGE, DIRECT OR INDIRECT, FROM THE USE OF THIS DOCUMENT, INCLUDING, WITHOUT LIMITATION, LOST PROFITS, LOST INVESTMENT, BUSINESS INTERRUPTION, GOODWILL OR LOST DATA, EVEN IF CA IS EXPRESSLY ADVISED IN ADVANCE OF THE POSSIBILITY OF SUCH DAMAGES.
About CA: CA (NASDAQ: CA) is the world’s leading independent IT management software company. With CA’s Enterprise IT Management (EITM) vision and expertise, organizations can more effectively govern, manage and secure IT to optimize business performance and sustain competitive advantage. For more information, visit www.ca.com.
ID1688-032910
SECTION 4: CONCLUSION
Service Validation and Testing facilitates ITSM by managing the certification of IT in the form of services needed by the business. However, doing so requires rigorous processes, appropriate communication plans and support commitments—as well as an outline detailing where to start and how to proceed through the journey.
Following the steps outlined in the CA Service Transition process map gives organizations a clear view of how their Service Validation and Testing journey will take shape, illustrating the key stops en route to achieving effective requirements alignment. This journey results in:
• Strategic re-orientation of testing towards business value
• Better prioritization and investment in services
• Improved risk management
• Stronger integration with other ITSM and ITIL best practices
SECTION 5: ABOUT THE AUTHOR
Malcolm Ryder has over 25 years experience in the IT industry, with expertise in the areasof service transition, IT value management, and IT strategy. For the last 15 years, Malcolm has worked in consulting and solution strategy roles with a heavy emphasis on service management systems with vendors, service providers and end-user customers. Malcolm has been a co-developer of multiple market-leading commercial ITSM solutions since the mid ’80s.
top related