"soa testing : 4 phased methodology and test automation framework"
TRANSCRIPT
![Page 1: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/1.jpg)
SOA Testing – SISE Approach & Scelta Test Automation
Framework
Sathyan S NairSenior Technical Analyst
December 13,2007
![Page 2: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/2.jpg)
Agenda
Needs & Solutions
Focus Areas & Challenges
SOA Lifecycle & SISE Approach
Scelta – The Automation Framework
Key – Benefits
Future Research
Open Discussion
![Page 3: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/3.jpg)
Needs & Solutions
![Page 4: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/4.jpg)
Business & Testing Needs
Business Needs► Evolve IT Solutions
► Avoid costly integration and re-engineering efforts
Testing Needs► How do we validate services for functional
requirements?► How do we manage the complexity of testing –
Integration of multiple services?► How do we validate shared services across
multiple applications, as services change and evolve?
![Page 5: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/5.jpg)
Address the Needs
How to address the needs?► Various stages of testing - Required to address
function as well as interaction complexities► Testers to wear additional hats and follow an
evolved approach► Automation Framework for multiple test
execution cycles
![Page 6: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/6.jpg)
Our Solution
![Page 7: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/7.jpg)
Focus Areas & Challenges
![Page 8: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/8.jpg)
How different is SOA Testing
Head-fewer services, without a UI
Services spread across different boundaries
Look beyond FT & PT – Integration, System & Workflow Testing
Requirements change impact – Across multiple application boundaries
Need for multiple end-to-end business process/workflow testing
![Page 9: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/9.jpg)
Addressing Challenges
![Page 10: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/10.jpg)
SOA Lifecycle – SISE Approach
![Page 11: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/11.jpg)
SOA Lifecycle – SISE Approach
![Page 12: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/12.jpg)
SISE Approach – Service/Component Testing What we do?
► Validation - Light-weight service Implementation
► Scripts Development - Assertions and Boundary Conditions
► “Good known data” based validation technique
http://www.webservicex.net/CreditCard.asmx?WSDL
Card Type – VISACard # - 7767545687679876
This Credit Card number is not valid.
Test Results
► Ensure the proper functioning of individual web-methods/services
![Page 13: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/13.jpg)
SISE Approach – Integration Testing
What we do?
► Validation – Communication between different systems
► Scripts Development – Messaging formats (namespace, SOAP compliance etc), Communication policies (parameters, return type etc), Security policies (Authentication, WS-Security etc)
► “Good known data” based validation technique
Integration Testing
Service C
Output
Output
Output
Service A
Service B
Results A and B Input
Input
Input
Test Results
► Different types of services across multiple systems seamlessly integrate and communicate with each other
![Page 14: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/14.jpg)
SISE Approach – System Testing
What we do?
► Validation – Communication between different systems
► Scripts Development – Messaging formats (namespace, SOAP compliance etc), Communication policies (parameters, return type etc), Security policies (Authentication, WS-Security etc)
► Mock services to handle – Asynchronous Messaging communications
► Message Checkpoints – Route messages to appropriate services
► Performance Monitors – Study the response times , uptimes etc
► Helps the team with known limitations, re-engineer and hardware recommendations
Test Results
► Different types of services across multiple systems seamlessly integrate and communicate with each other, under production env
► Overall transactions response time calculation
![Page 15: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/15.jpg)
SISE Approach – End-To-End Testing
What we do?
► Validation – Complete business cases/workflow
► Different types of business systems - Types of protocols include HTTP, HTTPS, SMTP, SOAP, XML-RPC, IRC etc.
► Mock services , Messages checkpoints & performance monitors deployed for appropriate usage
► Service change undergoes a complete end-to-end testing cycle
► Study of response time delays and key performance/response issues
Test Results
► Complete business workflow completes without any errors
► System satisfies the requirements
![Page 16: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/16.jpg)
Scelta – The Automation Framework
![Page 17: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/17.jpg)
Test Automation Framework
![Page 18: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/18.jpg)
Scelta – Input Data & Driver Components
Input Store
► XML driven Test Cases
► Business and technical users can create test cases
► Elements and schema guidelines for file creation
► Configuration file for implementation information
Workflow Manager
► Test execution - User choice (Business/Integration/security cases etc)
► Priority for test workflow/scenario/case
► Test execution – Environment configuration
Policy Manager
► Assign appropriate policy – Test cases
![Page 19: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/19.jpg)
Scelta – Service Validation Components
Service Validation
► Validate individual services/web-methods
► Assertions for different test-conditions
► Xpath query to fetch information
Response Manager
► Manages request and directs response
► Invokes mock services - For transactions with response delay time
► Message Routers – To deliver responses appropriately
Logs & Reports
► Centralized logs and reports collector
► Customized reports based on customer requirements
![Page 20: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/20.jpg)
Scelta – Performance Monitors & Metadata
Distributed Monitoring
► Service level monitors and agents
► Customized agent for collecting transactional information
Load Manager
► Workflow driven load testing
► Configurable load – Monitor performance of service
► Run periodically - Ensure Web service stability
Service Metadata
► Collects Historical information - Measure service reliability and quality
► Study service behavior (up/down time, time since last failure, service reliability history etc)
![Page 21: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/21.jpg)
Key Benefits – Approach & Framework
Multi-level validation–Defects fixes-Minimal effort & cost
Test Coverage - Different requirements validation
Minimal Defect leakage in production
Asynchronous test script execution
Service response facts , reliability and quality reports
Re-usable across ‘n’ # of test cycles
![Page 22: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/22.jpg)
Future Research
Support security related aspects
Finding services – Addressing testing pain-points
![Page 23: "SOA Testing : 4 Phased Methodology and Test Automation Framework"](https://reader036.vdocuments.us/reader036/viewer/2022062514/557d4defd8b42ae3298b4d49/html5/thumbnails/23.jpg)
Open Discussion