BIO PRESENTATION
International Conference On Software Testing Analysis and Review
May 15-19, 2006 Orlando, Florida USA
T12
Thursday, May 18, 2006 1:30PM
AUTOMATED SETUP AND TEAR
DOWN OF COMPLEX, MULTI-TIER
TEST CONFIGURATIONS
James Phillips Akimbi Systems
James Phillips
James Phillips brings more than 20 years of software industry experience to Akimbi Systems, which he co-founded in 2004. Prior to founding Akimbi, James was senior vice president of products and marketing at Actional Corporation (acquired by Progress Software) the pioneering leader in the service-oriented systems management software category. Previously, James was vice president of business development and product marketing at Ensim Corporation, where he successfully commercialized the company's technology assets. James has held leadership positions in software engineering, product management, business development and marketing management at Intel, Synopsys, Intuit, Central Point Software and Fifth Generation Systems. He is a popular speaker and writer, and has been featured in a number of industry conferences and publications. James holds a BS in mathematics and earned his MBA, with honors, from the University of Chicago.
T12
Automated Setup and Tear Down of Complex, Multi-tier Test ConfigurationsJames Phillips, Akimbi
Enterprise Software Architecture Evolution
Monolithic Client-server 3-tier, n-tier Service-oriented
Resulting Challenges for QA organizations
• Server Sprawl– 2-3 machines in application development and test for
every server in production (+ storage, networking …)– Server to Staff Ratios exceed 7:1 in some cases
• Setup and Provisioning Overhead– Repetitive system setup tasks overwhelming IT and
slowing software development cycles– Accounts for more than 50% of time expended in the
development and test cycle • Costly System Failures
– Difficult to reproduce and resolve software defects before production deployment
Gaps have opened in test automation solution continuum
design develop test deploy support
development
architecture
QA
data centeroperations
applicationsupport
IBM Rational Rose
Mercury TestDirector Mercury QTP
defects can’t be reproduced (interop and proximity)out-of-context unit testing
manual test system configuration
Visual StudioJUnit
Merant PVCS Serena TeamTrack
design develop test deploy support
A virtual lab automation system integrates with existing processes and solutions, filling the gaps
development
architecture
QA
data centeroperations
applicationsupport
IBM Rational Rose
Merant PVCS Serena TeamTrack
Mercury TestDirector
Visual StudioJUnit
Mercury QTP
VLA system VLA system
VLA system
Virtual Lab Automation – Shared Resource Pool, Automated Provisioning
Central QA Supportin Bangalore, India
Equity Capital Markets Trading System Developers
in New Jersey
Fixed Income AnalyticsSystem Developer
in New York
Akimbi SlingshotConfiguration Library(grouped machine images)
Servers are ProvisionedFrom Library
Shared pool of servers, storage and networking resources
Virtual Lab Automation – Architecture
Virtualized Server Pool
VLA Server Image Storage LibraryCIM Application Developer
CIM Application Developer
FP&A Application Developer
QA Automation Engineer
New Jersey Dev Facility
Walldorf Dev Facility
Bangalore Outsource Partner
VLA System Users
Vancouver Data Center
Self-service, instant “check out” from library
Automated Virtual Lab
Virtualized Server Pool
Akimbi Slingshot Server
1Image Storage Library
2
Application Developer
Walldorf Dev Facility
3
Effortless capture of configurations to the library
Automated Virtual Lab
Virtualized Server Pool
Akimbi Slingshot Server
1
Image Storage Library
2
Application Developer
Walldorf Dev Facility
3
Before Testing – QA creates and snapshots test case configurations
test server controlakimbi control
virtualtest servers
Test Lab in Palo AltoQA engineer
1. Create from scratch2. Clone existing snapshot3. Clone partial snapshot
QA engineer creates test case configurations:Servers, OS, Data, Application
2 Snapshots saved –single virtual test server or a bundle of servers for management as a unit.
1
Testing – Existing test tools provision test configurations using VLA Akimbi Snapshot
virtualtest servers
3 tier testconfiguration
reanimate test servers: server-OS-app
elapsed time for snapshot, clone and reanimation measured in seconds
Testing – Automated test executes as usual
virtualtest servers
3 tier testconfiguration
test inputs and results
Test success - Configuration is removed in preparation for next iteration
delete test servers
Test anomaly - Snapshot, create bug report and continue testing
virtualtest servers
1
2
anomaly detected
“create snapshot” command
3 snapshot taken and stored
4 problem report createdwith snapshot links
Debugging – Step 1: Developer receives bug report and clicks to restore environment
4
test server control
reanimatedvirtual
test servers
Test Lab
Test Lab in Palo AltoDeveloper in Austin
Bug report arrives toassigned developer
Developer clicks toreanimate testconfiguration –
before or after theanomaly was detected
2
1
Test server configurationreanimated
(following physical cues)
3
Debugging – Step 2: Developer debugs in exact environment in which bug occurred
test server controlakimbi control
remote connections
virtualtest servers
3 tier testconfiguration
Test Lab
Test Lab in Palo AltoDeveloper in Austin
Virtual Lab Automation - Benefits
Akimbi Slingshot Capability Customer Benefit
Pool and share server, networking, storage and other resources between software development, and test teams and individuals
Automatically and rapidly set up and tear down complex, multi-machine software configurations for use in development and test
Give every developer or test engineer the equivalent of their own fully-equipped data center with dedicated provisioning staff
Maintain a comprehensive library of customer and production system environments
Suspend and capture “ live” multi-machine configurations to a shared library
Efficiently move configurations between development and test facilities
• Reduce equipment-related capital and operating expenditures• Accelerate functional testing
• Shave man-weeks or months off every software development project• Increase the number of certified deployment configurations supported
by new software systems• Find and fix more bugs, earlier in the software development lifecycle
• Enable “ real world” unit testing• Slash time and energy wasted begging, stealing, re-configuring and
protecting hoarded servers, storage and networking equipment
• Eliminate upgrade breakage• Deliver better support users of a software system• Rapidly troubleshoot customer production problems
• Reproduce bugs reliably and reduce time spent in the debug phase• Reduce the number of latent software defects that slip into production
• Increase communication and productivity across distributed software development teams
• Enable confident and efficient outsourcing of software development and test activities
Leading organizations in deployment
Akimbi Systems1400 Fashion Island Blvd.Suite 900San Mateo, CA 94404(650) 887-2204 fax
James [email protected](650) 356-4401 direct(408) 832-3066 cell