testing centers of excellence - excelon development · by andrew soundranayagam & ashish usman...

VOLUME 7 • ISSUE 2 • FEBRUARY 2010$8.95 • www.stpcollaborative.com

Testing Centers of Excellence

CA is using its global test facilities to raise the bar on product quality and time to market.

Cover Story 1210 CRITICAL LESSONS LEARNED IN OUTSOURCED TESTING

BEAUTIFUL TESTING SATISFIES STAKEHOLDERS

CAREER ADVICE FROM THE EXPERTS

contentsDEPARTMENTS

6 Guest Editorial

7 Out of the Box

8 ST&Pedia

10 CEO Views

38 Case Study

VOLUME 7 • ISSUE 2 • FEBRUARY 2010

4

12 COVER STORY

Testing Centers of Excellence

By Harish Narayan

18 10 Key Lessons Learnedin Outsourced Testing

By Andrew Soundranayagam & Ashish Usman 22 Beautiful TestingSatisfies Stakeholders

By Rex Black

30 Career AdviceFrom the Experts

By Andrew Muns

5

A Publication

GET MORE ONLINE AT

Software Test & Performance Magazine (ISSN- #1548-3460) is published monthly except combined issues in July/August and November/December by Redwood Collaborative Media, 105 Maxess Avenue, Suite 207, Melville, NY, 11747. Periodicals postage paid at Hunting ton, NY and additional mailing offices. The price of a one year subscription is US $69 for postal addresses within North America; $119 elsewhere. Software Test & Performance Magazine is a registered trademark of Redwood Collaborative Media. All contents copyrighted 2009 Redwood Collaborative Media. All rights reserved. POSTMASTER: Send changes of address to Software Test & Performance Magazine, 105 Maxess Road, Suite 207, Melville, NY 11747. To contact Software Test & Performance Magazine subscriber services, please send an e-mail to membership-services@stpcollaborative.com or call 631-393-6051 ext. 200.

CONTRIBUTORS

REX BLACK

HARISH NARAYAN

ANDREW SOUNDRANAYAGAM

ASHISH USMAN

INTERVIEW

FORUM

NEWSLETTER

STP BLOGS

RESOURCES

stpcollaborative.com/knowledge/interviews.

stpcollaborative.com/testingcenters-forum.

stpcollaborative.wufoo.com/forms/ibm-presents-software-quality-optimization/

stpcollaborative.com/quality-software.

blogs.stpcollaborative.com/matt/.

stpcollaborative.com/resources.

WHITEPAPER

6

GUEST EDITORIALVOLUME 7 • ISSUE 2 • FEBRUARY 2010

Editor Amy Lipton alipton@stpcollaborative.com

Contributing Editors Joel Shore Matt Heusser Chris McMahon

Copy Editor Michele Pepe mpepe@stpcollaborative.com

Creative Director Scott Hansonshanson@redwoodcollaborative.com

Publisher Andrew Munsamuns@stpcollaborative.com

Media Sales Joe Altierijaltieri@redwoodcollaborative.com

Chief Marketing Officer Jennifer McClurejmcclure@redwoodcollaborative.com

Marketing Coordinator Teresa Cantwelltcantwell@redwoodcollaborative.com

Training & Professional Development Abbie Caracostasacaracostas@redwoodcollaborative.com

Social Media Director Rick Baucomrbaucom@redwoodcollaborative.com

Founder & CEO Andrew Muns

Chairman & President Peggy Libbey

724 South Tejon Street Suite C Colorado Springs, CO 80903 1.877.257.9531 1.719.472.0257 fax

www.stpcollaborative.com

Tough Choices

7

OUT OF THE BOXConcerity Announces Beta Launch of Concerity Analytics

uTest Rolls Out Updated Platform

Systems In Motion, Zephyr Team Up

Ascert Launches Driver Development Program

Automation Anywhere Aims to Test Anywhere

Free dynaTrace AJAX Tool Seeks Out Client-side Application Problems

Compuware Delivers First Phase of Vantage/ Gomez Integration

Coverity 5 Asks Which Defect Is the Worst

8

Sharing Corporate Goals and Strategy

Messaging

Voice Over IP (VOIP) Conferencing

Wikis

ST & PEDIA

Distributed Teamwork Demands Open Communication Channels

9

Shared Development and Test Machines

Video

E-Mail

VPN

Culture A naive way to attempt to bring

about collaboration on a large or distributed team is to make a lot of communication tools available and hope for the best.

10

CEO VIEWS

The Potential of Load Testing in the Cloud

11

12

Harish Narayan is a senior quality architect at CA, focused on corporate-wide quality improvement and process efficiency initiatives. He is a software engineering and information technology leader and has worked in the field since 1995. He is a transformative change agent and has experience building world-class quality functions and championing enterprise-wide process and capability improvements. Narayan has been a QA and test manager and architect, and has built and been part of testing centers of excellence in multiple companies. He also brings years of strategic planning, project management, requirements management, performance management and team-building skills and experience.

CA is using its global test facilities to raise the bar on product quality and time to market. Here’s a behind-the-scenes report.

INDIA TECHNOLOGY CENTER TEST EXECUTION COE

Testing Centers of Excellence By Harish Narayan

13

INDIA

CHINA

PRAGUE

14

Case Study 1

Case Study 2

15

CHINA TECHNOLOGY CENTER TEST EXECUTION COE

Case Study 1

The China COE’s objective now is

to balance team growth and efficiency without compromising quality.

16

PRAGUE TECHNOLOGY CENTER FUNCTIONAL SERVICE COE (AUTOMATION)

Case Study 1

Case Study 2

17

BOTTOM LINE

IS YOUR TESTING CENTER ‘EXCELLENT’?

Contributors include:

Ivan Chang – VP, Quality Assurance (CTC QA)

Milan Svatek – Sr. Director, Software Engineering (PTC)

Alex Kingham – Manager, Software Engineering (PTC)

Denis Tronin – Sr. Team Lead, Software Engineering (PTC)

Srdjan Nalis – Sr. Team Lead, Software Engineering (PTC)

Sankar Subramaniam – Director, Quality Assurance (ITC QA)

Sivakumar Ganesamurthy – Director, Quality Assurance (ITC QA)

Silvia Ahmed – VP, Quality Assurance (Wily QA)

Navid Rezvani – Sr. Director, Quality Assurance (Wily QA)

Index to Advertisers URL

Replay Solutions ................................ www.replaysolutions.com pg. 2

Soasta .............................................................www.Soasta.com pg. 3

STP Collaborative ...............www.STPCollaborative.com/Events pg. 28

STP Collaborative ....... www.STPCollaborative.com/Membership pg. 29

NeoLoad .........................................................www.neotys.com pg. 37

Web Performance ....................... www.webperformanceinc.com pg. 39

Hewlett-Packard ......................................www.hp.com/go/alm pg. 40

18

By Andrew Soundranayagam and Ashish Usman

19

This article is based on the authors’ 25 years of collective experience in the field of outsourced IT services. While it is not an exhaustive list, these 10 lessons represent some of the major pain points we have encountered over the years. Taking them into consideration is critical to a successful engagement.

The lessons below are not listed in any order of priority—rather, each requires attention by both the client and the service provider. A major theme running through all these lessons is to be prepared, and to do so through early and frequent communication. We believe that being aware of potential issues in advance is critical to allowing service providers to ask the right questions and manage client expectations, and to ensure the best possible outcomes.

Agile is a very high-level term that covers a wider spectrum of variants such as Scrum, AUP and XP.

20

21

CONCLUSIONS

22

23

For Whom Do We Test?

24

What Satisfies?

25

What Beauty Is External?

26

What Beauty Is Internal?

27

Conclusions

Software Test& PerformancePROFESSIONAL DEVELOPMENT

STP Public TrainingMarriott San Mateo San Mateo, California

April 19 – 23, 2010

introduces

Join the STP Collaborative Community to receive special members-only discounts on all STP Professional Development o!erings!

Learn more at www.regonline.com/STPTrainingSpring

Get in-depth professional education from industry experts! STP Public Classroom Training provides a wealth of benefits, including:

Experienced instructors

A structured environment focused on skills building

Hands-on exercises and real-world examples

The opportunity to gain peer-to-peer knowledge by networkingwith other software testing professionals

Save the date & stay tuned for more details!

Learn Refine Advance

James Bach

Career Advice From the Experts

In every art beginners must start with modelsof who have practiced the same art before them.

And it is not only a matter of looking at what has been and is being created. It is a matter of being drawn into the individual work of art, of realizing that it has been made by a human being and trying to discover the secret of its creation.

30

31

Vladimir Belorusets

32

Rex Black

33

Ross Collard

Jan Fish

Linda Hayes

I wish I had known that just because wedidn’t mind writing code to automate our

testing did not mean it was a good idea. We did not find out until much later that most of our customers had no skills or interest in writing code.

34

35

Elisabeth Hendrickson

36

Cem Kaner

I think there is a lot more that programmerswithin a test group can do to enhance the

productivity and capability of the test group.

37

38

CASE STUDY

Time to Get Serious About Agile

39