testing centers of excellence - excelon development · by andrew soundranayagam & ashish usman...

40
VOLUME 7 ISSUE 2 FEBRUARY 2010 $8.95 www.stpcollaborative.com Testing Centers of Excellence CA is using its global test facilities to raise the bar on product quality and time to market. Cover Story 12 10 CRITICAL LESSONS LEARNED IN OUTSOURCED TESTING BEAUTIFUL TESTING SATISFIES STAKEHOLDERS CAREER ADVICE FROM THE EXPERTS

Upload: others

Post on 21-Jul-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

VOLUME 7 • ISSUE 2 • FEBRUARY 2010$8.95 • www.stpcollaborative.com

Testing Centers of Excellence

CA is using its global test facilities to raise the bar on product quality and time to market.

Cover Story 1210 CRITICAL LESSONS LEARNED IN OUTSOURCED TESTING

BEAUTIFUL TESTING SATISFIES STAKEHOLDERS

CAREER ADVICE FROM THE EXPERTS

Page 2: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite
Page 3: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite
Page 4: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

contentsDEPARTMENTS

6 Guest Editorial

7 Out of the Box

8 ST&Pedia

10 CEO Views

38 Case Study

VOLUME 7 • ISSUE 2 • FEBRUARY 2010

4

12 COVER STORY

Testing Centers of Excellence

By Harish Narayan

18 10 Key Lessons Learnedin Outsourced Testing

By Andrew Soundranayagam & Ashish Usman 22 Beautiful TestingSatisfies Stakeholders

By Rex Black

30 Career AdviceFrom the Experts

By Andrew Muns

Page 5: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

5

A Publication

GET MORE ONLINE AT

Software Test & Performance Magazine (ISSN- #1548-3460) is published monthly except combined issues in July/August and November/December by Redwood Collaborative Media, 105 Maxess Avenue, Suite 207, Melville, NY, 11747. Periodicals postage paid at Hunting ton, NY and additional mailing offices. The price of a one year subscription is US $69 for postal addresses within North America; $119 elsewhere. Software Test & Performance Magazine is a registered trademark of Redwood Collaborative Media. All contents copyrighted 2009 Redwood Collaborative Media. All rights reserved. POSTMASTER: Send changes of address to Software Test & Performance Magazine, 105 Maxess Road, Suite 207, Melville, NY 11747. To contact Software Test & Performance Magazine subscriber services, please send an e-mail to [email protected] or call 631-393-6051 ext. 200.

CONTRIBUTORS

REX BLACK

HARISH NARAYAN

ANDREW SOUNDRANAYAGAM

ASHISH USMAN

INTERVIEW

FORUM

NEWSLETTER

STP BLOGS

RESOURCES

stpcollaborative.com/knowledge/interviews.

stpcollaborative.com/testingcenters-forum.

stpcollaborative.wufoo.com/forms/ibm-presents-software-quality-optimization/

stpcollaborative.com/quality-software.

blogs.stpcollaborative.com/matt/.

stpcollaborative.com/resources.

WHITEPAPER

Page 6: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

6

GUEST EDITORIALVOLUME 7 • ISSUE 2 • FEBRUARY 2010

Editor Amy Lipton [email protected]

Contributing Editors Joel Shore Matt Heusser Chris McMahon

Copy Editor Michele Pepe [email protected]

Creative Director Scott [email protected]

Publisher Andrew [email protected]

Media Sales Joe [email protected]

Chief Marketing Officer Jennifer [email protected]

Marketing Coordinator Teresa [email protected]

Training & Professional Development Abbie [email protected]

Social Media Director Rick [email protected]

Founder & CEO Andrew Muns

Chairman & President Peggy Libbey

724 South Tejon Street Suite C Colorado Springs, CO 80903 1.877.257.9531 1.719.472.0257 fax

www.stpcollaborative.com

Tough Choices

Page 7: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

7

OUT OF THE BOXConcerity Announces Beta Launch of Concerity Analytics

uTest Rolls Out Updated Platform

Systems In Motion, Zephyr Team Up

Ascert Launches Driver Development Program

Automation Anywhere Aims to Test Anywhere

Free dynaTrace AJAX Tool Seeks Out Client-side Application Problems

Compuware Delivers First Phase of Vantage/ Gomez Integration

Coverity 5 Asks Which Defect Is the Worst

Page 8: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

8

Sharing Corporate Goals and Strategy

Messaging

Voice Over IP (VOIP) Conferencing

Wikis

ST & PEDIA

Distributed Teamwork Demands Open Communication Channels

Page 9: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

9

Shared Development and Test Machines

Video

E-Mail

VPN

Culture A naive way to attempt to bring

about collaboration on a large or distributed team is to make a lot of communication tools available and hope for the best.

Page 10: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

10

CEO VIEWS

The Potential of Load Testing in the Cloud

Page 11: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

11

Page 12: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

12

Harish Narayan is a senior quality architect at CA, focused on corporate-wide quality improvement and process efficiency initiatives. He is a software engineering and information technology leader and has worked in the field since 1995. He is a transformative change agent and has experience building world-class quality functions and championing enterprise-wide process and capability improvements. Narayan has been a QA and test manager and architect, and has built and been part of testing centers of excellence in multiple companies. He also brings years of strategic planning, project management, requirements management, performance management and team-building skills and experience.

CA is using its global test facilities to raise the bar on product quality and time to market. Here’s a behind-the-scenes report.

INDIA TECHNOLOGY CENTER TEST EXECUTION COE

Testing Centers of Excellence By Harish Narayan

Page 13: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

13

INDIA

CHINA

PRAGUE

Page 14: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

14

Case Study 1

Case Study 2

Page 15: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

15

CHINA TECHNOLOGY CENTER TEST EXECUTION COE

Case Study 1

The China COE’s objective now is

to balance team growth and efficiency without compromising quality.

Page 16: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

16

PRAGUE TECHNOLOGY CENTER FUNCTIONAL SERVICE COE (AUTOMATION)

Case Study 1

Case Study 2

Page 17: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

17

BOTTOM LINE

IS YOUR TESTING CENTER ‘EXCELLENT’?

Contributors include:

Ivan Chang – VP, Quality Assurance (CTC QA)

Milan Svatek – Sr. Director, Software Engineering (PTC)

Alex Kingham – Manager, Software Engineering (PTC)

Denis Tronin – Sr. Team Lead, Software Engineering (PTC)

Srdjan Nalis – Sr. Team Lead, Software Engineering (PTC)

Sankar Subramaniam – Director, Quality Assurance (ITC QA)

Sivakumar Ganesamurthy – Director, Quality Assurance (ITC QA)

Silvia Ahmed – VP, Quality Assurance (Wily QA)

Navid Rezvani – Sr. Director, Quality Assurance (Wily QA)

Index to Advertisers URL

Replay Solutions ................................ www.replaysolutions.com pg. 2

Soasta .............................................................www.Soasta.com pg. 3

STP Collaborative ...............www.STPCollaborative.com/Events pg. 28

STP Collaborative ....... www.STPCollaborative.com/Membership pg. 29

NeoLoad .........................................................www.neotys.com pg. 37

Web Performance ....................... www.webperformanceinc.com pg. 39

Hewlett-Packard ......................................www.hp.com/go/alm pg. 40

Page 18: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

18

By Andrew Soundranayagam and Ashish Usman

Page 19: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

19

This article is based on the authors’ 25 years of collective experience in the field of outsourced IT services. While it is not an exhaustive list, these 10 lessons represent some of the major pain points we have encountered over the years. Taking them into consideration is critical to a successful engagement.

The lessons below are not listed in any order of priority—rather, each requires attention by both the client and the service provider. A major theme running through all these lessons is to be prepared, and to do so through early and frequent communication. We believe that being aware of potential issues in advance is critical to allowing service providers to ask the right questions and manage client expectations, and to ensure the best possible outcomes.

Agile is a very high-level term that covers a wider spectrum of variants such as Scrum, AUP and XP.

Page 20: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

20

Page 21: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

21

CONCLUSIONS

Page 22: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

22

Page 23: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

23

For Whom Do We Test?

Page 24: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

24

What Satisfies?

Page 25: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

25

What Beauty Is External?

Page 26: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

26

What Beauty Is Internal?

Page 27: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

27

Conclusions

Page 28: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

Software Test& PerformancePROFESSIONAL DEVELOPMENT

STP Public TrainingMarriott San Mateo San Mateo, California

April 19 – 23, 2010

introduces

Join the STP Collaborative Community to receive special members-only discounts on all STP Professional Development o!erings!

Learn more at www.regonline.com/STPTrainingSpring

Get in-depth professional education from industry experts! STP Public Classroom Training provides a wealth of benefits, including:

Experienced instructors

A structured environment focused on skills building

Hands-on exercises and real-world examples

The opportunity to gain peer-to-peer knowledge by networkingwith other software testing professionals

Save the date & stay tuned for more details!

Learn Refine Advance

Page 29: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite
Page 30: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

James Bach

Career Advice From the Experts

In every art beginners must start with modelsof who have practiced the same art before them.

And it is not only a matter of looking at what has been and is being created. It is a matter of being drawn into the individual work of art, of realizing that it has been made by a human being and trying to discover the secret of its creation.

30

Page 31: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

31

Vladimir Belorusets

Page 32: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

32

Rex Black

Page 33: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

33

Ross Collard

Jan Fish

Page 34: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

Linda Hayes

I wish I had known that just because wedidn’t mind writing code to automate our

testing did not mean it was a good idea. We did not find out until much later that most of our customers had no skills or interest in writing code.

34

Page 35: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

35

Elisabeth Hendrickson

Page 36: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

36

Cem Kaner

Page 37: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

I think there is a lot more that programmerswithin a test group can do to enhance the

productivity and capability of the test group.

37

Page 38: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

38

CASE STUDY

Time to Get Serious About Agile

Page 39: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite

39

Page 40: Testing Centers of Excellence - Excelon Development · By Andrew Soundranayagam & Ashish Usman 22 Satisfies Stakeholders / v Z ] Æ ( } u Z } } l ^ µ ] ( µ o d ] v P U _ ... Suite