test magazine - january 2015

INNOVATION FOR SOFTWARE QUALITY

VOLUME 6: ISSUE 6JANUARY 2015

www.testmagazine.co.uk

A SPY IN THE HOUSE

THE EUROPEAN SOFTWARE TESTER

TESTA 2014 A GLITTERING TRIUMPH

INSIDE:

HAS SOMEONE HIJACKED YOUR WEBCAM?

BIG DATA MANAGING THE DATA DELUGE

BREAKING DOWN BARRIERSCOLLABORATIVE WORKING WITH DevOps

JANUARY 2015 | www.testmagazine.co.uk PAGE 3

CONTENTS

THOUGHT LEADERSHIP 8 It Ain’t What You Do, It’s The Way That

You Do It Siva Ganesan, Vice President and Global Head

of Assurance Services at Tata Consultancy Services, argues that, in the era of DevOps, QA and testing need to have a continuous presence and influence throughout the software lifecycle, wearing multiple hats, and supporting all the phases.

NEWS 9 TESTA 2014

A look back at TESTA on 18th November, where the software industry got together to recognise, celebrate, and reward achievement.

16 ADVERTISING AND SALE OF MOBILE SPYWARE: FIRST CONVICTION

Danish citizen becomes first person to receive a criminal conviction in connection with the advertisement and sale of a mobile device spyware app.

18 RAYTHEON’S US NAVY SOFTWARE PROGRESS

Software development company Raytheon, which is contracted to build the software for the US Navy’s new Air and Missile Defense Radar (AMDR), says its method of developing and building software has been validated by recent results.

COVER STORY 22 A Spy in the House David Land talks to David Bryan, Principal

Security Consultant at Trustwave, about revelations of Russian hackers spying on webcams in British bedrooms and offices, and the importance of building security testing into the production process, prior to the product hitting the market.

INSIDE THIS ISSUE

DEVELOPMENT ISSUES 24 The Mainframe Gets Agile Steven Murray, Mainframe Solutions Director at

Compuware, discusses the crucial importance of bridging the cultural divide between web and distributed application development on the one hand, and mainstream development on the other.

SUPPLIER PROFILES 26 Looking to the Future One of the top firms worldwide in IT, consulting

and business solutions, Tata Consultancy Services took Leading Vendor at TESTA this year for its quality assurance and testing services. David Land, 2015 Chair of TESTA Judges, catches up with Prabhakar Karamsetty, Head of TCS’ Assurance Services Unit, UK & Europe, to discuss opportunities and challenges in the quality assurance space, and to ask him to predict the industry’s future with his crystal ball ...

30 Expanded Integrations PractiTest’s Marketing Manager, Ma’ayan

Leeper-Carr, talks to TEST Magazine about the testing software’s origin and evolution, and how it adapts to changing demands.

24.THE MAINFRAME

GETS AGILE

9. TESTA 2014

22.A SPY IN THE

HOUSE

In today’s vast and unpredictable technology landscape, does your QA and Testing function give you the con� dence of fail-safe systems and zero-risk performance? There exists a way: Tata Consultancy Services (TCS). With TCS’ independent enterprise testing arm, Assurance Services Unit (ASU), you can provide the certainty of risk-free systems to your customers, with market-proven, world-class experience, expertise and guidance. Visit tcs.com/assurance and you’re certain to learn more. Or write to us at: [email protected]

IT ServicesBusiness SolutionsConsulting

Scan the code to know about TCS Assurance Services

Assurance is the certainty of a risk-free experience. With confidence.


CONTENTS

IT INNOVATION 32 Breaking Down Barriers The concept of DevOps is to develop

collaborative working relationships between development and operations – areas which are traditionally non-aligned – and foster adoption of a common set of objectives, to deliver IT services that provide strategic value to the business, as Nigel Wilson, Head of Service Delivery at BJSS, explains.

TECHNOLOGICAL DEVELOPMENTS 34 Managing the Data Deluge

David Land talks to Iain Chidgey, General Manager of EMEA at Delphix, about the exponential rise in the quantity of data, both now and – with the coming Internet of Things – on the horizon.

THOUGHT LEADERSHIP 36 Peak Performance

With big online firms suffering losses during periods of peak demand, Archie Roboostoff, Product Director at Micro Focus, looks at the wisdom of utilising the cloud to undertake website testing, and thus avoiding peak period website casualties.

SUPPLIER PROFILE 38 Performance Matters Thibaud Bussière, CEO and co-founder of

Neotys, tells David Land about how the software company has evolved, and its current place in the market.

INSIDE THIS ISSUE

40.LOOKING AHEAD …

AND BACK!

LOOKING AHEAD … AND BACK! 40 Testing Times Sophie-Marie Odum, previous Editor of TEST

Magazine, reflects on her time within the software testing industry, and talks to industry leaders about their predictions for the next 12-18 months.

VIEWPOINT 45 Added Value Global Head of Testing Practice at ITC Infotech

Vijay Balasubramaniam discusses how testing is moving beyond the traditional time and materials based model, to a more outcome based perception of value.

LAST WORD 46 Documentary Evidence Despite his general adherence to the tenets of

the agile movement, with its belief that testing should be documentation free, Dave Whalen prefers documentation lite to documentation free. Here, he tells us why.

34.MANAGING THE

DATA DELUGE

WANTED: Developers who understand infrastructure. Automation-focused infrastructure engineers and

operations staff. Experienced and non-functional testers.

High-end technical delivery managers and architects with a solid background in IT development and operations.

32. BREAKING

DOWN BARRIERS

www.cognizant.com

The powerful solution to your testing challenges.

ADPART,thevisualmodelingtoolthatautomatesthecompletetestdesignprocess.Muchfasterthanmanualtestdesign,youcanavoidthemazeofwrittendocuments,accommodatefrequentchangesinrequirements,identifytheappropriatetestcasesanddetectdefectsearlier.Savingyouandyourorganizationtimeandmoney.What’smore,it’sfromCognizant,theworld’sleadingQualityEngineering&Assurancepractice.

— Model and embed business flows at any level

— Automated test scenario

— Automated test case generation, prioritisation and optimisation

— Automated requirement traceability and regression analysis

— Automated impact analysis for defects and change in requirements

— Rule-based test case generation

FindouthowADPARTcouldmaketestingquickerandsimplerforyou:

WWW.SEEING-THINGS-DIFFERENTLY.COM

faster

smarter

easier

TESTING

Cognizant _Adpart_FP_A4 v2.indd 1 29/09/2014 11:32


COLLABORATION IS KEYHello, and welcome to the January 2015 issue of TEST Magazine.

n all its forms, it seems, collaboration within software testing is an idea whose time has come. And nowhere could this be better exemplified than at TESTA, which took place on 18th November. Software testing’s annual

coming together to acknowledge, celebrate and reward achievement, it had a real buzz this year, a sense that the industry was moving forward as one, with a collaborative approach leading to greater achievements.

Mobile, DevOps, agile, migration to the cloud, the Internet of Things, big data … a whole lot of interrelated concepts pepper the pages of this issue. But what they share in common is collaboration.

Says Nigel Wilson, Head of Service Delivery at BJSS, in Breaking Down Barriers, “DevOps aims to break down the barriers and conflicting priorities which often exist between development and operations teams, such as project spend, application performance and functional requirements, thus allowing them to work together, to deliver systems into production reliably, safely and rapidly, and to operate and support them more effectively.”

In The Mainframe Gets Agile, Steven Murray, Mainframe Solutions Director at Compuware, argues for the crucial importance of bridging the cultural divide between web and distributed application development on the one hand, and mainstream development on the other.

The mainframe, which has sat at the heart of the business for an incredible 50 years, is now dealing with levels of workload that couldn’t even have been remotely envisaged when it was conceived.

Steven argues that, while agile development and the DevOps ethos are gaining traction in the world of web and distributed application development, having been a stable part of the IT environment for half a century, mainframe has been slow to embrace agile. If they are to safeguard the future of their IT services, IT leaders must encourage mainframe development teams to adopt the same modern ways of working as those their counterparts on distributed application teams have adopted.

In Added Value, Global Head of Testing Practice at ITC Infotech Vijay Balasubramaniam discusses how testing is moving beyond the traditional time and materials based model, to a more outcome based perception of value.

He says, “The concept of testing as a service is shifting from a predominantly ‘time and materials’ type of customer engagement, to a more ownership based, results oriented business model where, from a quality assurance perspective, what the customer expects at the end of a project is that the IT system which has been built and tested satisfies the business need for which it was specified.

“The number of test cases executed and bugs found is immaterial: what defines value is that, at the point when the system is built and has the go ahead, there are absolutely no problems, glitches or issues that impact on its deployment and smooth operation.

“The value of this new approach comes from the higher quality of the systems delivered. Typically, they have fewer defects and less downtime, meaning software that is truly fit for purpose, which integrates seamlessly with the business."

While collaboration is great of course, the downside of all this connectivity is the increased potential for breaches in data security. Following the notorious instances recently of Russian webcam hackers posting footage online of British gyms and bedrooms, in our cover story, A Spy in the House, we speak to David Bryan, Principal Security Consultant at Trustwave, about the importance of building security testing into the production process, prior to the product hitting the market.

Do you want

to write for TEST Magazine?

Email david.land@31media.

co.uk

I

LEADER

© 2015 31 Media Limited. All rights reserved.

TEST Magazine is edited, designed, and published by 31 Media Limited. No part of TEST Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher. A reprint service is available.

Opinions expressed in this journal do not necessarily reflect those of the editor or TEST Magazine or its publisher, 31 Media Limited.

ISSN 2040-01-60

EDITORDavid Land [email protected] Tel: +44 (0)203 056 4599

TO ADVERTISE CONTACT:Sarah Walsh [email protected] Tel: +44(0)203 668 6945

PRODUCTION & DESIGNTina Harris [email protected]

EDITORIAL & ADVERTISING ENQUIRIES31 Media Ltd, 41-42 Daisy Business Park, 19-35 Sylvan Grove, London, SE15 1PDTel: +44 (0) 870 863 6930

Email: [email protected] Web: www.testmagazine.co.uk

PRINTED BYPensord, Tram Road, Pontllanfraith, Blackwood, NP12 2YA

THIRTYONE

David Land Editor

JANUARY 2015 | www.testmagazine.co.ukPAGE 8

ow is success defined in today’s world of QA and testing? You might think that the answer was obvious: that it’s the delivery of secure, reliable software, which performs the task for which it was constructed as desired, and that

the quality, cost, delivery equation is the ultimate measure of success.

While quality, cost and delivery remain prime considerations however, in the fast paced digital world of today, there are other factors which have assumed greater significance. Nowadays, success can perhaps be defined as delivery of software that is done right first time, every time, irrespective of any changes in context, business, or technology. Achieving this means deploying an agile and lean approach, which is able to cope with high frequency releases, short development and testing cycles, and rapid speed to market. To enable success at this level, QA needs to be the common thread in DevOps (collaborative working relationships between development and operations).

The concept of DevOps has grown in prominence rapidly of late, in parallel with the adoption of cloud services: one of the Five Forces (as defined by TCS) behind the digital era: an age which demands we provide services continuously, anytime, anywhere and on any device.

DevOps brings together freethinking developers with process oriented operations professionals, to deliver accelerated release schedules with an efficiency and effectiveness that could only have been dreamed of back in the days when development and operations departments were hunkered down in their own silos. QA teams need to echo this collaborative and integrated approach, and apply it to the way they work with all functions in the software lifecycle.

It all starts with definition. Right at the beginning, when the roadmap is being set, there needs to be the correct level of specificity and granularity of business and technology requirements, validated by subject matter experts. Although QA teams aren’t usually expected to craft

the roadmap documents, they do need to ask the right questions, in a collaborative manner, to ensure that the job is done properly to minimise risk.

Moving on to build and assembly, the QA team needs to play the role of friendly critic, ensuring that there is the right level of detail in the assurance strategy, that test plans which factor in any co-dependencies are in place, using the correct test data and the right types of testing in the right environments. This isn’t about auditing; it’s about working in a collaborative partnership, with the common

goal of achieving excellence for the end user.

In production, usually after User Acceptance Testing has been completed, the QA teams

need to certify the go/no-go decision, and to be sure that, once the last set of fixes has been applied, the last set of results are not invalidated.

For mainstream production support, when new features or functionality are added, QA teams have to ensure that the right

level of impact analysis and change analysis has been carried out, and that there is an

effective regression test strategy in place, so that any change doesn’t destabilise something

that was working earlier.

At all of these stages, the level of automation and instrumentation necessary will vary according to the context. In some cases, to get the desired outputs at maximum speed and minimum cost and effort, it may be cost effective to apply customised tooling.

It’s no longer just about what QA does; it’s also about how QA does it. QA professionals need to be enablers, not blockers; advisers, not just testers; and they have to have a line of sight that cuts across the entire software lifecycle. Their aim is to help others succeed, for the benefit of the end customer, and ultimately the business.

IT AIN’T WHAT YOU DO, IT’S THE WAY THAT YOU DO ITFriend, philosopher, critic, guide: in the era of DevOps, QA and testing need to have a continuous presence and influence throughout the software lifecycle, wearing multiple hats, and supporting all the phases. Siva Ganesan, Vice President and Global Head of Assurance Services at Tata Consultancy Services, explains

H

NOWADAYS, SUCCESS CAN

PERHAPS BE DEFINED AS DELIVERY OF SOFTWARE

THAT IS DONE RIGHT FIRST TIME, EVERY TIME, IRRESPECTIVE OF ANY

CHANGES IN CONTEXT, BUSINESS, OR

TECHNOLOGY

THOUGHT LEADERSHIP

SIVA GANESAN VICE PRESIDENT AND GLOBAL HEAD,

ASSURANCE SERVICES, TATA CONSULTANCY SERVICES (TCS)


TESTA 2014

The European Software Testing Awards – better known to its friends as TESTA – saw over 500 top international professionals from software testing and IT gathered in the splendid halls of Old Billingsgate in the City of London, to celebrate the greatest achievements in the industry over the past 12 months.

Hosted by Hal Cruttenden, who has appeared on shows like Mock the Week, Have I Got News for You, and Live at the Apollo, it was a marvellous occasion. The turnout was fantastic, and the level of enthusiasm palpable.

A gourmet three course meal was served, while the wine flowed, and with it, the conversation. Connections were made, and ideas shared, as the industry came together as one.

The many great entries that TESTA attracted this year – with a 45% uplift year on year – demonstrates the high calibre of work that goes on within our industry, and meant that the judges had a tough job choosing only one winner in each category.

The TESTA judging panel is independently selected, and made up of highly experienced individuals from a range of disciplines within software testing and IT, and we would like to take this opportunity to offer them our thanks for their hard work, dedication and commitment. A massive thank you is also due to our headline sponsor, Borland Software, for its continued support. We would also like to thank all of our other sponsors and supporters, without whom TESTA would simply not be possible.

Speaking at TESTA, Grant Farrell, Managing Director of 31 Media said, “I’m frankly awed by the level of talent that TESTA attracts. Our purpose is to serve and unite the industry, and it gives me enormous personal satisfaction to see it all coming together. TESTA is surely a prime example of the truism that the whole is greater than the sum of its parts. Impressive though the individual components are that make it what it is, when they come together, something truly magical happens. I feel privileged to have witnessed that this evening.”

A prize draw in aid of Alzheimer’s Society was held during TESTA 2014, with a prize of a 128GB iPhone 6 Plus. A cheque for the amount collected – a fantastic £2560 – was presented by Grant Farrell to Meghan Booth and Soifra Petty from the Community Fundraising Team at Alzheimer’s Society, while the prize draw was won by SEGA’s Jim Woods.

David Land, Chair of the 2015 TESTA judging panel

CELEBRATING TECHNICAL EXCELLENCE

THE EUROPEAN SOFTWARE TESTING AWARDS

CELEBRATING TECHNICAL EXCELLENCE

THE EUROPEAN SOFTWARE TESTING AWARDS

Headline Sponsor

2014 The 2014 Winners:

The Cigniti Technologies Best Agile ProjectEPAM Systems

The Neotys Best Mobile ProjectWaitrose in partnership with Cognizant Technology Solutions

Best Test Automation ProjectTIBCO Jaspersoft

The Sogeti Green Testing Team of the YearTech Mahindra

Graduate Tester of the YearKieran Hunter, Cognizant Technology Solutions

Testing Manager of the YearPankaj Kumar, Tech Mahindra

Testing Innovator of the YearNeal Hardwick, Tech Mahindra

Best Overall Testing Project – Public SectorHome Office Technology – Test Design & Consultancy Services

Best Overall Testing Project – Finance SectorBarclays

Best Overall Testing Project – Retail SectorJohn Lewis in partnership with Cognizant Technology SolutionsCigniti Technologies (highly commended)

Best Overall Testing Project – Gaming SectorAditi Technologies

Best Use of ToolsLloyds Banking Group in partnership with Cognizant Technology Solutions

The Zephyr Testing Team of the YearAVEA and Ericsson

Testing Management Team of the YearThe IFDS Testing Management Team

Test Champion of the YearSamir Sinha, Tech Mahindra

The Maveric Systems Best Overall ProjectAditi Technologies

Best Overall Use of TechnologyTech Mahindra

The Sage Most Innovative ProjectProxama

King (highly commended)

Leading VendorTata Consultancy Services (TCS)

The Borland European Software Testing AwardHome Office Technology – Test Design & Consultancy Services

Lifetime Achievement AwardBob Bartlett

SponsorsHeadline Sponsor

Supported by

THE EUROPEAN SOFTWARE TESTER

MA

GA

ZIN

E


TESTA 2014

I’m constantly surprised at the innovation and creativity on show at TESTA. As the testing landscape gets more challenging, the entrants are having to raise their game every year –

and somehow they always manage it.

“TESTA is where innovation meets the testing challenge. The categories are diverse, and represent a great opportunity to see what’s happening in the world of testing. The teams,

people and projects showcase everything new in the world of testing.

“TESTA is a great opportunity to pick up best practice tips, meet people who could help improve what you do, and take your business forward.

Mark Conway, Director of Development, Borland

Software is ubiquitous in today’s world, and it plays a major role in both our work and personal lives. Given the pervasiveness of software and how dependent many of us are on it, it

is essential that those who can influence the quality of software keep their knowledge and skills up to date. Test Magazine and the National Software Testing Conference are great ways to learn about new and improved approaches to the field of software quality, while TESTA is a

fantastic way to recognise and reward the work being done in the industry.

James Murphy, Head of QA at Guardian News and Media and TESTA 2014 Judge

It’s great that TESTA rewards organisations and individuals who have shown a special capability in testing – people who aren’t usually in the spotlight.

“TESTA is a fantastic place and time to thank our partners and customers with a great evening, and an opportunity to demonstrate how Neotys contributes to the industry – the best Mobile

Project Award, which Neotys sponsored, provided us with great visibility.

Sylvain Fambon, Marketing Director, Neotys


TESTA 2014

We had a great time at TESTA 2014, and were proud to be finalists in four categories. We invited clients to join us on the night, and it was great to share the experience with them.

“TESTA is great, as it both recognises key players in the speclialist testing industry, rather than in the IT industry as a whole, and encourages them to strive for excellence and aim to be the

best, thus improving the skills level within our industry as a whole.

Samantha Mills, UK Marketing Manager, Sogeti

We were very happy to meet other testing professionals and exchange ideas and vision. It was exciting to see the strength of the testing world. As we invest heavily in innovative testing solutions, such as the Amdocs BEAT framework, TESTA was an opportunity to share some of it

with the testing community, and to understand its importance to our customers.

"TESTA is an important platform, which helps us engage with customers, to both understand their needs and plans, and to expand the testing value for our customers.

“Being part of a community of senior testing professionals is important for exchanging ideas and best practices. We were delighted to be a part of TESTA 2014, and hope to see even more

input from the telecommunications industry at TESTA 2015.

Maya Atlas-Rosen, Amdocs’ Testing Regional Lead EMEA and APAC

For more pictures visit www.softwaretestingawards.com


TEST Magazine: Congratulations on your success at TESTA – such an amazing achievement! How does it feel?

Tech Mahindra: Really great, thanks! We’re so proud of our five awards. They build upon the huge success and achievements of 2013, in which our Global Test Practice won two TESTA awards, and also achieved a global first distinction of TMMi level 5 certification at the first attempt. As TESTA is judged and awarded by peers in the software testing industry, it makes it all the more gratifying and relevant – and this year we had success in such a wide variety of categories. By the time of our fifth award, Hal Cruttenden –the evening’s host – was joking about seeing us again!

TEST: What impact has your winning had on staff morale?

TM: These industry accolades affect the morale of not only our testing staff, but those in our organisation more widely, and our customers too. We also recently won a Mahindra Innovation Award for our eConvergence test management application so, in terms of our achievements and celebrations, we are really matching last year!

We have expanded our Global Test Solutions and Services over the last couple of years, and the whole company is proud that we have done this in a way that maintains and drives the use of industry best practice and emerging techniques and methodologies.

We were very pleased and proud to have some key customers with us at TESTA this year – their projects and organisations having benefited directly from our managed test services.

TEST: What made you decide to enter the awards this year?

TM: Well, considering the success we had last year, we were certainly going to try and follow that this year! In terms of our wins, we had a great value from TESTA last year. Industry peer awards like this give new and prospective customers the confidence that you can deliver and achieve. We were delighted with our shortlistings, and so pleased to win in so many categories. Following this up next year will be a challenge – but we will certainly do our best and strive to achieve – look out TESTA 2015!

TEST: You won two awards in 2013, and now in 2014 you have won five. What developments have you got planned for 2015? Will you be going for another record year?

TM: It’s always onward and upward for us. In all our test services, our central theme is not just continuous improvement, but continuous innovation and continuous value-adds. With Test as a Service and Business Assurance as our key mantras – as we spoke about at the National Software Testing Conference in May – we will continue to invest, and to push the boundaries of efficiency and best practice for ourselves and all our customers – transparently and in partnership.

TEST: What factors do you feel led to you being been so successful in such a range of categories?

TM: We won in both individual and group categories. This demonstrates both our investment in our people, and the quality of our services and offerings. All of our individual winners – Samir Sinha (Test Champion of the Year), Pankaj Kumar (Testing Manager of the Year), and Neal Hardwick (Testing Innovator of the Year) – have been with our test practice for a number of years, and all have been key components in a wider senior management team, which has driven through order-of-magnitude growth in our test services.

Our team awards this year were a result of another key strategy – specialisms and best practice units within testing. In this case, the triumphs came via our Performance Engineering Unit, which goes from strength to strength, in terms of best non-functional testing solutions to fit any scale and budget.

TEST: Would you encourage others to enter next year? What do you feel are the benefits?

TM: Absolutely – and not only because of our success at the awards! It’s great to have a platform for testing and quality assurance in the industry. Even attending the awards – and the National Software Test Conference – you can see, feel and hear the level of enthusiasm, professionalism and pride, of like-minded professionals, who continually strive for success and quality on behalf of customers and end users.

Thank you TESTA, and all the best for next year!

SWEET SMELL OF SUCCESSTaking five awards – Sogeti Green Testing Team of the Year, Testing Manager of the Year, Testing Innovator of the Year, Test Champion of the Year, and Best Overall Use of Technology, Tech Mahindra won in more categories at TESTA than any other company. Here, they reveal the secret of their success

TESTA 2014

Connect with us:[email protected]

TESTING & PERFORMANCE.CONNECTED.

Tech Mahindra's Global Test Practice is the next generation concept in software testing, providing a superior and flexible alternative to standard off-shoring model. It is a multi-location based testing centre of excellence that combines best-in-class test processes, people, techniques, facilities and methodologies. It also offers a full range of scalable testing services, cutting-edge innovation along with optimum unit cost of testing.


have known Bob for about 20 years as a boss, a supplier and, I like to think, a mentor and friend. In a career spanning over four decades, he has, in the opinion of many, both changed the landscape of testing,

and defined today’s way of delivering business value through successful testing companies.

He has built businesses in the USA, UK, France, Germany, Ireland, South Africa and India, which have led the way in delivering testing, both strategically and operationally. While many of today’s self proclaimed testing gurus have an academic view of testing in the real world, this guy has been there and done it on more than one occasion.

Many of you reading this will have been guided or influenced by him over the years, and prospered in your careers as a result - I know I have. I have spent many hours debating testing with him - not necessarily agreeing, but always learning. Although constantly challenging, both technically and commercially, he has always cared, and shown a massive interest in the people who worked with him – supporting them where he could. As a business angel, he has also supported a number of businesses to real success.

Although he prefers to shun the limelight, Bob has been a visionary in the testing industry, and a major influence in the growth of the SIM Group of companies and, in the testing market, SQS (Software Quality Systems). When he speaks, people listen and learn – staff and customers alike. His knowledge, creativity, and approach to testing delivery, along with his commercial prowess, certainly made the people around him stronger, more successful, and confident about the future.

I can remember a heated discussion I had with a salesman on who ‘owns’ a customer: sales or delivery. Bob soon puts us both right, by telling us that neither of us owns a customer – the customer owns us! It is the customer who will decide who they trust, and who they want to work with. This simply sums up Bob’s attitude to adding value.

Now retired, and self appointed Chief Relaxation Officer at Life, Bob is currently enjoying travelling around the world with his wife Judy, drinking good wine and falling off bicycles – not related, I might add! A testing giant, a leader and jolly good chap, Bob deserves this accolade in recognition of a fantastic career and an amazing legacy.

TESTA 2014

LIFETIME ACHIEVEMENT AWARD: BOB BARTLETTIn a text edited from a speech he gave at TESTA 2014 Chris Ambler, Head of Testing, Capita Customer Management, discusses the career of Lifetime Achievement Award recipient Bob Bartlett

Standing ovation for Bob Bartlett

Chris Ambler on stage delivering the Lifetime Achievement Award speech, introducing Bob Bartlett

Bob Bartlett (L)

Lifetime Achievement Award 2014

Pictured with Chris Ambler (R)

I CAN REMEMBER A HEATED DISCUSSION I

HAD WITH A SALESMAN ABOUT WHO ‘OWNS’

A CUSTOMER: SALES OR DELIVERY. BOB SOON PUTS US

BOTH RIGHT, BY TELLING US THAT NEITHER OF US OWNS

A CUSTOMER – THE CUSTOMER OWNS US!

I


For the latest news, visit softwaretestingnews.co.uk and follow us @testmagazine

ADVERTISING AND SALE OF MOBILE SPYWARE: FIRST CONVICTION

A Danish citizen has been fined $500,000 in the USA, as he becomes first person to receive a criminal conviction in connection with the advertisement and sale of a mobile device spyware app. According to the statement of facts accompanying the plea agreement in the case, Hammad Akbar, 31, is the Chief Executive Officer of InvoCode Pvt Limited and Cubitium Limited, the companies that advertised and sold StealthGenie, an app that, once installed, could intercept all conversations and text messages sent using a phone. Undetectable by most users, it was advertised as being untraceable.

Akbar, who was arrested on 27 Sept 2014 in Los Angeles, pleaded guilty to sale of an interception device and advertisement of a known interception device. Accepting Akbar’s plea, US District Judge Leonie M Brinkema, in the Eastern District of Virginia, sentenced him to time served, and ordered him to pay a $500,000 fine. He was also ordered to forfeit the source code for StealthGenie to the government.

According to Akbar’s admissions, StealthGenie had numerous functions that enabled it, without the knowledge of the phone’s user, to intercept outgoing and incoming calls, emails, text messages, voicemail, and photographs. It could also turn on the phone’s microphone when it was not in use, to record sounds and conversations that occurred near the phone.

In order to install the app, the purchaser needed at least temporary possession of the target phone. During the installation process on an Android smartphone, for example, the person installing the app was required to grant a series of permissions that allowed the app to access privileged information on the device. Once the app was activated, it was started as a ‘background’ (hidden) service, and set up to launch automatically when the phone was switched on. The only time that the app interacted with the screen was during activation, and the icon for the app was removed from the phone’s menu. Akbar admitted that, because of these characteristics,

a typical smartphone user would not know that StealthGenie had been installed on their phone.

Says Assistant Attorney General Leslie R Caldwell, “Spyware is an electronic eavesdropping tool that secretly and illegally invades individual privacy. Make no mistake: selling spyware is a federal crime, and the Criminal Division will make a federal case out if it. Today’s guilty plea by a creator of the StealthGenie spyware is another demonstration of our commitment to prosecuting those who would invade personal privacy.

“Mr Akbar is the first ever person to admit criminal activity in advertising and selling spyware that invades an unwitting victim’s confidential communications”, says FBI Assistant Director in Charge Andrew G McCabe. “This illegal spyware provides individuals with an option to track a person’s every move without their knowledge. As technology evolves, the FBI will continue to evolve to protect consumers from those who sell illegal spyware.”

NEWS

TESTING ON THE CLOUD

TestCloud has launched a hosted testing service that’s optimised for mobile apps, but also works on websites. “Our goal is to move from ad hoc project testing, where most teams wait too long before a software release gets tested, to continuous testing that’s more integrated with the development process”, says Frederik Fleck, Chair of TestCloud. The service mixes crowd testing and automated testing of application features.

A dashboard shows how many bugs have been found, how severe they are, and where they are located. This information is collated into what TestCloud calls a release readiness score, which tells you how close an app is to being ready for general release.

“Mobile apps need much more testing”, says Fleck. “Because of the hundreds of devices out there, it’s basically not possible to test mobile in-house efficiently.”

The Overnight version (enabling you to test overnight or over the weekend) of TestCloud’s platform costs 1000 euros per month for two developers on a yearly contract. Each additional developer costs an extra 350 euros.

POLL RESULTSLast month we asked, ARE YOU SUPPORTING THE “RESET THE NET” CAMPAIGN?. Answer this month's poll at: www.testingmagazine.com

YES

NO

Sales

Yes

No

PEOPLE WITH AUTISM TRAIN FOR SOFTWARE TESTING

While, according to the charity Autism Speaks, about 85% of people who have autism in the United States are currently unemployed or underemployed, US social enterprise Meticulon is preparing individuals on the autism spectrum for a software testing career, and also helps them find jobs after training.

Says Autism Speaks, “Autism is a term used to describe complex disorders of brain development, and a person with autism often experiences difficulties with social interaction, communications and imagination. But they also often shine in areas of visual skills, music, maths and art.”

Although autistic people often exhibit great memory, an ability to match patterns, and attention to detail: skills that are particularly useful to software testing, they often have difficulty landing jobs. Says Meticulon, “Each year’s crop of autistic students – or Meticulon Consultants – is tested and evaluated to develop their MindMap – a unique profile of skills and ideal work environment, ultimately used to find these trained software testers an ideal job.”


BASESTONE AND CODACY TAKE TOP AWARDS AT PITCH

BaseStone, a collaboration tool for construction industry professionals, and Codacy, an automated code review tool that allows developers to save time and improve code quality, emerged from a shortlist of more than 200 companies from 36 countries, to take top honours at Pitch 2014, held during the Web Summit in Dublin, Ireland. Each company receives 10,000 euros.

The Coca-Cola Company, which sponsored the competition, will welcome both winners to its Atlanta headquarters in early 2015, for week long mentoring and strategy sessions.

“We’ll provide them with access to the Coca-Cola systems, resources, relationships and reach”, says Guy Wollaert, Coke’s Chief Technical and Innovation Officer, who served as one of the final round judges.

“While we’re a large corporation with operations all around the world, by partnering with startups like these, we’re learning to act small. This is a win-win relationship, and a win-win learning process.”

Says Codacy Co-Founder Jaime Jorge, “We’re looking forward to learning about marketing and also operations, which will benefit us as we grow and gain traction.”

Codacy breaks down code into categories, including complexity, code style and performance, helping developers find errors in the code they write. It recently announced a beta implementation of the ability for users to write customised tests in Javascript. “This is going to be very exciting”, says Head of Product Paul Bleicher. “It lets companies check for specific mistakes that are made over and over.”

The service also lets users set goals related to each category, and enables managers to see how each developer is performing. The platform can analyse code written in CSS, Javascript, PHP, Python and Scala. Java will be the next language added, and the company is also looking at adding Apple’s new programming language Swift.

The Company version of Codacy’s service costs $150 per month for up to 25 users. For that, you get a dedicated server and an unlimited number of repositories.

BaseStone actually didn’t make the shortlist of finalists to Pitch meanwhile, but was added at the last minute when another startup pulled out. “To go from being a wild card to winning is amazing”, says CEO Alex Siljanovski.

“We’re pretty fast and agile”, says Siljanovski, and adds that he thinks Coke can learn from his London-based company’s ability to move quickly and react to situations outside its control.

“As a startup, all the plans you have never go the way you want them to”, he says, “so the ability to adapt to changing circumstances is more important than being able to constantly and meticulously plan.”

NEWS

XBOSoft AND Go2Group COLLABORATION

XBOSoft and Go2Group are to share offices in Reston, Virginia, and Beijing, China, giving both companies access to clients and expertise across the globe, as they implement Atlassian (Go2’s oldest and largest partner) ALM (application lifecycle management) tools in conjunction with XBOSoft’s quality assurance services.

The companies started working together in 2008, with XBOSoft providing software QA services to Go2Group for its industry leading Atlassian plug-ins. Later, XBOSoft provided software testing services to Go2Group’s client base, as it was a natural fit for Go2Group’s ALM clients. Additionally, XBOSoft was able to offer its client base Go2Group’s products and services, to increase productivity throughout the development lifecycle.

ConnectALL, Go2Group’s premier product, is a one stop solution to multiple data sources, processes and systems, with a follow up Process Improvement Service to ensure the end product meets high quality standards. ConnectALL allows teams and technology to share knowledge and projects in the most efficient and coherent way, with reporting options and connectors such as bug, task, requirements, test case and test run issues.

“XBOSoft test automation services expand platform coverage and reduce regression cycle time, which have in the past been major obstacles for our clients”, says Brett Taylor, CEO of Go2Group. “The combined implementation of ConnectALL with XBOSoft’s QA services has enabled our clients to reduce risk and costs and improve overall software quality.”

XBOSoft’s Process Improvement Service identifies root causes of quality problems, and implements a roadmap for optimised testing processes.

Overall, XBOSoft and Go2Group’s partnerships allows high quality test cases and sprint velocity with reduced defects. “The partnership has opened up a raft of clients for us, which has meant our business has expanded significantly over the past few years”, says Phil Lew, CEO of XBOSoft.

RAYTHEON’S US NAVY SOFTWARE PROGRESS

Software development company Raytheon, which is contracted to build the software for the US Navy’s new Air and Missile Defense Radar (AMDR), says its method of developing and building software has been validated by recent results, adding that its agile build methods promoted accelerated development and early risk mitigation, and increased software maturity. The software was validated with a track-loop simulation, in which it detected an anti-air warfare target, and effectively tracked its path using data and feedback from the radar.

The process is segmented into monthly software increments that go through the full development cycle, including requirements, design, implementation, integration and test, to identify issues, errors or defects to be

resolved, before all code has been written and functionality built in. The test was part of the Navy’s AMDR Software Build Review, and came just eight months after Raytheon received the contract to develop the software.

Says Raytheon’s Tad Dickenson, AMDR Program Manager, “Thanks to leveraging the agile process, as well as reuse of radar software architecture and simulation data, and experience gained during the technology demonstration phase, execution of the AMDR programme continues on schedule. Realising the benefits of best practices and the talent and expertise of our team, the programme continues to advance toward targeted radar delivery for the first DDG 51 Flight III ship in 2019.


TWO MONTHS FROM IDEA TO LAUNCH

“Only two months before the launch of Adrian the Ant, Captain Spinks was just an idea, and now we’ve launched five games!”, says Captain Spinks founder, Michael Rodwell.

To go from idea to launch with five games titles in under two months, Captain Spinks Games utilised a 100% outsourced business model and development team. The first of five titles to be released was Adrian The Ant & The Sugar Factory, which came out at the end of October, with a further four games released on consecutive Thursdays for the following month.

In addition to operating with a 100% outsourced workforce, Captain Spinks achieved this rapid rate of games development using a technique known as app reskinning. This involves using pre-existing source code as a basis for the games’ development, with alterations to the graphics, sound, and programming, to breathe a new life and player experience into each game.

Says Michael Rodwell, “As soon as I saw the artwork for Adrian the Ant, I fell in love with the little guy. It’s great to see a theme come to life with such vibrancy and imagination.

“Players are very likely to experience ‘just one more try syndrome’, as they battle to keep Adrian safe from poison, garlic and lemon. This game is beautifully presented, with graphics

that ‘pop’, an infectious theme, and a difficulty level that will keep players engaged and coming back for more.

“It’s amazing how fast Captain Spinks has progressed. It was the approach we took that made the ambitious release schedule achievable. The integration and expertise of a varied team working together around the clock, along with outsourcing and app reskinning, allowed such tight timelines to be met, transforming an idea into reality in such a short space of time.

“Another advantage of app reskinning was that the underlying code has been extensively tested and used in the past. I personally know the developer of the code and, although he limits the number of reskins, we do select code that has been successfully reskinned in the past. This gives us a level of confidence that the project should run smoothly, and that we’re not reinventing the wheel when it comes to the underlying original code. The last thing we want is end up with code that is full of bugs, and requires lots of fixes prior to the development of the reskin!

“As the reskinning process primarly requires a reworking of the graphics and sounds, the actual functionality of the app/game ‘should’ remain intact from the source code. The majority of testing time is spent making sure the new elements are integrated correctly, and there are no issues

with incorrectly displaying graphics or ill fitted sounds. Our games are 2D and have hundreds of elements, compared to 3D games that have thousands. This greatly simplifies the process, and reduces the time required to integrate and test.

“There are layers of people involved – tester, artist, project manager, and then finally me. My role is to check that they app operates as expected, but this isn’t technical at all: it’s more from a usability standpoint.

“The person in the team who is solely dedicated to testing runs the games on various emulators and real devices – a tablet and a phone – on both iOS and Android, looking for errors. He’s directly in touch with the artist and sound integrator and, if an error is found, they work together on fixes.”

NEWS

EXPERITEST AND GALLOP IN PARTNERSHIP AS MOBILE TESTING SURGES

Experitest and Gallop Solutions have entered into a strategic partnership, enabling Gallop to offer Experitest’s Cross-platform mobile application testing solutions to large businesses throughout North America. The surge in mobile usage is increasing the demand for mobile testing tools, which streamline the process of testing applications over different devices with their own specific OS. With the launch of iOS8 and Android L, a majority of mobile users worldwide will soon be using new operating systems. Experitest’s SeeTest mobile testing tool enables testing across multiple devices and operating systems, all at once. According to Gallop, it was this flexibility which differentiated Experitest from its competitors.

Says Kalyana Rao Konda, President, Gallop Solutions, “The mobile world is getting increasingly competitive, demanding higher quality in shorter release cycles. Ensuring compatibility across screen sizes, operating

systems, devices and browsers, is critical. We wanted a solution and tool that would fit

the needs of any enterprise, ensuring security, maintainability and portability of tests. Experitest SeeTest supports all devices and operating systems, and fits in seamlessly with HP, Microsoft, IBM and other eco-systems. This partnership enables our customers to extend their ALM (application lifecycle

management) environments to mobile testing.”

Says Tali Barmeir, Experitest CEO, “We see a huge opportunity in this partnership with Gallop.

Gallop’s specialist co-located mobile software testing services, labs and career testers, create a compelling value for enterprises going mobile. Gallop’s experience as a specialist testing services provider, with our mobile testing solutions, gives enterprises a complete, comprehensive ROI (return on investment) rich solution that is time-effective.”


NEWS

UNDO RECORDING EXTENDS UndoDB FUNCTIONALITY

Reversible debugging tools provider Undo Software has introduced UndoDB Save-Load, as an extra cost option for its UndoDB reversible debugging tool. UndoDB enables developers to record, rewind and replay their code, to quickly find critical bugs, increase productivity, and meet development deadlines. UndoDB Save-Load extends UndoDB’s functionality by creating and saving an Undo Recording. This is a single file, containing information that allows UndoDB to completely reconstruct everything a program has done, including every memory access and every instruction executed.

Through Undo’s inter-machine replay technology, the Undo Recording can then be loaded on a different machine to that on which it was saved, and shared with other developers, increasing collaboration.

Developers can prioritise the order in which bugs are fixed, and jointly debug a particular failure. For

example, bugs that have a long run time or strike intermittently can be captured and then preserved for future analysis, without needing to restart the program or reproduce

the error, thereby saving time. As Undo Recordings can be

saved, UndoDB Save-Load protects against losing

data, avoiding the need to spend time re-running the program to reproduce the bug.

“As an existing user of UndoDB, we are already

seeing the benefits of reversible debugging when

writing code for our Electronic Design Automation (EDA) tools

and wireless sensor network projects”, says Luciano Lavagno, Professor of Electronics Engineering at Politecnico di Torino. “We’re looking forward to trying the new Save-Load option, because of its ability to save bugs for later analysis and thereby increase collaboration within our team. It would be invaluable when finding hard-to-reproduce bugs, such as heisenbugs, that seem to disappear when you try to study them.”

Additionally, the UndoDB Save-Load option enables organisations to reduce usage of expensive or

highly contended systems, such as supercomputer nodes, industrial control systems or prototype/bring-up devices, as debugging recordings from these systems can be saved and then loaded onto any UndoDB Save-Load licensed machine running Linux, even if it is running a different Linux distribution.

“Software development is a collaborative process, and UndoDB Save-Load extends this collaboration to the tracking down of bugs, by allowing developers to work together seamlessly,” said Greg Law, CEO and co-founder, Undo Software. “This collaboration is particularly useful when a developer or QA engineer discovers a bug in a colleague’s or vendor’s code, and wants to pass that issue on to the author of that code for analysis – as most development is done in teams this is a very common scenario. UndoDB is already proven to find critical bugs quickly, and the new Save-Load option increases flexibility, and makes it even easier to benefit from the power of reversible debugging.”

UndoDB Save-Load is available now as an option for new and existing UndoDB licences.

PoS MALWARE ALERT

Trend Micro is warning that a point-of-sale (PoS) malware family of viruses – known as TSPY_POSLOGR.K – is in circulation, with the potential – especially at this time of year – to do extensive damage.

It was around this time last year that US retailer Target suffered one of the largest data breaches in history. In a targeted attack that used the BlackPOS malware, a PoS RAM scraper malware family, cybercriminals gathered roughly 40 million credit and debit card numbers, as well as 70 million personal records of Target shoppers. Home Depot also suffered recently from a data breach, which has so far cost the hardware mart more than $43 million in expenses to investigate.

“Based on the other PoS malware behaviours we observed, it appears to be designed as multicomponent malware similar to an earlier BlackPOS variant”, says Trend Micro. “As it might require another component to retrieve the dumped data. It is highly possible that this is deployed as a package.

“The malware is dependent on its configuration file (which means that it’s starting to build flexibility). By default, the configuration file is not present in the system, so we cannot tell which default processes are being scanned or read. The malware also doesn’t display any known C&C communications, and still has debug strings in its code. Because of this, we believe that this PoS malware is still in the beta testing stage, or under development.”

ONE THIRD OF EUROPEAN BUSINESSES SET TO INTRODUCE WEARABLE TECHNOLOGY TO THE WORKPLACE IN 2015

A recent European survey of IT professionals by Ipswitch demonstrates that a large number of wearable devices will be introduced to workplaces over the next 12 months. It also reveals however that very little thought has been given to the impact this will have on network performance and security, with only 13% of organisations saying they have a policy in place to cover managing the impact of wearable technology.

Businesses in France and Germany appear to be adopting wearable technology at a faster pace than those in the UK. 34% of French and 33% of German businesses say they expect to introduce company owned wearable technology during 2015, while only 25% of UK businesses say they plan to do the same.

The survey also asked about expectations for employee owned wearable devices entering businesses. 36% of organisations said they expect an ‘influx’ of employee owned Apple Watches and other wearable technology in 2015.

However, despite expectations for corporately and personally owned wearable devices entering the workplace in 2015, 77% of businesses said they have no policy in place for managing it.

When asked about their focus for 2015, 11% agreed that they’d like to spend more time specifically planning for wearable technology, and 26% wanted to spend more time reviewing and tightening security policies.

Sogeti - a leading provider of Software Testing services:Winner of Testing Innovator of the Year Award, TESTA 2013

Ranked #1 for Outsourced Testing Services, Ovum 2011

Winner of Multiple Microsoft Partner Awards in 2013

Winner of multiple IBM Beacon Awards in each of the last 6 years

WANT TO FIND OUT MORE?Call: +44 (0) 20 7014 8900 Email: [email protected]

www.uk.sogeti.com

FINALISTS AT THE 2014 TESTA AWARDS

FINALIST

Best Overall Testing Project Gaming Sector

FINALIST

Graduate Tester of the YearPaul Foy

FINALIST

Testing Manager of the YearDavid Firth

FINALIST

Testing Innovator of the YearDaryl Searle

“Sogeti was able to understand the brief and

deliver testing in what was a very tight

timeframe, within the agreed budget, and

with very little input required from ourselves.”

Jason Stewart, E-Commerce Manager

Aberdeen International Airport

“ Key to the Group’s success is:

“its testing and process

expertise and levels of

customer intimacy.”

“High-touch, intimate

testing services.”


sed by criminals to access your computer or mobile device, either in order to take control of it, to obtain your private information, or to spy on you, remote access Trojan software (Rats) is downloaded invisibly with a legitimate

program, such as a game, or sent to you as an email attachment, and is increasingly used to take control of webcams.

Because they usually neither show up in lists of the programs or tasks you are using, nor can you generally notice that they are affecting your device’s

performance, Rats can be difficult to detect.

In some cases, Rats can control your webcam without activating the indicator light, so you are unaware that you are being watched. In addition to spying on you, or surreptitiously recording you, via your own webcam, Rats can monitor your online behavior, such as which websites you visit, whom you email, and what you type, and access your confidential information – including bank account and social security details.

David Land: We have heard about British webcams being hacked, including baby monitors, bedroom cameras and gym CCTVs. How widespread is the problem?

David Bryan, Principal Security Consultant at Trustwave: The problem is very widespread. A simple search turns up thousands of web cameras. When manufacturers develop these kinds of technologies, they often put

security on the backburner, or don’t consider it at all. They feel pressure to get the products to market

quickly, and therefore overlook security, leaving serious vulnerabilities.

According to our 2014 Security Pressures Report, which is a compilation of responses from a survey asking more than 800 full-time IT professionals worldwide about the pressures they face surrounding security, 79% felt pressurised in 2013 to roll out IT projects, despite concerns that the projects weren’t ready.

DL: The UK’s Information Commissioner Christopher Graham urged the Russian authorities to take immediate action to take down websites showing footage from hacked

webcams. Is this likely to happen? Would it have any effect?

DB: Taking down the site is not necessarily the solution. If it’s a site run by criminals, taking it down

certainly helps in the short term, but not in the long term, because the criminals will just find another vulnerability to exploit (or use the same one) in another set of cameras, and create a new website.

Sometimes, security researchers will find such security vulnerabilities and disclose them to the public. In these cases, it’s important to report them responsibly, so that they can be remedied. Security flaws and vulnerabilities must be made public by following a responsible disclosure process.

A SPY IN THE HOUSEDavid Land talks to David Bryan, Principal Security Consultant at Trustwave, about revelations of Russian hackers spying on webcams in British bedrooms and offices, and the importance of building security testing into the production process, prior to the product hitting the market

U

COVER STORY

DAVID BRYAN PRINCIPAL SECURITY CONSULTANT

TRUSTWAVE


DL: The Information Commissioner’s Office (ICO) has set out guidance on setting hard-to-guess passwords and other security measures that webcam owners can take to protect their privacy. Would this solve the problem?

DB: This will help to raise awareness, but ultimately the manufacturers of these internet-connected technologies, rather than having a default password, must make the setup secure by default. Two-factor authentication isn’t possible out of the box, but having a process to require the user to change the default password would be the way to go.

As criminals become more sophisticated, the key is to stay ahead of them. Manufacturers should bake encryption into the product - using TLS1.0 and higher - and not ‘propriety’ encryption. They should also perform automated vulnerability scanning and penetration testing on a regular basis, to help identify and remedy security weaknesses before criminals can exploit them. Scanning and testing should be done before the products hit the market.

If these kinds of technologies are being used in a business environment, businesses should perform risk assessments to identify where their valuable data lives, and if it touches these devices.

They should also perform scanning and testing across all of their assets, and implement security controls that specifically protect their attack vectors.

For consumers that use these products, unfortunately, using strong passwords and turning on any two factor authentication controls is pretty much all they can do. That’s why the onus is even more on the manufacturer of these technologies, to make sure security is baked in, so that when they go to market they don’t have gaping security flaws.

DL: Christopher Graham suggested consumers were too complacent about security, saying, “We have got to grow up about this sort of thing. These devices are very handy if you want to have remote access to make sure your child is OK, or the shop is all right but, unless you set a strong password, everyone else can access that, too.

“This isn’t just the boring old information commissioner saying ‘set a password’. This story is an illustration of what happens if you don’t do that. If you value your privacy, put in the basic security arrangements. It’s not difficult.” Is this true, or are more stringent precautions required?

DB: The steps he suggests would mitigate the problem with a good proportion of the cameras on the market today. However, there are also a set of cameras that allow you to just bypass the login screen, and go directly to the video feed.

Users should adhere to basic best security practices. However, more action needs to be taken on the manufacturer side as well. Manufacturers need to make security a business-as-usual imperative when developing these technologies, and make sure all vulnerabilities are patched before they go to market.

DL: The authorities seem to imply that this security breach is down to lax procedures on the part of owners, much like those targeted in the phone hacking scandal

not changing default passwords. But, presumably, if the stakes are high enough,

security can always be compromised. What’s to be done?

DB: These products need to have not only a secure development process built into them, but also have security reviews, like vulnerability scanning and penetration testing, completed by skilled mock attackers at the midway point in the development cycle, and at

least three months before the product ships to market.

Businesses that use these technologies also need to implement security controls that

include vulnerability scanning and penetration testing on a regular basis – not just annually. They need to perform risk assessments to identify where their valuable data lives, and then deploy security technologies to protect those attack vectors.

If they struggle with a lack of resources and manpower in-house to make sure their security controls are installed, updated based on the latest threats, monitored and working properly at all times, they should partner with a third party team of experts, whose sole responsibility is to focus on security, freeing up their in-house team to focus on revenue-generating priorities.

DL: Apart from posting ‘amusing’ pictures on the web, what’s in it for the hackers?

DB: During one of my penetration tests that I was running on a retail environment, I found a camera system that had a secondary account with a default password. I used that account to gain access to their camera system, and found a camera that was inside a manager’s office, pointed at the cash safe. Had I been a criminal, this would have been valuable intelligence about their security procedures, giving me details such as what time they do cash drops, and possibly revealing the combination.

Thankfully, I was hired to simulate a criminal, and help the businesses identify security weaknesses like this one – but it gives you an idea of how a criminal might leverage these flaws. This would be especially true, given that many of the IP addresses on the internet are now GeoCoded, meaning that if I know your IP address, there is a good chance that I know roughly where that device is located.

And if that device becomes a jumping point to other devices, like your wireless router, now I know your WiFi, service set identification (SSID), and possibly your media access control (MAC) address.

DL: Given their vulnerability, do the benefits of webcam type devices outweigh the risks?

DB: It really depends on the risk that the consumer is willing to take. Many of these devices make use of Universal Plug and Play (UPNP), which will automatically open holes in your firewall so that you – and anyone else on the internet – can access these devices. Additionally, consumers are using applications that front-end these services, so the user doesn’t know if the backend communication is encrypted or not.

If consumers want to be secure about their setup, I’d recommend getting something that can only be accessed using a virtual private network (VPN), or a central server that lives inside their network.

"WE HAVE GOT TO

GROW UP ABOUT THIS SORT OF THING. WEB

CAMS ARE VERY HANDY IF YOU WANT TO HAVE REMOTE ACCESS TO MAKE SURE YOUR

CHILD IS OK, OR THE SHOP IS ALL RIGHT BUT, UNLESS YOU SET A

STRONG PASSWORD, EVERYONE ELSE CAN ACCESS THAT,

TOO." – UK INFORMATION COMMISSIONER,

CHRISTOPHER GRAHAM

COVER STORY


ith the boom in the digital service economy showing no signs of abating, customers are demanding constant service improvements, which businesses must support with faster, better and increasingly functional applications,

placing significant pressure on enterprise IT.

The mainframe, which has sat at the heart of the business supporting innovation for an incredible 50 years, is now dealing with levels of workload that couldn’t even have been remotely envisaged when it was conceived.

To keep pace with customers and competitors, the teams that build, maintain and improve customer facing applications need to speed up the development process. The barriers that once existed between development and operations teams are coming down, as development teams begin to work more closely under the DevOps ethos, to ensure they’re labouring toward a common goal. That is also why development teams, even in the mainframe environment, are being forced away from

traditional waterfall development models, toward agile practices.

NEW LEASE OF LIFEAgile practices have been proven to have a positive impact on the development cycle. As well as helping application teams to become more dynamic, these new ways of working can also reduce inefficiencies in application DNA, and ensure the finished product is more closely aligned with user expectations and needs. Developing in small, incremental ‘sprints’, which allow for user feedback at every stage meanwhile, means applications can be perfectly refined for requirements along the way.

Quality and consistency can also be better assured, as the application is tested at every stage to ensure it all works together, rather than testing the parts of the whole in isolation. This means that developers can be much more effective and efficient, meeting user requirements

DEVELOPMENT ISSUES

THE MAINFRAME GETS AGILESteven Murray, Mainframe Solutions Director at Compuware, discusses the crucial importance of bridging the cultural divide between web and distributed application development on the one hand, and mainframe development on the other

STEVEN MURRAY MAINFRAME SOLUTIONS DIRECTOR

COMPUWARE

W

THE MAINFRAME

IS MORE CRITICAL TO TODAY’S

BUSINESSES THAN EVER, UNDERPINNING VITAL

FUNCTIONALITY IN OVER HALF OF ENTERPRISE

APPLICATIONS, WITH 81% OF CIOs BELIEVING IT WILL BE

A KEY BUSINESS ASSET OVER THE NEXT

DECADE


with greater accuracy, and reducing the time taken to do so, by eliminating the need to continually return to the drawing board. Indeed, as user requirements evolve during the development process, the end result is often completely different from what was described in the original brief.

However, while agile development and the DevOps ethos are gaining traction in the world of web and distributed application development, mainframers have been slow to embrace agile. Having been a stable part of the IT environment for half a century, many believe that there’s no need for mainframe development to change. But the mainframe is more critical to today’s businesses than it has ever been before, underpinning vital functionality in over half of enterprise applications.

It’s unsurprising therefore that research has shown that 81% of CIOs believe it will be a key business asset over the next decade.

With this in mind, the mainframe can’t be viewed as a second level priority. If they are to safeguard the future of their IT services, IT leaders must encourage their mainframe development teams to adopt the same modern ways of working as those adopted by their counterparts on distributed application teams.

OBSTACLES TO AGILITYThe first hurdle to overcome is that mainframe and distributed application teams have very different working practices and expectations. The critical nature of mainframe applications means that there is a culture of excellence and reliance on consistency, which have understandably made mainframe development teams resistant to change. Developers must tread carefully, to avoid making any changes that could have a negative impact or cause disruption.

In the distributed application world meanwhile, the opposite is true: dynamic and revolutionary changes are the key objective. As such, these teams have been quick to embrace agile development and DevOps approaches.

There are also mismatched expectations between the two functions. In the distributed application world, a single sprint could result in a dynamic change to an application, such as a brand new user interface. In the mainframe world, the change is likely to be less visible: for example, the addition of a new database field.

These differences make collaboration between the functions difficult, but both are essential for the success of modern IT projects. Furthermore, mainframe skills are becoming increasingly scarce and invaluable, so businesses can’t afford to push their developers into early retirement by rocking the boat too hard.

Limited collaboration between the two teams around what the customer needs can also lead to inefficiencies that drive up costs. For example, a mobile banking application could be unnecessarily requesting customer account details multiple times, thus creating more workload for the mainframe. Such inefficiencies can only be addressed if distributed application developers work closely with mainframe teams, describing the user story that defines how the finished application will be used.

Mainframe development teams can then identify how best to support it from their end.

However, with teams often dispersed over a wide geographic area and working in silos, this is becoming more difficult. The drive toward DevOps in the distributed application world has set the bar for how barriers between functions can be bridged, but geographic boundaries are harder to cross.

THE MAINFRAME ASSETAfter 50 years, the mainframe remains a consistent anchor for today’s enterprise IT. No other asset has proven as reliable, secure, or adaptable to the changing needs of business, and it’s clear that it will remain a critical resource for years to come. As such, IT leaders cannot afford to sit back and allow mainframe teams to fall behind the progress being made in other IT functions.

Neither can they force mainframe teams to completely revolutionise the way in which they work. A subtle and considered approach will ease the transition to agile development on the mainframe, helping to bring these irreplaceable and invaluable developers up to speed with distributed application developers, without alienating them.

STEPS TO MAINFRAME AGILE DEVELOPMENTThe good news is that, without forcing their teams to reinvent the wheel, there are a number of measures that IT leaders can take to address these challenges and ease the transition to agile development on the mainframe:

• Encourage face time Daily stand up meetings can be a great way of getting development teams to work more closely together, by getting them to spend a few minutes each day discussing what they’re working on and the progress they’ve made. Teams that work remotely can join by video conference, so nobody’s left out.

The aim is not only to keep projects on track, but to share expertise and discuss ways of working around any problems that arise. This can also help to ensure that mutual expectations are realistic, and mainframe and distributed application developers are working toward shared goals.

• Automate testing Agile development also increases the amount of testing needed to maintain the consistency that mainframe applications demand. Completing this manually would place far too great a strain on mainframe teams, so it is understandable that they would be resistant to change. However, automated testing tools can remove this burden, while ensuring that the same tests are applied after every development sprint, safeguarding consistency.

• Bridge the divide Introducing mainframe tools with more modern user interfaces can bring newer generations of developers that are accustomed to the distributed application world up to speed with their legacy counterparts. If possible, cross-training mainframe and distributed application developers will further help them to learn from one another, and gain a better understanding of their different ways of working.

DEVELOPMENT ISSUES


LOOKING TO THE FUTUREOne of the top firms worldwide in IT, consulting and business solutions, Tata Consultancy Services (TCS) took Leading Vendor at TESTA this year for its quality assurance and testing services. David Land, 2015 Chair of TESTA Judges, caught up with Prabhakar Karamsetty, Head of TCS’ Assurance Services Unit, UK & Europe, to discuss opportunities and challenges in the quality assurance space, and to ask him to predict the industry’s future with his crystal ball …

PRABHAKAR KARAMSETTYHEAD OF ASSURANCE SERVICES UNIT, UK & EUROPE

TATA CONSULTANCY SERVICES (TCS)

ssurance Services Unit (ASU), the enterprise quality assurance and testing arm of Tata Consultancy Services (TCS) operates in 56 countries, with over 27,000 specialised test professionals. ASU provides holistic end to end assurance, across various industries and technologies, at all stages of the test value chain, from

strategy, planning, consulting, advisory and management, to execution, completion and reporting. It offers functional, non-functional, automation, test environment and data services, with a focus on transformation and quality guardianship in test factory, test centre of excellence (CoE), and managed services models.

David Land: So, Prabhakar, what is your view on current developments and growth plans for the future?

Prabhakar Karamsetty: We believe that, in addition to rapid adoption of speed to market methodologies like DevOps, market growth for the next three to five years will be shaped by what we call the Digital Five Forces: mobile and pervasive computing; cloud; big data and analytics; artificial intelligence and robotics; and social media. Our growth plans are therefore designed to address this market disruption.

Our plans are broad-based and market leading across industries, with a focus on holistic growth across all the types of services we provide, and all the verticals in which we operate, always aiming to be a step ahead of the market and customer demands.

DL: Established in 1968, TCS has seen software testing evolve from its early origins. Can you talk about the changes that have occurred?

PK: Testing has matured from a standalone activity that happens at the end of the software development lifecycle (SDLC), to something that is truly value adding. We have shifted our testing focus from defect detection, to prediction and prevention. Our customers see a lot of value in this model, where we focus on quality and assurance to the left (early) part of the production release, and communication to the right.

DL: Can you please talk about TCS’ specialist skills and areas of focus?

PK: Over the years, TCS has built up expertise testing in niche areas like enterprise resource planning (ERP), the Digital Five Forces, and automation; and in non-functional services like performance engineering, security, accessibility, usability, browser compatibility, and so on.

We also provide specific testing services in different industry verticals, for example for point of sale (PoS) in retail, ATM testing in retail banking, third party product testing in investment banking and insurance, validation and

A

SUPPLIER PROFILE


TESTING IS MOVING OUT

OF THE REALM OF SCIENTIFIC CAUSE-

EFFECT RELATIONSHIPS, AND INTO THAT OF PSYCHOANALYSIS-BASED COGNITIVE

THEORIES

verification services in life sciences, ‘smart’ testing in utilities, and network testing in telecom. These services are delivered by a mix of skilled domain, technical, and testing associates and, in many cases, we have set up Test Labs globally to deliver these services

While we continue to focus on our core set of assurance services, TCS is making significant investments in R & D and intellectual property-based assurance solutions, like intelligent testing systems (ITS), which will help customers to digitise the complete test value chain.

DL: Can you please talk a bit about current market trends?

PK: As technology advances and gets more complex, customer priorities are changing and, with on-demand, cloud, pay per use, and so on, service models are also changing. With organisations moving toward agile and DevOps, the role of QA has become much wider and more important, and this trend is set to continue.

All of this demands a change in the way in which we deliver our services. With the spotlight on QA and testing departments like never before, QA teams are taking accountability for end results, which is why TCS’ services go beyond ‘testing’, and provide end-to-end domain-based ‘assurance’.

DL: That’s interesting. Can you please explain the significance of this?

PK: Each industry has its different needs and requirements, and what we offer needs to reflect this. In banking and financial services for example, multi channel, regulatory compliance and simplification are some of the key priorities. Assurance’s role here is bringing a seamless experience to the customer in a multi channel environment, certification services for regulatory compliance, and facilitation of automation, cloud and on demand services for simplification.

If you take retail, some of the priorities are online shopping, supply chain simplification, and enhanced store experience. And again, assurance is playing a key role. In travel and transportation, digital is a leading trend. In utilities, the installation of smart meters is one of the key priorities, while in manufacturing, supply chain optimisation is a priority.

In the majority of these verticals, the IT landscape across the industry value chain is changing from bespoke to products, and hence, requires assurance at all levels. In each instance, assurance has a

SUPPLIER PROFILE


significant role in reducing the total cost of ownership (TCO), improving quality and time to market.

DL: Can you please talk about the role of assurance in digital transformation programmes, and TCS’ unique offer in this area?

PK: With the ongoing digital transformation of the marketplace, the role that assurance plays is undergoing a fundamental change, to validating the ‘product’ or ‘experience’, rather than an ‘application’ or ‘change’. Customers aren’t any longer going to pay just for the functionality of a product; rather they will be concerned with the ‘experience’ or ‘ease of use’ of a specific product or service.

The implication of this is that, to exceed the expectations of the business and customers, business and IT teams supporting this transformation across organisations will have to refocus and rebalance their strategies.

To offer acceleration to this transformation, assurance services have naturally gained prominence in the lifecycle, and will continue to be a value generating lever for organisations that embark on this journey of digital transformation.

To put it simply, it’s no longer going to be enough to test against a discrete test scenario, based on the domain or technical aspects of the software. You will have to predict the various types of experience the user is likely to go through, and test for these as well. It’s quite exciting: testing is moving out of the realm of scientific cause-effect relationships, and into that of psychoanalysis-based cognitive theories.

We believe in testing and QA organisations providing ‘certification’ for not just the product, but also the business, and the customer. TCS works to this ethos, which is the main reason why we christened our quality assurance and testing arm Assurance Services Unit (ASU).

Business and customer assurance cannot be achieved overnight however, but requires a holistic ecosystem. Our assurance services and solutions ecosystem is built around the key principle of being a pervasive value generator across the business and IT lifecycle. We believe that building our ASU into the SDLC at the leftward end of the cycle can have significant value, ensuring that velocity and quality can be unlocked very early in the lifecycle, as well as later, achieving benefits in terms of both IT and business metrics.

People across the world are consuming more content by the day, across an ever growing number of multiple devices and platforms. The demand for seamless access to content grows exponentially and, as a result, internet protocol (IP) technology is now embedded in almost all end-user devices and network side technologies.

The implication of this is that we must have, and continue to build, unique, domain-centric and next generation IP, to realise and offer significant value to our customers, which in turn ensures realisation of early and sustainable business quality at an optimal TCO, and elevated speed. All of this is wrapped in our core value of Experience Certainty, hence we believe we are unique, and offer

a compelling proposition in the global market.

That's a long answer to a short question!

DL: Can you please discuss some instances of challenging assignments lately?

PK: While each assignment brings its own challenges, a specific example that springs to

mind is that of a large logistics company in Europe, for which TCS became the quality guardian vendor, and certified its complex, 150 country rollout.

To support the rollout, we established a test factory, to deliver functional, non-functional and automation services. To bring the best automation solutions, we leveraged TCS’ in-house tools and frameworks for enhanced productivity and efficiency.

In another example, TCS was involved in testing of complex digital architecture, and a European rollout, for a car rental company, which resulted in a reduction in customers’ queuing time of around 30%, and an improvement in bookings of around 4% per day.

TCS was also the Testing and QA partner with leading UK-based insurance company Legal & General in its Retail Distribution Review (RDR) – a large and complex legislative compliance programme – in which TCS not only provided testing services, but played a quality guardianship role.

Quoting the customer, Rob Wilkinson, RDR Test Programme Manager, Legal & General, UK, “Our RDR programme was hugely successful, delivering not only legislative compliance, but also a market leading PoS system for our advisers and partners. The TCS Assurance Services team played a critical part in our success. Not only did they show an unwavering dedication to delivery, they constantly reviewed the efficiency of testing, and worked with all parties to secure delivery. ‘Impossible’ was never part of their vocabulary. Every challenge was met with enthusiasm, and a determination to find a solution that was viable for all parties. I am proud to have worked with TCS on this programme, and look forward to collaborating with them again.”

DL: Looking to the future, what issues and trends do you see emerging for the industry?

PK: Staring into a crystal ball is never easy! However, it’s fair to predict that the next three to five years will be shaped by the ever increasing adoption of digital technologies. Due to the very nature of this disruption, the future of assurance is bright.

There will be significant adoption of extreme automation techniques and IP-led solutions, leading to the overall value chain being reshaped to offer enhanced value. There will be enhanced focus on assuring the business, and ensuring superior end consumer experience, which means assurance services will be increasingly in demand. Delivery models will be continually recast for assurance services, and adoption of cloud as a medium to offer these services will increase. Overall I can say that, while roles within assurance may change, the role of assurance will become much more significant, exciting, and all pervasive.

SUPPLIER PROFILE

AS WELL AS TAKING LEADING

VENDOR AT TESTA THIS YEAR, TCS ASSURANCE

SERVICES WAS RECENTLY RATED AS #1 IN NELSON HALL SOFTWARE TESTING VENDORS’ EVALUATION, AND ALSO POSITIONED AS LEADER IN EVEREST

PEAK MATRIX

Working in thetechnology sector?We’d love to meet you!

Our technology has the power to shape our future and secure our success.Our people hold the key to that power.

We have opportunities for you to develop your skills and career. Find out more at www.sage.co.uk/technologycareers

Are you interested in:• Software development, testing and leadership • Project and programme management• IT security and architecture • Business analysis

35754AL_Tech Group test awards magazine advert.indd 1 05/12/2014 16:36


complete test management solution, PractiTest can deal with every step of the QA process, from requirements, to tests, runs, and bug tracking. “Our vision is that testing tools are meant to simplify test management”, says

Ma’ayan Leeper-Carr, Marketing Manager at PractiTest. “Our test management tool feels natural to work with, and requires little to no effort to utilise, while enabling successful and speedy product releases.”

PractiTest was founded by testing and development veterans, who were looking for a software testing solution that would offer an alternative to the heavy and expensive enterprise products, but that on the other hand would be easy to use, and something with which testers, managers and developers would actually enjoy working.

A global company, catering to clients from New Zealand and Australia, via Asia, Africa and Europe, all the way to the US and Latin America, PractiTest has a solid base in the US and Europe, with significant presence in the UK, the Nordic region, and the Netherlands.

“As a ‘software as a service’ (SaaS) solution, PractiTest can offer the flexibility to work with distributed test teams, meaning projects can be worked on from anywhere, with access for anyone”, says Ma’ayan. “It’s designed to work fast, regardless of your internet connection or location, meaning you can work seamlessly with PractiTest whether you work in-house or offshore, outsource or crowdsource – or just in multiple geographic locations. Remember that, even if you don’t work distributed today, you may do so tomorrow.

SUPPLIER PROFILE

EXPANDED INTEGRATIONSPractiTest's Marketing Manager, Ma’ayan Leeper-Carr, talks to TEST Magazine about the testing software’s origin and evolution, and how it adapts to changing demands

A


“We feel that one of the biggest challenges when working on a testing project is how to manage your information and, more specifically, how to quickly find your test cases and generate good reports. PractiTest has its own unique solution – Hierarchical Filters – to help deal with this.”

Instead of rigid folders, Hierarchical Filters are dynamic filters. Added to the flexibility of PractiTest’s customisation settings, and the graphic information displayed on its dashboard, Hierarchical Filters are an efficient way to sort and manage the QA process of any project, promoting visibility and traceability, which are key for a smooth release. They enable anyone to define and organise the contents of their versions, features and product, to trace each requirement to test(s) and Issue(s), and to get visibility into their requirements, based on the status of the tests that cover them.

“The unique filters which are at the core of PractiTest’s solution give better support for agile teams as well, as they allow you to quickly arrange your data based on sprints, and also to reflect information based on user stories, epics and tasks”, says Ma’ayan.

“It seems to us that the most prominent issue in the market today is the increased demand for quick ‘time to market’ releases. This means that the pace of every part of product development – and testing in particular – is becoming ‘fast and furious’.

“When applied correctly, the trending shifts toward agile testing and scrum methodology both support this growing demand, and it’s important for us to offer a software solution that enables companies to reach these new goals in a highly competitive market.”

Being based on user stories and epics, while at the same time based on the different sprints of the team, PractiTest helps organise the work of the test team. It is also a lean solution, which helps you concentrate on testing – rather than on managing or maintaining your testing tool!

“PractiTest is always striving to tweak and improve what

is in fact already an ‘agile friendly’ solution, and thereby remain a top solution provider for such trends”, says Ma’ayan. “This can be seen in our wide range of options for seamless integration of PractiTest with a variety of popular bug tracking and automation tools – such as Selenium, QTP and TestComplete. Many of our customers benefit from the complementary work of PractiTest with Jira, with which we have created a powerful one and two way integration, which enables a comprehensive test management system, without deserting your existing bug tracker.

“You can even use PractiTest’s application programming interface (API) to connect external

frameworks such as Jenkins. And our clients have the option to develop their own API

for use with PractiTest if required. All this makes PractiTest the ideal choice for those searching for a test management solution which will work with their existing systems without having to shift over completely.”

Another strong element of PractiTest’s offer is its human methodological support, meaning it can provide expert

support of its platform throughout your project, helping you to customise and

use the software to get the most out of it.

PractiTest’s team is constantly growing and expanding. “We have recently been

approved as an Amazon Web Services Technology Partner”, says Ma’ayan. “We plan to continue expanding our integrations, and to look at ways to help QA Teams communicate their work with the rest of their organisations. We are bringing in some additional features for 2015, both large and small, which will make the lives of testers better.

PractiTest O’Hare Rosemont, 5600 N River Road, Suite #800, Chicago, IL 60018 Phone: US: +1-847-993-3064 | Int: +972-8-637-6997 |www.practitest.com | [email protected]

SUPPLIER PROFILE

BEING BASED ON USER STORIES

AND EPICS, WHILE AT THE SAME TIME BASED

ON THE DIFFERENT SPRINTS OF THE TEAM, PRACTITEST

HELPS ORGANISE THE WORK OF THE TEST TEAM. IT IS ALSO A LEAN SOLUTION, WHICH HELPS

YOU CONCENTRATE ON TESTING – RATHER THAN

ON MANAGING OR MAINTAINING YOUR

TESTING TOOL!


f you saw this job advert, how confident would you be that you could fit the bill? In fact, would you know anyone who could? As IT workers typically have a career legacy spanning either development or operations, not both, this

combination of skills is scarce. But, in 2015, workers with this skillset will be in high demand.

THE RISE OF DevOpsWith IT-enabled innovation a competitive differentiator for almost all type and size of organisation these days, agility in delivering IT systems is critical, as is the ability to run them reliably and cost effectively. This gives rise to the DevOps (development and operations) concept and, in turn, the demand for the skillset discussed here.

DevOps aims to break down the barriers and conflicting priorities which often exist between development and operations teams, such as project spend, application performance and functional requirements, thus allowing them to work together, to deliver systems into production reliably, safely and rapidly, and to operate and support them more effectively.

DevOps aims to develop a collaborative working relationship, and foster adoption of a common set of objectives, to deliver IT services that provide strategic value to the business. While DevOps is a culture rather than a specific technique or technology, it is heavily associated with a set of techniques collectively known as continuous delivery, which is a synthesis of concepts including lean production, continuous integration and continuous deployment.

CONFLICT BETWEEN DEVELOPMENT AND OPERATIONSUnfortunately, traditionally there has been little, if any, working partnership between the development and operations silos. Development and operations teams might work in different buildings – perhaps different continents. Even if they occupy the same time zone,

operations teams work 24-hour shift patterns, whereas development teams generally don’t. Their organisational structures and technical languages are usually distinct, and movement between teams that might foster a common understanding is rare: hence the scarcity of candidates who meet the criteria in the job advert at the start of the article.

The distinction between the roles of development and operations means most developers don’t have the depth of operational knowledge necessary to undertake complex infrastructure automation tasks, and few infrastructure and operations experts have the requisite software engineering skills to build software that performs reliably in production.

Although functional testing automation skills are now relatively widespread, DevOps can only work if testers are able to automate testing for the factors, such as performance, resilience and security, that matter most to operations. Few currently have this experience.

Any DevOps programme of enterprise scale will require first class, highly technically aware delivery managers and architects to plan and manage the necessary development, infrastructure, vendor management and organisational changes required.

IS DevOps WORTH IT?Today’s business challenges have pushed traditional delivery approaches to new levels. Therefore, the benefits of a DevOps approach far outweigh any potential difficulties in aligning the two areas. It delivers systems to the business faster and, through automated non-functional testing and shorter development iterations, reduces risk of production changes. In addition, it automates service management to support operational objectives, and improves understanding of all layers of the production environment stack, helping to prevent and resolve production issues.

Businesses might also find the differences are easier to overcome than originally thought. Typically, rather than a change in technology, it’s the change in culture that’s the issue. In addition to acquiring the right skills, successful DevOps requires careful management of organisational and cultural change.

Delivery of IT services is similar to a production line: speed of delivery and quality of the finished article are determined predominantly by the weakest link in the chain.

Until technical education and career progression catch up and encompass both sides, IT organisations can partner with specialist companies which are able provide both DevOps-trained experts and change management support. DevOps promises much, but needs to be underpinned by cultural change and the correct mix of skills.

BREAKING DOWN BARRIERSThe concept of DevOps is to develop collaborative working relationships between development and operations – areas which are traditionally non-aligned – and foster adoption of a common set of objectives, to deliver IT services that provide strategic value to the business, as Nigel Wilson, Head of Service Delivery at BJSS, explains

I

IT INNOVATION

NIGEL WILSON HEAD OF SERVICE DELIVERY

BJSS

WANTED: Developers who understand infrastructure. Automation-focused infrastructure engineers and operations staff. Experienced and non-functional testers. High-end technical delivery managers and architects with a solid background in IT development and operations.

www.neotys.com

Test Mag Agile Ad 02.indd 1 14-03-19 11:00 PM


he quantity of data that we now collect, store, analyse and distribute is larger than at any time in history, and is set to rise exponentially in the coming years. This is due in part to the rise in ubiquity of information-sensing mobile

devices, aerial sensory technologies, software logs, cameras, microphones, radio-frequency identification readers, wireless sensor networks and so on, and partly due to the coming Internet of Things, with billions of devices, such as heart monitoring implants and cars, having their own IP address.

The figures applied to the concept of big data change with time. What is considered ‘big’ today will not be considered big in years to come, but suffice to say that big data is a term for any collection of data so large and complex that it becomes difficult to process using traditional applications.

US-based company Delphix specialises in enabling more efficient data management, and making application development faster and more efficient. It aims to radically change the way companies deliver their data, by providing fast, flexible, efficient access, enabling agile data management, through intelligent software that eliminates redundant infrastructure and slow processes. As a result, it says, its customers deliver higher quality business applications in less time and at lower cost.

David Land: ‘Big data’ is certainly a buzzword at the moment, with the consensus seeming to be that the operative word is Big – the

quantity of data that the Internet of Things will generate will be huge. So, I would surmise that, regardless of how efficiently you de-duplicate and store your data, there is a management issue here.

Iain Chidgey, General Manager of EMEA (Europe, Middle East, Africa), Delphix: The growth of data is down to many things, but regardless of where it’s coming from, we estimate that 80% of the world’s data is probably copy data.

Production data and data copies are interlinked. The explosion in the so-called application economy – from the smart phone/tablet revolution to the development of SaaS and cloud – has meant a huge increase in data. Add the Internet of Things to the mix, and yes, data will increase further.

The issue is that current infrastructure is unable to support today’s needs, let alone tomorrow’s. The growth in the size and complexity of data has put huge pressures on application development, testing, and the supporting infrastructure. To work around this constraint, organisations are using subsets of data and/or taking less frequent snapshots. This means that the test data is often stale or a poor representation of production, which in turn leads to bugs and errors.

DL: Can you explain how the increased need for test environments and simulations places a demand on IT

for data copying?

IC: Take methodologies, such as agile and DevOps, which require a quick stand-up and

tear-down of environments. One of our customers

had 27 production databases totalling 3TB of data, which was being used by two waterfall teams. It was taking them

three weeks to stand-

MANAGING THE DATA DELUGEDavid Land talks to Iain Chidgey, General Manager of EMEA at Delphix, about the exponential rise in the quantity of data, both now and – with the coming Internet of Things – on the horizon

T

WHAT IS CONSIDERED

‘BIG’ TODAY WILL NOT BE CONSIDERED BIG IN

YEARS TO COME, BUT BIG DATA IS A TERM FOR ANY COLLECTION OF DATA SO

LARGE AND COMPLEX THAT IT BECOMES DIFFICULT

TO PROCESS USING TRADITIONAL

APPLICATIONS

TECHNOLOGICAL DEVELOPMENTS


up a single copy of a single production database. They wanted to move to 30 agile teams, each with a copy of all 27 databases. That’s 810 databases now at 90TB of data.

To realise the potential of agile, they wanted to stand-up and tear-down every single environment, every three weeks. Data virtualisation not only enables this to be done on-demand and in minutes, but requires around 95% less storage.

Another huge challenge is integration testing. You ask any online retailer about the difficulties in bringing together data from a single point in time from multiple sources, and they’ll tell you it’s a nightmare. Make that continuous integration testing, and the task becomes impossible. With Delphix, you can synchronise multiple sources, and automate the process, so you can do point-in-time data copies literally in minutes. Not only that, but you can roll back all the data sources in sync, at the click-of-a-button. To testing and QA, this is a revelation.

DL: In the current environment, how can developers and testers best be empowered?

IC: Delphix believes that developers and testers should be empowered to do their jobs without having to worry about the impact on infrastructure. The problem today is that not enough testing can be done, because of the difficulties in building test environments. Instead of refreshing your data every few days and batching together a set of tests to fit in with the schedule, now you need the ability to refresh and test your data whenever you like.

DL: On your website, under Application Development, you say that ‘Test instances are not agile’, ‘Databases are not agile’, and so on. Can you explain what agile testing has to do with big data, and efficient data storage?

IC: Agile has really helped development teams deliver code changes and new products much faster. The problem is that the provisioning environment hasn’t kept up, and is creating a bottleneck. In many cases, infrastructure has done a great job reducing the time it takes to deliver change through automation and scripting. However, this still means that development is waiting long periods for environments, and that provisioning is still linear.

Given that the biggest bottleneck is at the point where data is pushed downstream to testing, the need for quality data is even higher. Increasingly, we are seeing that a constant negotiation exists between adequate testing and the time/cost involved in supplying the data.

DL: Please explain the concept of ‘freedom from the tyranny of legacy’, which seems to be key to what you do.

IC: By ‘legacy’, we mean the age-old way of taking snapshots, clones, or using subsets of data. By ‘tyranny’

we mean that development and testing has had to live with the consequence of legacy for many years – waiting for days, weeks or months for environments, and working with old data because there hasn’t been a better way, until now.

DL: Please talk about how DevOps figures in your approach.

IC: DevOps is hot, and we spend a lot of time talking to companies adopting it all over the world. In the novel The Phoenix Project by Gene Kim, the two biggest IT constraints in realising DevOps are the setting-up of developer and QA environments.

The reason why DevOps is so relevant to Delphix is that we bridge the gap between needs: development (speed) and operations (cost and stability). Our agile data platform means database administrators (DBAs) aren’t spending their time on menial tasks, such as copying data, and can spend more time on complex projects that add business value. Each data copy is no longer taking up extra space, which provides extra storage. Ultimately, DevOps means that development is empowered to be creative, and testers are able to concentrate on testing more thoroughly.

DL: You say that inefficiency, when it comes to development and testing, is a cultural problem. Can you expand on this a bit?

IC: This goes back to the status quo. When new test data is required, it can involve sign off from your line manager, DBA, server admin, system admin, storage admin, network admin, and – if more storage is required – even additional purchasing. Each stage can have its own SLA (service level agreement), which is how it can takes weeks, even months, for this to happen.

The result is that infrastructure can’t handle any more requests, and gets into the habit of saying no. In tandem, development and testing get tired of receiving the same response, and stop asking. You end up with friction, and the two sides working against each other.

In addition, Delphix enables a new level of relationship between development and testing, meaning a tester can share their environment with a developer in an instant. As environments can be run in parallel, one with live data, where the problem exists, and yesterday’s, where it didn’t, triaging a problem can be easily achieved. The developer can run their latest branch of code with one set of data, while simultaneously having one shared with testing to resolve a minor bug.

The DevOps movement is about making everyone work on the same side, and toward the same business goals. By removing the bottleneck of data provisioning, Delphix goes a long way to enabling DevOps, and fundamentally helping to transform the whole process.”

TECHNOLOGICAL DEVELOPMENTS


n Cyber Monday last year, although online shoppers spent a record breaking £600 million, as they struggled to cope with the pressure of increased promotion-driven website traffic, many retailers failed to capitalise on potential

profits. When traffic peaked, big brands’ websites saw a drop in their load times – like eBay and Asos, which suffered a drop of 31% and 19% respectively – impacting customer experience as well as revenues.

This year, in the lead up to Black Friday and Cyber Monday, the busiest annual online shopping days of the year, although retail organisations had the opportunity to take extra precautions to ensure their mobile and web applications could cope with heavy peak loads, casualties were again reported.

RISKS OF NON-PERFORMANCEAccording to an independent global research study undertaken by Vanson Bourne, even minor delays to website response times can have a sizable impact on customer satisfaction, page views, conversion rates and site abandonment. Despite this, the survey further revealed, an astonishing 32% of organisations don’t know if their website is monitored on a 24/7 basis.

Although 79% of CIOs knew of the usual instances that drive peak traffic volumes meanwhile, 44% didn’t test their website’s performance to see if it could handle increased pressures.

The latest figures from Borland’s website performance monitoring tool demonstrate the potential impact of limited or no testing. Between Black Friday and Cyber Monday 2014, Asos, Amazon and Apple websites all suffered for intermittent periods.

Retailers cannot afford to lose customers just when their users need them most. Not only can it lead to an

astronomical loss of sales; it can also lead to a loss of reputation – something from which businesses may find harder to recover. In fact, Micro Focus’ CEBR (Centre for Economics & Business Research) research puts this loss globally at £36.7bn per year.

So, in the face of unusually high demand, such as that occasioned by promotional or seasonal trading, what approach can software testers take to ensure optimal website performance? How can they make sure these applications’ environments are constantly prepared for the extreme, while keeping within an inevitably tight IT budget?

TESTING IS IMPERATIVEFor applications to perform as expected in the real world, performance testing is imperative. In particular, business critical applications need thorough testing, to ensure they can bear the stresses and strains of the varying demands that companies have for their products and services.

TRADITIONAL TESTINGWhile traditional stress or performance testing, of both the application and application infrastructure, is well proven, it can be costly. It involves buying performance test software tooling, including the purchase, deployment and maintenance of the client and server infrastructures in order to simulate the load. It involves in addition the development of simulation scripts, user expertise and time. Thus, it can be a huge barrier to businesses investing in the tools that can ensure optimum web performance.

Similarly, traditional departmental silos can add another layer of complexity. While website performance was traditionally considered an IT problem, as marketers have become increasingly tech-savvy with the rise of digital marketing, a poor performing website has become a big problem for both departments.

THOUGHT LEADERSHIP

PEAK PERFORMANCEWith big online firms suffering losses during periods of peak demand, Archie Roboostoff, Product Director at Micro Focus, looks at the wisdom of utilising the cloud to undertake website testing, and thus avoiding peak period website casualties

ARCHIE ROBOOSTOFF PRODUCT DIRECTOR

MICRO FOCUS,

O


THOUGHT LEADERSHIP

CLOUD BASED TESTING Cloud based testing is an alternative to traditional solutions, which significantly reduces both the initial and ongoing costs, without compromising any of the rigour that is required.

Cloud based performance testing will ensure capacity even in the most extreme scenarios. The cloud infrastructure allows for real life testing for peak loads, using generators. This eliminates the effort and cost related to extending the on premise test infrastructure, which only the highest load scenarios would need.

In addition, cloud based services can provide a diagnosis of any performance related issues when they arise – giving teams the detailed diagnostics they need to pinpoint the nature and location of the problem. Combined with an on premise performance monitor, it’s straightforward to understand the demands on the server infrastructure in the data centre, providing end-to-end transparency.

BENEFITS OF CLOUD BASED TESTINGCloud-based performance testing offers a multitude of benefits to support the business without disruption:

•I t is extremely well suited to generating the peak demands required for enterprise performance testing.

• Peak load testing in the cloud takes advantage of the ability to run tests virtually on demand. Businesses simply schedule time for a test, and resources are automatically provisioned. This makes scheduling more flexible, helping to eradicate what can often be long delays with traditional testing, as internally managed hardware is deployed and verified by the IT department.

• Using cloud technologies can enable the performance management team to not only evaluate applications’

global readiness, but to conduct tests across the globe, by replicating virtual users

in a variety of locations, to ensure the website can handle users far and wide.

• The elasticity of the cloud means that you can scale computing resources up or down as needed, to ensure website performance is affordable. Using utility style pricing, businesses only pay for what they use. In comparison, to utilise a traditional solely on premise model, a company would have to acquire computing power to support very large user tests for the lifetime of the application.

MOVING TO A HYBRID MODELTo achieve high confidence in production performance, combining cloud capabilities with traditional approaches can often provide the optimal model, offering better agility and economy than using traditional methods alone.

Businesses should look to determine early on if a mixed cloud model is most suitable to their company – for example, a hybrid cloud that combines internet protocols with support for .NET, Java, Oracle, SAP, Siebel, COM and other enterprise application protocols.

Cloud-based testing meanwhile is the best environment for testing web 2.0 applications like AJAX, Silverlight and Flex, as more computing power is required to perform these more complex tests.

By implementing a performance testing solution via the cloud, software test teams can more effectively and affordably manage heavy loads on the company’s website and, as a result, the marketing department won’t suffer from wasted marketing efforts and low sales – to secure the business’ future success.

CLOUD BASED TESTING IS AN

ALTERNATIVE TO TRADITIONAL SOLUTIONS,

WHICH SIGNIFICANTLY REDUCES BOTH THE INITIAL

AND ONGOING COSTS, WITHOUT COMPROMISING

ANY OF THE RIGOUR THAT IS REQUIRED


SUPPLIER PROFILE

THIBAUD BUSSIÈRE CEO AND CO-FOUNDER

NEOTYS

THERE IS A CLEAR REQUIREMENT

FOR AGILITY IN THE PERFORMANCE TESTING

MARKET, SO THAT TESTING DOESN’T BECOME A BOTTLENECK.

PERFORMANCE TESTERS ARE EXPECTING SOLUTIONS THAT WILL HELP THEM BE FAST AND AUTOMATED, AND THAT FIT INTO AGILE DEVELOPMENT

PROCESSES


erformance testing has evolved a lot in recent years”, says Thibaud Bussière, CEO and co-founder of software company Neotys. “It used to be a discipline entirely owned by specialists, a sort of ‘black box’, whereas

now, performance is a strategic issue, which needs to be shared across teams. Because the user experience is at stake, project and product managers are involved.

“This has changed the way performance testing is managed. We are seeing more and more projects where performance KPIs are defined in the very early stages of the development cycle, and integrate the end user perspective, where the first application components are tested well before the application is fully assembled.”

Neotys was founded in 2005 when, says Thibaud, “We couldn’t find a testing solution on the market that would both help us solve performance issues, and match our needs in terms of technology support and budget. So we decided to create NeoLoad.”

A new breed of load and performance testing tool for web applications, NeoLoad was the first software to provide its type of automated design capabilities and ergonomic graphical user interface, enabling testers to test faster, but also allowing non-experts, like developers, to start testing under load.

NeoLoad has evolved since its inception, becoming the first to support new technologies like Flex, Google Web Toolkit, SPDY, and lately WebSocket. Keeping the focus firmly on providing continuous innovation in the field of load and performance testing has ensured that NeoLoad has remained successful. Today, it is adopted by more than 1600 organisations globally, including small and mid-sized companies, who appreciate the virtues of this powerful, easy, affordable and hyper-effective tool; and also some of the world’s largest organisations, who value this enterprise grade solution, the unrivalled TCO it enables, and the outstanding level of support provided.

Says Thibaud, “Many large accounts, who initially selected NeoLoad for a specific project have gone on to replace their legacy load testing solution at a corporate level, because NeoLoad was more efficient and provided a far better ROI.”

From its origins with NeoLoad, Neotys has gone on to serve QA, R&D, and operations teams with products that provide advanced features to support the latest technologies, which are easy to utilise, and enable users to deal with performance in agile contexts. With customers in over 60 countries, and dedicated operations in Europe, the USA, and India, Neotys works with a network of service partners around the world, including large system integrators like Atos, CGI, CapGemini and TCS, and also smaller consultancies that are highly specialised in performance testing.

“We are committed to delivering new products to the market”, says Thibaud. “We offer one of the best cloud testing platforms available, enabling users to access the infrastructure of many cloud providers around the world, from a centralised location, and we recently launched our application monitoring product, NeoSense. It’s designed to serve the needs of operations teams, who are a new type of user for us.

“Our vision is that performance matters for everyone involved in the product lifecycle, from project managers, to R&D, QA and operations. Providing innovative products around performance will continue to drive our growth in the coming years.

“Performance testing and monitoring is a highly complex discipline. Projects are often long, and associated with relatively high costs, in terms of engineering resources, software licenses and test infrastructure. In a context where development cycles and time to market are getting shorter, there is a clear requirement for agility in the performance testing market, so that testing doesn’t become a bottleneck. Performance testers are expecting solutions that will help them be fast and automated, and that fit into agile development processes.

“We are also seeing a growing demand for mobile performance testing. Web and mobile are hardwired into Neotys’ DNA, and we are committed to supporting the latest technologies in these areas. For example, we were the first to support WebSocket, and the first to provide a full mobile testing solution, including mobile network emulation.

“Our products help our customers to make difficult tasks easier, so they can isolate and fix performance issues earlier. This way, the domain of performance can be seen as a value for the software delivery chain, and not a source of problems.

“When people rate the overall end user experience these days, performance is the main factor. So performance testing has shifted from a ‘nice to have’ step, put off until the end of the project, to a strategic dimension. Applications are getting more and more complex, and test cycles are getting shorter. To deal with this, testers tend to automate performance testing as much as possible.

“Our technological expertise around web protocols and frameworks allows our customers to go very deeply into the testing and monitoring of their applications, even in more complex scenarios, while the quality of our products, and the support we offer, have enabled Neotys to maintain a steady organic growth, growing our client base, and enlarging our footprint with our existing customers.”

SUPPLIER PROFILE

PERFORMANCE MATTERSThibaud Bussière, CEO and co-founder of Neotys, tells David Land about how the software company has evolved, and its current place in the market

"P


TESTING TIMESSophie-Marie Odum, previous Editor of TEST Magazine, reflects on her time within the software testing industry, and talks to industry leaders about their predictions for the next 12-18 months

SOPHIE-MARIE ODUM PREVIOUS EDITOR

TEST MAGAZINE

LOOKING AHEAD … AND BACK!


gile, automation, and offshore testing are just some of the phrases that I learnt during my time as Editor of TEST Magazine, as well as a plethora of acronyms, of course!

I must admit that, when I started, my knowledge of software testing was limited, but I’ve always loved a challenge. I was guilty of thinking that testing and checking were one and the same but, as I acquainted myself with the industry, I soon learnt the difference. One of my early formative reads was the blog, Testing and Checking Refined, by James Bach and Michael Bolton – referred to as the ‘Godfathers of the industry’.

I quickly learnt that, while software testing is a unique field, which bridges the gap between business requirements and technical specifications, there is more than this to the average software tester. It’s not a career that a person chooses … rather, it’s the other way round. Born with an ability to identify defects, software testers have an inquisitive mind, and the right attitude to dive into something and find a way to break it.

Having come to realise that the software testing department within an organisation is vital, as companies won’t generally allow software to be released until it has been fully tested, I have a newfound respect and appreciation for software testers. But I also realise that, paradoxically, even though they are the gatekeepers to software quality, software testers aren’t always highly regarded within their organisations.

It’s been great to learn about, and investigate, the testing and development of new and futuristic technologies. Reflecting on the articles and interviews I have written for TEST, in addition to my first cover story in August 2013 (for sentimental reasons), my favourite is the April 2014 cover story, where I investigated the testing behind Biyo, the all-in-one PoS, which allows businesses to link their credit cards to their palms, closely followed by the CrowdEmotion (facial coding technology) and the Reset The Net articles, both in August 2014.

CHANGES IN THE INDUSTRYBack in July last year, it seemed the big debate in the industry was onshore vs offshore testing. At the time, the biggest argument for offshore testing was lower costs, but some felt that quality was compromised, and that it goes against agile processes. The debate continues, and will probably do so for a while.

With the rise of smart devices, mobile app testing was also a hot topic. And now, as we usher in a new era of the Internet of Things, we will see the role of software

testers evolve again, as many more ‘things’ will need to be tested. And, although I am leaving the industry, I’m interested in seeing what will happen with the new ISO 29919 standard …

During my time as Editor of TEST, it has been my privilege to work with some of the industry’s top players. I asked some of them for their thoughts on where the industry is now, and their predictions for what the next 12-18 months hold.

Says Shane Kelly, Head of QA and Test at William Hill, and Editorial Advisory Board member, “Now that automation is part of how we test, and not just an afterthought once a product has been delivered, the role of the Software Development Engineer in Test (SDET) in the Software Delivery Team has become even more important. This has been positive for the test industry as a whole, as we have been able to expand test coverage but reduce timelines, testing early and testing often.”

Says Rod Armstrong, Freelance Assurance Director, and Editorial Advisory Board member, “The value of good testing is now being more widely accepted. We are seeing the convergence of roles, and moving to a whole team testing approach: testers code and coders test. This has a positive impact, as all engineers learn new skills, and overall quality of output increases.”

But what of the next 12-18 months? Says Shane Kelly, “Until the universities start to add testing to their curricula, hiring a SDET will become increasingly difficult. And, as the laws of supply and demand dictate, this means that SDETs will become more expensive.”

Director of Development Services at Sega West, and Editorial Advisory Board member Jim Woods, adds, “This is a cyclical process. We will see increased complexity and change to processes and technology, which will require us to look again at how we test. SDETs are here to stay, and will become an increasingly valuable commodity.”

Rod Armstrong concurs, predicting that there will be, “… further convergence of roles, increased focus on automation at lower levels of the code base, and less emphasis on fullstack (e2e) automation within a browser session.”

As a word of advice, Rod adds, “Software delivery methods are changing fast. We all must be aware of this, and be part of the transition. Old school practices will fade over time. Make sure you stay relevant.”

Says James Murphy, Head of QA at Guardian News and Media, and TESTA judge, “Over the next 12-18 months, as the responsive enterprise model takes hold, agile will

A




be embraced by entire organisations, not just software development teams. Data driven decision making and fast feedback loops are being applied at all levels, enabling organisations to be responsive, react to changes quickly, and gain a competitive edge. Test teams must also learn from this, and ensure that they are equipped with the right skills and processes to remain relevant in these structures.

“Continuous Delivery, DevOps and A/B Testing enable teams to measure and understand the performance of their products in real time, while also enabling them to make optimisations to their products and measure the results.

“Fast deployments/roll-backs, automated testing, and real time monitoring mean that many organisations can not only get their products to market more quickly, but are also better equipped to deal with failure when it inevitably occurs. To work within these structures, testers need to be technical, and ensure that they are delivering value.

“As data moves to the cloud, many companies are choosing not to own their own data centres, and are instead turning to cloud providers such as Amazon Web Services to manage their infrastructure. The cloud enables teams to add computing and storage capacity quickly and seamlessly, which allows businesses to focus on the development of their products.

“This has provided a number of benefits, including reduced infrastructure costs, improved productivity of development teams, and better scalability. This has been of particular benefit for test teams who can use the cloud to improve their productivity. Services such Saucelabs and Browserstack make both automated and manual cross-browsers testing extremely easy.

“Another trend is reactive application engineering. With the increasing use of mobile and wearable technologies, and ever increasing expectations from users, it is quickly becoming apparent that the software architectures of yesterday aren’t capable of meeting the demands of today. Reactive systems are those that are designed to meet these new demands, by being engineered to be flexible, loosely coupled and scalable. We will be hearing a lot more about reactive systems in the coming months.”

AWARDS AND CONFERENCEThrough the magazine, websites, TESTA, the National Software Testing Conference and the Software Testing

Network, the team behind TEST Magazine has been working to serve and unite the industry for the past eight years.

Says Shane Kelly, “I always think it’s good to celebrate success and, as an

established industry, to recognise TEST Magazine is very important.”

Says James Murphy, “Software is ubiquitous in today’s world, and it plays a major role in both our work

and personal lives. Given the pervasiveness of software and how dependent many of us are on it, it is essential that those who can influence the quality of software keep their knowledge and skills up to date. TEST Magazine and the National Software Testing Conference are great ways to learn about new and improved approaches to the field of software quality, while TESTA is a fantastic way to recognise and reward the work being done in the industry.”

Jim Woods adds, “The conference provides a forum where we can start to share knowledge and ideas. Hopefully, this will be ongoing, and will continue to have a positive impact on this industry.”

Rod Armstrong meanwhile reflects, “These events have created an environment where we can celebrate our discipline, as well as getting an opportunity to hear what others are doing. Being part of the Editorial Advisory Board has been a great experience, enabling the opportunity to meet with likeminded people, and to ensure that the readership has a chance to read articles that are of interest, and sometimes beyond their current knowledge or understanding – perhaps upsetting the status quo!”

MISSION COMPLETEDThrough editing TEST Magazine, the recruitment of speakers for TESTA’s Judging Panel and the National Software Testing Conference, and the formation of the Editorial Advisory Board, it’s been a pleasure and personal triumph to bring the industry together.

I hope you feel I have introduced new and exciting ideas to TEST Magazine, and that it has continued to be the voice of the software testing industry. I would like to offer you my thanks for all for your support and help during my tenure. I feel privileged to have worked with so many dedicated professionals, who have such a passion for this industry.

When I started, my aim was to make a mark on the software testing industry. I believe my mission is complete.

THROUGH EDITING TEST MAGAZINE, THE

RECRUITMENT OF SPEAKERS FOR TESTA’S JUDGING PANEL

AND THE NATIONAL SOFTWARE TESTING CONFERENCE, AND THE FORMATION OF THE EDITORIAL BOARD, IT’S BEEN A PLEASURE

AND PERSONAL TRIUMPH TO BRING THE INDUSTRY

TOGETHER

Software

When it comes to crash testingWe’re no dummies

With access to digital content so readily available to your customers with the likes of Android, Apple and Microsoft putting instant access into the hands of more and more users across a huge range of devices every day, it has never been more important that your software works without failing. We’ve years of expertise ready and waiting to give you the simple and effective testing solutions you need to make sure your software and content delivers a bug-free, seamless and intuitive experience for your customers.

For further information and to download our complementary white paper on software testing please visit:

www.testroniclabs.com

Comprehensive Software QAManual and Automated Testing • Test Strategy • Compatibility • Back Office Validation • Mobile Testing • Website and Browser • HTML • Exploratory and Technical Testing • App Store Compliance • Load and Performance • Accessibility • User Experience

Testronic_TestMag_23x31.5.indd 1 16/04/2014 12:39


n the past, cost arbitrage used to be the main reason for using an IT service supplier: it was cheaper, and often more convenient, to use an offshore, outsourced testing partner. But as services providers have become more

knowledgeable about their customers’ business processes and underlying applications, the paradigm has started to shift to an outcome based perception of value.

A good example of this change is the manner in which vendors are starting to charge these days. In tandem with this, the concept of testing as a service is shifting from a predominantly ‘time and materials’ type of customer engagement, to a more ownership based, results oriented business model.

The traditional time and materials model reflected the number of people used and the hours of work delivered.

This then moved to a fixed price model which, while more milestone based, was still largely focused on resources used and the duration of the work. And now, we are moving to the next level where, from a quality assurance perspective, what the customer expects at the end of a project is that the IT system which has been built and tested satisfies the business need for which it was specified.

The number of test cases executed and bugs found is immaterial: what defines value is that, at the point when the system is built and has the go ahead, there are absolutely no problems, glitches or issues that impact on its deployment and smooth operation.

The value of this new approach comes from the higher quality of the systems delivered. Typically, they have fewer defects and less downtime, meaning software that is truly fit for purpose, which integrates seamlessly with the business.

STATES OF EVOLUTIONCustomer centred organisations are in different states of evolution. For those seeing cost arbitrage as their main driver, the time and materials engagement model remains quite effective. Others have moved on to the next level, and let the vendors take the bottom line in delivering the project, while yet other customer focused organisations have started to view their IT projects as a way of delivering increased business value, and have moved away from the simple calculation of the number of test cases and defects found, because these can only ever be an indirect way of delivering value. What really counts is whether the system was built as the business wanted it, and whether it works the way they wanted it to work.

BUILDING ADDITIONAL VALUEOnce we have baselined the current level of quality, how can we build additional value by increasing stability? We measure defect removal efficiency, and cost of quality and time to market improvements, to give us a full quantitative picture of where we need to be. A proportion of the fee is only paid when quantitative targets, agreed jointly between

the vendor and the customer, have been met. This goes some way toward eliminating issues with previous models, where the services vendor could benefit from the work continuing as long as possible.

With this new type of engagement – where vendors are committing to the quantitative values of cost, quality and time to market improvement – the end result becomes the business value of the IT system. Domain knowledge and competence are crucial parts of the new process for the vendor. The level of involvement depends on how many competence gaps exist, where they are, and what value for the customer additional skills can add.

Everyone involved in the process becomes a stakeholder, because if the quality is poor, it could impact the quantitative targets of all those taking part. Now there is a sense of ownership and contributing to what really counts for the business. Thus, there is more impetus for the whole team to really deliver.

There is joint ownership of the project, and the services vendor is no longer just an order taker, but has to understand the bigger picture – the key factors that make the customer more successful. Customer engagement is also richer under this approach. Previously, the service vendor was only worried about the tasks they had to perform, but now they go a step further and strive to ensure that their customers are successful with their own customers.

The real paradigm shift lies in the fact that service vendors need to make sure they are successful as a complete IT team, with the business requirements in mind. It’s about understanding and delivering the end value using a more collaborative approach. It is a joint way of looking at where the gaps are in delivering the required business value.

The new approach moves organisations that use it into a higher level of added value, and can offer them a real competitive edge. While doing this, it also offers the customer better metrics, quantifying the benefits that really matter to them through more detailed reporting.

Service providers that are looking at using the new outcome based value approach are able to move the quality assurance process to the next level, making business systems more stable, and helping the customer gain value from their IT implementations, and thus a quicker return on investment.

The organisations that find this engagement model to be a viable approach have accepted it wholeheartedly, because it offers a direct measure of business value. If they can adopt the business knowledge, as well as the process understanding and the associated best practices, it is set to become one of the main approaches that testing services vendors use. By doing so, they can ultimately become ‘off the shelf’ offerings for the vendor. And, as these practices become more widespread, the testing process is bound to become more specialised, and to add more value.

ADDED VALUEGlobal Head of Testing Practice at ITC Infotech Vijay Balasubramaniam discusses how testing is moving beyond the traditional time and materials based model, to a more outcome based perception of value

I

VIJAY BALASUBRAMANIAM GLOBAL HEAD OF TESTING PRACTICEITC INFOTECH

VIEWPOINT


LAST WORD

o Document or Not To Document?Sorry Mr Shakespeare.

One of the major tenets of the agile movement is that the testing process is supposed to be free of documentation, but I

prefer documentation-lite to documentation-free. Not just because I’m old and need everything written down. I do. But that’s not the point.

I hate documenting things. Writing test cases, or test plans, is definitely one of my least favorite activities. Unfortunately, it’s also necessary. Much more necessary today. So, as much as I hate it, document I must. The question is: how much documentation? The answer: it depends.

Don’t you just love a solid, concrete answer? Unfortunately, that’s the best I can do.

As with any type of communication, you need to consider your audience. What do they know? What is their background? What kind of experience do they have? What kind of experience do they need? Let me try to put things in perspective a bit.

I mentioned in a previous article that I have recently been given the task of testing an application for mobile devices. The first round of testing was on the iOS platform: iPads, iPods, and iPhones. Believe it or not, it was pretty straightforward. But now, it’s time to tackle Android.

Testing anything on Android is a huge task! There’s no way that I can tackle this daunting task alone. I need help ... and lots of it! Other people are going to need to be recruited to test.

My recruits will most likely not be experienced testers. As much as I hate to do it, I’m going to need to write test cases! Worse, I need to write them in such a way that anyone can follow them.

My ‘testers’ will need to know how to use a mobile device such as a phone or a tablet, but I can’t guarantee that they will.

What to do? I need to write test cases that are somewhat specific to cover the functionality adequately, but also somewhat broad, in that they can’t be specific to any particular device. And the tests need to be simple enough that they can be followed by any ‘tester’ with unknown experience, with any given device. Piece o’ cake? Not so much.

On the bright side, it’s been like all my Christmas and birthdays have come at once. I have a desk full of Android devices. I’ve got them all set up and ready to use for testing. Time to distribute them to my ‘test team’. Let the fun begin!

A COUPLE OF LESSONS: • Don’t assume anything. Just because someone can use a device from one vendor doesn’t mean they can use a device from another. I’ve been inundated with phone calls. How do I turn it on? How do I charge the battery? The screen went black, how to I get it back? Where is the ‘back’ button? Where is the keyboard?

• When in doubt, find a 10 year old. Kids just seem to ‘get’ technology. It makes sense to them. The rest of us need training – desperately! I work in the industry, and I don’t get it. Teach me like I’m a 10 year old.

My kids are actually older (much older), but they recently made the iOS-to-Android transition. They sat me down and gave me a lesson in Android 101. I feel better. But still ... document it!

So I’m back to the beginning. I need to write much better and more detailed test cases. Yes, I said ‘write’.

I also need to assume the recruited testers will have little-to-no experience with a given test device. If they do – awesome! Let’s assume they don’t. For the experienced user, the test cases may seem too detailed. To an inexperienced user, they may seem too vague, or not specific enough.

As my communications teacher always told me – write to the most inexperienced reader. I’m going to assume that they know very little about the devices. They may not know the difference between a short or long pull. What’s a ‘tap’? What’s a ‘drawer’? Swipe left? Swipe right? Icon? Tab bar?

DOCUMENT! DOCUMENT! DOCUMENT!As much as I may hate it, I need to write this stuff down. I’m also going to need to do a little training. We have a pretty experienced group of employees, with experience on both iOS and Android. They are very loyal to their devices. Android people don’t like iOS people, and vice versa.

Rather than play referee, I chose to let the Android people test Android, and iOS people test iOS, although I did grab a few people to test on both devices, to get some feedback on usability across devices.

We learnt a few things when it came to the user interface. The differences in terminology proved difficult. While I tried to keep most of the steps very generic, it wasn’t easy. I tell the user to select a record rather than tap it. I avoided terms like long pull or short pull. Instead, I told them to refresh the page. It was actually pretty successful in the end.

Remember when phones just called people and people called you? Me neither.

DAVE WHALENPRESIDENT AND SENIOR SOFTWARE ENTOMOLOGIST

WHALEN TECHNOLOGIESHTTP://SOFTWAREENTOMOLOGIST.WORDPRESS.COM

DOCUMENTARY EVIDENCEDespite his general adherence to the tenets of the agile movement, with its belief that testing should be documentation free, Dave Whalen prefers documentation lite to documentation free. Here, he tells us why

T

www .s oftware t es t i ngc onference . c om

• Two-day Event

• UK-based with a European reach• Speakers include the winners from The

European Software Testing Awards (TESTA)

• High profile round-table debate sessions

19-20 May 2015, The British Museum, London

National Software Testing ConferenceBreaking today’s boundaries to shape tomorrow

201

5

Headline Sponsor Supported by

MA

GA

ZIN

E

• A leading exhibition showcasing the latest products and services

• Supported by the industry leading journal, TEST Magazine

• Taking place in May 2015

• Multiple streams covering an array of testing subjects

Exhibitor

Customer expectations are soaring. So your apps and websites must work

right first time on any platform or device – anywhere in the world.

Cross-browser test in the Cloud with Borland and we can help you to

ensure consistent user experiences on a huge variety of browsers and

devices. What’s more, we increase your test coverage, improve visibility,

and reduce costs through automation.

See how Borland overcome the barriers to great user experiences by

allowing you to easily maintain app quality and performance while

managing costs.

Visit www.borland.com/anydevice and download our FREE White Paper:

The Mobile Multiplier Effect.

SUCCESSFUL SOFTWARE DELIVERY

GIVE EVERYONE EXCEPTIONALUSER EXPERIENCES ON ANY DEVICE TEST WITH BORLAND

Copyright© 2014 Micro Focus. All Rights Reserved.Portions Copyright © 1994-2009 Borland Software Corporation (a Micro Focus company).

UX_PressAd_a4_v1.4.indd 1 23/09/2014 10:35

test magazine - january 2015

Documents

testing services

software lifecycle

software qualityvolume

software industry

tata consultancy services

assurance services unit

testing function

quality assurance space