presentation bio return to main menu w6 · (ims tcp/ip otma connector) web/application server...

P R E S E N T A T I O N

International Conference On

Software Testing, Analysis & ReviewDEC 4-8, 2000 • COPENHAGEN, DENMARK

Presentation

Bio

Return to Main Menu W6

Wednesday, Dec 6, 2000

e-Business Testing:

Rainer Pirker & Andreas Rudolf

Focus on User Perception

IBM Global ServicesIBM Global Services© Copyright 2000 by IBM Corporation. All Rights Reserved

Rainer Pirker / Andreas RudolfIT Consultants

E-Business Testing:E-Business Testing:Focusing on User PerceptionFocusing on User Perception

EuroSTAR 2000 - CopenhagenEuroSTAR 2000 - Copenhagen

IBM Global Services© Copyright 2000 by IBM Corporation. All Rights Reserved

Presentation overview

Our e-business experience

Why focus on quality?

Key quality aspects for e-business solutions

Complexity of e-business solutions

Case Study:e-billing project / Telecomtest strategytest concepttest casesskills neededtest environment and toolscurrent project status

Critical success factors

Any questions ...


We have had experience of conducting large scale testing projects in many different industries

Our Department is strongly focused on:Test consulting (e-business, EURO, Y2k, new product developments etc.)Application Development Effectiveness consulting (supporting the customer to assess and improve their application development processes)

Our working experience is based on:a dozen large e-business/Y2k/EURO test projects since 1996 where we worked as consultants and as test managersdozens of projects for marketing and workshops for customerprojects in many industries: finance, insurance, transport, utilities


The traditional order placing and order receiving process is time consuming and error prone

1) Printing order

5) Typing in order, communicate with back-end system

Company A Company B

2) Carry order to fax machine4) Picking up

& Reading order

3) Faxing order

opt. 6) clarify order


In the e-business world the order process is fast but security and performance issues have to be addressed

Company A Company B

1) Submit order electronically

opt. 3) Handling exception by e-mail or phone

Back-end system

Firewall

2) Process order electronically


Only if you address the e-business risks from the beginning is a high return on investment in sight

Traditional solution:(+) creative, experienced human being, good exception handling(-) expensive workforce, processing is mainly done by humans (-) medium data quality, frequent 'media breaks'(+) "Human" firewall, resistant against "hacking"(-) service only during business hours

e-business solution:(RISK) single path, weak exception handling without human support(+) Processing is done by computers(+) high data quality and no media breaks (single electronic process)(RISK) hacking and spoofing are possible without a good security solution(RISK) customer expectations are 24h x 7 days service

To target these e-business risks, testing is key!


Key quality aspects for e-business solutions are:scaleability/performance, security, usability and reliability

Scaleability and

Performance

Usability

e-business quality

Security

Reliability

user perception


Key is how users perceive the performance of your site and service

Scalability and Performance:Permit a maximum of 8 seconds for loading a web page. When the loading time increases just 10% the bailout rate goes from 6% to 40% Knowing your requirements is key (number of concurrent users, user profiles, desired response time, throughput, type of sessions, transaction mix, modem speed etc.)Focus on the user's perception of performance:

loading the whole java applet vs. loading JIT the classes needed for each transactionloading large HTML tables vs. loading smaller ones as required by the userthe user will accept longer download times when the site and form are perceived as secure


Taking care of your customer's security fears is a further key to success

User security fearsSecurity perception is key!Good: inform the user of your security solution (SSL, length of key used for encryption etc.)Bad: your site's security problems are made public

Only a customer who trusts your e-business will continue to do business with you!

Company securitypassive Web attacks: wire tapping of password and data, analysis of the structure of the data streamactive Web attacks: spoof, mutilate and delete data, password guessingUse a secure system architecture (i.e. firewalls)For sensitive data (credit card numbers) use a secure protocol like SSL, a secure protocol like SSL can have a big performance impact if not handled by cypto hardwareDo not allow access your companies internal functions. Do not test call center interfaces over the internet


Usability is also key as users need to quickly and easily find the information they seek

Usability aspects:Navigation: usability is improved when the site supports a search facility, a site map and buttons for navigating to and from each page. The site should never contain any broken links.User interactions: the user's ability to interact with the system must be kept very simple and easy i.e. "one click" shopping.Help messages: especially business to customer (B2C) users cannot be trained in the normal way. Extensive help messages and FAQs are usefulComputer literacy: users find it important to be able to quickly locate the information they require on a Web page - graphical design is not the crucial point for most e-business users. Printing: users tend to print Web pages. Reading a page on the screen is about 25% slower than reading a hardcopy of the same textSupport channel: however good your usability is, strong e-mail and telephone support remains a necessity


If an error happens, a normal user has to understand what he has to do

After a search: What is the value for the user?

Is this understandable?

How should the user react?

No contact information is given!


Configuration testing is important also for "normal" HTML pages

HTML file generated from Word 97 and copied into existing template:

MS IE 5: Correct, headers bold, body text normal

Netscape Navigator 4.72: not correct, everything bold


24x7 operation and the high visibility of e-business solutions drives reliability requirements ever higher

Reliability:B2C: customers from across the world expect 24h x 7days operations which leaves little time for even short service windowsCustomers expect short down times and high mean time between failure rates (MTBF)Outages have a serious business impact:American Express outage cost: 167,000 USD / min Charles Schwab outage cost: 1M USD / min


The complexity of e-business architectures is such that end to end performance tests are definitely needed

The structure of an e-business solution is: a three-tiered architecture similar to Client/Server PLUS:

firewallsdistributed objectsauthentication, credit card verificationISPsetc.

HOST S/390

ITOC(IMS TCP/IP OTMA

Connector)

Web/ApplicationServer

AuthorisationDatabase

IMS

ITOC (backup)(IMS TCP/IP OTMA

Connector)

IMS (backup)

Firewall

Call CenterAppl.

Telecommunication Provider Intranet

Internet / e-billing inquiryUser

WebbrowserSimulator

WebbrowserSimulator

HT

ML

There are two basic types of e-business solution:new e-business solutions build new from ground up

Risk: high -> completely new system

Web enablement solutions of existing legacy applications (e-legacy)

Risk: medium: -> interface to existing production systems


Our case study is based on a telecommunication provider's first e-business project

The customerTelecommunication company5M customersTheir first e-business project

The projecte-billing solution which enables customers to obtain their telecommunication bill via the internetpilot targeted 500 customers

IBMA request to independently validate the solution was not received until after the pilot implementation had been completed


For rapid e-business testing a clearly defined approach is the foundation for success

Assessingthe

base

Defining thetest strategy

Defining thetest concept

Start End

Building upthe test

environment

Executing thetest cases

Defining thetest cases

This approach has been proven in many successful projects - at this level there is nothing e-business specific in there

e-business testing approach


First we assessed the existing base in terms of test cases, documentation, environment and tools

Assessing the base situation:Test cases: there were none documentedUser documentation: an user handbook was not available however, the functional specification was in good shapeRequirements, design and product documentation: The functional specifications were sufficient however, the non-functional requirements, like performance, transaction rate, reliability etc. were missingTest environment: we agreed with the customer that an isolated test environment should be built. Only the mainframe connection was to the production environment.

Assessingthe

base



Start End

Building upthe test

environment




Then we defined the test strategy, the heart of any testing project

Defining the test strategyDraw a system structure diagram - a diagram of the architecture was not available so we draw the picture to obtain an insight into possible performance bottlenecks and security implicationsFast start: a very tight time schedule of about 4 weeks forced us to do a really fast start in order to finish the engagement in timePerformance tests first: due to the customer's priorities we had to swap around the traditional approach and do the performance testing before functional testingBlack box end to end test: the customer was interested in the end to end results and how the end users perceived the system's performance

Assessingthe

base


Defining thetest conceptStart End

Building upthe test

environment




In the test concept we focus on What and How

Defining the test conceptDefine the What and HowHighlight 1: Check functional and non functional requirements - for the functional requirements the application and it's specifications were available and sufficient. Non functional requirements like performance, response time, transaction mix, reliability etc. were only partially available and had to be detailed out by us.Highlight 2: Define test cases for measuring end to end performance and for checking system security

Assessingthe

base


Definingthe testconcept

Start End

Building upthe test

environment


Definingthe testcases

Defining the test casesThe test cases were for business transactions which we found documented in the functional specifications and by exploring the live applicationHeadlines for the test cases were documented in MS Excel


The test environment has to be isolated from the production environment to get reproducible results

Building the test environmentTest environment: a dedicated and isolated test environment was built quite separate from production except for the connection to the mainframeFor defect tracking we used a Web based tool as the developers, customers and testers all worked at different locations.

Assessingthe

base



Start End

Building upthe test

environment


Defining the testcases

e-billing SubjectMatter Experts(Telco provider)

Software-developer(Software-

house)

IndependentTest Team

(IBM)

Independent TestTeam

(Businesspartner)

Problem-tracking

Internet

e-mail


In the case study the test system was a very close copy of the production system

HOST S/390

ITOC(IMS TCP/IP OTMA

Connector)

Web/ApplicationServer

AuthorisationDatabase

IMS

ITOC (backup)(IMS TCP/IP OTMA

Connector)

IMS (backup)

Firewall

Call CenterAppl.

Telecommunication Provider Intranet

Internet / e-billing inquiryUser

WebbrowserSimulator


Response times were measured for different user profiles and different numbers of concurrent users

0

20

40

60

80

100

120

140

160

5 user 10 user 20 user

Number of concurrent user

resp

on

se t

ime

(sec

)

Logon Display bill Bill comparison Display detailed bill Save bill Logoff

Response time of the different e-billing transactions


End to end measurements identified the application server (data conversion) as the performance bottleneck

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Logon Display bill Display detailed bill

e-billing application Authorisation/Logging Database Mainframe

Response time distribution of different e-billing transactions measured at the application server


Within four weeks the test was completed and the final report given to the customer

During this time the independent test team accomplished all of the planned tasks:

Test strategy definedTest plan definedEnd to End Performance Test cases executedMetrics and final report delivered


Quality testing had shown up a very large gap between actual performance and the requirements

The following parameters were measured on the web application server after it was identified early as the performance critical system in the architecture:

CPU LoadVirtual Memory Size

The performance results were about a factor of 50 worse than requirements. The development team was unable to quicklydetermine what needed to be done to meet requirements. The projectwas therefore suspended and restarted with a new development team, project management and quality assurance brief.

Metrics and reports delivered

Response times per transaction type and the number of concurrent users

Response times per transaction type for different user profiles - private customers, small- , medium businesses

To obtain a measure for user perceived performance, we measured the time before the first HTTP data packet arrived. This was interesting for transactions such as bill display were a large amount of data was being transferred. The user himself can see the first page of the bill before the download has finished. This depends on the HTML data stream.


Focusing on quality is key for e-business solutions

Summary and Critical Success Factors (1): Security issues can kill your e-business. Address these issues carefully in advance - use secure protocols, disclosures for the testers, etc..Non functional requirements: define then early in the projectUse test cases based on business transactionsUse an independent test team focused only on the requirements - testers who work also as developers do not usually have that "black box" view of the systemE-business test tool skills: it is necessary to have experience of Web based problem management and browser simulator tools if you need to deploy them fastQuality cannot be tested into a product. Plan for quality from the beginning using, for example, IBM's Lifecycle Testing approach


Focusing on quality is key for e-business solutions

Summary and Critical Success Factors (2): Web content development and e-business solution application development are quite different. Application development requires a strong quality, requirements and project oriented approach Test the performance of your e-business solution under real world conditions (end to end tests, load the server with the right transaction mix, SSL) and in a near real world test environmentUse Web based tools for problem management and test case documentationUser profiles: always keep in mind that there are a lot of different types of users out on the Web who are accessing your e-businessPlan and execute performance tests during all of the solution development phases


Any questions?

How to contact us:

Andreas Rudolf

e-mail:[email protected]

Phone: +43-1-21145-4347

Fax: +43-1-21145-4191

Rainer Pirker

e-mail:[email protected]

Phone: +43-1-21145-4163

Fax: +43-1-21145-4191

http://www.geocities.com/ad_consultants

Wednesday 6 December 2000

W 6 e-Business Testing: Focus on User Perception Rainer Pirker

Rainer Pirker is a consultant with IBM Application Development Effectiveness Practice. He specializes in test concepts and strategies to improve software quality. He has 15 years of experience in information system and application development, and has in-depth knowledge of banking, insurance and public institutions in the fields of executing, developing and improving tests.

Within IBM Rainer Pirker started as a tester and system specialist for a workflow product, became a test leader for a business application and is a test consultant, especially for application development process improvement in testing, e-business, EURO and Y2K problems.

From June 2000 he is test manager in a new development e-business project – this customer project is completely outsource to IBM and the topic is internet payment. The development team consist of about 20 people and the environment is WebSphere. Rainer is responsible for Unit tests up to extensive Performance- and Security tests, including multi platforms and integration of SAP.

Rainer has developed and presented several testing workshops at customers and IBM internal, creating some follow up business at customer sites.

He designed and implemented a test-management tool suite in Lotus Notes and MS Access. This tool suite covers business transaction-, test case-, equivalence classes- and scenario-definition, test case execution and problem-management and is in use at several customers and IBM internal in the central European region.

Before joining IBM, Rainer worked several years for software companies and for public institutions, where he designed and implemented accounting and education software.

Andreas Rudolf Professional Profile

Andreas Rudolf is certified consultant with IBM Application Development Effectiveness Practice. He specializes in concepts and strategies to improving the software application development process, in the last two years his main focus was on test concepts and quality assurance. Mr. Rudolf has over 10 years experience in workflow, application development and -testing. He worked for a variety of industries including telecommunication, banking and insurance. Professional Experience

Currently Mr. Rudolf assesses the application development process for an IT provider with the target to find and to improve the weak points in the requirements and solution validation process.

He was responsible establishing measurements and conducting performance tests for an e-billing project in the telecommunication industry.

He supported a year 2000 test project at a large bank in Germany as test consultant with is expertise in test case creation and execution. Due to the sound and proven methodology used in similar projects, the project finished 6 weeks before schedule.

For a large bank in Austria, Mr. Rudolf has written a concept for testing the whole software portfolio for year 2000 compliance. He built up and led a team of 20 people dedicated to this project. Due to high customer satisfaction with the project, a subsequent contract at this customer led to an assessment of their AD change and problem management processes.

For an insurance company, he built up a performance laboratory for client/server and e-business applications. It will help the client to achieve the performance target and test performance relevant software changes before going to production.

He managed the acceptance test of a central directory access service for a large telecommunication provider. By replacing the old service the customer could reduce his operating costs about 25% and could improve the response time about 20%.

For a telecommunication provider Mr. Rudolf designed the workflow structure of a tracking system for the manual laying of telephone wires. This project lead to an increase of customer satisfaction, since the process cycle time was shortened significantly.

While working for IBMs Vienna Software Development Laboratory he was responsible for the development of a platform and protocol transparent communication layer. His prototype, which integrated IBMs Workflow Manager with the Web, shaped IBMs Workflow strategy and generated customer leads.

Before joining IBM, he worked for a small software development firm, where he designed and implemented a low-cost network for educational use. At a subsequent release he reduced the initial hardware costs about 40%. Education and Personal History

Mr. Rudolf was co-author of the IBM redbook "cooperative processing in an object oriented" environment and he was speaker at international conferences including “Software Quality Week” and “EuroSTAR” with topics in the area of software testing. At the "International Workflow Conference" in Vienna he presented the topic "Workflow meets Cyberspace".

Mr. Rudolf attended the engineering school in Moedling, Austria, where he earned a bachelor degree in telecommunication and electronics. His native language is German and he speaks English fluently.

presentation bio return to main menu w6 · (ims tcp/ip otma connector) web/application server...

Documents