Testing Session

Testing Team-Release Management Team

Testing Session ConclusionsDocumentation

Documentation enhancements required - result codes specification, unspecified behaviour (empty data, etc), more detailed explanation of the logic of the service. Currently, the logic explanation remains vague. Overall, functional testing process went ok. Suggestion that categorisation of error codes could be useful.

Functional testing and installation testing overlap could be better utilized.

Testing manual (once service is installed) was suggested. This was because starting problem was encountered

Actions

ACTION: Assessment of documents and a report on documents required, documents missing, quality of documents for services, and timelines when documents are produced (and when they should be produced)

ACTION: Enhance templates and sample documents to specify what is required based on the experience so far

soapUI

SOAP UI doesn't help with automating generated requests. However, it helps in validating requests and responses against the rnc. It also helps in using java programs (for comparing against a database for example) as well as use of xpath expressions.

SOAP UI requires rnc specification changes to a certain extent. There is still a bit of manual work to configure the tests but once it is setup, it is easy.

Time frame assessment

Using scripts: Getting familiar with the a new product : 1 week Per functionality test scripts

development: ~2 weeks max Per RC testing: 1 day Report generation: included (as log files)

So far, per service testing using scripts (service has 3 functionalities and results in 5 RCs): ~ 7 weeks

with SOAP UI (relies a lot on the schema) Getting familiar with soap UI for a new-comer: 1 week Getting familiar with a new product: 1 week Per functionally test scripts development/setup: ~4 days Per RC testing: 3 hrs Report generation: configurable to generate logs (time required will be estimated)

Using SOAP UI, per service testing (service has 3 functionalities and results in 5 RCs): ~ 4 weeks

ACTION points: investigate SOAP UI (and other tools) and verify the time values and features of SOAP UI (especially reporting)

Performance Testing

Black box performance testing procedures document will be written up which will talk about performance benchmarks against parameters such as configuration (metadata), load, hardware, other processes running in parallel, Poisson distribution of sending requests.

ACTION: Performance testing document (have a look at other performance testing documents) .

ACTION:SOAP UI can help in performance testing?

Installation Testing

Testing team will help with installation testing. This will happen in parallel with the release management team doing installation testing (because Release management team will have a global view)