functional testing montpellier october 2006 perfsonar meeting michalis michael (cynet)
TRANSCRIPT
Functional TestingFunctional Testing
MontpellierMontpellierOctober 2006October 2006perfSONAR MeetingperfSONAR MeetingMichalis Michael (Cynet)Michalis Michael (Cynet)
Functional TestingFunctional Testing
• Service Oriented TestingService Oriented Testing
• Black Box TestingBlack Box Testing
• Behavioural TestingBehavioural Testing
• Automated TestingAutomated Testing
Functional Testing is Functional Testing is necessary to:necessary to:
• Make sure that the service actually Make sure that the service actually worksworks• This is accomplished by a wide range This is accomplished by a wide range of testsof tests• If the tests are successful the service If the tests are successful the service is functioning inside the specified is functioning inside the specified parametersparameters
Functional Testing processFunctional Testing process
• A wide range of requests is fed to the A wide range of requests is fed to the serviceservice
Requests that are well formatted containing valid Requests that are well formatted containing valid informationinformation
Requests that are malformedRequests that are malformed Requests containing invalid or non sufficient Requests containing invalid or non sufficient
informationinformation
• Responses are collected and checked for Responses are collected and checked for validityvalidity
• Responses information are cross-checked Responses information are cross-checked with information inside the service databasewith information inside the service database
•ProcessProcessCreate RequestsCreate Requests
Feed requests into ServiceFeed requests into Service
Log responsesLog responses
Check for validity/Correlate responsesCheck for validity/Correlate responses
against actual DB dataagainst actual DB data
Test PassedTest Passed Test FailedTest Failed
LS Functional TestsLS Functional Tests
• Focus on checking LS response to :Focus on checking LS response to : Well formatted requests containing valid Well formatted requests containing valid
informationinformation Requests that are not well formatted with or with Requests that are not well formatted with or with
out valid informationout valid information Requests that do not contain enough informationRequests that do not contain enough information• Requests with excess information also maybe fedRequests with excess information also maybe fed• Suggestions:Suggestions: Check LSCleanup (will do if time is enough)Check LSCleanup (will do if time is enough) Check the RRD MA against the LSCheck the RRD MA against the LS
A LSRegisterRequest exampleA LSRegisterRequest example
• The request is missing the Data element . The LS should return an error response
LS Functional Test – Parts neededLS Functional Test – Parts needed
• A request building script-class.A request building script-class.
• A script-class that feeds the requests to the A script-class that feeds the requests to the LS and stores the responses from the LS.LS and stores the responses from the LS.
(Requests are send via the SOAPClient class that is (Requests are send via the SOAPClient class that is already available in the release)already available in the release)
• A script-class that processes the responses A script-class that processes the responses (parser) and correlates the results with the actual (parser) and correlates the results with the actual data stored in the database (eXist). Here is data stored in the database (eXist). Here is decided whether the test has succeeded or failed.decided whether the test has succeeded or failed.
What has been done so farWhat has been done so far
• A document was send to the perf-dev A document was send to the perf-dev mailing list, listing the tests that are going to mailing list, listing the tests that are going to be done, in order to get feedback from the be done, in order to get feedback from the developersdevelopers
• Started working on the request generating Started working on the request generating class. Class for the LSRegisterRequest has class. Class for the LSRegisterRequest has been developedbeen developed
TO DOTO DO
• Finish the developing of the classes for Finish the developing of the classes for generating requests for the other types of generating requests for the other types of requestsrequests
• Make sure that the communication with the Make sure that the communication with the LS via the SOAPClient class is workingLS via the SOAPClient class is working
• Last ( but trickier ) the development of the Last ( but trickier ) the development of the script-class for parsing and checking the script-class for parsing and checking the responses from the LSresponses from the LS
Help NeededHelp Needed
• More like RFC documentation ( An accurate More like RFC documentation ( An accurate and detailed documentation is needed for and detailed documentation is needed for the tester to understand the way a service the tester to understand the way a service works and responds).works and responds).
• Maybe some information on how Maybe some information on how independent components of the services independent components of the services work, like the databases i.e.work, like the databases i.e.
• Suggestions for enriching and improving the Suggestions for enriching and improving the scheduled tests. scheduled tests.
An example – RRD MA An example – RRD MA
• Metadata Key RequestMetadata Key Request<?xml version='1.0' encoding='UTF-8'?><nmwg:message type="MetadataKeyRequest" id="mdrq1" … <nmwg:parameters id="msgparam1"> <nmwg:parameter name="authToken">PIONIER-Public</nmwg:parameter> <nmwg:parameter name="timeValue">1127250495</nmwg:parameter> <nmwg:parameter name="timeType">unix</nmwg:parameter> </nmwg:parameters>
<nmwg:metadata id="meta1"> <netutil:subject id="subj1"> <nmwgt:interface> <nmwgt:ifAddress type="ipv4">212.191.224.106</nmwgt:ifAddress> <nmwgt:ifName>ge-2/1/0.102</mwgt:ifName> <nmwgt:direction>out</nmwgt:direction> </nmwgt:interface> </netutil:subject> <nmwg:eventType>utilization</nmwg:eventType> </nmwg:metadata>
<nmwg:data id="1" metadataIdRef="meta1"/> </nmwg:message>
Test categories•Message Format•Message Values•Combination
Both correct and incorrectmessages are fed
An example – RRD MA An example – RRD MA
• Tested messagesTested messages• MetaData Key RequestMetaData Key Request• SetupData RequestSetupData Request• StoreRequestStoreRequest• All examples found inAll examples found in
• http://wiki.perfsonar.net/jra1-wiki/index.php/Schemata_and_Instanceshttp://wiki.perfsonar.net/jra1-wiki/index.php/Schemata_and_Instances
• Helped produce a number of testsHelped produce a number of tests
• Interaction with the LS will also be Interaction with the LS will also be tested. tested.
Current StatusCurrent Status
RRDMA Tests statusRRDMA Tests status1.1. Infrastructure completeInfrastructure complete
The main script - CoordinatorThe main script - Coordinator ExecutesExecutes
The creator script The creator script Creates the requests in xmlCreates the requests in xml filesfiles Notes the validity of the requests Notes the validity of the requests
The request scriptThe request script Actually used the SoapClient ClassActually used the SoapClient Class Feeds the xml requests to the service and outputs the responses to xml Feeds the xml requests to the service and outputs the responses to xml
filesfiles The parser scriptThe parser script
Checks the validity of the responsesChecks the validity of the responses Double-checks the responses with the actual data in the databaseDouble-checks the responses with the actual data in the database
Current Status 2Current Status 2
2.2. Creator ScriptCreator ScriptThe creator script is almost done.The creator script is almost done.Actually different creator scripts were created for different message Actually different creator scripts were created for different message
types.types.Minor changes are to be made when everything is completedMinor changes are to be made when everything is completed
3.3. Request ScriptRequest ScriptDoneDone
4.4. Parser ScriptParser ScriptWork in progress.Work in progress.So far 2 test types are recognised.So far 2 test types are recognised.Work is needed also on the double-checking mechanism.Work is needed also on the double-checking mechanism.
Current Status 3Current Status 3
RRDMA service has changed since RRDMA service has changed since perfSONAR 1.0. Major changes:perfSONAR 1.0. Major changes:
– 1.0 Uses Error Messages1.0 Uses Error Messages– 1.0 will use Error Codes1.0 will use Error CodesAn RRDMA 1.1 server should be able to interact An RRDMA 1.1 server should be able to interact
natively with a 1.0 clientnatively with a 1.0 clientThe functional testing team will check 1.1 version The functional testing team will check 1.1 version
against 1.1 and 1.0 tailored tests. against 1.1 and 1.0 tailored tests.
Q & AQ & A
• QuestionsQuestions
• CommentsComments