an overview of existing server compliance tests for 1.5 and 1.7

Paula O’BrienRonin Technologies, Inc.

(mailto:[email protected])

An Overview of Existing Server Compliance Tests for 1.5 and 1.7

Login

• Tests the following authentication:– Basic Authentication– Digest Authentication– UA-Auth (new) – 1.7 only

• Checks response for required capability urls: Login, Search, GetMetadata

• Checks response for optional capability urls: Action, ChangePassword, GetObject, LoginComplete, Update

Login, cont’d

• Checks for required response arguments (“Broker", "MemberName", "MetadataVersion", "User" )

• Checks for optional response arguments ("Balance", "TimeoutSeconds", "Expr", "OfficeList")

• Checks for a valid RETS version header– 1.5 or 1.7

• Checks for valid RETS-RESPONSE tags• Checks date format in header compares to

NOW in GMT time

Metadata• Tests the following types with required support for IDs of

0 and *. Validates the response formats and also the depth and breadth for the ID.– METADATA-SYSTEM– METADATA-RESOURCE– METADATA-CLASS

• Tests a user configured Id parameter of METADATA-CLASS:METADATA-TABLE

• Validates compact format metadata against a DTD created from the description in the specification.

• Validates Standard-XML format against the published dtds.– 1.5– 1.7

Metadata, cont’d

• METADATA-SYSTEM transaction is used to dynamically configure DMQL tests (parses metadata, validates types, displays metadata tree)

• Checks well-formed xml (for both compact and standard-xml).

• Checks for required response headers ("date", "cache-control", "content-type", "rets-version")

• Checks for optional response headers ("content-length", "transfer-encoding", "server")

Snippet of 1.7 Metadata Compact DTD

• • • • • • • • • <!ATTLIST RETS• ReplyCode CDATA #REQUIRED• ReplyText CDATA #REQUIRED• >• <!ELEMENT METADATA-SYSTEM (SYSTEM+, COMMENTS?, COLUMNS?, DATA*)>• <!ATTLIST METADATA-SYSTEM• Version CDATA #IMPLIED• Date CDATA #IMPLIED• >• <!ELEMENT METADATA-RESOURCE (COLUMNS?, DATA*)>• <!ATTLIST METADATA-RESOURCE• Version CDATA #IMPLIED• Date CDATA #IMPLIED• >• ….Compact metadata dtd too large to fit on one slide

Metadata Negative Testing

• Tests for appropriate RETS error codes in response to transactions.– Invalid Resource: Sets the ID parameter to

“RETSNegativeTestingResource:” + a valid Metadata-Table configured by the user.

– Invalid Type: sets the Type parameter to METADATA-RETSNegativeTesting.

– Requested DTD Unavailable: sets the Format parameter to STANDARD-XML:RETS-METADATA-20060101.dtd (which is not a valid RETS metadata dtd).

Search• Checks for required response headers ("date", "cache-

control", "content-type", "rets-version")• Checks for optional response headers ("content-length",

"transfer-encoding", "server")• If the content-type is text/xml, the response is validated

as well-formed xml – the <RETS> response is turned into a document and parsed

• The presence of multiple content-types are reported as ‘info’; not failures but warned that clients may have difficulty with multiple content-types

• When a test returns format=STANDARD-XML, the response is validated against the applicable dtd – 1.5 – 1.7

Search, cont’d• When a test returns Format=COMPACT or

COMPACT-DECODED, the response is validated against a dtd created using the definition described in the specification– 1.5– 1.7

• Standard names search test – StandardNames=1 and format=Standard-XML. A standard name (configured as a test parameter) is included in both the query and the select. The test checks the results for appropriate column selected and data type returned.

Search, cont’d• System names search test – StandardNames=0 and

format=Compact-DECODED. A system name (configured as a test parameter) is included in both the query and the select. Checks results to make sure column in select shows up in response with appropriate data type

• Test limit – Two tests, where limit=2 and format=Standard-XML and Limit=2 and format=Compact-Decoded. The test counts records returned for each transaction to make sure there are only 2 records returned.

• Test offset – Offset=1. The test checks that the server does not throw an error when offset submitted.

Search, cont’d• Test RestrictedIndicator – RestrictedIndicator=###. The

test checks that the server does not throw an error when RestrictedIndicator is submitted. Since the server may choose to omit the results of a restricted field rather than send back a restricted indicator, all that we can test for compliance is to make sure the server does not reject this parameter.

• Test count - three tests:– Count=0; tests to make certain that the <COUNT> tag is not

present the response.– Count=1; tests to make certain that the count tag is present in

the response and is formatted properly with a Records attribute.– Count=2; tests to make certain that ONLY the <COUNT> tag is

present in the response, with no result data, and that the Records attribute is formatted properly

Search Negative Testing• The purpose of this test is to ascertain whether the response

returns the appropriate RETS error code for the situation• In search, there are some error conditions (“query too complex”)

that cannot be tested uniformly across servers. Those types of errors are not included in negative testing.

• Unknown query field in select: the Select parameter is set to “RETSNegativeTestingField”.

• Unknown query field: the Query parameter is set to “(RETSNegativeTestingField=Cleveland)”.

• Invalid query syntax: the Query parameter includes a “>” symbol and ends with an “||”

• Requested dtd unavailable: The Format parameter is set to STANDARD-XML:RETS-20060101.dtd (which is not a valid dtd).

GetObject

• The test checks for required response headers:– content-type and mime-version– For multipart test:

• content-type MUST be multipart/parallel• boundary must be present

• The test checks for optional response headers:– Location – Description

• Test single response using the default id=0

GetObject, cont’d• Test multipart response using the id=*

– Multipart must be formatted properly – uses a multipart body parser to test this

• Tests do not check for Location=1 because servers have the option to return the object, not the URL, so only Location=0 must be supported and is the default.

• Negative testing tests for appropriate error responses:– Invalid resource: sets the Resource parameter to

RETSNegativeTesting.– Invalid type: sets the Type parameter to

RETSNegativeTesting.

Update

• The current test requires the user to manually key in the field/value pairs to set the Record field for updating (ListPrice=1&City=Cleveland, etc.)

• The test sends a user configured Validate parameter but other than making certain the server does not throw an error, the test does not do anything further with this feature at the product level.


Update, cont’d


• Check well-formed xml for the response body– this includes error and warning blocks (warning

blocks 1.7)• Validates the format and required elements of

the Update Response using a DTD created from the format defined by the specification– 1.5– 1.7

Update Response 1.7 DTD example:<!ATTLIST RETS

ReplyCode CDATA #REQUIREDReplyText CDATA #REQUIRED

><!ELEMENT TRANSACTIONID (#PCDATA)><!ELEMENT DELIMITER EMPTY><!ATTLIST DELIMITER value CDATA #REQUIRED><!ELEMENT COLUMNS (#PCDATA)><!ELEMENT DATA (#PCDATA)><!ELEMENT WARNINGDATA (#PCDATA)><!ELEMENT ERRORDATA (#PCDATA)><!ELEMENT ERRORBLOCK (ERRORDATA+)><!ELEMENT WARNINGBLOCK (WARNINGDATA+)><!ELEMENT RETS-STATUS EMPTY><!ELEMENT RETS (TRANSACTIONID,DELIMITER?,COLUMNS,DATA,ERRORBLOCK?,WARNINGBLOCK?)>

Update Negative Testing

• Tests for appropriate RETS error responses– Invalid Resource: test sets the Resource

parameter to InvalidResource.– Invalid Parameter: test sets a field/value pair

in the Record parameter to BadFieldName=Invalid.

DMQL Testing• Additional search tests to test DMQL support. • DMQL testing also tests metadata – Queries are built by

a user interface that parses the server’s metadata so the user can select the appropriate classes and then fields to use in numeric, date and character queries. Metadata with incorrectly identified data types will cause the tests to fail.

• Two DMQL test sets exist: one for Standard and one for System names.

• Inside of the Query parameter, values are case-sensitive and search results will fail if the case is incorrect.

DMQL Testing, cont’d• Numeric tests:

– Greater than = creates a query for with a numeric field using the + operator. Evaluates the results to make certain the value of the result returned for that field is greater than the value used in the Query.

– Less than = creates a query with a numeric field using a – operator. Evaluates the results to make certain the value of the result returned for that field is less than the value used in the Query.

– Range = creates a query with a numeric field using two values separated by the – operator and validates that the search result for that field falls within the range.

DMQL Testing, Cont’d• Character Tests:

– AND : creates a query using two character field=value pairs and the “,” operator between the pairs. Evaluates the search results to make certain that both equivalency conditions are met.

– OR: creates a query using two character field=value pairs and the “|” operator between the pairs. Evaluates the search results to make certain that one of the equivalency conditions has been met

– Contains: creates a query using field=*value*. Evaluates the search results to make certain the value is present within the result.

– Starts With: creates a query using field=value*. Evaluates the search results to make certain the result begins with the value.

DMQL Testing, cont’d

• In addition, all DMQL tests include tests for:– Required Search response headers– Optional Search response headers– Well-formed XML– Format=Standard-XML must validate against the

appropriate dtd– Format=COMPACT-DECODED must validate

against the search compact format DTD (created from the description in the specification).

DMQL Testing, cont’d• Date Tests:

– After this date: creates a query for a date field using the + operator. Evaluates the results to make certain the value of the date returned for that field occurs after the date used in the Query.

– Before this date: creates a query for a date field using the - operator. Evaluates the results to make certain the value of the date returned for that field occurs before the date used in the Query.

– Today: creates a query for a date using the token TODAY. Evaluates the results to make certain that the date returned is today’s date.

Logout



• Checks for optional logout response arguments: "Balance", "TimeoutSeconds", "Expr", "OfficeList"

• Validates format against a dtd created for the logout response defined in the specification

• Check well-formed xml

Example of Logout 1.7 response dtd:• • • • • • • • • <!ATTLIST RETS• ReplyCode CDATA #REQUIRED• ReplyText CDATA #REQUIRED• >• <!ELEMENT RETS-RESPONSE (#PCDATA)>• <!ELEMENT RETS-STATUS EMPTY>• <!ATTLIST RETS-STATUS• ReplyCode CDATA #IMPLIED• ReplyText CDATA #IMPLIED• >• • • <!ELEMENT RETS (RETS-RESPONSE?,RETS-STATUS?)>•

ServerInformation (1.7 only)• Tests the server information resource – a user may

configure optional request arguments for Resource, Class and StandardNames parameters. If none are supplied the test executes a transaction with no request arguments.



• Checks that the response is well-formed xml.• Validates the response against a ServerInformation dtd

developed using the response definition in the specification.

Example of ServerInformation dtd:• • • • • • • • • <!ATTLIST RETS• ReplyCode CDATA #REQUIRED• ReplyText CDATA #REQUIRED• >• <!ELEMENT Parameter (#PCDATA)>• <!ATTLIST Parameter• name (CurrentTimeStamp | LastTimeStamp | MinimumLimit | KeyLimit | ReplicationSupport |

CDATA) #REQUIRED• resource CDATA ""• class CDATA ""• >• <!ELEMENT ServerInformation (Parameter+)>• • • <!ELEMENT RETS (ServerInformation)>•

an overview of existing server compliance tests for 1.5 and 1.7

Documents