tab report to board 31 july 2014. completed work 2014 “keyword guidelines for oasis specifications...

TAB Report to Board

31 July 2014

Completed Work 2014 • “Keyword Guidelines for OASIS

Specifications and Standards”

• Systematic review of all 1st PRs + documentation to guide reviewers

• [comment?]-tagged HTML files for PR

• TAB public page and resources

• Open Standards Cup final selections

Public Reviews

• At Board request, TAB made PRs a 2013/2014 priority

- 4 WS-BRSP Profiles - BDX Location

- 3 Bindings for OBIX - CAMP v1.1

- 1 Encodings for OBIX - DSS Extension

- Trust Elevation Framework - 12 KMIP CSDs/CNDs

- MQTT and NIST Cybersecurity Framework

- OBIX v1.1

- 5 PKCS CSDs - SAML Conformance Clause

- TOSCA v1.0 - TGF v2.0

- 2 XACML Profiles

Public Reviews

• > 700 comments issued on 39 public reviews

• Review methodology • TAB only reviewed 1st PR

• Reivewed per TAB’s PR Checklist

• Comments tracked in JIRA & Excel exports made available

• Applied categorization at high-level

Issue categories• Normative: e.g normative vs. non-normative, use

of formal keywords

• Reference: e.g. incorrect, missing

• Style: e.g. formatting errors

• Technical: e.g. terminology, completeness

• Structure: e.g. misuse of sections & parts

• Conformance: e.g. referencing of normative content, relation to implementations, etc.

• Process: e.g. consistency with TC Process rules

Sample commentsConformance: “The conformance clauses <snip> are too vague for

implementers to attempt conformance with the specification. For example, 2.3, leaves many terms undefined and/or unclear.”

“The "table list" in the conformance clause has only one element...So if "an implementation may choose to implement and conform to one or more", then "Each of them is optional to implement" is just not true: this single list element is mandatory to support.”

Sample commentsReferences:

“Normative Reference currently reads <snip>. The correct citation is <snip> as reported by http://www.w3.org/2002/01/tr-automation/tr-biblio-ui”

“RFC 2630 is listed in non-normative references but is cited as a normative document in the definitions. Also RFC 2630 has been obsoleted by RFC3369, RFC3370, and RFC3369 has been obsoleted by RFC3852.”

Sample commentsTechnical:

“In the Process flow of Use Case 5, some apparent actors are used (see upper case words POLICY_AUTHOR, ATTRIBUTE_PROVIDER ) that have not been defined in the Actors section.”

“The paragraph under 17 Security Considerations reads <snip>. These are the only two uses of "principal" in the document. Both are requirements (MUST) and yet are undefined in the document.”

Analyzing the comments

Analyzing the comments

Next steps• Careful not to infer too much into data

• Caveats: differing levels of review from one to next, consolidation of comments, etc.

• Drill into the top categories – style, reference, normative – to see if there is more to learn

• Identify where best practice advice to editors might be of value

Work in progress

Conformance Clauses Guidelines upgrade

Specification Editors Best Practices Guidelines

Public review comment handling procedures and templates

Board Process Advisory / Consultation Work

?

Public review comment handling procedures and templates

New Work

• Charter Guidelines

• …