pmi validation and conformance testing for model-based ......semantic pmi • defined using a...

PMI Validation and Conformance Testing for Model-based Engineering

Joshua Lubell lubell@nist.gov

Systems Engineering Conference in Washington DC (SEDC) April 3, 2014

www.nist.gov/el/msid/infotest/mbe-pmi-validation.cfm

Engineering Laboratory Systems Integration Division

Outline

• Model-based systems engineering and PMI • PMI standards and “semantic PMI” • NIST digital test artifacts and V&V workflow • Software tools • Future directions Disclaimer: Any mention of commercial products within this presentation is for information only; it does not imply recommendation or endorsement by NIST.

Model-based SE and PMI

• Model must capture product’s functional and behavioral requirements

• Product and Manufacturing Information (PMI) – Specifies these requires in a production context – In a formal and precise language – Communicating allowable product geometry

variations in form, size, and orientation

• PMI is essential for manufacturing high-precision parts and assemblies

PMI Example from a Standard*

*ISO 1101, Geometrical product specifications (GPS)–geometrical tolerancing–tolerances of form. Orientation, location and run-out (2nd ed.) International Organization for Standardization, Geneva (Switzerland) (2004)

The Digital Thread

• Model Based Definition (MBD) – Created by Engineering – Used to complete Model Based Manufacturing

and Inspection (MBM and MBI)

• DT = MBD data + MBM data + MBI data • 3D models with “semantic” PMI makes it

possible

Semantic PMI • Defined using a computer-interpretable language • Allows software developers to automate various design

and manufacturing functions • Unambiguous representation and presentation of PMI

are critical for digital product definition • But PMI standards are complex • Potential is great for the same PMI to be interpreted

and presented differently by different software applications

• Incorrect PMI presentation and misinterpretation can result in significant delays and costly errors

NIST PMI Validation and Conformance Testing Project

Develop test case definitions, test case models and software algorithms sufficient to measure conformance of CAD systems to American Society of Mechanical Engineers (ASME) standards for PMI. ASME Y14.5-1994 - Dimensioning and Tolerancing ASME Y14.41-2003 - Digital Product Data Definition Practices

Project Components (1) • Test Cases

• Select 50 PMI annotation constructs (Atomic Test Case – ATC)

• PMI applied to 5 parts (Complex Test Case – CTC) • Each a collection of 10 ATC

• Expert review of the PMI

• Test Models • Each of the 5 parts modeled in 4 CAD systems

(CATIA, Creo, NX, SolidWorks) • Focus on Semantic PMI

• Precise, not graphical, machine interpretable, no encoded strings

Project Components (2) • Verification of CAD models

• Comparison to test case definitions and to each other • Identify and resolve modeling issues to ensure semantic

equivalence • Test of the CAD authoring system

• Release to implementer groups

• Generate derivative STEP, JT, and 3D PDF files • Validate derivative files to original CAD models • Import derivation STEP files to CAD systems

• Validate derivative CAD models to original CAD models • Test of CAD translating system

• Release test cases and CAD models to public

MBE PMI Validation Data Process Flow

Test Cases

CATIA V5 Models

Creo Models

NX Models

Verification

SolidWorks Models

Expert Review

Validation

STEP Files

JT Models

3D PDF Models

CAx-IF

JT-IF

3D PDF-IF

Implementer Forum = Vendor partnership to identify & resolve implementation issues

CATIA V5 Models

Creo Models

NX Models

SolidWorks Models

Test Cases versus Test Models *

Test Case – a PMI construct Test Model – its CAD implementation

*Figures from “NIST MBE PMI Validation Project Update,” ITI Transcendata, February 2013

“Complex” versus “Atomic”

5 Complex Test Cases • Manufacturable • Useful for integration

testing • Pre-requisite for developing

Atomic Test Cases

50 Atomic Test Cases • Many ATCs to one CTC • Useful for conformance

testing

Sample PMI Annotation Constructs

Complex Test Cases Spreadsheet

Atomic Test Cases Spreadsheet

Test Case Browser Demo

http://cic.nist.gov/mbe/PMITest/index.html

Test Cases Modeled in 4 CAD Systems Test Case CATIA V5 R21 Model

NX 8.0 Model SolidWorks 2012 Model

Creo 2.0 Model

Example Semantic Verification Issue

in-between designation and bounds (edges) are non-semantic.

Results • Test Cases

• Select 50 PMI annotation constructs • Applied to 5 parts (10 per part) • Documented with PDF, images, and explanatory text • Expert review of the PMI

• Test Models • Each of the 5 parts with PMI modeled in 4 CAD

systems • Verification of CAD models

• Comparison to test case definitions and to each other • Release to Implementor Forums

• Resolve modeling issues • Generate derivative STEP, JT files, and 3D PDF files • Validate derivative files to original CAD models (in

progress)

NIST STEP File Analyzer and

Semantic PMI • STEP File Analyzer parses ISO 10303 STEP

file and displays PMI data in spreadsheet • Recreates visual presentation of the

semantic PMI • Compare to test case definition • Identify source of discrepancies

• STEP file, CAD model, Analyzer

Semantic PMI Summary Worksheet • Recreate Visual Presentation of the Semantic PMI

• Datum systems, dimensional and geometric tolerances

STEP File Analyzer - Coverage Analysis

Column A – Types of PMI, numbers in parentheses refer to CAx-IF Recommended Practices Other Columns – One per STEP file, X indicates the file contains that type of PMI, but not the number of instances

Phase 2 Plans • Test Cases

• Select 50 more PMI annotation constructs (Atomic Test Case – ATC)

• Expert review of the PMI • Applied to 5 parts (Fully-Toleranced Test Case – FTC)

• Test Models • Each of the 5 parts with PMI modeled in 4 CAD systems

(more recent versions of CATIA, Creo, NX, SolidWorks) • Focus on Semantic PMI

• Verification of CAD models • Comparison to test case definitions and to each other • Implementor Forums (IFs) to resolve modeling issues

• Release to IFs • Generate derivative STEP, JT, and 3D PDF files • Validate derivative files to original CAD models • Import derivation STEP files to CAD systems

• Validate derivative CAD models to original CAD models

Summary

• Semantic PMI is a fundamental component of the Digital Thread

• Semantic PMI implementation is a challenge • NIST has a growing collection of digital test

artifacts and tools for verification and validation of CAD implementations of semantic PMI