1

Joint Working Group for DoD Systems Engineering

StandardizationSystems Engineering Standard

Technical Reviews & Audits StandardPresentation to NDIA SE Division

11 December 2013

Garry Roedler (WG Chair)

2

AgendaDoD Goals and Objectives

Background – Current Situation and Approach

Current Status

Looking Forward

3

Goal of DoD Standardization Initiative“Provide DoD and DoD contractors with a structured,

uniform approach” in the areas identified as gapsImprove communicationEnsure common expectationsAdd realism to bids

Standardization ObjectivesMutual-benefit stipulations:

Must meet both party’s needs and objectives Potential teaming partners must have existing experience with subject matter of

document and existing infrastructure for publishing standards Content of documents must be consistent with government needs

Ensure right kind of standard is developed Clear statement of requirements that speaks “common language” Specific enough to support realistic bidding and resourcing Proper contractual language – SHALL and should

Ensure right amount of the right standards are included Tailorable for different domains and contracting environments Consistent with DOD approach Conform to established, over-arching industry process standards

4

5

Current Situation

Each standard is intended to be put on future system contracts

Standard FocusResponsible SDO Approach

Expected Publication Notes

Systems Engineering IEEE-CS

ISO/IEC/IEEE 15288 (System Life Cycle Processes) will be used as basis for Systems Engineering needs.

Project is expected to produce an addendum to 15288 – “Standard for Application of Systems Engineering on Defense Programs". End of 2014

Full draft available for review in mid-2014

Increments under review now as drafted.

IEEE 15288.1

Technical Reviews and Audits IEEE-CS

No acceptable industry standard to use as base.

New full standard to be developed - “Standard for Application of Technical Reviews and Audits on Defense Programs". End of 2014

Full draft available for review in mid-2014

Increments under review now as drafted.

IEEE 15288.2

Configuration Management SAE Intl

EIA 649B will be used as basis for Configuration Mgt needs.

Project is expected to produce an addendum to EIA 649B. Mid 2015

Manufacturing Management SAE Intl

Recent selection - team is forming - approach to be announced. TBD

Intended to document best manufacturing practices aimed at promoting the timely development, production, modification, fielding, and sustainment of affordable products

Logistics Support SAE Intl

Revision of MIL-HDBK-502A, which provides application guidance to the TechAmerica Standard 0017

Adopted by DoD in June 2013

Reliability TBD Still under review TBD

6

DOD SE/TRA Standards Process

Present effort

7

Membership of DoD Joint SE Working Group Industry AssociationsAerospace CorpBAE SystemsBall AerospaceBoeingGeneral DynamicsHarris CorpL-3 ComLeidosLockheed MartinNorthrop GrummanRaytheonSAICUnited Technologies

AIA IEEE-CS/SA INCOSE ISO/IECNDIASAE Intl

Defense• Air Force (Multi-

parts)

• Army • Navy (Multi-parts)

• OSD – DASD (SE) • DAU• DCMA• DSPO

Leadership TeamChair, Garry Roedler, Lockheed MartinVice-chair, Dave Davis, USAF SMCSecretary, Brian Shaw, The Aerospace Corp.Technical Editor, Bill Bearden, INCOSE (SE)Technical Editor, Mark Henley, L-3 Com (TR&A)

Membership of WG Has Broad SpanWill be adding members to cover academia

8

Review of IEEE Standards Process

IEEE Process Stresses Consensus

9

Reviewers FocusNeed to ensure:

Technical accuracyUsability in defense environmentRequirements are relevant and reflect good practiceRequirements for deliverables are compatible with current

methods, and do not drive a particular method/techniqueRequirements do not result in unintended consequences

10

Schedule InformationBusiness rhythm/cadenceTR&A and SE will have 2-week phase separation for

incrementsMeetingDate +0 +7 +14 +21

WG Provide Inputs

Editor Prepares Draft

Leadership Team Initial Review

Draft to WG for Review

WG Review of Draft

+25 +28

Submit Review Comments to Editor

Editor Consolidates Comments

Review Issues from Comments at Mtg

IEEE Joint SE WG Schedule/Status

11

Integrated schedule balances efforts of both projects Draft development : on-going and on schedule Working draft: May 2014 Publication: December 2014

Each project = 7 development increments plus reviews and balloting 15288.1 (SE) 2 increments completed; 3rd increment in review. 15288.2 (TR&A) 2 increments completed; 3rd increment about to be released.

1

2

Joint WG for DoD SE Standardization - ScheduleD: draft issuedI: inputs for draft dueW: working group meeting discussion (Review Comments due 2 days prior to meeting) 8/

1/20

13

8/8/

2013

8/15

/201

3

8/22

/201

3

8/29

/201

3

9/5/

2013

9/12

/201

3

9/19

/201

3

9/26

/201

3

10/3

/201

3

10/1

0/20

13

10/1

7/20

13

10/2

4/20

13

10/3

1/20

13

11/7

/201

3

11/1

4/20

13

11/2

1/20

13

11/2

8/20

13

12/5

/201

3

12/1

2/20

13

12/1

9/20

13

12/2

6/20

13

1/2/

2014

1/9/

2014

1/16

/201

4

1/23

/201

4

1/30

/201

4

2/6/

2014

2/13

/201

4

2/20

/201

4

2/27

/201

4

3/6/

2014

3/13

/201

4

3/20

/201

4

3/27

/201

4

4/3/

2014

4/10

/201

4

4/17

/201

4

4/24

/201

4

5/1/

2014

5/8/

2014

5/15

/201

4

5/22

/201

4

5/29

/201

4

6/5/

2014

6/12

/201

4

6/19

/201

4

6/26

/201

4

7/3/

2014

7/10

/201

4

7/17

/201

4

7/24

/201

4

7/31

/201

4

8/7/

2014

8/14

/201

4

8/21

/201

4

8/28

/201

4

9/4/

2014

9/11

/201

4

9/18

/201

4

9/25

/201

4

10/2

/201

4

10/9

/201

4

10/1

6/20

14

10/2

3/20

14

10/3

0/20

14

11/6

/201

4

11/1

3/20

14

11/2

0/20

14

11/2

7/20

14

12/4

/201

4

12/1

0/20

14

12/2

0/20

14

12/2

5/20

14

SE Standard (addendum 15288.1)SE working group meetings 8/1 8/15 9/12 10/10 11/7 12/5 1/9 2/6 3/6 4/3 5/1 5/29 6/26 7/24 8/21 9/18 10/16 11/13 12/10SE Draft outl ine X

SE Inc #1:Business or mission analysis, Stakeholder needs and requirements definition process, System requirements definition process, Architecture definition process, Design definition process, System analysis process I D WSE Inc #2: Implementation process, Integration process, Verification process, Transition process, Validation process, Operation process I D WSE Inc #3: Maintenance process, Disposal process, Acquisition process, Supply process, Project planning process, Project assessment and control process I D WSE Inc #4: Decision management process, Risk management process, Configuration management process, Information management process, Measurement process, Qualifty assurance process I D W

SE Inc #5: Lifecycle model management process, Infrastructure management proces, Portfolio management process, Human resource management process, Quality management process, Knowledge management process I D WSE Inc #6:Clauses 1 - 5, annexes I D WSE Inc #7: additional info needed I D WSE Working 15288.1 draft complete D WSE Mandatory Editorial Coordination (MEC) 5/8SE Establish Ballot Group (draft near completion) 5/15SE Draft Complete – Ready for Ballot 6/5SE Establish Small Ballot Comment Resolution Group 6/5SE Ballot start 6/19SE Ballot close/results 7/24SE Recirculation updates/ballot (as needed) 8/14 9/4 9/25SE Submit to RevCom (Hard Date - deadline for meeting) 10/16SE RevCom Meeting 12/10SE Publish 12/20

TRA 15288.2 Develoment Schedule - Rev 2TRA working group meetings 8/1 9/5 10/3 10/24 11/21 12/19 1/23 2/20 3/20 4/17 5/15 5/29 6/26 7/24 8/21 9/18 10/16 11/13Draft Outline XTRA Inc #1: 3 reviews/audits - ITR, SFR, SRR I D WTRA Inc #2: 3 reviews/audits - SAR, SSR, PDR I D WTRA Inc #3: 3 reviews/audits - CDR, IRR, TRR I D WTRA Inc #4: 3 reviews/audits - FRR, FCA, SVR I D WTRA Inc #1: 3 reviews/audits - PCA, P/MRR, ISR I D WTRA Inc #6: front matter, annexes] I D WTRA Inc #7: any additional info needed I D WTRA Working Draft Complete I D WTRA Mandatory Editorial Coordination (MEC) 5/8TRA Close Ballot Group Invitation (draft near completion)

5/15TRA Draft Complete – Ready for Ballot 6/5TRA Establish Small Ballot Comment Resolution Group 6/5TRA Ballot start 6/19TRA Ballot close/results 7/24TRA Recirculation updates/ballot (as needed) 8/14 9/4 9/25TRA Submit to RevCom (Hard Date - deadline for meeting) 10/16TRA RevCom Meeting 12/10TRA Publish 12/20

Note: 2014 RevCom MeetingsAug 19-21; Dec 8-10

15

28

8.1

15

28

8.2

Draft Development

Balloting Period1

2

Progress:

Currently on track

Joint DoD SE WG - Integrated Schedule

12

Looking Forward2014

Development and publication of aligned base documents ISO/IEC/IEEE 15288 (DIS 1Q14; FDIS 3Q14; Publish 4Q2014) IEEE 15288.1 (draft 2Q14; publish 4Q2014) IEEE 15288.2 (draft 2Q14; publish 4Q2014) Also, INCOSE SE Handbook (2Q2014)

Communicate across other working groups to ensure consistency of terms, concepts, and other information Liaisons established with CM and Mfg Mgt Standards WGs Communication across WGs to agree on resolution of identified issues

Start preparation for adoption, governance, support, use 2015

Adoption of standards (early 2015)Complete preparation for use of the standards

Ali

gn

men

t of

SE

R

eso

urc

es

13

Back-up Charts

14

Source ProductsSource Products for SE Standard

ISO/IEC/IEEE 15288 (CD.2), 2013 ISO/IEC/IEEE 15289, Content of life-cycle

information products (documentation), 2011 SMC-S-001, Systems Engineering

Requirements and Products, July 2013 DoDI 5000.02 NATO AAP-48, NATO System Life Cycle

Processes, July 2012 (Addendum Standard to 15288; focused on NATO Armament Systems)

Defense Acquisition Guide (DAG), Chapter 4, Systems Engineering, 2013

INCOSE SE Handbook V3.2 /V4.0 Draft SEBoK EIA 632, Engineering of a System ISO/IEC/IEEE 24748-4 NAVSEA Instruction & Policy 5000-009 for

Systems Engineering Technical Reviews

• Source Products for TR&A Standard– ISO/IEC/IEEE 15288 (CD.2), 2013– DoDI 5000.02– SMC-S-021, Technical Reviews and Audits

for Systems, Equipment and Computer Software, Volume 1, September 2009

– NAVSEAINST 5009.9, Naval Systems Engineering Technical Review Handbook, July 2009

– NAVAIRINST 4355.19E, Systems Engineering Technical Review Process

– NAVAIR Systems Engineering Technical Review Process Handbook, initial release

– Defense Acquisition Guidebook, Chapter 4.Source Products include a combination of Industry and DoD resources

15

Source Products UsageApproach for Use of Source Products

Use as body of knowledge for defense programsIdentify relevant information that should be considered

Identify differences or additional tasks that need to be considered Identify key outputs and their attributes Identify essential guidance needed to provide common understanding

Assimilate key information at level that applies to most defense programs State in terms of “what”, not “how”

16

Review of PARs and Technical ApproachStandard for Application of SE on Defense Programs

Summary of PAR for Systems Engineering Identifier of Standard – IEEE Std 15288.1Title: Standard for Application of Systems Engineering on Defense Programs

– Scope: • System life cycle processes,

activities, and tasks of ISO/IEC/IEEE 15288 for use on any defense system across the life cycle

– Purpose:• This standard implements

ISO/IEC/IEEE 15288 for use by United States Department of Defense (DoD) organizations and other defense agencies in acquiring systems or systems engineering support.

– Need: • Provide the defense specific

language and terminology for the standard to ensure the correct application of acquirer-supplier requirements for a defense prgm.

– Technical Approach:• Addendum to ISO/IEC/IEEE 15288

and will: – Not repeat processes and

information in 15288– Include defense specific

language and terminology – Include necessary tailoring or

changes to existing elements– Include any additional

explanation or guidance

17

Review of PARs and Technical ApproachStandard for Application of Technical Reviews & Audits

Summary of PAR for Technical Reviews & Audits Identifier of Standard – IEEE Std 15288.2Title: Standard for Application of Technical Reviews and Audits on Defense

Programs– Scope:

• Establishes the requirements for technical reviews and audits to be performed throughout the acquisition lifecycle for the U.S. Department of Defense (DoD) and other defense agencies.

– Purpose:• Amplify ISO/IEC/IEEE 15288 Clause 6.3.2.3.a

for selection, negotiation, agreement, and performance of the necessary technical reviews and audits, while allowing tailoring flexibility for the variety of acquisition situations/ environments when the technical reviews or audits are conducted.

– Need: • Provide the defense specific language and

terminology for the standard to ensure the correct application of acquirer-supplier requirements for a defense program.

– Technical Approach:• Standard will be in the form of a full

standard that has links to ISO/IEC/IEEE 15288 and will:– Elaborate on the activities and tasks related to

TR&A– Include defense specific language and

terminology needed for the standard– Include the criteria for reviews & audits– Include the expected/required outcomes/

products of reviews & audits– Include any additional explanation or guidance