kpi for design team

8/11/2019 Kpi for Design Team

1/23

Presentedby

Ronald Collett

Numetrics Management Systems

Santa Clara, CA

www.numetrics.com

Key Performance IndicatorsOf Methodology Capabilities


2/23

Measuring Design Productivity is the Foundationof Assessing Design Methodology Capabilities

Design productivity is a proxy for design processquality

High productivity means high design output per engineer

High productivity is a reflection of:

Engineering skill and management skills,

Tools, flows, methodology, infrastructure

To be used as a proxy, productivity calculationmust contemplate that the product designed is

released to volume productionReleasing to volume production implies that the product itself

offers the requisite functionality, performance, price,

reliability, form factor, etc. (i.e. the right value proposition)


3/23

Basic Productivity Definition

PRODUCTIVITY =OUTPUT

LABOR INPUT


4/23

Measuring Manufacturing Productivity IsStraightforward

MANUFACTURING

PRODUCTIVITY

VALUE-ADDED

LABOR INPUT=

VALUE-ADDED = PRODUCT SELLING PRICE - COST OF MATERIALS


5/23

Measuring Design Productivity Is MuchMore Difficult

????????Effort (Person-weeks)

=DESIGNPRODUCTIVITY


6/23

Dissecting the Numerator of the Design ProductivityMetric--Whats the Best Measure of What a Design TeamProduces?

Overly simplistic and INACCURATE measure of what adesign team produces: Total transistors in the design

EFFORT (MAN-WEEKS)TOTAL TRANSISTOR COUNT


7/23

There is Almost Zero Correlation betweenTransistor Count and Project Effort

ProjectEffort(PersonWeeks)

Raw Transistors in Millions

Relationship Between Raw Transistors and Project Effort*

R2= 0.105p = 0.008

= IC Design Project

0

500

1,000

1,500

2,000

2,500

3,000

3,500

0 2 4 6 8 10 12 14 16


8/23

Measuring Design Productivity

P =

Design

Output

Design

Effort

Units Influencing Factors

Factors that explain highor low productivity:

- Engineering Skill Levels- EDA Tools- Design Flow- Process Stability- Customer Relationship- Management Support

- Etc...

Transistor CountCircuit Type

Reuse LevelsTimingDensityEtc.

Numetrics ComplexityUnits (NCUs)

Person-Weeks

(Direct MeasureOf Staff & Schedule)

A V St C l ti E i t B t


9/23

A Very Strong Correlation Exists BetweenNumetrics Complexity Unit (NCU) Calculationand Project Effort

Numetrics Complexity Units (NCUs) in Millions

Y = 179 + 544 * XR2= 0.520p = 0.000

Relationship BetweenNCUs and Project Effort*

Project

Effort(Person

Weeks)

0

500

1,000

1,500

2,000

2,500

3,000

3,500

0 0.5 1 1.5 2 2.5 3 3.5


10/23

Numetrics Normalization Methodology Yields an R-squaredValue of 0.52 (Project Effort vs. NCUs/Chip)

Accuracy of the Normalization Methodology

R2= 0.105p = 0.008

Actual Transistors per chip (Millions)

Project Effort(Person-weeks)

NCUs per chip (Millions)

Actual Transistor Count

vs. Project Effort

Project Effort(Person-weeks)

Numetr ics Complexi ty Uni t Count

vs . Project Effort

= IC Design Projects

R2= 0.520p = 0.000

= IC Design Projects

0

500

1,000

1,500

2,000

2,500

3,000

3,500

0 2 4 6 8 10 12 14 160

500

1,000

1,500

2,000

2,500

3,000

3,500

0 1 2 3

N i D i P d i i M S


11/23

Low Productivity Project

High Productivity Project

Comparing Design CapabilityWith DPMS

Comparing Design CapabilityWithout DPMS

Numetrics Design Productivity Management System

(DPMS) Quantifies Design Productivity and, therefore,Design Quality

High ProductivityDesign Project

Low ProductivityDesign Project


12/23

Other Factors Explain the Difference in Design

Effort Between Projects of Similar Complexity

Engineering Capability

Inherent Design Complexity

Leadership

EDA Tools/Flows/Methodology

External Factors

69%39%

IC Design

Effort


13/23

Key Performance Indicators are a Prerequisitefor Determining Quality of Design Process

IC Design Productivity IC Development Cost

IC Reuse LeverageIC Design Capacity

0 5000

500 4500

1000 4000

1500 3500

20002500

3000

$0

$20

$40 $60

$80

0 20000

2000 18000

4000 16000

6000 14000

800010000

12000

0%

20%

40% 60%

80%

100%

NCUs perperson week

PercentEffort

Saved perIC Design

Dollars perNCors

$100

Performanceof a Particular Project

Industry Average

NCU= Numetrics Complexity Unit

NCUs per week

Th P f M i D i P Q lit b


14/23

The Power of Measuring Design Process Quality byObserving Three Key Performance IndicatorsSimultaneously

DesignCapacity(Log

Scale)

NCUsd

esignedperWee

k)

Design Productivity (Log Scale)

(NCUs designed per Person-Week)

ASSP Project Distribution by Design Productivity, Design Capacity & Development Cost*

Design Productivity Industry Average

5% Trim Mean

Design Capacity Industry Average

5% Trim Mean

Low-Cost Project(Dev Cost < $5.55)

Mid-Cost Project($5.55 $13.40)

100

1,000

10,000

100,000

10 100 1,000 10,000 100,000

Dev. Cost=$ per NCU


15/23

Comparing the Quality of Two Different Design Flows

Design Capacity(NCUs designed

per week)

LOW

HIGH

Design Productivity(NCUs designed per Person-Week)

LOW HIGH

Average Productivity

Average Capacity

OLD Design Flow NEW Design Flow



16/23

Two Steps are Needed to CompareDifferent Chip Design Projects

1. Design complexity normalization is used to

Account for differences in reuse levels, circuit

types, process technology, timing, and other

circuit design characteristics.

2. Grouping similar projects by design application,

project scope, team goals, etc.

C bi i N li ti ith G i f Si il


17/23

Combining Normalization with Grouping of SimilarProjects (in terms of design application, circuit content,

etc.) Provides for Best-in-Class Assessment

Low Cost Project

Best-in-ClassQuadrant

Design Capacity(NCUs designedper Week)

LOW

HIGH

Design Productivity(NCUs designed per Person-Week)

LOW HIGH

Average Productivity

Average Capacity

Mid-Cost Project High Cost Project

Analog & Mixed-Signal ICs

for Communications

Applications



18/23

Five Sets of Key Performance Metrics

Cycle Time Metrics

Project Effort Metrics

Project Cost Metrics

Design Reuse Metrics

Technology Metrics


19/23

Cycle Time Metrics

Design CapacityNCUs /Week (Numetrics Complexity Units designed per

Week)

Design Cycle TimeTime from Project Start to Release to Volume Production

Project Schedule Slippageas a Percentage of Planned Schedule

First Prototype Turnaround Time1st Tapeout to 1st Packaged Prototypes Received from Fab

Time Allocation by First TapeoutTime Consumed Prior to 1st Tapeout

Time Consumed After 1st Tapeout


20/23

DPMS Yields a Profile of Project Effort and

Duration for Each Design Phase

Project Duration = End Date - Start Date

Example Project Staffing Profile(People versus Time)

Industry StandardDefinition

Project Effort = (Phase Duration FTE)

Full-timeEquivalentPeople

(F

TE)

0

5

10

15

ProjectStart

Milestone1

ProjectEnd

Phase 1

First Tapeout*

Industry Standard

Definition

Milestone3 Milestone

4

Milestone5

Milestone2

Phase 2

Phase 3

Phase4 Phase 6

Phase5

E i i M i DPMS t


21/23

Engineering Managers are using DPMS toAnalyze Cycle Time Improvements

Phase Duration Improvement

Phase 1High LevelDesign &

Partitioning

Phase 2RTL

Design

Phase 3LogicDesign

Phase 4Test InsertionAPR & Timing

Verification

Phase 51st Proto

Fabrication.

Phase 6System

Validation

18.0(17%) 20.3

(19%) 15.4

(14%) 9.5

(9%) 10.0

(9%) 34.5

(32%)

6.3(26%)

5.6(23%)

6.6(27%)

24.5 Weeks

107.8 Weeks

1.7(7%)

2.3(9%)

2.0(8%)

Projects

Startedin 1996

-28%CAGR

-33%CAGR

-49%CAGR

-35%CAGR

-39%CAGR

-40%CAGR

TTM REDUCTION-37% CAGR

ProjectsStartedin 1999


22/23

Cycle Time Metrics (contd)

Design Phase Improvements(if a standard template is used)

Relative Capacity (for netlist-handoff ASIC only)Physical Design Cycle Time

Number of Silicon Spins

No. of Planned Spins

Actual Metal-only Spins & Actual All-layer Spins


23/23

Summary and Conclusions

Measuring design productivity is the cornerstonefor measuring design methodology efficacy

Quantifying design complexity is a prerequisite to

measuring design productivity--requires a robustnormalization approach in order to comparedesigns fairly

Numetrics measurement system is now being used

across the semiconductor and systems industry

Quality of design process has become tantamountto quality of manufacturing process

kpi for design team

Documents