industry pulse: trends in functional verification chief scientist verification design verification...
TRANSCRIPT
Industry Pulse:Trends in Functional Verification
Chief Scientist Verification
Design Verification Technology
MemoCODE 2013
Harry Foster
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Extrapolating From Current Conditions Disregards Future Innovation
HF, MemoCODE, 20132
“In 1910, in the early history telephony, a Bell telephone statistician projected a massive ramp-up in switchboard operator jobs as telephone use grew, until “every woman in America” would be required.”
Source: Future Savvy: Identifying trends to Make Better Decisions, Manage Uncertainty, and Profit From Change Adam Gordon, 2008
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Functional Verification Market According to EDAC
2010 20120
200
400
600
800
1000
700
965
Million
s (
$)
HF, MemoCODE, 20133
38% Growth Between 2010 2012
EDAC: Market Statistics Service 2007 Annual Summary Report
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Functional Verification Market According to EDAC
2010 20120
100
200
300
400 380430
190
365
130
170
Simulation 13% GrowthEmulation 94% GrowthFormal 31% Growth
Million
s (
$)
HF, MemoCODE, 20134
EDAC: Market Statistics Service 2007 Annual Summary Report
www.mentor.com© 2013 Mentor Graphics Corp. Company ConfidentialHF, MemoCODE, 20135
Conducted by Wilson Research Group
— Commissioned by Mentor Graphics
— Format followed 2002, 2004 Collett studies for trend
analysis, as well as the 2007 FarWest Research Study
Worldwide study
— Overall confidence of 95% plus/minus 4.05%
This was a blind study!
— To eliminate any bias in the results
This was a balanced study!
— No single vendor dominated responses
Wilson Research Group
2012 Wilson Research Group Functional Verification Study
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Who Participated In The Survey Participant’s market segment
HF, MemoCODE, 20136
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
ASIC Vendor
Fabless IC Vendor
IC Manu-facturer
Systems Company
Design Services Company
IP Vendor Company
Other0%
10%
20%
30%
40%
50%
60%
Non-FPGA 2012FPGA 2012
Stu
dy P
art
icip
ants
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Who Participated In The Survey Participant’s job title
HF, MemoCODE, 20137
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
HARDWARE
DESIG
NER
VERIFIC
ATION E
NGI...
SYST
EM A
RCHITEC
T
TEAM
LEA
DER
SOFT
WARE
ENGIN
EER
MANAGER
CAD SUPP
ORT EN
GI...
OTHER
0%
10%
20%
30%
40%
50%
60%
Non-FPGA 2012
FPGA 2012
Stu
dy P
art
icip
ants
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential8
Overview
Beyond Theory
Beyond Standards
Beyond the Status Quo
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
BEYOND THEORY
Beyond Theory in Terms of Rising Complexity
HF, MemoCODE, 20139
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential10
Difference Between Theory and Practice
HF, MemoCODE, 2013
In theory there is no difference
between theory and practice, but in
practice there is.
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential11
Difference Between Theory and Practice
HF, MemoCODE, 2013
Theory: Everything is clear, but nothing works.
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential12
Difference Between Theory and Practice
HF, MemoCODE, 2013
Practice: Everything works, but nothing is clear.
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential13
Difference Between Theory and Practice
HF, MemoCODE, 2013
The problem is sometimes theory meets practice:
Nothing works and nothing is clear.
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential14
Beyond Theory in Terms of Rising Complexity
HF, MemoCODE, 2013
What does this really mean?
What makes things complex?
How do we measure complexity?
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential15
What Makes Something Complex?
HF, MemoCODE, 2013
System consisting of many interconnected
parts— Examining the individual parts tells you nothing about the
system
Complex does not necessarily mean complicated
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential16
Designs are Getting More Complex
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
0.25
µ or LA
RGER
0.18
µ
0.15
µ
0.13
µ
0.09
µ
0.06
5µ
0.04
5µ
0.03
2µ o
r sm
a...
0%
5%
10%
15%
20%
25%
30%
35%
40%
2007
Stu
dy P
art
icip
ants
2007: Mean 90nm2010: Mean 65nm2012: Mean 45nm
Process Geometry
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential17
Designs are Getting More Complex
HF, MemoCODE, 2013
100K
or LE
SS
>100K
- 50
0K
>500K
- 1M
>1M -
5M
>5M -
10M
>10M
- 20
M
>20M
-40M
>40M
- 60
M
>60M
0%
5%
10%
15%
20%
25%
30%
2002200720102012
Non-F
PG
A S
tudy P
art
icip
ants
About a 1/3rd of designs below 5M gatesAbout a 1/3rd of design between 5M - 20M gatesAbout a 1/3rd of designs great than 20M gates
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
Number of gates of logic and datapath, excluding memories
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential18
Designs are Getting More Complex
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
2002 2007 2010 20120
2
4
6
8
10
12
0.4
2.7
6.1
11.1
No
n-F
PG
A M
ea
n D
esig
n S
ize
G
ate
s (
M)
Mean number of gates of logic and datapath, excluding memories trends
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential19
Designs are Getting More Complex
NONE 1 2 3 4 5 or MORE0%
10%
20%
30%
40%
50%
60%
21% 22%
28%
6%
10% 13%
2004
2007
2010
2012
Number of Embedded Processors for Non-FPGA Designs
Non-F
PG
A S
tudy P
art
icip
ants
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 2013
79% of designs contain one or more embedded processors
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential20
Designs are Getting More Complex
2004 2007 2010 20120
1
2
3
1.06
1.46
1.96
2.25
No
n-F
PG
A M
ea
n n
um
be
r o
f E
mb
ed
de
d P
roce
sso
rs
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 2013
Mean number of embedded processors continues to rise
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential21
FPGAs are Getting Complex Too!
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
NONE 1 2 3 4 5 or MORE0%
10%
20%
30%
40%
44%
38%
11%
1%4%
2%
21% 22%
28%
6%
10% 13%
FPGA
Non-FPGA
Number of embedded processors
Surv
ey P
art
icip
ants
56% of FPGAs contain one or more embedded processors
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential22
How do we measure complexity?
HF, MemoCODE, 2013
Computational complexity theory used in computer science
There are no generally accepted metrics!
O(1)
O(logn)O(n)
O(nlogn)
O(n^2)
O(2^n)O(n!)
Op
era
tion
s
Elements
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential23
Is bug density a good proxy?
Single, sequential data streams— Floating point unit— Graphics shading unit— DSP convolution unit — MPEG decode— . . .
Multiple, concurrent data streams— Cross bar— Bus traffic controller— DMA controller— Standard I/F (e.g., PCIe)— . . .
Channel
CompressedAudio
Data Link LayerTX
RX
PHY
Sequential data streams1x number of bugs
Concurrent data streams5x number of bugs
Encoder Decoder
HF, MemoCODE, 2013
-Ted Scardamalia, internal IBM study
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential24
Packet-Based Design
Transaction Layer Packet
Reformater
Data LinkLayer PacketReformater
Retry Buffer Arbiter
Tx
Rx
FromFabric
ToPHY
From Rx Channel
Concurrency is Complicated to Verify
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential25
Maybe effort is a good proxy?
HF, MemoCODE, 2013
Aff
ord
ab
leExp
en
siv
eP
roh
ibit
ive
Cost/
Eff
ort
Complexity
Critical Threshold
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential26
Verification Consumes Majority of Project Time
1%-20% 21%-30% 31%-40% 41%-50% 51%-60% 61%-70% 71%-80% >80%0%
5%
10%
15%
20%
25%
Total Project Time Spent in Verification
2007
2010
2012
Time (Percent)
Non-F
PG
A S
tudy P
art
icip
ants 2007: Mean 49%
2010: Mean 56%2012: Mean 56%
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential27
More and More Verification Engineers Mean peak number of design vs. verification engineers
2007 2010 2012
7.8 8.1 8.5
4.8
7.68.4
~ 1-to-1 ratio of peak design
and verification engineers58% 11%
4% 5%
Verification Engineers
Design Engineers
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential28
Where Verification Engineers Spend Their Time
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
16%
22%
23%
36%4%
Test Planning
Testbench Development
Creating and Running Test
Debug
Other
More time spent in debug than any other task!
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential29
54%
2007
46%
2007
47%
2012
53%
2012
Doing Design Doing Verification
Design Engineer Project Time2007 - 2012
15%
Incr
ease
HF, MemoCODE, 2013
Designers Doing More and More Verification
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential30
2007 2012 2017 2022 2027 2032 20370
20
40
60
80
100
At this rate…In 25 years, ALL of a designer’s
time will be devoted to
verification
Project Time 2007 - 2037
Design
Verification
Time Design Engineers Spends Doing:
Tim
e (
Perc
ent)
HF, MemoCODE, 2013
Time Designers Spends in Design vs. Verification
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Design Reuse Trends
HF, MemoCODE, 201331
0%
10%
20%
30%
40%41%
33%
13% 13%
28%
35%
22%
15%
2007
2012
Design Composition
Non-F
PG
A S
tudy P
art
icip
ants
Source: Wilson Research Group and Mentor Graphics.
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential32
Verification Reuse
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
0%
10%
20%
30%
40%
50%50%
41%
8%
39%
44%
17%
2007
2012
Testbench Composition
Non-F
PG
A S
tudy P
art
icip
ants
Mean testbench composition trends
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential33
With All This Effort, How are We Doing?
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential34
Project’s Schedule Completion Trends
More than 10%
EARLY
10% EARLY
ON-SCHED
ULE
10% BEHIND SCHED
ULE
20% 30% 40% 50% >50% BEHIND SCHED
ULE
0%
5%
10%
15%
20%
25%
30%
35%
2007
2010
2012
Stu
dy P
art
icip
ants
2007: 67% behind schedule2010: 66% behind schedule2012: 67% behind schedule
Behind ScheduleAhead of schedule
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential35
FPGA vs. Non-FPGA Completion Trends
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
More than 10%
EARLY
10% EARLY
ON-SCHED
ULE
10% BEHIND SCHED
ULE
20% 30% 40% 50% >50% BEHIND SCHED
ULE
0%
5%
10%
15%
20%
25%
30%
35%
Non-FPGA
FPGA
Non-FPGA vs. FPGA completion compared to project's original schedule
Stu
dy P
art
icip
ants
Non-FPGA: 67% behind schedule FPGA: 67% behind schedule
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential36
Required Number of Spins
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
1 FIRST
SILICON SUCCESS
2 3 4 5 6 7 SPINS or
MORE
0%
10%
20%
30%
40%
50%
2004
2007
2010
2012
Number of Required Spins
Non-F
PG
A S
tudy P
art
icip
ants
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential37
Types of Flaws
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
LOGIC
OR F
UNCTI
ONAL
CLOCKIN
G
TUNIN
G A
NALO
G C
IRCU
IT
CROSS
TALK
POW
ER C
ONSU
MPTI
ON
MIX
ED-S
IGNAL
INTE
RFACE
YIEL
D OR R
ELIA
BILIT
Y
TIMIN
G – P
ATH T
OO S
LOW
FIRM
WARE
TIMIN
G – P
ATH T
OO F
AST
IR D
ROPS
OTH
ER
0%
10%
20%
30%
40%
50%
60%
2004
2007
2010
2012
Trends in Types of Flaws Resulting in Respins
Non-F
PG
A S
tudy P
art
icip
ants
* Multiple answers possible
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential38
Root Cause of Functional Flaws
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
2002
2004
2007
2010
2012
Root Cause of Functional Flaws
Non-F
PG
A
Stu
dy P
art
icip
ants
* Multiple answers possible
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential© 2010 Mentor Graphics Corp. Company Confidentialwww.mentor.comHF, MemoCODE, 201339
Cost of Find Functional Flaws
Silicon Debug, Doug Josephson and Bob Gottlieb, (Paul Ryan)D. Gizopoulos (ed.), Advances in Electronic Testing: Challenges and Methodologies, Springer, 2006
Relative Cost Of Finding Bugs
$1
$10
$100
$1,000
$10,000
$100,000
$1,000,000
$10,000,000
Design Cycle
Co
st
To
Fix
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
BEYOND STANDARDS
Beyond arguing over who won the standards war
HF, MemoCODE, 201340
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential41
Standardization of Languages
VHDL Verilog Synopsys Vera
System C SystemVer-ilog
Specman e C/C++ OTHER Testbench
0%
20%
40%
60%
80%
2007
2010
2012
Languages Used for Verification (testbenches)
Non-F
PG
A S
tudy P
art
icip
ants
SystemVerilog grew 8.3% between 2010 and 2012
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential42
SystemVerilog Adoption by Design Size
< 5M 5 - 20M > 20M0%
20%
40%
60%
80%
100%
59%
71%
89%
SystemVerilog Adoption by Design Size (Gate Count Excluding Memories)
Non-F
PG
A S
tudy P
art
icip
ants
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential43
Standardization in Base Class Libraries
0%
10%
20%
30%
40%
20102012
Testbench Methodologies and Base-Class Libraries
Non-F
PG
A S
tudy P
art
icip
ants
486% UVM growth between 2010 and 201246% UVM projected growth in the next twelve months
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential44
Standardization of the SoC Verification Process
Ten years ago, IC/ASIC verification was partitioned
into two main steps:
HF, MemoCODE, 2013
Integration
Verification
Block-Level
Verification
Block Full Chip
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential45
Standardization of the SoC Verification Process
Emerging from ad hoc to systematic processes
HF, MemoCODE, 2013
Block-Level
Verification
Interconnect
Verification
Integration
Verification
Application / SW
Verification
IP Subsystem SoC System
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
BEYOND THE STATUS QUO
Beyond surviving by maintaining the status quo
HF, MemoCODE, 201346
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential47
The Verification Paradox
A good verification process lets you get the most out of best-in-class verification tools
Start
ToolsAd Hoc
Processes
6-9%Cost
Increase
Start
Process Tools20-30%
Cost Savings
Source: Cisco Momentum Research Group
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential48
Standardization of the SoC Verification Process
HF, MemoCODE, 2013
Block-Level
Verification
Interconnect
Verification
Integration
Verification
Application / SW
Verification
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential49
Use of Advanced Verification Techniques
HF, MemoCODE, 2013
Constrained-Random Simu-lation
Functional coverage
Assertions
Code coverage
0% 10% 20% 30% 40% 50% 60% 70% 80%
62%
71%
68%
70%
41%
40%
37%
48%
2007
2012
Non-FPGA Study Participants
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential50
Directed vs Constrained-Random Simulation
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
2010 20120%
20%
40%
60%
80%
100%
44% 51%
56% 49%
Directed
Constrined-Random
Mean Directed vs. Constrained-Random Simulation Trends
Non-F
PG
A S
tudy P
art
icip
ants
16%
Increase
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential51
Larger Designs Use More Formal
< 5M 5 - 20M > 20M0%
10%
20%
30%
40%
20%
26%
41%
Formal Property Checking Adoption by Design Size (Gate Count Excluding Memories)
Non-F
PG
A S
tudy P
art
icip
ants
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential52
The Evolution of Formal Technology
FormalPropertyChecking
FullyAutomatic
Formal
AutomatedApplications
LowEffort
HighEffort
FormalExperts Everyone
1990s Today2000s
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential53
Standardization of the SoC Verification Process
HF, MemoCODE, 2013
A57A57A57A57
Block-Level
Verification
Interconnect
Verification
Integration
Verification
Application / SW
Verification
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential54
Standardization of the SoC Verification Process
HF, MemoCODE, 2013
Block-Level
Verification
Interconnect
Verification
Integration
Verification
Application / SW
Verification
— IP Blocks connectivity— Access all memories— Access all registers, such as control— Configurations work— Functional scenarios and use-cases— Verify multiple clock domain crossings
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential55
Mean Number of Clock Domains by Design Size
< 5M 5 - 20M > 20M0
2
4
6
8
10
12
14
5.21
9.36000000000001
11.48
Mean Number of Clock Domains by Design Size (Gates Excluding Memories)
Mean N
um
ber
of
Clo
ck D
om
ain
s
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential56
Standardization of the SoC Verification Process
HF, MemoCODE, 2013
Block-Level
Verification
Interconnect
Verification
Integration
Verification
Application / SW
Verification
— Boot OS— Load System Drivers— Run Application SW
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
SoC Design & Verification Involves Lots of SW
HF, MemoCODE, 201357
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012$0
$20
$40
$60
$80
$100
Total SW Engineering Costs + ESA Tool CostsTotal HW Engineering Costs + EDA Tool Costs
Source: ITRS 2010, Impact of Design Technology on SoC Consumer Portable Implementation Cost
($ M
)
It’s the software, stupid! -Gary Smith
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential58
As Design Sizes Increase…Emulation Up, FPGA Prototyping Down in 2012
HW Acceleration/Emulation FPGA Prototyping0%
10%
20%
30%
40%
50%
60%
18%
32%
50%
45%
57%
28%
< 5M 5 - 20M> 20M
Adoption by design size for those doing HW acceleration/emulation and FPGA Prototyping
Non-F
PG
A S
tudy P
art
icip
ants
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential59
Integrated Simulation/Emulation/Software Verification Environments Emerge
HF, MemoCODE, 2013
Virtual Prototype
Processor Debug
JTAG
SW Debug
OVM/UVMSystemC/C++
Monitors
TestbenchAcceleration
Assertions & Checkers
Physical Devices
Protocol Solutions
System Level
Virtual Devices & Transactors
Protocol Solutions
Design under Test
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential60
Coverage and Power
HF, MemoCODE, 2013
Block-Level
Verification
Interconnect
Verification
Integration
Verification
Application / SW
Verification
Block Subsystem SoC System
Across all aspects of verification
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential61
Signoff criteria trends
The Rising Importance of Coverage
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
* Multiple answers possible
OTHER
WHEN THE PROJECT PLAN SAYS SIGN-OFF, REGARDLESS OF STATUS
WHEN WE CAN NO-LONGER THINK OF ANY MORE TESTS TO WRITE
WHEN THE RATE OF BUGS FOUND PER WEEK DROPS BELOW A SPECIFIED GOAL
WHEN THE EMULATED OR PROTOTYPED DESIGN IS WORKING IN-SITU
WHEN COVERAGE SAYS WE HAVE ACHIEVED A TARGET
WHEN THE PROJECT PLAN SAYS SIGN-OFF, ASSUMING VERIFICATION LOOKS OK
WHEN ALL TESTS DOCUMENTED IN THE VERIFICATION PLAN ARE COMPLETE AND PASS
0% 10% 20% 30% 40% 50%
20072012
Non-FPGA Study Participants
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential62
Unified Coverage Interoperability Standard
HF, MemoCODE, 2013
New Accellera UCIS Standard Announced at DAC 2012
UCIS API
Coverage Database
Simulation
Formal
Emulation
Analysis Testplan
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential63
Trends in Power Management Verification
Power domain power down/power up
Transitions between system
power states
Interactions between power domains
Application-level power management
0% 20% 40% 60% 80%
Aspects of Non-FPGA Power Managed Design That Are Verified
67% of the industry actively manages power
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Power Trends
HF - January 2013 Master Set, WRG & MG Study Results
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
0%
5%
10%
15%
20%
25%
30%
35%
Percentage of total simulations that were power-aware
Non-F
PG
A S
tudy P
art
icip
ants
About 10% of power managed designsperform no power-aware simulation!
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Power Trends
HF - January 2013 Master Set, WRG & MG Study Results
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
0%
5%
10%
15%
20%
25%
30%
35%
Percentate of Verification Resources Focused on Power Management
Non-F
PG
A S
tudy P
art
icip
ants
Median Verification Resources: 20% -29%
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Power Trends
HF, MemoCODE, 201366
CPF 1.0 CPF 1.1 CPF 2.0 UPF 1.0 (Accellera)
UPF 2.0 (IEEE 1801-
2009)
Internal/Proprietary
0%
5%
10%
15%
20%
25%
6% 6%
24%
15%
27%
22%
Non-F
PG
A S
tudy P
art
icip
ants
* Multiple answers possible
Notation used to describe power intent
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential67
Beyond the Status Quo
HF, MemoCODE, 2013
Block-Level
Verification
Interconnect
Verification
Integration
Verification
Application / SW
Verification
IP Subsystem SoC System
Standardization of the SoC Verification Process
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
BACK TO THE FUTURE
The Productivity Gap
HF, MemoCODE, 201368
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential69
Design Productivity Gap
Silicon Density
Design Productivity
Cap
acit
y
Time
Source: SEMATECH
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential70
Verification Productivity
Verification Productivity Gap
Source: SEMATECH
Silicon Density
Design Productivity
Cap
acit
y
Time
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Closing The Verification Gap
HF, MemoCODE, 20137171
Reuse Abstraction
Acceleration Methodology
?
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
72
100s of lines of TLM
Billions of Transistors
100s of Millions of Gates
Millions of Lines of RTL
#include "systemc.h" SC_MODULE(adder) // module (class) declaration { sc_in<int> a, b; // ports sc_out<int> sum; void do_add() // process { sum = a + b; } SC_CTOR(adder) // constructor { SC_METHOD(do_add); // register do_add to kernel sensitive << a << b; // sensitivity list of do_add } };
Managing Complexity
HF, MemoCODE, 201372
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
73
RTLclk
Cycle Accurateclk
Timed TLM
Untimed TLM
C/C++
Simulation
1x (7 days)
10,000x (1 min)
10x
100x
1,000x
Protocol
Protocol
Transaction
Functionarguments
Transaction
Productivity Gains Through Abstraction
HF, MemoCODE, 201373
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
What’s the advantage of SystemC compared with RTL?
There are two different aspects – quality and time schedule. Today, a full chip on SystemC will run around 10 MHz, and you will never reach that speed using RTL or a lower-level abstraction. It’s similar to a prototype speedup. Previously with RTL designs, our bug rate was in the range of 10 to 50 bugs per square millimeter. Now we are at less than one bug per millimeter squared. So we have both quality and speed of development.
Source: EETimes, 2007, Laurent Ducousso, who manages intellectual-property (IP) verification for STMicroelectronics’ Home Entertainment Division
Bug Prevention vs. Bug Hunting
HF, MemoCODE, 201374
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential75
Summary
Beyond theory in terms of rising complexity Beyond arguing over who won the standards
wars Beyond surviving by maintaining the status
quo
HF, MemoCODE, 2013
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential