1
110001011010011110100111011011010011110011
www.bitvis.no Your partner for SW and FPGA
A game changer for VHDL verification:
- Advanced VHDL Verification - Made simple - For anyone
NMI FPGA Network: Design tools & methodologies
October 11, 2016
www.bitvis.no A game changer for VHDL Verification 2
The 2016 Wilson Research Group Functional Verification Study (1)
2
www.bitvis.no
The 2016 Wilson Research Group Functional Verification Study (2)
A game changer for VHDL Verification 3
www.bitvis.no
Quality and Efficiency enablers
A game changer for VHDL Verification 4
Structure & Architecture
Overview, Readability, Simplicity
Modifiability, Maintainability, Extendibility
Debuggability
Reusability
3
www.bitvis.no
UVVM Universal VHDL Verification Methodology
A game changer for VHDL Verification 5
Free and Open source code
• Downloadable from GitHub or Bitvis.no
Two levels of functionality:
• Utility Library (and BFMs) : May be used stand-alone
• VVC Framework (and VVCs) : Uses Utility Library
Structure & Architecture
Overview, Readability, Simplicity
Modifiability, Maintainability, Extendibility
Debuggability
Reusability
UVVM: Targets exactly these challenges
www.bitvis.no
UVVM Utility Library
Previously 'Bitvis Utility Library'. Free & open source since 2013
A basic VHDL testbench infrastructure
• Logging and verbosity control
• Alert handling incl. optional positive acknowledge
• Extended string handling
• Simplified randomisation
• check_value(), check_value_in_range()
• check_stable(), await_stable()
• await_change(), await_value()
• report_alert_counters()
• BFM support and examples
• etc....
Very user friendly
Extremely low user threshold
Advanced funct. when needed
Up and running in less than an hour
Well documented
A game changer for VHDL Verification 6
4
www.bitvis.no A game changer for VHDL Verification 7
UVVM Utility Library
www.bitvis.no A game changer for VHDL Verification 8
UVVM Utility Library
5
www.bitvis.no A game changer for VHDL Verification 9
UVVM Utility Library - BFMs
www.bitvis.no
BFMs to handle interfaces
Handle transactions at a higher level
E.g. Read, Write, Send packet, Config, etc
BFM: - A model or model set (or API) for handling transactions on a physical interface.
- Models the environment - e.g. a bus master
A game changer for VHDL Verification 10
6
www.bitvis.no
BFM: - A model or model set (or API) for handling transactions on a physical interface.
- Models the environment - e.g. a bus master
BFMs to handle interfaces
Handle transactions at a higher level
E.g. Read, Write, Send packet, Config, etc
Example: BFM procedure for a CPU access to a module's register E.g. write 0xF0 into a register at address 0x22
Replaced by:
write(x”22”, x”F0”);
cs <= ’1’;
we <= ’1’;
addr <= x”22”;
data <= x”F0”;
wait until rising_edge(clk);
wait until falling_edge(clk);
cs <= ’0’;
we <= ’0’;
or:
sbi_write(x”22”, x”F0”);
SBI: Simple Bus Interface - Single cycle - Optional ready - Dead simple
A game changer for VHDL Verification 11
www.bitvis.no
BFMs to handle interfaces
Handle transactions at a higher level
E.g. Read, Write, Send packet, Config, etc
More understandable for anyone
Simpler code & Improved overview
Uniform style, method, sequence, result
Easy to add several very useful features
Replaced by:
write(x”22”, x”F0”);
cs <= ’1’;
we <= ’1’;
addr <= x”22”;
data <= x”F0”;
wait until rising_edge(clk);
wait until falling_edge(clk);
cs <= ’0’;
we <= ’0’;
or:
sbi_write(x”22”, x”F0”);
A game changer for VHDL Verification 12
7
www.bitvis.no
What about more complex DUTs?
More interfaces
More simultaneous activity on interfaces
Value related corner cases
Cycle related corner cases
May need structured Constrained Random?
May need structured Functional Coverage?
Need some kind of Scoreboarding
Reusable IP/Libraries will save time
A game changer for VHDL Verification 13
www.bitvis.no
What about more complex DUTs?
More interfaces
More simultaneous activity on interfaces
Value related corner cases
Cycle related corner cases
May need structured Constrained Random?
May need structured Functional Coverage?
Need some kind of Scoreboarding
Reusable IP/Libraries will save time
Must have a good architecture and interface control
A game changer for VHDL Verification 14
8
www.bitvis.no
Chip level functional scenario
A game changer for VHDL Verification 15
PIF
SPI
P3
ETH ETH
P1
P2
uart
DMA
Intr ctrl
Sequential testing
Find some bugs?
Then behaves fine
Value related corners?
Parallel operation
Multi-end
cycle related corners
Bugs are hard to find
www.bitvis.no
Typical testbench approaches
A game changer for VHDL Verification 16
How do designers handle Cycle related corner cases
Test Controller
Clock Generator
Testcase Sequencer Adding threads CCL
Ad hoc "structure"
Lab ?
?
PIF
SPI
P3
ETH ETH
P1
P2
uart
DMA
Intr ctrl
- Fixed structure
Testcase Sequencer
???
- Overview??? - Reuse??? - Bad synchronisation
Good architecture, overview, reuse & sync.
9
www.bitvis.no
Wouldn't it be nice if we could ...
handle any number of interfaces in a structured manner?
reuse major TB elements between module TBs?
reuse major module TB elements in the FPGA TB?
read the test sequencer almost as simple pseudo code?
recognise the verification spec. in the test sequencer?
understand the sequence of event
- just from looking at the test sequencer
Is that feasible at all?
Wishful thinking?
A game changer for VHDL Verification 17
www.bitvis.no
BFM procedures only allow sequential operation (in test sequencer)
Parallelism requires BFMs in separate processes or entities
• But - processes may only have one single thread of execution
Entities / Components
Allow multiple active threads (processes or more entities)
May add dedicated processes for any extended functionality
Enables encapsulation
Makes re-use far more efficient
Using Components is the best solution - exactly as for Design
'VHDL Verification Components' : VVC
Parallelism & Encapsulation
A game changer for VHDL Verification 18
10
www.bitvis.no
VHDL Verification Component - VVC
A game changer for VHDL Verification 19
VHDL entity / component
Allows multiple thread(s)
• May include multiple processes or sub-components
Autonomous operation
Controllable via commands
Encapsulated complete verification functionality
More structured
Better overview
Easy to extend
Easy to reuse
A standard interface towards a bus structure / sequencer?
All issues exactly as for Design !!!
www.bitvis.no
VVC – In its simplest form
A game changer for VHDL Verification 20
DUT (UART VC)
“Ext.” I/O Other
Ports
Clocks
Bus
interface
Clock
Generation
Testcase
Sequencer
SBI_
VVC
Going from BFM to VVC
Using Bus access (SBI) as example - E.g. write to a register in DUT
Sequencer command using BFM:
sbi_write(C_ADDR_TX, x"2A");
Minimum VVC 1. Interpret command from sequencer in zero time
2. Execute respective BFM towards DUT
Sequencer command using VVC: sbi_write(SBI_VVCT, C_ADDR_TX, x"2A");
Results in above BFM being executed from VVC towards DUT
Very simple VVC – already allows simultaneous execution of BFMs on different interfaces
11
www.bitvis.no
TB implementation & understanding - Three main areas
A game changer for VHDL Verification 21
Clock
Generation
Testcase
Sequencer
SBI_VVC
UART_TX_VVC
UART (DUT)
RX Other
Ports
Clocks
Bus
interface
TX UART_RX_VVC
1: The complete Testbench with Test Harness
2: The Verification Components
3: The Central Test Sequencer
www.bitvis.no
1:The UVVM testbench
A game changer for VHDL Verification 22
UVVM is LEGO-like Testbench
Test harness
Build test harness
• Instantiate DUT and VVCs
• Connect VVCs to DUT
Build TB with test sequencer
• Instantiate test harness
• Include VVC methods pkg Connections included
• No additional connections
• May have VVCs anywhere
DUT
VVC A
VVC B
VVC C
Test seq.
VVC A library
VVC B library
VVC C library
VVC D
12
www.bitvis.no
2: VVC: VHDL Verification
Component
A game changer for VHDL Verification 23
SBI_VVC
Testcase Sequencer SBI_VVC
UART (DUT)
RX Other Ports
Clocks
Bus interface TX
Interpreter
- Is command for me?
- Is it to be queued?
- If not: Case on what to do
Executor
- Fetch from queue
- Case on what to do
- Call relevant BFM(s) & Execute transaction
Command Queue
Same main architecture in every VVC
• >90% same code in Interpreters
• Same command queue
• 90% same code in Executors - apart from BFM calls
VVC Generation
UART BFM to UART_VVC:
in less than 30 min
*_VVC
Clock
Generation
Testcase
Sequencer
SBI_VVC
UART_TX_VVC
UART (DUT)
R
X
Other
Ports
Clocks
Bus
interfac
e
T
X
UART_RX_VVC
UART_RX_VVC
Baudrate Checker
(1:Testbench : Easy to implement & understand by anyone)
Now - what about these VVCs?
www.bitvis.no A game changer for VHDL Verification 24
2: VVC: VHDL Verification
Component
*_VVC
(1:Testbench : Easy to implement & understand by anyone)
Now - what about these VVCs?
Interpreter
- Is command for me?
- Is it to be queued?
- If not: Case on what to do
Executor
- Fetch from queue
- Case on what to do
- Call relevant BFM(s) & Execute transaction
Command Queue
Bit-rate checker
Frame-rate checker
Gap checker
Queue
Response-Executor
Clock
Generation
Testcase
Sequencer
SBI_VVC
UART_TX_VVC
UART (DUT)
R
X
Other
Ports
Clocks
Bus
interfac
e
T
X
UART_RX_VVC
UART_RX_VVC
Baudrate Checker
13
www.bitvis.no
(Based on very structured TB and VVCs)
The sequencer is the most important part of the Testbench
Most man-hours will be (or should be) spent here
MUST be easy to understand, modify, maintain, ....
3: The test sequencer
A game changer for VHDL Verification 25
Clock
Generation
Testcase
Sequencer
SBI_VVC
UART_TX_VVC
UART (DUT)
RX Other
Ports
Clocks
Bus
interface
TX UART_RX_VVC
UART_RX_VVC
Baudrate Checker
Next: For each verification plan step - Check UVVM code in test sequencer
www.bitvis.no
Verification plan - In plain English (1)
A game changer for VHDL Verification 26
1. First Apply data to UART RX, then Wait for RX interrupt
then Read reg RX_DATA. (to show standard serial sequencer)
Pure sequential - Using Sequential BFMs only
uart_transmit(x"A1", "First byte on UART RX");
await_value(rx_empty, '0', 0, 12*bit_period, ERROR, message);
sbi_check(C_ADDR_RX_DATA, x"A1", "UART RX first byte");
Clock
Generation
Testcase
Sequencer
SBI_VVC
UART_TX_VVC
UART (DUT)
R
X
Other
Ports
Clocks
Bus
interfac
e
T
X
UART_RX_VVC
UART_RX_VVC
Baudrate Checker
Pure sequential - using UVVM VVC Framework + VVCs
uart_transmit(UART_VVCT,1, x"A1", "First byte on UART RX");
await_value(rx_empty, '0', 0, 12*bit_period, ERROR, message);
sbi_check(SBI_VVCT,1, C_ADDR_RX_DATA, x"A1", "UART RX first byte");
[ await_completion(SBI_VVCT,1, C_TIMEOUT_SBI_CHECK); ]
14
www.bitvis.no
2. Apply data to UART RX and read reg RX_DATA at the same time
Verification plan - In plain English (2)
A game changer for VHDL Verification 27
Parallel operation - using UVVM VVC Framework + VVCs
uart_transmit(UART_VVCT,1, x"A1", "First byte on UART RX");
sbi_check(SBI_VVCT,1, C_ADDR_RX_DATA, x"A1", "UART RX first byte");
Clock
Generation
Testcase
Sequencer
SBI_VVC
UART_TX_VVC
UART (DUT)
R
X
Other
Ports
Clocks
Bus
interfac
e
T
X
UART_RX_VVC
UART_RX_VVC
Baudrate Checker
www.bitvis.no
3. Read reg RX_DATA when this has just been up-loaded
Verification plan - In plain English (3)
A game changer for VHDL Verification 28
Parallel operation - using UVVM VVC Framework + VVCs
uart_transmit(UART_VVCT,1, x"A1", "First byte on UART RX");
insert_delay(SBI_VVCT,1, C_FRAME_TIME + 2 * C_CLK_PERIOD);
sbi_check(SBI_VVCT,1, C_ADDR_RX_DATA, x"A1", "UART RX first byte");
N
Clock
Generation
Testcase
Sequencer
SBI_VVC
UART_TX_VVC
UART (DUT)
R
X
Other
Ports
Clocks
Bus
interfac
e
T
X
UART_RX_VVC
UART_RX_VVC
Baudrate Checker
15
www.bitvis.no
Verification plan - In plain English (4)
A game changer for VHDL Verification 29
4. Extend the range in 3. to include all possible cycle corner cases
Similarly for Write on reg TX_DATA wrt. transmitting on TX
Similarly to check TX vs RX independence
Parallel operation - using UVVM VVC Framework + VVCs
for i in -C_CYCLES_BEFORE to C_CYCLES_AFTER loop
uart_transmit(UART_VVCT,1, my_data(i), "New byte on UART RX");
insert_delay(SBI_VVCT,1, C_FRAME_TIME + i * C_CLK_PERIOD);
sbi_check(SBI_VVCT,1, C_ADDR_RX_DATA, my_data(i-1), message);
await_completion(UART_VVCT,1, "Finish before next transmit");
end loop;
Make your own super-procedure
uart_transmit_sbi_check(
C_START_VALUE, -C_CYCLES_BEFORE, C_CYCLES_AFTER);
Even better: Make the VVCs handle the sequences
sbi_write(SBI_VVCT,1, C_ADDR_TX_DATA, RANDOM_TO_BUFFER, 1, 256);
uart_expect(UART_VVCT, 1, RX, FROM_BUFFER, 1, 256);
i
i
Clock
Generation
Testcase
Sequencer
SBI_VVC
UART_TX_VVC
UART (DUT)
R
X
Other
Ports
Clocks
Bus
interfac
e
T
X
UART_RX_VVC
UART_RX_VVC
Baudrate Checker
www.bitvis.no
Value related corner cases
A game changer for VHDL Verification 30
• Direct testing
• Constrained random
• Use std VHDL, Utility Library or OSVVM to generate values
• Apply from test sequencer as previously shown :
• 1. As individually initiated single accesses - as shown earlier
• 2. As a chunk of single accesses distributed "simultaneously"
• 3. As a single command to initiate multiple accesses from VVC
• Functional Coverage
• OSVVM is currently the best system for functional coverage
• Control from test sequencer as follows:
uart_transmit(UART_VVCT,1,TX, RANDOM, C_BUFFER_2, 256, msg);
uart_transmit(UART_VVCT,1,TX, RANDOM, C_BUFFER_1, DATA_COVERAGE, msg);
Clock
Generation
Testcase
Sequencer
SBI_VVC
UART_TX_VVC
UART (DUT)
R
X
Other
Ports
Clocks
Bus
interfac
e
T
X
UART_RX_VVC
UART_RX_VVC
Baudrate Checker
16
www.bitvis.no
Value related corner cases - Full solution
A game changer for VHDL Verification 31
UVVM vs OSVVM (Randomisation and Coverage)
UVVM & OSVVM are Complementary
Unified VHDL Verif. Meth.
Plug-in: OSVVM Random+Coverage - (with UVVM Utility Library?) - inside UVVM BFMs - inside UVVM VVCs - inside UVVM Test Sequencers
UVVM Utility Library
Logging, Alerts, Check value,
Stability, Awaits
UVVM VVC Framework TB Architecture, VVC architecture, Commands, Debug
OSVVM Randomisation
and Functional Coverage
BFMs & VVCs AXI4-lite/stream, Avalon, UART, I2C, SPI, SBI, GPIO, ...
www.bitvis.no
Debugging (1) - Verbosity ctrl
A game changer for VHDL Verification 32
• Important for both the DUT, TB and VVCs
• UVVM built-in messaging simplifies TB/VVC debugging
Showing only the distribution of commands from the test sequencer to the VVCs
2045ns TB seq.(uvvm) ->uart_transmit(UART_VVC,1,TX, x"AA"): . [15]
2045ns TB seq.(uvvm) ->await_completion(UART_VVC,1,TX, 2080 ns): . [16]
4005ns TB seq.(uvvm) ->sbi_check(SBI_VVC,1, x"00", x"AA"): RX_DATA. [17]
4005ns TB seq.(uvvm) ->await_completion(SBI_VVC,1, 2080 ns): . [18]
Showing only the completed commands (executed BFMs)
3805ns UART_VVC,1,TX uart transmit(x"AA") completed. [15]
4017ns SBI_VVC,1 sbi_check(x"00", x"AA")=> OK, read data = x"AA". RX_DATA [17]
Time stamp
Scope
Message
Showing distribution and execution - together
2045ns TB seq.(uvvm) ->uart_transmit(UART_VVC,1,TX, x"AA"): . [15]
2045ns TB seq.(uvvm) ->await_completion(UART_VVC,1,TX, 2080 ns): . [16]
3805ns UART_VVC,1,TX uart transmit(x"AA") completed. [15]
4005ns TB seq.(uvvm) ->sbi_check(SBI_VVC,1, x"00", x"AA"): RX_DATA. [17]
4005ns TB seq.(uvvm) ->await_completion(SBI_VVC,1, 2080 ns): . [18]
4017ns SBI_VVC,1 sbi_check(x"00", x"AA")=> OK, read data = x"AA". RX_DATA [17]
Clock
Generation
Testcase
Sequencer
SBI_VVC
UART_TX_VVC
UART (DUT)
R
X
Other
Ports
Clocks
Bus
interfac
e
T
X
UART_RX_VVC
UART_RX_VVC
Baudrate Checker
17
www.bitvis.no A game changer for VHDL Verification 33
Debugging (2) - Follow command
Showing all log messages, but for uart_transmit() only
2045ns TB seq.(uvvm) ->uart_transmit(UART_VVC,1,TX, x"AA"): . [15]
2045ns UART_VVC,1,TX uart_transmit(UART_VVC,1,TX, x"AA"). Command received [15]
2045ns TB seq.(uvvm) ACK received [15]
2045ns UART_VVC,1,TX uart_transmit(UART_VVC,1,TX, x"AA") Will be executed [15]
3805ns UART_VVC,1,TX uart transmit(x"AA") completed. [15]
3805ns UART_VVC,1,TX ..Executor: Waiting for command
3805ns UART_VVC,1,TX ..Interpreter: Waiting for command
Debugging the Testbench or VVC (Only showing uart_transmit)
Normal view: Showing executed commands only
3805ns UART_VVC,1,TX uart transmit(x"AA") completed. [15]
4017ns SBI_VVC,1 sbi_check(x"00", x"AA")=> OK, read data = x"AA" RX_DATA [17]
Command index
Clock
Generation
Testcase
Sequencer
SBI_VVC
UART_TX_VVC
UART (DUT)
R
X
Other
Ports
Clocks
Bus
interfac
e
T
X
UART_RX_VVC
UART_RX_VVC
Baudrate Checker
www.bitvis.no
Scoreboards
A game changer for VHDL Verification 34
Check that the result is as expected
Simplest Scoreboard is just a FIFO
Complex Scoreboards may include
• Out of order transactions
• One to one, one to many, many to one, many to many
May include modelling -including generation of the expected result
18
www.bitvis.no
Scoreboards in UVVM
A game changer for VHDL Verification 35
Level 0: Entry Level:
• Centr. seq. direct words
• Centr. seq. direct block
Level 1
• Centr. seq. indirect block
Level 2
• Centr. seq. requests local CR
• Centr. seq. requests local Coverage
Level 3 (roadmap)
• Introducing independent Monitors
• Introducing central scoreboard and modelling
Clock
Generation
Testcase
Sequencer
SBI_VVC
UART_TX_VVC
UART (DUT)
R
X
Other
Ports
Clocks
Bus
interfac
e
T
X
UART_RX_VVC
UART_RX_VVC
Baudrate Checker
Word or Block of words from Sequencer to VVC-A + VVC-B
Reference to block of words from Sequencer to VVC-A + VVC-B
CR and Functional Coverage request from Sequencer
to VVC-A + VVC-B
Monitors and scoreboards outside VVCs
www.bitvis.no
UVVM is just a step-by-step evolution on VHDL (2008)
UVVM can run on Modelsim, Questa, Active HDL, Riviera Pro
UVM is far more comprehensive and complex than UVVM
• But UVVM is sufficient for almost all testbenches
UVVM user threshold is a fraction of the UVM threshold
UVVM allows a gentle introduction to modern verification
UVVM allows the use of low cost tools
UVVM vs UVM
A game changer for VHDL Verification 36
Using VHDL for FPGA verification is:
• more cost efficient for almost all VHDL designers
• more time efficient for almost all VHDL designers
19
www.bitvis.no
Wouldn't it be nice if we could ...
handle any number of interfaces in a structured manner?
reuse major TB elements between module TBs?
reuse major module TB elements in the FPGA TB?
read the test sequencer almost as simple pseudo code?
recognise the verification spec. in the test sequencer?
understand the sequence of event
- just from looking at the test sequencer
Wishful thinking - Recap
A game changer for VHDL Verification 37
UVVM
Already available Open Source BFMs & VVCs:
AXI4-lite, AXI4-stream, Avalon MM, SBI, SPI, I2C, UART, GPIO
www.bitvis.no
UVVM is gaining momentum
A game changer for VHDL Verification 38
UVVM VVC Framework - Released February 2016
Great feedback on LinkedIn
Really good feedback from users (industrial and academia)
Presented at various conferences - with huge interest: - 'FPGA-Kongress' in Munich, July - 'FPGAworld' in Stockholm, September Next: 'DVCon-Europe in Munich, October
Next course
- Dec 6-8, München: Accelerating FPGA VHDL Verification (In Cooperation with Trias Mikroelektronik Gmbh)
Will soon set up more courses for 2017.
20
www.bitvis.no
Conclusions
A game changer for VHDL Verification 39
VHDL is the best language for Verification for almost all VHDL designers.
Understanding, Maintainability & Reuse are critical
Testbench architecture is key
No standard exists, but is really needed
UVVM handles the critical issues
UVVM yields a very good architecture at all levels
UVVM significantly improves Efficiency and Quality
UVVM allows the best reuse possible for VHDL verification
www.bitvis.no
Your partner for Embedded software and FPGA
Quality in every bit
A game changer for VHDL verification:
- Advanced VHDL Verification - Made simple - For anyone
Thank you