Making a Simple, Structured and Efficient Testbench Step-by-step

Slides:



Advertisements
Similar presentations
Configuration management
Advertisements

Designing Reusable Frameworks for Test Automation
© 2003 Xilinx, Inc. All Rights Reserved Course Wrap Up DSP Design Flow.
Using emulation for RTL performance verification
Bitvis Utility Library Concepts and usage Your partner for SW and FPGA
MapleLeaf, LLC SDLC Methodology. MapleLeaf, LLC, has established standard phases and processes in regards to project management methodologies for planning.
Internal Logic Analyzer Final presentation-part B
Software Frame Simulator (SFS) Technion CS Computer Communications Lab (236340) in cooperation with ECI telecom Uri Ferri & Ynon Cohen January 2007.
1/1/ / faculty of Electrical Engineering eindhoven university of technology Introduction Part 3: Input/output and co-processors dr.ir. A.C. Verschueren.
Alternate Software Development Methodologies
MotoHawk Training Model-Based Design of Embedded Systems.
Software Engineering COMP 201
Introductory Comments Regarding Hardware Description Languages.
Chapter 16 Control Unit Operation No HW problems on this chapter. It is important to understand this material on the architecture of computer control units,
Software Testing and Quality Assurance
Chapter 16 Control Unit Implemntation. A Basic Computer Model.
Chapter 15 IA 64 Architecture Review Predication Predication Registers Speculation Control Data Software Pipelining Prolog, Kernel, & Epilog phases Automatic.
Prof. John Nestor ECE Department Lafayette College Easton, Pennsylvania ECE VLSI System Design Lecture 4 - Advanced Verilog.
EE694v-Verification-Lect5-1- Lecture 5 - Verification Tools Automation improves the efficiency and reliability of the verification process Some tools,
Implementation. We we came from… Planning Analysis Design Implementation Identify Problem/Value. Feasibility Analysis. Project Management. Understand.
02/10/06EECS150 Lab Lecture #41 Debugging EECS150 Spring 2006 – Lab Lecture #4 Philip Godoy Greg Gibeling.
System/Software Testing
INFO 637Lecture #81 Software Engineering Process II Integration and System Testing INFO 637 Glenn Booker.
Workshop - November Toulouse Ronan LUCAS - Magillem Design Services 07/04/2011.
Finite State Machines. Binary encoded state machines –The number of flip-flops is the smallest number m such that 2 m  n, where n is the number of states.
Eric Keller, Evan Green Princeton University PRESTO /22/08 Virtualizing the Data Plane Through Source Code Merging.
© Janice Regan, CMPT 300, May CMPT 300 Introduction to Operating Systems Principles of I/0 hardware.
ASIC/FPGA design flow. FPGA Design Flow Detailed (RTL) Design Detailed (RTL) Design Ideas (Specifications) Design Ideas (Specifications) Device Programming.
1 Integration Verification: Re-Create or Re-Use? Nick Gatherer Trident Digital Systems.
Design Verification An Overview. Powerful HDL Verification Solutions for the Industry’s Highest Density Devices  What is driving the FPGA Verification.
Moving into Implementation SYSTEMS ANALYSIS AND DESIGN, 6 TH EDITION DENNIS, WIXOM, AND ROTH © 2015 JOHN WILEY & SONS. ALL RIGHTS RESERVED.Roberta M. Roth.
FPGA-Based System Design: Chapter 6 Copyright  2004 Prentice Hall PTR Topics n Design methodologies.
Slide 1 Construction (Testing) Chapter 15 Alan Dennis, Barbara Wixom, and David Tegarden John Wiley & Sons, Inc. Slides by Fred Niederman Edited by Solomon.
Verification Plan & Levels of Verification
The First in GPON Verification Classic Mistakes Verification Leadership Seminar Racheli Ganot FlexLight Networks.
VHDL Project Specification Naser Mohammadzadeh. Schedule  due date: Tir 18 th 2.
© 2012 xtUML.org Bill Chown – Mentor Graphics Model Driven Engineering.
Boost Verification Results by Bridging the Hw/Sw Testbench Gap by Matthew Ballance Verification Technologist Mentor Graphics.
Accessing I/O Devices Processor Memory BUS I/O Device 1 I/O Device 2.
ECE 545 Project 2 Specification. Project 2 (15 points) – due Tuesday, December 19, noon Application: cryptography OR digital signal processing optimized.
UVVM Utility Library Concepts and usage Your partner for SW and FPGA
Sub-Nyquist Sampling Algorithm Implementation on Flex Rio
CHAPTER 8 Developing Hard Macros The topics are: Overview Hard macro design issues Hard macro design process Physical design for hard macros Block integration.
This material exempt per Department of Commerce license exception TSU Xilinx On-Chip Debug.
Verification of FT System Using Simulation Petr Grillinger.
Firmware - 1 CMS Upgrade Workshop October SLHC CMS Firmware SLHC CMS Firmware Organization, Validation, and Commissioning M. Schulte, University.
Assignment write a short notes on 1.Manufacturing Testing. 2.Functional Testing. 3.Files and Text I/O. 4.Differentiate the cpld and fpga architecture.
Part A Final Dor Obstbaum Kami Elbaz Advisor: Moshe Porian August 2012 FPGA S ETTING U SING F LASH.
MVC WITH CODEIGNITER Presented By Bhanu Priya.
Lecture 1 – Overview (rSp06) ©2008 Joanne DeGroat, ECE, OSU -1- Functional Verification of Hardware Designs EE764 – Functional Verification of Hardware.
EE694v-Verification-Lect7-1- Verification Plan & Levels of Verification The Verification Plan Yesterdays and today’s design environment Design specification.
Basic Elements of Processor ALU Registers Internal data pahs External data paths Control Unit.
Test Plan: Introduction o Primary focus: developer testing –Implementation phase –Release testing –Maintenance and enhancement o Secondary focus: formal.
CDA 4253 FPGA System Design Hao Zheng Dept of Comp Sci & Eng USF.
Detecting Hardware Trojans in Unspecified Functionality Using Mutation Testing Nicole Fern K.-T. Tim Cheng UC Santa Barbara 1.
Software Development Process CS 360 Lecture 3. Software Process The software process is a structured set of activities required to develop a software.
C++ for Engineers and Scientists, Second Edition 1 Problem Solution and Software Development Software development procedure: method for solving problems.
Testing Overview Software Reliability Techniques Testing Concepts CEN 4010 Class 24 – 11/17.
Lecture 1 – Overview (rSp06) ©2008 Joanne DeGroat, ECE, OSU -1- Functional Verification of Hardware Designs EE764 – Functional Verification of Hardware.
Instructions to build HIBI_PE_DMA testing system (SOPC+NIOS II Eclipse) Lasse Lehtonen Last modification:
Presenter: Yi-Ting Chung Fast and Scalable Hybrid Functional Verification and Debug with Dynamically Reconfigurable Co- simulation.
Introduction Edited by Enas Naffar using the following textbooks: - A concise introduction to Software Engineering - Software Engineering for students-
Introduction Edited by Enas Naffar using the following textbooks: - A concise introduction to Software Engineering - Software Engineering for students-
Software life cycle models
Verification Plan & Levels of Verification
Debugging EECS150 Fall Lab Lecture #4 Sarah Swisher
UNIT 5 EMBEDDED SYSTEM DEVELOPMENT
UNIT 5 EMBEDDED SYSTEM DEVELOPMENT
Debugging EECS150 Fall Lab Lecture #4 Sarah Swisher
Test Cases, Test Suites and Test Case management systems
Presentation transcript:

Making a Simple, Structured and Efficient Testbench Step-by-step Espen Tallaksen www.bitvis.no Making a simple, structured and efficient Testbench, Step-by-step

About Bitvis Leading Vendor Independent Design Centre in Norway FPGA and Embedded SW services for customers From specification to final product – or any phase in between Good overview of pitfalls, time wasters and risks Focus on methodology, quality, efficiency and customers Located in Asker outside Oslo Making a simple, structured and efficient Testbench, Step-by-step

Products and courses from Bitvis Products from Bitvis 'Bitvis Utility Library' (Free and Open source, Directly downloadable) - Currently being used world wide 'UVVM' (Universal VHDL Verification Methodology) (UVL for VHDL) To be released 2014, Q4 'RegisterWizard', For generation of SW, Doc. and VHDL (bus IF, regs, etc.) To be released 2014, Q4 Courses from Bitvis 'FPGA Development Best Practices' - A two day course - A pragmatic approach to improving quality and efficiency. - So far Denmark, Sweden and Norway. May be held anywhere on request See our website for more offers http://bitvis.no/services/fpga-courses/ Making a simple, structured and efficient Testbench, Step-by-step

Why Testbenches and Simulation Cost of corrections Design stage Spec. Volum/ Field Ease of correction & debugging Far more control and observability Far faster iterations May have a structured bottom-­up verification. Detect bugs that cannot or most probably will not be detected in a lab-test Most bugs can be found with short simulations. Far more control and observability * Variables and intermediate signals can be viewed. * Environment and test­driver can also be viewed. - Must often coordinate I/O and internal state to verify corner cases. * Single stepping through code and signals is possible * “Embedded analysers” often sample on clock edges. Simulators show detailed signal sequences. Far faster iterations * even more important for time consuming P&R May have a structured bottom-­up verification. *** Detect bugs that most probably will not be detected in a lab-test * Detect bugs in functionality outside currently known scope. * Detect bugs that occur in abnormal situations * Detect bugs hard to provoke with current HW, SW or Test system Most bugs can be found with short simulations Making a simple, structured and efficient Testbench, Step-by-step

Verification: State of the FPGA community Even for quite simple modules (e.g. IRQC, SPI, I2C, UART): Average verification workload: 3 days to 3 weeks Often inadequate coverage Multiple Design, Synthesis, P&R iterations on full FPGA A low quality testbench Hopeless to understand for anyone else Difficult to extend Terrible to modify Tedious debugging Why? How can we improve? Making a simple, structured and efficient Testbench, Step-by-step

The Verification Paradox For proper simulation of FPGAs: More lines of verification code than for design source More time consuming than design Yet - a far lower effort is spent on verification Lower effort on partitioning Lower effort on structuring Lower effort on documentation A good TB is key to success: Simple 500 MH project: Saves 50-100 MH Complex 5000 MH project: Saves 500-1500 MH Significantly improves TTM, LCC, Quality Making a simple, structured and efficient Testbench, Step-by-step

Different TB requirements (1) The required level of verification differs a lot Low verification complexity single data path FPGA with some additional functionality modules with no or very simple corner cases E.g. GPIO, simple IRQC, CRC, FEC, decoder... Higher verification complexity:  Still needs basic infrastructure – as a platform  Need more advanced verification solutions on top Low verification complexity  Simple Testbench But – how simple? Making a simple, structured and efficient Testbench, Step-by-step

How simple should a TB be? Understanding somebody’s design module is often very hard Understanding somebody’s testbench is normally impossible And modifying it… ? Testbench Issues: Purpose: To verify DUT requirements Focus: Sufficient functional coverage with a minimum effort Test Sequencer Requirements:  Simple to write  Simple to understand and modify - by anyone  Simple to execute, debug and understand reports & results Making a simple, structured and efficient Testbench, Step-by-step

The development stages Ideally: - Specify  Design  Simulate  Synthesize  Test - Verification should start as early as possible But: Specification changes all the time…  Design must be extended or modified  Testbench or test cases must be extended or modified  Simulations must be re-executed  and the result must be checked again TB Readability, Flexibility and Extendibility is important Simulation result checking and readability is important Making a simple, structured and efficient Testbench, Step-by-step

Verification stages Will go through, stage-by-stage Make a verification specification – but avoid overhead and aim for single source Define your TB architecture and concept - but consider what is already available Implement TB architecture If available – start TB implementation from a template Implement test cases - In sensible steps and starting ASAP But also focus on: Simulation execution, Reporting, Debugging, Results Iterations Making a simple, structured and efficient Testbench, Step-by-step

irq_source(n) IRQC / n clk irq2cpu_ack bus interface arst irq2cpu The IRQC Bus interface irq_source(0) S Q R IER(0) IRR(0) ITR(0) ICR(0) D IPR(0) IPR(1..n) (Handling of one single irq_source. To be repeated for all sources.) irq2cpu irq2cpu_ack (from CPU) (to CPU) igr IRQ2CPU_ENA IRQ2CPU_ALLOWED Note: Uppercase names indicate software accessible registers (or dummy registers) irq_source(n) IRQC / n clk irq2cpu_ack bus interface arst irq2cpu Making a simple, structured and efficient Testbench, Step-by-step

1: Make a verification specification A verification specification is always needed, but The verification spec. should not be too extensive/detailed Required details could be added in a later spec. iteration A separate verification spec. document is normally not needed irq_source(n) S Q R IER(0) IRR(n) ITR(n) ICR(n) D IPR(n) irq2cpu irq2cpu_ack igr IRQ2CPU_ENA IRQ2CPU_ALLOWED Verification specification IRQC: Check defaults on output ports Check register defaults and access (write + read) Check register trigger/clear mechanism Check interrupt sources, IER, IPR and irq2cpu Check autonomy for all interrupts Check irq acknowledge and re-enable Making a simple, structured and efficient Testbench, Step-by-step

2: Define your TB architecture Two main categories for almost all modules - Defined by degree of interaction on interfaces 1. Basic TB infrastructure only 2. TB for simultaneous handling of multiple interfaces IRQC has very simple interfaces Very little and no critical interaction No contention issues Still – A bad design may always fail for corner cases Hence IRQC TB  Simple Verification  Simple TB arch./concept  Basic TB infrastructure Making a simple, structured and efficient Testbench, Step-by-step

Simple TB architecture Avoid all complex issues No support for handling simultaneous interfaces No queuing of commands Use a single, simple, understandable test sequencer Required support Logging Alert handling & reports Continuous actions Actions not handled by sequencer Min. interaction with sequencer Repeated actions Avoid unstructured copying Provide more info for multi-usage TB_IRQC DUT (IRQC) Test sequencer Support processes Support procedures & functions Making a simple, structured and efficient Testbench, Step-by-step

3: Implement TB architecture *** Inside TB architecture: ------------------------------------------------ -- Main process and test sequencer p_main: process begin log(ID_LOG_HDR, "Start simulation TB_IRQC"); clock_ena <= true; -- start clock generator *** *** Actual test sequence. To be filled in --========================================================== log(ID_LOG_HDR, "SIMULATION COMPLETED"); clock_ena <= false; -- to gracefully stop the simulation assert false report "End of simulation. (***Ignore this provoked failure.)" severity failure; wait; -- to stop completely end process p_main; Ready to implement the first tests TB workload so far: 10-30 min. Assuming no template is available… Generate TB entity with DUT instantiated Push-button in several tools (MsD, Emacs, etc.) Add support process for clock generation Allow enable/disable from seq. Add test sequencer process Main process in TB Controls everything! From TB initialization To termination of simulation TB_IRQC DUT (IRQC) Test sequencer Support processes Support procedures & functions Making a simple, structured and efficient Testbench, Step-by-step

4a: Implement First tests set_inputs_passive(VOID); apply_reset(VOID); log(ID_LOG_HDR, "Check defaults on output ports"); --------------------------------------------------------------- check_value(irq2cpu, '0', ERROR, "irq2cpu must be default inactive"); check_value(dout, x"00", ERROR, “dout must be default inactive"); Inside p_main process test sequencer process) First tests are important To verify compilation and elaboration OK To see that our Testbench is up and running To actually verify our first tests of the DUT First test: Output defaults Very simple test Allows module to be integrated First example on active support Using support procedures TB_IRQC DUT (IRQC) Test sequencer Support processes Support procedures & functions The above code will result in the following log/transcript: BV: 0 ns irqc_tb All inputs set passive BV: 60 ns irqc_tb Pulsed reset-signal - active for 10T BV: BV: 60 ns irqc_tb Check defaults on output ports BV:---------------------------------------------------------------------- BV: 60 ns irqc_tb check_value(sl 0)=> OK. irq2cpu must be default inactive BV: 60 ns irqc_tb check_value(slv x00)=> OK. dout must be default inactive Progress information Detailed progress information. Only interesting initially and for debug… Making a simple, structured and efficient Testbench, Step-by-step

Verbosity control Method to control amount of information To allow only selected groups of messages Allows reduction/increase of information – without modifying the code (comment/uncomment) Makes it possible to get a different set of messages depending on current simulation focus - e.g.: Debugging testcase or verification component Debugging a specific interface on DUT Debugging a data flow through DUT General simulation progress report. E.g. Modelsim vsim options: -version Print the version of the simulator -quiet Do not report 'Loading...' messages +sdf_verbose Display SDF annotator status messages  verbosity switches turning on or off a given set of messages Making a simple, structured and efficient Testbench, Step-by-step

Priority based verbosity control Every message has a given priority. Typically 1 for a high priority message and 6 for a low priority message. Every possible message from any part of the testbench has a given priority - normally defined when writing the message. E.g. log(2, "Packet header received”) Verbosity level will determine which messages to show E.g. a verbosity level of 2 means - Only priority 1 and priority 2 messages are let through (i.e. shown in the transcript or log); - whereas all other messages are blocked. Messages could be defined anywhere In Sequencer, BFMs, verification components, other processes, etc.. SystemVerilog’s verbosity control is basically priority based Making a simple, structured and efficient Testbench, Step-by-step

ID based verbosity control ID based verbosity allows log messages with different IDs to be shown or blocked depending on whether a given ID is enabled or disabled. Every message has a given ID Could be a string, a number, an enumerated, etc…. There are no priorities E.g. ID=58 and ID=216 are not prioritised in any way Example: log(216, "Packet header received”) Verbosity control will determine which messages to show Test sequencer may enable or disable message IDs dynamically depending on which messages are of interest. SystemC’s verbosity control is ID based Making a simple, structured and efficient Testbench, Step-by-step

Verbosity Control – Using Enumerated ID based verbosity control. Enumerated IDs A set of predefined IDs. E.g. ID_LOG_HDR or ID_BFM Example: log(ID_BFM, "UART byte received") Verbosity control Enable an ID using enable_log_msg(ID_BFM) Disable an ID using disable_log_msg(ID_BFM) Only enable or disable from the test sequencer! HHH X ID based verbosity control. - Enumerated IDs *** A set of predefined IDs *** E.g. ID_LOG_HDR or ID_BFM - Example: log(ID_BFM, "Packet header received”)  - Verbosity control - Enable an ID using enable_log_msg(ID_BFM) - Disable an ID using disable_log_msg(ID_BFM) - Only enable or disable from the test sequencer! ▼ Making a simple, structured and efficient Testbench, Step-by-step

Why ID based verbosity Far more flexible and controllable Real priorities are dependent on the situation application development stage problem at hand ID’s allows full control of verbosity May still define numeric priorities locally E.g. to indicate your assumed priorities May then choose to use numeric priority or ID ID-based verbosity is a superset of priority-based. Making a simple, structured and efficient Testbench, Step-by-step

The need for verbosity control Verbosity control is not required for the simplest TBs, but... Provides an advantage - even for most really simple TBs More efficient to use the same methodology for simple, medium and advanced TBs Most TBs normally end up more advanced than assumed Yields no overhead in even the simplest test sequencer Now – back to simple TB Making a simple, structured and efficient Testbench, Step-by-step

Back to TB: Do not waste time Most designers are unstructured wrt. verification Do not wait with your testbench and verification Do not dive into your first test case with no structure Do not use stupid testbench generators Do not write stimuli sequences by pure assignments Do not check results by checking outputs in the wave view Identify the need for subprograms – ASAP Immediately write subprogram when detecting repeated code Write a “self checking” testbench Requires good check subprograms Good Comments/documentation is NEVER wasted Provide in sequencer and in log/transcript Making a simple, structured and efficient Testbench, Step-by-step

Identify the need for subprograms *** Inside p_main process (test sequencer process) log_hdr(“Reset"); --------------------------------------------------------------- set_inputs_passive(VOID); apply_reset(VOID); log_hdr("Check defaults on output ports"); check_value(irq2cpu, '0', ERROR, "irq2cpu must be default inactive"); check_value(pif_irqc_dout, X"00", ERROR, “pif_irqc_dout must be default inactive"); Obvious subprogram candidates for IRQC Register Access Signal checkers Interrupt source pulsing? Interrupt acknowledge pulsing? (Report/log method) (Alert-handling) (reset, set_passive, …) Split in two categories Local – IRQC-dedicated Declare locally Common – non dedicated Declare in common package Share with others Local procedure – dedicated to inputs Evaluate whether this should be a common, slightly more general procedure… Common procedures. Logging and checking is required in all testbenches. irq_source(n) S Q R IER(0) IRR(n) ITR(n) ICR(n) D IPR(n) irq2cpu irq2cpu_ack igr IRQ2CPU_ENA IRQ2CPU_ALLOWED Making a simple, structured and efficient Testbench, Step-by-step

4b: Next test: Register Access First check register defaults Then general access (Write + Read) Provide Check-procedure – using Read Make dedicated procedures Will be common for many register interfaces  Bus access procedures are used to set up signal sequences to access internal registers  BFM : Bus Functional Model Making a simple, structured and efficient Testbench, Step-by-step

BFM / TLM Purpose Handle transactions at a high level E.g. Read, Write, Send packet, Config, etc More understandable for anyone Simpler code & Improved overview Uniform style, method, sequence, result Easy to add several very useful features Example: BFM for a CPU access to a module's register E.g. write 0xF0 (“11110000”) into a register at address 0x22 (“100010”) cs <= ’1’; we <= ’1’; addr <= ”00100010”; data <= ”11110000”; wait until rising_edge(clk); wait until falling_edge(clk); cs <= ’0’; we <= ’0’; Replaced by write(x”22”, x”F0”); Making a simple, structured and efficient Testbench, Step-by-step

Major BFM quality differences Inside a ”normal” BFM Pure Read, Write or Check transaction Additionally - Inside some BFMs Syncronization of access to the relevant clock Additionally - Inside good BFMs Normalisation of inputs Sanity-check on inputs Configuration of behaviour Logging of all accesses – with parameters and result Severity control and alert handling Verbosity control to potentially suppress log Making a simple, structured and efficient Testbench, Step-by-step

Making common, general BFMs function normalise( *** parameters (incl. implicitly constrained vectors) ) is *** Constants and Variables begin *** checks for legal constrained ranges *** check if (restriction = ALLOW_WIDER_SOURCE) then check_value(***Complex check and reporting***); *** corresponding checks for ALLOW_SHORTER_SOURCE, etc. *** Adapt value to target signal range and return end; Value and Signal widths are initially unknown Hence use unconstrained input vector for addr & data But unconstrained vectors do not define direction/range e.g constant data_exp : in std_logic_vector; Unconstrained data must often be “interpreted” Comparisons, Bit-manipulation, etc… Must Normalise Must Check for sanity – for early problem detection Making a simple, structured and efficient Testbench, Step-by-step

Make and run first BFM - Summary As soon as register interface is implemented: Make BFMs for write, read, check – in a common package Then make simplified BFMs First check defaults, and then write followed by check procedure sbi_write ( constant addr : in unsigned; constant data_in : in std_logic_vector; signal cs : inout std_logic; signal addr : inout unsigned; signal wr : inout std_logic; signal rd : inout std_logic; signal din : inout std_logic_vector; signal rdy : in std_logic; signal clk : in std_logic; constant clk_period : in time; constant msg : in string; constant scope : in string; *** more *** ) is begin normalise, sanity check, adapt, synch, write, wait?, alert?, log? end; procedure write ( constant addr : in unsigned; constant data_in : in std_logic_vector; constant msg : in string := "" ) is begin sbi_write(addr, data_in, pif_irqc_cs, pif_addr, pif_wr, pif_rd, pif_din, pif_rdy, clk, C_T_CLK, msg , C_SCOPE, ***more***); end; write(x"1", x"0F", "IER"); check(x"1", x"0F", error, "Pure readback"); Only if allowed by verbosity control if check ok BV: 172 ns. irqc_tb SBI write(x1, x00000F) completed. IER BV: 192 ns. irqc_tb SBI check(x1, ==> x00000F) completed. BV: Pure readback if check not ok BV: 172 ns. irqc_tb SBI write(x1, x00000F) completed. IER BV: BV:============================================================== BV: ERROR: BV: 192 ns. irqc_tb BV: value was: 'x00000E'. expected 'x00000F'. BV: (From SBI check(x1, x0F, Pure readback)) Not shown (or counted) if set to ignore Error Will break simulation if stop limit is reached Making a simple, structured and efficient Testbench, Step-by-step

TB: Workload so far? Total so far: 2-7 h Initial architecture Note: Assuming a TB infrastructure for log/alert/checks Otherwise only slightly more, but far less log/report/debug) Initial architecture 10-30 min First tests: Incl. simple local procedures Simple BFM set for a known protocol - More for protocol-oriented BFMs 1-4 h Overhead for Good BFM (Normalise, Log, Checks, etc.) - Given a good BFM template 30-60 min Reg. access tests and reg. interaction Total so far: 2-7 h Making a simple, structured and efficient Testbench, Step-by-step

4c: Continuing with TB for IRQC log(ID_LOG_HDR, "Check register trigger/clear mechanism", C_SCOPE); ------------------------------------------------------------ write(C_ADDR_ITR, x"AA", "ITR : Set interrupts"); check(C_ADDR_IRR, x"AA", ERROR, "IRR"); write(C_ADDR_ITR, x"55", "ITR : Set more interrupts"); check(C_ADDR_IRR, x"FF", ERROR, "IRR"); write(C_ADDR_ICR, x"71", "ICR : Clear interrupts"); check(C_ADDR_IRR, x"8E", ERROR, "IRR"); write(C_ADDR_ICR, x"85", "ICR : Clear interrupts"); check(C_ADDR_IRR, x"0A", ERROR, "IRR"); check(C_ADDR_IRR, x"5F", ERROR, "IRR"); write(C_ADDR_ICR, x"5F", "ICR : Clear interrupts"); check(C_ADDR_IRR, x"00", ERROR, "IRR"); ***Etc… Checking register interactions Just a series of Write and Check procedures Checking signal stability E.g. interrupt to CPU stable on or off Waiting for an event- inside a “window” E.g. interrupt to occur wrt. activated source *** time_tmp is set at a time from which the interrupt should be stable check_stable(irq2cpu, (now - time_tmp), error, "No previous active irq2cpu"); await_value(irq2cpu, '1', 0 ns, C_T_CLK, ERROR, "IRQ2CPU expected by now"); HHH ***  Making a simple, structured and efficient Testbench, Step-by-step

Making local procedures Local procedures should be made for any repeated actions for which a common (central) procedure does not make sense E.g. pulsing the interrupt acknowledge: Then evaluate whether this should be a common, slightly more general procedure… procedure pulse_irq2cpu_ack( constant dummy : in t_void ) is variable initial_value : std_logic := irq2cpu_ack; begin check_value(std_match(irq2cpu_ack, '0'), tb_warning, “Test seq.”, "irq2cpu_ack='1' when pulse_irq2cpu_ack() called"); wait until falling_edge(clk); irq2cpu_ack <= '1'; wait until rising_edge(clk); wait for (C_T_CLK / 4); irq2cpu_ack <= '0'; log(ID_SEQUENCER_SUB, “Test seq.”, "Pulsed irq2cpu_ack from 0 to 1 to 0"); end; In addition to the actual signal toggling – several tasks should be added. Check that ack is not already active HHH Important aspect of making a good testbench ***   Synchronise pulse on and off Report the pulse to the log Dedicated msg ID Making a simple, structured and efficient Testbench, Step-by-step

4d: Completing the Test case Add missing sections acc. to verif. spec. Fill inn all sections Section headers  Log headers Details inside sections  Std. log messages End simulation by reporting all alerts Making a simple, structured and efficient Testbench, Step-by-step

Report summaries report_alert_counters() Reports all alert counters =========================================================================== BV: *** SUMMARY OF ALL ALERTS *** BV: ====================================================================== BV: REGARDED EXPECTED IGNORED Comment? BV: NOTE : 0 0 0 ok BV: TB_NOTE : 0 0 0 ok BV: WARNING : 0 0 0 ok BV: TB_WARNING : 0 0 0 ok BV: MANUAL_CHECK : 0 0 0 ok BV: ERROR : 0 0 0 ok BV: TB_ERROR : 0 0 0 ok BV: FAILURE : 0 0 0 ok BV: TB_FAILURE : 0 0 0 ok BV: >> No mismatch between counted and expected serious alerts Making a simple, structured and efficient Testbench, Step-by-step

Verbosity control – revisited Bitvis: 0.0 ns TB seq. Start Simulation of TB for IRQC Bitvis: -------------------------------------------------------------------------- Bitvis: Bitvis: 110.0 ns TB seq. Check defaults on output ports Bitvis: 110.0 ns TB seq. Check register defaults and access (write + read) Bitvis: 217.5 ns TB seq. Check register trigger/clear mechanism Bitvis: 337.5 ns TB seq. Check interrupt sources, IER, IPR and irq2cpu Bitvis: 637.5 ns TB seq. Check autonomy for all interrupts Bitvis: 2167.5 ns TB seq. Check irq acknowledge and re-enable Bitvis: 2367.5 ns TB seq. Check Reset Bitvis: ===================================================================== Bitvis: *** SUMMARY OF ALL ALERTS *** etc…… Log-result for TB for IRQC Only ID_LOG_HDR enabled: 32 lines Log headers only All IDs enabled: 500 lines All details IRQC is an extremely simple module… Making a simple, structured and efficient Testbench, Step-by-step

Reduced design time Early structured TB & test cases improve efficiency Forces you to think about the spec. in a different way Avoids wasting time on “simple/stupid” simulations Do NOT make initial “force/check” verification Yields a good TB & Test case structure - right away and provides a “shell” for filling in tests as needed Early verification of selected issues is useful May add verification tasks as IRQC is being designed No time wasted – as all verification is useful Continuous TB update on spec/design-changes Late changes are far simpler to handle Making a simple, structured and efficient Testbench, Step-by-step

Reduced debug time Far better simulation debug support Logging sequences and every single action Possibility to reduce amount of logging Good alert messages with mismatch report And preceeding progress report Far better quality on lab-releases Minimised need for lab debug Easier to make new test cases to trace lab problems Making a simple, structured and efficient Testbench, Step-by-step

Documentation May easily extract required documentation Examples from IRQC testbench Verification specification Check register trigger/clear mechanism 1 Code sectioning for multiple test areas 2 Code commenting ITR should set IRR Simulation status when completed (No alerts reported) 4 Intermediate progress report “Check register trigger/clear mechanism” 3 Alerts “ERROR: ITR should set IRR” Check-reports “Checked: ITR should set IRR” Debug support “ERROR: ITR should set IRR; Was ‘00110011’, expected ‘11111111’” Progress report and debug support “pif_write(x3, x00A5A5) completed. ITR” May easily extract required documentation Single source for all documentation/commenting Making a simple, structured and efficient Testbench, Step-by-step

Bitvis Utility Library Open source VHDL library – Released April 2013 Initial version VHDL 2002/08 compliant only VHDL '93 version released August 2013 Important features Logging and Verbosity control Alert handling and reporting Simple randomisation Basic testbench checking and await procedures Simplicity is key Strong focus on allowing real simple infrastructure usage Project adaptable behaviour and log layout Quick Reference for all methods is provided Advanced method versions allow added complexity - Open source VHDL library – to be released tomorrow X VHDL 2002 compliant (using 'ieee_proposed' library) X VHDL '93 version will be provided if sufficient demand  - Important features - Logging and Verbosity control *** As shown in this presentation - Alert handling and reporting - Simple randomisation - Basic testbench checking and await procedures - Simplicity is key - Strong focus on allowing real simple infrastructure usage *** Very low user threshold. X Project adaptable behaviour and log layout X Quick Reference for all methods is provided - Advanced method versions allow added complexity *** More parameters for more complex operation ▼ Making a simple, structured and efficient Testbench, Step-by-step

Verification effort for a single module/FPGA Coverage Total of 4 hours for complete TB for IRQC... Structured approach using an already available infrastructure 100% Structured approach Typical Project Approved No “wasted” time up front Straight on coding Time (MH) Making a simple, structured and efficient Testbench, Step-by-step

Conclusions for my TB for IRQC A uniform and structured methodology More compact and understandable code Single source for spec, code comments and logging Far easier to write, read, modify and extend a test case Easy to execute and understand output/status Debugging is much faster with a good progress report Total of 4 hours for complete TB for IRQC  Very efficient TB implementation and design debug Making a simple, structured and efficient Testbench, Step-by-step

Making a Simple, Structured and Efficient Testbench Step-by-step The end. Making a simple, structured and efficient Testbench, Step-by-step 42