09mec017

Design & Implementation of MitigationTechniques for Single Event Upset

in SRAM FPGA

Major Project Report

Submitted in partial fulfillment of the requirements

for the degree of

Master of Technology

in

Electronics and Communication

(VLSI Design)

By

SAVANI VIJAY GOPALBHAI

(09MEC017)

Department of Electronics and Communication Engineering

Institute of Technology

Nirma University

Ahmedabad-382 481

May 2011

Design & Implementation of MitigationTechniques for Single Event Upset

in SRAM FPGA

Major Project Report

Submitted in partial fulfillment of the requirements

for the degree of

Master of Technology in Electronics and Communication

(VLSI Design)

By

SAVANI VIJAY GOPALBHAI

(09MEC017)

Guided By

Prof. N. P. Gajjar

Department of Electronics and Communication Engineering

Institute of Technology

Nirma University

Ahmedabad-382 481

May 2011

iii

DECLARATION

I here by declare that,

1 ) This thesis comprises my original work towards the degree of Master of Technology

in Electronics and Communication (VLSI Design) at Nirma University and has

not been submitted elsewhere for a Degree.

2 ) Due acknowledgement has been made in the text to all other material used.

- Vijay G Savani

iv

CERTIFICATE

This is to certify that the M.Tech Major Project thesis entitled “Design & Imple-

mentation of Mitigation Techniques for Single Event Upset in SRAM

FPGA” submitted by Savani Vijay Gopalbhai (09MEC017) towards the par-

tial fulfillment of the requirements for the Degree of Master of Technology (Electronics

& Communication) in the field of VLSI Design at Institute of Technology, Nirma

University, Ahmedabad, is the record of the work carried out under our supervision

and guidance. The work submitted has in our opinion reached a level required for

being accepted for examination. The result embodied in this project work to the best

of our knowledge has not been submitted to any other University or Institute for the

award of any Degree.

Date: Place: Ahmedabad

Prof. N. P. Gajjar Dr. N. M. Devashrayee

Sr. Associate Professor, PG Co-ordintor,

EC-VLSI Design, VLSI Design,

Institute of Technology, Institute of Technology,

Nirma University, Ahmedabad. Nirma University, Ahmedabad.

Prof. A. S. Ranade Dr. K Kotecha

HOD, EE Department, Director,

Institute of Technology, Institute of Technology,

Nirma University, Ahmedabad. Nirma University, Ahmedabad.

v

ACKNOWLEDGEMENT

First, I would like to thank “Almighty GOD” for the wonderful life that I have.

Thank you very much GOD for keeping the courage and the enthusiasm in me every

day. I would like to thank my FAMILY for always looking after me and for contin-

uous encouragement.

I express my sense of gratitude to my project guide Prof. N. P. Gajjar, Sr. Asso-

ciate Professor, EC Department, Institute of Technology, Nirma University, for all of

his guidance, encouragement and patience which enable me to carry out my project

work. He inspired me with his original and innovative ideas and provided his technical

guidance in the project. Without his never ending support this project would never

have been completed.

It is my pleasure to thank Dr. N. M. Devashrayee (PG Coordinator-VLSI Design),

Department of Electronics and Communication Engineering, Institute of Technology,

Nirma University, Ahmedabad, for his support and for the knowledge and experience

they have helped me obtain.

Appreciation is expressed for all research colleagues & all those with whom I worked

and interacted at Institute of Technology, for their help and co-operation in the course

of my project work. I am also thankful to Nirma University Management for providing

me an opportunity to work in the prime institute of the country and providing me

excellent working environment together with required resources.

Vijay G. Savani

(09MEC017),

M.Tech (VLSI Design),

Institute of Technology,

Nirma University.

vi

ABSTRACT

The FPGAs (SRAM BASED), operating in space environment, are perturbed by

charged particles, which affect the circuit in different ways. This work details the

mitigation techniques for one of these effects called Single Event Upset(SEUs).

With the progress of technology, the highly scaled devices exhibit an increased sen-

sitivity to SEUs due to a reduced feature size and a proportional increase in device

density. FPGA based designs are more susceptible to SEUs compared to ASIC de-

signs and it is more harmful when FPGAs are used in space and defence applications.

The Goal of the project is to Design, Develop and Implement the miti-

gation techniques for SEU, which can be carried out using various techniques.

This thesis addresses some developed solutions to turn CMOS memory cells SEU

immune by system where software solutions and hardware redundancy is used to mit-

igate SEU. Various design based solutions like Spatial Redundancy(Triple Modular

Redundancy), Temporal Redundancy and Scrubbing(Reconfiguration) are discussed

in the thesis. Apart from the mitigation, infrequent and unpredictable nature of real

SEUs, small scale testing of their effects and system verification is impractical. To

perform this tasks the SEU monitor system is designed and implemented on FPGA.

The system along with controller macro can emulate an SEU by deliberately injecting

an error into the FPGA configuration so that its subsequent detection and correction

can be confirmed. It can also be used to assess SEU mitigation circuits implemented

in a design. Finally, the Self Correcting System is implemented using the SEU mon-

itor system and self reconfigurable system, which detects and corrects the error. If

the correction of error is not possible then it reconfigure the user design. The thesis

describes the operation and architecture of the proposed logic design as well as its

implementation in Xilinx virtex-5 FPGA.

Contents

Declaration iii

Certificate iv

Acknowledgement v

Abstract vi

Contents vii

List of Figures ix

List of Tables xi

1 Introduction 11.1 Radiation Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Different Types of Single Event Effects . . . . . . . . . . . . . . . . . 41.3 Effects of SEUs on FPGAs . . . . . . . . . . . . . . . . . . . . . . . . 51.4 Report Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 SEU Mitigation Techniques 102.1 Technology Based Techniques . . . . . . . . . . . . . . . . . . . . . . 112.2 Design Based Techniques . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2.1 Spatial Redundancy . . . . . . . . . . . . . . . . . . . . . . . 112.2.2 Temporal Redundancy . . . . . . . . . . . . . . . . . . . . . . 122.2.3 Scrubbing(Reconfiguration) . . . . . . . . . . . . . . . . . . . 13

3 Implementation of Mitigation using TMR & PR 153.1 Basics of Triple Modular Redundancy . . . . . . . . . . . . . . . . . . 153.2 Granularity of TMR . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.3 Various TMR Implementation Methodology . . . . . . . . . . . . . . 17

3.3.1 TMR with Single Voter . . . . . . . . . . . . . . . . . . . . . . 173.3.2 TMR with Triplicated Voter . . . . . . . . . . . . . . . . . . . 183.3.3 Temporal Data Sampling . . . . . . . . . . . . . . . . . . . . . 18

vii

CONTENTS viii

3.3.4 Optimal Design of TMR . . . . . . . . . . . . . . . . . . . . . 213.4 Basics of Partial Reconfiguration . . . . . . . . . . . . . . . . . . . . 233.5 Methods of Partial Reconfiguration . . . . . . . . . . . . . . . . . . . 24

3.5.1 Module Based Partial Reconfiguration . . . . . . . . . . . . . 243.5.2 Difference Based Partial Reconfiguration . . . . . . . . . . . . 27

3.6 Bus Macro Communication . . . . . . . . . . . . . . . . . . . . . . . 273.7 Programming Medium for Configuration . . . . . . . . . . . . . . . . 303.8 Development of a Self Reconfigurable System . . . . . . . . . . . . . . 33

4 Error Detection and Correction 374.1 Configuration Memory . . . . . . . . . . . . . . . . . . . . . . . . . . 374.2 FRAME ECC Primitives . . . . . . . . . . . . . . . . . . . . . . . . . 38

4.2.1 Readback CRC Algorithm . . . . . . . . . . . . . . . . . . . . 424.3 Internal Configuration Access Port (ICAP) . . . . . . . . . . . . . . . 43

5 Implementation of Self Correcting System 445.1 SEU Controller macro . . . . . . . . . . . . . . . . . . . . . . . . . . 445.2 SEU Monitor System . . . . . . . . . . . . . . . . . . . . . . . . . . . 485.3 Self Correcting System . . . . . . . . . . . . . . . . . . . . . . . . . . 51

6 Implementation of SEU mitigation Techniques 536.1 TMR Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . 536.2 Implementation of Self Reconfigurable System . . . . . . . . . . . . . 576.3 Implementation of Systems . . . . . . . . . . . . . . . . . . . . . . . . 62

7 Conclusion and Future Work 677.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 677.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

A VIRTEX-5 Platform FPGA 69A.1 Virtex-5 Device Functional Blocks: . . . . . . . . . . . . . . . . . . . 70

B Tools & PR Design Implementation Flow 72B.1 Tools and Design Board . . . . . . . . . . . . . . . . . . . . . . . . . 72

B.1.1 Xilinx EDK 9.2.02 . . . . . . . . . . . . . . . . . . . . . . . . 72B.1.2 Xilinx ISE 9.2.04i with Partial-Reconfiguration Patch . . . . . 73B.1.3 Xilinx PlanAhead 10.1 . . . . . . . . . . . . . . . . . . . . . . 73B.1.4 Virtex-V Evaluation Board . . . . . . . . . . . . . . . . . . . . 74

B.2 PR Design Implementation Flow . . . . . . . . . . . . . . . . . . . . 75

Glossary 82

References 83

List of Publication 86

List of Figures

1.1 SEU in CMOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 6-Transistor Based SRAM Storage Cell . . . . . . . . . . . . . . . . . 61.3 SRAM based FPGA architecture(regular array) . . . . . . . . . . . . 71.4 SEU Sensitive Configuration Bit Storage . . . . . . . . . . . . . . . . 8

2.1 Basics of TMR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.2 Temporal Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.1 Majority Voter Circuit and Truth Table . . . . . . . . . . . . . . . . 163.2 TMR of One-Bit Counter . . . . . . . . . . . . . . . . . . . . . . . . 183.3 Single Voter TMR Counter with Sequential SEUs . . . . . . . . . . . 183.4 Triple Voted TMR of One-Bit Counter . . . . . . . . . . . . . . . . . 193.5 Triple Voted TMR Counter with Sequential SEUs . . . . . . . . . . . 193.6 Proposed Temporal Data Sampling . . . . . . . . . . . . . . . . . . . 203.7 Clocking Scheme for Temporal Sampling . . . . . . . . . . . . . . . . 213.8 TMR register with voters and scrubbing . . . . . . . . . . . . . . . . 223.9 Module Based Partial Reconfiguration Design Flow . . . . . . . . . . 263.10 Bus macro used between PR Logic and Fixed Logic . . . . . . . . . . 283.11 Bus Macro between two Region . . . . . . . . . . . . . . . . . . . . . 293.12 Physical Implementation of Bus Macro . . . . . . . . . . . . . . . . . 293.13 Methods of FPGA Reconfiguration . . . . . . . . . . . . . . . . . . . 303.14 Bitstream Flow for the reconfiguration . . . . . . . . . . . . . . . . . 323.15 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.16 Up Sampler block diagram . . . . . . . . . . . . . . . . . . . . . . . . 343.17 Down Sampler block diagram . . . . . . . . . . . . . . . . . . . . . . 353.18 Interconnection of the reconfigurable system . . . . . . . . . . . . . . 35

4.1 Configuration Memory Data Frame . . . . . . . . . . . . . . . . . . . 384.2 FRAME ECC primitive (Virtex-5) . . . . . . . . . . . . . . . . . . . 394.3 ICAP Primitive (Virtex-5) . . . . . . . . . . . . . . . . . . . . . . . . 43

5.1 SEU Controller macro . . . . . . . . . . . . . . . . . . . . . . . . . . 455.2 SEU Monitor System . . . . . . . . . . . . . . . . . . . . . . . . . . . 485.3 Port mapping of System . . . . . . . . . . . . . . . . . . . . . . . . . 49

ix

LIST OF FIGURES x

5.4 Internal signal Integration . . . . . . . . . . . . . . . . . . . . . . . . 495.5 Software for Monitor System using SDK . . . . . . . . . . . . . . . . 505.6 Block Diagram of Self Correcting System . . . . . . . . . . . . . . . . 51

6.1 TMR with single voter(Technology Schematic) . . . . . . . . . . . . . 536.2 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 546.3 Implementation of TMR with triple voter . . . . . . . . . . . . . . . . 546.4 Technology Schematic of TMR(with Voters and Scrubbing) . . . . . . 556.5 Temporal Data Sampling Technology Schematic . . . . . . . . . . . . 556.6 Up Sampler RTL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 576.7 Up Sampler Simulation Results . . . . . . . . . . . . . . . . . . . . . 586.8 Implementation of Down Sampler . . . . . . . . . . . . . . . . . . . . 586.9 Testing of Reconfigurable System . . . . . . . . . . . . . . . . . . . . 596.10 Finding no of clock cycles to Reconfigure . . . . . . . . . . . . . . . . 596.11 Dynamic PR(Performing Reconfiguration of UP Sampler) . . . . . . . 606.12 RTL of SEU Monitor System . . . . . . . . . . . . . . . . . . . . . . 626.13 Testing of SEU Monitor System . . . . . . . . . . . . . . . . . . . . . 626.14 Checking CRC of SEU Monitor System . . . . . . . . . . . . . . . . . 636.15 Error Injection using Monitor System . . . . . . . . . . . . . . . . . . 646.16 Testing of Self Correcting System . . . . . . . . . . . . . . . . . . . . 66

A.1 Virtex FPGA architecture . . . . . . . . . . . . . . . . . . . . . . . . 69

B.1 PlanAhead PR Design Flow . . . . . . . . . . . . . . . . . . . . . . . 74B.2 Create XPS Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75B.3 Add Software Application . . . . . . . . . . . . . . . . . . . . . . . . 77B.4 Draw Partial Module . . . . . . . . . . . . . . . . . . . . . . . . . . . 78B.5 DRC Run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79B.6 Luanch Run of static RM . . . . . . . . . . . . . . . . . . . . . . . . 80B.7 PR assemble of Design . . . . . . . . . . . . . . . . . . . . . . . . . . 80

List of Tables

4.1 Interpretation of syndrome bit . . . . . . . . . . . . . . . . . . . . . . 404.2 Frame ECC Error Codes . . . . . . . . . . . . . . . . . . . . . . . . . 404.3 Parity Bit Error Diagnosis . . . . . . . . . . . . . . . . . . . . . . . . 42

5.1 Controller Macro Operational Modes . . . . . . . . . . . . . . . . . . 47

6.1 Comparison of TMR techniques . . . . . . . . . . . . . . . . . . . . . 566.2 Device Utilization Summary (For combined Up and Down Sampler) . 606.3 Area Comparison for Up and Down Sampler . . . . . . . . . . . . . . 616.4 Timing Analysis for Reconfiguration . . . . . . . . . . . . . . . . . . 616.5 Resource Utilization comparison for Monitor System . . . . . . . . . 65

xi

Chapter 1

Introduction

Given the remarkable success of reprogrammable logic devices in areas such as telecom-

munications and defence applications, it is only natural that an interest should arise

in their use for Space based Electronics solutions as well. However, while such devices

present numerous advantages in terms of design flexibility, they come with the draw-

back of being susceptible to bit upsets induced by radiation, more commonly known

as “Single Events Upsets (SEUs)”.

FPGAs have been very attractive for space applications over the past decade. Indeed,

the main advantage provided by gate arrays is the elimination of the large overhead

cost of developing custom ASICs. One of the major reasons being the prospect of

performing post-launch design optimizations or changes in spacecraft objectives. An-

other advantage, their inherent re-programmability feature has been fully exploited

for prototyping purposes. While antifuse technology has several inherent limitations

that make SRAM-based FPGAs more attractive. First, once a device is programmed,

it cannot be changed, additional devices have to be programmed and physically re-

place the installed devices. Second, available antifuse gate arrays are considerably

smaller in gate count than SRAM configurable gate arrays [7]. But the problem is

that the FPGAs are known to be highly susceptible to SEUs. SEUs can result in

deviations from expected component behavior.

1

CHAPTER 1. INTRODUCTION 2

1.1 Radiation Effects

There are two main categories of radiation effects that are relevant for Static Random

Access Memory (SRAM) based Field Programmable Gate Arrays (FPGAs) in space.

1. Total Dose Effects

2. Single Event Effects (SEEs)

Total Dose Effects are cumulative effects that induce degradation of electrical pa-

rameters at the device, circuit, and system levels. They are induced by the total

amount of ionizing energy deposited by photons or particles such as electrons, pro-

tons, or heavy ions. Single Event Effects are induced by the passage of a single high

energy proton or heavy ion through a device or a sensitive region of a microcircuit.

SEEs in digital integrated circuits (ICs) can be either destructive (e.g., Single-Event

Latch-up), or Non-destructive (e.g., Single Event Upset), such as the occurrence of

transient faults in combinational and sequential logic [7].

Single event upset (SEU) is defined by NASA as “Radiation induced errors in

microelectronic circuits caused when charged particles (usually from the

radiation belts or from cosmic rays) lose energy by ionizing the medium

through which they pass, leaving behind a wake of electron hole pairs”[7].

When designing in-orbit, space-based, or extra-terrestrial applications in hostile radi-

ation environments, users must consider the effects of charged particles such as heavy

ions or protons. As these charged particles travel through the FPGA as shown in

figure 1.1, which can alter the logic state of any static memory element, resulting in

single event upsets (SEUs). An SEU in the configuration memory array can have

adverse effect on expected FPGA functionality [2].

Recent advancements in the semiconductor industry have led to the fabrication of

highly scaled devices. These devices exhibit an increased sensitivity to SEUs due to


Figure 1.1: SEU in CMOS

a reduced feature size and a proportional increase in device density. In the case of

programmable logic devices such as FPGAs, SEU effects can be much more severe.

Since FPGAs utilize a configuration memory array to define the logic function, an

SEU occurring in a single bit in the array can lead to an unexpected alteration of the

original design [8].

SEUs are soft errors, and are nondestructive. In FPGAs, a multitude of latches, also

called memory cells or RAM bits, define all logic functions and on-chip interconnects.

Such latches are similar to the 6-transistor storage cells (as shown in figure 1.2, whose

function is described in section 1.3) used in SRAMs, which has proved to be sensitive

to single event upsets caused by high-energy neutrons. The faults have been observed

as bit errors in memories.

As the microelectronics industry has advanced, Integrated Circuit (IC) design in gen-

eral and reconfigurable architectures (FPGAs, reconfigurable SoC and etc) in par-

ticular have experienced dramatic increase in density and speed due to decrease in

feature sizes with which these devices are manufactured. The effects of scaling on the

single event response of microelectronics are a direct result of the physics of energy

loss, charge collection, and upset due to a cosmic ray striking a junction in an IC


device. When an energetic ion passes through any material it loses energy through

interactions with the bound electrons, causing an ionization of the material and the

formation of a dense track of electron-hole pairs. The rate at which the ion loses

energy is the stopping power (dE/dx). The incremental energy dE is usually mea-

sured in units of MeV while the material thickness is usually measured as a mass

thickness in units of mg/cm2. The radiation effects community has adopted the term

LET (Linear Energy Transfer) for the stopping power. An ion with an LET of 100

MeV-cm2/mg deposits approximately 1pC of electron-hole pairs along each micron of

its track through silicon [1].

1.2 Different Types of Single Event Effects

The single event effects have different categories as follow. These are based on the

definitions provided by the Joint Electron Device Engineering Council (JEDEC) asso-

ciation [16]. Any observable or measurable change in state or performance occurring

in a microelectronic device, component or system that can be digital, analog or opti-

cal, resulting from a single energetic particle strike called Single Event Effects (SEEs).

SEEs comprise SEUs and several other effects defined below.

1. Single Event Upset (SEU): A soft error resulting from a transient signal induced

by a single energetic particle strike.

2. Single Event Transient (SET): A transitory voltage spike (change of state) at a

node of an integrated circuit resulting from single particle collisions.

3. Single Hard Error (SHE): An SEU which causes a permanent change to the

operation of a device. An example is a stuck bit in a memory device.

4. Single Event Latchup (SEL): An abnormal high-current state caused when single

energetic particles pass through sensitive regions of the device, resulting in (in

most cases) loss of device functionality.


5. Single Event Burnout (SEB): An event caused by a single particle collision in-

ducing a localized high-current state in a device, and resulting in its destruction.

1.3 Effects of SEUs on FPGAs

In the presence of electric fields, the electron-hole pairs quickly separate as they drift in

opposite directions in the field and are quickly collected by whatever voltage sources

are responsible for the field, thus producing a current transient. In bulk CMOS

designs, such electric fields are present across every PN junction in the device. If an

ion strikes a junction connected to a signal node, a current transient is subsequently

observed on the signal node as the electric fields in the junction and funnel regions

separate the electron and hole carriers [1].

SEUs are inherently transient, and might go undetected in several cases. But in

storage circuits like latches and memory structures these events get stored. The

SRAM structure that commonly acts as the configuration memory for FPGAs is

based on the 6-transistor storage cell. However, the cell is known to be vulnerable

to SEUs, consequently compromising the functionality of the FPGA [8]. Figure 1.2

shows a possibility of how an SEU might affect the cell.

The nMOS pass gates T5 and T6 are used to read/write values to the cell. For a

write operation, bit is set to the desired logic, say ‘0’ and ˜bit to logic ‘1’. The pass

transistors are then activated via the RD/WR Enable line and the value is latched in

by the cross-coupled inverters. Similarly to read the state of the cell the pass gates

are once again enabled. Now suppose that the cell holds a logic ‘0’ when a current

pulse from a particle collision arrives at node A. The pulse travels to the gate of

T3 and if its magnitude is high enough, is capable of turning on T3 thereby pulling

node B to logic ‘0’. The positive feedback ensures that the change propagates to

the other two transistors, eventually resulting in a logic ‘1’ getting stored in the cell.

If the structure originally held a logic ‘1’, then a similar event occurring at node B


Figure 1.2: 6-Transistor Based SRAM Storage Cell

would result in a logic reversal. It is important to note that the occurrence of such

logic changes also depends on the design and a particle collision might not necessarily

result in a bit-flip [8].

Basically the FPGA memory is divided into two part one is user and other is config-

uration memory and an SEU can affect either of the two. The former case results in

an undefined state getting latched in a register element. Though this might cause a

temporary disruption in normal functionality, the original design remains unchanged.

In the latter case however, bit-flips can cause an alteration of the logic and the af-

fected area needs to be reprogrammed to remove the fault. Faults in this category

can be either in the routing, logic resources or in the IOBs.

Xilinx’s FPGA

In general, Xilinx FPGAs have an array composed of configurable logic blocks (CLBs)

surrounded by programmable input/output blocks (IOBs), all interconnected by a


hierarchy of fast and versatile routing resources. Each CLB has a set of look-up

tables (LUT), multiplexers, and flip-flops, which are divided into slices. A LUT is

a logic structure able to implement a Boolean function as a truth table. The CLBs

provide the functional elements for constructing logic while the IOBs provide the

interface between the package pins and the CLBs. Figure 1.3 shows SRAM based

FPGA architecture based on regular array.

Figure 1.3: SRAM based FPGA architecture(regular array)

The CLBs are interconnected through a general routing matrix (GRM) that comprises

an array of routing switches located at the intersections of horizontal and vertical

routing channels. The FPGA matrix also has dedicated memory blocks called Block

select RAMs (BRAMs), clock delay locked loops (DLLs) for clock distribution delay

compensation and clock domain control and other components that vary according to

the FPGA family. These devices are quickly programmed by loading a configuration

bitstream (collection of configuration bits) into the device. The device functionality

can be changed at anytime by loading in a new bitstream. The bitstream is divided

into frames and contains all the information to configure the programmable storage

elements in the matrix located in the LUT and flip-flops, configuration cells and

interconnections. The characteristic of the CLB logic and slice may change consistent


with the FPGA family.

Figure 1.4: SEU Sensitive Configuration Bit Storage

There are mainly five area’s of CLB that are affected by SEU as shown in

figure 1.4.

1. Upsets in the logic (LUT)

2. Upsets in the customization routing bits inside the CLB

3. Upsets in the routing connecting CLBs and pins

4. Upsets in the CLB flip-flops (flip-flops)

5. Upsets in BRAM


1.4 Report Organization

The rest of the report is organized as follows.

Chapter 2, SEU Mitigation Techniques, describes the various techniques to mitigate

the effect of SEUs.

Chapter 3, Implementation of Mitigation using TMR & PR, describes the various

types of TMR that can be implement to overcome the effect of single event

upset. It also describes the Methods for reconfiguration of the FPGA devices

using External or Internal interfacing and DPR as a one of the solution to

mitigate the SEU.

Chapter 4, Error Detection and Correction, describes Emulation of SEU means,

Error Detection and Correction of SEU using FRAME ECC and ICAP.

Chapter 5, Implementation of Self Correcting System, shows design and develop-

ment of the SEU monitor system which uses the SEU controller macro and

soft core processor to detect, inject and correct the error. Latter part of chap-

ter shows the design and implementation of self correcting system using SEU

monitor system and the DPR of FPGA.

Chapter 6, Implementation of SEU mitigation Techniques, shows implementation

and results of different types of TMR, Dynamic Partial Reconfiguration and

Finally the SEU monitor system & Self Correcting System to mitigate the SEUs.

Chapter 7, Conclusion and Future Work, Finally this chapter gives concluding

remarks of the implementation of different techniques to mitigation the SEUs

and the future work that can be extended.

Chapter 2

SEU Mitigation Techniques

Radiation hardening against SEUs can be done in many ways. It can either be inher-

ently built into the device during the manufacturing phase or be realized by making

modifications to the design. The most commonly used approach incorporates redun-

dant components that reduce chances of interruptions in the operation due to upsets.

Correction of an upset following detection is usually desirable to prevent accumula-

tion of errors especially in the case of FPGAs. This section discusses different types

of methods that are commonly employed to cope with the SEU problem. One must

be aware nonetheless that an overhead is always involved, either in the

form of cost, power consumption or area.

These thesis is mainly concern about design based mitigation techniques and finally

the system has been design to inject, detect and correct the SEU. These major tech-

niques broadly divided in two categories [8].

1. Technology Based Techniques

2. Design Based Techniques

10

CHAPTER 2. SEU MITIGATION TECHNIQUES 11

2.1 Technology Based Techniques

Silicon on Insulator(SOI)

In this method a different substrate consisting of an insulated SiO2 layer over a silicon

substrate is used in place of the conventional silicon bulk substrate. Bulk structures

used when fabricating CMOS based devices utilize wells to provide isolation between

adjacent configurations. They can also provide some resistance against SEUs by

reducing the charge collection area. Another common type of SOI is called Silicon-

on-Sapphire (SOS). In this case a thin epitaxial Si layer is grown over a sapphire

substrate. Both SOI and SOS improve device resistance against SELs and can also

reduce the number of SEU events, but the fabrication processes are known to be

costly.

Epitaxial CMOS Process

This is a widely used technique, though not as advanced as SOI. An epitaxial layer is

grown over a highly doped Si substrate. The low resistivity resulting from the highly

doped region, limits its charge-collection thereby improving its immunity against

SEUs. This arrangement offers limited SEL protection since the parasitic latch-up

elements are still present unlike in SOIs.

2.2 Design Based Techniques

2.2.1 Spatial Redundancy

A classic example of spatial redundancy is Triple Modular Redundancy (TMR). TMR

is a widespread method and is applied to several domains other than SEU mitigation.

It works on the principle of using three identical design blocks and then voting on

their results to detect failures. An output is obtained as long as at least two of the


blocks are functioning correctly. If a module encounters an error, it is detected by

the voter and can then be corrected by reconfiguring the block. The module can

represent a physical hardware resource such as a processor or a board or can be

redundant logic on the same chip. An apparent weakness of the simple approach

described is that an upset in the voter or in the input cannot be countered as shown

in figure 2.1. This can be resolved by triplicating the voter, and supplying the input

from three identical computations through different paths. Because the scrubbing

does not correct the content of the Configurable Logic Block (CLB) flip-flops, it is

necessary to have a feedback path, to correct the content of the flip-flop at the next

clock cycle. In addition to the TMR scheme for functional blocks inputs and outputs

need to be triplicate. The outputs are tied together externally, using minority voting

to prevent conflicts in the I/O current [14][12].

Figure 2.1: Basics of TMR

2.2.2 Temporal Redundancy

A variant of TMR provides redundancy in the time domain. This technique is gen-

erally used to offer mitigation against error in inputs or in combinatorial logic. It

functions by sampling the signal at different time periods as shown in figure 2.2. A

delay element is added to CLKA and is used to clock elements B and C. Assuming


positive edge-triggered flip-flops, if an error is present in the input during the rising

edge of CLKA, it gets latched in block A. However, the transient dies off before blocks

B and C get affected and hence the error can be voted out. This scheme requires

fewer elements than spatial TMR, but pays the price in terms of a lower speed.

Figure 2.2: Temporal Redundancy

2.2.3 Scrubbing(Reconfiguration)

The use of hardware redundancy by itself is not sufficient to avoid errors in the FPGA;

it is mandatory to reload the bitstream constantly to avoid the accumulation of faults

[4]. This continuous reload of the bitstream is called “scrubbing”. This method uses

inherent functions of these devices like readback / reconfiguration to provide SEU

mitigation. This is done in two phases, detection and correction. There are two

methods to do this.

First involves performing a readback operation that reads back the current data

present in the configuration memory. A bit-by-bit comparison is then carried out

between the readback file and the original (golden) bitstream to determine the number

and position of the upsets. The device is then ‘scrubbed’ to remove the faults by


reconfiguring the device with the golden configuration [8]. Several modifications to

this flow can be made to speed up the process. The easiest way is to skip the detection

phase altogether and scrub the device periodically at a predefined rate called the

‘Scrub Rate’. This has the added advantage of faster upset correction as well as a

reduced overhead, since the detection process requires additional memory to perform

the comparison.

In Second method, the error is detected using ECC facility [32][34] of FPGA and

correction using the partial reconfiguration or reconfiguration. This gives the designer

the option to scrub only affected frames instead of the entire device, greatly lowering

the scrubbing duration. The scrub rate needs to be carefully selected depending on

the estimated SEU rate. Ideally it should be high enough to correct a fault before

the arrival of another upset.

Scrubbing is commonly used in conjunction with the other mitigation techniques

discussed earlier, like TMR, to give a higher degree of immunity against SEUs.

Chapter 3

Implementation of Mitigation

using TMR & PR

3.1 Basics of Triple Modular Redundancy

Several well-known and proven SEU mitigation techniques can be implemented by

means of hardware logic for digital microcircuits as discussed in previous chapter.

All of these techniques are required to work with existing semiconductors. To keep

the trade-offs of cost versus functionality in check, component-level SEU mitigation

techniques prove beneficial to reduce radiation-induced failures. One common re-

quirement to these techniques is the ability to perform a localized reset to the specific

circuitry by an external reset mechanism, typically implemented with a SEU-tolerant

FPGA. Triple Voting is the most common and easy to implement technique. It is

termed as Triple Modular Redundancy (TMR).

TMR uses redundant hardware to mask circuit faults. A circuit protected by TMR in

its most basic form has three redundant copies of the original circuit and a majority

voter. A fault in any one of the three replicates of the original circuit does not produce

an error at the output because the majority voter selects the correct output from the

15

CHAPTER 3. IMPLEMENTATION OF MITIGATION USING TMR & PR 16

other two replicates. Triplicate voters are often used to avoid a single point of failure.

Basic structure of voter circuit and its functional truth table is shown in Figure 3.1.

The basic concept of triple redundancy is that a sensitive circuit can be hardened

to SEUs by implementing three copies of the same circuit and performing a bit-wise

“majority vote” on the output of the triplicate circuit. The function of the majority

voter is to output the logic value (“1” or “0”) that corresponds to at least two of its

inputs. For example, if two or more of the voter’s inputs are a “1”, then the output of

the voter is a “1”. If the inputs of the voter are labeled A, B, and C, and the output

V, respectively, then the boolean equation for the voter is: V= AB + AC + BC [9].

Figure 3.1: Majority Voter Circuit and Truth Table

3.2 Granularity of TMR

There basically three types of TMR as per the granularity of module triplication

implementated into the design.

1. Local TMR

• Triplicate sequential elements only (Flip-Flops, Shift registers, block RAMs

and sequential DSPs.)


• Reduces SEE occurrence of frequency dependant SET capture.

• Clock trees, global routers and I/Os ate still susceptible [15].

2. Distributed TMR

• Triplicate sequential and combinational logic (Global routes and IO not Trip-

licated)

• Reduces SEE occurrence

• Clock trees, global routes and IO still susceptible [15].

3. Global TMR

• Triplicated entire design including global buffers (Sequential elements, Com-

binational logic, Voters, Global buffers).

• Very high level of radiation protection

• Preferred scheme for commercial SRAM based FPGAs [15].

3.3 Various TMR Implementation Methodology

3.3.1 TMR with Single Voter

Here the design is implemented using triplicated logic design as shown in figure 3.2

with single Voter. Here the design module is triplicated and the voter is not tripli-

cated. But in the actual condition all the signals will not be same due to the SEU.

Mainly, when the sequential circuit is used, the output changes irrespective of each

other. This might change the value of all the three inputs to the voter and the con-

dition shown in the figure 3.3 can be observed [9]. Implementation of this method is

shown in section 6.1 figure (6.1).


Figure 3.2: TMR of One-Bit Counter

Figure 3.3: Single Voter TMR Counter with Sequential SEUs

3.3.2 TMR with Triplicated Voter

To avoid the problem discussed in the previous section, Voter is also triplicated in

the designed with the triplicated design module for the sequential circuits as shown

in figure 3.4. The functionality of this design shown in the figure 3.5[9]. section

6.1(figure6.1) shows the implementation of given method for one bit counter.

3.3.3 Temporal Data Sampling

The first key step in new proposed technique is “Temporal Data Sampling”. A sim-

ple embodiment of the Temporal Data Sampling is shown in the Figure 3.6. The


Figure 3.4: Triple Voted TMR of One-Bit Counter

Figure 3.5: Triple Voted TMR Counter with Sequential SEUs

circuit consists of five “edge-sensitive flip-flops”. Each flip-flop operates in “Sam-

pling Mode” when its respective clock signal is in high state and in “Blocking

Mode” when clock signal is low. In “Blocking Mode” flip flop holds the data and

data changes are blocked. In “sampling mode” the flip-flop behaves transparent to

the incoming data [1].

The Temporal Sampling stage helps to store data samples at different time intervals.

These samples are used in voting logic to eliminate single event upsets. Three different

clocks (CLKA, CLKB & CLKC) are used. These three clocks are derivative of the

main clock and have a 90-degree phase shift and 25% duty cycle to cope with the

SEEs as shown in Figure 3.7.


Figure 3.6: Proposed Temporal Data Sampling

If SEU is observed on any one of the clock lines, the phase shift in the remaining

clock signals will help the respective set of flip-flops to store the correct data at

different time intervals, hence voiding the effect of spurious glitch on the clock line

due to radiation. Any transients due to radiation last for small period of time and if

it happens at the negative edge of any clock signal, it will die out before the other

temporal flip-flop start their operation due to phase shift in clock signals. Therefore,

this clocking scheme will help to cope with all the single event transients either in

data line or any one of the clock signals. A conventional (SEU susceptible) sequential

circuit would satisfy timing constraints such that the maximum combinatorial logic

transition time would be less that the period of the master clock minus setup time for

the D Flip-Flop. In this proposed technique, data is released on the edge of CLKC,

and must reach the next sampling stage before the edge of CLKA (minus the setup

time). The insertion of the two extra clock phases (CLKB and CLKC) is required

for the additional temporal sampling. Figure 3.7 illustrates the clocking scheme.

Implementation of this particular method is shown in figure 6.5.

The effective, on-chip computational frequency is exactly one half the frequency of

the master clock. Therefore speed penalty by a factor of two has been incurred to

ensure complete upset immunity. The proposed design technique has two stages,


Figure 3.7: Clocking Scheme for Temporal Sampling

data sampling and data release stage. Flip-flops L1, L3 and L5 constitute the data

sampling stage while L2, L4 and L5 constitute data release stage of the proposed

technique. Flip-flop L5 is common in both stages. The sampling stage FFs captures

data at different time intervals based on their respective clock signals. CLKC serves

as sampling clock as well as sample release clock. For any given data, two samples

of data are stored at different time intervals (CLKA, CLKB). Third data sample is

stored at time t (CLKC) and at the same time previously stored samples are released

to majority voting logic along with this data Sample [1].

3.3.4 Optimal Design of TMR

The majority voters perform a very important task in the TMR approach, because

they are able to block the effects of an upset through the logic at the final output.

In this way, the voters can be placed in the end of the combinational and sequential

logic blocks, creating barriers for the upset effects [3].

If an upset occurs in one of the redundant combination logic parts (LUTs or routing),

its effect will remain until the load of the next bit stream. The constant reconfigu-

ration of the device avoids the accumulation of upsets in the programmable matrix.

This continuous loading of the bit stream is called scrubbing, and it does interrupt


the application. It is important to notice that in throughput logic structures com-

posed by registers, the only way to correct an upset in a register is by loading a new

data in the input of the register, or by implementing this refreshing structure with

voters (as shown in figure 3.8). In the case of the registers, it is not possible to load

the configuration bit stream without interrupting the application because the correct

state of the register cannot be saved and loaded by the bit stream.

Figure 3.8: TMR register with voters and scrubbing

State machine logic is any structure where registered output, at any register stage

within the module, is fed back into any prior stage within the module, forming a

registered logic loop. This structure is used in accumulators, counters, or any custom

state machine or state sequencer where the given state of the internal registers is

dependent on its own previous state. In this case, it is necessary to triplicate the

logic and have majority voters in the outputs. The register cannot be locked in a

wrong value, and for this reason there is a voter for each redundant logic part in the

feedback path, making the system able to recover by itself. The structure presented in

figure 3.8 can be used. One majority voter can be implemented by one LUT. Because

the LUT can be upset (permanent effect), the voters are also triplicate. In this way,

if one voter is upset, there are still two voters working properly. The primary purpose

of using a TMR design methodology is to remove all single points of failure from the

design [3].


Since the full triple module redundancy generates every logic path triplicate, the

TMR output majority voters, inside the output logic block, allows converging the

output again to one signal outside the FPGA.

3.4 Basics of Partial Reconfiguration

Throughout the research and development of a system that is capable of detecting

and gracefully recovering from errors in its circuitry, many new technologies and

concepts have been encountered. partial reconfiguration of the FPGA is one of them.

Partial Reconfiguration is the capability of reprogramming a portion of

an FPGA while the rest of the part does not change . The concept of partial

reconfiguration is a relatively new and uncommon technology used in FPGA design.

In order to create a system that has no down time while repairing modules in error,

partial reconfiguration must be used to reprogram these modules without affecting

the rest of the system. Certain areas of a device can be reconfigured while other

areas remain operational and unaffected by reprogramming. Partial Reconfiguration

is done when the device is active.

Reconfiguration of the FPGA (Partial/Full) is the one of the solution against the

SEUs. For these there may be two possibilities, one is reconfiguring the device at

frequent rate (the frequency of reconfiguration must more than the frequency of oc-

cupance of error). The other solution is to reconfiguration of the device as and when

error has been detected. This chapter describe the development of first method and

chapter 6 shows the implementation and results of the designed system. Chapter 5

gives the details of the second method, which we called as running repair strategy

[10].

Normally, reconfiguring an FPGA requires it to be held in reset while an external

controller reloads a design onto it. Partial reconfiguration allows for critical parts of

the design to continue operating while a controller either on the FPGA or off of it


loads a partial design into a reconfigurable module. Partial reconfiguration also can

be used to save space for multiple designs by only storing the partial designs that

change between designs. Xilinx is the one of a few reconfigurable device producers

in the market. A family of its FPGAs (Virtex Series) provides an important feature

called partial reconfiguration.

Partial Reconfiguration can be divided into two groups:

1. Dynamic (or active) partial reconfiguration, also known as an active partial

reconfiguration, permits to change the part of the device while the rest of an FPGA

is still running. User design is not suspended and no reset and start-up sequence

is necessary.

2. Static (or shutdown) partial reconfiguration, the device is not active during

the reconfiguration process. While the partial data is sent into the FPGA, the

rest of the device is stopped (in the shutdown mode) and brought up after the

configuration is completed. The non-reconfigurable area of the FPGA is held in

reset and the FPGA enters the start-up sequence after partial reconfiguration is

completed

3.5 Methods of Partial Reconfiguration

3.5.1 Module Based Partial Reconfiguration

Module based partial reconfiguration permits to reconfigure distinct modular parts

of the design. To ensure the communication across the reconfigurable module bound-

aries, special bus macros ought to be prepared (which will be explained in setion3.6).

It works as a fixed routing bridge that connects the reconfigurable module with the

rest part of the design. Module-based partial reconfiguration requires performing a

set of specific guidelines during at the stage of design specification. Finally for each


reconfigurable module of the design, separate bit-stream is created. Such a bit-stream

is used to perform the partial reconfiguration of an FPGA. Module based design typi-

cally requires floor planning to specify where all of the reconfigurable modules will be

placed on the physical layout of the FPGA. When the device is to be reprogrammed,

the entire reconfigurable module is overwritten with the new partial bit stream [17].

I. Multi-Column Partial Reconfiguration, Independent Designs : For designs where

the modules are completely independent (no common I/O except for clocks;

no communication between modules). In this case, reconfiguring one module

does not affect the operation of another module. The Module-Based Partial

Reconfiguration flow is used for these designs.

II. Multi-Column Partial Reconfiguration, Communication between Designs : For

modules that communicate with each other, a special bus macro allows signals

to cross over a partial reconfiguration boundary. Without this special consider-

ation, inter-module communication would not be feasible as it is impossible to

guarantee routing between modules. The bus macro provides a fixed “Bus” of

inter-design communication. Each time partial reconfiguration is performed, the

bus macro is used to establish unchanging routing channels between modules,

guaranteeing correct connections. The Module-Based Partial Reconfiguration

flow is used for these designs.

Module Based Design Flow : similar to the standard design flow this flow

comprises the following steps. The detail flow of these approach with required tool

sets is shown in Appendix B.

1. Design Entry and Synthesis both the top-level design and modules are created

using an HDL (Verilog or VHDL) or any other established design entry method.

To synthesize them Xilinx synthesis tool, Xilinx Synthesis Technology (XST), can

be used. This tool produces a netlist in NGC format.


2. Design Implementation This step consists of the three phases:

(1) Initial Budgeting - Creating the constraints and floor plan for the top-

level design.

(2) Active Module Implementation - Implementing the top-level design with

one module expanded at a time.

(3) Final Assembly - Assembling the top-level design and all implemented

modules into a complete design.

Figure 3.9: Module Based Partial Reconfiguration Design Flow

Modular Design allows large designs to be partitioned into self-contained modules

that can be developed in parallel and independently to save time which is based on

the Xilinx Modular Design methodology. Later on, implemented modules are merged

into one complete FPGA design [17]. Figure 3.9 shows the overview of this flow.


3.5.2 Difference Based Partial Reconfiguration

Difference based partial reconfiguration can be used when a small change is made to

the design. It is especially useful in case of changing LUT equations or dedicated

memory blocks content. The partial bit-stream contains only information about dif-

ferences between the current design structure (that resides in the FPGA) and the new

content of an FPGA. There are two ways of difference based reconfiguration known as

a front-end and back-end. The first one is based on the modification of the design in

the hardware description languages (HDLs). It is clear that such a solution requires

full repeating of the synthesis and implementation processes. The back end difference

based partial reconfiguration permits to make changes at the implementation stage

of the prototyping flow. Therefore there is no need for re-synthesis of the design. The

usage of both methods (either front-end or back-end) leads to creation of a partial

bit-stream that can be used for a partial reconfiguration of the FPGA [11].

3.6 Bus Macro Communication

Reconfigurable modules communicate with other modules, both fixed and reconfig-

urable, by using a special Bus Macro. Figure 3.10 shows, how the bus macro is used

between fixed logic and PR logic.

To facilitate communication across reconfigurable module boundaries, yet still con-

form to the Partial Reconfiguration requirement that routing resources across such

boundaries be completely fixed and static, the use of a special bus macro is required.

The Figure 3.11 shows the left half “A” is a module and the right-half “B” is another

module. Either A, B or both could be partially reconfigurable. To support communi-

cation between modules A and B, a special bus macro is used. Partial Reconfiguration

requires the signals used as communication paths between or through reconfigurable

modules must use fixed routing resources. That is, the routing resources used for


Figure 3.10: Bus macro used between PR Logic and Fixed Logic

such intermodule signals must not change when a module is reconfigured. It is a

pre-routed hard macro used to specify the exact routing channels and will not change

from compilation to compilation.

For each of the different design implementations, there is absolutely no variation in

the bus macro routing. Route locking is required because if any of the designs choose

a different routing for the bus macro, it will not align properly with other designs

and the communication between the two halves is effectively broken. The current

implementation of the bus macro uses eight 3-state buffers (TBUFs) hooked up in

an arrangement that allows one bit of information to travel either left-to-right or

right-to-left, using one TBUF long line per bit as shown in the figure 3.11. Each row

of the device can support four bits of a bus macro. The bus macro position exactly

straddles the dividing line between design A and B, using four columns of TBUFs

on the A side, and four columns of TBUFs on the B side. Design A only connects

to the TBUFs in the two or four columns on the Design A side. Likewise, Design

B only connects to the TBUFs in the two or four columns on the Design B side.

The “fixed bridge” that is pre-routed is comprised of the TBUF output long lines

to ensure reliable communication between the two sides. The figure 3.12 shows the


Figure 3.11: Bus Macro between two Region

physical implementation of a bus macro.

Figure 3.12: Physical Implementation of Bus Macro

The bus macro must be physically locked in such a way as to straddle the boundary

line between A and B, and it must be locked in exactly the same position for all

compilations. The bus macro can be wired so that signals can go in either direc-

tion (left-to-right or right-to-left). It is strongly recommended that once direction is

defined, it should not change for that particular FPGA design. Bus macro signals


should neither be bidirectional nor reconfigurable. The number of bus macro commu-

nication channels is limited by the number of horizontal long line routing resources

available in each CLB tile.

3.7 Programming Medium for Configuration

Partial Reconfiguration can be implemented through a JTAG (External) connec-

tion to PC or internally though custom logic or an on board processor (Internal).

Partially reprogramming the FPGA through internal circuitry, referred to as self-

reconfiguration, is a much more useful method of partial reconfiguration since it

eliminates the need for an external PC. The partial bit streams are stored in memory

and are written to the Internal Configuration Access Port (ICAP) of the FPGA in

order to reconfigure the specified region of the board with the new logic [5][18]. For

the Reconfiguration of Device there are two methods as shown in figure 3.13.

Figure 3.13: Methods of FPGA Reconfiguration

One is Self-Reconfiguration and another is External-Reconfiguration. In the first

method we require the use of ICAP or SelectMAP and the controller, while in the


latter case the configuration is carried out by JTAG, UART or other methods of

external reconfiguration of FPGA.

SelectMAP Interface: The SelectMAP interface provides an 8-bit, 16-bit,

or 32-bit bidirectional data bus interface to the FPGA configuration logic. It can

be used either in Master Mode with the CCLK signal considered as an output from

the FPGA, or in Slave mode with the CCLK signal considered as an input. In slave

mode, SelectMAP allows for both configuration and readback, while in master mode

only configuration is possible. The clock is setup to be generated from outside the

FPGA itself while readback may be required for user verification purposes.

Internal Configuration Access Port(ICAP): The ICAP port is the

element provided by Xilinx that allows the Self-Reconfiguration in Xilinx devices.

Which is almost same as SelectMAP as explained in previously. The fixed part of the

FPGA needs a mechanism to reprogram the reconfigurable part. This mechanism is

provided by Xilinx in some of their FPGA families and is called Internal Configura-

tion Access Port(ICAP). The ICAP interface is a subset of the SelectMAP interface,

and it allows the internal logic to access the configuration data of the FPGA. The

ICAP interface is located in the lower right hand corner of the FPGA, so this intro-

duces a restriction to our system. The fixed part responsible for reprogramming the

FPGA must be located on the right hand side. The ICAP can accept data up to 50

MHz without handshaking protocol, but by controlling the CCLK input data can be

downloaded at lower rates. The ICAP port only accepts partial bitstreams because

it cannot stop the FPGA during the reconfiguration process.

In order to manage the ICAP controller from the Softcore Processor(Power PC micro-

processor or Microblaze), a controller is connected to the system bus. This controller

has two registers mapped into the memory of the microprocessor, a data register

where the data to be written/read to the ICAP port is contained, and a control reg-

ister to indicate when the transference has to begin or when it has finished. ICAP

Controller module takes care of that the process is completed or not according to the


configuration command. In this way it will control the configuration of the FPGA.

While OPB [19]/PLB [22] controller modules control the bus interfacing of the ICAP

to the processor IP Core [20] and update the FPGA according to the configuration.

The partial reconfigure bitstreams are stored into the BRAM or SRAM as per ap-

plication requirement. So the reconfiguration time of the FPGA is reduced and the

performance of the Dynamic Partial Reconfiguration is increased.

Bitstream flow using ICAP: The partial bit stream is stored into the ex-

ternal storage memory and it transfered to the FPGA. The figure 3.14 shows, How

the bit stream is transfered into FPGA from external memory into FPGA.

Figure 3.14: Bitstream Flow for the reconfiguration

Embedded Processor:

For the Reconfiguration there are some Soft IP core [18] requires so that it will give the

correct functionality. A self-reconfiguring system on a Xilinx Virtex FPGA is imple-

mented by making use ICAP. A partial bitstream is written to the ICAP, which then

reconfigures the specified portions of the FPGA with the new logic. Communication

with the ICAP can be implemented through either the Softcore processor(IP) or

through a custom VHDL logic design.


Basically two IP core are available.

1. Power PC [28]

2. Micro Blaze [26]

For these project, Microblaze soft core processor have been used to implement the

desisng. The MicroBlaze embedded processor soft core is a reduced instruction set

computer (RISC) optimized for implementation in Xilinx Field Programmable Gate

Arrays (FPGAs) Features. The MicroBlaze soft core processor is highly configurable,

allowing you to select a specific set of features required by your design.

3.8 Development of a Self Reconfigurable System

Figure 3.15: System Architecture


The system has been developed to give the proof of concept of DPR and how it is used

to mitigate the SEUs. Figure 3.15 shows the block diagram of the developed system

to implement the design. In these Designed System, To give the proof of concept and

to explore the efficiency of partial reconfiguration, one reconfiguration region(RR)

and two RM has been developed. Reconfiguration modules are sampler (one is dig-

ital upsampler and another is digital downsampler). They were designed and

configured using dynamic partial reconfiguration. The system accepts data from in-

put serially and gives output serially with different sampling rate depending upon

currently which reconfigurable module is loaded into to FPGA.

Figure 3.16: Up Sampler block diagram

Figure 3.16 show the Block diagram of the up sampler (RM) and the figure 3.17

shows the Block diagram of the down sampler design. The RTL schematic and the

simulation results of up sampler is shown in figure 6.6 and 6.7 respectively. Same for

the down sampler figure 6.8(a) and 6.8(b).

The figure 3.18 shows the interconnecting of various blocks in the design of self re-

configurable system. Right now the control of reconfiguration is with the use of serial

communication on hyper terminal. The algorithm can be developed to automatically

reconfigure the system at desired rate which is the method to mitigate the SEUs as

discussed in the section 2.2


Figure 3.17: Down Sampler block diagram

Figure 3.18: Interconnection of the reconfigurable system

The following terminology is used in PR

(i) Partial Reconfigurable Region (PRR) : area which is defined as a reconfigurable

region from the entire region while doing FloorPlanning.

(ii) Reconfigurable Module (RM) : The different designs or modules which are loaded

into FPGA during PR.

(iii) Static Logic : The fixed part of the design.


(iv) Bus Macro : As defined in section 3.6.

(v) Partial Bitstream : The bit file which is stored into the external compact flash

card.

(vi) Merged Bitstream : The final bit file which contains the Hardware and Software

parts of the design.

The implementation results of the system is shown in chapter 6. For the detail analysis

of DPR for Area and Timing requirement also has been carried out and the results

are also available in Chapter 6.

Chapter 4

Error Detection and Correction

4.1 Configuration Memory

Like any other RAM, the configuration memory of an FPGA is partitioned into words,

also called frames, which represent the smallest addressable unit of the memory for

write and read operations. Values stored in static memory cells control the config-

urable logic elements and interconnect resources. These values load into the memory

cells on power-up, and can reload if necessary to change the function of the device.

The configuration memory cells lie closely to the specific functions they control and

are laid out in a regular pattern. A data-frame is a 1-bit slice of the memory array

along the vertical axis. The configuration data is written to the configuration mem-

ory from configuration registers one data-frame at a time. Therefore, the smallest

portion of configuration data that may be read from, or written to, the configuration

memory is one data-frame. As Shown in figure 4.1, a single data-frame contains por-

tions of configuration data for each and every block that lies in that column. Hence,

multiple data-frames are required to describe the complete width of a column. In

order to read and write individual data-frames, each must be uniquely addressed by

the configuration logic [6].

37

CHAPTER 4. ERROR DETECTION AND CORRECTION 38

Figure 4.1: Configuration Memory Data Frame

Virtex-5 frames consist of 1,312 bits [34][32]. Each frame includes a 12-bit field (bits

640 to 651) consisting of 11 Hamming bits and an overall parity bit for the frame

data to provide the potential for single error correction (SEC) as well as double

error detection (DED) in the frame data and 16 unused bits (Bits 656 to 671). The

parity and Hamming bits are generated external to the FPGA by the configuration

Bitstream generation software and are subsequently downloaded with the application

specific configuration data. However, system memory data subject to change during

the operation of the FPGA, such as the contents of block RAMs and look-up tables

(LUTs) used as distributed RAMs, are not covered by the parity and Hamming bits

[35][21].

4.2 FRAME ECC Primitives

Virtex-5 provide a specialized primitive, called FRAME ECC (Error Correcting Code)

as shown in figure 4.2, for detection and identification of single and double bit errors

in the frame data [1][32]. This primitive is designed to detect single or double bit

errors in configuration frame data. It uses SECDED (Hamming code) parity values


based on the frame data generated by BitGen (configuration Bitstream generation

software).

Figure 4.2: FRAME ECC primitive (Virtex-5)

During read back (each frame read from the configuration memory), the Frame ECC

module calculates the Hamming bits as well as the overall parity for the frame data,

and compares these bits with the Hamming bits and parity for that frame stored

in the configuration memory. Based on this comparison, the FRAME ECC module

produces indications for no error, single bit error, and double bit error conditions in

addition to a syndrome indicating the location of single bit errors.

If the bits have not changed from the original programmed values, then the syndrome

bits are all 0s. If a single bit has changed, including any of the ECC bits, then the

location of the bit is indicated by syndrome bits 10 to 0 and the syndrome bit 11 is

1. If two bits have changed, then syndrome bit 11 is 0 and the remaining bits are non

zero and meaningless. If more than two bits have changed, then the syndrome bits

are indeterminate. The error output of the block is asserted if one or two bits have

changed, indicating that action needs to be taken. The syndrome bits S[10:0] are

derived from the Hamming parity bits, while S[11] is derived from the overall parity

bit. The table 4.1 shows the interpretation of syndrome bit [32].


SYNDROME BITS INTERPRETATION

S[11] = 0 S[10:0] = 0 No errorS[11] = 1 S[10:0] 6= 0 Single bit (SED) error;

S[10:0] denotes location of bit topatch (indirectly)

S[11] = 1 S[10:0] = 0 Single-bit error;overall parity bit p[11] is in error

S[11] = 0 S[10:0] 6= 0 Double-bit error, not correctable

Table 4.1: Interpretation of syndrome bit

System memory contents block RAMs and LUT RAMs, for example are masked from

the internal parity and Hamming calculation by the Frame ECC. Table 4.2 [1][34]

shows the summary of type of error depending upon the hamming bit and parity bit

while the syndrome valid signal is asserted high.

Type of Error Condition(syndrome valid = 1)

No bit error Hamming match, no parity error1-bit correctable error (SEC) Hamming mismatch, parity error2-bit error detection (DED) Hamming mismatch, no parity error

Table 4.2: Frame ECC Error Codes

Repair implementation using FRAME ECC

To use the Frame ECC logic, FRAME ECC VIRTEX5 must be instantiated in the

user’s design, and readback must be performed through SelectMAP, JTAG, or ICAP.

At the end of each frame of readback, the syndrome valid signal is asserted for one

cycle of the readback clock. The number of cycles required to read back a frame varies

with the interface used. The FRAME ECC VIRTEX5 logic does not repair changed


bits; this requires a user design. The design must be able to store at least one frame

of data, or be able to fetch original frames of data for reload.

Following is a repair implementation steps:

(I). A frame is read out through ICAP and stored in block RAM. The frame

address must be generated as each frame is read.

(II). If an error is indicated by the error output of the FRAME ECC block, then

the read back is halted and the syndrome value is saved. If bit 11 is 0, then

the whole frame must be restored. If bit 11 is 1, then bits 10:0 are used to

locate the error bit in the saved frame, and the bit is inverted.

(III). The repaired frame is then written back into the frame address generated in

step (I).

(IV). Readback then begins again with the next frame address.

In case of a single-bit error in the frame data, the syndrome bits S[10:0] points to

the flipped bit in the address space from 704 (location of the first bit in the frame)

to 2047 (last bit in the frame). To convert the syndrome value S [10:0] to the index

of the flipped bit in the range 0 to 1311, subtract 704 decimal (2C0 hexadecimal or

01011000000 binary) if the syndrome is less than 1,024 decimal; otherwise, subtract

736 decimal (2E0 hexadecimal or 01011100000 binary). This is equivalent to sub-

tracting 22 or 23 decimal from S [10:5]. An efficient algorithm for determining the

bit-offset of the error in the range 0-1311 is shown in Equation 4.1 [32], where S[10:0]

are the Frame ECC syndrome outputs [13][34].

Offset ={S [10:5] – 6’d22 – S [10], S [5:0]} (4.1)

If the binary value of syndrome [10:0] is 0 or a power of 2, then the error is located


in one of the parity bits, in which case the location of the bit error is determined as

shown in Table 4.3 [34].

SYNDROME[11:0] Offset SYNDROME[11:0] Offset

100000000001 640 100001000000 646100000000010 641 100001000000 647100000000100 642 100100000000 648100000001000 643 101000000000 649100000010000 644 110000000000 650100000100000 645 100000000000 651

Table 4.3: Parity Bit Error Diagnosis

4.2.1 Readback CRC Algorithm

The read back CRC in the virtex-5 FPGA is performed in this manner

(I.) Continuous Readback of configuration data in the background of a user design.

(II.) Dedicated logic Readback continuously in the background to check the CRC

of the configuration memory content.

(III.) The first round of Readback CRC value is latched as the golden value for later

comparison.

(IV.) The subsequent rounds of Readback CRC value are compared against the

golden value.

(V.) When a CRC mismatch is found, the CRCERROR pin of the FRAME ECC

VIRTEX5 primitive is driven high.


4.3 Internal Configuration Access Port (ICAP)

In addition to this, Virtex-5 contains a 32-bit internal configuration access port

(ICAP) primitives that provides access to the configuration memory from within

the FPGA core. Figure 4.3 shows the ICAP primitive of virtex-5 FPGA. The details

functionality was discussed in chapter 3.

Figure 4.3: ICAP Primitive (Virtex-5)

The next chapter describe’s the design of monitor system which uses the FRAME ECC

and ICAP primitives of virtex5 FPGA.

Chapter 5

Implementation of Self Correcting

System

5.1 SEU Controller macro

Since the FRAME ECC primitive itselt does not provide error correction, circuitry

must be added in the FPGA fabric that uses the ICAP and Frame ECC modules to

cycle through all frames and to detect and correct SEUs in the configuration memory.

The Frame ECC function is also performed each time a frame is read via the ICAP.

So, in this project for detection and correction of bit error, SEU controller macro have

been used, which uses the FRAME ECC and ICAP primitives of Virtex5 as shown

in the figure 5.1.

Basically the SEU controller macro uses the ICAP VIRTEX5 and FRAME ECC

VIRTEX5 primitives to clock and observe the readback CRC circuit, which performs

the SEU detection as described earlier. The macro also includes a controller that

connects to the other ports of these primitives to perform the operations necessary

to locate and correct SEU errors using the built-in ECC facility. For test purposes,

the connection to ICAP is used to facilitate the controlled injection of configuration

44

CHAPTER 5. IMPLEMENTATION OF SELF CORRECTING SYSTEM 45

errors.

Figure 5.1: SEU Controller macro

The functionality of all ports of controller macro is listed be-

low [10]

• mode[2:0]: This 3-bit value specifies the operational mode of the macro as

shown in Table 5.1. The value provided on the mode[2:0] is only read by the

macro on the rising edge of clk when the mode en signal is active high.

• mode en: Apply an active-High signal to this input to instruct the macro to

read the value provided on the mode[2:0] on the next rising edge of CLK.

• rst: Active-High input to instruct the macro to reset on the rising edge of CLK.

The reset clears the operational mode to 000 and initializes the error injection

pointer to zero. (Macro reset is not possible when the BUSY signal is active)

• clk: Input clock used by all elements within the macro and used to clock the

built-in readback CRC logic of the Virtex-5 device.

• seu detect: An active high level on this output signifies that a configuration

error has been detected by the readback CRC circuit. This signal has a direct


connection to the CRCERROR output of the FRAME ECC VIRTEX5 primi-

tive used within the macro.

• detection active: This output signal has a direct connection to the SYN-

DROME VALID output of the FRAME ECC VIRTEX5 primitive used within

the macro and can be used to confirm that readback CRC is actively scanning

the device configuration in order to detect SEUs.

• error inject: This active-High input should only be used in conjunction with

modes 4, 5, 6, and 7 (MODE[2:0] = 100, 101, 110, or 111) to initiate the action

associated with the particular mode. Each cycle of the ERROR INJECT signal

requires a hand-shaking interaction with the BUSY output using the following

sequence:

1. Confirm BUSY is Low or wait for it to become Low.

2. Assert (drive High) ERROR INJECT.

3. Wait for BUSY to become high.

4. Deassert (drive Low) ERROR INJECT.

• busy: Active high output indicating the macro is actively engaged in perform-

ing a task such as error correction or error injection. The main purpose of this

signal is in the hand-shake sequencing of the ERROR INJECT input and to

determine when an operation has been completed so that the macro is ready to

continue with the same or a different task.

• addr[34:0]: This 35-bit input is only used in conjunction with mode 4 (MODE[2:0]

= 100) and defines the location at which an error will be injected.

• error: The macro will drive this output High to indicate when an operation

has been unsuccessful There are two possible reasons why error injection can

fail to happen


1. All frames contain 16 unused bits which cannot be changed.

2. second error injection is performed at the same location (without previ-

ously correcting it using mode 1)

The table 5.1 shows the different operating modes of the controller macro [10].

MODE mode[2:0] Description

0 000 Detection only (default at power-on or following reset)1 001 Detection and automatic correction.2 010 Reserved (Do not use)3 011 Reserved (Do not use4 100 Inject configuration error at location = ADDR5 101 Increment the error injection pointer(ADDR)6 110 Reset the error injection index (ADDR = 0)

Table 5.1: Controller Macro Operational Modes

The operation of the SEU controller macro is as follows:

(1). A 1312-bit frame of configuration memory is read through the ICAP as forty-one

words, each of 32-bits length. The frame data is stored in a block RAM.

(2). If an error is indicated by the outputs of the FRAME ECC primitive, the type

of error is determined as shown in Table 4.1 ( If bit S[11] =0, then the whole

frame must be restored. If bit S[11] =1, then bits S[10:0] are used to locate the

error bit in the saved frame, and the bit is inverted) If the error indicates a

double-bit error, the error output of the SEU controller is latched high and read

back continues with the next frame of configuration memory. If a single-bit error

is indicated, the location of the bit is determined from the syndrome and the

erroneous bit is corrected (i.e. inverted) in the frame data stored in the block

RAM.


(3). The repaired frame is written back into the configuration memory at the same

frame address from which it was read.

(4). Read back resumes with the first frame of configuration memory in the configu-

ration column containing the newly repaired frame.

(5). When a configuration column has been completely read and repaired, the SEU

controller advances to the next configuration column in the array.

5.2 SEU Monitor System

Apart from the SEU controller macro, we require the monitor system to control,

monitor and to provide the user interface to check the functionality of the macro by

injecting, detecting and correcting the error into the configuration memory. In this

method we require the soft processor to monitor and to control the functionality of

SEU controller macro and to provide the user interface. In this project I have used

the MicroBlaze as a soft core processor and UART (RS232) as a standard IO of the

entire system to provide the interface for the user.

Figure 5.2: SEU Monitor System


Figure 5.2 shows the Block Diagram of the created monitor system. The SEU Mon-

itor system can emulate an SEU by deliberately injecting an error into the FPGA

configuration so that its subsequent detection and correction can be confirmed.

Design(SEU monitor system) have been created using Xilinx’s ISE, EDK and SDK

software to handle the functionality of this macro as shown in figure 5.2. Figure 5.3

shows port mapping of the system which has been created using xilinx EDK.

Figure 5.3: Port mapping of System

Figure 5.4: Internal signal Integration


The internal signal connection of the monitor system and the SEU controller macro is

shown in figure 5.4. The interfacing between the system and macro has been created

using the GPIO of the soft processor. The software part of the system has been

written using SDK (shown in figure 5.5).

Figure 5.5: Software for Monitor System using SDK

The software having following facilities to test the system. Bold character value

shows the the particular key required to press from the hyper terminal to initiate the

particular functionality.

(1). S : To Know the Macro Status

(2). C : To Know the CRC Scan Status & FRAME COUNT

(3). R : To Reset the SEU CONTROLLER

(4). M : To Set the Operation Mode

(5). A : To Set the Address(Error Injection)

(6). I : To inject Error at Specified Address


(7). Z : To Make the Address Pointer Zero

The implementation and testing of the system has been carried out on ML505 Board

having Virtex5 (XC5VLX110T-1FF1136) FPGA and the implementation an the test-

ing results are shown in the chapter 6. The same design have been designed and

synthesize for the virtex6 (XC6VLX240T-1FF1156) for the resource utilization com-

parison and the results of that is shown in the table 6.5, in implementation and results

chapter. This system can be integrated in any user design to facilitate the

functionality of SEU detection, injection and correction.

5.3 Self Correcting System

Finally the all the modules have been intergraded into one system called Self cor-

recting System. The SEU controller macro have been integrated in to the previously

created Self reconfigurable system. The Block Diagram of the self correcting system

is shown in figure 5.6.

Figure 5.6: Block Diagram of Self Correcting System


The concept of the final design is like this, the user design is triplicated by means of

any previously discussed TMR method and that would be the Reconfigurable Mod-

ule(RM), then using the SEU Monitor system we can emulate the SEU by injecting

the error, detecting and then correcting the bit error by means of the facility of SEU

monitor System and if this is not possible then the reconfiguration is carried out by

two modes(Manual or Self). The System operates in different three modes.

1. SEU Emulation mode

2. Manual Reconfiguration Mode

3. Automatic Reconfiguration mode

First mode (SEU Emulation mode), is the same as only for the emulating the

SEUs by injecting, detecting and correcting the error into configuration memory,

which is same as the only monitor system created earlier.

The second mode (Manual Reconfiguration Mode) adds the feature of reconfig-

uration of the user logic into first mode. In this reconfiguration is carried out if the

error is detected and the monitor system can not correct it or as and when user is

need to do so. but the reconfiguration is carried out by user interface manually.

Third mode (Automatic Reconfiguration mode) is automatic mode. If the sys-

tem is kept into this mode, the system keep macro in mode 1 which is automatic

detection and correction and if the correction is not possible by macro then it do the

reconfiguration of the user logic.

Implementation and testing of the system has been carried out on ML505 Board hav-

ing Virtex5 (XC5VLX110T-1FF1136) FPGA and the implementation an the testing

results are shown in the figure 6.16 chapter 6.

Chapter 6

Implementation of SEU mitigation

Techniques

6.1 TMR Implementation

This section covers implementation of various methodology of TMR as discussed in

chapter 3.

Figure 6.1: TMR with single voter(Technology Schematic)

53

CHAPTER 6. IMPLEMENTATION OF SEU MITIGATION TECHNIQUES 54

Figure 6.2: Simulation Results

TMR with single voter: Various methodologies of TMR has been implemented

for the one bit counter and the ML505 evaluation board with XC5VLX110T-f1136

FPGA have been used for the design which have been simulated using the Model-

Sim simulator. Figure 6.1 shows the technology schematic of one bit counter where

the logic is triplicated and the voter is not triplicated and the figure 6.2 shows the

simulation results for the same.

(a) Technology Schematic (b) Simulation Results

Figure 6.3: Implementation of TMR with triple voter

TMR with triple voter: Figure 6.3(a) shows the technology schematic of one-bit

counter where the logic is triplicated as well as voter is also triplicated and the result


of its simulation is shown in figure 6.3(b).

Figure 6.4: Technology Schematic of TMR(with Voters and Scrubbing)

TMR registers with Voters and Scrubbing: The Figure 6.4 shows implemen-

tation of TMR registers with Voters and Scrubbing for the one-bit counter. This

method has been discussed in the topic Optimal Design of TMR of section 3.3.

Figure 6.5: Temporal Data Sampling Technology Schematic

TMR with Temporal Data Sampling: Figure 6.5 shows implementation of TMR

with Temporal Data Sampling, the technique which has been discussed in section 3.3,

for the one-bit counter.


Comparison of TMR techniques All the TMR Methodology have been imple-

mented on Xilinx Virtex5 XC5VLX110T-f1136.. Table 6.1 shows the compar-

ison of all the techniques of TMR for simple One bit Counter.

Name ofTechnique

Delay(Min Periodns)/(Max Fre-quencyMHz)

Area(#ofSliceRegis-ters/SliceLUTs)

Pros Cons

Without TMR 1.154/866.551 1/1 SimpleHardware

No SEUDetection

Single voterTMR

1.386/721.501 3/5 Less Hardwarerequired forVoter Circuit

Not efficient forsequentialcircuits

Triple voterTMR

1.386/721.501 3/5 Efficient for se-quential circuitwith SEU

More Hardwarefor Votercircuitcompared toabove method

TMR registerswith VotersandScrubbing

1.061/942.507 3/5 Self Correction More hardwarecomplexity

TMR withTemporal DataSampling

0.812/1231.527 5/2 Also DetectsDouble Error

More no of FFand Clock re-quired and ef-fective clock isHalf of the orig-inal CLK

Table 6.1: Comparison of TMR techniques


6.2 Implementation of Self Reconfigurable System

For the proof of concept of Reconfiguring system, system have been designed as

discussed in the chapter 3 section 3.8, which uses the Microblaze softcore processor,

ICAP provided by xilinx and RS-232 interface. As a reconfiguration module design

of upsampler and downsampler have been used.

The goal of this reconfiguring system is to reprogram a portion of the hardware

without affecting the performance of the remaining static hardware. The partially

reconfigurable sampler accepts data serially and gives output serially with down sam-

pled or up sampled depending upon which module is currently used. A successful

reconfiguring system will allow for the partially reconfigurable module (Upsampler or

Downsampler or Blank) to be reprogrammed by a pressing the appropriate switch on

keyboard through hyper terminal without affecting the static module. Static module

consist of start or stop button.

Figure 6.6 shows the RTL of the upsampler module and the simulation results is

shown in figure 6.7.

Figure 6.6: Up Sampler RTL


Figure 6.7: Up Sampler Simulation Results

RTL Schematic of downsampler module is shown in figure 6.8(a), where as figure

6.8(b) shows the simulation results for the same.

(a) RTL Schematic (b) Simulation Results

Figure 6.8: Implementation of Down Sampler

The figure 6.9 shows the testing of Self Reconfigurable System on hyper terminal

and the figure 6.11 shows Performing Reconfiguration of UP Sampler by appropriate

command through user interface. The design has been extended by adding the timer

in the soft core processor to find out time(no of clock cycles) require to reconfigure

the reconfigurable module dynamically and the testing shown in figure 6.10.


Figure 6.9: Testing of Reconfigurable System

Figure 6.10: Finding no of clock cycles to Reconfigure


Figure 6.11: Dynamic PR(Performing Reconfiguration of UP Sampler)

In the next part the table 6.2 shows the Device Utilization Summary for the combined

Up and Down Sampler. Table 6.3 shows the Area Comparison for combined and

individual module, Up and Down Sampler. Timing Analysis for Reconfiguration is

listed in the table 6.4.

Device Used: Xilinx Virtex5 XC5VLX110T-1ff1136

Logic Utilization Used Available UtilizationNumber of Slice Registers 3255 69120 4.70%Number of Slice LUTs 3245 69120 4.70%Number of fully used Bit Slices 1328 5172 25.67%Number of bonded IOBs 22 640 3.43%Number of Block RAM/FIFO 18 148 12.16%Number of BUFG/BUFGCTRLs 5 32 15.62%Number of DCM ADVs 1 12 8.33%Number of DSP48Es 3 64 4.68%

Table 6.2: Device Utilization Summary (For combined Up and Down Sampler)


Device utiliza-tion summary:

Available Upsam-pleralone

Downsam-pleralone

Combineup anddownsampler

Average% ofSaving

Number ofSlice Registers

69120 3 8 11 45.46 %

Number ofSlice LUTs

69120 4 9 12 50.00 %

Number usedas Logic

69120 4 9 12 50.00 %

Number ofLUT Flip Floppairs used

– 7 17 12 8.34 %

Table 6.3: Area Comparison for Up and Down Sampler

ReconfigureModule

Size of bit file Time Require toReconfigure(Using No of Count)

No of ProcessorClock Cycle

DownSampler

43.379 Kbytes 0.99 µsec (990 nsec) 1 Cycle

Up Sampler 39.993 Kbytes 1.16 µsec (1160 nsec) 1 CycleBlank 39.863 Kbytes 0.97 µsec (970 nsec) 1 Cycle

Table 6.4: Timing Analysis for Reconfiguration


6.3 Implementation of Systems

SEU Monitor system: Figure 6.12 show the RTL of SEU Controller Monitor

system implemented on virtex5 FPGA.

Figure 6.12: RTL of SEU Monitor System

Figure 6.13: Testing of SEU Monitor System


Figure 6.13 show the Testing of SEU Monitor System of SEU Controller Monitor

system through Hyper Terminal. Working of CRC through SEU Monitor System

has been carried out which is shown in figure 6.14. As discussed earlier about the

emulation of SEUs thorough this monitor system performed by injecting the error

into configuration memory and then correcting, whose result is shown in figure 6.15.

Figure 6.14: Checking CRC of SEU Monitor System


Figure 6.15: Error Injection using Monitor System


Device Virtex-5 XC5VLX110T-1ff1136 Virtex-6 XC6VLX240T-1ff1156Resource Available Used Utilization Available Used Utilization

Total Num-ber SliceRegisters

69,120 2,599 3.76% 301440 155 0.05%

Number ofLUTs

69,120 2,220 3.76% 150720 214 0.14%

Numberof bondedIOBs

640 4 0.62% 600 4 0.66%

Numberof BlockRAM/FIFO

148 66 44.59% 416 2 0.05%

Numberof BUFG/BUFGC-TRLs

32 4 12.5% 32 2 6.25%

Number ofICAP VIR-TEX’s

2 1 50% 2 1 50%

Number ofFRAMEECC VIR-TEX

1 1 100% 1 1 100%

Table 6.5: Resource Utilization comparison for Monitor System

Self Correcting System: Finally, the Implementation and testing of the Self

Correcting System is shown in figure 6.16 and the SEU Monitor System is also im-

plemented on virtex6 for area utilization comparison Table 6.5 list this comparison.


Figure 6.16: Testing of Self Correcting System

Chapter 7

Conclusion and Future Work

7.1 Conclusion

The SEU effects can be nullified using the TMR and temporal sampling methods very

effectively. But, these methods increase the area overhead and also the latency of the

systems. The methods described can also be modified to remove the double event

upset effect. Small area overhead can be tolerated for the settlement of the single event

upset effect especially when the FPGAs are used in the space applications. Partial

Reconfiguration method is extremely useful for the space application as the SEU has

to be mitigated. Small FPGA can also be used for larger design by dividing the

design in the modules and we load the module into the FPGA as and when required

by the application in time multiplex way. In Dyanmic Partial Reconfiguration the

total area and time require to reconfigure the design is very less compare to the full

reconfiguration.

Due to the infrequent and unpredictable nature of real SEUs, small scale testing of

their effects and system verification is impractical. The SEU monitor system is suc-

cessfully implemented to solve this problems. The implemented design (which uses

the FRAME ECC and ICAP primitives) is capable of injecting, detecting and correc-

67

CHAPTER 7. CONCLUSION AND FUTURE WORK 68

tion the of single-bit errors for all Virtex-5 FPGAs. The design is easily integrated in

any existing user design with minimal resource overhead for detection and correction

of single bit errors. The Entire Monitor System requires 66 Block RAM and about

2,599 logic slices in case of Virtex-5(XC5VLX110T-1ff1136) and 2 Block RAM and

155 logic slice in case of virtex-6(XC6VLX240T-1ff1156),which is almost only 3.76%

and 0.05% of available resource respectively.

Finally, The Self Correcting System is implemented using the SEU monitor system

and self reconfigurable system, which detects and corrects the error. If the correction

of error is not possible then it reconfigure the user design.

7.2 Future Work

1. The Designed System can be modified, which improves the timing requirement

of reconfiguration by means of some techniques like using DMA for the accessing

the memory.

2. The Designed System can be modified, which would be area efficient and will

work in time multiplex way.

Appendix A

VIRTEX-5 Platform FPGA

Architecture Overview

Figure A.1: Virtex FPGA architecture

Virtex-5 devices are user-programmable gate arrays with various configurable ele-

ments and embedded cores optimized for high-density and high-performance system

designs. [30]

69

APPENDIX A. VIRTEX-5 PLATFORM FPGA 70

Available in five platforms LX, LXT, SXT, TXT, and FXT[29]

(1). Virtex-5 LX: High-performance general logic applications

(2). Virtex-5 LXT: High-performance logic with advanced serial connectivity

(3). Virtex-5 SXT: High-performance signal processing applications with advanced

serial connectivity

(4). Virtex-5 TXT: High-performance systems with double density advanced serial

connectivity

(5). Virtex-5 FXT: High-performance embedded systems with advanced serial con-

nectivity

A.1 Virtex-5 Device Functional Blocks:

1. I/O blocks provide the interface between package pins and the internal con-

figurable logic. Most popular and leading-edge I/O standards are supported by

programmable I/O blocks (IOBs). The IOBs can be connected to very flexible

ChipSync logic for enhanced source-synchronous interfacing. Source-synchronous

optimizations include per-bit deskew (on both input and output signals), data

serializers/deserializers, clock dividers, and dedicated I/O and local clocking re-

sources.

2. Configurable Logic Blocks (CLBs), the basic logic elements for Xilinx FPGAs,

provide combinatorial and synchronous logic as well as distributed memory and

SRL32 shift register capability. Virtex-5 FPGA CLBs are based on real 6-input

look-up table technology.

3. Block RAM modules provide flexible 36 Kbit true Dual Port RAM that are

cascadable to form larger memory blocks. In addition, Virtex-5 FPGA block RAMs

APPENDIX A. VIRTEX-5 PLATFORM FPGA 71

contain optional programmable FIFO logic for increased device utilization. Each

block RAM can also be configured as two independent 18 Kbit true dual-port

RAM blocks.

4. Cascadable embedded DSP48E slices with 25 x 18 two’s complement mul-

tipliers and 48-bit adder/subtracter/accumulator provide massively parallel DSP

algorithm support. In addition, each DSP48E slice can be used to perform bitwise

logical functions.

5. Clock Management Tile (CMT) blocks provide the most flexible, highest-

performance clocking for FPGAs. Each CMT contains two Digital Clock Manager

(DCM) blocks (self-calibrating, fully digital), and one PLL block (self calibrating,

analog) for clock distribution delay compensation, clock multiplication/division.

All programmable elements, including the routing resources, are controlled by val-

ues stored in static storage elements. These values are loaded into the FPGA dur-

ing configuration and can be reloaded to change the functions of the programmable

elements.[30].

Appendix B

Tools & PR Design

Implementation Flow

B.1 Tools and Design Board

Several hardware and software tools are necessary for the completion of this project.

These tools include the physical FPGA and its associated development board that

allowed for continual reprogramming of test systems as well as many features for

data storage and output display. Xilinx has also supplied a suite of tools that are

necessary for this project. These Xilinx software tools are used for developing the

hardware and software aspects of the system. Although many of these tools have

included documentation from Xilinx, their support of partially reconfigurable systems

is currently somewhat lacking.

B.1.1 Xilinx EDK 9.2.02

Xilinx EDK is a software development kit used to utilize the functionality of the

onboard Microblaze cores of the Virtex FPGA chip. The EDK provides us with IP

72

APPENDIX B. TOOLS & PR DESIGN IMPLEMENTATION FLOW 73

Cores to bridge functionality of hardware designs on the FPGA with software designs

on the Microblaze. This is an attractive feature as this project will implement VHDL

designs on the FPGA while being able to partially reprogram those designs using

the Microblaze. Most of the software developed for this project will be created using

the Xilinx EDK interface. After successful software build, the EDK can synthesize

the correct VHDL wrappers to instantiate the Microblaze core and can inject the

Microblaze instructions into a blockRAM on the FPGA for the Microblaze to read

and process [25][23][24].

B.1.2 Xilinx ISE 9.2.04i with Partial-Reconfiguration Patch

Xilinx ISE is a popular FPGA development tool used widely in industry and in

educational institutions. This tool allows for complete FPGA development. ISE can

read in VHDL/Verilog/Schematic(..And many more formats) modules created for a

project design and synthesize them into logic elements to be placed on an FPGA. ISE

can automatically interpret your HDL syntax, synthesize your description, place and

route the logic elements and then provide a software BIT file description to connect

these logic elements together to create the circuit described in the HDL. All these tool

flow steps require their own respective application program to perform the function.

ISE calls these programs automatically in a pleasant GUI so that the user can create

FPGA designs in seconds.

B.1.3 Xilinx PlanAhead 10.1

Xilinx PlanAhead is a FloorPlanning tool provided by Xilinx to allow developers

flexibility on how there synthesis designs should be placed on the FPGA floorplan.

This tool is useful in the designs where locations of logic elements are an important

factor to the performance of the application. For the scope of this project, PlanA-

head has partial reconfiguration options which make it a required tool in the partial


reconfiguration tool flow. The Figure B.1 shows Summarized PlanAhead PR Design

Flow.

Figure B.1: PlanAhead PR Design Flow

B.1.4 Virtex-V Evaluation Board

These boards provide a variety of features in order to implement our entire design

with partial reconfiguration, SEU Detection and correction, and finally the integration

of entire design. This board contains a Virtex FPGA chip from Xilinx. Out of the

many features which this board provides, this project just utilizes the FPGA, onboard

memory storage, Compact Flash memory card and the inputs/outputs provided by

RS-232, LEDs, and Push-Button.[27]. In this Project work I have used Virtex-5

evaluation board with XC5VLX110T-f1136 FPGA.


B.2 PR Design Implementation Flow

When creating a self-reconfigurable system there are many specific considerations to

take. This production flow will cover all specific changes which must occur in all tool

flows in order to integrate the Microblaze wrappers with any custom partial or static

VHDL modules. Following these steps it should be possible to recreate the entire

design. The following tool flow is best used with the exact versions of Xilinx tools

described in Section Tools. This tool flow is not guaranteed with any older or newer

version of these Xilinx tools [31].

1. Open EDK and create a new project with the project wizard as shown in fig B.2.

Specify your specific board settings. Add desirable peripherals. Set boot memory

from the ilmb cntlr and unselect the option of memory test and Peripheral selftest.

Figure B.2: Create XPS Project

2. Once the project has been created, right click on clock generator 0 instance and

select delete instance. A selection box will open up. Select Delete instance but

keep its ports and click OK.


3. Using Project tab, double-click on system.mhs entry to open the file in an editor

window.

4. Replace dcm clk s with sys clk s.

5. Add the following line after sys rst pin to bring the dcm lock signal from the top-

level PORT DCM all locked pin = DCM all locked, DIR = I. Now Save the file

and close it.

6. Using IP Catalog tab, add three instances of XPS gpio and name them as icap go

out, icap done in, and icap bitstreamlength for User ICAP interface processor or

Add one XPS gpio for HWICAP processor.

7. Configure icap go out to have 1 bit databus (to provide start configuring signal),

unidirectional, output only.

8. Name GPIO d out port connection to icap go.

9. Similarly, configure icap done in to have 1 bit databus (to provide start configuring

signal), unidirectional, input only.

10. Name GPIO in port connection to reconfig done.

11. Configure icap bitstreamlength for 32-bit, output only.

12. Name GPIO d out as bitstreamlength as this port will provide the bitstream length

to the icap processor.

13. From IP Catalog tab, add an instance of icap processor and ICAP INTERFACE

from Project Local pcores folder.

14. Expand three xps gpio and icap processor instances in System Assembly View

window with Bus Interface filter in effect, and connect various busses.

15. Select Addresses filter and set 64K bytes for icap interface 0, icap bitstreamlength,

icap done in and icap go out each and then click Generate Addresses.


16. Select Ports filter and connect each port to the appropriate signals in the system

and make BM enable an external port.

17. Select Software Platform Settings, and click on xilfatfs check box to select the

fatfs file system support.

18. In Application tab of XPS, double-click on Add Software Application Project as

shown in fig B.3, type TestApp (Any name for the Software Project) in the Project

Name field, and click OK.

Figure B.3: Add Software Application

19. Right-click on Sources and select Add Existing Files.(If already available other

wise create it using SDK)

20. Browse to directory where your TestApp is stored and add one (main.c) source file

and the Right-click on Headers and select Add existing Files (Browse to TestApp

src directory and add one (icap interface.h) source file).

21. In Applications tab, deselect Mark to Initialize BRAMs for Default: microb-

laze 0 bootloop application and select Mark to Initialize BRAMs for TestApp project.

22. Select Software Build All Users Applications to run LibGen to generate library

files and compiler to compile the application.


23. Open ISE and create a new project with your board settings. Create a top level

VHDL module. This module will include port mapping references to any static or

partial designs.

24. In your top-level ISE project add an existing source (not a copy). Import the

*.XMP file created in your EDK project into ISE.

25. Select top.vhd in Sources window and double-click Synthesize to synthesize the

design.

26. Open PlanAhead and create a new project. Import the top-level NGC file created

during synthesis. Point reference to any folders which contain NGC files for child

modules described within the top-level NGC. Select the correct part number for

your board; import the UCF file created in ISE and click finish.

27. For every static module, right click on the static module and click create new

pblock.

28. click on site constrain and put all Bus Macro within the pblock as shown in fig

B.4.

Figure B.4: Draw Partial Module

29. For every partial module on the floorplan, add the MODE attribute to it and set

it to Reconfigurable.


30. Next perform Tools=>RunDRC (shown in fig B.5, in the window click on generate

script files only and click on finish.

Figure B.5: DRC Run

31. At this stage, the ExploreAhead Runs tab should show static and two RM modules

entry(as we have two RM in our Design).

32. Before we run the PR Implementation flow, we need to set path to system sys-

tem stub.bmm file in order to generate system stub bd.bmm file after the implemen-

tation. The system stub.bmm file describes the BRAM composition (logical) where

program will be stored when the FPGA is configured. The system stub bd.bmm

file describes actual BRAMs used in the implementation.

33. Select static in ExploreAhead Runs tab, and select options tab in Run Properties

view. Select -bm option under ngdbuild and click on down arrow.

34. Browse to icap ../edk/implementation, select system stub.bmmfile and click Open

and then Click Apply to have the option in effect.

35. Select static in ExploreAhead Runs tab, right-click, and select Launch Run.

36. simultaneously Select both the RM in ExploreAhead Runs tab, right-click, and

select Launch Run as shown in fig B.6.


Figure B.6: Luanch Run of static RM

37. The last step in the PR Implementation flow is to run PR Assemble and PR Verify

design steps, which can be launched simultaneously by right-clicking one of the RM

modules and selecting Run PR Assemble as shown in figure B.7.

Figure B.7: PR assemble of Design

38. Browse to ../project 1/project 1.runs/floorplan 1/merge directory and copy pblock

reconfig sampler blank.bit, up partial.bit and down partial.bit and place

them asblank.bit, up.bit and down.bit respectively in ../image folder.

39. Browse to ../project 1/project 1.runs/floorplan 1/merge directory and copy static full.bit

and place it in ../edk/implementation folder.

40. Open EDK shell. In EDK shell, change directory to ../edk and then execute the


following command to generate download.bit file (having software component in-

cluded) from static full.bit (having just hardware component) [33].

data2mem -bm implementation/system stub bd -bt implementation

/static full.bit -bd TestApp/executable.elf tag microblaze 0 o b imple-

mentation/download.bit

This will generate download.bit in edk/implementation directory.

41. In EDK shell, execute following command to generate system.ace file using follow-

ing command.

xmd -tcl genace.tcl -jprog -target mdm -hw implementation /down-

load.bit -elf TestApp/executable.elf -board ml505 -ace system.ace

This will generate system.ace file in edk directory.

42. Place Compact Flash memory card in a CF writer.

43. Using Windows Explorer copy all files (3 bit and 1 ace) from ..lab/image folder

into CF card (Make sure that there are no files in CF card before copying).

44. Place the CF card into ML505 board, start HyperTerminal window with 115200

baud, and power ON the ML505 board.

45. Test the design.

Glossary

Glossary of significant acronyms used in this thesis

FPGA Field Programmable Gate ArrayASIC Application Specific Integrated CircuitCMOS Complementary Metal Oxide SemiconductorCLB Configurable Logic BlockLUT Look Up TableNASA National Aeronautics and Space AdministrationSEU Single Event UpsetSEE Single Event EffectSET Single Event TransientSHE Single Hard ErrorSEL Single Event LatchupSOI Silicon On InsulatorSOS Silicon On SapphireTMR Triple Modular RedundancyJTAG Joint Test Action GroupICAP Internal Configuration Access PortPR Partial ReconfigurationDPR Dynamic Partial ReconfigurationPRR Partial Reconfigurable RegionRM Reconfigurable ModuleECC Error Correcting CodingCRC Cyclic Redundancy CheckSECDED Single Error Correction Double Error DetectionDED Double Error DetectionMDD Microprocessor Driver DescriptionMHS Microprocessor Hardware SpecificationMPD Microprocessor Peripheral DescriptionMSS Microprocessor Software SpecificationPLB Processor Local BusBMM Block RAM Memory MapBDD Black Box DefinitionDCM Digital Clock ManagerUCF User Constrain FileEDK Embedded Development KitSDK Software Development KitDMA direct Memory Access

References

[1] A Stoica , T Arslan and S Baloch. “Design of a Single Event Upset(SEU) Mit-igation Technique for Programmable Devices”. In 7th International Symposiumon Quality Electronic Design(ISQED), 2006.

[2] Carl Carmichael , Brendan Bridgford and Chen Wei Tseng. “Single Event UpsetMitigation Selection Guide”, XAPP987(v1.0). Xilinx Inc., March 2008.

[3] L Sterpone , F Lima Kastensmidt and M Sonza Reorda. “On the OptimalDesign of Triple Modular Redundancy Logic for SRAM based FPGAs”. In Design,Automation and Test in Europe Conference and Exhibition(DATE), 2005.

[4] Michael Caffrey and Anthony Salazar. “Correcting Single-Event Upsets ThroughVirtex Partial Configuration”, XAPP216(v1.0). Los Alamos National Laborato-ries, June 2000.

[5] Ming Liu , Zhonghai Lu and Wolfgang Kuehn. “Run-time partial reconfigurationspeed investigation and architectural design space exploration”. In Internationalconference on Field programmable Logic and it’s Appplication, FPL09, September2009.

[6] Phil Blain , Carl Carmichael and Michael Caffrey. “SEU Mitigation Techniquesfor Virtex FPGAs in Space Applications”. In Los Alamos National laboratory,Carmichael.

[7] Philippe Adell and Greg Allen. “Assessing and Mitigating Radiation Effects inXilinx FPGAs”. Technical report, Jet Propulsion Laboratory, California Instituteof Technology Pasadena, 2008.

[8] Shadab Gopinath Ambat. “Single Event Upset Detection in Field ProgrammableGate Arrays”. Technical report, University of Kentucky, February 2008.

[9] Carl Carmichael. “Triple Module Redundancy Design Techniques for Virtex FP-GAs”, XAPP197(v1.0.1). Xilinx Inc., July 2006.

[10] Ken Chapman. “SEU Strategies for Virtex-5 Devices”, XAPP864(v2.0). XilinxInc., April, 2010.

83

REFERENCES 84

[11] Emi Eto. “Difference Based Partial Reconfiguration”, XAPP290(v2.0). XilinxInc., December 2007.

[12] Sandi Habinc. “Functional Triple Modular Redundancy(FTMR)”. Technicalreport, European Space Agency Contract Report, December 2002.

[13] L Jones. Single Event Upset (SEU) Detection and Correction Using Virtex-4Devices, XAPP714(v 1.5). Xilinx Inc., June 2007.

[14] Anthony Lai. “Mitigation techniques for electronics in Single Event Upset envi-ronments”. Technical report, Military Embedded Systems, January 2006.

[15] Oliver Neumann. “Radiation Effects Cookbook”. Technical report, MentorGraphics, 2009.

[16] JEDEC Standard. “JEDEC Dictionary of Terms for Solid State Technology”.JESD88B, 3, May 2006.

[17] Xilinx Inc. “Two flows for partial reconfiguration: Module based or small bitmanipulations”, xapp290(v1.0), May 2002.

[18] Xilinx Inc. “In-Circuit Partial Reconfiguration of RocketIO Attributes”,XAPP662(v2.4), May 2004.

[19] Xilinx Inc. “OPB HWICAP v1.3”, DS280, March 2004.

[20] Xilinx Inc. “Processor IP Reference guide”, v1.9, January 2004.

[21] Xilinx Inc. “Virtex Series Configuration Architecture User Guide”,XAPP151(v1.7), October 2004.

[22] Xilinx Inc. “PLB IPIF v2.02a”, DS448, April 2005.

[23] Xilinx Inc. “EDK 9.2 MicroBlaze Tutorial in Virtex”, WT001(v4.0), October2007.

[24] Xilinx Inc. “EDK Concepts, Tools, and Technique-A Hands On Guide to Effec-tive Embedded System Design”, May 2007.

[25] Xilinx Inc. “Embedded System Tools Reference Manual”, EDK 9.2i, September2007.

[26] Xilinx Inc. “MicroBlaze Processor Reference Guide Embedded Development KitEDK 9.2i”, UG081(v9.0), January 2008.

[27] Xilinx Inc. “ML505/ML506/ML507 Evaluation Platform User Guide”,UG347(v3.1.1), October 2009.

REFERENCES 85

[28] Xilinx Inc. “Virtex-4 FPGA Embedded Processor Block with PowerPC 405 Pro-cessor(v2.01b)”, DS306, April 2009.

[29] Xilinx Inc. “Virtex-5 Family Overview”, DS100(v5.0), February 2009.

[30] Xilinx Inc. “Virtex 5 FPGA user guide”, UG190(v5.3), November 2009.

[31] Xilinx Inc. “Early access partial reconfiguration user guide for ISE 9.2.04i”,UG208(v1.2), August 2010.

[32] Xilinx Inc. “Virtex5 configuration guide”, UG191(v3.9.1), August 2010.

[33] Xilinx Inc. “Data2MEM User Guide”, UG658(v 11.2), June, 2009.

[34] Xilinx Inc. “Virtex-4 FPGA Configuration User Guide”, UG071 (v1.11), June,2009.

[35] Xilinx Inc. “Correcting Single-Event Upsets in Virtex-4 Platform FPGA Con-figuration Memory”, XAPP988 (v1.0), March, 2008.

List of Publication

1. Vijay Savani, Akash Mecwan, N. P. Gajjar. “Single Event Upset MitigationTechniques Implementation Based on TMR”, In Nirma University Inter-national conference on Current Trends in Technology. Institute of Technology,Nirma University, December 2010.

2. Vijay Savani, Akash Mecwan, N. P. Gajjar. “Dynamic Partial Reconfig-uration of FPGA for SEU Mitigation and Area Efficiency”, In In-ternational Journal for Engineering and Technology, volume 2, No.2, pages 17.,http://www. ijict.org , December 2011.

3. Vijay Savani, Akash Mecwan, N. P. Gajjar. “Design and Implementation ofSEU Monitor System for SEU Detection and Correction in virtex-5FPGA”, In Integration the VLSI journal(ELSEVIER), Comunicated(In review).

09mec017

Documents

exploreahead

runsoorplan

cclk signal

ip catalog

advanced serial

triple modular

select delete

soft core