FAST A/D SAMPLER

FINAL PRESENTATIONPresented By: Tal Goihman, Irit KaufmanInstructor: Mony Orbach

Winter 2012

Goals Project Goal:

Design and implement an A/D system using Xilinx Virtex6 development board for sampling at highest possible rate.

Sample to virtex6 development board DDR memory.

Transfer the sampled data to PC memory through PCIe and save the data to disk.

H/W Block Diagram

PCI-E Connector

FMC Conn

Virtex6 FPGA

ML605 development board

DDR3

FMC125 Fast A/D

A/D IC

PC

ML605 development board

FMC125 Fast A/D

A/D SamplerThe FMC125 is a Quad-Channel ADC that provides four 8-bit ADC channels enabling simultaneous sampling of 1, 2, or 4 channels @ 5 , 2.5 , 1.25Gsps respectively. Problems:

4DSP provides free of charge a reference design only for 4ch @ 1.25Gsps. A reference design for 1ch @ 5Gsps priced at 2300 EU.

4DSP reference design

Design Block Diagram

“wormhole”AXI4-liteAXI4

Implemented

In XPSMicroBlaze

DDR3Memory

Controller (MIG)

AXI Master

UART

ADC Control

FMC125 Aggregator

CDMAPCI

Express

AXI Slave

AXI MasterMailbox

AXI Slave

Timer

Implemented

In ISE project Navigator

Block Diagram: Main Data Channel

FMC125 delivers data using 4 128-bit lanes of a proprietary bus running at 125Mhz into the aggregator.

Aggregator unites and synchronizes the different channels into a 512-bit bus.

AXI Master sends the data over a 256-bit wide AXI bus running at 200Mhz to the AXI Slave interface of the Memory Controller.

Memory controller handles Writes to DDR3.

FIFO’s and H/W Flow control in every component’s input (and some components output) throughout the channel to achieve highest possible bandwidth.

Throughput is measured using a timer from the start of the write operation until assertion of a write done signal from the AXI master

DDR3Memory

Controller (MIG)

AXI MasterFMC125 Aggregator AXI Slave

AXI Bus background Xilinx has adopted AXI bus, which is a

standard bus protocol from ARM used in modern ARM SoC.

CharacteristicsMemory mapped, 32-bit addressesWrite Address, write data, write response,

read address, read dataVariable width, clock & burst length over a

single bus.

AXI write example

Memory & Memory controller 512MB DDR3 64-bit @ 400Mhz (800MT/s)

Theoretical bandwidth of 800MT/s * 64bit / 8 = 6.25GB/s

Main channel matched to this theoretical bandwidth (200Mhz * 256bit /8 = 6.25GB/s)

Memory & controller isn’t perfect, has a utilization factor.

We achieved 5.13GB/s Throughput (82% utilization!)

Design Block Diagram

“wormhole”AXI4-liteAXI4

Implemented

In XPSMicroBlaze

DDR3Memory

Controller (MIG)

AXI Master

UART

ADC Control

FMC125 Aggregator

CDMAPCI

Express

AXI Slave

AXI MasterMailbox

AXI Slave

Timer

Implemented

In ISE project Navigator

Block Diagram: memory to PCIe After the data is sampled to memory the system

transfers the first chunk to the Host PC’s DMA buffer.

The subsequent chunks are transferred upon receiving a command from the Host PC.

Data is read from memory and transferred to PCIe by the DMA engine Throughput was matched to PCIe Transferred in 16MB chunks (the chosen DMA buffer size) Address translation occurs in the PCIe core

Block Diagram: PCIe Xilinx Available solutions include:

A PCIe integrated block with support for up to x8 gen2. Hard to work with:○ Need to know inner workings of PCIe○ Need to implement several proprietary interfaces with many rules and

signals

A wrapper for memory mapped AXI with support for up to x4 gen1 / x2 gen 2○ Connects to a standard AXI4 bus○ Chosen in our design due to high ROI

We chose to use the wrapper due to high ROI x4 gen1 configuration, up to 1GB/s One 64bit AXI BAR for sample data transfer with Configurable address

translation, one PCIe BAR for mailbox communication.

Block Diagram: MicroBlaze MicroBlaze is a soft-core processor by Xilinx.

Runs firmware written in C from a dedicated 64KB BRAM.

Firmware communicates with Host PC’s software through a PCIe mailbox Supports a predefined set of commands to accomplish the

functional use case and provide debug capabilities Initializes and configures the entire system per to the Host

PC’s instructions. Provide visual status and information to the user through

UART massages.

Block Diagram: ADC ctrl ADC ctrl is a custom core deigned to enable

MicroBlaze to communicate with other custom cores

Developed using the Create peripharel wizrd in XPS and implements mamory mapped register access through AXI4-lite

Provides the following capabilities: Translate the wormhole transactions to / from the FMC125

core. Provide AXI Master with burst count, write start, test signals Provides the aggregator with enabled ADC channels

Software Written in C# .NET 4 in VS2012, GUI in WPF

Enables simple interaction with the system

User adjustable burst size, burst count, active channels,output file location andexternal trigger mode

compatible with Windows7/8 x64.

Software: Jungo WinDriver Easy driver creation

Simple interface for DMA buffer allocation, read and write operations

Slightly problematic from a .NET environment since it lacks proper documentation, and the wizard generates a non working solution

Reference for .NET and DMA was found in a bundled example for PLX chips

Test Environment Verification of sampled data on DDR3 is accomplished by

firmware on MicroBlaze comparing read data with predetermined written pattern

FMC125 incoming data is observable through ChipScope.

Bandwidth was calculated by measuring the time to write a chunk of data using a Timer.

Data transfer from memory to PCIe is validated through observing the memory on system and comparing with the data received from PCIe manually.

QUESTIONS?THANKS

Backup Documantaion