characterization presentation neural network implementation on fpga supervisor: chen koren maria...
Post on 19-Dec-2015
221 views
TRANSCRIPT
Characterization Presentation Characterization Presentation
Neural Network Neural Network Implementation On Implementation On
FPGAFPGASupervisor: Chen KorenSupervisor: Chen Koren
Maria Nemets 309326767Maria Nemets 309326767Maxim Zavodchik 310623772Maxim Zavodchik 310623772
IntroductionIntroduction
Neural Network is a sort of artificial Neural Network is a sort of artificial intelligenceintelligence
NN is an imitation of a human body NN is an imitation of a human body A NN acquires knowledge through learning. A NN acquires knowledge through learning. NN can learn in real timeNN can learn in real time Applications: Applications:
Optical Characters RecognitionOptical Characters Recognition Voice RecognitionVoice Recognition Financial ForecastingFinancial Forecasting Medical DiagnosisMedical Diagnosis
Theoretical BackgroundTheoretical Background
Neuron Model (Perceptron):Neuron Model (Perceptron): The single neuron is a device that implements
the function Activate function – some bounded
continuously rising function (for example, sigmoid).
X is an input vector W is a weight vector
Knowledge is stored within inter-neuron connection strengths
01
_ ( )m
j jj
Activate func w x w
Learning of the neural Learning of the neural networknetwork
Popular implementation is a multilayer feed forward network
Neural network learns from examplesNeural network learns from examples Learning is changing weights to implement the Learning is changing weights to implement the
desired classification functiondesired classification function There are algorithms which change the weights There are algorithms which change the weights
in the networkin the network We will use Back Propagation learning algorithmWe will use Back Propagation learning algorithm
Minimizes the difference between output and training Minimizes the difference between output and training example.example.
Weight changes depend on the units in the layer below.Weight changes depend on the units in the layer below.
Project ObjectivesProject Objectives
Implementing Neural Network on FPGAImplementing Neural Network on FPGA Creating modular designCreating modular design Implementing in software (Matlab)Implementing in software (Matlab) Creating PC InterfaceCreating PC Interface Performance Analyze:Performance Analyze:
Area on chipArea on chip InterconnectionsInterconnections Speed vs. software implementationSpeed vs. software implementation FrequencyFrequency CostCost
Development StagesDevelopment Stages Choosing application: 0-9 Digits RecognitionChoosing application: 0-9 Digits Recognition Implementing a single neuron in VHDLImplementing a single neuron in VHDL Determining network structure Determining network structure Building NN in Matlab, training and testing itBuilding NN in Matlab, training and testing it Determining weights length and resolutionDetermining weights length and resolution Choosing the proper hardware: FPGA and Choosing the proper hardware: FPGA and
MemoryMemory Planning FSM controllerPlanning FSM controller Implementing the NN in VHDL Implementing the NN in VHDL Simulation and SynthesisSimulation and Synthesis Performance analysis: hardware vs. software Performance analysis: hardware vs. software
System InterfaceSystem Interface
InputsInputs Binary image ( 8x8 pixels )Binary image ( 8x8 pixels ) Weights – The size will be determined Weights – The size will be determined Getting inputs by blocksGetting inputs by blocks
OutputsOutputs Vector size of 4 bits coding the digit or Vector size of 4 bits coding the digit or
indicating a failureindicating a failure
Neuron ArchitectureNeuron Architecture
Multipliers for calculatingMultipliers for calculating Adder for calculating theAdder for calculating the ROM implementing Activation functionROM implementing Activation function SRAM saving the weightsSRAM saving the weights SRAM saving the input imageSRAM saving the input image FSM ControllerFSM Controller
j jw x
01
m
j jj
w x w
Neuron ArchitectureNeuron Architecture
Hardware RequirementsHardware Requirements
Memec Development BoardMemec Development Board Virtex II Pro FPGAVirtex II Pro FPGA
44 18x18-bit Multipliers44 18x18-bit Multipliers 88 KB of on-chip memory88 KB of on-chip memory Power PC 4.05 CPU corePower PC 4.05 CPU core
32 MB of SDRAM32 MB of SDRAM 100MHz & 125 MHz clocks100MHz & 125 MHz clocks 2x16 LCD panel2x16 LCD panel Serial port interfaceSerial port interface
Hardware RequirementsHardware Requirements
Load Weight
s
To SDRA
M
Load Image From User
Load Weight
s
To BRAM
Calculate Intermedia
te Sum
Calculate
Output
Return Result
To User
Special ProblemsSpecial Problems Problem:Problem: NN interconnection complexity NN interconnection complexitySolution:Solution: Designing sub-networks for each digit Designing sub-networks for each digit
Problem:Problem: Numbers representation Numbers representationSolution:Solution: Fixed-Point representation avoiding Floating Fixed-Point representation avoiding Floating Point operations Point operations
Problem:Problem: Determining accuracy Determining accuracy Solution:Solution: Choosing proper resolution from Matlab Choosing proper resolution from Matlab simulationssimulations
Problem:Problem: Trade-off between Area and Concurrency Trade-off between Area and Concurrency Solution:Solution: Searching for big FPGA which enables Searching for big FPGA which enables maximal parallelizing maximal parallelizing
Special ProblemsSpecial Problems Problem:Problem: Modulation vs. Speed Modulation vs. Speed
Solution:Solution: Simulating the hardware NN and finding the Simulating the hardware NN and finding the trade-offtrade-off
Problem:Problem: Large Memory size Large Memory size
Solution:Solution: Finding FPGA with big memory or using outside Finding FPGA with big memory or using outside
memory coresmemory cores
Problem:Problem: Sigmoid function is continuous and infinite Sigmoid function is continuous and infinite
Solution:Solution: - Function quantization according - Function quantization according
to our resolution to our resolution
- Saving values for arguments in limited range - Saving values for arguments in limited range
ScheduleSchedule
Date Assignment31.10-
5.11Learning the theoretical backgroundTraining and testing an example NN in
Matlab
7.11-12.11
Implementing a single Neuron in VHDL, synthesizing and checking its area
14.11-19.11
Finding suitable Hardware solution (Area, Finding suitable Hardware solution (Area, Memory)Memory)
17.11 Characterization Presentation
21.11-3.12
Building training setBuilding our NN in Matlab, training and
testingDetermining weight length and needed
resolution
5.12-10.12
Determining NN structureDetermining NN structure
Proposing final architecture Proposing final architecture
ScheduleSchedule
Date: Assignment12.12-17.12
Implementing Sigmoid functionImplementing Sigmoid functionPlanning the FSM ControllerPlanning the FSM Controller
19.12-7.01
Implementing the NN in VHDL
20.1220.12 Midterm presentation
9.01-21.01
Simulating the NN Synthesis
23.01-28.01
Analyzing performanceWriting project book
21.0221.02 Final presentation