armarm--based based socsoc design design –– cmos image sensor · 2016-10-31 · –– cmos...

2010-11-29

1

ARMARM--Based Based SoCSoC Design Design –– CMOS Image SensorCMOS Image Sensor

2

AgendaAgenda

CMOS Image Sensor

System Structure

Project Overview Compile the Hardware

Build the Software

Simulate the Reference Design

Click t

o buy NOW!

PDF-XChange

www.docu-track.com Clic

k to buy N

OW!PDF-XChange

www.docu-track.com

http://www.docu-track.com/buy/


2010-11-29

2

CMOS Image SensorCMOS Image Sensor

4

CISCIS

CMOS Image Sensor

Low Power (CCD 1/10 ~1/100)

(18 /cycle, CCD : 8 /cycle)

, , , CCTV

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

3

5

CCDCCD

Charged Coupled Device

: Pixel ( , )

CCD

:

ADC, DSP

CMOS

: DSLR Camera

6

CIS FeatureCIS Feature

CMOS Image Sensor

VGA(640 x 480), CIF(352 x 288) Resolution Support

I2C Compatible Serial Bus Interface

R,G,B to YCbCr Color Space Conversion

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

4

7


Key Specification

8


Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

5

9


Block Diagram

10

Pin DescriptionPin Description

Pin Description

NAME TYPE DESCRIPTION

AVDD AP Analog Power

AVSS AG Analog Ground

DVDD P Digital Power

DVSS G Digital Ground

LVALID O Data Valid indicator signal

FSYNC O Frame Synchronous Signal

MCLK I System Clock input

RSTB I, PU System Power on Reset, Active Low

VCLK O Output Pixel Data Synchronous Clock

SDA B Serial Bus data

SCK I Serial Bus clock

D[7:0] O 8 bit image data

STROBE O Full line shut open timer indicator

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

6

11

DC Operating ConditionsDC Operating Conditions

12


Clock Scheme

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

7

13


Frame Timing for YCbCr Format Output

14


YCbCr422 8Bit Interface Format Output

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

8

15


I2C Control

16

AC Timing TableAC Timing Table

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

9

17

I2C Control Register ListI2C Control Register List

18


Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

10

19


20


Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

11

21


22


Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

12

23

Register Register

System StructureSystem Structure

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

13

25

Block DiagramBlock Diagram

26

I2C ControllerI2C Controller

CMOS Image Sensor Setting.

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

14

27

CIS ControllerCIS Controller

CIS 8bit YCbCr Image .

8Bit YCbCr 16 SRAM Controller

28

SRAM ControllerSRAM Controller

CIS Controller 16 .

AHB Bus ARM Stripe

Timing Waveform of Read Cycle

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

15

29

SRAM ControllerSRAM Controller

Timing Waveform of Write Cycle(/OE = Low fixed)

30

DMA ControllerDMA Controller

Transmit images from SDRAM to LCD Driver

Consumes a large amount of memory bandwidth

Configured by the slave interface

Base address of frame buffer register

Size of image to be transferred register

Enable DMA control register

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

16

31

LCD DriverLCD Driver

32

Color Space Converter(ARM Stripe)Color Space Converter(ARM Stripe)

CIS YCbCr(YUV) Image Data

TFT LCD Display RGB.

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

17

33

Operation FlowOperation Flow

1. CMOS I2C Setting : I2C Control

2. CIS 1 Frame Image Saved (SRAM) : CMOS CTL

3. Image Data Transmit to Stripe : slave_sram_iface

4. Stripe : Image Data Conversion : YCbCr -> RGB

DMA Enable

5. DMA Controller : Conversion RGB Data

6. LCD Driver : Data Display

Project OverviewProject Overview

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

18

35

Design StepDesign Step

Compile the Hardware

Build the Software

Download

36

Directory StructureDirectory Structure

cis_lcd_reference_design

ads

common

rtl

Contains the Quartus II project

Contains the software design files

Contains the hardware design source files

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

19

37

RequriementsRequriements

Quartus II v4.0 (or later version)

ARM Development Suite v1.2

Mentor Graphics ModelSim v5.7 (or later)

Compile the HardwareCompile the Hardware

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

20

39

Create ProjectCreate Project

Run Quartus II

Create New ProjectName : arm_top

Top-level Entity : arm_top

Target : EPXA4F672C3

40

Add H/W design filesAdd H/W design files

Menu > Project > Add/Remove … Click

Add *.v to project

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

21

41

Add FileAdd FileSlave_sram_iface.v

I2c_ctl.v

Cmos_ctl.v

Stripe.v

Addr_mux.v

Mux2.v

Sys_pll.v

Imag_rom.v

Slave_interface.v

Tft_lcd.v

Line_buffer.v

Default_slave.v

Response_and_data_mux.v

Slave_decoder.v

Video_component.v

Video_dma.v

Video_dma_controller.vqm

Single_transaction_slave.v

Regfile.v

42

Project HierarchyProject Hierarchy

ARM_TOPStripe

Sys_pll

Slave_decoder

Default slave

Response_and_data_mux

Video_dmaVideo_dma_controller

Tft_lcd

Slave_interface

Line_buffer

Slave_sram_ifaceSingle_transaction_slave

regfile

addr_mux

mux2

CMOS_CTL

i2C_CTL

IMAG_ROM

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

22

43

Create Create ARM StripeARM Stripe

Tools > MegaWizard .. Click

44

ARMARM--Based ExcaliburBased Excalibur

Megafunction : ARM-Based Excalibur

Output file : Verilog HDL

Output file : stripe.v Click Click

File name

Click

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

23

45

System SettingSystem Setting

Device SelectionExcalibur_ARM

EPXA4

Reset OperationBoot from Flash

Byte OrderLittle Endian

Reserve PinsEBI Automatically Enabled

Enable UART Pins

Disable Trace Pins

46

StripeStripe--toto--PLD InterfacePLD Interface

Bridges

Stripe to PLD

PLD to Stripe

Interrupts

PLD to Stripe interrupt - Mode 3

Trace/Debug

Disable Trace Extensions

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

24

47

ClocksClocks

Clk_ref

25 MHz

AHB1/AHB2

Enable PLL1

AHB1: 200 MHz

AHB2: 100 MHz

SDRAM Controller

Enable PLL2

133MHz

Serial Programming

Ignore

48

Memory MapMemory MapRegisters

Default: 0x7FFFC000; 16K

SRAM0 : OFFSRAM1 : OFFDPRAM0 : OFFDPRAM1: OFFSDRAM0 :

0x00000000; 64M, 32bitShow details > CL -> 2

SDRAM1 : OFFEBI0

0x40000000; 4M

EBI1, EBI2, EBI3: OFFPLD0

80000000; 2G

PLD1, PLD2, PLD3: OFF

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

25

4949

EBI0 SettingsEBI0 Settings

Synchronous

Wait Cycles: 8

CS Polarity: Active Low

Data Width: 16 Bits

Bus Clock Divide: 1

50

Arm_topArm_top (1 of (1 of 4)4)

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

26

51

Arm_topArm_top (2 of (2 of 4)4)

52

TopTop Level Design (3 of Level Design (3 of 4)4)

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

27

53

TopTop Level Design Level Design (4 (4 of of 4)4)

54

System_pllSystem_pll blockblock

Designed PLL

Generate two 50MHz clock

Megawizard™

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

28

55

Default_slaveDefault_slave blockblock

AMBA Bus Interface

HREADY, HRESP[1:0] Signal generation

56

Slave_decoderSlave_decoder blockblock

Address decoder blockHsel_slave_iface : 0x80000000

Hsel_slave_sram_iface : 0x90000000

Hsel_interrupt_iface : 0xa0000000

Hsel_default_slave : etc

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

29

57

Slave_decoderSlave_decoder blockblock

assign haddr_sig = haddr[31:28];

always @(haddr_sig)

case (haddr_sig)

4'b1000 : begin

hsel_slave_iface <= 1;

hsel_slave_sram_iface <= 0;

hsel_interrupt_iface <= 0;

hsel_default_slave <= 0;

end

4'b1001 : begin

hsel_slave_iface <= 0;

hsel_slave_sram_iface <= 1;

hsel_interrupt_iface <= 0;

hsel_default_slave <= 0;

end

4'b1010 : beginhsel_slave_iface <= 0;hsel_slave_sram_iface <= 0;hsel_interrupt_iface <= 1;hsel_default_slave <= 0;

end

4'b1011 : beginhsel_slave_iface <= 0;hsel_slave_sram_iface <= 0;hsel_interrupt_iface <= 0;hsel_default_slave <= 1;

endendcase

58

Response_and_data_muxResponse_and_data_mux blockblock

Video_dma to Stripe Data Interface BlockHsel_slave_iface : hrdata_slave_iface trans

Hsel_slave_sram_iface : hrdata_slave_sram_iface

Hsel_default_slave : 0

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

30

59

Response_and_data_muxResponse_and_data_mux blockblock

assign hsel = {hsel_slave_iface, hsel_slave_sram_iface, hsel_default_slave, 1'b0};

// hrdata mux

always @(hsel,hready_slave_iface,hresp_slave_iface,hrdata_slave_iface,

hready_slave_sram_iface,hresp_slave_sram_iface,hrdata_slave_sram_iface,

hready_default_slave,hresp_default_slave)

case (hsel)

4'b1000 : begin

hready <= hready_slave_iface;

hresp <= hresp_slave_iface;

hrdata <= hrdata_slave_iface;

end

4'b0100 : begin

hready <= hready_slave_sram_iface;

hresp <= hresp_slave_sram_iface;

hrdata <= hrdata_slave_sram_iface;

end

4'b0010 : beginhready <= hready_default_slave;hresp <= hresp_default_slave;hrdata <= 0;

end

default : beginhready <= 1;hresp <= 0;hrdata <= 0;

end

endcase

60

Video DMA blockVideo DMA block

AMBA interface DMA controller

TFT LCD controller

DPRAM(line_buffer)

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

31

61

Ahb2cis blockAhb2cis block

AMBA Interface – single_transaction_slave

SRAM Controller – regfile

CIS Controller – cmos_ctl, i2c_ctl

62


I2C CMOS

CMOS Camera Register ROM

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

32

63


I2C Control

64


Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

33

65

CMOS CTLCMOS CTL

Video Input

: CMOS PIXEL Clock : 25MHz

Data BUS : 8 bit multiplexed YCbCr

: SRAM 1 frame Data

66

CMOS CTLCMOS CTL

assign irq = (~cmos_fsync);

assign cmos_data = (cmos_fsync) ? 16'bz : cmos_ydata;

assign cmosram_we = store_ena;

assign cmosram_cs = store_ena;

~~~~~

if (store_flag)

begin

cmos_ydata[7:0] = cmos_y; // CbCr

cmos_ydata[15:8] = cmos_datatemp;

store_ena = 0;

store_flag = 0;

end

else

begin

cmos_datatemp = cmos_y; // Y

store_ena = 1;

store_flag = 1;

end

always @(posedge store_ena or posedge cmos_fsync)beginif (cmos_fsync)cmos_addr = 0;

elsecmos_addr = cmos_addr + 1;

end

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

34

67

CMOS_CTLCMOS_CTL

YCbCr422 8Bit Interface Format Output

68

Slave_sram_ifaceSlave_sram_iface

AMBA Interface

Transmit SRAM Data to ARM Stripe

16bit Data => 32bit Data

Slave_sram_ifaceSingle_transaction_slave

regfile

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

35

69

Single_transaction_slaveSingle_transaction_slave

ADDRESS_PHASE:

Begin

if(HSEL == 1'b0 | wait_sig ==1'b0)

begin // Slave not selected

slave_state <= ADDRESS_PHASE;

hready_r <= 1'b1;

HRESP <= 0;

latch_bus <= 1'b0;

end

else if(HTRANS == 2'b00) //IDLE

begin


hready_r <= 1'b1;

HRESP <= 0;

latch_bus <=1'b0;

end

else if(HTRANS == 2'b10) // NONSEQbegin // Single transaction

haddress_r <= HADDRESS;hwrite_r <= HWRITE;hburst_r <= HBURST;hsize_r <= HSIZE;htrans_r <= HTRANS;slave_state <= WAIT_PHASE;HRESP <= 0;latch_bus <= 1'b1;state_cnt <= 0;

if(HWRITE)hready_r <= 1'b1;

else hready_r <= 1'b0;

end // elseif

end // Address Phase

70

Single_transaction_slaveSingle_transaction_slave

WAIT_PHASE: begin

haddress_r <= HADDRESS;

hwrite_r <= HWRITE;

hburst_r <= HBURST;

hsize_r <= HSIZE;

htrans_r <= HTRANS;

HRESP <= 0;

if (state_cnt == 6) begin

state_cnt <= 0;

hready_r <= 1;


latch_bus <= 0;

end

else begin

state_cnt <= state_cnt + 1;

hready_r <= 0;

slave_state <= WAIT_PHASE;

latch_bus <= 1;

end

end

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

36

71

regfileregfile

Sram_stateSram_oe, Sram_we, Sram_cs

Addr_stateSRAM Address counter

address

Data_stateData Write

16 SRAM 32-> AHB

72

Device SettingsDevice Settings

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

37

73

Pin Assignment & CompilePin Assignment & Compile

Assign Pin to signals & Synthesis

Compile All H/W Design

74

Pin Assignment(2)Pin Assignment(2)

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

38

75

Pin Assignment(3)Pin Assignment(3)

Build the SoftwareBuild the Software

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

39

77

Add Software filesAdd Software files

Browse \software folder

Add below files

Armc_start.s

Retarget.c

Uartcomm.c

Irq.c

Main.c

Exception.c

78

Software Build SettingsSoftware Build Settings

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

40

79

Software Build SettingsSoftware Build Settings

80

Build SoftwareBuild Software

Build All Software files

Menu Processing > Start Software Build

Application Application CodeCode

PLD ImagePLD Image

Flash Flash Programming Programming

FileFile

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

41

81

Main.cMain.c

volatile unsigned int *DMA;volatile unsigned int *SRAM;volatile unsigned int *INT;

int main(void){

DMA = (volatile unsigned int *)EXC_PLD_BLOCK0_BASE;

// initialize the video driver

*(DMA+1) = HRES|VRES; // image size (240X320)

while(1);}

82

Irq.cIrq.c

void CIrqHandler(void){volatile int irqID;

irqID = *INT_ID(EXC_INT_CTRL00_BASE);

switch (irqID){case PLDINT1_IRQ_PRI:break;

case PLDINT2_IRQ_PRI:cmos_image_handler();break;

case PLDINT3_IRQ_PRI:lan_irq_handler();break;

case PLDINT4_IRQ_PRI:usb_irq_handler();break;

case UART_IRQ_PRI:uart_irq_handler();break;

default:/* This shouldn't happen, but let's trap it in case *///printf("Unknown irq %#x",irqID);break;

}

return;}

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

42

83

Irq.cIrq.c

void cmos_image_handler(void){int i,j,k;

volatile unsigned int *VGA = (volatile unsigned int *) EXC_PLD_BLOCK0_BASE;volatile unsigned int *SRAM = (volatile unsigned int *)0x90000000;volatile unsigned int *INT = (volatile unsigned int *)0xa0000000;

*INT_MC(EXC_INT_CTRL00_BASE) |= INT_MC_P2_MSK;*INT = 0x0;

#if 1 /* 240 * 320*/

k=0;

for(i=0;i<240;i++) {for(j=0;j<160;j++) {

ycbcr422[k++] = SRAM[i*160+j];}

}#endif

84

Irq.cIrq.c#if 1 /* START_CONVERSION */

k = 0; Cr2 = 0;for(i=0;i<240;i++)for(j=0;j<320;j++) {

Cr1 = Cr2;Y1 = (ycbcr422[k] & 0xff00);Cb1= (ycbcr422[k] & 0xff) -128;Y2 = (ycbcr422[k] & 0xff000000) >> 16;Cr2= ((ycbcr422[k] & 0xff0000) >> 16) - 128;r = Y1 + 351*Cr1;g = Y1 - 179*Cr1 - 86*Cb1;b = Y1 + 443*Cb1;

if(r>65280 || g>65280 ||b>65280)rgb[j][i] = 0xffff;

elsergb[j][i] = (r&0xf800) | ((g>>5)& 0x7e0) | (b>>11&0x1f);r = Y2 + 351*Cr2;g = Y2 - 179*Cr2 - 86*Cb1;b = Y2 + 443*Cb1;

if(r>65280 || g>65280 ||b>65280)rgb[++j][i] = 0xffff;

elsergb[++j][i] = (r&0xf800) | ((g>>5)& 0x7e0) | (b>>11&0x1f);k++;

}#endif /* __END_OF_CONVERSION */

t = (unsigned int)&rgb[0][0];*VGA = t;*(VGA+2) = 0x00000001; // start driver

*INT_MS(EXC_INT_CTRL00_BASE) |= INT_MS_P2_MSK;

}

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

43

DownloadDownload

8686

Target ImplementationTarget Implementation

Check Power

Connect ByteBlaster cable between PC and Target Platform

SoC-Entry II Dip-Switch S1 & S6 are all off in the bottom-center placed

Turn on Target Board Power

Run \Prog_files\Prog_hw.bat

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

44

87

Exc_flash_programmerExc_flash_programmer

Run Dos Command Window

To download image file Type “Exc_flash_programmer –p –v –g arm_top.hex” <enter>

88

DownloadDownload

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



2010-11-29

45

89

SummarySummary

CIS & I2C Controller

System component

H/W Design and Software Design

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



armarm--based based socsoc design design –– cmos image sensor · 2016-10-31 · –– cmos...

Documents