Digital Camera Design

Agenda

• Digital video formats• Image sensor technology• Sensor interface with CoolRunner-II• LCD• CoolRunner-II system design

Digital Video

• RGB– Digitized version of analog RGB– Most common in computer graphics

• YCrCb – Digitized version of analog YUV and YPbPr– Format used by DVD and digital television

Color Formats• RGB

– Primary additive colors: red, green, & blue– Not efficient format for image processing

• YCrCb – Luminance, Chroma Red, Chroma Blue– 8-bit ranges: Y (16 to 235), Cr & Cb (16 to 240)– 4:2:2 & 4:4:4 formats determine when Cr and Cb are sampled with Y

Range White Yellow Cyan Green Magenta Red Blue Black

R 0 to 255 255 255 0 0 255 255 0 0

G 0 to 255 255 255 255 255 0 0 0 0

B 0 to 255 255 0 255 0 255 0 255 0

Digital Video Capture

• Video is series of still images (which each consist of lines of data)• Control signals:

– VSYNC: Vertical sync (asserted during frame scan time)– HSYNC: Horizontal sync (asserted during line scan time)– BLANK: blanking– CLK: 1x or 2x sample clock

• Digital output represents data of each pixel (RGB or YCrCb)• Data sampled on rising edge of sample clock

Output Timing

CCD CMOS

Need specialized fabrication methodology Standard fabrication

Require numerous power supplies andvoltage regulators for operation

Single 3.3V (or 5V) supply needed

Typically requires 2–5 watts per digital output Requires 20–50 milliwatts per pixel output

Ideal for low light applications Lower power consumption

Ideal for handheld, portable devices

Technology allows integrating with imageprocessing: SOC

Image Sensors• CCD (charge-coupled device) vs. CMOS

• CMOS manufacturers: Kodak, Micron, Hynix, OmniVision, Mitsubishi, ST Microelectronics, Toshiba

MI-SOC-0343

• Complete CMOS image sensor camera-on-chip solution• “Active pixel” sensor architecture• Image sensor core + digital image processing technology• Outputs digitally processed RGB or YCrCb data• Programmable control registers• 640 x 480 VGA image array

Active Pixel Architecture

Photodiode Active-Pixel Architecture Photogate Active-Pixel Architecture

• CMOS sensorphotosite area breakdown25% = circuitrySensor fill factor = 75 %

• Active pixel structure prevents background image noise

Image courtesy of Micron

Sensing Color

• Funnel light to the photosensitive portion of each pixel• CFA allows each RGB color to be measured independently

Image courtesy of Micron

Bayer CFA

• Invented by Kodak• Able to separately measure red, green, & blue photons• 2x2 repeating arrangement• Output: sequential RGB (sRGB)

G R G R G R G RB G B G B G B GG R G R G R G RB G B G B G B GG R G R G R G RB G B G B G B G

Image courtesy of Micron

MI-SOC-0343 Diagram

Active Pixel Array(640 x 480)

Image Core Register Set

Analog Processing ADC

Imager Core

Image Flow Processor

Register Set

Data Output& Timing

Image Flow Processor

Serial Interface(SCLK, SDATA)

• Color Correction• Gamma Control• Sharpening Control• Saturation Control• AWB• AE• Defect Correction• Lens Detection

Image Flow Processor

• Interpolates 1 color/pixel into 3-colors/pixel by filling in missing data based on adjacent pixels

• Includes: lens shading, edge detection, aperture correction, color correction, AWB, AE, gamma correction, color saturation control, and zoom features

• 77 addressable control registers• Output either 4:2:2 YCrCb (CCIR656) or 4:4:4 565RGB data

CoolRunner-II Interface

MI-SOC-0343

SCLK, SDATASHIP

Interface Control Logic

Image Grabber

Control Logic

FRAME_VALID

LINE_VALID

PIX_CLK

SRAMInterface Logic

SRAM

Address / Data / Control

AddressCounterTop Level

Control Logic

DOUT (7:0)

24 MHz Clock

LCD

• Best LCD solution for video application: color active matrix TFT• Transmissive or transflective polarization type• TFT modules offer sub-pixels (3 per pixel) to represent each RGB

color• LCD (with an integrated driver) accessible via parallel data

interface

Optrex LCD

• 6.4” Transmissive Color TFT• Dual CCT backlight• 640 x (RGB) x 480 (VGA format)• Thin form factor• Parallel 6-bit RGB data interface

LCD Panel(640 x 480)

TimingLogic

Vertical IC Driver

CCT

Horiz

onta

l IC

Drive

r

CCT

CoolRunner-II Interface

OptrexT-51382D

LCD Interface

Control Logic

VSYNCHSYNC

CLK

SRAMInterface Logic

SRAM

Address / Data / Control

Top LevelControl Logic

R (5:0)G (5:0)B (5:0)

PWMControl Logic

K2607 Inverter

System Design

LCDPanel

LCD Interface

Control Logic

SRAMInterface Logic

SRAM

MainControl Logic

PWMLogic

Inverter

CMOSImageSensor

SHIP Interface

Control Logic

Image Grabber

Control Logic

Cell Phone Camera Expansion

CMOSImageSensor

uP

KEYBOARD

DISPLAY

DRAM

EPROM

SRAMDSP

D/A

A/D SPEECHCODER

SPEECHDECODER

CHANNEL CODER

CHANNEL DECODE

CHANNEL SPREAD

RAKERCVR

RF MOD

RF DEMOD

DUPLEXER

RFAMP

RF RCV/AMP

Summary

• CoolRunner-II allocation– Design uses a 384 macrocell device (with approx. 80% utilization)– Varies based on internal ROM size for CMOS image sensor register writes

• CoolRunner-II is ideal for digital camera applications in portable and handheld devices– Provides a low power flexible interface with CMOS image sensors– Provides a low power data allocation and memory resource solution

• CoolRunner-II can be used to extend existing design (ie. PDA, cell phone) and add digital camera functionality