Digital Camera Design
Agenda
• Digital video formats• Image sensor technology• Sensor interface with CoolRunner-II• LCD• CoolRunner-II system design
Digital Video
• RGB– Digitized version of analog RGB– Most common in computer graphics
• YCrCb – Digitized version of analog YUV and YPbPr– Format used by DVD and digital television
Color Formats• RGB
– Primary additive colors: red, green, & blue– Not efficient format for image processing
• YCrCb – Luminance, Chroma Red, Chroma Blue– 8-bit ranges: Y (16 to 235), Cr & Cb (16 to 240)– 4:2:2 & 4:4:4 formats determine when Cr and Cb are sampled with Y
Range White Yellow Cyan Green Magenta Red Blue Black
R 0 to 255 255 255 0 0 255 255 0 0
G 0 to 255 255 255 255 255 0 0 0 0
B 0 to 255 255 0 255 0 255 0 255 0
Digital Video Capture
• Video is series of still images (which each consist of lines of data)• Control signals:
– VSYNC: Vertical sync (asserted during frame scan time)– HSYNC: Horizontal sync (asserted during line scan time)– BLANK: blanking– CLK: 1x or 2x sample clock
• Digital output represents data of each pixel (RGB or YCrCb)• Data sampled on rising edge of sample clock
Output Timing
CCD CMOS
Need specialized fabrication methodology Standard fabrication
Require numerous power supplies andvoltage regulators for operation
Single 3.3V (or 5V) supply needed
Typically requires 2–5 watts per digital output Requires 20–50 milliwatts per pixel output
Ideal for low light applications Lower power consumption
Ideal for handheld, portable devices
Technology allows integrating with imageprocessing: SOC
Image Sensors• CCD (charge-coupled device) vs. CMOS
• CMOS manufacturers: Kodak, Micron, Hynix, OmniVision, Mitsubishi, ST Microelectronics, Toshiba
MI-SOC-0343
• Complete CMOS image sensor camera-on-chip solution• “Active pixel” sensor architecture• Image sensor core + digital image processing technology• Outputs digitally processed RGB or YCrCb data• Programmable control registers• 640 x 480 VGA image array
Active Pixel Architecture
Photodiode Active-Pixel Architecture Photogate Active-Pixel Architecture
• CMOS sensorphotosite area breakdown25% = circuitrySensor fill factor = 75 %
• Active pixel structure prevents background image noise
Image courtesy of Micron
Sensing Color
• Funnel light to the photosensitive portion of each pixel• CFA allows each RGB color to be measured independently
Image courtesy of Micron
Bayer CFA
• Invented by Kodak• Able to separately measure red, green, & blue photons• 2x2 repeating arrangement• Output: sequential RGB (sRGB)
G R G R G R G RB G B G B G B GG R G R G R G RB G B G B G B GG R G R G R G RB G B G B G B G
Image courtesy of Micron
MI-SOC-0343 Diagram
Active Pixel Array(640 x 480)
Image Core Register Set
Analog Processing ADC
Imager Core
Image Flow Processor
Register Set
Data Output& Timing
Image Flow Processor
Serial Interface(SCLK, SDATA)
• Color Correction• Gamma Control• Sharpening Control• Saturation Control• AWB• AE• Defect Correction• Lens Detection
Image Flow Processor
• Interpolates 1 color/pixel into 3-colors/pixel by filling in missing data based on adjacent pixels
• Includes: lens shading, edge detection, aperture correction, color correction, AWB, AE, gamma correction, color saturation control, and zoom features
• 77 addressable control registers• Output either 4:2:2 YCrCb (CCIR656) or 4:4:4 565RGB data
CoolRunner-II Interface
MI-SOC-0343
SCLK, SDATASHIP
Interface Control Logic
Image Grabber
Control Logic
FRAME_VALID
LINE_VALID
PIX_CLK
SRAMInterface Logic
SRAM
Address / Data / Control
AddressCounterTop Level
Control Logic
DOUT (7:0)
24 MHz Clock
LCD
• Best LCD solution for video application: color active matrix TFT• Transmissive or transflective polarization type• TFT modules offer sub-pixels (3 per pixel) to represent each RGB
color• LCD (with an integrated driver) accessible via parallel data
interface
Optrex LCD
• 6.4” Transmissive Color TFT• Dual CCT backlight• 640 x (RGB) x 480 (VGA format)• Thin form factor• Parallel 6-bit RGB data interface
LCD Panel(640 x 480)
TimingLogic
Vertical IC Driver
CCT
Horiz
onta
l IC
Drive
r
CCT
CoolRunner-II Interface
OptrexT-51382D
LCD Interface
Control Logic
VSYNCHSYNC
CLK
SRAMInterface Logic
SRAM
Address / Data / Control
Top LevelControl Logic
R (5:0)G (5:0)B (5:0)
PWMControl Logic
K2607 Inverter
System Design
LCDPanel
LCD Interface
Control Logic
SRAMInterface Logic
SRAM
MainControl Logic
PWMLogic
Inverter
CMOSImageSensor
SHIP Interface
Control Logic
Image Grabber
Control Logic
Cell Phone Camera Expansion
CMOSImageSensor
uP
KEYBOARD
DISPLAY
DRAM
EPROM
SRAMDSP
D/A
A/D SPEECHCODER
SPEECHDECODER
CHANNEL CODER
CHANNEL DECODE
CHANNEL SPREAD
RAKERCVR
RF MOD
RF DEMOD
DUPLEXER
RFAMP
RF RCV/AMP
Summary
• CoolRunner-II allocation– Design uses a 384 macrocell device (with approx. 80% utilization)– Varies based on internal ROM size for CMOS image sensor register writes
• CoolRunner-II is ideal for digital camera applications in portable and handheld devices– Provides a low power flexible interface with CMOS image sensors– Provides a low power data allocation and memory resource solution
• CoolRunner-II can be used to extend existing design (ie. PDA, cell phone) and add digital camera functionality