USB host for web camera

connection

Final presentation – Part A

Presenters:

Alexander Shapiro

Sergey Alexandrov

Supervisor:

Mike Sumszyk

High Speed Digital Systems Lab

Agenda

Project objectives. System architecture. Project flow.

USB basics. ISP 1362. Isochronous transfer.YUV video.

Part B project objectivesDemo

Project Objectives

Implementation of SOPC interface for USB webcam. Learn and implement USB host protocol. Learn and implement webcam protocol. Learn SOPC Development tools. Understand NIOS II architecture. Building NIOS II softcore system.

Receive picture from the webcam. Convert YUV to RGB and display in Matlab

Development Environment

Altera DE2 board Provides USB support for host and device application. VGA output (up to1600 x 1200 at 100-Hz refresh rate).

Cyclone II FPGA Programmable chip to implement any design.

Programming tools: Quartus II, Nios II IDE, SOPC Builder

Debug tools: Hardware and software USB analyzer.

System Architecture

Cyclone II FPGA

Philips ISP1362USB Controller

VGA D/A converter

USB 1.0 Port

VGA Port

On Chip Memory

NIOS IISoftCore

Avalon Switch

On- ChipMemory

DRAMMemory

InputUSB Philips ISP1362 controller.Transfer rate: 12 Mb/s.

OutputVGA DAC (10-bit triple DACs).Up to 1600x1200 at 100-Hz.

Logic Flow diagram

DATA in YUY2 formatPayload

1. Check errors2. Compose YUV frame

3.Decode to RGB4. Send to VGA buffer

Project Flow

Learn stage

Working environmentSOPC builderNIOS II

USB and ISP1362 documentation Take mouse demo as basis for USB

system.

Host implementation

Initialize connection and open communication port.

Determine the kind of device. Request configuration descriptor. Learn basic device capabilities.

Webcam interface implementation

Probe and commit sequence. Negotiation between Host and Video streaming

interface. Consists of Host`s requests with supported

streaming parameters (bit rate, resolution) and Client`s confirmation/denial

Select video interface. Device is configured and ready to work.

Probe & Commit flow

Host and device negotiate acceptable configuration.

It may take several iterations to “agree”

Webcam interface implementation

Mid-project milestone

Working webcam interface on NIOS II system Initialization Configuration Bulk data transfer No isochronous traffic support

ISO transfer over Philips ISP1362 Isochronous transfer is the most challenging aspect

of the USB protocol. Main issue – absence of code examples.

Almost no help could be obtained from forums and documentation.

No guidelines for implementation in Programming Guide. Was implemented from scratch. USB analyzer was irreplaceable in understanding

and debugging.

ISO transfer over USB

SOF

DATA 0IN Payload

SOF

DATA 0IN Payload

DATA 0IN Payload

DATA 0IN Payload

. . . Host periodically sends ISO IN requests.

Device responds with ISO packets.

Each packet consists of payload and data.

Header holds error checks and End Of Frame indicators.

DATA is video stream in YUV2

Isochronous transfer buffering

Challenge: Data to be delivered and processed each mSec. CPU on embedded system cannot handle data each mSec.

Solution: ISP 1362 supports double buffering for ISO transfer One buffer is proceeded while the second is filled. 1024 bytes in each buffer. Toggle rate is configurable.

Debug tools: USB analyzer

USB HW analyzer is irreplaceable in debug. Shows real traffic between host and device Triggers on defined events Clear log of USB packets Showed what packets Windows

sends to USB device. Software analyzer can show only driver level packets.

Helped find source of bugs. Made it possible to ensure that

host/device behaves according to our expectation.

Decode YUY2 video format

2 RGB pixels are defined in each macropixel (four

bytes). They share U and V components. Natural bitrate is 2 bytes/pixel

Demands efficient real-time decoder to RGB

Part B Project Objectives

USB 1.1 Video implementation. Integrate VGA controller. Implement real-time video streaming

USB 2 support.Migration to DE-3 environment.

Code optimization.DMAHardware YUV to RGB decoder

USB 2 Video streaming to monitor

References

Philips Embedded Programming guide Philips ISP 1362 Data Sheet USB 2.0 Specification USB Video Class Specification USB Complete by Jan Axelson Google