DSPInteraction and

Media

Video and MultimediaImaging and Communication

Sensing and Devices

Mark SmithSensing and Mobility

HP Labs, Palo Altomsmith@hpl.hp.com

HP labs’ roles

• participate in hp strategy creation

• deliver technology that enables hp to win in hp’s selected strategies through:

– breakthrough technologies

– technology advancements

• create opportunities that go beyond hp’s current strategies

• invest in fundamental science and technology in areas of interest to hp

HP 9100Compute

r

HP’sLaserJetPrinter

Hand-Held

Calculator

AtomicClock

Access7Telecom

Monitoring

System

Real timeUltra-SoundCardiac Analysis

PA-RISC DigitalPhotography

Molecular Computing

IA-64 / EPICComputer

Architecture

DesktopScientificCalculato

r

InkJetPrinter

Atomic Resolution

Storage

Web QoS

Future

Audio Oscillato

rin

Disney’s“Fantasi

a”

Tradition of Innovation

JetSendProtocol

High-Frequenc

yCounter

HP 95LXPalmtop

PC

World’sBrightest

LEDs

‘98‘72‘67 ‘86‘51 ‘91 ‘97’95 ‘2000‘68 ‘841939 ‘66 ’94’93

OmniBook 300Lightest

Portable PC

WebWristwatch

(w/ Swatch)

Change:

Applications deploy

an end to end space.

• Applications are network sourced.

• Services lie outside the network.

• Devices are aggregated into what the application and user wants.

Content Transaction

End to End Application Space

Vehicle Space

Living Space

HPL Streaming Media Technologies

StreamingMedia

IndustrySplicing

DigitalVCR Independent

RegionCoding

SmartRecoding

Rate Changing

Error-ResilientVideo Coding

Low-PowerVideo

Compression

MPEG2to H.263

WirelessVideo

DigitalTelevisionIndustryMPEG4

to MPEG2

Object-BasedVideo Processing

HP Technologies

SecureVideo

Streaming

Problem Statement

ProcessDecode Encode

How do we process compressed media streams?

1 Gbps 1 Gbps20 Mbps20 Mbps

2,000 MOPS 20,000 MOPS

• Difficulties:– Computational requirements: 22,000 MOP

overhead from decode and encode operations (plus additional processing)

– Bandwidth requirements: Need to process uncompressed data at 1 Gbps

– Quality issues: Even without processing, the decode/encode cycle causes quality degradation.

010001101001101101

100110101011100010

CDP

ProcessDecode Encode

1 Gbps 1 Gbps 20 Mbps20 Mbps

2,000 MOPS 20,000 MOPS

Compressed Domain

Processing

• In the digital world, media streams are stored, transported, and distributed in compressed form.

How do we process compressed video streams?

Use efficient compressed-domain processing

algorithms.

100110101011100010

010001101001101101

CITMO 2002

Streaming Media System

HP SmartStream Router

Full resolutionDecode and Display

Low resolutionDecode and Display

Low-bandwidthwireless link

High-bandwidthLAN

Full resolutionCapture and

Encode

Transcoder

HP Streaming System

Media streaming and transcoding over packet

networks.

Secure Delivery and Transcoding of

Compressed Video Streams

HP SmartStream Router

Full resolution Decrypt,Decode, and Display

Low resolution Decrypt,Decode, and Display

Low-bandwidthwireless link

High-bandwidthLAN

Capture,Encode,

and Encrypt

SecureTranscoder

HP Secure Streaming System

Secure media streaming and transcoding over packet

networks.

Secure Transcoding

HP CapShare 910 information appliance

One of the most DSP intensive things I ever worked on!

Scanning motion up to 1 mph

Image processing path

Path Reconstructionand Page Orientation

Image Rectification Image Stitching

2D optical navigator• Image X-correlation

processor• 2D motion & contact

detection• Derivative used in optical

mice• Prototype:

• 600 dpi resolution• 32x64 pixels• 1.5 GOPS computation

rate• 25K frames/second (1

mph)• 180 mW power dissipation

• Commercial versions:• 18x18 pixels• 2.3K frames/sec• 75 mW power dissipation

Device Operation

Grazing Illumination

Optics

Object surface Texture

Surface Texture Image

2D Minima Interpolation

Compute Minima Translation

9 LocalPixels

9 Nearest Pixel Correlations

Final navigator design

SecurePAD—Dynamic Security

Goal of providing access, security and accountability based on who, where and how a resource is used.

Biomedical, Health, Sensors and Algorithms

Aggregate sensor data to give physical meaning.Used by services to provide:

PersonalizationPrivacy assuranceConnectivity optimization

Current Efforts:Device PossessionBiometric physiological stateContext based Privacy AlgorithmsSensor enhanced Security Algorithms

In the watch band

The Djammer: DSP for interactive MP3

Wearable, grab and shake design

USB sensor interface allows multipledevices and use with BlueTooth earpiece

Music interaction and scratching is doneusing an optical sensor and accelerometers

Prototype used 802.11b for the wirelesslink. Bluetooth is also possible.

DJammer = DJ + jam session

What is new about the DJammer is:

1. It allows mobile users to interact with the digital music presented to them.

2. It allows users to augment music contents through a digital realization of music scratching (pitch control).

3. It permits the resulting augmented music to be communicated by streaming to other DJammer users allowing further augmentation and streaming.

4. It is capable of being aggregated into sessions with other dissimilar devices allowing sharing of music.

MEMS Micropositioners– Folded flexures

provide x and y axis motion, while limiting z displacement

– Motor fabrication integrated with storage mechanism fabrication

A

A’

D.A. Horsley et al., Proc. of the 2000 Solid State Sensor & Actuator Workshop,Hilton Head, SC, June 4-8, 2000.

Atomic Resolution

StorageProbe-based storage using phase-change medium, array of electron emitters, MEMS micropositioner

– Write / Erase >apply strong electron

beam from tip >heating/cooling profile

determines write / erase.

– Read>apply weak electron

beam from tip>detect resistance of

written bits>Optical reading

techniques>SEM , others...

”Avoiding a Data Crunch”, J. W. Toigo, Scientific American 282, 58 (2000)

ARS Medium working!

• HP Logo – written on

phase change material

– conductivity image, not topological image

Molecular ElectronicsPushing the Limits

Scaling of electronic devices

101 100 10-1102

104

106

108

Number of chip components

Feature size (microns)

1010

1012

1018

1014

1016

10-2 10-3

Classical Age

Historical Trend

SIA Roadmap2010

CMOS

19952000

2005

1970

1980

1990

4oK

Quantum Age

77oK

295oK

Quantum State Switch

Silicide Nanowires on Siparallel quantum wires –

nine nanometer separation

Atomic Image

HP Company Technology Modelradical ideas

current products

technology advancements

breakthrough technologies

product organization

s

labs

next generation products

labstransfers

molecularelectronics

bistable displays

atomic resolution storage

InternetAppliancesStreaming

MediaCommunicationand InternetworkingCoolTown

Imaging

PrintingWirelessNetworking

StorageSystems

ComputerArchitectures

ComputerGraphics

SecurityAlgorithms

Compression

SoftwareEngineering