UAV Video Understanding Using GIS Products

Discriminators• Discriminators:

– Know-how in UAV domain; UAV video understanding; Geo-registration; mosaicing;

• Why is this technical product the best of class?

– Fully automated and robust UAV video understanding.

Objectives

Operational Objectives–Provides robust and automated UAV video understanding

Technical Objectives–UAV video understanding based on existing GSI products, such as maps, DTED

–UAV video understanding based GSI knowledgebase.

Development Roadmap

Maturity–A->A+, research phase, level 4-5

Plan to future product ( could be multi-year)

–Automated UAV video annotation

–Automated event detection from UAV video.

Business Parameters

NGA/ARAD/IC.

Special Ops, Army, Navy, AFRL

UAV Based Automatic Road Following

Discriminators• Discriminators:

– Know-how in UAV domain; UAV video understanding; Geo-registration; mosaicing;

• Why is this technical product the best of class?

– Fully automated UAV operation with adv. video understanding tech.

Objectives

Operational Objectives–Following a road or a pipe, etc in UAV video or by controlling UAV cameras

Technical Objectives–Road or pipe recognition and tracking

–UAV sensor control

Development Roadmap

Maturity–A->A+, research phase, level 4-5

Plan to future product ( could be multi-year)

–Detect events related to road, pipe lines or borders.

Business Parameters

Army Night Vision Lab asked us to solve this problem.

Special Ops, Army, Navy–IED detection, road, oil pipe, border protection

Information Triage

Discriminators• What sets Sarnoff apart from the

competition?– Image/video understanding

– Geo-registration

– WordNet+ from Princeton University

• Why is this technical product the best of class? – First integrated info triage solution

Objectives

• Operational Objectives

•Triage vast amount of info from different modality into an integrated intelligence source.

• Technical Objectives

•Cross-cuing among modalities

•Multi-level info triage

•Cognitive UI

Development Roadmap• Current maturity: A->A+, level 4

• Plan to future product ( could be multi-year)

•Information triage system that takes in information from different sources and integrates them.

Business Parameters•Current relationship: NGA/ARDA/IC

•Suggested Future Relationships

•DARPA/DoD

Geo-Spatial Image/Video Management

Discriminators• Discriminators:

– User intent estimation (ARDA/VACE)

– Video understanding (ARDA/VACE)

– DVL

– TerraSite

• Why is this the best?– High level understanding, cognitive GUI

Objectives

Operational Objectives– Develop geo-spatial image and video management system based on high level image/video understanding

Technical Objectives– Improve effective image/video summarization and indexing capabilities

– Develop cognitive GUI that understand user intent and learns from interaction

Development Roadmap

Maturity–A+, research phase, level 5

Plan to future product ( could be multi-year)

–A geo-spatial info. management sys.

Business Parameters

NGA, CIA, DIA– Manage vast amount geo-spatial images and videos

Automated Performance Evaluation for Convoy Training

Discriminators• Discriminators:

– Vision systems for automotive applications

– Automated performance evaluation for Military Operations in Urban Terrain (ONR program).

• Why is this the best?– Provides feedback for real convoy training

Objectives

Operational Objectives– Develop automated performance evaluation for convoy operation

Technical Objectives– Record info. related to convoy operation using audio/visual, GPS info. and metadata

– Develop performance metric and training method for convoy training

– Develop convoy operation knowledgebase

Development Roadmap

Maturity–A->A+, research phase, level 4-5

Plan to future product ( could be multi-year)

–Autonomous convoy operation

Business Parameters

ONR, Army, Marine– Training how to drive convoy

SaccadeCam: Pro-Active Motion Imagery Acquisition for Persistent Surveillance

Impacts• Provides wide are persistent surveillance and

reconnaissance

– Reduces or eliminates missed targets & events.

– Significantly increases the area that can be monitored by a single UAV.

• Automates motion imagery acquisition and improves the efficiency and effectiveness of the acquisition system.

• Greatly reduces the number of sensor operators and analyst needed for a mission.

• Provides actionable intelligence instead of raw data using information-push based on user’s intent.

• Significantly reduces the response time and enables total situational awareness.

Novel Ideas• Automate motion imagery acquisition processing and

free cameras from direct operator control.

• Inspired by the Saccade motion of eye movement, SaccadeCam moves the “fixation” or Field-Of-View (FOV) of a camera constantly and at high speed.

• Where and when to look, also referred as the Saccade motion of the camera, is determined by analyzing acquired imagery, estimating operator’s intent, underlying tasks and other sensor information, such as wide FOV camera, SARS, radar or LIDAR.

• Present motion imagery to an analyst based on his/her interests and underlying tasks.

Tasks and Efforts

Tasks Efforts1. Proof-of-concept demonstration 5 month2. System design 2 months3. Software system development 9 months a) Saccade motion planning b) Multi-layer mosaic representation c) Operator intent understanding4. Prototype system development 12 months5. Advance capability demonstration 6 months6. Final system integration 2 months

Total 36 months

Potential Clients: I2WD, NVESD2006 ROM Estimate: TBD

25W Fuel Cell Power System

• Lightweight fuel cell stacks enabled by the Sarnoff’s proprietary metal/ceramic bipolar plate technology saves weight and volume, increasing Power Density.

• Sarnoff’s Technology will enable system:

– 20-50% reduction in system weight and volume

– 30% reduction in system start-up time

– 10% higher energy efficiency

Objectives

Development Roadmap• Current Status:

Demonstrated metal/ceramic bipolar plates for lightweight fuel cell stacks

-- >5000 hrs stable operation

-- 50% reduction in weight stack and volume

Business Parameters

Current Soldier Batteries (LM-145, LI-145): ~ 150 Wh/kg

• Project Plan (24 Months):Sarnoff FC Stack Traditional FC Stack

•Teamed with a Fuel Cell System Integrator and a Supplier of Military Equipment. Contracted to build two (2) 15 W metal plate PEM fuel cell stacks (TRL5 level) using hydrogen as fuel.

•Demonstrate the metal bipolar plates using methanol (direct & reformed) as fuel.

• Sarnoff focuses on lightweight high performance fuel cells stack development ,and we will team with a experienced fuel cell system integrator to demonstrate the fuel cell power system.

•Soldier Portable Power Requirements:

• Power Density: 600-1000 Wh/kg

• Life: 2000-5000hrs

•Sarnoff Technical Objective

• Power Density: 1000 Wh/kg

• Life: 5000-6000 hrs

•Use multiple fuel to provide power

• Hydrogen/Methanol/other hydrocarbons (diesel)

Discriminators

Task Name

Task 1. Brassboard system demonstration1.1 Lightweight stack development1.2 Brassboard System Integration Milestone: TRL-4 system demonstration

Task 2. TRL-5 System Demonstration2.1 Lightweight stack Optimization2.2 Micro BoP components development2.3 System Integrationa and miniturazation Milestone: TRL-5 system demonstration

9/29

12/28

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q12006 2007 2008

Potential Client: C2D, I2WD2006 ROM Estimate: TBD

Purpose:Extend position, navigation and C2 capabilities to the dismounted Land Warrior operating in an indoor or urban environment. Merge enhanced commercial location technology and ACIN networking technology to provide Blue Force pos/nav tracking in conditions where standard techniques (GPS etc.) are blocked.

Product:• Hand Held position / navigation / messaging

device.• Portable self-configuring outdoor

infrastructure

Payoff:• Leverages commercial technologies and

recent developments in the BFAPS and ACIN-CSUE programs to provide a deployable system solution.

• Supports communications and map based location between team leader and team members.

• Provides C2 capability from remote location• Demonstration of system in FY05

• Leverage Commercial Technologies

• Address Need for Blue Force Awareness in Urban Areas

Description: Due (Months ARO):

Monthly Technical Reports Monthly

Detailed Program Plan and Schedule 1

System architecture description report 3

Demonstration of Indoor location capability 6

Demonstration of Map based location capability 8

Demonstration of self configuration capability 11

Final Report 12

Delivery of One (1) System (4 handheld devices and outdoorInfrastructure)

12

Schedule

Ad HocWireless LAN

ReferenceBeacon

Differential GPSBase Station

Ad Hoc NetworkInterface Node

AnalysisStation

TagLocationReceiver

Position / Navigation for Situational Awareness Client: C2D2006 ROM Estimate: TBD

Transferring Commercial Protocols to JTRSObjectives

Development Roadmap

Business Parameters

Discriminators•Sarnoff’s proven SDR capability with application to DoD waveforms

•Sarnoff’s capability in digital design for high-speed signal processing

•Sarnoff’s experience with many commercial waveforms and standards

• Proof of concept to support JTRS stated goal of commercial waveform support

• Demonstrate re-configurability

• Small form factor

Current Status

18 Month Development Schedule

- Developed low cost, embedded SDR platform to support Cluster 5 SFF requirements under ACIN in 2004

- Developed SoC for North American DTV reception. ATSC A/54

- Demonstrated a small form factor (SFF) radio targeted to Cluster 5 requirements in field demonstration

-The system was demonstrated under the 2004 ACIN program using the Soldier Radio Waveform, EW Mode, 8 MHz operation.

- Sarnoff’s vision processor the Acadia™ chip- Sarnoff’s DTV demodulator- Sarnoff’s MPEG Audio/Video decoder

-Soldier Radio Waveform for Cluster 5 SFF Radio-A/54, ITU j.83b, DOCSIS, CDMA?,GSM?,MPEG, H.264, WMA,MP3

- The goal is device that is portable (hand-held)- Low power- Use COTS components

-Show the ability to demodulate two different commercial waveforms-ie: DTV-8VSB and DVB-H (primary functionality)

-Demonstrate the ability to make use of existing infrastructure in other countries-Analysis of what it would take to port a commercial broadcast standard to JTRS capable system.

6 12 18 months

Develop DVB-H

DTV 8VSB to FPGA

DVB-H to FPGA

System Evaluation and Final Report

•$2,012,000 required over 18 months• $1,575,000 in year one• $455,000 in year two

•Peak loading 5 staff members