Interactive Visual System

By

Arthur Evans, John Sikorski, and Patricia Thomas

The Client

Graduate student and professor at Rensselaer Polytechnic Institute

Implemented CAVIAR on a desktop PC

Interested in a mobile version

CAVIAR

CAVIAR George Nagy, DocLab RPI, Troy, NY 12180

13

The Idea

Image recognition is difficult Humans excel at segmentation Computers excel at large scale memory and

calculations Combine the two for optimal performance

Faster than a layperson and more accurate than a fully automatic system

Requirements

Demonstrate user interaction with an image on a handheld computer.

Allow for three methods of user interaction Pointing to image Outline part of image User response to queries

Top three matches must be displayed with target image.

Create a log file to log user actions and time.

Hardware

Our Handheld Criteria

Must run java

Must take a camera attachment

Must take a wireless LAN card

Personal Digital Assistants (PDA’s) PDA’s run Personal Information Management

(PIM) software. Address books Calendars

Limited computational power. Examples:

Palm Pilot Handspring Visor

Handheld Computers

Extensive computational power. Designed as portable computers. Fill the gap between desktop/laptop PC’s and

PDA’s. Examples:

Compaq Ipaq series Sharp Zaurus

Hardware Selection

Sharp Zaurus SL-5500 200 MHz Intel StrongArm processor 64 MB memory 1 Compact Flash port and 1 SD port

Sharp CE-AG06 camera attachment Plugs into Compact Flash port

Linksys wireless LAN card (WCF11) 802.11b Plugs into Compact Flash port

Design

Data Flow Diagram

HandheldCamera

PDA Stylus Pad

User commands and data

Photo of flower

PDA screen display

Server/Pattern Recognition Software

FlowerDataobject

Encapsulate Flower Image & results of user interaction

into flowerData object

Transmit flowerData Object to Remote

Server

Load user selected photo

Store image as local file

Transmit receive confirmation back to

user

Transmit classification decision to user

Package Structure

GUI – implemented with AWT MainGUI class provides layout for panels Each ‘action’ implemented as a class that extends java.awt.Panel Additional supporting classes

Data FlowerData class encapsulates image and data to send to

server. FlowerMatch class encapsulates image matches from recognition

engine. Additional supporting classes

Remote - access recognition engine via RMI Well defined protocol Modular design Consists of several server and client side classes

Java 2 Micro Edition (J2ME)

Recent specification

Provides Java functionality to a broad range of devices

Uses concept of Configurations and Profiles

Basic J2ME Architecture

Operating System

Java Virtual Machine

Configuration

Profile

Configurations

Tied to JVM on a low level Consider memory, processor, display and

connectivity Currently 2 are defined

Connected, Limited Device Protocol (CLDC) Connected Device Protocol (CDC)

Profiles

Sits on top of a Configuration Provides implementation API Currently 2 are defined:

Mobile Information Device Protocol (MIDP) Typically implemented on PDA’s and cellular phones

Personal Profile Targets high end handheld Reference implementation available for the Zaurus

Personal Java

Implementation by Insignia Systems – Jeode

Not a J2ME specification

Based on JDK 1.1.8

Implements most Personal Profile specifications

Summary

Successfully demonstrated user interaction on a handheld

Wireless connection to server Created structure for IVS application

Easily extendible Identify different objects by changing database

and data points Exceeds client expectations

Next Semester

Add pattern recognition Call C++ code with JNI? Rewrite in java?

Add segmentation Automatic User interaction