mobile vision for ambient learning in urban environments€¦ · lucas paletta, institute of...

MLEARN 2004, Lago Bracciano, ItalyLucas Paletta, Institute of Digital Image Processing

Mobile Vision for Ambient LearningMobile Vision for Ambient Learningin Urban Environmentsin Urban Environments

G. Fritz, C. Seifert, P. G. Fritz, C. Seifert, P. LuleyLuley, L. , L. Paletta Paletta and A. and A. AlmerAlmer

JOANNEUM RESEARCH JOANNEUM RESEARCH ForschungsgesellschaftForschungsgesellschaftInstitute of Digital Image ProcessingInstitute of Digital Image Processing

Graz, AustriaGraz, Austria


< Mobile vision for personal assistance< Automated mobile image analysis< Outdoors object detection and identification< Vision providing object and context awareness

< Urban environments< Tourist city information systems< Shopping assistants< Assistive technology for the visually impaired

< City exploration and learning< Intuitive multimodal Human Computer Interfaces

(HCI)< Contextual learning

MotivationMotivation


CMU Pittsburgh, USACMU Pittsburgh, USA(Yang et al. 2001)

ARAR--PDA, GermanyPDA, Germany

Multimedia VTT, Multimedia VTT, FinlandFinland

MotivationMotivationMobile Vision Mobile Vision -- State of State of thethe ArtArt


MotivationMotivationJR JR Outdoors Outdoors Object RecognitionObject Recognition

< EU Cognitive Vision Projects< EC Project DETECT: logo detection

< EC Project MACS: purposive robot vision

< Mobile Vision

< Industrial Projects< Mobile road sign inventory

< Traffic surveillance systems

< JRP ASF Cognitive Vision –Key Technology for Personal Assistants

< Attentive recognition strategies

< Contextual memories


MotivationMotivationUrban Information SystemsUrban Information Systems

< Vision enhanced city explorer

< Information in field of view

< No maps or addresses


Mobile Mobile Learning Learning in Urban in Urban EnvironmentsEnvironmentsUrban Urban ObjectsObjects


Mobile Mobile LearningLearning in Urban in Urban EnvironmentsEnvironmentsAmbient Ambient LearningLearning

< Exploration of urban environment

< Responsive to objects in the field of view

< Object awareness


Mobile Mobile LearningLearning in Urban in Urban EnvironmentsEnvironmentsSystem System ArchitectureArchitecture


Mobile Mobile LearningLearning in Urban in Urban EnvironmentsEnvironmentsMultimedia User InterfaceMultimedia User Interface


Computer Vision Computer Vision MethodologyMethodologyObject RecognitionObject Recognition ExperimentExperiment

< Local environment (Hauptplatz Graz)

< 10 objects (7 buildings, 3 statues)

< Each from 3 different viewpoints

< Imagettes 10x10, 15x15, 20x20, 25x25

< Informative features and voting


Computer Vision Computer Vision MethodologyMethodologyLearning DiscriminativeLearning Discriminative FeaturesFeatures

ROI for training samples

Discriminative imagettes


Computer Vision Computer Vision MethodologyMethodologyDiscriminative Discriminative FeaturesFeatures

Discriminative features Voting of responses Integration of votes


Computer Vision Computer Vision MethodologyMethodologySightSight IdentificationIdentification

Opera houseOpera house

Hauptplatz Hauptplatz buildingbuilding

Hauptplatz Hauptplatz buildingbuilding

Church Church statuestatue


Computer Vision Computer Vision MethodologyMethodologyViewpointViewpoint ChangesChanges

Glacis Glacis avenue buildingavenue building

ROI for training samples


Computer Vision Computer Vision MethodologyMethodologyLocal Local Information and Information and AccuracyAccuracy

original information selected accuracy

COIL-20

Fritz et al. AAAI 2004

))((log))|( ggg |oP|P(oH ko

kak

∑Ω∈

−=Ω


Computer Vision Computer Vision MethodologyMethodologyInformative Informative RepresentationsRepresentations

< Entropy threshold Θ

< Maximum a posteriori interpretation

< Sparse object representation (≤10%)

all imagettes

informative imagettes


Computer Vision Computer Vision MethodologyMethodologyRobust on Robust on OcclusionOcclusion and and NoiseNoise

40% occlusion

80% occlusion

50% Gaussian noise


Computer Vision Computer Vision MethodologyMethodologyOcclusion Occlusion Performance Performance AnalysisAnalysis

No selection

Fritz et al. ICPR 2004

INFORMATIVEINFORMATIVE imagettes

ALLALL imagettes


< Conclusion< Mobile vision and object recognition< Learning from city exploration

< Ongoing and future work< Extended scope - objects, viewpoints, illumination< Application using camera phones< Logging tourists

< localisation< trajectories< focus of attention

ConclusionConclusion and Outlookand Outlook


LocalisationLocalisation

3a2

1a

1b

1c3b

4

1d

LegendGPS-based position estimationObjectObject capture position (OCP)GPS-indexed OCP

2


Thank You for Your AttentionThank You for Your Attention !!

mobile vision for ambient learning in urban environments€¦ · lucas paletta, institute of...

Documents