The role of context in

Dag Elgesem &

Dept. of Information Science and Media

in image interpretation

& Joan Nordbotten

Media Science, University of Bergen

Mobile Phone Visual QueriesImage + GPS + time

1

Resolving Ambiguity

Our approach:

based on:• Object identification• Location information• Intention probabilities

3

CONTEXT

The information that must be

common knowledge

between user and system

for communication to succeed.

5

MMIR: Mobile Multi-modal Information Retrieval

MMIR

Probalilities

Events, history

...........

Known objects

7

But, what is the question?

Foreground/background:The sailboat? Statsraad Lemkul

The wharf? Bryggen i Bergen

Whole/component:The cathedral? Nidaros Domkirken

The statues?

2

Visual query interpretation

MMIR

DecisionTree

StatsraadLemkul...

Bryggen...

Other

p1

p2 p3

4

Elements of common knowledge

• Object representation

• Image structures –> real world objects

• Interpretation options

• Option probilities

6

Summary

Given

– A set of non-ambigous images with

– Location(s) for source photos and

– An effort value for each location

Than

– Map query object(s) to stored images

– Determine probability of intended query

8

Context Aware Image management

VERDIKT: pr.nr.

CAIM web-site: http://caim.uib.no

Thank you for your interest o n@ n om .u .noa i f edia ib