A Linear-Algebraic Technique with an Application in Semantic Image

RetrievalInternational Conference on Image and Video Retrieval 2006

Jonathon S. Hare and Paul H. LewisIntelligence, Agents, Multimedia Group

School of Electronics and Computer ScienceUniversity of Southampton{jsh2 | phl}@ecs.soton.ac.uk

& Peter G.B. Enser and Christine J. Sandom

School of Computing, Mathematical and Information SciencesUniversity of Brighton

{p.g.b.enser | c.sandom}@bton.ac.uk

Contents

Introduction

Using Linear Algebra to Associate Images and Terms

A Simple Semantic-Space

foo

Key

SKYTREE

MOUNTAIN

CABLE CAR

Visual terms

Keywords

Documents

Experimental Results

Real World Applications

In “The Bridging of the Semantic Gap in Visual Information Retrieval” project we are exploring how test-bed ontologies combined with content-based techniques and annotation can help meet the needs of real users in limited domains.

In particular, we are investigating how the factorisation technique works with real image collections.

Real World Applications The Kennel Club Data-set

Images of dog related activities from the Kennel Club.

About 3000 annotated images (noisy keywords).

~3600 unannotated images.

Images indexed with quantised DoG/SIFT features.

3000 term vocabulary, trained on Washington data-set.

Naively applied the factorisation technique, without any cleaning of the keywords.

Real World Applications The Kennel Club Data-set :: Demo