computer vision and media group: selected previous work

24/10/02 AutoArch Overview

Computer Vision and Media Group:

Selected Previous Work

David Gibson, Neill CampbellColin Dalton

Department of Computer ScienceUniversity of Bristol


Duck: The AutomaticGeneration of 3D Models

• Generating 3D computer models is difficult• Put object on turntable• Take 8 pictures of it from different angles• Crank the handle…• No skilled user or expensive equipment• Make avatars by spinning person on chair


Cog and Stepper

• Automatically inject ‘life’ into computer animations

• 3D swathe through 4D space time• Where space is 3D computer model• Or just to make things look strange!


Casablanca: Motion Ripper

• Computer animation driven by film• Animator labels a small number of points• System then tracks these points over all

frames• Motions are extracted and used to drive

animation


Laughing ManMotion Ripper Part 2

• Automatic video creation• Points are marked and tracked• System learns the motions• System generates new motions which are

different but ‘correct’• Forever!


AutoArch: The Automatic Archiving of Wildlife Film

Footage

David Gibson, Neill CampbellDavid Tweed, Sarah Porter

Department of Computer ScienceUniversity of Bristol


Motivation

• BBC Natural History Unit• Manual archiving/meta data generation• Reuse problematic

– Inefficient/time consuming– Expensive– Limited access

• Obvious need to automate


Objectives

• Generate efficient visual representations– Video segmentation– Visual browsing/summarisation– Visual searching

• Generate as much meta data automatically– Camera motions/effects– Scene structure– Scene content


System Overview

ShotSegmentation

VisualSummarisation

MotionAnalysis

Colour/TextureAnalysis

Meta data extraction algorithms

Catalogue Entry

Visualisation based algorithms

Visualisation and Searching


Video Segmentation


Visual Summarisation

• Key frame extraction


Visual Summarisation Tree

Entir

e sh

ot

Level of detail


Visual Searching

• Layered 2D representationof high D clip space


Motion Analysis using point tracking

•Camera Motion Estimation•Event/Area of Interest Detection•Gait Analysis•Foreground/Background Separation•Combine with Colour and Texture for Classification•See cheetah track avi


Camera Pan

BCD0111.09_0085.epslines = 47, curls = 98, shorts = 5long lines = 47, mode = 95.00, mean = 95.21, std = 4.15zoom centre = (603.01, 63.65), val = -0.2356zoom residual per line = 22.92zoom residual #2 per line = 28.92Average line vector: 109.94 -8.27

pan/tilt angle: 94.30, vector: (109.94 -8.27)pan/tilt residual per line = 21.67pan/tilt residual #2 per line = 33.38percentage of lines within 5% of mode: 89.36


Camera Zoom

BCD0113.15_0067.epslines = 142, curls = 1, shorts = 7long lines = 134, mode = 340.00, mean = 227.24, std = 128.76zoom centre = (182.97, 55.52), val = 0.2063zoom residual per line = 4.86zoom residual #2 per line = 6.90Average line vector: -3.81 17.28pan/tilt angle: 347.57, vector: (-3.81 17.28)pan/tilt residual per line = 13.85pan/tilt residual #2 per line = 16.13percentage of lines within 5% of mode: 17.16


Tracking Failure

This could be an interestingevent in its self: flocking,herding, close up of lots ofactivity, shot grouping, etc.


Event/Area of InterestDetection


Frequency Analysis:Gait Detection

FFT

After trajectory segmentation


Foreground/BackgroundExtraction

Feature space #1

Feat

ure

spac

e #2

Foregroundmodel

Backgroundmodel

Which pixelsare foreground?


Animal IdentificationGive models a name:

= cheetah

= elephant

= zebra

= lion


Some Problems

• Noise in images• Noise in measurements• Camouflage• Occlusion• Answer: Need higher level models• See next few slides


Model Based Tracking


Lion Tracking

• Synchronise horse model with lion points• Move and deform horse model to lion points• See avi• To do: Improve spatial deformation, especially for

legs, using colour and texture


Multiple Object Tracking


Conclusions

• Visualisation is very powerful• Combined with text is even better!• Assists searching and communication• Lots of meta data can be auto generated• Assists archiving• Help to prioritise manual archiving• Can be applied to any visual media

computer vision and media group: selected previous work

Documents

pantilt residual

zoom residual

average line vector

d computer models

d computer modelor

pantilt angle

zoom centre

long lines