query interfaces for a 3d model search engine

Query Interfaces forQuery Interfaces fora 3D Model Search Enginea 3D Model Search EngineQuery Interfaces forQuery Interfaces fora 3D Model Search Enginea 3D Model Search Engine

Patrick Min

Tom Funkhouser, Misha Kazhdan, Joyce Chen, Alex Halderman, David Dobkin, David

Jacobs

DLI2 meeting, Edinburgh, 25 June 2002

Patrick Min

Tom Funkhouser, Misha Kazhdan, Joyce Chen, Alex Halderman, David Dobkin, David

Jacobs

DLI2 meeting, Edinburgh, 25 June 2002

Online DataOnline DataOnline DataOnline Data

On-line multimedia data is changing the way we get and use information

Registered Saddlebred out of Famous Sultan Supreme line. 100% sound. 16 year old, flashy, chestnut w/white, loving, high energy horse, needs experienced rider. Was shown professionally in early years as gaited saddlebred. Most recently shown and always placed in training and first level dressage shows.Currently used as dressage/pleasure horse, jumps, loves trailriding. www.dreamhorse.com

2D ImageText Audio

What about 3D Models?What about 3D Models?

3D Models3D Models3D Models3D Models

are becoming easier to create, use and distribute

are becoming easier to create, use and distribute• Laser range scanners: supply

Cyberware Scanner

are becoming easier to create, use and distribute• Laser range scanners: supply• Fast graphics cards: demand• World Wide Web: distribution

Protein Data BankCyberware Scanner Large Model Archive

Georgia Tech

Protein Data BankCyberware Scanner Large Model Archive

Georgia Tech

One day 3D models will be as ubiquitous as images are now

Research ChallengesResearch ChallengesResearch ChallengesResearch Challenges

Current research: • How do we construct 3D models ?

Digital Michelangelo ProjectStanford Graphics Laboratory

Research ChallengesResearch ChallengesResearch ChallengesResearch Challenges

Future research: • How do we find 3D models ?• How do we understand them ?

Digital Michelangelo ProjectStanford Graphics Laboratory

Current research: • How do we construct 3D models ?

Finding 3D ModelsFinding 3D ModelsFinding 3D ModelsFinding 3D Models

Project GoalProject GoalProject GoalProject Goal

A Search Engine for 3D Models on the Web

3D Model Search Engine3D Model Search Engine3D Model Search Engine3D Model Search Engine

Crawler Repository of 3D Models

World Wide Web

CrawlerCrawlerCrawlerIndexer

TextMatcher

3D Index

2D Index

Text Index

2DMatcher

3DMatcher

User QueryInterface

Matches

On-line

Off-line

World Wide Web

TextMatcher

3D Index

2D Index

Text Index

2DMatcher

3DMatcher

User QueryInterface

On-line

Off-line

Acquisition

Matches

World Wide Web

TextMatcher

3D Index

2D Index

Text Index

2DMatcher

3DMatcher

User QueryInterface

On-line

Off-line

Analysis

Matches

World Wide Web

TextMatcher

3D Index

2D Index

Text Index

2DMatcher

3DMatcher

User QueryInterface

On-line

Off-line

Matching

Matches

World Wide Web

TextMatcher

3D Index

2D Index

Text Index

2DMatcher

3DMatcher

User QueryInterface

On-line

Off-line

Querying

Matches

World Wide Web

TextMatcher

3D Index

2D Index

Text Index

2DMatcher

3DMatcher

User QueryInterface

On-line

Off-line

Querying

Matches

Analysis

Matching

Main Challenge & today's topicMain Challenge & today's topicMain Challenge & today's topicMain Challenge & today's topic

Which Query Interfaces are Useful

for a 3D Model Search Engine?

User QueryInterface

Matches

Desired Properties of Query InterfacesDesired Properties of Query InterfacesDesired Properties of Query InterfacesDesired Properties of Query Interfaces

easy to use• also for novice users

possible to quickly specify query efficient matching method effective

• high number of relevant results

Available Data for QueriesAvailable Data for QueriesAvailable Data for QueriesAvailable Data for Queries

http://www.3dspline.com

Example of a 3D Model available on the Web:

Appearance Shape

Text QueriesText QueriesText QueriesText Queries

+ easy to use

+ possible to quickly specify query

+ efficient matching method

+/- effective

Effective Text QueriesEffective Text QueriesEffective Text QueriesEffective Text Queries

caffeine

dog fork

Text Query ProblemsText Query ProblemsText Query ProblemsText Query Problems

objects are badly annotated:• ferrar03.3ds, ferrair, rahmen, …

keywords are too common• face, plane

keywords are unknown to user, or annotator not possible to anticipate every relevant text

query• elf, fairy-tale creature, puppet, person, ...

We hypothesize shape can help

keywords are unknown to user, or annotator not possible to anticipate every relevant text

query• elf, fairy-tale creature, puppet, person, ...

OutlineOutlineOutlineOutline

Introduction Text Queries 3D Shape Queries 2D Shape Queries Results Conclusions Future Work

3D Shape Queries3D Shape Queries3D Shape Queries3D Shape Queries

use 3D shape to find 3D shape

3D Model Queries3D Model Queries3D Model Queries3D Model Queries

(create &) upload 3D model

select 3D model on results page

("shape iteration")

3D Matching Method3D Matching Method3D Matching Method3D Matching Method

Shape Iteration User StudyShape Iteration User StudyShape Iteration User StudyShape Iteration User Study

Test Design:• 18 students in an introductory CS class• input: on website, initial text keyword

search, then “find similar object”

• where similar means similar text or similar shape (unknown to user)

How does shape combine with text in interactive searches?

Shape Iteration ResultsShape Iteration ResultsShape Iteration ResultsShape Iteration Results

shape iteration finds target object more often

in fewer iterations and in less time

Similarity

3D Model Query Problems3D Model Query Problems3D Model Query Problems3D Model Query Problems

upload 3D model:• such a model may not be available• creating one with standard tools is hard

shape iteration:• have to get such a result first

3D Sketch Queries3D Sketch Queries3D Sketch Queries3D Sketch Queries

Sketch 3D model using simple modeling program• e.g. Teddy, Sketch

Teddy: A Sketching Interface for 3D Freeform DesignIgarashi et al., SIGGRAPH 1999

Teddy Sample ResultsTeddy Sample ResultsTeddy Sample ResultsTeddy Sample Results

3D Sketch Query Problems3D Sketch Query Problems3D Sketch Query Problems3D Sketch Query Problems

Teddy: supports only coarse, blobby shapes of

genus 0 still hard to use for novice users

3D Sketch Query Properties3D Sketch Query Properties3D Sketch Query Properties3D Sketch Query Properties

- easy to use

- possible to quickly specify query

+ efficient matching method

+/- effective

So we need a simpler query interface

“paintbrush” style interface:• simple outline sketches• easy to use for most users

Multiple 2D SketchesMultiple 2D SketchesMultiple 2D SketchesMultiple 2D Sketches

2D Sketch Matching Method2D Sketch Matching Method2D Sketch Matching Method2D Sketch Matching Method

7 projections per database modelQuery sketch

2D Sketch Matching Method2D Sketch Matching Method2D Sketch Matching Method2D Sketch Matching Method

compute 2D shape descriptor in similar fashion

intersect with circles instead of spheres

2D Sketch User Study2D Sketch User Study2D Sketch User Study2D Sketch User Study

2D Sketch Interface User Study2D Sketch Interface User Study2D Sketch Interface User Study2D Sketch Interface User Study

Does shape add to text?

Test design:• 43 students in introductory CS class• goal: retrieve target objects• input: text keywords and sketches on

Example InputExample InputExample InputExample Input

2D Sketch User Study Results2D Sketch User Study Results2D Sketch User Study Results2D Sketch User Study Results

Distinguishing keywords:• text works well• even better with sketch

Class is too large:• text is too inspecific• sketch works best

text and shape can complement each other

sketch allows refinement within a class

The drawing is free-form:• hard to draw accurately• hard to get proportions right

Image matching method:• user and database outline have to look

similar

Search Engine Usage ResultsSearch Engine Usage ResultsSearch Engine Usage ResultsSearch Engine Usage Results

Search Engine ResultsSearch Engine ResultsSearch Engine ResultsSearch Engine Results

online since early November 2001 20,707 models indexed, 2,873 commercial one typical week of usage:

• 4,522 queries • 1,346 unique hosts from 55 different

countries• 1,029 models downloaded

Text vs. Shape

about 70 % text, 30 % shape-based

2Dtext & 2D3Dtext & 3Dsimilar shapeupload model

Shape Query Type

about 58 % shape iteration, 38 % 2D/text & 2D

Number of Hosts per DayNumber of Hosts per DayNumber of Hosts per DayNumber of Hosts per Day

jan aprmarfeb may

average about 175 hosts and 525 searches/day

% New Hosts per Day% New Hosts per Day% New Hosts per Day% New Hosts per Day

jan aprmarfeb may

about 20 % returning hosts per day

# Searches per User# Searches per User# Searches per User# Searches per User

1 3 5 7 9 11 13 15 17 19 21 23 25

about 40 % does 3 or more searches per day

# searches

# hosts

Conclusions and Future WorkConclusions and Future WorkConclusions and Future WorkConclusions and Future Work

ConclusionsConclusionsConclusionsConclusions

shape-based search can help each interface has its strengths and

weaknesses• so useful to combine them

find similar shape

Future WorkFuture WorkFuture WorkFuture Work

2D Structural Interface primitive drawing user study evaluation of several interfaces

• oval + rectangle• skeletal graphs

3D Structural Interface

Future Work IIFuture Work IIFuture Work IIFuture Work II

3D Model Database

fully automated acquisition and model processing

support for more model formats

AcknowledgementsAcknowledgementsAcknowledgementsAcknowledgements

Funding NSF (CCR-0093343, 11S-0121446, CCR-99-88173, DGE-

9972930) Army Research Organization (DAAD19-99-1-0205)

Alfred P. Sloan Fellowship (for Prof. Tom Funkhouser)People Brian Kernighan and the students of CS109 and CS111 Bernard Chazelle, Adam Finkelstein,

Szymon Rusinkiewicz, Emil Praun Princeton CS department technical staff Princeton Shape Analysis Group3D Models Viewpoint Datalabs, http://www.viewpoint.com De Espona Infografica, http://www.deespona.com

http://shape.cs.princeton.edu

Test DatabaseTest DatabaseTest DatabaseTest Database

50 models, 10 categories of 5 models each

query interfaces for a 3d model search engine

d model available

demandworld wide web

distributelaser range

distribution3d modelsare

supply3d modelsare

distribute3d modelsare

supplyfast graphics

query interfaces fora

Documents

comp 4010: lecture 4 - 3d user interfaces for vr

poster: realdance: an exploration of 3d spatial interfaces...

merging source query interfaces on web databases

query operational data is collected by probes,...

3d interfaces for mobile phones_public_20080529

flow web: a graph based user interface for 3d flow field...

from single image query to detailed 3d reconstruction ·...

ivoa registry interfaces version 1 - arxiv.org · ivoa...

games and 3d user interfaces:games and 3d user interfaces

an interactive clustering-based approach to integrating...

graphical query interfaces for semistructured data: the...

3d user interfaces for information systems based on usixml

comp 4010 lecture7 3d user interfaces for virtual reality

rising trends in 3d user interfaces trends in 3d user...

3d sub-query expansion for improving sketch-based multi...

demonstration of user interfaces for querying in 3d

a hierarchical approach to model web query interfaces for...

new directions in 3d user interfaces -...

three-dimensional cluster analysis identifies interfaces...

ranked queries over sources with boolean query interfaces ...