query interfaces for a 3d model search engine
DESCRIPTION
Query Interfaces for a 3D Model Search Engine. Patrick Min Tom Funkhouser, Misha Kazhdan, Joyce Chen, Alex Halderman, David Dobkin, David Jacobs DLI2 meeting, Edinburgh, 25 June 2002. Online Data. On-line multimedia data is changing the way we get and use information. - PowerPoint PPT PresentationTRANSCRIPT
Query Interfaces forQuery Interfaces fora 3D Model Search Enginea 3D Model Search EngineQuery Interfaces forQuery Interfaces fora 3D Model Search Enginea 3D Model Search Engine
Patrick Min
Tom Funkhouser, Misha Kazhdan, Joyce Chen, Alex Halderman, David Dobkin, David
Jacobs
DLI2 meeting, Edinburgh, 25 June 2002
Patrick Min
Tom Funkhouser, Misha Kazhdan, Joyce Chen, Alex Halderman, David Dobkin, David
Jacobs
DLI2 meeting, Edinburgh, 25 June 2002
Online DataOnline DataOnline DataOnline Data
On-line multimedia data is changing the way we get and use information
Registered Saddlebred out of Famous Sultan Supreme line. 100% sound. 16 year old, flashy, chestnut w/white, loving, high energy horse, needs experienced rider. Was shown professionally in early years as gaited saddlebred. Most recently shown and always placed in training and first level dressage shows.Currently used as dressage/pleasure horse, jumps, loves trailriding. www.dreamhorse.com
2D ImageText Audio
What about 3D Models?What about 3D Models?
3D Models3D Models3D Models3D Models
are becoming easier to create, use and distribute
3D Models3D Models3D Models3D Models
are becoming easier to create, use and distribute• Laser range scanners: supply
Cyberware Scanner
3D Models3D Models3D Models3D Models
are becoming easier to create, use and distribute• Laser range scanners: supply• Fast graphics cards: demand• World Wide Web: distribution
Protein Data BankCyberware Scanner Large Model Archive
Georgia Tech
Protein Data BankCyberware Scanner Large Model Archive
Georgia Tech
3D Models3D Models3D Models3D Models
are becoming easier to create, use and distribute• Laser range scanners: supply• Fast graphics cards: demand• World Wide Web: distribution
One day 3D models will be as ubiquitous as images are now
are becoming easier to create, use and distribute• Laser range scanners: supply• Fast graphics cards: demand• World Wide Web: distribution
One day 3D models will be as ubiquitous as images are now
Research ChallengesResearch ChallengesResearch ChallengesResearch Challenges
Current research: • How do we construct 3D models ?
Digital Michelangelo ProjectStanford Graphics Laboratory
Research ChallengesResearch ChallengesResearch ChallengesResearch Challenges
Future research: • How do we find 3D models ?• How do we understand them ?
Digital Michelangelo ProjectStanford Graphics Laboratory
Current research: • How do we construct 3D models ?
Finding 3D ModelsFinding 3D ModelsFinding 3D ModelsFinding 3D Models
car
Project GoalProject GoalProject GoalProject Goal
A Search Engine for 3D Models on the Web
car
3D Model Search Engine3D Model Search Engine3D Model Search Engine3D Model Search Engine
Crawler Repository of 3D Models
World Wide Web
CrawlerCrawlerCrawlerIndexer
TextMatcher
3D Index
2D Index
Text Index
2DMatcher
3DMatcher
User QueryInterface
Query
Matches
On-line
Off-line
3D Model Search Engine3D Model Search Engine3D Model Search Engine3D Model Search Engine
Crawler Repository of 3D Models
World Wide Web
CrawlerCrawlerCrawlerIndexer
TextMatcher
3D Index
2D Index
Text Index
2DMatcher
3DMatcher
User QueryInterface
Query
On-line
Off-line
Acquisition
Matches
3D Model Search Engine3D Model Search Engine3D Model Search Engine3D Model Search Engine
Crawler Repository of 3D Models
World Wide Web
CrawlerCrawlerCrawlerIndexer
TextMatcher
3D Index
2D Index
Text Index
2DMatcher
3DMatcher
User QueryInterface
Query
On-line
Off-line
Analysis
Matches
3D Model Search Engine3D Model Search Engine3D Model Search Engine3D Model Search Engine
Crawler Repository of 3D Models
World Wide Web
CrawlerCrawlerCrawlerIndexer
TextMatcher
3D Index
2D Index
Text Index
2DMatcher
3DMatcher
User QueryInterface
Query
On-line
Off-line
Matching
Matches
3D Model Search Engine3D Model Search Engine3D Model Search Engine3D Model Search Engine
Crawler Repository of 3D Models
World Wide Web
CrawlerCrawlerCrawlerIndexer
TextMatcher
3D Index
2D Index
Text Index
2DMatcher
3DMatcher
User QueryInterface
Query
On-line
Off-line
Querying
Matches
3D Model Search Engine3D Model Search Engine3D Model Search Engine3D Model Search Engine
Crawler Repository of 3D Models
World Wide Web
CrawlerCrawlerCrawlerIndexer
TextMatcher
3D Index
2D Index
Text Index
2DMatcher
3DMatcher
User QueryInterface
Query
On-line
Off-line
Querying
Matches
Analysis
Matching
Main Challenge & today's topicMain Challenge & today's topicMain Challenge & today's topicMain Challenge & today's topic
Which Query Interfaces are Useful
for a 3D Model Search Engine?
User QueryInterface
Query
Matches
Desired Properties of Query InterfacesDesired Properties of Query InterfacesDesired Properties of Query InterfacesDesired Properties of Query Interfaces
easy to use• also for novice users
possible to quickly specify query efficient matching method effective
• high number of relevant results
Available Data for QueriesAvailable Data for QueriesAvailable Data for QueriesAvailable Data for Queries
http://www.3dspline.com
Example of a 3D Model available on the Web:
Text
Appearance Shape
Available Data for QueriesAvailable Data for QueriesAvailable Data for QueriesAvailable Data for Queries
Text
Appearance Shape
Available Data for QueriesAvailable Data for QueriesAvailable Data for QueriesAvailable Data for Queries
Text QueriesText QueriesText QueriesText Queries
Text QueriesText QueriesText QueriesText Queries
+ easy to use
+ possible to quickly specify query
+ efficient matching method
+/- effective
Effective Text QueriesEffective Text QueriesEffective Text QueriesEffective Text Queries
747
caffeine
dog fork
dna
Text Query ProblemsText Query ProblemsText Query ProblemsText Query Problems
objects are badly annotated:• ferrar03.3ds, ferrair, rahmen, …
keywords are too common• face, plane
Text Query ProblemsText Query ProblemsText Query ProblemsText Query Problems
objects are badly annotated:• ferrar03.3ds, ferrair, rahmen, …
keywords are too common• face, plane
keywords are unknown to user, or annotator not possible to anticipate every relevant text
query• elf, fairy-tale creature, puppet, person, ...
Text Query ProblemsText Query ProblemsText Query ProblemsText Query Problems
We hypothesize shape can help
objects are badly annotated:• ferrar03.3ds, ferrair, rahmen, …
keywords are too common• face, plane
keywords are unknown to user, or annotator not possible to anticipate every relevant text
query• elf, fairy-tale creature, puppet, person, ...
OutlineOutlineOutlineOutline
Introduction Text Queries 3D Shape Queries 2D Shape Queries Results Conclusions Future Work
3D Shape Queries3D Shape Queries3D Shape Queries3D Shape Queries
use 3D shape to find 3D shape
3D Model Queries3D Model Queries3D Model Queries3D Model Queries
(create &) upload 3D model
select 3D model on results page
("shape iteration")
3D Matching Method3D Matching Method3D Matching Method3D Matching Method
3D Matching Method3D Matching Method3D Matching Method3D Matching Method
Shape Iteration User StudyShape Iteration User StudyShape Iteration User StudyShape Iteration User Study
Shape Iteration User StudyShape Iteration User StudyShape Iteration User StudyShape Iteration User Study
Test Design:• 18 students in an introductory CS class• input: on website, initial text keyword
search, then “find similar object”
• where similar means similar text or similar shape (unknown to user)
How does shape combine with text in interactive searches?
Shape Iteration ResultsShape Iteration ResultsShape Iteration ResultsShape Iteration Results
shape iteration finds target object more often
in fewer iterations and in less time
Similarity
3D Model Query Problems3D Model Query Problems3D Model Query Problems3D Model Query Problems
upload 3D model:• such a model may not be available• creating one with standard tools is hard
shape iteration:• have to get such a result first
3D Sketch Queries3D Sketch Queries3D Sketch Queries3D Sketch Queries
Sketch 3D model using simple modeling program• e.g. Teddy, Sketch
Teddy: A Sketching Interface for 3D Freeform DesignIgarashi et al., SIGGRAPH 1999
3D Sketch Queries3D Sketch Queries3D Sketch Queries3D Sketch Queries
Teddy: A Sketching Interface for 3D Freeform DesignIgarashi et al., SIGGRAPH 1999
Teddy Sample ResultsTeddy Sample ResultsTeddy Sample ResultsTeddy Sample Results
3D Sketch Query Problems3D Sketch Query Problems3D Sketch Query Problems3D Sketch Query Problems
Teddy: supports only coarse, blobby shapes of
genus 0 still hard to use for novice users
3D Sketch Query Properties3D Sketch Query Properties3D Sketch Query Properties3D Sketch Query Properties
- easy to use
- possible to quickly specify query
+ efficient matching method
+/- effective
So we need a simpler query interface
2D Sketch Queries2D Sketch Queries2D Sketch Queries2D Sketch Queries
2D Sketch Queries2D Sketch Queries2D Sketch Queries2D Sketch Queries
“paintbrush” style interface:• simple outline sketches• easy to use for most users
Multiple 2D SketchesMultiple 2D SketchesMultiple 2D SketchesMultiple 2D Sketches
2D Sketch Matching Method2D Sketch Matching Method2D Sketch Matching Method2D Sketch Matching Method
2
1
3
4 7
65
7 projections per database modelQuery sketch
2D Sketch Matching Method2D Sketch Matching Method2D Sketch Matching Method2D Sketch Matching Method
compute 2D shape descriptor in similar fashion
intersect with circles instead of spheres
2D Sketch User Study2D Sketch User Study2D Sketch User Study2D Sketch User Study
2D Sketch Interface User Study2D Sketch Interface User Study2D Sketch Interface User Study2D Sketch Interface User Study
Does shape add to text?
Test design:• 43 students in introductory CS class• goal: retrieve target objects• input: text keywords and sketches on
paper
Example InputExample InputExample InputExample Input
2D Sketch User Study Results2D Sketch User Study Results2D Sketch User Study Results2D Sketch User Study Results
Distinguishing keywords:• text works well• even better with sketch
Class is too large:• text is too inspecific• sketch works best
2D Sketch User Study Results2D Sketch User Study Results2D Sketch User Study Results2D Sketch User Study Results
text and shape can complement each other
sketch allows refinement within a class
2D Sketch User Study Results2D Sketch User Study Results2D Sketch User Study Results2D Sketch User Study Results
2D Sketch Query Problems2D Sketch Query Problems2D Sketch Query Problems2D Sketch Query Problems
The drawing is free-form:• hard to draw accurately• hard to get proportions right
Dog
DC10
2D Sketch Query Problems2D Sketch Query Problems2D Sketch Query Problems2D Sketch Query Problems
The drawing is free-form:• hard to draw accurately• hard to get proportions right
Image matching method:• user and database outline have to look
similar
Dog
DC10
Search Engine Usage ResultsSearch Engine Usage ResultsSearch Engine Usage ResultsSearch Engine Usage Results
Search Engine ResultsSearch Engine ResultsSearch Engine ResultsSearch Engine Results
online since early November 2001 20,707 models indexed, 2,873 commercial one typical week of usage:
• 4,522 queries • 1,346 unique hosts from 55 different
countries• 1,029 models downloaded
Search Engine ResultsSearch Engine ResultsSearch Engine ResultsSearch Engine Results
text
shape
empty
Text vs. Shape
about 70 % text, 30 % shape-based
Search Engine ResultsSearch Engine ResultsSearch Engine ResultsSearch Engine Results
2Dtext & 2D3Dtext & 3Dsimilar shapeupload model
Shape Query Type
about 58 % shape iteration, 38 % 2D/text & 2D
Number of Hosts per DayNumber of Hosts per DayNumber of Hosts per DayNumber of Hosts per Day
jan aprmarfeb may
average about 175 hosts and 525 searches/day
% New Hosts per Day% New Hosts per Day% New Hosts per Day% New Hosts per Day
jan aprmarfeb may
about 20 % returning hosts per day
# Searches per User# Searches per User# Searches per User# Searches per User
1 3 5 7 9 11 13 15 17 19 21 23 25
1000
2000
3000
4000
5000
6000
7000
about 40 % does 3 or more searches per day
# searches
# hosts
Conclusions and Future WorkConclusions and Future WorkConclusions and Future WorkConclusions and Future Work
ConclusionsConclusionsConclusionsConclusions
shape-based search can help each interface has its strengths and
weaknesses• so useful to combine them
table
find similar shape
Future WorkFuture WorkFuture WorkFuture Work
2D Structural Interface primitive drawing user study evaluation of several interfaces
• oval + rectangle• skeletal graphs
3D Structural Interface
Future Work IIFuture Work IIFuture Work IIFuture Work II
3D Model Database
fully automated acquisition and model processing
support for more model formats
AcknowledgementsAcknowledgementsAcknowledgementsAcknowledgements
Funding NSF (CCR-0093343, 11S-0121446, CCR-99-88173, DGE-
9972930) Army Research Organization (DAAD19-99-1-0205)
Alfred P. Sloan Fellowship (for Prof. Tom Funkhouser)People Brian Kernighan and the students of CS109 and CS111 Bernard Chazelle, Adam Finkelstein,
Szymon Rusinkiewicz, Emil Praun Princeton CS department technical staff Princeton Shape Analysis Group3D Models Viewpoint Datalabs, http://www.viewpoint.com De Espona Infografica, http://www.deespona.com
http://shape.cs.princeton.edu
3D Model Search Engine3D Model Search Engine3D Model Search Engine3D Model Search Engine
Test DatabaseTest DatabaseTest DatabaseTest Database
50 models, 10 categories of 5 models each