research area b leif kobbelt. communication system interface research area b 2
TRANSCRIPT
![Page 1: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/1.jpg)
Research Area B
Leif Kobbelt
![Page 2: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/2.jpg)
Communication
System
Interface
Research Area B
2
![Page 3: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/3.jpg)
Research Area B
3
Interface
System
Communication
A
C
D
![Page 4: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/4.jpg)
• definition of „application“ within UMIC• find new and relevant applications (killer app)
• combine existing technology
• identify application profiles
• types of data
• amount of data
• latency requirements
• input / output devices
• golden demo• good : convey the message
• bad : no basis research, commercial competitors
4
Research Area B
![Page 5: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/5.jpg)
Future Mobile Applications
5
fund
amen
tal
algor
ithm
s syste
m
desig
n
evalu
ation
com
mer
cializ
ation
![Page 6: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/6.jpg)
Future Mobile Applications
6
fundam
enta
l
algorit
hms
syst
em
desig
n
eval
uatio
n
com
mer
cializ
ation
![Page 7: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/7.jpg)
Future Mobile Applications
7
fundam
enta
l
algorit
hms
syst
em
desig
n
eval
uatio
n
com
mer
cializ
ation
communication
it security
computer graphics
computer vision
![Page 8: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/8.jpg)
Future Mobile Applications
8
fundam
enta
l
algorit
hms
syst
em
desig
n
eval
uatio
n
com
mer
cializ
ation
software engineering
security
computer graphics & vision
interface design
![Page 9: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/9.jpg)
Future Mobile Applications
9
fundam
enta
l
algorit
hms
syst
em
desig
n
eval
uatio
n
com
mer
cializ
ation
graphical UIs
prototypes
user studies
...
![Page 10: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/10.jpg)
Visual Computing for Future Mobile Applications
Bastian Leibe
![Page 11: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/11.jpg)
Target Scenario: Pedestrian Navigation
11
Aachen Cathedral
Mobile visual search Simply point the camera to any object/building of interest. Images are transmitted to a central server for recognition.
![Page 12: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/12.jpg)
Target Scenario: Pedestrian Navigation
12
Aachen Cathedral
Mobile visual search Simply point the camera to any object/building of interest. Images are transmitted to a central server for recognition. Object-specific content is sent back to for visualization on the
mobile phone (mobile AR).
![Page 13: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/13.jpg)
LocalizeMe Demo
13
P. Steingrube, T. Weyand, T. Sattler, A. Schmitz, B. Leibe, L. Kobbelt
![Page 14: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/14.jpg)
Mobile Service Structure
14
Localization Service
Image Database
3D Model
Render Server
Information Service
Cultural Database
Internet
Compound
Application
MobileClient
ServerServerServer
User Interface
![Page 15: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/15.jpg)
Localization: Large-Scale Image Matching
How can we perform this matching step efficiently?
15
Database with thousands (millions) of images
??
Mobile photo
![Page 16: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/16.jpg)
Localization: Large-Scale Image Matching
16
Database with thousands (millions) of images
Mobile photo
Local features(~1000 per image) …
“Visual vocabulary”(~1M feature clusters)
Shortlist of candidate matches(~100 images)
Shortlist of candidate matches(~100 images)
Matching (nearest neighbors in 128D space)
![Page 17: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/17.jpg)
Localization: Geometric Verification
17
Mobile photo (for each image in shortlist)
xAj
xBj
XjAssumption: corresponding 3D structure
T. Sattler, B. Leibe, L. Kobbelt, SCRAMSAC: Improving RANSAC’s Efficiency with a Spatial Consistency Filter. International Conference on Computer Vision, 2009.
Find a rigid geometric transformation to verify that the matched features correspond to the same 3D structure. Problem: efficient processing with many outliers.
![Page 18: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/18.jpg)
Summary: Visualization Service
18
Image
Local Feature
Extractor
Local Feature
Database
Image Database
Feature Matching
ImageImageCandidate Matches
Geometric Verificatio
n
DetermineLocation
On mobile device
On server side
![Page 19: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/19.jpg)
Mobile Service Structure
19
Localization Service
Image Database
3D Model
Render Server
Information Service
Cultural Database
Internet
Compound
Application
MobileClient
ServerServerServer
User Interface
![Page 20: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/20.jpg)
World-Scale Mining for Content Creation
20
e.g. Wikipedia match
Mining geotagged images Extracted Image clusters
Automaticannotation
& verification
Frequent tags
![Page 21: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/21.jpg)
Example: Automatic Landmark Detection
21
Matched images for Aachen city hall (subset)
T. Weyand, B. Leibe
![Page 22: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/22.jpg)
How Does This Scale?
Feasibility study Pairwise matching on 500,000 geotagged images of Paris How many matching images can we find at a certain location?
Touristic sites and central roads are well-covered.
22
T. Weyand, B. Leibe
![Page 23: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/23.jpg)
Mobile Service Structure
23
Localization Service
Image Database
3D Model
Render Server
Information Service
Cultural Database
Internet
Compound
Application
MobileClient
ServerServerServer
User Interface
![Page 24: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/24.jpg)
Virtual City Model
24
Floor Plan Map (2D)
Street Graph (2D)
Height Field(3D)
Synthetic Textures
Photographic
Textures
Building Model
Landscape Model
Optimized Octree Data
Structure
Estate Plan(2D)
![Page 25: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/25.jpg)
Results: Virtual Aachen Model
25
G. Fabritius, J. Kraßnigg, L. Krecklau, C. Manthei, A. Hornung, M. Habbecke, L. Kobbelt
![Page 26: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/26.jpg)
Alternative 1: Mobile Rendering
26
Rendering quality ona regular PC
C. Schreder, A. Schmitz, L. Kobbelt
Mobile rendering Very limited memory! Need to precompute
octree structure Dynamic transmission
of geometry & textures
![Page 27: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/27.jpg)
Alternative 2: Stream Rendering
27
Rendering on server
Internet
Current prototype Transmit images in
UDP packages Tradeoff: compression
vs. framerate
![Page 28: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/28.jpg)
Mobile Service Structure
28
Localization Service
Image Database
3D Model
Render Server
Information Service
Cultural Database
Internet
Compound
Application
MobileClient
ServerServerServer
User Interface
![Page 29: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/29.jpg)
Application Study: Looking Through Time
How did Aachen’s cathedral look in past centuries? Camera phone as a “magic lens” to reveal past building states How should the interface be designed for such an application? Mock-up prototype (using Motion Capture system for tracking) Evaluation in user studies
29
T. Palm, J. Borchers
![Page 30: Research Area B Leif Kobbelt. Communication System Interface Research Area B 2](https://reader036.vdocuments.us/reader036/viewer/2022070412/56649ddb5503460f94ad1868/html5/thumbnails/30.jpg)
Research in Novel Interaction Techniques
30
T. Karrer, M. Weiss, M. Wittenhagen, G. Herkenrath, J. Borchers
TWEND Twisting and bending as new
interaction gestures in futuremobile devices
E.g. for interaction with anelectronic map or an e-book.
PocketDRAGON Directly drag objects along their
movement trajectory to preciselynavigate to a specific point in avideo sequence
Mobile implementation madepossible through communicationwith server