Upload File

learning 3d geometry - cs.cornell.edu

98
ⓘ

Upload: others

Post on 13-May-2022

4 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Learning 3D geometry - cs.cornell.edu

ⓘ

Page 2: Learning 3D geometry - cs.cornell.edu
Page 3: Learning 3D geometry - cs.cornell.edu

•

–

•

•

•

•

https://apps.engineering.cornell.edu/CourseEval/
Page 4: Learning 3D geometry - cs.cornell.edu

Third Place

Page 5: Learning 3D geometry - cs.cornell.edu
Page 6: Learning 3D geometry - cs.cornell.edu

Second Place

Page 7: Learning 3D geometry - cs.cornell.edu
Page 8: Learning 3D geometry - cs.cornell.edu

First Place

Page 9: Learning 3D geometry - cs.cornell.edu
Page 10: Learning 3D geometry - cs.cornell.edu
Page 11: Learning 3D geometry - cs.cornell.edu
Page 12: Learning 3D geometry - cs.cornell.edu
Page 13: Learning 3D geometry - cs.cornell.edu
http://go/im2depth
Page 14: Learning 3D geometry - cs.cornell.edu
Page 15: Learning 3D geometry - cs.cornell.edu

●

https://diode-dataset.org/
Page 16: Learning 3D geometry - cs.cornell.edu

•

•

•

Page 17: Learning 3D geometry - cs.cornell.edu
Page 18: Learning 3D geometry - cs.cornell.edu
Page 19: Learning 3D geometry - cs.cornell.edu

●

Page 20: Learning 3D geometry - cs.cornell.edu
Page 21: Learning 3D geometry - cs.cornell.edu

[Snavely, Seitz, Szeliski. Photo Tourism. SIGGRAPH 2006]

Page 22: Learning 3D geometry - cs.cornell.edu
Page 23: Learning 3D geometry - cs.cornell.edu
Page 24: Learning 3D geometry - cs.cornell.edu

●

●

Page 25: Learning 3D geometry - cs.cornell.edu

Input Ground truth Predicted depth

Page 26: Learning 3D geometry - cs.cornell.edu
Page 27: Learning 3D geometry - cs.cornell.edu
Page 28: Learning 3D geometry - cs.cornell.edu
Page 29: Learning 3D geometry - cs.cornell.edu
Page 30: Learning 3D geometry - cs.cornell.edu

●

●

●

Page 31: Learning 3D geometry - cs.cornell.edu

●

●

https://shihmengli.github.io/3D-Photo-Inpainting/
Page 32: Learning 3D geometry - cs.cornell.edu
Page 33: Learning 3D geometry - cs.cornell.edu

Stereo Photography

Page 34: Learning 3D geometry - cs.cornell.edu

Stereo Photography

Viewing Devices

Page 35: Learning 3D geometry - cs.cornell.edu

Queen Victoria at World Fair, 1851

Stereo Photography

Page 36: Learning 3D geometry - cs.cornell.edu

Stereo Photography

Page 37: Learning 3D geometry - cs.cornell.edu

Issue: Narrow Baseline

~1.5 cm~6.5 cm

Page 38: Learning 3D geometry - cs.cornell.edu

Left

Page 39: Learning 3D geometry - cs.cornell.edu

Right

Page 40: Learning 3D geometry - cs.cornell.edu
Page 41: Learning 3D geometry - cs.cornell.edu

Problem Statement

View Synthesis

model

…

Output

…

Output Input

Page 42: Learning 3D geometry - cs.cornell.edu

Challenges

……InputOutput Output

Extrapolation

Large disocclusion

Non-Lambertian Effects

Reflections, transparencies, etc.

Page 43: Learning 3D geometry - cs.cornell.edu

Prior Methods: No Shared Scene Representation

Input views

Neural Net

Output views

[Flynn et al., 2015]

[Kalantari et al. 2016]

Page 44: Learning 3D geometry - cs.cornell.edu

Input views

Neural Net

Output views

Prior Methods: No Shared Scene Representation

[Flynn et al., 2015]

[Kalantari et al. 2016]

Page 45: Learning 3D geometry - cs.cornell.edu

Input views

Neural Net

Output views

…

Rendered

independently

Prior Methods: No Shared Scene Representation

[Flynn et al., 2015]

[Kalantari et al. 2016]

Page 46: Learning 3D geometry - cs.cornell.edu

Input views

Scene

Representation

Ours: Shared Scene Representation

Neural Net

Output views

…

Page 47: Learning 3D geometry - cs.cornell.edu

Stereo Magnification: Learning View Synthesis using Multiplane Images

Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, Noah Snavely

SIGGRAPH 2018

Page 48: Learning 3D geometry - cs.cornell.edu

Multiplane Camera (1937)

Image credits: Disney https://www.youtube.com/watch?v=kN-eCBAOw60 (from 1957)

https://www.youtube.com/watch?v=kN-eCBAOw60
Page 49: Learning 3D geometry - cs.cornell.edu

Multiplane Images (MPIs)

Reference

Viewpoint

Each plane is at a fixed

depth and encoded by

an RGBA image

Page 50: Learning 3D geometry - cs.cornell.edu

View Synthesis using Multiplane Images

Reference

Viewpoint

Homography

Target

Viewpoint

Over

Page 51: Learning 3D geometry - cs.cornell.edu

View Synthesis using Multiplane Images

Reference

Viewpoint

Homography

Target

Viewpoint

Over

Synthesized image

Page 52: Learning 3D geometry - cs.cornell.edu
Page 53: Learning 3D geometry - cs.cornell.edu
Page 54: Learning 3D geometry - cs.cornell.edu

• Models disocclusion

• Models soft edges and

non-Lambertian effects

• Efficient for view synthesis

• Differentiable rendering

Properties of Multiplane Images

Page 55: Learning 3D geometry - cs.cornell.edu

Learning Multiplane Images

Input views

Multiplane Image

Alpha

RGB

Neural net

Page 56: Learning 3D geometry - cs.cornell.edu

Learning Multiplane Images

Input views

Rendered views

…

Ground-truthMultiplane Image

Alpha

RGB

Neural net

Page 57: Learning 3D geometry - cs.cornell.edu

Training Data

…

Input views Target view

( )

( )

( )

,

,

,

Need massive set of triplets with known

camera poses

Page 58: Learning 3D geometry - cs.cornell.edu
Page 59: Learning 3D geometry - cs.cornell.edu
Page 60: Learning 3D geometry - cs.cornell.edu
https://google.github.io/realestate10k/
Page 61: Learning 3D geometry - cs.cornell.edu

Sampling Training Examples

… …

Input TargetInput(Extrapolated)

Page 62: Learning 3D geometry - cs.cornell.edu

Sampling Training Examples

… …

InputTargetInput(Interpolated)

Page 63: Learning 3D geometry - cs.cornell.edu

Results

Page 64: Learning 3D geometry - cs.cornell.edu

Left

Page 65: Learning 3D geometry - cs.cornell.edu

Right

Page 66: Learning 3D geometry - cs.cornell.edu

Output

Page 67: Learning 3D geometry - cs.cornell.edu

Image 1

Page 68: Learning 3D geometry - cs.cornell.edu

Image 2

Page 69: Learning 3D geometry - cs.cornell.edu

Output

Page 70: Learning 3D geometry - cs.cornell.edu

Reference input view

Plane 0

Plane 13

Plane 9

Plane 16

Plane 24 Plane 26

Page 71: Learning 3D geometry - cs.cornell.edu
Page 72: Learning 3D geometry - cs.cornell.edu

1.4 cm 6.3 cm

Extrapolating Cellphone Footage

Page 73: Learning 3D geometry - cs.cornell.edu
Page 74: Learning 3D geometry - cs.cornell.edu
Page 75: Learning 3D geometry - cs.cornell.edu

●

●

●

○

○

Page 76: Learning 3D geometry - cs.cornell.edu
Page 77: Learning 3D geometry - cs.cornell.edu
Page 78: Learning 3D geometry - cs.cornell.edu
Page 79: Learning 3D geometry - cs.cornell.edu
Page 80: Learning 3D geometry - cs.cornell.edu
https://www.matthewtancik.com/nerf
Page 81: Learning 3D geometry - cs.cornell.edu
Page 82: Learning 3D geometry - cs.cornell.edu
Page 83: Learning 3D geometry - cs.cornell.edu
https://nex-mpi.github.io/
Page 84: Learning 3D geometry - cs.cornell.edu
https://nerf-w.github.io/
Page 85: Learning 3D geometry - cs.cornell.edu
Page 86: Learning 3D geometry - cs.cornell.edu

●

●

●

Page 87: Learning 3D geometry - cs.cornell.edu
https://www.balletforadults.com/back-to-basics-the-five-positions-of-the-arms/
Page 88: Learning 3D geometry - cs.cornell.edu
Page 89: Learning 3D geometry - cs.cornell.edu

●

●

●

Page 90: Learning 3D geometry - cs.cornell.edu
Page 91: Learning 3D geometry - cs.cornell.edu
Page 92: Learning 3D geometry - cs.cornell.edu
Page 93: Learning 3D geometry - cs.cornell.edu
Page 94: Learning 3D geometry - cs.cornell.edu
Page 95: Learning 3D geometry - cs.cornell.edu

●

●

●

○

Page 96: Learning 3D geometry - cs.cornell.edu

●

●

●

○

Page 97: Learning 3D geometry - cs.cornell.edu
Page 98: Learning 3D geometry - cs.cornell.edu

3D Coordinate Geometry

Geometry Processing Pipeline 3D Scanning, Smoothing…., 3D ...€¦ · 2014.09.08 1 FU Berlin –AG Geom Sunil Yadav ABV Seminar 17.08.2017 Geometry Processing Pipeline 3D Scanning,

CS664 Computer Vision 9. Camera Geometry - cs.cornell.edu · 17 Fundamental and Essential Matrix Linear algebra formulation of the epipolar geometry Fundamental matrix, F, maps point

Classes - cs.cornell.edu

7.60 3D Geometry Review.notebook

Pyramids - cs.cornell.edu

3D Turtle Geometry: Artwork, Theory, Program Equivalence ...wstomv/publications/3d... · 3D Turtle Geometry: Artwork, Theory, Program Equivalence and Symmetry Tom Verhoe Faculty of

3D Modeling 3D Geometry, Topology and Semantics · 3D Modeling 3D Geometry, Topology and Semantics . Christian Clemen ... A single slide on IFC: information exchange in general !!!

DM Chapter 4 3D Geometry

3D Geometry Tutorials - David Eberly

cs2110Hierarchy - cs.cornell.edu

Reverse Engineering Niloy J. Mitra. Reverse Engineering Capturing Geometry: 3D Scanners laser scanner 3D geometry

Epipolar Geometry. Two-view geometry Epipolar geometry 3D reconstruction F-matrix comp. Structure comp

Maths 3d Geometry

3D Geometry for Computer Graphics

Worldwide Pose Estimation using 3D Point Cloudslandmark.cs.cornell.edu/docs/global_pose.pdf · Worldwide Pose Estimation using 3D Point Clouds ... † Cornell University {snavely,dph}@cs.cornell.edu

Applications of projective geometry€¦ · 3D projective geometry These concepts generalize naturally to 3D • Homogeneous coordinates – Projective 3D points have four coords:

Geometry - NJCTLcontent.njctl.org/courses/math/geometry-2015-16/3d-geometry/3d... · 2 Geometry 3D Geometry 2015-10-28

Creating and processing 3D geometry

Using MMX™ Instructions to Perform 3D Geometry ...€¦ · Using MMX™ Instructions to Perform 3D Geometry Transformations March 1996 3 2.0. REVIEW OF 3D GEOMETRY Common geometrical

27. 3D GEOMETRY

Step1 of 3D Geometry Creation

Recursion - cs.cornell.edu

3D Printing Oriented Design: Geometry and Optimizationstaff.ustc.edu.cn/~lgliu/Courses/SigAsia_2014_course_3Dprinting/... · 3D Printing Oriented Design: Geometry and Optimization

Rasterization - cs.cornell.edu

Objects - cs.cornell.edu

55:148 Digital Image Processing 3D Vision, Geometry Topicsuser.engineering.uiowa.edu/~aip/Lectures/Lecture06... · 3D Vision, Geometry Topics: Basics of projective geometry Points

Meshes - cs.cornell.edu

Formal Foundations of 3D Geometry to Model Robot …...Formal Foundations of 3D Geometry to Model Robot Manipulators Basic Elements of 3D Geometry Robot Manipulators with Matrices

Geometry - Center For Teaching & Learningcontent.njctl.org/courses/math/geometry-2015-16/3d-geometry/3d... · Geometry 3D Geometry 2015-10-28 Slide 3 / 311 Table of Contents Intro

Testing - cs.cornell.edu

Working in the Geometry Modes of Plaxis 3D - … · • The Geometry modes of Plaxis 3D comprises the ... • Defining 3D geometry using mouse input is ... and interchange (*.DXF)

cs2110ShortestPath - cs.cornell.edu

Claritas – 3D Land Geometry Example Workflow for an Example 3D Land Geometry Dataset supplied by USGS

3.1 Geometry of 2D shapes 3.2 Geometry of 3D Objects 3.5 ... · 3.2 Geometry of 3D Objects 3.3 Geometry of straight lines 3.4 Transformation geometry ... that have 2 parallel (circular