virtual reality motion parallax with the facebook...

Virtual Reality Motion Parallax with the Facebook Surround-360 David Lindell and Jayant Thatte Stanford University Motivation New Technique Related Work Experimental Results Incorporating motion parallax into virtual reality (VR) can create a more convincing and comfortable VR experience. Although growing interest in VR has led to the development of 360-degree media platforms, including camera rigs and smaller handheld cameras, most systems fail to incorporate motion parallax rendering. Systems which do support motion parallax require a complicated acquisition process or a large amount of data must be stored to render the scenes. We implement a framework to capture natural scene images using the Facebook Surround-360 1 and render 360-degree stereo images with motion-parallax from an easily compressible data format. We alter the Facebook Surround-360 rendering code to produce DASPs with an extended stereo baseline corresponding to a larger ODS viewing circle. Motion parallax rendering is supported for eye positions within the larger viewing circle. Image Acquisition with Surround-360 Render DASP using extended stereo baseline, estimate depth from disparity Synthesize stereo views with motion parallax using DASP [1] S. Peleg, M. Ben-Ezra, and Y. Pritch. Omnistereo: Panoramic stereo imaging. IEEE Trans. Pattern Anal. Mach. Intell., 23(3):279–290, 2001. [2] R. Anderson, D. Gallup, J. T. Barron, J. Kontkanen, N. Snavely, C. Hernandez, S. Agarwal, and S. M. Seitz. Jump: virtual reality video. ACM TOG, 35(6):198, 2016. [3] H.-Y. Shum and L.-W. He. Rendering with concentric mosaics. In Proceedings SIGGRAPH 99, pages 299–306., 1999. [4] J. Carranza, C. Theobalt, M. A. Magnor, and H.-P. Seidel. Free-viewpoint video of human actors. In ACM TOG, volume 22, pages 569–577. ACM, 2003. [5] J. Huang, Z. Chen, D. Ceylan, and H. Jin, 6-DOF VR videos with a single 360-camera. In CVPR, 2017 (submitted) [6] J. Thatte, J. B. Boin, H. Lakshman, and B. Girod, Depth augmented stereo panorama for cinematic virtual reality with head-motion parallax. In 2016 ICME, pp. 1–6, July 2016. 1. https://github.com/facebook/Surround360 Source: Facebook From Simulated DASP From Surround-360 DASP Format/Method Scene Capture Data Footprint Motion parallax Omnidirectional stereo (ODS) panorama Google Jump [2], FB Surround-360 1 Simple Small No Concentric Mosaics [3] Medium Medium Yes Free viewpoint rendering [4] Hard Medium/Large Yes Structure from motion [5] Hard Medium/Large Yes Depth-augmented stereo panorama (DASP) Simple Small Yes

Upload: others

Post on 17-Oct-2020

2 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Page 1: Virtual Reality Motion Parallax with the Facebook Surround-360stanford.edu/class/ee367/Winter2017/lindell_ee367_win17_poster.pdf · Source: Facebook From Simulated DASP From Surround-360

Virtual Reality Motion Parallax with the Facebook Surround-360David Lindell and Jayant Thatte

Stanford UniversityMotivation

New Technique

Related Work

Experimental Results

Incorporating motion parallax into virtual reality (VR) can create amore convincing and comfortable VR experience. Although growinginterest in VR has led to the development of 360-degree mediaplatforms, including camera rigs and smaller handheld cameras, mostsystems fail to incorporate motion parallax rendering.

Systems which do support motion parallax require a complicatedacquisition process or a large amount of data must be stored torender the scenes.

We implement a framework to capture natural scene images usingthe Facebook Surround-3601 and render 360-degree stereo imageswith motion-parallax from an easily compressible data format.

We alter the Facebook Surround-360 rendering code to produce DASPs with an extended stereobaseline corresponding to a larger ODS viewing circle. Motion parallax rendering is supported foreye positions within the larger viewing circle.

Image Acquisition with Surround-360

Render DASP using extended stereo

baseline, estimate depth from disparity

Synthesize stereo views with motion

parallax using DASP

[1] S. Peleg, M. Ben-Ezra, and Y. Pritch. Omnistereo: Panoramic stereo imaging. IEEE Trans. Pattern Anal. Mach. Intell.,23(3):279–290, 2001.

[2] R. Anderson, D. Gallup, J. T. Barron, J. Kontkanen, N. Snavely, C. Hernandez, S. Agarwal, and S. M. Seitz. Jump: virtual reality video. ACM TOG, 35(6):198, 2016.

[3] H.-Y. Shum and L.-W. He. Rendering with concentric mosaics. In Proceedings SIGGRAPH 99, pages 299–306., 1999.[4] J. Carranza, C. Theobalt, M. A. Magnor, and H.-P. Seidel. Free-viewpoint video of human actors. In ACM TOG,

volume 22, pages 569–577. ACM, 2003.[5] J. Huang, Z. Chen, D. Ceylan, and H. Jin, 6-DOF VR videos with a single 360-camera. In CVPR, 2017 (submitted)[6] J. Thatte, J. B. Boin, H. Lakshman, and B. Girod, Depth augmented stereo panorama for cinematic virtual reality

with head-motion parallax. In 2016 ICME, pp. 1–6, July 2016.1. https://github.com/facebook/Surround360

Source: Facebook

From Simulated DASP From Surround-360 DASP

Format/MethodScene

CaptureData

FootprintMotion parallax

Omnidirectional stereo (ODS) panoramaGoogle Jump [2], FB Surround-3601 Simple Small No

Concentric Mosaics [3] Medium Medium Yes

Free viewpoint rendering [4] Hard Medium/Large Yes

Structure from motion [5] Hard Medium/Large Yes

Depth-augmented stereo panorama (DASP) Simple Small Yes

Recommendations For Surround Sound Productioncontent.grammy.com/PDFs/Recording_Academy/Producers_And_Engineers/Surround...postproduction, gaming, etc.) in surround sound. ... as a