screen-space triangulation for interactive point rendering · screen-space triangulation for...
TRANSCRIPT
![Page 1: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/1.jpg)
Screen -Space Triangulation for Interactive Point Rendering
Reinhold Preiner
Institute of Computer Graphics and Algorithms
Vienna University of Technology
![Page 2: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/2.jpg)
1
Motivation
High-quality point rendering mostly implies some kind of continuous surface reconstruction
Using Point Properties for renderingpoint normals (local surface orientation)splat radii (connectivity)
![Page 3: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/3.jpg)
2
Motivation
Huge point clouds: time-consuming Preprocessing
St. Stephans Cathedral Domitilla Catacomb460 Million Points 1.9 Billion PointsNormal Est.: ~ 17h Normal Est.: ~ 21h
![Page 4: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/4.jpg)
3
Motivation
Can we achieve comparable quality on rendering without precomputed attributes?
� Our Approach: Reconstruct normal and connectivity info on-the-fly during rendering in screen-space on the GPU
AdvantagesNo time-consuming preprocessingSaves memory for storing attributes (normals, radii)Rendering/Reconstruction independent from data layout (Hierarchical, Out-Of-Core, …)
Possible ApplicationsFast on-site preview of scanned point cloudsInstant rendering of 4D point streams
![Page 5: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/5.jpg)
4
Overview of Our Approach
Input: Point data projected to screenPositionColor (optional)
Output: Reconstructed frame buffersDepthNormalColor (optional)
Use for further deferred shading,illumination, …
![Page 6: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/6.jpg)
5
Surface reconstruction - FAQ
Given a surface sample � whichneighbors to use for reconstruction?
KNN, FDN, …robust statistics, LMS … ?
![Page 7: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/7.jpg)
6
Surface reconstruction - FAQ
Given a surface sample � whichneighbors to use for reconstruction?
KNN, FDN, …robust statistics, LMS …
Given a local neighborhood � how to reconstruct surface?
surface fitting, forward search, …
![Page 8: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/8.jpg)
Given a surface sample � whichneighbors to use for reconstruction?
KNN, FDN, …robust statistics, LMS …
� Screen -Space Nearest Neighbor Search
Given a local neighborhood � how to reconstruct surface?
surface fitting, forward search, …
� Normal Estimation & Triangulation
7
Surface reconstruction - Our Approach
![Page 9: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/9.jpg)
8
Input: projected point bufferInitial search radius r
How to quickly find and store k nearest neighbors of each point Q in the input buffer?
Divide screen space region around Q in 8 segmentsStoring nearest neighbor of each segment in 2 RGBA Textures
Screen-Space Nearest Neighbor Search� Screen -Space Nearest Neighbor Search
![Page 10: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/10.jpg)
9
Screen-Space Nearest Neighbor Search
Pass 1: for each Pi, render search splat of radius rStore min. world space distances dmin at pixel Q
Pass 2: Render Search Splats Pi againCompare distance PiQ with saved dmin
� Screen -Space Nearest Neighbor Search
![Page 11: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/11.jpg)
10
Screen-Space Nearest Neighbor Search
Lookup the neighbor points and calculate normal
� Normal Estimation
Neighbor Buffers
Point Position Buffer
Point Normal Buffer
![Page 12: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/12.jpg)
11
Screen-Space Nearest Neighbor Search
Triangulation in Geometry Shader
� Triangulation
Sparse Input Buffers
Depth
Normal
Color
Final BuffersNeighbor Buffers
![Page 13: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/13.jpg)
12
Temporal Coherence Depth Culling
Depth Buffer
i+1
i
Input Points Depth Culled Input Points
RE
PR
OJE
CT
![Page 14: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/14.jpg)
13
Search Radii
Maintain a search radius bufferAdapt radii over timeStart with initial search radius r0
Define increase factor α > 1
Frame i: if #neighbors too small (e.g. < 3)
r i+1 = r i * αelse
r i+1 = max( distance(neighbor k) ), k = 1…8
![Page 15: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/15.jpg)
14
Algorithm - Summary
1) Project points to screenDepth cull with depth buffer from previous frame
2) Update search radii3) Perform neighbor search4) Normal estimation5) Triangulation
![Page 16: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/16.jpg)
15
Results
Comparison to point splatting
Box Splatting Gauss Splatting SST
![Page 17: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/17.jpg)
16
Results
Gauss Splats (Precomputed) SST
![Page 18: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/18.jpg)
17
Results
Normalestimation only locally � noise sensitive
precomputed Normals SST Normals
![Page 19: Screen-Space Triangulation for Interactive Point Rendering · Screen-Space Triangulation for Interactive Point Rendering Reinhold Preiner Institute of Computer Graphics and Algorithms](https://reader031.vdocuments.us/reader031/viewer/2022022016/5b658d6d7f8b9ab63a8bd3a7/html5/thumbnails/19.jpg)
18
Conclusion / Outlook
Interactive rendering without precomputationQuality comparable to Gauss splats
DrawbacksTemporal Coherence Artifacts, FlickeringSome degrees of freedom (r0, α)Normal estimation only local
Future WorkIntroduce denoising of normals by geometry-aware filterEstimate absolute radii per frame (get rid of TC, r0 and α)� instantly estimate good splat radii � Gauss splatting?