The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

A View-Independent Graphics Rendering

Architecture

Jason Stewart, Eric P. Bennett, and Leonard McMillan

Presented by Anselmo Lastra

University of North Carolina at Chapel Hill

Graphics Hardware 2004 Grenoble, France

Graphics Hardware 2004 Grenoble, France

PixelView

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Why View-Independence?

Why View-Independence?

Decouples Rendering from Viewing

Eliminates latencyProvides uniform frameratesAllows increased shading complexity

Needed for future applicationsShared multi-user virtual environmentsTrue three-dimensional (Autostereo) displays

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

True 3-Dimensional Displays

True 3-Dimensional Displays

Promising 3-D Display Technologies

Lenticular, and fly’s-eye opticsBarrier-based methodsReflective opticsHolographic optics

Matusik’s lenticular 3D display

Benton’s holographic 3D display

Technology is MaturingProblem:

How to generatethe content?Requires 1000’s of simultaneous views

Isaksen’s fly’s-eye display

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Using Today’s ArchitectureUsing Today’s Architecture

I guess you could… buy 1024 GPUs10 years of Moore’s law would yield…

4 doublings in performance (only need 64 GPUs)At least 2 doublings in power (only needs 10 KW)

There has to be a better wayTraditional graphics architectures are inefficient for view-independent graphics

resourceswasted

power moreyInefficenc

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Previous WorkPrevious Work

Low Latency Rendering3D Light field Viewing H/W [Regan 99] Frameless Redering [Bishop 94 ]Just-in-time Pixels [Mine & Bishop 93]

View-Independent RenderingMultiple viewpoint rendering [Halle 98]

4D ParameterizationLight Field [Levoy & Hanrahan 96], Lumigraph [Gortler 96]

Micropolygon RasterizationReyes [Cook 87]Reyes streaming H/W pipeline [Owens 02]

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Rearchitecting the Pipeline

Rearchitecting the Pipeline

Classic “view-dependent” pipeline

Geometry RasterizeVertex

Processing

FragmentProcessing

Visibility 2DFramebuffer

Scanout

View Specification Sampling

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Rearchitecting the Pipeline

Rearchitecting the Pipeline

Proposed “view-independent” pipeline

GeometryFragment

ProcessingSubdivideinto points

Point-Casting

Scatter 4DFramebuffer

Scanout

View Specification

Sampling

Hardware Prototype

Host PC

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

PixelView PrototypePixelView Prototype

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

PixelView PrototypePixelView Prototype

100 Mhz XILINX SpartanII-E FPGA300k Gates16MB100MHzSDRAM5000 lines of Verilog6000 linesof C# onhost PC

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

PixelView DemoPixelView Demo

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

System PartitioningSystem Partitioning

The prototype pipeline implementation

GeometryFragment

ProcessingSubdivideinto points

Point-Casting

Scatter 4DFramebuffer

Scanout

View Specification

Sampling

Hardware Prototype

Host PC

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

PixelView 4D FramebufferPixelView 4D Framebuffer

16 MBytes of Framebuffer memory

Reconfigurable (ex. 8x8x256x256x(rgb + z))16 bits (5/6/5) rgb16 bits z

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

View SelectionView Selection

Every view is just a 2D planar slice through the 4D framebufferWhich, after some simplifying assumptions,reduces to:

vvv

uuu

ttt

sss

CjBiAjiv

CjBiAjiu

CjBiAjit

CjBiAjis

),(

),(

),(

),(

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Linear Expression Evaluators

Linear Expression Evaluators

Simple datapath replicated for each ofs, t, u, and vPixel rateTrivialH/W costEasy toparallelizeDrop inreplacement for traditional scanout

+

BiAiACCrow ACCcol

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Memory Access PatternsMemory Access Patterns

For each view the LEE generates:s(i,j), t(i,j), u(i,j), v(i,j)

S (5.7 format)

U (10.6 format)

T (5.7 format)

V (10.6 format)

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Scan out PerformanceScan out Performance

Typical < 10% Memory bandwidth required for scan out 640x480 VGA100 MHz SDRAM, and order of magnitude behind the state of the art (DDR @ 500MHz)Can easily support multiple simultaneous “views”

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Filling the FramebufferFilling the Framebuffer

Elemental Rendering Primitive is the Outgoing Radiance from a Point

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Point CastingPoint Casting

Instead of the natural “planar” radiance parameterizationabout each pointWe align with parameterization planesSimplifies mapping ),,(),,()(

),,(),,()(

zyxBtzyxAtr

zyxBszyxAsq

PrPr

PqPq

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

In Other Words…In Other Words…

Parameterize outgoing radiance on fixed planes and resample it.

q

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Unexplored View Coherence

Unexplored View Coherence

Outgoing radiance from a point is smoother than spatial variationsToday’s architectures do not exploit thisStill amplespatial coherenceWe support 2formats

Uniform colorSpatially varying

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Unexplored CoherenceUnexplored Coherence

Outgoing radiance from a point is smoother than spatial variationsToday’s architectures do not exploit this

Lambertian

View-dependency

Occlusion

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Geometry SubdivisionGeometry Subdivision

Subdivide until primitive is “point-sized”Backward compatibility with polygonsReminiscent of Reyes rendering pipeline

Every primitive requires a world-spacesubdivision methodHowever, Reyes subdivision is view-dependent (the stopping criterion is based on pixel grid)

Probably better methods

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Prototype LimitationsPrototype Limitations

Points are transferred via USB 1.1Achieved 80,000 points/second (which means 5,120,000 rays/second)Each pointcast requires at least 64 readsRequires 17% of memory B/WCould easily include 4 or more parallel point-casting unitsEntire design uses 23% of chip

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

A Practical PixelView System

A Practical PixelView System

The prototype demonstrates feasibility, but what would a real system entail?Improve:

ScalabilityField of ViewSubdivision TechniquesOutput Bandwidth

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Distributing the Frame Buffer

Distributing the Frame Buffer

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Expanding Field of ViewExpanding Field of View

6 Slabs @ 64x64x1024x1024x(8+8) = 64 GBytes

But … 1MB was huge for a 64k PDP-11

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Addressing Output Bandwidth

Addressing Output Bandwidth

Currently we can support only a handful of dynamic views out of the framebufferAn Autostereoscopic Displays would require every pixel on every frameHigh-speed interconnects are available >5GHz per pin without compression

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Improving SubdivisionImproving Subdivision

Our conservative subdivision methods oversample by a factor of 4 or more after factoring out depth complexity Fast, on-the-fly, hardware friendly, uniform subdivision would be great

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

ConclusionsConclusions

PixelView simultaneously supports low latency and complex shadingPixelView supports a wide range of primitives and IBR data structuresPixelView is scalable to:

A full field of viewHigh resolutionsMulti-user environments

PixelView can power the next generation of display technologies

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Thank You – Questions?Thank You – Questions?

A View-Independent Graphics Rendering

Architecture

PixelView