Taku Komura

Volume Illumination and Segmentation

Computer Animation and Visualisation – Lecture 13

Taku Komura

Institute for Perception, Action & BehaviourSchool of Informatics

Taku Komura

Overview

Volume illumination Segmentation

Taku Komura

Volume Illumination Why do we want to

illuminate volumes? illumination helps us to

better understand 3D structure

—displays visual cues to surface orientation

—highlight significant gradients within volume

Taku Komura

Light Propagation in Volumes Lighting in volume

− In ray-casting (the method in the last lecture) only transmission and emission considered

− can also:— reflect light— scatter light into different directions

Taku Komura

Illumination of Volumes

For every voxel ray intersects, need to consider:

• Light absorbed.• Light emitted.• Light scattered out of the ray.• Light scattered into the ray.

Taku Komura

Illumination of Volumes

For every voxel ray intersects, need to consider:

• Light absorbed.• Light emitted.• Light scattered out of the ray.• Light scattered into the ray.

Taku Komura

Example : multiple scattering

Light scattered multiple times to produce simulation of a cloud

Taku Komura

Example : sub-surface scattering

Varying how light is scattered inside a surface affects perception

− hence useful in visualisation

− why ? - prior visual experience, perception of distance ?

Very large statue

Medium sized

Small statue.

Taku Komura

Simple Tricks: Subsurface Scattering

The more the light travels in a material, the more it gets scattered and absorbed Reduce the intensity of the light which has travelled longer

Taku Komura

Volume Illumination - ?

Scattering is too costly so we usually do not take them into account when doing volume rendering

But we still can add slight shadows to the volume by illuminating them

Taku Komura

What are we illuminating ?

embedded (iso-) surface sharp gradients in the scalar value

Taku Komura

Shading an Embedded iso-surface

Classify volume with a step function Use regular specular / diffuse surface shading Remember for lighting equations require illumination direction

− camera model (position)

− surface orientation

− need to calculate and store surface normal

Taku Komura

Estimating the surface normal from the depth

Use distance map to the iso-surface value

1. Determine the threshold value2. Determine the surface voxels based on the

threshold3. Compute the normal vectors based on centred

difference method

For example, if we sample the centre of the voxels,

We can extract the normal vectors of the region where

the scalar values are changing significantly,

i.e. boundary of tissues

Taku Komura

Result : illuminated iso-surface

Surface normals recovered from depth map of surface

MIP technique

Shaded embedded iso-surface.

Taku Komura

Illuminating Opacity (Scalar) Gradient

Illuminate “scalar gradient” instead of iso-surface

− requirement : estimate and store gradient at every voxel

CompositeShaded opacity gradient(shades changes in opacity)

Taku Komura

Illumination : storing normal vectors

Visualisation is interactive

− compute normal vectors for surface/gradient once

− store normal

− perform interactive shading calculations

Storage :

− 2563 data set of 1-byte scalars ~16Mb

− normal vector (stored as floating point(4-byte)) ~ 200Mb!

− Solution : quantise direction & magnitude as small number of bits

Taku Komura

Illumination : storing normal vectors

Subdivide an octahedron into a sphere.

Number the vertices.

Encode the direction according to the nearest vertex that the vector passes through.

For infinite light sources, only need to calculate the shading values once and store these in a table.

Quantize vector direction into one of N directions on a sub-divided sphere

Taku Komura

Overview

Volume illumination Segmentation

Taku Komura

Medical Segmentation

Segmenting the data into different tissues

The data can be either volume / image

Taku Komura

Segmentation of CT imaging CT value corresponds to density → easier to segment

Sometimes, you might want to segment tissues of the same CT values

− Segmenting muscles

Taku Komura

How to segment MRI data?

Liver

Trachea

Lungs

Gall Bladder

How to extract an organ here?

A Dog’s thorax

MRI values are not directly proportional to density

Taku Komura

Image Processing on MRI Data From image processing / computer vision

− segment image slices— noise removal (smoothing)— edge detection (changes in colour)— Region growing— thresholding – all scalar values in

between upper and lower limits

Liver outlined in red

Image processing

− not strictly visualisation

− additional tool in visualisation pipeline

− see Advanced Vision course

Taku Komura

Region Growing

start from initial image patch grow outwards including similar regions stop when boundary or distinct change in

value reaching

Taku Komura

Example : MRI segmented knee

Original MRI slice (top, left), Segmentation (top, right), Overlay (bottom).

Courtesy : Brigham & Women's Hospital

Taku Komura

“Segmentation” of dog organs

Segmentation of organ in each image slice (2D)

− visualisation of segmented 2D image stack as 3D volume

3D model of Liver.Liver outlined in red

Taku Komura

Triangulated 3D model of dog liver

Vertices in one contour need to be

matched with those in the next slice to

produce triangulated mesh.

Problem - different number of vertices in

each curve

Taku Komura

Problem 1 : Grow “hole”

High slice

Middle slice

Taku Komura

Problem 2 : split contours

Known as the “branching problem”.

Middle slice Low slice

Taku Komura

[Geiger '93]

OR OR

Possible Ambiguities Contour connection method

needs to handle these cases

Solution: NUAGES [Geiger '93]

− fill convex hull of contour with tetrahedra

− discard those outside the contours or with only one edge on each level

− 3D contour connection = remaining exterior tetrahedra faces

Taku Komura

NUAGES : Triangulate Contour Triangulate contour vertices using Delaunay Triangulation

− no inside / outside constraint

− forms planar convex hull of each contour

[Geiger '93]

Taku Komura

Delaunay triangulation

Input : vertices

Output : triangles composed by the vertices

The triangulation DT(P) of P such that no point in P is inside the circumcircle of any triangle in DT(P)

maximizes the minimum angle of all the angles of the triangles in the triangulation

Gives nice set of triangles for finite element analysis

Connecting the centres of the circumcircles produce the Voronoi diagram

Taku Komura

NUAGES : Form Tetrahedra

Form tetrahedra by joining closest vertex on opposite contour

− tetrahedra t1 and t2 formed with closest opposite vertex enclosed within area of triangle

− The vertex on the opposite side that is closest to the circumcenter is selected

− tetrahedra t12 formed by two edges that cross

− Vertices are added if necessary

[Geiger '93]

Taku Komura

NUAGES : Form Tetrahedra

Form tetrahedra by joining closest vertex on opposite contour

− t1 : A triangle on Plane 1 and point on t2

− t2 : A triangle on Plane 2 and point on t1

− t12 : connecting two edges each on t1 and t2

[Geiger '93]

Taku Komura

NUAGES : Remove Tetrahedra

Remove : Tetrahedra with an edge outside of the contour.

Remove : Non-solid ones

[Geiger '93]

Non-solid : tetrahedra not connected to t1 nor t2

Taku Komura

Problems The closest point on the other side might be too far

away We insert points by projecting the Voronoi skeleton of

the top contour to the bottom

Taku Komura

Voronoi diagram − We are given a set of points P.

− Each point p has a Voronoi cell V(p).

− V(P) consists of all points closer to p than any other points in P

− Created by connecting the center of circumcircles of the triangles produced by Delaunay triangulation

Taku Komura

Creating skeletons by the Voronoi diagrams

By connecting the edges of the Voronoi diagram of the points sampled over a closed polygon, we can produce its skeleton

The skeleton inside the polygon is called internal Voronoi skeleton (IVS), and that outside is called external Voronoi skeleton (EVS)

Taku Komura

Whether we insert points We check if the IVS of one side intersects with the

EVS of the other side If they do, the EVS is projected to the other side

and points are sampled over it

Taku Komura

NUAGES : Results

Use exterior triangles of remaining tetrahedra to form outside boundary of the shape

Upper Contour

Lower Contour

Connected : top down view

Connected : side view

[Geiger '93]

Taku Komura

NUAGES : Results

Multiple medical image slices joined by NUAGES

− http://www-sop.inria.fr/prisme/fiches/Medical/

Taku Komura

Dog Liver : Final Model

Completed model – despite presence of contour holes/ branches (NUAGES)

Taku Komura

Fully automatic segmentation is difficult

although you apply all great computer vision techniques

Segmentation is a low level procedure

− Based on colour information

− But users have high level knowledge

Taku Komura

Volume Catcher We need to give some knowledge to the system

− A good user interface can improve the efficiency of segmentation

− Sketch-based interface , Owada et al. I3D 2005

http://www.youtube.com/watch?v=2Utj9L6dLrE

Taku Komura

Specifying the Region of Interest The user specifies which area he/she is interested in The system finds out the color information inside and

outside the volume data The color information is used for region growing /

segmentation

Taku Komura

Contour-based Interface for Refining Volume Segmentation

An interface to refine the contours directly in 3D http://www.riken.jp/brict/Ijiri/movies/contourrefine.wmv

http://www.youtube.com/my_videos_upload?forceui=3

Taku Komura

Summary Subsurface Scattering

Volume Illumination

Segmentation

Readings Jensen et al. “A Practical Model for Subsurface Light Transport”,

SIGGRAPH 2001

http://http.developer.nvidia.com/GPUGems3/gpugems3_ch14.html

Marc Levoy, “Display of Surfaces from Volume Data”, IEEE Computer Graphics and Applications,Vol. 8, No. 3, May, 1988, pp. 29-37.

S Owada, et al. “Volume Catcher”, I3D 2005

Takashi Ijiri and Hideo Yokota: Contour-based Interface for Refining Volume Segmentation Computer Graphics Forum, 29(2007), 7, 2153-2160. (Pacific Graphics2010)