3d fib-sem image segmentation and advanced …efug.imec.be/efug2012_14_lichau.pdf3d fib-sem image...

Data: U Alberta Data: U Alberta

Data: NIST

3D FIB-SEM Image Segmentation

and Advanced Characterization

EFUG Meeting 2012 October 1st - Cagliari, Italy

Daniel.Lichau@vsg3d.com

Outline

About VSG

3D imaging from visualization to characterization

FIB/SEM reconstruction

Image processing and segmentation for characterization

Registration and data fusion

Solving Visualization and Analysis challenges since 1986

Open Inventor ®

3D Development Toolkit

• Oil & Gas, Geosciences, Mining

• Medical and Life Sciences

• Engineering and Simulation

Amira ®

3D Visualization Software

• Life Sciences

• Biomedical Research

• Pharmaceutical Industry

Avizo ®

3D Analysis Software

• Materials and Geoscience

• Industrial Inspection

• Engineering & Simulation

Visilog 2D Image Processing

• Biology

• Pharmaceutical

• Materials Research

On August 1, 2012,

FEI acquired VSG

and

Widely used in Materials Research, Industrial Inspection, Geoscience, Life Sciences

Any material, any size, any scale, any imaging modality

Research, prototyping, and industrial applications

The power of FIB-SEM 3D Imaging

Application areas Geosciences Reservoir rocks, CO2 sequestration Life sciences Tissue Material sciences Fuel cells Catalysts Ceramics Polycrystalline Metals

Semiconductors Failure analysis

Defect inspection in nanowires

Nanyang Technological University, Singapore

From visualization to characterization

Characterization of delaminations at a chip / molding

compound interface. Courtesy A. Rucki

BGA defect analysis Modeling topology

Intermetallic microstructural analysis in tin-plated copper (tin whiskers)

NIST and VSG Marsh et al. 2010. Microscopy and Microanalysis.

Case Studies: Metals

Global Metrics

Fractal dimension of basal surface: 2.31

Degree of anisotropy of interspersed: 0.598

3D Density of interspersed: 0.13 grain / μm3

Population Metrics [Combinatorial filters]

Volume

Surface Area

Length

Width

Aspect Ratio

Orientation

Etc.

Case Study: Solid Oxide Fuel Cells

Characterizing Porous Materials

• Image-based quantification

• Total porosity, Connected porosity, Included Porosity

• Tortuosity, coordination

• Propagation distance

• Modeling-based quantification with Avizo XLab

• Permeability Tensor and Absolute Permeability

• Molecular Diffusivity

• Formation Factor

• Heat Conductivity (coming soon)

Data generously shared by MNT Lab University of Alberta

Image to Simulation Workflow

Skeletonization

CAE solvers Abaqus, Ansys, Comsol,

Fluent, OpenFoam, StarCCM+, etc.

Pore Network Modeling

Mesh generation

Direct calculation of physical property

Avizo XLab

micro-CT image stack

Avizo Fire

Avizo Fire Avizo Fire

Image Segmentation

Avizo Wind

Avizo Fire

Absolute permeability

Molecular diffusivity

Electrical resistivity

Mesh Generation workflow

Segmentation Reconstruction 3D Grid Generation Export

DXF, STL…

The problem with real-world imaging

Idealized imaging

High resolution Resolve small features

High contrast Differentiate different materials High signal fidelity Uniform response High spatial fidelity Distortion free

The problem with real-world imaging

Idealized imaging

High resolution Resolve small features

High contrast Differentiate different materials High signal fidelity Uniform response High spatial fidelity Distortion free Ideal image

The problem with real-world imaging

Idealized imaging

Noisy image

High resolution Resolve small features

High contrast Differentiate different materials High signal fidelity Noise, Non-uniform illumination High spatial fidelity Distortion free

The problem with real-world imaging

Idealized imaging

High resolution Resolve small features

High contrast Differentiate different materials High signal fidelity Noise, Non-uniform illumination High spatial fidelity Distortion free

The problem with real-world imaging

Idealized imaging

Noisy image

High resolution Resolve small features

High contrast Differentiate different materials High signal fidelity Noise, Non-uniform illumination High spatial fidelity Distortion free

The problem with real-world imaging

Idealized imaging

Noisy image with foreshortening

High resolution Resolve small features

High contrast Differentiate different materials High signal fidelity Noise, Non-uniform illumination High spatial fidelity Foreshortening, shear, Misalignment

Signal fidelity artifacts manifest themselves in the histogram

FIB-SEM Shale

Signal fidelity artifacts manifest themselves in the histogram

FIB-SEM Steel

Unprocessed

Signal fidelity artifacts manifest themselves in the histogram

Data generously shared by Georgs-Marienhutte

FIB-SEM Steel

Shadow corrected

Signal fidelity artifacts manifest themselves in the histogram

Data generously shared by Georgs-Marienhutte

FIB-SEM Steel

“Denoised”

Signal fidelity artifacts manifest themselves in the histogram

Data generously shared by Georgs-Marienhutte

FIB-SEM Imaging artifacts

Artifacts

Geometrical distortions Foreshortening Stack alignment Z-shear

Signal fidelity artifacts Noise Shadowing Curtaining Charging Pore-backs Pore halos

Common workflow to interpreting 3d structure

Common workflow to interpreting 3d structure

Visualization Volume Rendering, Surface, Slice

Image processing Greyscale transforms Geometry transforms Image filters Mathematical morphology Image segmentation

Quantitative Analysis Image to table

Simulation Image to simulation pathway

Generic image formats TIFF, BMP, JPG

Microscopy image formats MRC

Home-grown research formats Raw (sometimes with header)

Light Microscopy PSF Z-drop Non-uniform illumination

TEM Tomography Missing wedge CTF

FIB-SEM Shadowning Mis-alignment Shearing

Artifacts may be technique specific

2D Alignment

2D Alignment Selective alignment by intensities or 3D mask

Shearing correction

Shadowing Trench walls and floor can occlude signal and cause locally darker regions Flatfield correction

Detector noise Curtaining Image Smoothing

Pore backs Pore halos Relief artifacts

FIB-SEM Geometrical

• Alignment • XY displacement • Calibration of slice thickness (Z)

• Foreshortening correction (Y) • Shearing (along Z-direction) • Masking

Shadowing Noise Curtaining

Artifacts may be technique specific

From greyscale to binary First stage of segmentation

Thresholding or other binarization

Refining the binary image Second stage of segmentation

Grain boundary reconstruction

Indexing all binary objects

Connected component labeling

Population measurements

Properties upscaling – shale rock permeability case

Medical CT: 0.4mm

Resolution?

Medical CT: 0.6mm uCT: 0.009mm

Solid (e.g., organic matter)/Fluid (e.g., oil)

Substitution?

Scanning Electron Microscope

Downscale and Upscale

Heterogeneity segmentation

1mm

H1: Pores 7.9%

H2: Organic Matter: 14.6%

H3: Mineral 77.5%

uCT: 25um resolution

FIB-SEM: 20nm resolution

FIB-SEM: 20nm resolution

Pores 4.6% OM: 0% Mineral: 95.4%

Pores 37.7% OM: 58.3% Mineral: 4%

2um

1um

data1

data3

data2

Choosing the ROI

CLSM SEM

CLSM FIB-SEM

Sample: Mung bean root nodule colonized by nitrogen-fixing bacteria

D-A-CH FIB Workshop 2011 Zürich

Correlative microscopy

Tool for manual and automatic slice alignment

Image segmentation and advanced characterization of 3D FIB-SEM Reconstructions

Data: U Alberta Data: U Alberta Data: NIST Data: ExxonMobil

Summary

• Universal workflow applies

• Analysis hinges on segmentation

• Microstructure analysis is easy

• Numerical modeling is getting easier

• Increasing use of data registration and fusion

Catalog of artifacts

• Geometrical artifacts

• Rigid alignment

• Ineleastic alignment

• Shear

• Foreshortening

• Signal fidelity artifacts

• Charging

• Curtaining

• Noise

• Shadowing

• Pore-backs

• Pore halos

Avizo 7.1 coming soon

Enhanced FIB Stack Wizard

Filter Sandbox

Registration and data fusion tutorials

New Animation Producer

XLab new solvers

Molecular Diffusion

Electrical Resisivity

New volume to surface mapping

Surface rendering optimized

Colormap port enhanced

And more

Data: U Alberta Data: U Alberta

Data: NIST

THA NK YOU

Da n i e l . L i c ha u@vs g3d . c om