image mosaic techniques for the restoration of virtual heritage yong-moo kwon, ig-jae kim, tae-sung...

Image Mosaic Techniquesfor the Restoration of Virtual

Heritage

Yong-Moo Kwon, Ig-Jae Kim, Tae-Sung Lee, Se-Un Ryu, Jae-Kyung Seol

KISTKOREA

2003. 8. 28

Contents

Revisiting Image Mosaic Technique

Our Researches for Image Mosaic

IR Reflectography Image Mosaic

X-Ray Image Mosaic

Summary

Revisiting Image Mosaic Technique

Image Mosaicing

Panorama Image

Image Based Rendering (IBR)

Basic Algorithm Registration using Features

Image Warping based on Homography

Matrix

Blending Images

Target Dimension in view of Image Mosaicing

2D Target

Planar Paintings Image

Homography Technique Feature-Based Image Mosaicing

3D Target

3D Real World Image

Limitation using Homography Due to Depth Difference b/w Features in Target

Our Research for Image Mosaic

2D Target

IR Reflectography

Mural Underdrawings Mosaic

Special 3D Target

X-Ray Imaging

Old Sword X-Ray Image Mosaic

Research Topics

How to extract and use Features

Imaging Media (IR, X-Ray)

Feature’s characteristics are different from the previous

ones

IR Image Mosaic

IR Reflectography System IR Source

IR Filter

IR Camera

Murals

IR Reflectography Principle

Back Frame

UnderDrawing

Color Painting,

Dust

Visible Light IR

ReflectionReflection

Absorbed

IR Reflectography Camera

IR Camera : Super eye C2847 (~1.9 ㎛ ) Hamamatsu

IR Source : ~1.9 ㎛ IR Filter : CVI Laser Corp.

NIR bandwidth filter 800nm ~ 2000nm Pass

Bandpass Filter every 100 nm bandpass filter (800nm, 900, …, 2000nm)

IR Characteristics to Mural according to WL

IR Camera HAMAMATSU Super eye C2847 WL Range : 0.4 ㎛ ~ 1.9 ㎛

IR Source HAMAMATSU C1385-02

Sony PC-115 Digital Image Capture Night Shot

Filter HAMAMATSU IR-D80A : 0.8 ㎛ ~ 1.9 ㎛

CVI Laser corporation Near IR Interference BP filter 800nm, 900nm, … 2000nm

IR Image Mosaic for Mural Underdrawing

Basic Method Automatic Feature Extraction Registration Using Features Image Warping Image Blending

Main Considerations IR Wavelength Characteristics

Penetration Ratio into Paintings Color (Red, Green, Blue etc) Color Painting Depth

Our Approach

▶ Automatic Feature Extraction & Registration

- Cross Points in IR Underdrawing Image- Grid Pattern for Blank Space

▶ Adaptive Overlapping Area For Image Blending

- Trade-Off between Registration and Blending * Large Overlapping Area: Good for Registration

* Small Overlapping Area: Good for Blending

▶ Use feature of IR Spectrum

- Use Different IR Wavelength according to paining color

Automatic Feature Extraction

Feature of Korea Murals- Many Blank Space- Not so much good features

1> Visible Light Pattern

2> Twice Captures- w/o IR Filter- w/i IR Filter

IR Image Mosaic

- Homography Estimation using Grid Image & IR Image

- Apply Homography to IR Image

X-RAY Image Mosaic

Why we use X-ray Technique ?

Old Sword

Old Sword is inside Sword Cover

Weak for Touch & Manipulation

Can’t Open Sword Cover

Use X-Ray Technique for the restoration

of Old Sword inside Sword Cover

Sword for experiment

Schema of a x-ray imaging using a linear X-Ray Camera

1. X-Ray Image

2. X-Ray Tube

3. X-Rays

4. X-Ray Detector

5. PC

6. Object

Why X-Ray Image Mosaic ?

For High Resolution Imaging

Multiple X-Ray Imaging

Setting Object

X-Ray Image Capture

Move Object Upward or Downward Step-By-Step

Stitching X-Ray Images into High Resolution Image

X-Ray Imaging Principle

Basic Principle

X-Ray Particle Penetrates through Target

One Point Depth -> Grey Value Pixel

Dependency

Target Depth

Target Material

X-Ray Image Characteristics: 2D or 3D ?

Target Dimension in view of Image Mosaic

Well Controlled Penetration Angle

Image Pixel Depends on Penetration Angle

Usually Same Penetration Angle for Each Capture

Orthogonal axis Movement according to X-Ray Beam

Just Planar 2D Image Using CCD Camera

Object -> X-Ray Camera -> CCD Camera

2D Target: Homography Technique

X-RAY Image Equipment

X-TEK X-Ray System X-Ray Source & Object (Sword)

X-RAY Image Capture For High-Resolution Restoration

Multiple X-Ray Imaging Image Stitching Technique Feature-based Registration

Problem ? Difficult to use features in X-ray Image Using Feature Pattern

Feature Extraction

Feature Extraction From Known Pattern Circle Type & Rectangular Type

Circle Type -> Pattern Matching Rectangular -> Feature Points

Feature Extraction

Binary pattern for feature ID

Feature Extraction

Method Circle Type Pattern -> Apply Image Labeling Rectangular Type Pattern -> Corner Detection

2

2

yyx

yxx

DDD

DDDC

2

))((4)( 22222222

yxyxyxyx DDDDDDDD

- For every pixel of image, computes first derivatives Dx and Dy.

- The eigenvalues are found by solving det(C- λI )= 0

If λ1, λ2 > t, where t is some threshold, then a corner is found at that location

Feature Point Matching

Semi-Auto(Present) Automatic Feature

Extraction of Rectangle Type pattern

Manual Matching Automatic Matching (On-going)

Classify the features using pattern ID from Circle Type Pattern

Homography Matrix Apply LS-Method(Least

Square Method) using Matched feature Points

Semi-auto Demo

Implemented S/W

X-rayImage FileHandling

Feature Extraction &

Select Points

Homography Matrix

Estimation & Stitching

Generated High-Resolution X-ray

Image

More Experimentation

Summary

Application of Image Mosaicing Techniques

Infrared Image

X-Ray Image

Our Approach

Feature Pattern

Automatic Feature Extraction & Registration

Homography Technique

Imaging Media (IR, X-Ray) & Feature’s

Characteristics

Thank You !

ymk@cherry.kist.re.kr