Videogrammetric Technique for

Aerospace Applications: From Model Attitude and Deformation to

Surface Geometry

Tianshu Liu

Department of Mechanical and Aerospace Engineering

Western Michigan University, Kalamazoo, MI 28004

A. W. Burner, T. W. Jones, D. A. Barrows

NASA Langley Research Center

Hampton, VA 23681

Objectives

• Mapping surface quantities onto surface

• Measurement of static and dynamic aeroelastic

deformation and extracting vibration modes

• Measurement of attitudes and positions of models

and control surface

• Reconstruction of 3D velocity fields

Outline

• Application Examples:

Static and Dynamic Wing Deformation

Model Position and Attitudes

Vision-Based Autonomous Landing

Surface Geometry

• Videogrammetric Attitude and Deformation

Measurement (VMD) Technique:

Hardware (Cameras & Lighting)

Software (Calibration & Intersection)

• Conclusions

• Videogrammetric deformation technique has been

developed for wind tunnel testing at NASA Langley

(Burner 1996, 1997, Liu et al. 2000, Burner & Liu

2001, Liu et al. 2012)

• Viedogrammetry has been used in large-scale space

structures at NASA Langley (Pappa 2002, Jones

2005)

The Working Principle:

Non-Topographic Photogrammetry

Camera Calibration/Orientation

Objective: to determine camera exterior and interior orientation

parameters in the collinearity relation between object space

and image plane.

The Collinearity Equations

Tnnnn )Z,Y,(X=P

Tnnn )y,(x=p

)Z,Y,X,,,( ccc

)/SS,P,P,K,K,y,xc,( vh2121pp

Parameters:

Object point coordinates:

Image point coordinates:

Exterior orientation:

Interior orientation and lens distortion:

n = 1, 2, 3 ...

.coscosm,cossinm,sinm

,cossinsinsincosm

,coscossinsinsinm

,sincosm,sinsincossincosm

,sincoscossinsinm,coscosm

333231

23

22

2113

1211

Rotational Matrix

& Euler Angles:

)ZZ(m)YY(m)XX(m

)ZZ(m)YY(m)XX(mcyyy

)ZZ(m)YY(m)XX(m

)ZZ(m)YY(m)XX(mcxxx

c33c32c31

c23c22c21

p

c33c32c31

c13c12c11

p

Lens Distortion Model

dr xxx

dr yyy

,

4

p2

2

p1r r)x'x(Kr)x'x(Kx

4

p2

2

p1r r)y'y(Kr)y'y(Ky

)y'y)(x'x(P2])x'x(2r[Px pp2

2

p

2

1d

)y'y)(x'x(P2])y'y(2r[Py pp1

2

p

2

2d

2p

2p

2 )y'y()x'x(r

,

,

,

.

where

K1 > 0

K1 < 0

Camera Calibration/Orientation

Object point coordinates: Tnnnn )Z,Y,(X=P n = 1, 2, 3 ...

Image point coordinates: T

nnn )y,(x=p

Given Data:

Unknowns:

Exterior orientation: )Z,Y,X,,,( ccc

Interior orientation &

lens distortion: )/SS,P,P,K,K,y,xc,( vh2121pp

Photogrammetric Intersection

Object point coordinates: Tnnnn )Z,Y,(X=P n = 1, 2, 3 ...

Unknowns:

Given Data:

Tnnn )y,(x=p )Z,Y,X,,,( ccc )/SS,P,P,K,K,y,xc,( vh2121pp

Alternative Forms of Collinearity Equations

3

222p

2

3

111p

1

X

X

)(

)(

c

xxx

X

X

)(

)(

c

xxx

c3

c2

c3

c1

XXm

XXm

XXm

XXm

T321)X,X,X(X

( 1m , 2m , 3m ) Projection form in the frame

are coordinates on ( 1m , 2m , 3m )

0)( c1 XXW

0)( c2 XXW

Projection form least-squares estimation

Typical Methods for Camera Calibration

Iterative Least-Squares Method (The Bundle Method):

Determination of a full set of the parameters

Matrix singularity, Multiple camera stations and roll angles,

Initial guess, Expertise to operate.

Direct Linear Transformation (DLT):

Simplicity and no initial guess when lens distortion is neglected.

Loss of simplicity and large errors when lens distortion exists.

Methods in Comupter Vision (Tsai’s Two-Step Method):

Fast and nearly-automatic calibration.

Not solving the collinearity equations,

Limitations of radial alignment constraint.

Direct Linear Transformation (DLT)

Linear Treatment of a Nonlinear Problem

0)1ZLYLXL)(ydy(LZLYLXL

0)1ZLYLXL)(xdx(LZLYLXL

111098765

111094321

The DLT equations (Abdel-Aziz & Karara 1971):

111 L,L The DLT parameters: )Z,Y,X,,,( ccc

)y,x(c, pp

The over-determined system:

T111 )L,L( L

TMM11 )y,x,y,x( Cwhere

B is the 2M11 configuration matrix

CLB

The least-squares solution:

CBB)(B=L T1T

Optimization Method for Camera Calibration (Liu et al. AIAA J. 2000)

Two separated, but interacting procedures:

(1) Resection for the exterior orientation parameters;

(2) Optimization for the interior orientation

and additional parameters, i.e.,

Objective function std(xp) min

(1) A single image, single station method

(2) Automatic calibration when combined with DLT

(3) Determination of a full set of 14 camera parameters

(4) Capability to obtain spatial coordinates for

a multiple-camera system

0

1

2

3

4

5

0

2

4

68

1012

1416

02

46

810

1214

Z (in)

X (in

)

Y (in) x (mm)

-8 -6 -4 -2 0 2

y (

mm

)

-2

0

2

4

6

8

Object space Image plane

imaging

Simulation: Targets on a Step

Topology of the Objective Function std(xp) near

a Minimum-Point in the Parameter Space

Effects of Three-Dimensionality and Noise

on Topology of std(xp)

(a) H = 6 in, (b) H = 2 in. (a) disturbance level of 6 m;

(b) disturbance level of 1 m

3D Effect Noise Effect

Camera Calibration Set-Up

Calibration target-plate

Camera

Parameters c(mm) xp (mm) yp (mm) Sh /Sv K1 (mm-2

) K2 (mm-4

) P1 (mm-1

) P2 (mm-1

)

Optimization 8.133 -0.156 0.2014 0.99238 0.0026 3.310-5

1.810-4

310-5

Optical techniques 8.137 -0.168 0.2010 0.99244 0.0027 4.510-5

1.710-4

710-5

Calibration for a Hitachi CCD camera

with a 8 mm Cosmicar TV lens

c (mm)

10 20 30 40 50 60 70 80 90

xp o

r y

p (

mm

)

-1

0

1

2

Laser illumination technique

Optimization algorithm

xp

yp

c (mm)

10 20 30 40 50 60 70 80 90

K1 (m

m-2

)

-0.0020

-0.0015

-0.0010

-0.0005

0.0000

0.0005

0.0010

Optimization algorithm

Burner (1995)

Calibration for a Hitachi CCD Camera with a Sony Zoom Lens

Zoom setting (mm)

1020304050607080

Principal distance c (mm)

10

20

30

40

50

60

70

80

90

Optimization algorithm

Linear fit

Calibration for a Hitachi CCD Camera

with a Sony Zoom Lens

Effect of Zoom Setting Effect of Step Height

Calculations of Target Coordinates Multiple-Camera Solution

Photogrammetric Intersection:

0)(,0)( c(n)2(n)c(n)1(n) XXWXXW . ( 2,1n )

Generalized Longuet-Higgins Relation:

Image 1 Image 2

Epipolar Line

h(1)x

h(2)x

Epipolar Line

0)xx(Q)x(x )1(hh(1))2(hh(2)

Epipolar Geometry on Point Correspondence:

Geometric illustration of the single-camera solution

Calculations of Target Coordinates Single-Camera Solution

Single-Camera System Two-Camera System

Basic Systems

Single-Camera VMD System in Wind Tunnel

LIMITED EXCLUSIVE RIGHTS NOTICEThese data are subject to Limited Exclusive Rights

under Government contract No. NAS1-20220.

Camera Calibration/Orientation

in Large Wind Tunnels

Video Cameras

Still Cameras

Targets: Retro-Reflecting Targets

Targets: Retro-Reflecting Targets

Targets: Laser-Projecting Targets

(a) Test configuration for proof-of-concept experiment

Camera 1

Camera 2 Laser

Membrane Structure

Diffractive Pattern Generator

(a)

(b) (c) (d)

Flowchart of Software

Key Procedure

Target Tracking and Centroid Calculation

• Real-time target tracking.

• Automatic recovery of lost targets based on

memory of target locations.

• Adjustable intensity level threshold.

• Capability of tracking PSP targets.

target

target in next frame

searching box

Measurement Uncertainty

AOAac

(degree)

-40 -30 -20 -10 0 10 20 30 40

Ran

do

m E

rro

r (d

eg

ree

)

-0.030

-0.025

-0.020

-0.015

-0.010

-0.005

0.000

0.005

0.010

0.015

0.020

0.025

0.030

Test 1

Test 2

Test 3 0.01 deg.

Centroid variation in x direction (pixel)

-0.04-0.03-0.02-0.010.000.010.020.030.04

Population0

20

40

60

80

100

120

Standard deviation = 0.0081 pixels

(a)

Accuracy: 0.01 degrees in angle measurement

0.001 inches in displacement measurement

Wing Twist Calculation

Deformation and Attitude Measurements

Static Deformation of a Semi-Span Model

-3

-2

-1

0

1

0.30.4

0.50.6

0.7

0.8

0.9

1.0

-5

0

5

10

15

20

25

Tw

ist (d

eg

)

Wing Span (%)

AOA (deg)

Spanwise location

0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

De

fle

ction

(in

)

0.0

0.5

1.0

1.5

2.0

2.5

AOA = 0 deg

AOA = 5 deg

AOA = 10 deg

AOA = 15 deg

AOA = 20 deg

Static Deformation of a High-Speed Research

Model

Reconstructing Deformed Wing Surface Geometry

The body and local coordinate systems and the wing airfoil

sections along the quarter-chord line

4/c0

4/c0

twtw

twtw

4/c

4/c

ZZ

XX

cossin

sincos

ZZ

XX

Twisted Wing

Sections

Reconstructed Deformed Wing Quarter-chord line

Wing twist

Reconstructed Wing Surface

Rotor Blades in Ames 40-by-80-Ft Wind Tunnel

Rotor Blades in Ames 40-by-80-Ft Wind Tunnel

Thin Wing Vibration Measurements

Biologically-Inspired Concept

Camera Calibration/Orientation in Wind Tunnel

Two-Step Target Plate

for Camera Calibration

Two-Step Target Plate Aligned with

The Tunnel Coordinate System

Orthogonal Eigenfunctions Based on Beam Functions

6

1r

rr )t()y,x(w)t,y,x(w

1st bending 2nd bending

1st torsion

Vibration Control with Flexible Fins

Semispan: 71.5 mm (6.75 in)

Chord: 101.6 mm (4 in)

Thickness: 0.22 mm

Rectangular Flat-Plate Wing:

(Cantilever part: 6 in)

Mylar Fins:

Length: 0.2 chord

Thickness: 0.089 mm

5 segments in semispan

located at 0.1c

Time-Dependent Wing Surface at AoA = 8 deg at 13 m/s

Reconstructed from 6 Eigen Modes

Baseline Wing Wing with Fins

Maximum, Median and Minimum Amplitudes

Amplitude of 1st Torsion Mode at AoA = 8 deg at 13 m/s

Spectra of 1st Torsion Mode at AoA = 8 deg at 13 m/s

Applications in Global Flow Diagnostics

Motion Field on Surface

Image

(Cattafesta et al. 1996)

Skin Friction Topology of Wing-Body Junction Flows

GLOF Images Taken at Five Viewing Angles and Positions

AoA = 6 deg,

U = 27 m/s,

Re based on max

thickness:

151,000,

UV LED Power:

12 W

Long-pass Filter:

550 nm

Reconstructed Intensity Distribution on Surface of

the Wing-Body Junction via Photogrammetry

GLOF Intensity Distribution Surface Mesh

(240,372 grid points)

Reconstructed Skin Friction Field on Surface of

the Wing-Body Junction for AoA = 6 deg

0S#N#

6N#

6S#

• Pattern/target Recognition

• Position, attitude and velocity determination

NASA LaRC OV-10:

Two cameras at wing tips;

One camera at left vertical

tail

Vision-Based Autonomous Landing

(Liu & Fleming 2006)

Vision-Based Autonomous Landing

Natural Features on Ground:

High contrast, parallel

runway edges

3D Target Field for Camera Orientation

Determining Aircraft Position and Attitude

Two-Camera Method

Requirements:

• Two calibrated cameras

• Two parallel runway edges

(Edge detection)

,

3

1i

iaci321312131g3 r|PPPP|/PPPP ee

Aircraft Euler Angles and Position Coordinates

Relation between the ground and aircraft coordinate systems

Triangle

3

iaci12121g1 r|PP|/PP

1i

ee

3

1i

iaci221g321g3g2 r|PP|/PP eeee

3332 r/rtan

31rsin

1121 r/rtan

Roll, Pitch

and Yaw Angles:

Camera Parameters:

118)40,(4477,)Z,Y,X(acacac gOgOgO

)Z,Y,Xκ,φ,ω,( ccc )ft16,0,0,0,88,85( ooo

)ft16,0,0,0,88,85( ooo

feet Position & Attitude: o2 o5

o3

Simulation Case #2

Left-wing camera Right-wing camera

Random Noise Effects (Case #2)

Position Errors Attitude Errors

Surface Geometry of Gossamer Space Structures

1 2

3 4

(McInnes 1999, 2002, West & Derbes 2000,

Spieth & Zubrin 1999, Pappa et al. 2002,

Jones et al. 2006)

The photogrammetric method can be integrated

with other image-based techniques

in wind tunnel testing.

Conclusions

Videogrammetric technique is very useful

as a remote and non-contact measurement

method in wind tunnel testing, ground-based

and flight testing.