georeferencing of images by exploiting geometric distortions in stereo images of uk dmc

1

Mirza Muhammad Waqar

GEOREFERENCING OF IMAGES BY EXPLOITING GEOMETRIC DISTORTIONS IN

STEREO IMAGES OF UK DMC

Final Defense

2

Advisor Dr. Rafia Mumtaz (SEECS, NUST)

Guidance & Examination Committee Dr. Ejaz Hussain (IGIS, NUST) Dr. Rizwan Bulbul (IGIS, NUST) Muhammad Hussan (IGIS, NUST)

3

Contents

Overview Novel Contribution Objectives & Scope Research Novelty Literature Review Thermo elastic Model Model Inversion Results Conclusions

Georeferencing It is the process of assigning

geographic coordinates to a digital image.

It is a process for correcting spatial location and orientation of a satellite image

Types of Georeferencing Direct Georeferencing Indirect Georeferencing

Required for Geospatial Analysis

Change Detection Urban Planning

Map update

Overview4

5

Overview Traditional Method of Georeferencing

Require Ground Control Points (GCPs) GCPs are acquired manually and hence an expensive task Regions like deserts which lack salient features and have

homogeneous texture, selection of GCP is difficult A large number are required for complex terrain with varied surface

elevation. Accuracy of exterior orientation depends on the accuracy of GCP Not suitable for push broom imagery as every scanned line

possesses a different set of exterior orientation parameters Proposed method is based on Direct Georeferencing

Does not require Ground Control Points (GCPs) Utilizes

Satellite position/velocity data Attitude data Sensor Configuration to determine the pixel’s geo location.

6

Objectives

The primary goal of this research is to provide accurate georeferenced imagery using no GCPs by mitigating thermo elastic effect.

This will be accomplished by the following objectives

Modeling thermo elastic effect as a transformation matrix

Model inversion in order to extract the thermal deformation from the image offsets

Find geodetic coordinates by correcting pointing error Validate the complete model by testing it on UK-DMC

imagery

7

Novel Contributions

Modeled the thermo elastic effect as transformation matrix by exploiting the inter image offsets present between the pair of images.

Inverted the model to extract the thermal deformation knowledge to remove the pointing error.

Developed a new direct georeferencing method capable of modeling the pointing errors.

8

UK DMC – Sensor GeometryLaunch by Surrey Space Center, UK

in 2003 operated by DMC international imaging

Sensor: Push broom Made up of 6 CCD Channels Making 2 banks of 3 (Port and Star-

Board Array) Separate by an angle β across track Separate by an angle α along track Both Port & Star-board Array

have10000 linear CCD detectors 19500 pixel image width with 500

pixels overlap

GSD = 32 m Spectral Bands: Green, Red, NIR Ground Swath = 640 km

DMC Multi Spectral Imager (MSI)

Star-Boar

dArra

y

Port Array

9

Direct Georeferencing Direct Georeferencing

Transforms the image coordinate in camera frame to geodetic frame. Satellite onboard attitude data Satellite position and velocity data

One crucial point Accuracy of estimated geo-locations is

directly dependent on the accuracy of onboard sensor measurements

Main Advantage Requires no GCPs Becoming more robust and accurate

every year with the ongoing development in GPS and inertial equipment.

Can be applied with aerial camera, hyperspectral scanner, Synthetic Aperture Radar, LIDAR

10

Thermo-elastic Effect

Due to thermo-elasticity Satellite cools and heat up

periodically which causes it to contract and expand.

This introduces changes in the relative orientations of the attitude sensors (e.g. star tracker) and the imager.

(Attitude of Imager)

(From Attitude Sensor)

11

Literature Review on Thermo elastic Effect

In 2003, work on “Active pixel array devices in space missions“ has been done and it has been found that

The X-ray Telescope for NASA’s Swift mission incorporates a Telescope Alignment Monitor (TAM) to measure thermo-elastic misalignments between the telescope and the spacecraft star tracker.

In 2005, research on “Pleiades-HR Image System Products, Quality And Geometric Accuracy” has been done.

It has been found that attitude sensors are mechanically fixed on the telescope support to minimized the thermo elastic distortions.

12

Radhadevi et al in “In-flight geometric calibration of fore and aft cameras of CARTOSAT-1”devised a method for in-flight geometric calibration for Cartosat-1 The objective of this study is to ensure the best

absolute 3D pointing accuracy and relative location accuracy of the cameras.

It is concluded that accuracy of direct orientation observation could be brought down to better than 100 m with the inclusion of in-flight calibrated parameters in to the adjustment model.

Literature Review on Thermo elastic Effect

13

In 2008, a research is conducted on, “Attitude Performance Requirements and Budgeting for RASAT Satellite” In this study various sources of errors are

identified for RASAT satellite. These includes Star Tracker Errors Controller Errors Thermo elastic Error

Finally these errors are combined together in an error budgeting tool for analysis.

Literature Review on Thermo elastic Effect

14

Research Novelty In direct geo-referencing the major

source of error Thermo-elastic effect

Creates misalignment between the imaging sensor and the attitude sensor onboard.

Measured and realized orientations will be different.

Mechanical design changes to reduce the distortion by mounting the

imager/star-tracker assembly together on compliant mounts

But this solution will be costly and require changes in the physical mounting of the sensors.

Towards such ends we proposed to model the thermo elastic effect as a transformation matrix by using the inter-image offsets present

between the pair of images.

(Attitude of Imager)

(From Attitude Sensor)

15

Modeling Thermo-elastic Effect

The thermal deformation could be a rotation, scaling or some sort shearing in the pixels.T = Trotation + Tscale + Tshear Rotation

It has the major effect on the accuracy of geo-locations. Small deviations from the true orientation cause a large

displacement on the ground. Scaling

Due temperature changes the focal length of the imager contracts and expands.

This effects the scaling in the height (Z-direction) Shearing

Non parallel projection of imager results in shearing of pixels.

16

Modeling Thermo-Elastic Effect as a Rotation

This can be determined by

Rotation of Imager Determined by inter-image

offsets present between stereo pair.

Mathematical model has been developed

The inter image offsets equations are the function of senor configuration angles and attitude components.

(From Attitude Sensor)

(Attitude of Imager)

17 Let the port and starboard pixel in body frame be and Let T be the matrix that represents the thermo elastic

deformation.

Let A be the attitude matrix

Next step is to find the equations for the inter-image offsets.

Modeling Thermo-Elastic Effect as a Rotation

Column Shift Determine the corresponding Pixels Measure the distance of port and starboard pixel

End of their respective array Difference in the distances gives the column shift

Row Shift Time separation Delay for the corresponding

point to appear in the second image

18

Where

Where

Modeling Thermo-Elastic Effect as a Rotation

19

Direction in which satellite is moving

Δr

Δc

Δr be the row shiftΔc be the column shift

Star Board Array

Port Array

Overlapping region

Max Row ShiftMin Column Shift

Min Row ShiftMin Column Shift

Modeling Thermo-Elastic Effect as a Rotation

Inter-Image Offsets20

Row Shift Column Shift

At Nominal Attitude and nominal thermo-elastic effect

T =

Offset Modeling as Parabola & Straight Line

21

Offset Modeling Column shift represents

parabola Row shift represents

straight line Column shift as Parabola

Express parabola parameters in terms of attitude components.

H (peak value of parabola)

xo (x-intercept for peak value of parabola)

a (shape of parabola)

Attitude matrix

Where

Thermo elastic matrix

Para

bola

Offset Modeling as Parabola & Straight Line

22

Row shift as straight line

Express slope m and intercept c in terms of attitude parameters

Stra

ight

Lin

e

23

Effect of Attitude on the Offset Parameters

Image offset Para

meters

Roll Pitch Yaw

c 0.19 0.37 0.18m 2.66 0.04 0γp 0 0.01 0.002H 0 0 2.70

Model Inversion - Modeling Thermo elastic Effect as Rotation

Attitude is determined Estimating the unknown

components of attitude matrix by solving the offsets equations linearly. Use properties of attitude matrix

Mapping the estimated components to general attitude matrix

Determine the attitude by using the standard equations

Properties of Attitude Matrix

24

Attitude matrix

25

Modeling Thermo-elastic Effect as Scaling

Temperature changes effects the focal length of the imager Introduces height changes along the Zaxis

Change in scale (along z-axis) effects ground separation of the port and starboard imaging planes.

This will effect the row shift magnitude. Hence row shift constant equation will be used to

extract

26

The cofactor of the Tscaling matrix can be written as,

Where

The cofactor terms appear in expression of row and column offset parameters.

Modeling Thermo-elastic Effect as Scaling

27

Modeling Thermo-elastic Effect as Scaling

Column offset Equations

Where

28

Modeling Thermo-elastic Effect as Scaling

Row Offset Equations Similarly using cofactors, the expression for b1,

b2, b3, b4, b5, b6, b7 and b8 can be reduced to

Using above values, the row shift parameters can be written as

29

Simulations – Sensitivity of Scaling w.r.t offset parameters

30

Scale Extraction from row offset

With scaling matrix the row offset equation will take the form

From the above equation can be found as

31

Modeling Thermo-elastic Effect as Shearing

Shearing slides one edge of an image along the X or Y axis, creating a parallelogram.

The amount of the shear is controlled by a shear angle

32

The cofactors of shear matrix can be written as

The cofactor terms appear in the expression of row and column offset parameters.

Hence

Modeling Thermo-elastic Effect as Shearing

33

Column Offsets Equations

Row Offset Equations

Modeling Thermo-elastic Effect as Shearing

Where

34

Simulations – Sensitivity of Shearing w.r.t Offset

Parameters

35

Shear Extraction from column offset

Shear pushes the pixels across the track thus effecting the column offset. Therefore column offset will be used to find the shear factor

where

36

Thermo-elastic Matrix

Thermo-elastic matrix will be the sum of rotation, scale and shear extracted from the image offsets

Now inserting this matrix between the body and orbital frame will mitigate the misalignment between the imager and the attitude sensor.

37

Direct Georeferencing using onboard Attitude Sensor

Study Area: UK Date: May 23, 2004 Rows: 16250 Columns: 10000 GPS Positions

X = 4500717.0 m Y = -55020.32 m Z = 5445201.0 m

Attitude Values Roll = 32 centidegree Pitch = -25

centidegree Yaw = 47 centidegree

38

Attitude & GPS data Provided by SSTL

GPS Data

Attitude Data

39

Direct Georeferencing using exterior orientation data

By using the onboard exterior orientation data, the accuracy of 10-15Km has been achieved.

40

Row Shift Column Shift

Offset estimation using Window based Scheme

41

Measured Image offsets over the entire Scene length

42

Stereo Angle Estimation

Prior to model inversion, the sensor configuration angles of UK DMC must be determined which are α and β.

The β is being determined by SpaceMetric β = 12.6448o.

However the magnitude of α can be found from Rearranging equation of row offset’s constant

For UK DMC band 3 image pair, the value of α was found to be 0.04560.

43

Direct Georeferencing using Imager Attitude

By using imager attitude, the accuracy of 7-10Km has been achieved.

44

Thermo elastic Rotation

45

Direct Georeferencing using thermo elastic Rotation

By applying the thermo elastic Rotation

The accuracy of 1-5Km has been achieved.

46

Thermo elastic Scale

47

Direct Georeferencing using thermo elastic Rotation and Scale

By applying the thermo elastic Rotation Scale

The accuracy of 1-3Km has been achieved.

48

Thermo elastic Shear

49

Georeferencing using thermo elastic Rotation, Scale and Shear

By applying the thermo elastic Rotation Scale Shear

The accuracy of 1-1.5Km has been achieved.

50

Potential Benefits No GCPs required:

The major benefit of this approach is that it does not based on GCPs. Collection of GCPs is time consuming and expensive Difficult to collect GCPs having homogeneous texture Large number of GCPs are required for complex terrain Unsuitable for pushbroom imagery

Thermo elastic correction To date, modeling of thermo elastic effect as a transformation matrix has

not been explored. The thermo elastic effect is deemed as major source of error in this work Not only provide cost effective solution but also mitigate pointing error

No additional hardware Low cost and low mass system for obtaining geolocation Can work with conventional EO cameras with no additional hardware

Frequent Attitude Observations Due to small baseline or angular separation between the sensors, the

registration time for the corresponding pixels is very small which results in rapid attitude observations.

51

Conclusions A novel method for measuring geolocation without GCPs has been

developed. The error in geolocations estimate has been addressed by modeling thermo

elastic effect as a deformation matrix. Mathematical model has been developed by exploiting inter image offset

between pair of images. The possible deformations that has been considered are

Rotation, Scaling & Shearing These deformations have been modeled individually and has been

summed at the end to represent the entire thermo elastic deformation. The row and column offset parameters have been simulated individually

to determine The best candidate for extracting these individual deformation.

Developed mathematical model has been validated on UK DMC imagery. Accuracy of 1-1.5Km is achieved using newly developed georeferencing

method.

52

Future Work

Developing Generic Model for thermo elastic effect

Can be applied on other celestial bodies (e.g. Moon)

53

1. Rafia, M, P.L.Palmer,. Waqar, M.M. Georeferencing of images without Ground Control Points by Exploiting Geometric Distortions in Stereo Earth Images. Journal of Remote sensing of the Environment (Manuscript Submitted) Impact Factor ~ 3.95

2. Waqar, M.M., Johum, F.M., Rafia, M., Ejaz, H. (2012) Development of New Indices for extraction of Built-up area & Bare Soil from Landsat Data. Journal of Geophysics & Remote Sensing. (Manuscript Accepted).

3. Waqar, M.M., Rafia, M., Sufyan, N., Mustafa, M. (2012) Accuracy Assessment of Geo-locations using Multi-Resolution Interpolated DEMs. 2012 4th International Conference on Digital Image Processing. Kuala Lumpur, Malaysia. Published by SPIE.

Research Publications

54

Acknowledgement

55

Questions - Discussion