introduction of ship detection technology using sar and ... kazuo ouchi (p… · 09/07/2007 ·...

Introduction of Ship Detection Technology Using SAR and Trend of Ship Classification

Kazuo Ouchi Former Professor, National Defense Academy, Japan

Agenda

1. Purpose and need for ship detection and classification systems by synthetic aperture radar (SAR)

2. Ship detection by SAR: Algorithms and examples

3. Ship classification: Algorithms, examples, and current trend

4. Summary: Flow of ship monitoring system

First, Look at Optical Image at Night by SUOMI

Suomi (NOAA) 2011~

“Father of satellite meteorology”

Verner E. Suomi (1915 – 1995)

Sept. 2012

Night Time Light over Yellow Sea

Sept. 2012

Exclusive Economic Zone: EEZ

Korean EEZ Korean TZ (Transitional Zone) Chinese and Korean Prov. EEZs Chinese and Korean PMZ (Provisional Waters Zone) Chinese EEZ Chinese TZ (Transitional Zone)

Overlay of Night Time Light and EEZs

Korean EEZ Korean TZ (Transitional Zone) Chinese and Korean Prov. EEZs Chinese and Korean PMZ (Provisional Waters Zone) Chinese EEZ Chinese TZ (Transitional Zone)

Current Vessel Monitoring Systems

○ VMS : Vessel Monitoring System

○ LRIT : Long-Range Identification and Tracking of ships

○ VTS : Vessel Traffic Service

○ AIS : Automatic Identification System

Problem: Ships Without AIS/VMS/LRIT Signals

© JCG

Illegal oil release

Taiwan

Illegal oil release Poaching boat

Problem: Ships Without AIS/VMS/LRIT Signals

© JCG

Illegal oil release

Taiwan

Illegal oil release Poaching boat

Need for Vessel Monitoring Systems from Space with Imaging Sensors

Ship Detection and Identification with Imaging Sensors

Two types of airborne and satellite-borne imaging sensors:

1) Optical sensors

2) Imaging radars

Pleiades-1A: 50 cm resolution

cloud

Optical sensors are useful, but the problem is cloud cover !!!

Cloud Cover of the World during 2007 to 2009

Use Synthetic Aperture Radar (SAR) : All-Weather and Day-and-Night Data Acquisition Capability

Optical Image of the Tokyo Bay SAR Image of the Tokyo Bay

ALOS-Optical ALOS-PALSAR (L-band)

Simultaneously acquired optical and SAR images

Ship Detection by SAR

Principle: Radar Backscatter from Ships and Sea

TerraSAR-X Image of Gibraltar Strait

TSX StripMap HH 2007/07/09 06:29 UTC

Az.

Rg.

Selected Ship Detection Algorithms

1) CFAR (Constant False Alarm Rate) : Most frequently used algorithm. A threshold is set for a required false alarm rate (FAR) using a probably density function (PDF) of the background clutter. Resultant FAR is constant.

2) Adaptive Threshold : The value of target pixel is compared with the statistical values (mean, standard deviation) of background pixel. A threshold is variable and FAR is not constant.

3) Multi-Look / Sub-Look Detection : Uses the inter-look correlation property between moving windows in multi-/ sub-look images. Can detect images with low SNR.

4) Polarimetric Analyses : Polarimetric approach such as scattering power decomposition, eigenvalue analysis, inter-polarization correlation, etc. Not many datat by spaceborne SAR.

５) Many Others : Wavelet Transform, Notch filter, Standard deviation filter, Along-track InSAR, etc.

Selected Institutes and Ship Detection Algorithms

Institute System Name Algorithm

JRC / TNO (ESA / Netherland) SUMO/K CFAR / K-distribution

CCRS / MDA / RSI (Canada) OMW Modified CFAR / K-distribution

Veridian Syst. Div. (USA) --- Adaptive Threshold : IT = < I > + c σI

QinetiQ (UK) MaST Adaptive Threshold : IT = < I > + c σI

Kongsberg / FFI (Norway) MEOS Adaptive Threshold : IT = < I > + c σI

CLS (France) OMW CFAR

NOAA / NESDES (USA) AKDEMO CFAR

EMSA (EU) VDS CFAR

Edisoft / ISEL-Lisbon (Portugal) VDC softwear/SUMO Adaptive Threshold : IT = c (< I > + σI) / Sub-Look

MSS / MELCO (Japan) HuygensWorks Standard Deviation Filter

GMV (Spain) / IHI (Japan) SIMONS/ iOMS Wavelet/ Adaptive Threshold

eOsphere Limited (UK) R&G Median filter/Pauli Decomposition

Open University/IHI (UK/Japan) --- Sub-Look Entropy/Coherence

IT: Threshold intensity/amplitude, σI: Standard deviation, c: Constant

Example: Cell-Averaging CFAR A

mpl

itude

A

mpl

itude

Range or azimuth

Targets

Range or azimuth

Original signal, z

Clutter

Variance 分散値

After CFAR operation

Mean SAR Image

Moving window

Moving Window

Buffer Window (Not used)

Signal/Target Window

Background Window

After thresholding

FAR does not change even if σz changes

False alarm rate is constant !!!

Example: Adaptive Threshold Algorithm

Yes No

Target cell

Reference cells

Compare the value of the target cell and the mean + standard deviation

Reference cells

Example: Adaptive Threshold

Pixel value of target cell ≤ Threshold

No ship

Pixel value of target cell > Threshold

Ship

Target cell

Buffer window

Background window

Threshold = Constant x (Mean + Standard Deviation)

Example: Adaptive Threshold

Pixel value of target cell > Threshold

Ship

After threshold

False alarm rate is not constant !!!

Ship Detection with Polarimetric SAR Data

HH HV VV

Polarimetric SAR data: Current status

More information than single-polarization data Resolution of present polarimetric SAR is not fine compared with single- and dual-polarization SAR Resolution may be increased by digital-beam forming SAR in future

Ship Detection by 4-Component Scattering Power Decomposition

ALOS-PALSAR PLR 2008.10.09

HH VV

HV ★ Single-bounce surface scattering component

★ Double-bounce scattering component

★ Volume scattering component

★ Helix scattering component

Single-bounce Double-bounce Volume Helix

Ship Detection by 4-Component Scattering Power Decomposition

Single-bounce surface scattering component Double-bounce scattering component

Volume scattering power component

HH-VV Co-polarization Phase Difference

2012/09/20/ 08:28:10-12 (UT)

HH amplitude image HH-VV phase difference image

HH-VV Co-polarization Phase Difference

HH-VV phase difference image

Detection of Ship Wakes and Kelvin Waves

Ship wake

Ship

Ship wake V-shaped Kelvin waves

Smooth

Equilibrium rough surface

Water motion

Rough

Ship Wakes and Kelvin Waves: Gibraltar Strait

TSX StripMap HH 2007/07/09 06:29 UTC

Az.

Rg.

Ship Classification by SAR Passenger Ro-Ro Passenger Cargo

Battleships

LNG Tanker

Product Tanker

Battleship roles

Sushi

Fishing Boat

Pirates

Squid Fishing

SAR Images of Different Types of Ships: Tokyo Bay

VV

EMILIA

OCEAN SOUTH

NYK SILVIA

136 x 25 m

166 x 28 m

210 x 32 m

TSX SL 3.2 x 1.17 [m] (azimuth x slant-range)

Range range

azimuth

2008.12.06: TSX SL (Asc.) 3.2 x 1.17 (az x sr)

HH

187 x 23 m

TerraSAR-X over Tokyo Bay Azimuth

2011.05.30

CARIBE MAIDEN (General Cargo)

SAR Images of Different Types of Ships: Tokyo Bay

VV

EMILIA

OCEAN SOUTH

NYK SILVIA

136 x 25 m

166 x 28 m

210 x 32 m

TSX SL 3.2 x 1.17 [m] (azimuth x slant-range)

Range range

azimuth

2008.12.06: TSX SL (Asc.) 3.2 x 1.17 (az x sr)

HH

187 x 23 m

TerraSAR-X over Tokyo Bay Azimuth

2011.05.30

CARIBE MAIDEN (General Cargo)

Progress of ship classification is slow in comparison with detection, mainly because of resolution and complexity of imaging process.

Ship Classification Algorithms

As resolution is becoming finer in recent years, some algorithms have been emerging. The main approaches are as follows.

Feature Extraction

Feature-based Template Matching

Machine Learning

Multi-channel

Possible features include: • Length, width • Moments of inertia • Fractal measures • Wavelet coefficient • Target and shadow contour

• Image itself • Energy in brightest pixels • Polarimetric channel ratio • Attributed Scattering Center • Topographic Features

Pattern Matching SAR image

ISAR image

Feature Vector Method by GMV Spain: SIMONS

Reference parametric vectors by GRECOSAR ** simulator

* G. Margarit and A. Tabasco, “Ship classification in single-pol SAR images based on fuzzy logic,” IEEE Trans. Geosci. Remote Sens., vol. 49, no. 8, pp. 3129-3138, 2011. ** G. Margarit, J. J. Mallorqui, J. M. Rius, and C. Lopez-Martinez, “On the usage of GRECOSAR, an orbital polarimetric SAR simulator of complex targets, to vessel classification studies,” IEEE Trans. Geosci. Remote Sens., vol. 44, no. 12, pp. 3517-3525, 2006.

SIMONS (Ship Monitoring Service)*

SIMONS: Classification Results of ENVISAT-ASAR

* G. Margarit and A. Tabasco, “Ship classification in single-pol SAR images based on fuzzy logic,” IEEE Trans. Geosci.Remote Sens., vol. 49, no. 8, pp. 3129-3138, 2011.

ENVISAT-ASAR data: 30 m medium resolution

9 ship categories:

Oil tanker, Coaster, Bulk Carrier, Reefer, Cruise, Container, Carr Ferry, Medium fishing boat, Small fishing boat

Using 311 ships with AIS pols Classification accuracy ~ 70%

Important: Accuracy depends also on the number and types of categories !!!

SIMONS: Experiment in Tokyo Bay

NDA power-driven craft

S-band radar X-band radar

Ground-Truth Data Collection

AIS

Visual observation with a video camera

Yokohama

Tokyo Bay

Ground-truth data collection

NDA

TerraSAR-X Ground-Truth Data

2012/09/20/ 08:28:10-12 (UT) HH

Fumika

Shoei Maru

Tosei

Zenken Maru

Nippon Maru

Blue Ridge Highway

Hokuyo Maru Izumi Maru

Mie Maru

Vsg Pride

Chang Yu

Kyokuyo Maru Chang Yu

Mie Maru Kyokuyo Maru

Blue Ridge Highway Zenken Maru

Ysg Pride Izumi Maru

Hokuyo Maru

Shoei Maru Nippon Maru

JCG 1 JCG 1

No AIS 4

No AIS 1

No AIS 3

NDA

No AIS 7

Visible + No AIS

Visible + AIS

Ground-Truth by AIS + Visual

No AIS 4

Tosei

Fumika

No AIS 6

No AIS 5

No AIS 5

JCG 2

No AIS 2

Felicity Ace Felicity Ace

Yutoku Maru

Yutoku Maru No AIS 1

Name True Category: L/W [m]

Classified Category: L/W [m]

Photographs AIS + JCG

SAR image HH-polarization

Blue Ridge

Highway

Vehicles Carrier: 180/32

Class 3: Vehicles Carrier: 152.83/22.38

Zenken Maru

Waste Disposal Cargo: 98/18

Class 5: Coaster/LPG Tanker/Regional Cargo:

94.73/17.41

VSG-Pride

General Cargo: 91/15


105.01/22.38

Izumi Maru


Class 3: Vehicles Carrier: 116.81/9.95

Hokuyo Maru

Limestone Carrier: 123/20


103.42/7.46

Chang Yu



93.61/22.32

Blue: Correct Red: Incorrect

SIMONS: Classification Results

37

Name True Category: L/W [m]

Classified Category: L/W [m]

Photographs AIS + JCG

SAR image HH-polarization

Mie Maru General Cargo: 75/13

Class 5: Coaster/LPG Tanker/Regional

Cargo: 59.64/7.46

Kyokyuyo Maru

Cargo: 61/9 Class 5: Coaster/LPG Tanker/Regional

Cargo: 64.08/9.95

JCG 1* Patrol Craft: 27/5.6 Class 6: Patrol/Small Ship: 24.22/14.92

JCG 2** Patrol Craft: 23/6 (Stationary)

Class 6: Patrol/Small Ship:: 14.94/9.95

* JCG 1 (Japan Coast Guard 1): Cruising with high speeds at 51 deg. from range ** JCG 2 (Japan Coast Guard 2): Stationary

Blue: Correct Red: Incorrect

SIMONS: Classification Results

Accuracy: ~ 87 % 38

Classified 16 ships + 5 ships in the TerraSAR-X image over the Melilla Bay in Mediterranean

Lockheed Martin Canada: Hierarchical Classification

Length [m]

Prob

abili

ty D

ensi

ty F

unct

ion

Classification accuracy:

Frigate 77% Destroyer 96% Cruiser 82% Battleship 62%

Aircraft Carrier 75%

Same principle as SIMONS of GMV Spain, but hierarchical neural net classifier with high-resolution airborne SAR: Many parametric vector components

Target Recognition of Vehicles: Deep Learning Theory

Wang et. al., “ Application of deep-learning algorithms to MSTAR data,” Proc. IEEE IGARSS 2015, pp. 3743-3745. Chen et. al., “Classification using the deep convolution networks for SAR images, IEEE Trans. Geosci. Remote Sens., vol. 54, pp. 4806-4817, 2016.

Summary: Flow of Ship Monitoring System

1. Produce SAR image 2. Land/island masking 3. Apply ship detection algorithm

Image format is different depending on detection and classification methods

Mask out land, islands and other manmade objects from archive data or SAR image

CFAR: Constant False Alarm Rate ATM: Adaptive threshold Method MLCC: Multi-Look Cross- Correlation PolSAR: Polarimetric SAR

etc.

5. Check and change, if necessary, threshold

4. Apply wake detection algorithm

6. Comparison with AIS/ VMS data

Check if ship wakes are visible for improvement of detection accuracy

If the systems are not fully automatic, an operator checks ambiguity ghost images, false alarm images in rough seas under high sea states, others

Compare with AIS/VMS and/or maritime radar data and find non-AIS ships


7. Apply classification algorithms to non-AIS ships

Classify the ships’ category, and cruising speed, if possible


8. Reporting the results

Near real-time, if possible, is desirable

9. Action

Thank you for your attention !!!

introduction of ship detection technology using sar and ... kazuo ouchi (p… · 09/07/2007 ·...

Documents