• K means local minimao Runs k means moreo Learn ocean and

whale centroids and initialize to those

• Better water removal

• 4-fold CV• 1 fold with at least 1 image

for each whale

Testing

Introduction Detection and Identification Algorithms

Dataset Results

The North Atlantic Right Whale is a critically endangered species that lives alongthe east coast of the United States. There are only an estimated 500 whales left inthe ocean and to aid conservation efforts, they must be tracked and cataloguedfrom aerial imagery [1,2].

Future Work

Credit: Woods Hole Oceanographic Institution Graphics

The task of identifying individual whales isusually performed by highly experiencedresearchers and marine biologists under stricttime and budget constraints.

Currently, the manual matching is conductedwith the aid of an online catalog and somecustom software [3] which is extremely timeconsuming.

[1] Kaggle Inc. Right Whale Recognition,https://www.kaggle.com/c/noaa-right-whale-recognition[2] NOAA Fisheries. North Atlantic Right Whales (eubalaena glacialis),http://www.nmfs.noaa.gov/pr/species/mammals/whales/north-atlantic-right-whale.html[3] DIGITS: Digital Image Gathering and Information Tracking System,http://www.neaq.org/conservation_and_research/projects/endangered_species_habitats/right_whale_research/right_whale_projects/monitoring_individuals_and_family_trees/identifying_with_photographs/digits.php

Raw Images

• Identify ROI• Save negative regions• Train detector

Cascade Object Detector

• Get 50 largest bounding boxes• Convert to mask

Whale Extraction• Dilate• Erode• Take largest blob

Mask Manipulation

• Detect SIFT features [4]• Apply PCA [5]

SIFT Extraction

• Convert RGB to HSV space• Random sample for cluster

centroids

Color Space Change

• K-means clustering, k = 4• Largest cluster is ocean• Invert to get whale mask

K-means Clustering

[4] VLFeat Team. (2013). VLFeat MATLAB Toolbox

http://www.vlfeat.org/index.html

This work is based on a Kaggle competition [1] that aims touse machine learning techniques to identify Right Whalesin a large dataset of aerial imagery.

# of images 11,000

# of training images 4,500

# of unique whales 447

Image resolution 4 – 17 MP

Poor lighting and

lack of contrast

from reflections on

water

Occlusions from

the sea foam

Distortion from the

water

Varying actions

and orientations

(breaching,

feeding)

• Pick whale with largest match %

SIFT Matching

• Remove uncommon features between images

Prune Features

• SVM feature is a single SIFT feature• Each SIFT feature is labeled for a

given whale• Entire image is then labeled as most

common label for SIFT features

SVM

Detection Identification

Detection

Identification

• SIFT comparison improvements to reduce false positives

• Continue working on SVM• Use other learning methods (e.g.

Convolutional Neural Networks)• Better SIFT features, or more

localized to certain parts of the whale (e.g. “face”, back, tail)

Ocean mask

K-mean mask

K-mean result

COD mask

COD result[5] Farag, A. A., & Elhabian, S. (2005). A Tutorial on Principal Component Analysis

http://dai.fmph.uniba.sk/courses/ml/sl/PCA.pdf

Color Space Detection Identification

ComputationTime

DetectionStep

Time(s/img)

K means ~60

COD ~10

SIFT ~5

IdentificationSteps

Time(s/img)

Pruning ~1

SIFT Matching ~80

SVM Testing ~500

Same whale:

Wrong whale:

Whale Detection and Identification from Aerial ImageryAditya Mahajan, Adrien Perkins

Department of Aeronautics and Astronautics, Stanford University

58 matches / 168 features

53 matches / 6654 features

Bad lighting will still show the whale in HSV space

HSV space candifferentiate whale, water and foam

Deep parts (like the tail)are also more apparent in HSV space

COD K-means

COD K-means

COD K-means