a uav vision system for airborne surveillance

A UAV Vision System for Airborne Surveillance

M.Kontitsis, K. Valavanis

IEEE 2004 International Conference on Robotics and Automation

Technical University of Crete

University of South Florida

N. Tsourveloudis

Objectives

• Present a methodology for the design of a machine vision system for aerial surveillance by Unmanned Aerial Vehicles (UAVs)

• Identify specified thermal source• Perform these functions on board the

UAV in Real time• Flexible enough to be used in a variety

of applications

Machine Vision System

IR/NIR image Noise reduction

Feature extraction

(Size, Mean intensity)

Feature vectors classification

Alarm on/off

Persistence

Input Images

IR (3μm ~ 14μm)

8bit grayscale

Near IR camera (1μm ~ 3μm)

Noise Reduction

5x5 spatial Gaussian filter

+ Smoothes noise while preserving most of the features on the image

Feature Extraction

Size of region using a region growing algorithm

Mean intensity of region defined as

This module attempts to extract information about the regions on the image

Feature Vector Classification Subsystem

Mean Intensity of Region

Size of Region

Target Identification Possibility

Fuzzy Classifier

Mean Intensity Membership Functions

Grayscale values

HighMidLow

Region Size Membership Functions

Pixels

Small Medium Large

Objective ID Possibility Membership Functions

Output of the Fuzzy Classifier

Classification Example

Classification Result

p>0.8

0.5<p<0.8

p<0.5

Alarm raising

Persistent classification of a certain region as of High Possibility raises the alarm

The region that raised the alarm is pin-pointed by a red cross

The alarm stays on even if the thermal source is temporarily occluded by surroundings or lost due to violent camera vibrations

Alarm raising

Mechanism used : Alarm Registry

Region Coordinates Variable persistancei1,j1 p1

i2,j2 p2

…. ……..

in,jn pn

If pi > Ton => Activate alarm

If pi < Toff => Deactivate alarm

Complexity

Noise Reduction O(n2) for (nxn) image

Region Growing O(n2) for (nxn) image

Fuzzy Logic Classifier* O(nxm)

*in its current implementation

n inputs, m rules

Case Study: Forest fires

Adjusting membership functions manually

Classification Example (objective present)

Thermal source (fire)

Classification Result (objective present)

possibility>0.7 0.5< possibility <0.7 possibility <0.5

Classification Example (objective absent)

Classification Result (objective absent)

possibility>0.7 0.5< possibility <0.7 possibility <0.5

Classification Result (Video) (objective present)

Classification Result (Video) (objective absent)

Automatic Parameter Selection

aij bij cij dij for i=1 and j =1,2,3 which define the form of the membership functions of Mean Intensity

aijbij cij dij

Basic Elements of the Genetic Algorithm

Fitness function

fitness(x)=1 correct deactivation of the alarm

fitness(x)=0 in any other case

correct activation of the alarm

Chromosome => parameters x=(aij bij cij dij)

Basic Elements of the Genetic Algorithm

Selection operator selects individuals for mating as many times as the ratio of their fitness to the total fitness of the population

Crossover operator crossover probability pc=0.7

Mutation operator mutation probability pm=0.001

Mean Intensity M. F. as evolved by GA

Result (using GA for parameter selection)

Remarks

Adjustable for a variety of applications

Real time execution

Correct identification rate of about 90%

False alarms not entirely avoided (especially in the system evolved by the GA)