indoor simulation of wildfire detection and monitoring ... · indoor simulation of wild re...
TRANSCRIPT
Indoor Simulation of Wildfire Detection and Monitoringusing UAV
by
Hao Cheng Zhu
A thesis submitted in conformity with the requirementsfor the degree of Masters of Applied Science
Graduate Department of Aerospace EngineeringUniversity of Toronto
c© Copyright 2014 by Hao Cheng Zhu
Abstract
Indoor Simulation of Wildfire Detection and Monitoring using UAV
Hao Cheng Zhu
Masters of Applied Science
Graduate Department of Aerospace Engineering
University of Toronto
2014
Wildfires have been a major source of environmental and economical damage for Canada.
Wildfire detection and monitoring using UAVs have garnered great interest. This thesis
project investigates the use of colour-based vision feedback for tracking a dynamically ex-
panding wildfire simulated in a fully coloured environment. A custom Indoor Simulation
Platform is developed to test and debug various navigation algorithms using a physical
quadrotor. The proposed fire perimeter tracking navigation algorithm has been tested
in both computational simulation and on the Indoor Simulation Platform, and could be
used in an outdoor mission with high confidence of success.
ii
Acknowledgements
First and foremost, I would like to thank Dr. Hugh H.T. Liu for his continuous support
and guidance over the course of these two years. His mentor-ship has been critical in
the completion of this thesis project. Furthermore, I would like to thank my two RAC
members, Dr. Grant and Dr. Damaren, for their feedbacks and comments throughout
this project. I would also like to thank Yasir Malang, a summer intern at the FSC
lab, for his contribution in the initial construction of the Indoor Simulation Platform.
Additionally, Yasir assisted in the configuration of the ground station computer. I would
also like to acknowledge Marlon Koga, another summer intern at the FSC lab, for his
contribution in parameter estimation based on experimental data. Finally, I have to give
special thanks my friends and family for their continued encouragement and support over
these two years, providing me with hope and motivation that are necessary to complete
my project.
iii
Contents
1 Introduction 1
1.1 Background and Motivation . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Research Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.2 Constraints and Limitations . . . . . . . . . . . . . . . . . . . . . 3
1.3 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3.1 Fire Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3.2 Quadrotor Dynamic Model . . . . . . . . . . . . . . . . . . . . . . 5
1.3.3 Edge Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3.4 Edge Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3.5 Visual Servoing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2 Fire Perimeter Tracking 9
2.1 Fire Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1.1 Static Elliptic Approximation . . . . . . . . . . . . . . . . . . . . 10
2.1.2 Dynamic Elliptic Approximation . . . . . . . . . . . . . . . . . . 12
2.1.3 Approximation under Non-Uniform Conditions . . . . . . . . . . . 17
2.2 Edge Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3 Perimeter Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3.1 Reverse Elliptic Approximation . . . . . . . . . . . . . . . . . . . 25
2.3.2 Full Perimeter Reconstruction . . . . . . . . . . . . . . . . . . . . 27
2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3 Quadrotor Navigation 31
3.1 Quadrotor Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.1.1 Translational Dynamics . . . . . . . . . . . . . . . . . . . . . . . 32
3.1.2 Rotational Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . 33
iv
3.1.3 Motion Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.2 Quadrotor Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.3 PBVS Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.3.1 Reference Frame Transformation . . . . . . . . . . . . . . . . . . 38
3.4 Parameter Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4 Indoor Simulation Platform 46
4.1 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.2 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.3 Experimentation Phases . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
5 Numerical Results 57
5.1 Quadrotor Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.1.1 Controller Validation . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.1.2 Way-Point Navigation . . . . . . . . . . . . . . . . . . . . . . . . 58
5.1.3 Quadrotor Fire Perimeter Tracking . . . . . . . . . . . . . . . . . 62
5.2 Fire Perimeter Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . 63
6 Experimental Results 65
6.1 Quadrotor Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
6.1.1 Way-Point Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . 65
6.1.2 Target Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
6.1.3 Line Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.1.4 Dynamic Edge Tracking . . . . . . . . . . . . . . . . . . . . . . . 71
6.2 Fire Perimeter Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . 73
6.3 Outdoor Extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
7 Conclusions 78
References 78
v
List of Tables
2.1 Cell properties table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.1 Hardware specifications for FPV259 quadrotor frame . . . . . . . . . . . 49
4.2 Hardware specifications for ArduPilot Mega 2.6 . . . . . . . . . . . . . . 50
vi
List of Figures
2.1 The ROS-based elliptic approximation method . . . . . . . . . . . . . . . 10
2.2 Simulated fire propagation after 5, 20, and 100 seconds . . . . . . . . . . 12
2.3 The contour that identifies the new fire front lies tangent to two adjacent
ellipses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4 Geometric method of finding the tangent between two circles. . . . . . . 14
2.5 Fire perimeter simulation using dynamic elliptic approximation . . . . . 17
2.6 Finding the tangent between two different circles. . . . . . . . . . . . . . 18
2.7 Fire perimeter simulation under non-uniform conditions . . . . . . . . . . 20
2.8 The top down view of a wildfire projected onto the ISP. . . . . . . . . . . 21
2.9 The propsed process of edge extraction . . . . . . . . . . . . . . . . . . . 21
2.10 Side profile of quadrotor and location of new way-point on image . . . . 24
2.11 The spiral path created by the UAV when tracking a continuously expand-
ing fire perimeter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.12 The reverse elliptic approximation result and comparison . . . . . . . . . 27
2.13 The path of a UAV tracking the fire front over three time steps . . . . . 27
3.1 The “X” configuration used by the ArduPilot Mega [26] . . . . . . . . . . 32
3.2 Principle of quadrotor dynamics . . . . . . . . . . . . . . . . . . . . . . . 32
3.3 Controller structure used for the quadrotor . . . . . . . . . . . . . . . . . 37
3.4 PBVS flow chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.5 The image reference frame . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.6 The pixel-centimetre ratio at a certain altitude . . . . . . . . . . . . . . . 39
3.7 The contribution of quadrotor’s pitch angle to total pitch angle difference
for target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.8 The experimental parameter estimation block diagram [28] . . . . . . . . 44
4.1 The overall system schematic of the ISP. . . . . . . . . . . . . . . . . . . 47
4.2 The environment projection platform with angled projection. . . . . . . . 47
4.3 An example of the projected wildfire environment. . . . . . . . . . . . . . 48
vii
4.4 The two quadrotor frames that have been used in this project . . . . . . 49
4.5 Structure of the ISP software system . . . . . . . . . . . . . . . . . . . . 51
4.6 The interaction between ground station and vision client for edge extraction 52
4.7 The reverse elliptic approximation result and comparison . . . . . . . . . 53
4.8 The test plan for simulation and physical experiment . . . . . . . . . . . 55
5.1 Step response for angle tracking . . . . . . . . . . . . . . . . . . . . . . . 58
5.2 The simulated way-point based quadrotor mission . . . . . . . . . . . . . 59
5.3 Yaw angle tracking in simulated way-point navigation mission . . . . . . 59
5.4 Altitude tracking in simulated way-point navigation mission . . . . . . . 60
5.5 Position tracking in simulated way-point navigation mission . . . . . . . 60
5.6 Roll and pitch angle tracking in simulated way-point navigation mission . 61
5.7 Lateral error in simulated way-point navigation mission . . . . . . . . . . 62
5.8 Simulated fire edge tracking with quadrotor . . . . . . . . . . . . . . . . 62
5.9 Finding local fire perimeter position using reverse elliptic approximation 63
5.10 Full perimeter approximation using elliptic curve fitting technique . . . . 64
6.1 Way-point tracking experiment position results . . . . . . . . . . . . . . . 66
6.2 Way-point tracking experiment heading angle and altitude results . . . . 67
6.3 The target used in the PBVS experiment . . . . . . . . . . . . . . . . . . 67
6.4 Determining the centre position of the target object via thresholding . . 68
6.5 Top down view of the target estimation test . . . . . . . . . . . . . . . . 69
6.6 Top down view of the PBVS tracking test . . . . . . . . . . . . . . . . . 70
6.7 X and Y-axis PBVS tracking result . . . . . . . . . . . . . . . . . . . . . 71
6.8 Static straight edge being projected onto the glass floor . . . . . . . . . . 71
6.9 Top down view of static line tracking . . . . . . . . . . . . . . . . . . . . 72
6.10 The initial state of fire perimeter and the segment projected . . . . . . . 73
6.11 The progression of the dynamically growing fire in three time frames . . . 73
6.12 Dynamic edge tracking result . . . . . . . . . . . . . . . . . . . . . . . . 74
6.13 Detected edges and approximation of the segment . . . . . . . . . . . . . 74
6.14 Comparison of estimated, projected, and actual fire perimeters . . . . . . 75
6.15 FSC Octorotor flying a aerial imaging mission with full payload . . . . . 76
6.16 Edge extraction of outdoor aerial wildfire image . . . . . . . . . . . . . . 77
viii
Nomenclature
ROShead Rate of Spread in the direction of the wind
ROStail Rate of Spread opposite of the wind
ROSside Rate of Spread perpendicular to the wind
a Major axis of ellipse
b Minor axis of ellipse
p Distance between ignition point and centre of ellipse
θwind Direction of the wind
Tellipse Transformation matrix that transforms ellipse into a circle
Ci Centre of circles
Pi Intersection points between tangent line and circles
ri Radius of circles
ξ Neighbouring pixel
c(ξ, x) Geometric closeness between two pixels
s(f(ξ), f(x)) Photometric similarity between two pixels
Gx, Gy Gradient strength in x and y direction
F1−4 Thrust from each motor
θ, φ, ψ Pitch, roll, yaw Euler angle
p, q, r Roll, pitch, yaw rate in body-fixed frame
x, y, z x, y, z acceleration in inertial frame
U1−4 Intermediate input values
Ω1−4 Rotational velocity of each motor
Ixx, Iyy, Izz Moment of inertia for x, y, and z axis
ix
Acronyms
FSC Flight Systems and Control lab
UAV Unmanned Aerial Vehicle
LASE Low Altitude Short Endurance
ISP Indoor Simulation Platform
PBVS Position Based Visual Servoing
IBVS Image Based Visual Servoing
ROS Rate of Spread
ESC Electronic Speed Controller
RGB Red-Green-Blue colour space
HSV Hue-Saturation-Value colour space
APM ArduPilot Mega
PID Proportional-Integral-Derivative
PI Proportional-Integral
IMU Inertial Measurement Unit
GPS Global Positioning System
MP Mission Planner
LiPo Lithium Polymer
DCM Direction Cosine Matrix
x
Chapter 1
Introduction
1.1 Background and Motivation
Canada experiences over 5,000 wildfire occurrences on average each year, affecting al-
most 2 million hectares of land [1]. In 2014, the area damaged by wildfires stands at
a staggering 3.5 million hectares in less than 4,000 occurrences. Real-time monitoring
and early detection of wildfires is a very important field of research due to the massive
environmental and economic damage done by wildfires around the world and in Canada.
Current wildfire detection and monitoring methods include using ground-based watch
towers, satellite imaging, as well as manned aircraft missions. Ground-based watch tow-
ers are of limited use due to their inflexible nature. Even with more modern automatic
fire and smoke detection systems, the detection range for each tower is restricted to ap-
proximately 20km in good visibility [2]. Satellites used for fire detection can be divided
into Low Earth Orbit (LEO) satellites and geostationary satellites. LEO observation
satellites have an orbiting period, which limits their value in tracking wildfires in real
time. The LEO Landsat satellites can provide a resolution of 15m per pixel, but have an
orbiting period of 14 days [3]. The Geostationary Operational Environmental Satellites
(GOES), in contrast, can provide an updated image every 15-30 minutes, but the reso-
lution is only 1km for visual image and 4km for thermal images [3]. In addition, satellite
images can only be taken from a fixed angle, making them impossible to actively avoid
any smoke or cloud cover obstructions. Manned aircraft are much more manoeuvrable
compared to satellites, flies below clouds and can avoid smoke; however, manned aircraft
missions can be risky for the operators and these missions are expensive to carry out.
The use of unmanned aerial vehicles (UAVs) to complement existing wildfire detection
and monitoring methods such as manned missions has become an active area of research
due to the unique attributes of a UAV mission.
1
Chapter 1. Introduction 2
Low altitude UAVs are greatly desired for wildfire detection and monitoring missions
because they can safely fly at an altitude that can avoid cloud cover. They are able to
survey the field at close range and take high resolution images. Low altitude UAVs are
relatively small in size compared to some of the high altitude ones used by the military;
therefore, they generally have shorter endurance. Multirotor is one type of low altitude
UAV that has very high manoeuvrability and is capable of vertical take-off and landing.
They are very useful in missions that have forested areas where sufficiently large clear
areas for take-off and landing are hard to find.
In general, the development of vehicle and its fire detection function needs to be
tested at each design stage, from numerical simulations to field experiments. Numerical
modelling and simulation may not be accurate enough, and field testing is time consum-
ing. Thus, the development of an indoor test platform for hardware-in-loop simulation
is proposed to bridge the gap, and to provide an efficient design-validation cycle.
1.2 Literature Review
Fire edge detection using UAVs requires the integration of multiple sub-systems and spans
multiple areas of research. These components and areas of reserach include environment
simulation, micro-UAV control, autonomous edge tracking, vision-based edge detection,
and navigation using visual servoing.
1.2.1 Fire Model
A fire model in some form is necessary in the simulation in order to move the navigation
algorithm from the more deterministic line tracking mission to a more stochastic envi-
ronment. The so-called physical fire model and approximation fire model are two main
types of methods to describe the spread of a fire. Although they both use cell-based sim-
ulation where each grid cell contains a number of states, such as unburned, burning, and
burnt status, they differ fundamentally in how these states are determined. A physical
fire model, such as the Rothermel model, calculates the temperature of each cell based
on physical parameters of the cell [4]. Fuel parameters, including the mineral content,
moisture content, as well as vegetation density, and environment parameters, including
the wind speed, wind direction, as well as terrain incline, are used as direct inputs to
the physical fire model. The physical fire model uses these input parameters and vari-
ous thermal equations to calculate the temperature of each cell at every time interval.
The states of these cells are assigned based on the temperature. For example, a cell is
Chapter 1. Introduction 3
labelled as burning when it reaches a certain temperature threshold for a period of time.
In general, the physical fire model is more computationally intensive, but it is also more
accurate than other approximation methods. The physical fire model is used when the
estimated temperature of each cell is desired.
The approximation fire model, on the other hand, uses geometry to estimate the
location of the fire perimeter. The most common and most basic approximation method
is the elliptic approximation. The simplest case of an elliptic approximation is when all
parameters are constant and uniform over the simulated area. The fire expands out from
the original ignition point in an ellipse shape depending on the wind direction and speed.
The wind parameters are converted into head, tail, and side rates of spread (ROS) before
being passed into the ellipse approximation model. These ROS would be used to calculate
the angle, major-axis, and minor-axis of the ellipse that would represent the perimeter
of the fire [5]. This perimeter would then be overlaid onto the simulation field consisting
of grid cells, and the states of these cells are determined by where the cells are located
relative to the elliptic perimeter, such as in the simulation developed by Trunfio [6]. The
elliptic approximation method could be extended to include changing wind parameters
and non-uniform fuel parameters.
The type of fire model to be used for this project is mainly determined by the com-
plexity and accuracy requirement of the fire simulation. The environment simulation for
this project does not have to be overly complex since the ISP is largely a proof of concept.
The approximation model would be the better choice over the physical model due to its
simplicity. Additionally, the output of the fire model for this project is the position of
the fire perimeter, which is direct output of the approximation fire model. While the
approximation model used in this particular project will only take into account a limited
number of field conditions, the model itself is expandable to make the simulation much
more realistic.
1.2.2 Quadrotor Dynamic Model
Although fixed-wing aircrafts are better suited for outdoor fire detection and monitoring
missions, they cannot be operated in the limited indoor lab space. As a result, the more
manoeuvrable quadrotors are used to in the indoor set up. A dynamic model for the
quadrotor is needed for controller development in both simulation and on the physical
vehicle. The quadrotor dynamic model has been presented by many researchers, both
in the original non-linear and linearized form. The full non-linear dynamic model of a
quadrotor can be derived from Newton’s Second Law, as presented by many researchers
Chapter 1. Introduction 4
including Leishman and Beard [7][8]. The Stanford Testbed of Autonomous Rotorcraft
for Multi-Agent Control (STARMAC) team presented a simplified non-linear model as
well as a linearized quadrotor model for simulation and development [9]. A small roll
and pitch angle assumption is used in the simplified non-linear model to reduce the need
of a rotation matrix between body-fixed frame angles and inertial frame angles. Small
angle approximation is used during linearization to make the model simple and straight
forward. This linearization is done at the equilibrium state of hover, and the linearized
model describes the behavior of the quadrotor within a small roll and pitch angle limit.
The same derivations from basic physic laws can be used to find the quadrotor dy-
namic model. This project will adopt the simplified non-linear model in mathematical
simulation for a balance of accuracy and complexity.
1.2.3 Edge Detection
While many of the edge tracking strategies proposed assume that the edge information is
available, it is important that a realistic edge detection and extraction strategy is imple-
mented to complete the feedback loop. Visual cameras will be the feedback sensors used
in this project. A number of different image processing algorithms and filters can help
with edge extraction from a visual image. Canny Edge Detector, originally proposed by
Canny [10], is the most common image processing method for detecting edges. Canny
Edge Detector utilizes the colour gradient of an image to find the areas that have the
highest contrast, or greatest change in colour. These areas are extracted as edges via hys-
teresis. Canny Edge Detector is very sensitive to noise, which means filtering the image
beforehand is important if a specific edge is desired. Thresholding is a common filtering
technique used to extract an object of interest that has a unique colour. Thresholding
works the best when the object’s colour makes it stand out from the background. A more
advanced thresholding technique is described by Hong [11]. The thresholding technique
described by Hong uses fuzzy sets to determine the object of interest. Qiu [12] presented
an image based autoadaptive edge detection technique that is specifically designed to
detect the edges of a real flame, based on the luminance property of a fire. Multiple
image processing techniques can be combined to increase accuracy and robustness, such
as the curve tracking algorithm proposed by Chen [13]. In some cases, categorizing the
detected edges based on their shapes is just as important as detecting them. Generic
Edge Tokens (GET) is one of the many methods to organize detected edges [14]. The
GET model contains a set of segments and partition points, and can be applied to any
edge. These edges are then classified into one of four types of straight lines, or one of
Chapter 1. Introduction 5
four types of curves [14].
For the purpose of this physical experiment conducted in the ISP, Canny Edge De-
tector will be used in conjunction with various filters to ensure that the fire edge is the
only edge being extracted. The exact method of image processing will be determined
based on the actual experimental set up and the quality of the images collected.
1.2.4 Edge Tracking
Aerial fire edge tracking is often simplified into a line following problem, as many re-
searchers, such as Casbeer and Sujit [15][16], make the assumption that the fire can be
defined by a clear distinctive edge in their papers regarding fire edge tracking. Casbeer
presented a fire edge tracking method that would keep the UAV a certain distance away
from the fire to minimize damage. His method assumes that the edge of the fire can be
specified by a curve and that it is in view at all times. In the case where the fire edge
is not guaranteed to be in view of the camera, Kawano [17] proposed a line following
algorithm with a dead-reckoning technique that can estimate the position of the robot.
The dead-reckoning technique computes the position of a robot by taking into account
of its velocity vector and time passed since previous update. This method is relatively
accurate on ground robots where the velocity with respect to ground can be accurately
measured. Dead-reckoning may not be as accurate on UAVs as ground robot due to the
increased uncertainty with the operating environment, such as effect of gusts and air
density. Holt and Beard [18] presented a road following algorithm for UAVs based on
proportional navigation. The experimental results included show good tracking accuracy.
Simpler control strategies, such as a bang-bang controller, have been presented by various
researchers including Clark [19] and Kemp [20] to be used as benchmarking baseline for
other more complex algorithms.
The environment simulation created in the ISP is purely visual based. The UAV does
not need to maintain a certain distance away from the fire for safety reasons, as described
in Casbeer’s paper. Therefore, the tracking algorithm used for this project will attempt
to ensure the UAV is on top of the fire edge at all times, using a combination of Casbeer’s
method for heading control and a simpler bang-bang controller for lateral error control.
The advantage of a bang-bang lateral error controller is that it is easy to create an offset
for the vehicle, with or without an angled camera. In a real fire tracking mission, it
would be a good idea for the UAV to fly a certain distance away from the actual fire to
prevent heat damage and turbulence. The fire in the simulated environment is assumed
to be expanding much slower than the velocity of the UAV in the ISP; thus, it can be
Chapter 1. Introduction 6
assumed that the fire edge is in view of the camera at all time.
1.2.5 Visual Servoing
Visual servoing is a technique that can be applied to different types of mobile robots,
including UAVs. There are two types of visual servoing techniques for navigation: Po-
sition Based Visual Servoing (PBVS) and Image Based Visual Servoing (IBVS). PBVS
feedback requires the knowledge of the absolute position of both the vehicle and the tar-
get, whereas IBVS only needs the knowledge of relative position between the two. Many
researchers, such as Rangavjhala [21], Thuilot [22], and Bachiller [23], have presented
papers on PBVS control for robotics. On the other hand, the IBVS technique has been
used by researchers such as Begum [24] and Fu [25], on mobile robotics navigation. The
total loss of target is a major problem for navigation based on visual servoing. Fu at-
tempts to negate this problem with the use of an odometry-based fail-safe system. In
this case, the robot would continue to navigate using on board Lidar sensors.
Both visual servoing techniques can be used for the navigation control of a UAV to
track the edge of a fire. The ultimate goal of fire edge tracking is to gather location data
of the fire edge. For this purpose, PBVS would be the better choice because it computes
the target’s global position while navigating. IBVS, on the other hand, only calculates
the relative position of the fire edge, which means an extra step is needed to find the
global position of the fire edge afterwards.
1.3 Research Objectives
1.3.1 Objectives
The primary goal of this thesis project is to develop a wildfire perimeter tracking al-
gorithm for UAVs. This trajectory tracking algorithm needs to be able to detect the
location of a dynamically expanding fire perimeter, and guide the UAV to fly along the
perimeter over the course of the mission. This algorithm should be tested in both com-
putational simulation, such as MATLAB, and a hardware-in-the-loop environment with
a physical aerial vehicle in a simulated environment using visual feedback. The following
two objectives summarizes this goal:
1. To design a fire perimeter tracking algorithm that makes the aerial vehicle follow
the expanding fire edge and relay position data of the fire to the ground station,
and
Chapter 1. Introduction 7
2. To develop an integrated hardware-in-the-loop Indoor Simulation Platform (ISP)
that allows the testing of such navigation algorithms inside the lab.
1.3.2 Constraints and Limitations
One of the primary requirements of the navigation algorithm is that it has to make the
UAV track the edge of a wildfire. The sensors that could be used by a UAV to detect a
wildfire in real life are visual or thermal cameras. In fact, both types of cameras are often
used in the real world scenario to provide backup and to increase detection accuracy. The
extraction of a fire edge from a thermal image and a visual image is similar, although
edge extraction is more robust with thermal image due to less background noise. A visual
camera is used for this thesis project because the it is not practical to generate a simulated
environment with temperature difference indoors on the first iteration of the ISP, which
is only a proof of concept system. This indoor constraint limits the fire environment
to a simplified representation of an actual wildfire. Specifically, the environment used
for testing will have varying wind conditions, but uniform and constant vegetation and
terrain parameters.
1.4 Thesis Outline
This thesis includes five chapters. Chapter 1 is an introduction to the problem that
this research project attempts to solve. It also provides a summary of related work
previously done by other researchers. Chapter 2 describes the fire model used to generate
a simulation environment, provides an edge extraction algorithm that can detect the
fire edge from visual images, and proposes a fire perimeter reconstruction technique
to analyze the data captured by the UAV. Chapter 3 presents the quadrotor model,
control structure, and the navigation algorithm developed for experimentation. Chapter
4 contains an overview of both the hardware and software components of the Indoor
Simulation Platform that is being developed for the experiments. Chapter 5 presents the
numerical results of the quadrotor controller tests, as well as the fire edge tracking tests
in MATLAB simulation. Experimental results for both preliminary tests, such as target
position estimation and PBVS target tracking, and the final fire edge tracking tests on
the ISP can be found in Chapter 6. Finally, Chapter 7 provides a summary for this
project and discusses possible future research.
Chapter 2
Fire Perimeter Tracking
A fire model is needed in order to generate a simulated environment with a dynamically
growing wildfire. Once this simulated environment is created, the UAV would use an edge
detection algorithm to extract the edge of the fire perimeter for navigation purposes. Data
collected from the quadrotor tracking mission would be used for perimeter reconstruction
for the local segment that is tracked, and potentially for the entire fire.
2.1 Fire Model
The physical model and the approximation model are two common ways to create a
wildfire simulation. The elliptic approximation model is chosen for this research project
for two reasons. First of all, the scope of this project constrains the wildfire simulation
to constant fuel parameters. This means the vegetation density, wind speed, and wind
direction are the same across the entire simulation space, which in turn means that the
rate of spread at every point in the simulated field is the same. This suggests that
the elliptic approximation fire model is sufficient of the purpose of this project, and the
additional complexity of the Rothermel model is not needed. Secondly, the desired output
of the fire simulation is a top down view of an expanding fire perimeter, as opposed to
temperature at each point on the grid. The elliptic approximation model generates the
fire perimeter directly, whereas the physical fire model needs to compute the temperature
first before creating the fire perimeter. Thus, the elliptic approximation model is chosen
to generate the wildfire simulation.
8
Chapter 2. Fire Perimeter Tracking 9
2.1.1 Static Elliptic Approximation
The basis of the elliptic approximation model is that under a no wind condition, a field
with constant fuel would result in the same rate of spread (ROS) in all directions. This
would mean that the fire perimeter could be approximated as a circle. Under windy
conditions, the front ROS, which has the same heading angle as the wind direction, will
increase. The tail ROS, which is opposite to the wind direction, will decrease. The
two side ROS, which are perpendicular to the wind direction, will remain the same.
The circular fire perimeter will turn into an ellipse in this case. Figure 2.1 shows the
relationship between the ignition point, the ROS, the wind direction, and the elliptic
shape of a fire perimeter.
Figure 2.1: The ROS-based elliptic approximation method
The ellipse created by the wind has the same direction as the wind direction. The
following equations calculate the total spread of the fire in all four directions for every
time period dt: headcurrenttailcurrent
sidecurrent
=
headprevtailprev
sideprev
+ ∆t×
ROSheadROStail
ROSside
(2.1)
a =headcurrent + tailcurrent
2(2.2)
b = sidecurrent (2.3)
p = a− tailcurrent (2.4)
Chapter 2. Fire Perimeter Tracking 10
C = (Ix + p× cos(θ), Iy + p× sin(θ)) (2.5)
where a and b are the major and minor axes of the ellipse respectively. For plotting
purposes, the centre of the ellipse C is needed, which is distance p away from ignition
point I = (Ix, Iy). The ellipse can be created using the general ellipse formula:
(x cos(θwind) + y sin(θwind))2
a2+
(x sin(θwind)− y cos(θwind))2
b2= 1 (2.6)
The pseudo code for a fire simulation using static elliptic approximation method is
as follows:
Create simulation field of x× y grid cellsSet all cell values to 0Set all cell state to unburnedwhile time is less than final time do
Calculate current head, tail, and side spread from Ignition Point (Eqn 2.1)Calculate a, b, p, C based on current head, tail, and side distances (Eqn 2.2 - 2.5)Update ellipse perimeter arrayfor Each cell in the simulation field do
if cell is on the ellipse perimeter AND cell is unburned thenSet cell state to burning
else if cell state is burning thenIncrement cell value
else if cell state is burning AND cell value reaches threshold thenSet cell state to burnt
end ifend forColour simulation field with predetermined colour schemeIncrement time
end while
The simulated field is a square grid with 100 cells on each side. Each cell is associated
with one of three states: unburned, burning, and burnt. Each cell is also associated with
a colour and a value that represents how much fuel has been burned. The unburned cells
are coloured green and have a numerical value of 0, representing healthy vegetation. The
burning cells start with yellow colour with a numerical value of 8, but quickly changes
to orange and then red where the value would eventually become 10. This is made
to distinguish between those that have just started burning and those that have been
burning for a while. The burnt cells have a brown colour with a numerical value of 10,
and are not considered to be on fire because they have used up all available fuel. The
comparison between cell colour, value, and state can be found in Table 2.1.
Chapter 2. Fire Perimeter Tracking 11
Value Colour State0 Green Unburned
1 - 7 NA NA8 Yellow Burning9 Red Burning10 Brown Burnt
Table 2.1: Cell properties table
Figure 2.2 demonstrates how elliptic approximation fire model propagates the fire
perimeter over 100 seconds. The environment used for this demonstration is a 100 × 100
grid.
(a) t = 5s (b) t = 20s (c) t = 100s
Figure 2.2: Simulated fire propagation after 5, 20, and 100 seconds
2.1.2 Dynamic Elliptic Approximation
The elliptic approximation can be expanded with dynamic wind conditions. This means
that while the fuel parameters on the ground remain constant and uniform, the wind
speed and direction can vary with time. The effect of the wind on every grid cell at each
time step is uniform. Under such conditions, the fire edge cannot be approximated by
a single ellipse. Instead, elliptic approximation would need to be applied to a number
of discretized points along the original fire front. The Dynamic Elliptic Approximation
includes three steps:
1. Discretize the fire front at time t = 0 into a number of points.
2. Each of the discretized points act as a new ignition point, and apply basic elliptic
approximation on each of these ignition points.
Chapter 2. Fire Perimeter Tracking 12
3. Find a contour that envelops all the individual new ellipses, and this contour be-
comes the new fire front at time t = 1.
The calculations for the small individual ellipses are the same as the static elliptic
approximation, which requires the knowledge of ignition point, wind speed, and wind
direction. The contour that envelops these small ellipses would lie tangent to all of them;
thus, the task of finding this tangent can be broken down into finding tangent lines
between every two adjacent ellipses, as illustrated in Figure 2.3.
Figure 2.3: The contour that identifies the new fire front lies tangent to two adjacentellipses
Points P1,t and P2,t are two points on the original fire perimeter at time t. They
become the two adjacent ignition points after discretization has been applied to the
original fire perimeter. After one time period has passed, the two ignition points expand
based on the basic elliptic approximation rules and becomes points P1,t+1 and P2,t+1
respectively. The line that is tangent to P1,t+1 and P2,t+1 forms a segment of the new fire
perimeter at time t+ 1.
The method that is used to find these tangent lines has three steps:
1. Find and apply the transformation matrix T to transform the two adjacent ellipses
into circles.
2. Find a line that is tangent to both circles.
3. Apply inverse transformation matrix T−1 to transform the two circles back into the
original ellipses, along with the newly determined tangent line.
Chapter 2. Fire Perimeter Tracking 13
The transformation matrix has 2 components. It must first rotate the ellipses back
to a 0 degree position, and then compress the major axis of the ellipses by a factor of
b/a. The transformation is defined by Tellipse = TCompTRot:[XT
YT
]= Tellipse
[x
y
](2.7)
TComp =
b/a 0
0 1
TRot =
cosθ sinθ
−sinθ cosθ
(2.8)
where θ is the angle of the ellipses, which is the same as the wind direction; a and b
are the major and minor axes of the ellipse respectively;[x y
]Tare the points on the
original ellipse; and[XT YT
]Tare the points on the circle after the transformation.
The entire ellipse is first rotated clockwise by an angle of θ such that the new ellipse is
horizontal with respect to the x-axis. The ellipse is then shrunk by a factor of b/a along
its major axis so that it becomes a circle with radius r = b. Once the two ellipses have
been transformed into two circles, a line segment representing the outer tangent of these
two circles needs to be calculated. Figure 2.4 shows the geometric method of finding a
tangent line between two circles with same radius.
Figure 2.4: Geometric method of finding the tangent between two circles.
The two circles are centred at C1 = (xc1, yc1) and C2 = (xc2, yc2), and both have radius
r. Line P1P2 is the line that is tangent to both circles. AP1 and BC2 are perpendicular
Chapter 2. Fire Perimeter Tracking 14
to the x-axis. 6 AP1C1 and 6 BC1C2 are equal and are named 6 α. 6 α can be described
by the following relationships based on 4BC1C2:sin(α) = yc2−yc1d
cos(α) = xc2−xc1d
(2.9)
The intersection points P1 and P2 can be determined using the knowledge of sin(α)
and cos(α).
P1,x = xc1 − r × sin(α)
P1,y = yc1 + r × cos(α)
P2,x = xc2 − r × sin(α)
P2,y = yc2 + r × cos(α)
(2.10)
Once P1P2 are found, an inverse transformation Tinv = R−1ellipse would be applied so
that this line is tangent to the two original ellipses. In order to create a fire simulation
based on dynamic elliptic approximation, the static elliptic approximation procedure
needs to be converted into a function. This function would take the following inputs:
ignition point, head ROS, tail ROS, and side ROS; and would provide the following
outputs: 2× n array of x and y position of the discretized ellipse perimeter.
function createEllipse(I, ROShead, ROStail, ROSside)Calculate head, tail, and side spread distancesCalculate ellipse parameters a, b, p, C based on head, tail, and side spread distancesreturn Ellipse parameters
end function
Using the function above, a fire simulation with the dynamic elliptic approximation
can be created.
Figure 2.5 demonstrates the use of the dynamic elliptic approximation to simulate
fire perimeter growth over one time step.
The black circle is the original ignition point of the fire. The black ellipse is the original
fire perimeter. The small blue ellipses are the results of individual elliptic approximations
applied to the discretized fire perimeter. Finally, the red ellipse is the new fire perimeter,
which envelopes the individual ellipses.
Chapter 2. Fire Perimeter Tracking 15
Require: createEllipseCreate simulation field of x× y grid cellsSet all cell values to 0Set all cell state to unburnedwhile time is less than final time do
if time is 0 thenCall createEllipse on original ignition pointCompute perimeter array based on returned ellipse parameters
elsefor Each point in perimeter array at time t-1 do
Call createEllipse with selected point as ignition pointSave ellipse parameters into parameter array
end forfor Each entry in ellipse parameter array do
Calculate outer tangent line between current and next ellipse using Eqn 2.9and 2.10
Save tangent line points in perimeter arrayend forDetermine cell state using same logic as static case
end ifColour simulation field with predetermined colour schemeIncrement time
end while
Chapter 2. Fire Perimeter Tracking 16
Figure 2.5: Fire perimeter simulation using dynamic elliptic approximation
2.1.3 Approximation under Non-Uniform Conditions
The dynamic elliptic approximation can be expanded to simulate non-uniform conditions.
A non-uniform condition is defined as wind and fuel conditions that changes with both
time and location. At any time instance t, the wind speed, wind direction, and fuel
parameter on the simulation field can be different depending on where the wind data is
being measured. For example, if a part of the simulation field is set to contain higher
density vegetation, then there is more fuel for burning at that location compared to rest
of the field. Generally, the rate of spread increases when there is more fuel present or
the wind speed is higher, and vice versa.
The difference between dynamic elliptic approximation under uniform conditions and
non-uniform conditions is that two different transformation matrices, and by extension,
two inverse transformation matrices, are needed. The environment parameters could
be different between two neighbouring cells, resulting in different ROS for each cell.
This implies that the radii of the two neighbouring circles after transformation could be
different. As a result, the general version of the tangent line equations are used. Figure
2.6 shows the geometric method of finding tangent line between two circles with different
radii.
Chapter 2. Fire Perimeter Tracking 17
Figure 2.6: Finding the tangent between two different circles.
Once again, the two circles are centred at C1 = (xc1, yc1) and C2 = (xc2, yc2). Circle
1 has radius r1 and Circle 2 has radius r2. Line P1P2 is the line that is tangent to both
circles. AP1 and BC2 are perpendicular to the x-axis; C1P1 and C2P2 are perpendicular
to the tangent line P1P2; and C1C is parallel to the tangent line P1P2. 6 AP1C1 is 6 δ,
where 6 δ = 6 α + 6 β. 6 α and 6 β can be described by the following relationships based
on 4BC1C2 and 4C1C2C:
sin(a) = yc2−yc1d
cos(a) = xc2−xc1d
sin(b) = rc2−rc1d
cos(b) = C1Cd
(2.11)
where d =√
(yc2 − yc1)2 + (xc2 − xc1)2
C1C =√d2 − (r2 − r1)2
(2.12)
sin(δ) = sin(α + β) = sin(α) cos(β) + cos(α) sin(β)
cos(δ) = cos(α + β) = cos(α) cos(β) + sin(α) sin(β)(2.13)
Chapter 2. Fire Perimeter Tracking 18
Finally, the locations of P1 and P2 can be calculated using the following equations:
P1,x = xc1 − r1 × sin(δ)
P1,y = yc1 + r1 × cos(δ)
P2,x = xc2 − r2 × sin(δ)
P2,y = yc2 + r2 × cos(δ)
(2.14)
The pseudo code for applying the non-uniform condition version of dynamic elliptic
approximation in to the fire model is very similar to that of the uniform version described
before. The only differences lie within the need to calculate ROS for each grid cell and
the use of Eqn 2.11 - 2.14 for calculating tangent line segments.
Require: createEllipseCreate simulation field of x× y grid cellsSet all cell values to 0Set all cell state to unburnedwhile time is less than final time do
if time is 0 thenCall createEllipse on original ignition pointCompute perimeter array based on returned ellipse parameters
elsefor Each point in perimeter array at time t-1 do
Calculate ROS values for the selected pointCall createEllipse with selected point as ignition pointSave ellipse parameters into parameter array
end forfor Each entry in ellipse parameter array do
Calculate outer tangent line between current and next ellipse using Eqn 2.11and 2.14
Save tangent line points in perimeter arrayend forDetermine cell state using same logic as static case
end ifColour simulation field with predetermined colour schemeIncrement time
end while
Figure 2.7 shows the result of applying non-uniform condition extension to the dy-
namic elliptic approximation. The lower right area is impacted by wind blowing hori-
zontally towards the right, whereas the wind direction for the rest of the simulation field
are towards the top right. The ROS can be affected by many different sources, including
Chapter 2. Fire Perimeter Tracking 19
wind parameters, fuel parameters, and terrain parameters. The dynamic elliptic approx-
imation can be used to simulate a growing wildfire as long as these parameters can be
quantitatively converted into the three ROS values at each cell location.
Figure 2.7: Fire perimeter simulation under non-uniform conditions
2.2 Edge Detection
The environment being projected for this research project is the aforementioned wildfire
expansion scenario. There are four distinct colours involved in this projection: green,
yellow, red, and brown, as shown in Figure 2.8.
The dynamic wildfire environment will be projected onto a 2m x 2m glass floor, and
a 250mm diameter quadrotor will fly on top and attempt to follow the edge of the fire
from one side to the other. Further details of the experimental set up of the ISP is
provided in Chapter 4. Due to the limited space, a portion of the environment image
from Figure 2.8 would be projected. The camera on board the micro-quadrotor would
capture a portion of the projected image, and sends it to the ground station for edge
detection. In theory, a good image should contain the edge of the fire, as well as both
the green unburned area and the burning area. The tasks of the ground station include
Chapter 2. Fire Perimeter Tracking 20
Figure 2.8: The top down view of a wildfire projected onto the ISP.
detecting the edge, finding a target way-point from the detected edge, and converting
the target way-point from image coordinate to the global Optitrack coordinate before
sending it as a way-point command to the autopilot.
Based on the fact that the colour information for the region of interest in the images
is known before hand, a colour-based thresholding edge extraction technique is proposed.
One of the major issues with the captured images is that the image can be noisy due to
the pattern on the semi-transparent film used on the ISP. As a result, pre-filtering the
raw image before running the edge extraction algorithm is required. The proposed steps
for extracting the edge from each image is shown in Figure 2.9.
Figure 2.9: The propsed process of edge extraction
There are a number of smoothing filters that can be used to reduce noise on an
image, such as gaussian filter, median filter, and Bilateral filter. Both the gaussian filter
and median filter blurs the entire image, including the edges. The Bilateral filter, in
contrast, blurs the surface while keeping the sharpness of the edges, making it ideal for
edge detection.
Chapter 2. Fire Perimeter Tracking 21
A bilateral filter is used first to remove the texture of the semi-transparent film by
blurring areas with similar colours while keeping edges sharp. Red Green Blue (RGB)
to Hue Saturation Value (HSV) colour-space conversion transforms the image into the
intensity-based colour-space, so that the thresholding process afterwards can pick out
different shades of the same colour easier. Thresholding would then find areas in the
image with red colour and turn that white, while changing everything else to black. Once
the image is in black and white, a Canny edge detector is used to find the edges. Since
Canny edge detection could potentially result in multiple broken edges, morphological
transformation is used to connect these edges based on spacial distance. Finally, contours
are extracted from the image with edge data, in the format of arrays of x and y positions
for further processing. Edge extraction for this research project is implemented with the
OpenCV library.
The bilateral filter contains both a domain filter and a range filter. A domain fil-
ter targets the geometric distance between target pixels and its neighbours; whereas a
range filter targets the photometric similarities [27]. Together, the bilateral filter can be
described by the following equation:
h(x) = k−1(x)
∫ ∞−∞
f(ξ)c(ξ, x)s(f(ξ), f(x))dξ (2.15)
where h(x) is the output image matrix, f(x) is the input image matrix where the target
pixel x is compared with its neighbouring pixels ξ. c(ξ, x) is the geometric distance
between neighbouring pixels ξ and the target pixel x. s(f(ξ), f(x)) is the photometric
similarity, or how closely the two pixels are related in their colour properties. Finally,
k(x) is the normalization term, defined as:
k(x) =
∫ ∞−∞
∫ ∞−∞
c(ξ, x)s(f(ξ), f(x))dξ (2.16)
In essence, the bilateral filter only averages the colour of neighbouring pixels if their
colour properties and distances are closely related. Visually, this means that areas with
similar colours, or non-edges, get averaged; whereas areas with different colours, or edges,
remain sharp.
Thresholding in HSV colour space is a hysteresis process including the hue, saturation,
and value ranges. HSV colour space has the advantage over RGB colour space in that the
colour information is stored in the Hue range alone. The Saturation and Value ranges
indicate the colourfulness and the brightness of the colour defined by Hue. This means
that when thresholding for red colour, as in this research, the Hue hysteresis curve is
the most critical. As long as the Hue is correctly thresholded, Saturation and Value
Chapter 2. Fire Perimeter Tracking 22
ranges can be determined by how big the range in different shades of red and brightness
is needed. The sample image shows that although the colour is relatively uniform, the
brightness could change significantly due to the bright spot produced by the projector
bulb. In fact, any region could be turned into a white colour if the bright spot falls on it.
This means that both Saturation and Value range should be relatively large to account
for this variation.
The OpenCV library specifies that each of the three ranges are defined by numbers
from 0 to 179. The red colour in the OpenCV Hue range is between 160 and 179. The
edge extraction program uses this set of minimum and maximum Hue values to find red
areas in the image regardless if it’s lighter or darker in colour. All red pixels are turned
into white and all non-red pixels are turned into black in the binary output.
Canny Edge Detection was originally proposed by Canny as a four step computational
edge detection process [10]. The four steps are Gaussian filter, intensity gradient, non-
maximum suppression, and hysteresis. The Gaussian filter smooths the image; however,
unlike the bilateral filter that is applied before, Gaussian filter averages over the entire
image and blurs the edges as well. An intensity gradient is the first derivative of colour,
and it can be applied to the image to find the rate of change of colour for every pixel. An
edge pixel would have a very high intensity gradient, whereas a surface pixel would have
relatively low intensity gradient. Canny Edge Detector uses the Sobel Operator, which
is based on the intensity gradient of a visual image, for edge detection.
Gx =
−1 0 +1
−2 0 +2
−1 0 +1
(2.17)
Gy =
−1 −2 −1
0 0 0
+1 +2 +1
(2.18)
The intensity gradient has two directions: x and y direction. The overall gradient
strength and direction of an edge can be calculated as:
G =√G2x +G2
y (2.19)
θ = arctan(Gy
Gx
) (2.20)
The combination of non-maximum suppression and hysteresis techniques ensure that
false edges are eliminated from the final result. A single pixel is used to represent each po-
Chapter 2. Fire Perimeter Tracking 23
tential edge with the application of non-maximum suppression, following which hysteresis
eliminates potential edge pixels based on a set of minimum and maximum threshold val-
ues that defines edge pixels. Edge pixels lower than the minimum threshold value or
higher than the maximum value are considered non-edges. Those that lie within the
threshold range are only considered an edge if they are connected to another edge.
The OpenCV library contains pre-made functions for many of the filters used in the
edge detection process, including bilateral filter and canny edge detector. In an OpenCV
implementation, converting from edge images to contour arrays is necessary if further
processing on the extracted edge is needed, as arrays of x and y values are much more
useful than a binary image of 1’s and 0’s. The concept of determining a new target
way-point is shown in Figure 2.10.
(a) Position of camera and field of view (b) Edge image captured by the camera
Figure 2.10: Side profile of quadrotor and location of new way-point on image
While the quadrotor system will be discussed in detail in Chapter 4, it is important to
note where the camera is mounted. The left image shows the position of the camera with
respect to the quadrotor, and the camera’s field of view. Since the camera is mounted
towards the front of the quadrotor, the vertical centre of this image is at a location in
front of the quadrotor’s position. This is illustrated on the right image, where the blue
dotted line is the vertical centre of the image and the black dotted line below is the centre
of the quadrotor. Assuming that the fire edge is in the image, and that the quadrotor is
flying relatively parallel to the fire edge, then the red solid curve is the fire edge that is
in view of the camera. The position of the quadrotor would be the intersection between
the horizontal centre line and the black dotted line. The intersection between the fire
edge and the vertical centre line, the green dot, would be a position on the fire edge but
in front of the quadrotor. If this green dot acts as the next way-point, then it can be
certain that the quadrotor would always move forward and track the fire edge.
The pseudo code for an OpenCV implementation of the fire edge extraction algorithm
is presented below, which would extract the fire edge from the image and determine the
Chapter 2. Fire Perimeter Tracking 24
location of the next way-point in the image coordinate.
Image resolution is 640× 480while exit key not pressed do
Receive RGB image from cameraUse Bilateral Filter to remove noiseConvert from RGB to HSV colour spaceThreshold image for red colour, return binary imageUse Canny Edge Detector, return binary image with white edges on black back-
groundPerform morphological transformation to remove gaps in edgesConvert edges from binary image to arrays of contoursFind longest contourFind the point from the longest contour with y value of 240
end while
2.3 Perimeter Reconstruction
The ultimate goal of the fire perimeter tracking mission is to recreate the fire perimeter
and be able to both move backwards to see the history of the fire progression and move
forward to predict the fire growth. The quadrotor that flies alongside the fire would result
in a spiral path as the fire continuously expands over time, similar to the one shown in
Figure 2.11. Reverse elliptic approximation is a method developed to synchronize all the
location data captured over a time period, similar to the spiral UAV path in Figure 2.11,
into a single time instance.
2.3.1 Reverse Elliptic Approximation
Reverse elliptic approximation is fundamentally the same as the dynamic elliptic ap-
proximation, except that it works in reverse. The reverse elliptic approximation steps
backwards in time instead of forward, finding the inner tangents as opposed to the outer
tangents of two ellipses. This is accomplished by using a negative angle α in Equa-
tion 2.10. Figure 2.12 shows the result of applying the reverse elliptic approximation
procedure after a forward elliptic approximation.
Figure 2.12a shows a forward elliptic approximation from the original fire perimeter in
black to the new fire perimeter in blue; Figure 2.12b shows a reverse elliptic approximation
being applied to the new fire perimeter; and Figure 2.12c shows the difference between
the original fire perimeter in black and the new approximated perimeter at the same
Chapter 2. Fire Perimeter Tracking 25
Figure 2.11: The spiral path created by the UAV when tracking a continuously expandingfire perimeter
time instance in red. Any difference between the black and red perimeters are due to
perimeter discretization.
If the fire is assumed to be much larger than the distance that a single UAV can
cover, then the local fire perimeter at different time instances can be approximated to be
parallel to each other, as illustrated in Figure 2.13.
P1, P2, and P3 are three unique positions on the fire perimeter. The fire perimeter
expands upwards with time, and the UAV tracks the fire perimeter from right to left.
The UAV reaches point P1 at time t, point P2 at time t + ∆t, and finally P3 at time
t + 2∆t. The overall UAV path is indicated by the solid arrow. The recorded positions
of the fire perimeter from the UAV’s perspective are: P1,t, P2,t+∆t, and P3,t+2∆t. In order
to synchronize the three positions into a single time frame, a series of reverse elliptic
approximations need to be applied. The reverse elliptic approximations used here are
slightly modified, in that the line segment to be determined is the tangent to an ellipse
and an adjacent point. Based on the scenario in Figure 2.13, for example, the red ellipse
with an ignition point at P2,t+∆t can be created. It would have its head direction pointed
downward. The tangent between point P1,t and this ellipse would form the line P1,tP2,t.
Chapter 2. Fire Perimeter Tracking 26
(a) Forward Approximation (b) Reverse Approximation (c) Result Comparison
Figure 2.12: The reverse elliptic approximation result and comparison
(a) (b)
Figure 2.13: The path of a UAV tracking the fire front over three time steps
In this example, P2,t+∆t would be called the Reverse Ignition Point, P1,t is the Static
Point, and P2,t is the Intersection Point. Applying this technique twice on P3,t+2∆t would
create the line P2,tP3,t. This line segment can be combined with the line segment P1,tP2,t
to create the desired local fire front at time t. The pseudo code for calculating the fire
perimeter at time t = 0 using the UAV’s location data from time t = tfinal is as follows:
2.3.2 Full Perimeter Reconstruction
If the quadrotor is capable of perfect tracking, then the result of the reverse elliptic ap-
proximation would be the same as the fire perimeter that has been tracked. However,
perfect tracking does not exist, and the position data obtained from a quadrotor prob-
ably does not describe the fire perimeter perfectly. Elliptic curve fitting and 2nd order
polynomial curve fitting are two methods can be used to approximate the fire perimeter
being tracked.
Assuming that the overall fire perimeter grows in an elliptic fashion, then elliptic
curve fitting would provide an approximation on the entire fire perimeter, as opposed to
Chapter 2. Fire Perimeter Tracking 27
UAVPath (size 2×n) contains UAV’s x and y position data, where n = tfinal× intervalCreate empty UAVX and UAVY matrices (size n× n)for k increments from 1 to n do
UAVX k’th diagonal value = UAVPath x at index kUAVY k’th diagonal value = UAVPath y at index k
end forfor i increments from 2 and n do
for j decrements from i to 2 doFind head, tail, side ROS at current timeReverse Ignition Point P1 = (UAVX(j, i), UAV Y (j, i))Static Point P2 = (UAV X(j − 1, i− 1), UAV Y (j − 1, i− 1))Apply reverse elliptic approximation on P1
Find tangent line between ellipse and P2, where Intersection Point Pinter is thepoint where the tangent line intersects the ellipse
UAV X(j − 1, i)andUAV Y (j − 1, i) are defined by Pinterend for
end for
only the section that the quadrotor has monitored. The quadratic form of the general
ellipse equation is as follows:
Ax2 + 2Bxy + Cy2 + 2Dx+ 2Fy +G = 0 (2.21)
where x and y are points representing the ellipse, and A,B,C,D, F,G are parameters
that need to be calculated. This means that a minimum of six unique points that lie
on the ellipse are needed. A visualization of the ellipse can be drawn once these six
parameters are obtained. The centre of the ellipse is:x = CD−BFB2−AC
y = AF−BDB2−AC
(2.22)
and the semi-axes lengths of the ellipse are:a =
√2(AF 2+CD2+GB2−2BDF−ACG)
(B2−AC)[√
(A−C)2+4B2−(A+C)]
b =
√2(AF 2+CD2+GB2−2BDF−ACG)
(B2−AC)[−√
(A−C)2+4B2−(A+C)]
(2.23)
Finally, the angle between the major axis of the ellipse and the x-axis can be calculated
with one of the four cases:
Chapter 2. Fire Perimeter Tracking 28
θellipse =
0 if b = 0 and a < c
12π if B = 0 and A > C
12
cot−1(A−C2B
) if B 6= 0 and A < C
π2
+ 12
cot−1(A−C2B
) if B 6= 0 and A > C
(2.24)
Due to the fact that the ellipse that is being fitted is free in size, location and angle, the
best fitted ellipse may not necessarily be the closest to the original fire perimeter if there
are noises in the data. In an extreme example, it is possible for the fitted ellipse to be 90
degrees offset from the actual perimeter. This means that the data that are suppose to
be on the major axis are fitted to the minor axis, making the fitted ellipse very different
from the original. In these cases, the approximated perimeter that are further away from
the location of data collection will contain too much error to be of any use. If the segment
that is close to the data collection location is the only portion that is useful, then the
curve can also be approximated using a 2nd order polynomial curve fitting technique.
2nd order polynomial curve fitting is much simpler. The general 2nd order polynomial
equation is as follows:
y = Ax2 +Bx+ C (2.25)
2nd order polynomial curve fitting requires a minimum of 3 sets of data points, and
the data can be fitted using the least square method. It should provide a result that is
very close to the tracked segment of the actual fire perimeter.
2.4 Summary
An elliptic approximation fire model is chosen for generating the dynamically growing
wildfire because it provides the position of the fire perimeter without the need to create
a temperature model for the environment. This elliptic approximation model can be
expanded to include both dynamic and non-uniform field conditions. The edge detection
method proposed for this project includes a bilateral filter to pre-process the image, a
Canny Edge Detector to extract the edges, and a few other post-process filters to ensure
that only a specific edge is being extracted. Both elliptic curve fitting and 2nd order
polynomial curve fitting have been proposed for reconstructing the perimeter. Elliptic
curve fitting can provide a good estimate of the entire fire perimeter if the noise in data
is low. If the data contains a lot of noise, however, it is possible that elliptic curve
fitting does not return a valid solution. Additionally, fire does not necessarily grow in an
Chapter 2. Fire Perimeter Tracking 29
elliptic fashion, which prevents effective prediction of how the fire expands in areas not
monitored by the quadrotor. In these scenarios, 2nd order polynomial curve fitting may
be a better alternative. 2nd order polynomial can only provide a good approximation of
the section of the fire perimeter that has been monitored, but it is much less susceptible
to noise.
Chapter 3
Quadrotor Navigation
A quadrotor dynamic model and controllers are required to test the navigation algorithm
in simulation. The quadrotor model can be derived from basic physics laws such as the
Newton’s Second Law. The controller structure used in the simulation will match that of
the actual autopilot for the micro-quadrotor. Finally, a PBVS navigation method will be
used in conjunction with edge detection to tracking the fire perimeter via visual camera.
3.1 Quadrotor Model
A standard non-linear quadrotor model is used for fire edge tracking simulation in MAT-
LAB. While many papers use the simpler “plus” configuration of quadrotor model, a
slightly more complex “X” type quadrotor model is needed for this project in order to
match with the hardware configuration. The “X” configuration is used on the micro-
quadrotor so that the camera can be mounted towards the front of the vehicle without
significantly disrupting air flow from the propellers. ArduPilot Mega is an open source
autopilot that is being used on the physical micro-quadrotor [26]. The ArduPilot Mega
has its own way of numbering the motors, and the simulation model will reflect this cus-
tom motor order for consistency. Figure 3.1 shows the APM version of the “X” quadrotor
configuration.
Quadrotor attitude and position can be calculated based on the thrust and the torque
each propeller produces during flight. The translational and rotational dynamics of a
quadrotor can be modelled separately. Figure 3.2 shows the principles behind quadrotor
dynamics.
Each motor provides a certain amount of torque to the rotor, and the rotor creates an
upward thrust, indicated by red arrows, as well as a certain amount of torque, indicated
by blue arrows. The quadrotor rolls and pitchs by creating a difference in thrust between
30
Chapter 3. Quadrotor Navigation 31
Figure 3.1: The “X” configuration used by the ArduPilot Mega [26]
Figure 3.2: Principle of quadrotor dynamics
pairs of motors. For example, the quadrotor rolls to the left if motors 2 and 3 generate
less thrust than 1 and 4. At the same time, the quadrotor would start moving towards
the left because the total thrust vector is always perpendicular to the quadrotor frame.
The quadrotor can yaw counter-clockwise if the motors that generate counter-clockwise
torques are faster than those that generate clockwise force. In this case, a counter-
clockwise yaw can be produce by having motors 1 and 2 turn faster than 3 and 4,
creating a difference in overall torque.
3.1.1 Translational Dynamics
The dynamics of a quadrotor are governed by Newton’s Second Law, F = ma, where m
is the mass of the quadrotor. This can be expanded in the inertial reference frame:
Chapter 3. Quadrotor Navigation 32
xyz
=1
m
FxFyFz
(3.1)
The thrusts provided by the motors, F1−4, are defined in the body-fixed frame. Thus,
the rotation matrix RIB1 transforming from body-fixed frame to inertial frame is required.
RIB1 =
cψcθ −sψcφ + cψsθsφ sψsφ + cψsθcφ
sψcθ cψcφ + sψsθsφ cψsφ + sψsθcφ
−sθ cθsφ cθcφ
(3.2)
where c denotes cos and s denotes sin. φ is the roll angle, θ is the pitch angle, and ψ is
the yaw angle of the quadrotor. The overall translational dynamics becomes:xyz
=1
mRIB1
0
0
F1 + F2 + F3 + F4
− 0
0
mg
(3.3)
This can be simplified into three equations that describes the quadrotor’s translational
motion: x = 1
m(sin(φ)sin(ψ) + cos(φ)cos(ψ)sin(θ))U1
y = 1m
(cos(φ)sin(θ)sin(ψ)− cos(ψ)sin(θ))U1
z = 1mcos(θ)cos(φ)U1 − g
(3.4)
where U1 = F1 + F2 + F3 + F4.
3.1.2 Rotational Dynamics
The rotational dynamics of a rigid body in the body frame is governed by the Euler’s
equation:
T = IRα + ω × IRω (3.5)
where T is the torque, α is the angular acceleration, and ω is the angular rate in body-
fixed frame. IR is the moment of inertia matrix and is defined as:
IR =
Ixx Ixy Ixz
Iyx Iyy Iyz
Izx Izy Izz
(3.6)
Chapter 3. Quadrotor Navigation 33
where Ixx, Iyy, Iz are the moments of inertia of the quadrotor with respect to the roll,
pitch, and yaw axis. The rest of the terms in the matrix are the products of inertia.
Due to the fact that quadrotors have mirror symmetry on the XZ and Y Z planes, the
products of inertia are 0. Expanding the above equation would result in the following
equations in terms of angular rates p, q, r in the body-fixed frame:p = 1
Ixx(Mx − (Izz − Iyy)qr)
q = 1Iyy
(My − (Ixx− Izz)pr)
r = 1IzzMz
(3.7)
where Mx,My,Mz are the moments with respect to the body frame. To convert angular
rates in the body-fixed frame to Euler angular rates, θ, φ, ψ in inertial frame, the rotation
matrix RIB2 is used:
RIB2 =
1 sφtθ cφtθ
0 cφ −sφ0
sφcθ
sθcθ
(3.8)
φθψ
= RBI2
pqr
(3.9)
where s denotes sin, c denotes cos, and t denotes tan.
3.1.3 Motion Equations
Combining the translational and rotational dynamic equations, a quadrotor’s motions
can be described by the following set of equations:
Chapter 3. Quadrotor Navigation 34
x = 1m
(sin(φ)sin(ψ) + cos(φ)cos(ψ)sin(θ))U1
y = 1m
(cos(φ)sin(θ)sin(ψ)− cos(ψ)sin(θ))U1
z = 1mcos(θ)cos(φ)U1 − g
p = 1Ixx
(U2 − (Izz − Iyy)qr)
q = 1Iyy
(U3 − (Ixx − Izz)pr)
r = 1IzzU4
φ = p+ qsin(φ)tan(θ) + rcos(φ)tan(θ)
θ = qcos(φ)− rsin(φ)
ψ = q sin(φ)cos(θ)
+ r sin(θ)cos(θ)
(3.10)
The inputs, Ui, are intermediate values and are defined as:
U1 =∑4
i=1 Fi
U2 = l(−Fleft + Fright)
U3 = l(Ffront − Fback)
U4 = KtorqueKthrust
(F1 + F2 − F3 − F4)
(3.11)
Ffront = F1 + F3
Fback = F2 + F4
Fleft = F2 + F3
Fright = F1 + F4
(3.12)
Since the rotors of a quadrotor are mostly perpendicular to the direction of travel, the
impact of air speed through the rotors on the thrust they generate is small. In addition,
translational velocity of the quadrotor is very low in the indoor space, so that any changes
in thrust due to change in air flow are also small and negligible. The thrust and torque
can be approximated by the following relationship with the propellers’ angular velocity
Ωi. l is the length of the motor arm of the quadrotor.Fi = KthrustΩ2i
Ti = KtorqueΩ2i
(3.13)
Due to limited space in the indoor environment that the ISP resides in, the body angles
of the quadrotor can be assumed to be small. As a result, the Eular angular rates φ, θ,
ψ and the body angular rates p, q, r can be approximated to be the same. Additionally,
Chapter 3. Quadrotor Navigation 35
gyroscopic effects generated by moments of inertia in the off axes are negligible when the
angular rates in all three axes are small. As a result, Eqn 3.10 can be simplified to the
following:
x = 1m
(sin(φ)sin(ψ) + cos(φ)cos(ψ)sin(θ))U1
y = 1m
(cos(φ)sin(θ)sin(ψ)− cos(ψ)sin(θ))U1
z = 1mcos(θ)cos(φ)U1 − g
φ = 1IxxU2
θ = 1IyyU3
ψ = 1IzzU4
(3.14)
The mass moments of inertia of the quadrotor on the x, y, and z axis can be approx-
imated with a disk, such that:
Ixx = Iyy =1
2Izz (3.15)
3.2 Quadrotor Control
The quadrotor has four output states: longitudinal position, lateral position, altitude,
and heading angle. Longitudinal position is controlled by the pitch angle of the quadrotor,
while lateral position is controlled by the roll angle. Pitch, roll, yaw, and altitude are
determined by the thrust on each motor. The overall control structure is provided by the
ArduPilot, and the controllers in the simulation are made to match this structure. The
controller structure is shown in Figure 3.3.
The quadrotor is entirely controlled by PID controllers. Four control loops are used for
longitudinal and lateral position control, shown in Figure 3.3 as XB and YB respectively,
while two are used for altitude and heading angle control. A PI controller is used to
convert displacement error to desired velocity, as well as to convert angular error to
desired angular velocity. The PI controller ensures that the relatively slow changing
parameters, position and angle, have no steady state error. In real world operations,
flying in constant wind can result in steady state error in both quadrotor’s pitch angle,
roll angle, and horizontal positions, if the integral terms are not tuned. In contrast, a PID
controller is used for horizontal velocity and angular rate. These controllers are designed
to provide quick responses to the relatively fast changing parameters. The heading angle
control structure is similar to that of the pitch or roll angle. There is a PI controller for
angle error, and a PID controller for angular rate error. The altitude control structure,
on the other hand, is similar to the outer loop of horizontal position control, which has
Chapter 3. Quadrotor Navigation 36
Figure 3.3: Controller structure used for the quadrotor
a PI controller for altitude error and a PID controller for climb rate. The PID gains
are custom tuned for the simulation and hardware setup. The tuning for the numerical
simulation model is done mostly by analyzing the step response behavior, whereas the
tuning for the micro-quadrotor is done by a combination of analyzing behavior based on
telemetry log files and trial and error.
On the ArduPilot, the IMU on the autopilot outputs the angles θ, φ, ψ and the angular
rates θ, φ, ψ. The inertial navigation, on the other hand, estimates the position X, Y, Z
and linear velocity X, Y , Z of the quadrotor in both the body-fixed frame and the inertial
frame.
Both the simulated autopilot and the ArduPilot need to be able to operate in two
modes: way-point command mode and velocity command mode. In way-point command
mode, the user specifies way-points for the quadrotor. A way-point contains X, Y, and
Z parameters. The way-point command mode can be used by any generic navigation
algorithms that has target locations as outputs. The longitudinal error represents how far
the quadrotor is from the target; and the lateral error represents how far the quadrotor has
drifted from its intended path between two way-points. In contrast, velocity command
mode is used when the user not only cares about where the quadrotor needs to go, but
also how it gets there. Many edge tracking based navigation algorithms uses velocity
command mode to ensure that the vehicle moves along the edge at a constant velocity
Chapter 3. Quadrotor Navigation 37
and the vehicle does not deviate from the edge being tracked. In this case, the horizontal
velocities are constrained, which requires the full PID controllers.
3.3 PBVS Navigation
Position Based Visual Servoing (PBVS) has been chosen as the navigation technique over
Image Based Visual Servoing (IBVS) for this thesis project primarily because the position
of the target, which in this case is the fire perimeter, is desired for post experiment data
processing. In this case, PBVS provides this position data while navigating. Figure 3.4
shows how the PBVS navigation algorithm works. Additionally, the ArduPilot is set up
as a way-point based autopilot that responds to way-point commands.
Figure 3.4: PBVS flow chart
A PBVS navigation algorithm can be divided into three parts. Firstly, the vehicle
needs to have an autopilot that uses the way-point as navigation commands. Secondly,
there needs to be a way to determine the target way-point. Lastly, the target way-points
have to be transformed into the inertial coordinates. In this particular case, both the
autopilot in the simulation and ArduPilot do indeed work with way-point commands.
The target way-point is determined by the edge detection algorithm described in Section
2.2 in experiment, and is simulated with the actual fire perimeter with Gaussian noise
added in MATLAB simulation. The edge detection algorithm outputs way-points in
the image coordinate, but ArduPilot requires target way-points in the format of x and
y position, in the inertial coordinates that is being used by Optitrack. Therefore, a
reference frame transformation is needed to connect the two existing blocks.
3.3.1 Reference Frame Transformation
There are three steps involved in transforming a set of coordinates from the image refer-
ence frame into the global reference frame. Assuming that the camera is pointed vertically
downwards when an image is taken, then any pixel that is not at the centre of the image
could be represented by a variation in pitch and roll angles, ∆θi and ∆φi where subscript
i denotes the image frame, as shown in Figure 3.5.
Chapter 3. Quadrotor Navigation 38
Figure 3.5: The image reference frame
C is the centre of the image, P is the point of interest, and O is the origin in the
image frame. ∆θi and ∆φi are calculated by first converting all pixel distances in image
frame into distances in the global frame via a pixel-to-centimeter ratio, and then taking
the arctangent of the horizontal and vertical distances. Although the pixel to centimetre
ratio is different depending on the altitude of the camera, the angles created by the
distance are the same because they form similar triangles, as shown in Figure 3.6. Thus,
the angles can be calculated using a specific pixel-centimetre ratio with a specific altitude
regardless of the altitude that the image is actually taken at.
Figure 3.6: The pixel-centimetre ratio at a certain altitude
The image that the on board camera captures has a resolution of 640× 480px, which
means that xi = 640px and yi = 480px. When the camera is pointed vertically downwards
at a height of h = 35cm, the centimetre distances that is captured by the camera has
Chapter 3. Quadrotor Navigation 39
been experimentally determined to be x = 42cm and y = 31.5cm. The calculations for
∆θ and ∆φ using this set of experimental data are as follows:∆θi = atan(∆yi ×K)
∆φi = atan(∆xi ×K)
K = 31.5(480×35)
(3.16)
where K is a constant that is related to the pixel-centimetre ratio and the specific altitude
for this ratio.
Due to the fact that camera is fixed downwards in relation to the quadrotor frame
and the projection plane is flat, any tilt on the image frame relative to the global frame
are induced by the pitch and roll of the quadrotor, as illustrated in Figure 3.7.
Figure 3.7: The contribution of quadrotor’s pitch angle to total pitch angle difference fortarget
The offset distance between the centre of the camera and the centre of the quadrotor is
very small in relation to the altitude that the quadrotor is flying at; thus, the camera can
be assumed to be at the centre of the quadrotor for this reference frame transformation
process. Based on Figure 3.7, the total pitch and roll angle needed to reach the target
is the combination of the quadrotor’s pitch and roll (θ and φ), and the target’s relative
pitch and roll in the image frame (∆θi and ∆φi):θtotal = θ −∆θi
φtotal = φ−∆φi(3.17)
In this particular case, a position pitch angle in the image plane corresponds to a
Chapter 3. Quadrotor Navigation 40
negative pitch angle in the inertial frame of the quadrotor. The same rotation matrix
between body frame and inertia frame (Eqn 3.2) can be applied to transform the angles
experienced in the image frame into the global frame. Both the x, y, z position and the
φ, θ, ψ attitude angles of the quadrotor are known, which means that the reference frame
transformation can be done with the following calculations:R1 = cos(φtotal) sin(θtotal) cos(ψ)− sin(φtotal) sin(ψ)
R2 = cos(φtotal) sin(θtotal) sin(ψ) + sin(φtotal) cos(ψ)
R3 = cos(θtotal) cos(φtotal)
(3.18)
xtarget = xquad + L×R1
ytarget = yquad + L×R2
ztarget = zquad + L×R3
(3.19)
Although the parameter L is an unknown, the altitude of quadrotor is known and the
altitude of the target is zero. Therefore, parameter L can be calculated as:
L =−zquadR3
(3.20)
The pseudo code for PBVS implementation in MATLAB can be found below. The
PBVS navigation implementation for the physical experiment uses the same concept, but
is different because the Indoor Simulation Platform is an integration of many separate
components. As a result, data needs to be sent between components to ensure proper
execution of the algorithm. The implementation of this algorithm on the physical system
will be discussed in the next chapter.
while time < tfinal doFind fire edge in pre-defined image windowDetermine next target way-point based on intersectionTransform way-point to global coordinateSend new way-point command to autopilotwhile way-point has not been reached do
Check distance to way-pointend while
end while
Chapter 3. Quadrotor Navigation 41
3.4 Parameter Estimation
The above simulations are completed with the theoretical quadrotor model, where var-
ious parameters are estimated based on geometry of the physical quadrotor. Some of
these parameters, such as mass, can be easily measured; however, other parameters, such
as moments of inertial, are more difficult. A more accurate parameter estimation of the
quadrotor can be accomplished by using maximum likelihood method. This method has
been investigated by Marlon Koga, a summer intern at the FSC lab. This method is
based on the idea that a mathematical model can be found given the input sequence
to the autopilot and the physical output results from the quadrotor [28]. In theory, the
calculations for the model parameters would be trivial. In the real world, however, the
noise in the physical system means that Kalman Filter is required to estimate these pa-
rameters. The new state-space equations for a system that contains random disturbance
is as follows:
x(t) = Ax(t) +Bu(t) +Bww(t) (3.21)
y(t) = Cx(t) +Du(t) (3.22)
z(i) = y(i) + v(i), i = 1, 2, 3, 4...N (3.23)
where Bw is the disturbance matrix, w(t) is the random disturbances, z(i) is the mea-
surements, and v(i) is the measurement noise given in discrete time steps. Both w(t)
and v(i) are Gaussian distributions. The likelihood for a series of measurements ZN =
[z(1), z(2), ..., z(N)]T given a theoretical model M and a set of inputs is given by the
following equation:
L(ZN |M) =n∏i=1
(2π)−n0/2|β(i)|−1/2exp[−1
2νT (i)β−1ν(i)] (3.24)
where the predicted measurement yp, innovation ν, and innovation covariance matrix β
are defined as:
yp ≡ E[z(i)|Zi−1,M ] = E[z(i)|M ] (3.25)
ν(i) ≡ z(i)− yp(i) (3.26)
β(i) ≡ Cov[z(i)|Zi−1,M ] = E[ν(i)νT (i)] (3.27)
The desired system parameters γ that approximates the real parameters γ can be
found by maximizing the likelihood equation:
Chapter 3. Quadrotor Navigation 42
γ = argmax(L(ZN |M)) = argmin1
2
N∑i=1
[νT (i)β−1(i)ν(i) + ln|β(i)|] (3.28)
Thus, the problem has been reduced to a form that supports a discrete time Kalman
Filter algorithm where the prediction equations are:
xp(i) = Adx(i− 1) +Bdu(i− 1) (3.29)
Pp(i) = AdP (i− 1)ATd +BwdQBTwd (3.30)
innovations are:
β(i) = CdPp(i)CTd +R (3.31)
ν(i) = z(i)− Cdxp(i)−Ddu(i) (3.32)
and the measurement updates are:
x(i) = xp(i) +K(i)ν(i) (3.33)
K(i) = Pp(i)CTd β−1(i) (3.34)
P (i) = [I −K(i)Cd]Pp(i) (3.35)
where Ad, Bd, Cd, Dd and Bwd are the discrete time version of the state space matrices.
Figure 3.8 shows the block diagram for parameter estimation with experimental data,
incorporating the Kalman Filter estimation method.
The experimental input data consists of the 4 desired forces the autopilot sends to the
motors. The output data are the angles and angular rates from the IMU, the position
Optitrack, and estimated linear velocities from inertial navigation. These data are not
logged at the same frequency; thus, data synchronization is needed to match the frequency
of the output data. In this case, all data are modified to have a frequency of 50Hz, even
if they are actually logged at a lower frequency. The experimental inputs are passed into
the Kalman Filter as inputs, and the predictions of the Kalman Filter are compared with
the experimental output data. The cost function and the model are updated based on the
discrepancies between the two outputs. Three parameters are subjected to estimation:
m, Ixx, and Iyy. The result of this estimation are as follows:
Chapter 3. Quadrotor Navigation 43
Figure 3.8: The experimental parameter estimation block diagram [28]
m = 511grams
Ixx = 0.0010kgm2
Iyy = 0.0014kgm2
(3.36)
The mass parameter can be experimentally verified to be highly accurate. The actual
weight of the quadrotor is measured to be 510 grams. There are two ways of calculating
the moments of inertia theoretically. The first method involve breaking up the quadrotor
into separate simple geometry components and add up all the individual moments of
inertia. The second method simple approximates the quadrotor as a disk, and calculate
the moments of inertia using the volume and dimensions of the quadrotor. Although the
break-up method would provide a more accurate result, both methods would normally
produce the same value for Ixx and Iyy, and neither can fully take into account of the
mechanical imperfections of the physical hardware. The quadrotor has many different
components that are not mounted at the centre of gravity, such as batteries, radio, and
camera. While much effort has been made to balance the centre of gravity, there is
only so much that can be done given the limited space on the micro-quadrotors and
the required payload for this mission. Therefore, the moments of inertia for the actual
quadrotor on the roll and pitch axes are most likely not the same. The experimentally
estimated parameters above indicate this important difference. Due to the fact that the
quadrotor used in indoor experiments have gone through multiple hardware upgrades,
Chapter 3. Quadrotor Navigation 44
this estimation is only done on the final hardware setup. Therefore, this estimation result
is not used in the numerical simulations for this thesis project.
3.5 Summary
The quadrotor dynamic model used in the numerical simulation is the “X” configuration
of the simplified non-linear model used by many other researchers. The “X” configuration
ensures maximum thrust with the camera mounted in front of the micro-quadrotor. A
set of intermediate inputs is needed for the “X” configuration compared to the standard
“Plus” configuration. The controller has four control signals controlling four states: X
position, Y position, altitude, and heading angle. X and Y are the left and forward
directions in body-fixed frame respectively, which translate into desired roll and pitch
angles. The controller gains for the simulation model and the physical micro-quadrotor
are tuned separately based on step response behaviors. PBVS navigation has been chosen
for the fire edge tracking mission. The input of the PBVS algorithm is target way-point
provided by edge detection. A reference frame transformation is required to transform
the target way-point from image frame in pixels to the global frame in cm. A parameter
estimation method has been investigated by the summer student Marlon, and has proven
to be accurate in estimating the mass and moments of inertia of the quadrotor being
used for experiments.
Chapter 4
Indoor Simulation Platform
The ISP is an integrated hardware-in-the-loop experimentation environment being de-
veloped at the Flight Systems and Control (FSC) Lab. The system used for this thesis
project is the first iteration in both hardware and software, and serves as a proof of con-
cept for future developments. This version of ISP is capable of generating a simulated
environment and having a single micro-quadrotor navigate within the environment via
visual feedback.
4.1 Hardware
There are four major hardware components and four major software components for the
ISP. The four hardware components are: a quadrotor that would act as the vehicle for
carrying out various navigation missions, an environment projection that can simulate
a top down view of any environment, an indoor navigation system that helps locate the
quadrotor, and finally a ground station that is capable of synchronizing all the data and
commands the quadrotor. The overall system schematic can be seen in Figure 4.1.
The black dotted box encircles everything that is on board the micro-quadrotor, which
includes the autopilot, various sensors, and other quadrotor hardware. The black arrows
indicate wired connections; the blue dotted arrows show wireless connections; and the
red dotted arrows describe relationships other than data transmission.
The environment projection platform was designed and constructed previously by
Yasir Malang, a summer intern at the FSC lab. It consists of an angled projector that
projects the image downwards at a mirror. The image is reflected by the mirror upwards
to a semi-transparent film under a large piece of glass floor. The projection platform
setup can be see in Figure 4.2.
An example of the projected environment can be seen in Figure 4.3. The projected
45
Chapter 4. Indoor Simulation Platform 46
Figure 4.1: The overall system schematic of the ISP.
Figure 4.2: The environment projection platform with angled projection.
elliptical fire edge can be seen from the top of the glass floor, so that the image can be
captured by a downward looking camera mounted on the quadrotor for position feedback.
The glass floor measures about 2m × 2m, which limits horizontal dimension of the
simulation space. The glass floor is also 1.5m above ground, due to the angled mirror
assembly underneath. This restricts the vertical dimension of the simulation space to no
more than 2m. A set of six Optitrack cameras are used as the indoor positioning system.
The cameras are V100:R2 models. In order for the cameras to provide a trackable area
of 2m × 2m horizontally, the vertical tracking is essentially limited to a maximum of 1m.
The lack of large open space in the lab limits the size of the aerial vehicle used on in
the experiment to a micro-sized quadrotor. A micro-quadrotor has very limited payload
Chapter 4. Indoor Simulation Platform 47
Figure 4.3: An example of the projected wildfire environment.
capacity, which has to be split between an autopilot, a battery, and a camera system. In
order to ensure reasonable flight time, the autopilot and the camera system would have
to be small and lightweight. A small autopilot means limited computational resources
and some of the more resource intensive operations, such as image processing, would have
to be done on the ground station instead. In turn, this means that data transmission
between the ground station and the autopilot is crucial. A lightweight camera system,
on the other hand, means that the image quality would suffer, and that there would be
very limited, if any, physical stabilization of the camera. With these operational limits
in mind, a micro-sized quadrotor with diagonal arm length of less than 250mm is needed
for ISP operation.
A number of hardware options have been considered for the ISP micro quadrotor.
The main constraints on selecting an appropriate micro quadrotor for this project are
the size, payload, and endurance. The micro quadrotor must have a diameter no more
than 250mm; it must be able to carry a fully functional autopilot that is capable of
carrying out high level navigation commands, as well as a small camera with associated
video transmission module; finally, it should have a maximum flight time of at least 5
minutes.
The quadrotor frame and autopilot controller are selected separately as there are a
number of mix and match combinations. The frame selection included a FPV250 frame
from HobbyKing, Kmel Robotics quadrotor package, and a custom built carbon fiber
frame. The Kmel Robotics package, which contains both a carbon fibre frame and a
Gumstix based autopilot, was rejected because a flexible open source system is desired.
The custom built carbon fibre frame was chosen initially due to the flexible nature of
the design. Although there are a number of autopilot options to choose from within
the price range, only the ArduPilot is open source in both software and hardware. As
such, the ArduPilot is chosen as the autopilot for the ISP. Two carbon fibre frames
Chapter 4. Indoor Simulation Platform 48
were constructed for ISP; however, the design and available materials could not meet
the vibration requirements of the ArduPilot. A switch is made to the FPV250 frame,
and a number of damping solutions are applied on the ArduPilot, so that the vibration
requirements could finally be met.
(a) Custom built frame (b) FPV250 frame
Figure 4.4: The two quadrotor frames that have been used in this project
Figure 4.4 shows the two quadrotor frames that have been tested. The one on the left
is the custom built carbon fibre frame that ultimately did not pass the vibration test.
The one on the right is the FPV250 frame from HobbyKing, which is currently used on
the ISP. The FPV250 frame has the following technical specifications:
Manufacturer HobbyKingConfiguration Quad X
Diagonal Length 250mmPropeller 5 x 3
Motor Multistar 1704Electronic Speed Controller 12Amp ESC
Battery 1300mAH 3 Cell LiPoTotal Weight 450gEndurance 5minAutopilot ArduPilot Mega 2.6
Table 4.1: Hardware specifications for FPV259 quadrotor frame
The ArduPilot Mega 2.6 (APM) is an open source autopilot module. It has the
following specifications:
The ArduPilot has an integrated Inertial Measurement Unit (IMU) that contains a
3-axis rate gyro and accelerometer. Additionally, it uses Direction Cosine Matrix (DCM)
for attitude estimation and inertial navigation. It has an internal task scheduler and
Chapter 4. Indoor Simulation Platform 49
Manufacturer 3D Robotics
Sensors
3-Axis Gyro3-Axis Accelerometer
CompassBarometric Altimeter
Battery Current SensorOptional GPS
Chipset ATMega 2560On Board Memory 4MB Dataflash Chip
Table 4.2: Hardware specifications for ArduPilot Mega 2.6
runs four loops at different frequencies. The fastest loop, running at a frequency of
100Hz, is the IMU loop. Its tasks include monitoring and logging IMU data as well as
motor control. The medium loop, running at 50Hz, contains inner loop controls such
as pitch and roll angle controllers. The slow loop, running at 10Hz, handles navigation
control. Lastly, the very slow loop running at 1Hz is tasked with miscellaneous activities
that are not as time sensitive as quadrotor navigation, such as some of the logging and
communication tasks.
4.2 Software
The software, on the other hand, includes an autopilot that can be modified for various
navigation algorithms, an image processing module for controller feedback, an environ-
ment simulation module to generate the environment projection, and data synchroniza-
tion on the ground station to ensure that every module is communicating with each other
properly. Figure 4.5 shows the relationship between different ISP modules that need to
be developed. The blue dotted box is the software package on board the micro quadrotor,
whereas the red dotted box is the software on the ground station.
The software package being carried by the micro quadrotor consists of the autopilot
and a video transmission module. The images taken by the camera is transmitted to a
receiver that is attached to the ground station computer wirelessly. The autopilot controls
the movement of the quadrotor, which includes both high level navigation controllers
and low level stabilization controllers. The navigation commands are generated by the
ground station and sent to the autopilot. Due to the limited size and payload capability
of the micro quadrotor, the on board computational resource is very limited. As a result,
some essential calculations, such as position tracking, image processing, and navigation
Chapter 4. Indoor Simulation Platform 50
Figure 4.5: Structure of the ISP software system
command generation, would be done remotely on the ground station.
The ground station software used for ISP is a modified version of the Mission Planner
(MP), which is the default open source ground station software used by ArduPilot. MP
is made up of multiple threads. The most important of these is the Mavlink thread.
Mavlink is the data transmission protocol used by the Ardupilot to communicate with
its proprietory ground station software, Mission Planner(MP). Mavlink itself is an open
source communication library developed with UAV data transmission as its primary
purpose. The Mavlink thread handles the flow of data between the autopilot and the
ground station. Information such as quadrotor attitude would be received by the ground
station; and in return, navigation commands would be transmitted to the quadrotor.
The radio telemetry that handles the transmission of data operates at 900MHz and has
a baud rate of 57600 bits per second. Due to the fact that MP is programmed in C#,
and Optitrack library is created in C, it would be very difficult to integrate the Optitrack
library directly into MP. Therefore, a separate Optitrack Client is created in C to decode
the data being sent by the Optitrack cameras. Data from the Optitrack Client are sent
to the MP via internal network. Optitrack Listener is a thread created in MP for the
sole purpose of receiving and processing data being sent by the Optitrack Client on the
internal network.
A similar configuration has to be made for image processing. The OpenCV graphics
Chapter 4. Indoor Simulation Platform 51
library is used for the ISP, but it only supports C or C++ environments. A Vision Client
is created to receive the image from the camera receiver and process the image before
sending valuable information to the MP via a separate internal network. Once again, a
Vision Listener thread is made in the MP to receive this information so that all these
external data can be integrated in the MP before creating the appropriate navigation
commands. For simplicity in data encoding and decoding, the Mission Planner does not
send any real data to the external client. As a result, data can only travel from the Vision
Client to the Mission Planner while the Mission Planner can only send acknowledgements
to the Vision Client. This results in the interaction process shown in Figure 4.6.
Figure 4.6: The interaction between ground station and vision client for edge extraction
Processes in the Mission Planner are inside the red dotted box, and processes that
are part of the Vision Client are inside the blue dotted box. The interaction starts as
soon as the Vision Client receives an image from the camera. A request to save the
current states of the quadrotor is sent by the Vision Client to the Mission Planner.
The states saved include the altitude, heading angle, pitch angle, yaw angle, and the
position. These information will be used to convert target way-point information from
image pixel coordinate to global optitrack coordinate for navigation purposes. When the
Mission Planner receives this request, it would save the current states data, and return
an acknowledgement to the Vision Client. The image processing sequence starts once
the acknowledgement has been received. The Vision Client would then send the point of
Chapter 4. Indoor Simulation Platform 52
interest data, which in this case would be the target way-point in image pixel coordinates,
to the Mission Planner after image processing has been completed. The Mission Planner
is responsible to convert this target way-point from image pixel coordinates into global
coordinates. X and Y are usually longitude and latitude values in GPS coordinates for
outdoor missions and Z is usually the altitude above ground measured in meters. In
an indoor experiment, however, the quadrotor would be tracked by Optitrack instead of
GPS. Therefore, X, Y, and Z are the coordinates in the Optitrack reference frame with
units in mm. The simulation developed in MATLAB for this thesis project will use the
Optitrack reference frame. Finally, the global target way-point defined by X, Y, and Z
is sent to the autopilot as a navigation command, completing the PBVS feedback cycle.
As with all wireless data transmission, noise can be occasionally added to the data
during transmission. Figure 4.7 shows a comparison of a sample image captured by the
on board camera with and without noise.
(a) Normal image (b) Image under interference
Figure 4.7: The reverse elliptic approximation result and comparison
Figure 4.7a shows the image captured under normal circumstances, while Figure 4.7b
shows images with occasional interference. Interference could potentially come from two
sources. Due to the size of the micro-quadrotor, all the electronic components are very
close to each other. The four motors and the four ESCs could generate large amounts
of electro-magnetic interference for both the camera and wireless transmission module.
In addition, the wireless transmission module is relatively low power because of the
battery limit on board the micro-quadrotor; therefore, noise from the surroundings could
also degrade the image quality during wireless transmission. The chance of receiving
an image with as much interference as the one shown in Figure 4.7b is low; thus, it is
possible to completely discard images like these during image processing as they offer no
real value in terms of navigation information.
The pseudo code for the Vision Client and Mission Planner are shown below. The
Vision Client code can be changed for different PBVS missions. For example, a target
tracking PBVS mission would use an image processing algorithm that detects the target
Chapter 4. Indoor Simulation Platform 53
object, and sets the location of the target object as the next way-point. The Mission
Planner component of the PBVS code does not need to be changed.
Vision Clientwhile Exit key not pressed do
Receive image from cameraSend request to MP to save quadrotor stateswhile MP acknowledgement not received do
Wait for MP responseend whileFind fire edge from image and determine next way-point (image coordinate)Send way-point (image coordinate) to MPwhile MP acknowledgement not received do
Wait for MP responseend while
end while
Mission Planner Vision Threadwhile Exit key not pressed do
while Request from Vision Client not received doWait for request
end whileSave current quadrotor statesSend acknowledgement to Vision Clientwhile Way-point from Vision Client not received do
Wait for way-pointend whileUse saved quadrotor states to transform way-point into Optitrack reference frameSend way-point (Optitrack coordinate) to autopilot
end while
The Optitrack Client transmits quadrotor’s attitude and position data to the MP
at 10Hz, and the Vision Client transmits the new target way-point to the MP at 1Hz.
Although the maximum update rate for Optitrack hardware is 25Hz and the maximum
refresh rate for image processing is 10Hz, these two components are throttled because
such a high update rate on quadrotor position control has been found to not be necessary.
The environment generation for the ISP is independent of everything else in the
system. The current design of ISP requires a pre-rendered environment, either static or
dynamic, to be projected on to the the glass floor via a separate computer. The wildfire
projection used in this research, for example, is a video captured from a MATLAB
simulation. The rest of the system, such as the ground station and the autopilot, are not
connected to the environment projection in any way.
Chapter 4. Indoor Simulation Platform 54
4.3 Experimentation Phases
A test plan that consists of multiple phases has been created for both computational sim-
ulation and physical experiments. This test plan, illustrated in Figure 4.8, ensures that
the system is fully tested on its basic functions before moving on to the final objectives.
Each of the blocks in Figure 4.8 is an experiment.
The ones on the left side in red are the experiments that need to be done in the
simulation. The first of these tests is controller verification. This would validate both
the quadrotor model and the controllers, including controller gains, in the simulation. A
well tuned controller is essential to future quadrotor experiments, and individual control
channels will be checked. Once the controllers have been tested, the way-point navigation
experiment will be done in simulation. This experiment validates the quadrotor’s ability
to follow way-point commands, which is important for a PBVS navigation. Finally, the
dynamic fire edge tracking can be tested in simulation.
Figure 4.8: The test plan for simulation and physical experiment
The indoor experiments are listed on the right side, in blue boxes. Due to the fact
that ArduPilot is a commercial open source autopilot, the confidence in the controller im-
plementation is high and the controller gains are the only variables that need to be tuned.
Therefore, the indoor physical experiment starts with way-point navigation test, which
would validate the quadrotor’s navigation functions using manual way-point commands.
Chapter 4. Indoor Simulation Platform 55
A target estimation test is needed afterwards to verify the image processing algorithm
and reference frame transformation code. Once the target can be estimated accurately,
the quadrotor would need to run a PBVS target tracking test. This ensures that the vi-
sion feedback loop is working properly and navigates based on image processing results,
instead of manual way-point commands. An intermediate edge tracking experiment needs
to be carried out to test the edge extraction code on a static fire edge. Finally, the full
dynamic fire edge tracking experiment would be completed once all sub-systems of the
experiment set up has been validated.
4.4 Summary
The ISP is an integration of multiple hardware and software systems, such as micro-
quadrotor, Optitrack cameras, ground station software, and image processing program.
The hardware of ISP, the micro-quadrotor in particular, gets updated over the course
of this thesis project in order to meet the specifications of the system. The software of
ISP is concentrated on the ground station computer, where three separate programs are
inter-connected in operation via an internal network. Together, they provide real-time
tracking, image processing, way-point identification, as well as wireless communication
capabilities. The indoor experiment has been broken down into several phases, each one
designed to verify the capabilities of the entire system while minimizing the chance of
catastrophic failure.
Chapter 5
Numerical Results
Numerical simulations of a quadrotor fire edge tracking mission has been carried out
in MATLAB. Quadrotor controllers, including attitude and position control, have to be
verified first to ensure that the quadrotor behavior in simulation is correct. Then, the
quadrotor is simulated with the dynamically expanding fire to see how well the edge
tracking strategy works. Finally, an attempt is made to reconstruct the fire perimeter
based on the data collected by the quadrotor while tracking the fire edge.
5.1 Quadrotor Navigation
In order to develop a simulation of the quadrotor edge tracking mission and verify the
fire perimeter estimation method proposed in this research, a simulated quadrotor with
functional navigation controllers is required. A non-linear quadrotor model is created
in MATLAB with the controller structure presented in Section 2.2.2. Various system
parameters used by this model are either measured, such as the weight of the quadrotor,
or approximated, such as the moments of inertia, based on the micro-quadrotor that is
being used on the ISP.
5.1.1 Controller Validation
Although the behaviours of the simulated quadrotor should be close to that of the actual
quadrotor, they are not an exact match. As a result, the various PID gains used for
the simulated controllers are different from those used in the ArduPilot. The controller
structure can be divided into two controller loop categories. The outer loop includes
the position controller, the yaw controller, and the altitude controller. Controllers in
this category are directly affected by the navigation commands. The inner loop includes
56
Chapter 5. Numerical Results 57
the roll and pitch controllers. These two controllers use the outputs of the outer loop
position controllers as reference signals. Inner loop PID controllers are tuned first and
usually require fast response to any changes in reference signals. After which, the outer
loop controllers can be tuned with the assumption that the inner loops are effectively a
part of the model.
On a quadrotor, roll and pitch behaviours should be the same in theory; therefore,
the roll and pitch controllers would use the same set of PID gains. Figure 5.1 shows the
simulated step response of roll angle tracking using the inner controllers.
(a) Angle and angular rate vs. time (b) Roll Moment vs. time
Figure 5.1: Step response for angle tracking
U2 from Eqn 3.14 is the roll moment. The pitch angle response would be exactly the
same. The quadrotor is capable of achieving the desired roll angle of 1 radian within 0.4
seconds. The force applied has a limiter of ±2N that prevents the model from generating
a force greater than the actual quadrotor could.
5.1.2 Way-Point Navigation
The three outer loop controllers of the quadrotor can be examined with a simulated way-
point based mission. The quadrotor starts at the origin on the ground and a series of
two way-points are defined for this mission, in the format of[x y altitude
]with unit
in meters . The first way-point is located at[0 5 1
], and the second way-point is at[
4 8 2]. The overall result of this simulation is shown in Figure 5.2
The simulation ran at a discrete time step of 0.01 second. The green dots indicate the
location of the home position of the quadrotor and the two way-points. The blue circles
represent the location of the quadrotor over time, and the vertical blue lines represent
Chapter 5. Numerical Results 58
Figure 5.2: The simulated way-point based quadrotor mission
the altitude of the quadrotor at every 100 time steps. The yaw angle tracking result is
shown in Figure 5.3.
Figure 5.3: Yaw angle tracking in simulated way-point navigation mission
The reference yaw signal is shown in red and the output of the quadrotor is shown
in blue. The yaw angle controller is capable of reaching the reference yaw angle within
1 second, and has no steady state error and minimum overshoot. Figure 5.4 shows the
altitude tracking and vertical velocity of the quadrotor in simulation.
A vertical velocity limit has been placed on the quadrotor model in order to produce
Chapter 5. Numerical Results 59
Figure 5.4: Altitude tracking in simulated way-point navigation mission
a more gradual climb that is similar to commercial autopilots, including the ArduPilot.
The climb rate limit prevents any noticeable overshoots in the system. Figure 5.5 shows
the position tracking results.
Figure 5.5: Position tracking in simulated way-point navigation mission
The X and Y position tracking of the quadrotor is limited by a maximum linear
velocity of 0.5m/s. Although most quadrotors in the real world would travel between
5m/s and 10m/s, these velocities are too fast for the ISP application. The velocity of
0.5m/s is more reasonable when flying indoors. A minor cross channel interference can
be observed at the start, when the quadrotor tries to turn from its initial heading of
0 deg to a reference heading of 90 deg in order to point at the next way-point. This
Chapter 5. Numerical Results 60
type of cross channel interference, between heading angle and position, is due to the
quadrotor attempting to move in a certain horizontal position, thus having a roll or pitch
angle, while changing its heading. The direction of horizontal movement changes in the
inertial frame when the quadrotor has a non-zero roll angle and a changing heading
angle. Although the PID controller will eventually fix this drift once the heading angle
stabilizes, it cannot provide an instant response to prevent such movement. Figure 5.6
shows the roll and pitch response of the quadrotor based on the requirements of the
position controllers.
Figure 5.6: Roll and pitch angle tracking in simulated way-point navigation mission
The reference roll and pitch angles are the outputs from the horizontal position con-
trollers. It can be observed that the responses are very quick and accurate. The steady
state errors in roll and pitch angles are also minimum. Finally the lateral error of the
quadrotor can be seen in Figure 5.7.
The lateral error plot shows how far off the quadrotor is from the intended track
linking between two way-points. Overall, the lateral error is constantly decreasing and
is negligible, even though it does not quite reach zero.
The way-point mission simulation proves that the PID gains for the controllers as
well as the controller structure can achieve good navigation results. The quadrotor in
simulation is capable of moving from way-point to way-point with very good tracking
performance and minimum error. All four control channels are stable and the simulated
quadrotor can be used in different way-point based missions.
Scaling up is possible because everything in the ISP, from fire simulation to the micro-
quadrotor, are scalable. In fact, wildfires in real world are most likely to be much larger
Chapter 5. Numerical Results 61
Figure 5.7: Lateral error in simulated way-point navigation mission
in size compared to the UAV. As a result, the local fire edge should appear more like a
straight line than a curve, making tracking easier for the UAV.
5.1.3 Quadrotor Fire Perimeter Tracking
A simulation of a quadrotor tracking the expanding fire edge is created to see how well the
fire perimeter could be approximated from the position data collected by the quadrotor
in theory. The environment used is a wildfire scenario with uniform but dynamic wind
conditions. Figure 5.8 shows the edge tracking result of a simulated quadrotor on a
dynamically expanding fire perimeter with ideal tracking.
(a) Isometric view (b) Bird’s eye view
Figure 5.8: Simulated fire edge tracking with quadrotor
Figure 5.8a is the isometric view of the simulation and presents the quadrotor’s move-
Chapter 5. Numerical Results 62
ment in the 3-dimensional space. Figure 5.8b presents the bird’s eye view of both the fire
perimeter over time and the quadrotor movement. Each of the red ellipses represent the
location of the fire perimeter at a time instance. The wind initially blows at a 45 degree
angle, but changes to 90 degrees in the middle of the simulation. This can be observed
from the difference in the spacings between the red ellipses in Figure 5.8b.
5.2 Fire Perimeter Reconstruction
The reverse elliptic approximation can be applied to the position data that forms the
quadrotor’s flight path to determine the fire perimeter location and shape at a single
time instance. Figure 5.9 shows the result of reverse elliptic approximation using the
technique presented in Section 2.3.1.
Figure 5.9: Finding local fire perimeter position using reverse elliptic approximation
The green ellipse represents the ground truth position of the fire perimeter at the time
instance when the quadrotor first encounters the fire. The goal is to estimate the shape
and position of this particular perimeter using the position data of the quadrotor and
the available wind information. The black line is the estimation result using the reverse
elliptic approximation technique. The estimation result is very similar to the actual fire
perimeter when the quadrotor is capable of ideal tracking. In fact, it can be said that
the estimated perimeter is a discretization of the fire perimeter.
The next step is to approximate the rest of the fire perimeter using the estimated local
fire perimeter. The aforementioned elliptic curve fitting technique is used to approximate
the entire fire perimeter. Figure 5.10 shows the result of this curve fitting technique
presented in Section 2.3.2.
Chapter 5. Numerical Results 63
(a) No noise (b) Gaussian noise
Figure 5.10: Full perimeter approximation using elliptic curve fitting technique
The quadrotor in Figure 5.10a has ideal tracking capabilities and experiences no noise
on its sensors. As a result, the ellipse curve fitting produces excellent results. In con-
trast, the elliptic perimeter approximated is very different from the actual perimeter if
the quadrotor tracking is not perfect, as can be seen in Figure 5.10b, where a gaussian
error with a mean of 0m and a standard deviation of 1m is added to the position data of
the quadrotor. A quadrotor’s tracking capability heavily relies on the sensor accuracy,
autopilot implementation, and the integration electromechanical components. As a re-
sult, ideal tracking does not exist in the real world and the results to be expected from
the ISP experiment should be closer to that of Figure 5.10b.
In the non-ideal tracking case, only the segment close to the reverse elliptic approxi-
mation result is accurate enough for any further processing. The rest of the ellipse does
not provide any useful information regarding the shape of the entire fire perimeter. In
the segment tracked by the quadrotor, 2nd order polynomial curve fitting provides bet-
ter results than ellipse fitting. Additionally, ellipse curve fitting can only provide good
estimation of the entire ellipse if the fire is assumed to be expanding elliptically. This is a
valid assumption in the constant and uniform parameter conditions, as well as dynamic
wind condition with limited change in wind parameters. However, this assumption would
not be valid in a real world scenario.
Chapter 6
Experimental Results
6.1 Quadrotor Navigation
Flight tests in the indoor environment can be dangerous for both the vehicle and po-
tentially the occupants; thus, the experimental phase of this research project is divided
into multiple phases, each assigned a small objective. The result of meeting each of these
objectives would mean that the ISP system is ready to move on to the next phase of the
experiment.
The progression of the experiment starts with verification of the quadrotor’s way-point
tracking capability within the Optitrack environment, which involves position feedback
control. Once this capability has been met, the next objective would be target tracking.
The quadrotor would be tasked with tracking a moving target using vision feedback.
Then the quadrotor needs to be able to switch from tracking a single object to a static
line. It would need to detect and follow the edge of the line from one side of the ISP
to the other. Finally, the quadrotor would be used to track a dynamically expanding
fire perimeter. The data collected from this test would be used for the fire perimeter
reconstruction experiment.
6.1.1 Way-Point Tracking
Position-based navigation is built upon the quadrotor’s ability to track a way-point. This
capability is evaluated in terms of the quadrotor’s horizontal position, heading angle, and
altitude. The quadrotor should be able to not only maintain its position at a single way-
point, but also move from one way-point to another.
In the first phase of the experiment, the quadrotor is commanded to hover at an
initial way-point, with additional way-point commands sent to the quadrotor manually.
64
Chapter 6. Experimental Results 65
Figure 6.1 shows the quadrotor’s way-point tracking capabilities in this experiment.
(a) X-axis position control (b) Y-axis position control
Figure 6.1: Way-point tracking experiment position results
Figure 6.1a shows the position of the quadrotor in the x-axis, along with the reference
position, which is the position of the way-point. Figure 6.1b shows the desired and actual
position data in the y-axis. The quadrotor takes off in manual mode, with the autopilot
switched on at the 10 second mark. The way-point moves in the x-axis at the 20 second
mark, and moves in the y-axis at approximately 45 second mark. Overall, the deviation
from the desired position is less than ±5cm. The quadrotor also tracks any changes
in the way-point very closely, provided that the change is gradual. Gradual changes in
way-points are needed to prevent excessive overshoot.
Figure 6.2a shows the heading angle of the quadrotor throughout this experiment.
The heading angle is fixed to 90 degrees in the Optitrack reference frame. The quadrotor
is capable of maintaining this heading angle command to an error of ±5 degrees.
Figure 6.2b shows the altitude of the quadrotor. The command altitude for this
experiment is 60cm. The altitude controller is engaged at the 10 second mark, and
any altitude changes prior to this is due to manual throttle control. The quadrotor can
maintain its altitude fairly well during hover, but has exhibits an initial drop in altitude
when it is commanded to move to another position.
Although the position of the way-points in this experiment are manually entered into
the system, the system is set up so that the ground station can generate its own way-
point commands based on the image processing results, which would become a part of
the PBVS control loop.
Chapter 6. Experimental Results 66
(a) Heading angle control (b) Altitude control
Figure 6.2: Way-point tracking experiment heading angle and altitude results
6.1.2 Target Tracking
A PBVS target tracking experiment is carried out to make sure that the quadrotor is
capable of navigating using automated way-point commands. The PBVS target tracking
experiment requires the quadrotor to follow a uniquely coloured object above the glass
floor of the ISP. The quadrotor’s altitude is manually specified to be 40cm, and its heading
angle is fixed to 180 degrees. The target object used in this experiment is a piece of blue
foam fitted with Optitrack tracker balls on top of a long rod, as illustrated in Figure 6.3.
Figure 6.3: The target used in the PBVS experiment
The long rod allows the target object to be moved around on top of the ISP glass
floor manually. The target is tracked by the Optitrack system via the tracker balls, so
that the quadrotor’s movement can be compared with the ground truth position data of
the target from Optitrack.
A relatively simple colour thresholding object tracking algorithm is used in this PBVS
Chapter 6. Experimental Results 67
experiment, because the target has a unique blue colour compared to the white back-
ground. A colour threshold filter for the blue colour is applied to the raw image, which
turns the pixels that are considered part of the target white, and pixels that are not the
target black. Then, an area based centre of mass calculation, provided by the OpenCV
library, is used to determine the location of target’s centre within the image. Figure 6.4
shows the result of applying the blue threshold filter on the raw image captured by the
on board camera on the left, and the position of the centre of the target object as the
quadrotor is hovering on the right.
(a) Threshold result (b) Target tracking
Figure 6.4: Determining the centre position of the target object via thresholding
The red line in Figure 6.4b indicates the movement of the centre of the object within
the image frame. In this particular case, however, this is a result of the movement of
the quadrotor in hovering mode with the target object being stationary. In the PBVS
target tracking test, the location of the centre of the target would be transformed from
the image coordinates to the global Optitrack coordinates. This location would become
the new target way-point for the quadrotor. The target way-point is being updated by
the ground station continuously throughout the target tracking mission. As a result, it is
important to make sure that this reference frame transformation process is tested before
the actual PBVS experiment is carried out. Figure 6.5 shows the result of the target
position estimation test.
The quadrotor is commanded to hover around the way-point[1600 1200
], which
represents 1600mm on the x-axis and 1200mm on the y-axis. While the quadrotor is
hovering, the target is moved around on the ISP glass floor, still within the field of view
of the on board camera. The estimated target position in blue follows the actual target
position in red very closely. The actual target position is taken from Optitrack and is
considered as the ground truth in this experiment. The maximum error in the X direction
Chapter 6. Experimental Results 68
Figure 6.5: Top down view of the target estimation test
is 60mm with a mean error of 7.8mm, whereas the maximum error in the Y direction
is 120mm with a mean error of only 6.5mm. The image processing and reference frame
transformation processes produce very accurate position estimates of the target object.
Finally, the PBVS experiment can be carried out knowing that the ISP could accurately
estimate the position of the target object. Figure 6.6 shows the result of this PBVS test.
The separate tracking responses for x and y-axis are shown in Figure 6.7.
The red line represents the position of the target over the duration of the PBVS
test, as measured by Optitrack. The blue line represents the estimated target position,
calculated by the Vision Client and ground station from the images taken by the on
board camera. The black line represents the flight path of quadrotor, whose goal is to
track the target by ensuring that the target remains at the centre of the image. The
target estimation is very accurate and the tracking by the quadrotor is good although
not exact. It can be observed that on both axes, the quadrotor’s position lags behind
the position of the target, which is to be expected in an object tracking test.
6.1.3 Line Tracking
The PBVS line tracking test is an extension of the target tracking test. In this test,
the target way-point is a point in front of the quadrotor that is also on the line that
has been detected. The line comes from a projection of a static image that includes a
straight fire edge. For scaling reasons, the projected fire edge has to be large relative to
Chapter 6. Experimental Results 69
Figure 6.6: Top down view of the PBVS tracking test
the quadrotor. Figure 6.8 shows the original image that is being projected.
The fire is located at the bottom of the image, and the fire edge would be expanding
upwards if this was a dynamic image instead of a static one. The fire edge retains the
yellow outer edge that represents newly burning areas, the red inner edge that represents
the areas that have been burning for a while, and the dark red areas at the bottom have
been burnt out. Figure 6.9 shows the top down view of the static line tracking result.
Due to the position of the projector relative to the ISP, the fire edge does not appear in
the middle of the glass floor as it does in the original image. The actual position of the
edge is measured by Optitrack markers based on the projection.
The green line represents the total width of the fire edge, which includes the yellow
and the red areas. The red line is the centre of the fire edge; the blue line is the position
of the fire edge estimated by the ground station, which is effectively the position of the
target way-points; and finally the black line is the path that the quadrotor took to track
the fire edge. The estimated fire edge position varies in the y-axis but overall remains
bounded to the upper and lower fire edge boundaries. The quadrotor’s roll angle affects
the accuracy of edge detection by altering the lighting of the raw image. The closeness
in colour space between yellow and different shades of red can affect the accuracy of the
estimation as well. As a result, it is acceptable for the estimated fire edge to be between
the upper and lower boundaries of the projected image. The x-axis tracking result for
the quadrotor is not as important as the y-axis, since the target way-point would always
Chapter 6. Experimental Results 70
Figure 6.7: X and Y-axis PBVS tracking result
Figure 6.8: Static straight edge being projected onto the glass floor
be a set distance in front of the quadrotor and the quadrotor should move forward in the
x-axis at a relatively constant velocity. Additionally, this plot depicts that the quadrotor
both overshoots and lags behind when trying to track the target way-points in the y-
axis. The lag is considered normal behaviour for tracking. Neither the overshoot nor the
lag are detrimental to the quadrotor’s line tracking capability and the test is considered
successful.
6.1.4 Dynamic Edge Tracking
The environment being projected on to the ISP glass floor is a simulation of dynamically
expanding fire edge created in MATLAB. This projection also uses the same scale as that
Chapter 6. Experimental Results 71
Figure 6.9: Top down view of static line tracking
of the static fire edge, which is a portion of the full perimeter simulation. Figure 6.10
shows the initial state of the entire fire perimeter, as well as the segment of the perimeter
that is actually projected.
The red ellipse is the fire perimeter on the simulation field. The blue rectangle
is the window used for capturing the video of the simulation. Everything within the
blue window is projected onto the ISP glass floor. Figure 6.11 shows the change of the
projected fire perimeter over time.
Figure 6.11a shows the initial state of the fire perimeter segment that is projected
onto the glass floor; 6.11b shows the perimeter segment after 40 seconds; and finally,
6.11c shows the final state of the fire perimeter segment after 80 seconds. The quadrotor
is capable of tracking this edge from right to left, flying counter-clockwise around the
fire, within 80 seconds. Figure 6.12 shows the quadrotor dynamic edge tracking result.
Similar to the static edge tracking result, the blue line is the estimated target position,
and the green line is the quadrotor flight path. The red lines, in this case, are the locations
of the fire perimeter at different time instances. The lowest red line is the initial fire
perimeter, which corresponds to Figure 6.11a; whereas the highest red line is the final
fire perimeter position, which corresponds to Figure 6.11c. The expansion of the fire
perimeter can be observed in the estimated target position, as the blue line moves from
lower right side to the upper left side. The quadrotor was able to keep the fire edge in
Chapter 6. Experimental Results 72
Figure 6.10: The initial state of fire perimeter and the segment projected
(a) (b) (c)
Figure 6.11: The progression of the dynamically growing fire in three time frames
view at all times and followed the expansion of the fire edge throughout this experiment.
6.2 Fire Perimeter Reconstruction
An attempt was made to use the quadrotor position data to approximate the location
of the fire perimeter, similar to the simulation; however, the quadrotor could not track
the actual fire perimeter as accurately as hoped. Although the quadrotor was able to
maintain the fire edge in view of the camera at all times, its own position was not as stable
as in the simulation. This means that the position of the quadrotor cannot accurately
represent the position of the fire edge. The estimated target positions, on the other
hand, are much more accurate and are used for fire perimeter reconstruction instead.
The images are sent from the on board camera to the Vision Client at a rate of 30 frames
Chapter 6. Experimental Results 73
Figure 6.12: Dynamic edge tracking result
per second; however, the Vision Client is set to only receive and process the images at 2
frames per second. Every 10th image that has been processed is saved for post analysis
purposes, which results in 1 image every 5 seconds. Over the course of 80 seconds of
flight, 16 images are saved for fire perimeter reconstruction. Figure 6.13 shows the raw
positions of the detected edges over the course of the dynamic edge tracking experiment
on the left, and the result of second order polynomial curve fitting after aligning the
curves to a single time step on the right.
(a) (b)
Figure 6.13: Detected edges and approximation of the segment
Chapter 6. Experimental Results 74
Figure 6.13a includes 16 edges. Each of these are from a different time instance, at
least 5 seconds apart, with the first edge on the right side and last on the left. The
locations of these edges show the progression of the fire perimeter, which moves from
bottom to top. These edges are synchronized into a single time instance using the reverse
elliptic approximation technique presented in Section 2.3.1. The resulting edges at the
initial time instance are coloured in red in Figure 6.13b. The blue line is the 2nd order
polynomial curve fitting result, which should be similar to the actual projected curve.
A comparison between the estimated perimeter, projected perimeter, and the actual
perimeter in simulation are shown in Figure 6.14.
(a) (b)
Figure 6.14: Comparison of estimated, projected, and actual fire perimeters
The red ground truth perimeter is the elliptic curve from the original MATLAB simu-
lation of a dynamically growing fire, after being converted into the Optitrack coordinate.
The blue projected perimeter is the curve that has been projected onto the ISP glass
floor. These two curves should be exactly the same in theory, and they do appear to
be very similar. The green detected curve is the same second order polynomial approx-
imation from Figure 6.13b. In this case, the detected curve is slightly higher than the
original. This can be explained by the fact that colour based detection for a projected
image is not always exact.
Figure 6.14b shows the ground truth perimeter in red, the second order approximation
curve in blue, and an elliptic estimation of the full fire perimeter in green. As with the
full fire perimeter result from the simulation experiment, the elliptic curve fitting can
only ensure reasonable approximation of the fire perimeter around the area where data
is collected. The perimeter is less likely to match that of the ground truth the further
away it is from the quadrotor’s tracking path. Elliptic curve fitting is very sensitive to
noise in the data. In certain cases, the parameters returned by the elliptic curve fitting
Chapter 6. Experimental Results 75
algorithm are imaginary numbers due to bad data and the total freedom in placing the
ellipse. It may be possible to minimize this by restricting the angle of the ellipse, since
the historic wind data could be obtained.
6.3 Outdoor Extension
The goal of the ISP is to provide a testing platform for various aerial trajectory tracking
and path planning algorithms. Although the wildfire perimeter tracking project is limited
to indoor experimentations, the Flight Systems and Control lab has an octorotor platform
for outdoor experimentations. The FSC octorotor consists of a Tarot T15 octorotor frame
with Pixhawk PX4 autopilot. The payload consists of a point and shoot camera with
satellite tagging and a 2 degrees of freedom gimbal. Figure 6.15 shows the octorotor
system.
Figure 6.15: FSC Octorotor flying a aerial imaging mission with full payload
There is a very close connection between this particular system and that of the ISP.
The PX4 autopilot is an open source autopilot developed by 3DRobotics, the same com-
pany that made the ArduPilot Mega. In fact, the PX4 and the ArduPilot uses the same
code. Additionally, they share the same ground station software and use the same com-
munication protocol library Mavlink. This means that modifications that have been done
on the ISP ground station software and the ISP quadrotor autopilot could also be done
on the outdoor octorotor platform. The same wildfire edge tracking algorithm, for ex-
ample, could be loaded onto the octorotor platform with relative ease, making the flight
tests completed on the ISP an integral part of any future outdoor octorotor flight tests
using the same algorithm. An edge detection analysis has been completed on an actual
aerial wildfire image using the same fire edge extraction algorithm used in the indoor
experiment. The result reveals that the algorithm developed for the indoor experiment
Chapter 6. Experimental Results 76
can be used for real world outdoor fire edge tracking experiment with minor adjustments.
Figure 6.16 shows the edge detection process being done on the sample aerial wildfire
image taken by CTV News in BC [29].
(a) Original[29] (b) Bilateral Filter (c) Thresholded
(d) Edge Detection (e) Selected Contour
Figure 6.16: Edge extraction of outdoor aerial wildfire image
The edge detection process used for the outdoor wildfire edge detection is very similar
to the process used for fire edge detection in the ISP experiment. First, a bilateral filter
is applied to the raw image to smooth out the noise. Then, a colour based threshold
filter is applied to pick out the colour that represents the object of interest. This would
be red in the case of fire detection. A Canny edge detector is used to pick out the edges
from the thresholded image. For way-point generation, the longest edge is picked as the
contour of interest and a new way-point can be found at the intersection between this
contour and the vertical centre line of the image. The image processing algorithm is able
to pick out the edge of the fire with relatively good accuracy and the way-point output
from this algorithm could be used as the feedback for navigation controller, similar to
the process integrated in the ISP system.
Chapter 7
Conclusions
The primary objective of this project was to develop a fire edge tracking algorithm
that could be tested in realistic scenarios. These scenarios consisted of a fully coloured
dynamically expanding fire perimeter, and a physical quadrotor that had to track this
moving fire edge. This navigation algorithm has been tested in both computational
simulation and on a physical indoor test bed, the ISP. The quadrotor was able to track
the expanding fire edge, and provided the desired position data of the fire. Although
approximation of the entire fire perimeter is unlikely in a real world scenario, the segment
of the fire that the quadrotor tracked could be accurately plotted. This allows for future
fire growth prediction, and provide valuable information for disaster relief personnel.
The Indoor Simulation Platform created for this thesis project provides indoor test-
ing capabilities of a wild fire tracking algorithm. This integrated test bed also allows for
experimentation of other various UAV path planning and trajectory tracking algorithms.
The development of this platform increases algorithm debugging and functionality im-
provement efficiency greatly by moving many of the validation tests from outdoor to the
much more easily accessible indoor environment.
In addition, this system is fully compatible with the FSC octorotor system that is
used for outdoor flight tests. The octorotor uses an autopilot that shares the same code
as the one used for the ISP; therefore, it is relatively easy to insert the tracking code into
the octorotor system. This outdoor experiment could further verify the usefulness of the
ISP as an intermediate testing platform for UAV navigation algorithms.
77
References
[1] N. R. Canada, “National Wildland Fire Situation Report,” http://cwfis.cfs.nrcan.
gc.ca/report, accessed: 2014-07-08.
[2] D. Schroeder, “Evaluation of Three Wildfire Smoke Detection Systems,” http:
//wildfire.fpinnovations.ca/61/EvaluationOfThreeWildfireSmokeDetectionSystem.
pdf, accessed: 2014-09-17.
[3] NASA, “Global Fire Monitoring - NASA and NOAA Missions for Monitoring Global
Fires,” http://earthobservatory.nasa.gov/Features/GlobalFire/fire 5.php, accessed:
2014-09-17.
[4] R. Rothermel et al., “A Mathematical Model for Predicting Fire Spread in Wildland
Fuels,” in USDA Forest Service Research Paper, Ogden, Utah, 1972.
[5] D. Anderson et al., “Modelling the Spread of Grass Fires,” in Journal of the Aus-
tralian Mathematical Society, 1982, pp. 451–466.
[6] G. Trunfio et al., “A New Algorithm for Simulating Wildfire Spread Through Cel-
lular Automata,” in ACM Transactions on Modelling and Computer Simulation,
2011.
[7] R. Leishman et al., “Quadrotors and accelerometers: State estimation with an im-
proved dynamic model,” Control Systems, IEEE, vol. 34, no. 1, pp. 28–41, Feb 2014.
[8] A. Salih et al., “Modelling and PID Controller Design for a Quadrotor Unmanned Air
Vehicle,” in International Conference on Automation Quality and Testing Robotics,
Cluj-Napoca, Romania, 2010, pp. 1–5.
[9] S. Waslander et al., “Multi-Agent Quadrotor Testbed Control Design: Integral Slid-
ing Mode vs. Reinforcement Learning,” in Intelligent Robots and Systems, Edmon-
ton, Canada, 2005, pp. 3712–3717.
78
References 79
[10] J. Canny, “A Computational Approach to Edge Detection,” IEEE Transaction on
Pattern Analysis and Machine Intelligence, vol. PAMI-8, no. 6, pp. 679–698, Nov
1986.
[11] D. Hong et al., “A Threshold Strategy for Edge Tracking,” in IEEE International
Conference on Fuzzy Systems, New Orleans, USA, 1996, pp. 1886–1891.
[12] T. Qiu and Y. Yan, “An Autoadaptive Edge-Detection Algorithm for Flame and
Fire Image Processing,” IEEE Transactions on Instrumentation and Measurement,
vol. 61, pp. 1486–1493, May 2012.
[13] H. Chen and Q. Gao, “Integrating Color and Gradient into Real-time Curve Track-
ing,” in Canadian Conference on Computer and Robot Vision, Windsor, Canada,
2008, pp. 294–300.
[14] X. Zheng and Q. Gao, “Generic Edge Tokens: Representation, Segmentation and
Grouping,” in International Conference on Vision Interface, Halifax, Canada, 2003,
pp. 426–460.
[15] D. Casbeer et al., “Forest Fire Monitoring with Multiple Small UAVs,” in American
Control Conference, Portland, OR, 2005, pp. 3530–3535.
[16] P. Sujit et al., “Cooperative Forest Fire Monitoring Using Multiple UAVs,” in IEEE
Conference on Decision and Control, New Orleans, LA, 2007, pp. 4875–4880.
[17] T. Kawano et al., “Generating Target Path For Tracing a Line Before Mission the
Traced Line of Dead Angle of Camera,” in SICE-ICASE International Joint Con-
ference, Bexco, Korea, 2006, pp. 5286–5289.
[18] R. Holt, R. Beard et al., “Vision-Based Road-Following Using Proportional Naviga-
tion,” in Journal of Intelligent Robotic Systems, vol. 57, 2009, pp. 193–216.
[19] J. Clark and R. Fierro, “Cooperative Hybrid Control of Robotic Sensors for Perime-
ter Detection and Tracking,” in American Control Conference, Portland, OR, 2005,
pp. 3500–3505.
[20] M. Kemp et al., “Multi-UUV Perimeter Surveillance,” in IEEE Autonomous Under-
water Vehicles, 2004, pp. 102–107.
[21] S. Rangavajhala and A. Shirkhodaie, “Visual-Servoing and Navigation of Mobile
Robots for Target Tracking Applications,” in Southeastern Symposium on System
Theory, 2001, pp. 257–262.
References 80
[22] B. Thuilot et al., “Position Based Visual Servoing: Keeping the Object in the Field
of Vision,” in International Conference on Robotics and Automation, Washington,
DC, 2002, pp. 1624–1629.
[23] M. Bachiller et al., “Well Structured Robot Positioning Control Strategy for Position
Based Visual Servoing,” in International Conference on Robotics and Automation,
Seoul, Korea, 2001, pp. 2541–2546.
[24] A. Begum et al., “A Simple Visual Servoing and Navigation Algorithm for an Om-
nidirectional Robot,” in International Conference on Human-Centric Computing,
Cebu, Philippines, 2010, pp. 1–5.
[25] W. Fu et al., “Visual Servoing Based Mobile Robot Navigation Able to Deal with
Complete Target Loss,” in International Conference on Methods and Models in Au-
tomation and Robotics, Miedzyzdroje, Poland, 2013, pp. 502–507.
[26] 3DRobotics, “Connecting RC Outputs to the Motors,” http://copter.ardupilot.com/
wiki/connecting-your-rc-input-and-motors/, accessed: 2014-07-23.
[27] C. Tomasi and R. Manduchi, “Bilateral Filtering for Gray and Color Images,” in
International Conference on Computer Vision, Bombey, India, 1998, pp. 839–846.
[28] M. Koga, “Summer Project Report: Quadrotor Parameter Estimation,” Aug, 2014.
[29] CTVNews, “Wildfires Prompt Evacuations in BC, Alberta,” http://www.ctvnews.
ca/canada/wildfires-prompt-evacuations-in-b-c-alberta-1.1916251, accessed: 2014-
07-23.