persistent uav service: overviewxs3d.kaist.ac.kr/lab activity/2014 spring lab... · guidance,...

2014 spring lab seminar

Persistent UAV service: Overview

xS3D

Department of Industrial and Systems Engineering

KAIST, South Korea

Thursday, March 6, 2014


Presentation Overview

• Motivation for persistent service

• Overall orchestration of UAV service system

• UAV service system : Components and prototype

- Central planning

- UAV guidance system

- Automatic replenishment station

- System demonstration

• Concluding remarks


Motivation

• Large expensive UAVs

– Usually military purpose

– Operate for many hours

– Travel long distances

• Small inexpensive UAVs

– A lot of application area such as tracking, communication relay,

environmental / fire / national boundary monitoring, cartography, disaster

relief and so on.

– Limited duration of mission

– Limited distance

• To increase the usability of small UAVs, systems for persistent operation is required

– Collection of UAVs, refueling stations, automatic operation system

– Methods to orchestrate their operations (scheduling issue)


Overall orchestration of UAV service system

Service station 1

Service station 2

Object 1

Object 2

Object 3Service station 3

UAV 1

UAV 2

UAV 4

UAV 3

UAV 5

Moving

objective

trajectory

UAV service

system

Persistent UAV service

Random arrival

of customer

information Random path and

duration as

customer request

Vision

technology

Heterogeneous

UAVs

UAV operation

system

Central

planning

Automatic

replenishment

station



Service station 1

Service station 2

Object 1

Object 2


UAV 1

UAV 2

UAV 4

UAV 3

UAV 5

Moving

objective

trajectory

UAV service

system


UAV operation

system

Automatic

replenishment

station

Central

planning

Vision

technology

Heterogeneous

UAVs

Random path and

duration as

customer request



Service station 1

Service station 2

Object 1

Object 2


UAV 1

UAV 2

UAV 4

UAV 3

UAV 5

Moving

objective

trajectory

UAV service

system


UAV operation

system

Automatic

replenishment

station

Central

planning

Vision

technology

Heterogeneous

UAVs


System description

• Persistent UAV system with distributed multiple service stations – Follow deterministic time-space trajectories without interruption

– Capacitated UAVs can use any station and should return after mission

0 100 300 200 400 500 600

100

200

300

Station 1

Station 2


System description

Customer & UAV information Optimization algorithm Detail Schedule


System Description

■ To follow a time-space trajectory, the trajectory is divided into pieces (split jobs)

Start

End

Service station 1

Service station 2

1

2

3

14 4

5

6 7

∙ ∙ 13

▪ Objective moves

- From point (50,250) to (950,750)

- From 13:06 to 13:20

2 2 2 2( ) ( ) ( ) ( )i j i j j i j i

ij e s e s ji e s e sd x x y y d x x y y

Split

job

Start

point

End

point

Start

time

End

time

1 50,250 150,250 13:06 13:07

2 150,250 250,250 13:07 13:08

3 250,250 350,250 13:08 13:09

4 350,250 450,250 13:09 13:10

5 450,250 550,250 13:10 13:11

6 550,250 650,250 13:11 13:12

7 650,250 750,250 13:12 13:13

8 750,250 850,250 13:13 13:14

9 850,250 950,250 13:14 13:15

10 950,250 950,350 13:15 13:16

11 950,350 950,450 13:16 13:17

12 950,450 950,550 13:17 13:18

13 950,550 950,650 13:18 13:19

14 950,650 950,750 13:19 13:20


Mathematical Model

i, j : Indices for jobs

s : Index for stations

k : Index for UAVs

r : Index of a UAV’s rth flight

NJ : Number of split jobs

NUAV : Number of UAVs in the system

NSTA : Number of recharge stations

NR : Maximum number of flight per UAV during the time horizon

M : Large positive number

(xjs, y

js) : Start point of split job j

(xje, y

je) : End point of split job j

Dij : Distance from split job ith finish point to split job jth start point, Dij ≠ Dji

Ei : Start time of split job i

Pi : Processing time or split job i

qk : Maximum traveling time of UAV k

Sok : Initial location(station) of UAV k

TSk : Travel speed of UAV k

• Notations


Mathematical Model

ΩJ : = {1, …, NJ}, Set of split jobs

ΩJD : = {1, …, NJ+1}, Set of split jobs and dummy jobs

ΩSS : = {NJ+2, NJ+4, …, NJ+2∙ NSTA}, set of UAV flight start station

ΩSE : = {NJ+3, NJ+5, …, NJ +2∙ NSTA+1}, set of UAV flight end station

ΩA : = (ΩJD U ΩSS U ΩSE) = {1,…, NJ+2∙NSTA+1}, set of all jobs and recharge stations

▪ Xijkr = 1 if UAV k processes split job j or recharges at station j after processing split job i or recharging at station i during the rth flight; 0, otherwise

▪ Cikr is job i’s start time by UAV k during its rth flight or UAV k’s recharge start time at station i; otherwise its value is 0.

▪ Yikr = 1 if UAV k processes split job i during its rth flight; 0, otherwise.

• Notation

• Decision Variables


Mathematical Model

1, 1 ( , 1... 1, )JD JD

iskr s ikr R SE

i i

X X k K r N s

1 ( , )SE JD

iskr

s i

X k K r R

1 ( , )SS JD

sjkr

s j

X k K r R

, 1 1 ( )ok

JD

s jk

j

X k K

Subject to

1, 1 ( , 1... 1, )skr s kr R SEC C k K r N s

1 ( )A

ijkr J

k K r R i

X j

0 ( , , )A A

ijkr jikr JD

j j

X X i k K r R

0 ( , , )JD

iskr SS

i

X k K r R s

A A

ij ijkr

k K r R i j

D X

Minimize

Network flow constraints

Initial recharge station constraints

Split job assignment constraints

• Mathematical formulation


Mathematical Model

/ (1 ) ( , , , )ikr i ij k jkr ijkr JD SS JD SEC P D TS C M X i j k K r R

( , , )JD SE

ijkr ikr J

j

X Y i k K r R

( , , )ikr ikr JM Y C i k K r R

( )ikr i J

k K r R

C E i

/ ( , )A A JD A

ij k ijkr i ijkr k

i j i j

D TS X P X q k K r R

, 1, ( , , )sdkr d s kr SSX X k K r R s

0 ( , , )dikr idkr JX X k K r R i

0 ( , , )ikr AC k K r R i

{0,1} ( , , , )ijkr A AX k K r R i j

{0,1} ( , , )ikr AY k K r R i

Start time consts

Fuel and battery constraints

Dummy job constraints

Decision variables

• Mathematical formulation


UAV guidance system

1280 720 pixel front camera

320 240 pixel belly camera

< AR drone 2.0 >

< Ipad 3>

< WIFI >


UAV guidance system

■ Roles of UAV guidance system

1. Receive and implement the schedule from the MILP.

2. Convert the video from the UAV cameras into usable information for directing the motion of

the UAVs

3. Enable a human overseer to monitor the UAV progress via video and adjust feedback control

gain values for various situations

4. Allows for a human overseer to initiate emergency actions such as immediate landing.


UAV guidance system

Descriptions

① : Front(Bottom) Camera Video

② : Start Procedure Interface

③ : Color Filtered Video

④ : Control Gain Adjustment Sliders

⑤ : Emergency Landing Button

1. The color video from the camera is acquired via

TCP port and processed using OpenCV framework.

2. The image is separated into three RGB channels.

These three images are used to determine the color

of the targeted image.

3. Control inputs including the longitudinal-lateral tilt

angles, height and yaw angular velocity are calculated

from the number and mean coordinate of target pixels

in the processed image.


Automatic replenishment station

▪ Each AR Drone 2.0 uses a three cell lithium

polymer battery

▪ four copper leads (three for each terminal and

one for the ground terminal) were threaded from

the battery inside the UAV to the four feet of the

drone

▪ The service station consists of four pads, one for

each foot of the drone.

▪ Each such pad connects to the UAV battery via

the leads on the drone feet


System demonstration


System demonstration

UAV Start

station

Assigned

job

End

station

Service

start

time

Service

end

time

1 1 1,2,3,4 2 2 10

2 2 5,6,7,8 3 10 18

■ Demonstration description

Station 1Station 2

Station 3

UAV 1

UAV 2

Hand-off

Split

job 1

Split

job 2

Split

job 8

Split

job 7

∙ ∙ ∙

< Schedule by MILP >

< Demonstration layout >

5m


Experiments

■ Demonstration video


Concluding remarks

• To increase the usability of small UAVs, prototype systems for persistent operation is developed.

• As a components of persistent UAV service system, MILP for deriving UAV schedules, UAV

guidance system , automatic replenishment station were suggested.

• MILP generates UAV schedules using customer information and system resource information

such as location of station and number & location of UAVs.

• UAV guidance system provides uninterrupted customer tracking service by using vision

technology.

• Demonstration shows the orchestrationof those system components and applicability of proposed

UAV service system.


Literature Review

• Scheduling methods without a distance or time restriction – T. Shima and C. Schumacher, “Assignment of cooperating UAVs to simultaneous tasks using genetic algorithm,” In Proc. AIAA

Guidance, Navigation, and Control Conference and Exhibit, San Francisco, 2005 – J. Zeng, X. Yang L. Yang and G. Shen, “Modeling for UAV resource scheduling under mission synchronization,” Journal of Systems

Engineering and Electronics, Vol. 21, No. 5, 2010, pp. 821-826

• Scheduling methods for limited flight duration – A. L. Weinstein and C. Schumacher, “UAV scheduling via the vehicle routing problem with time windows,” In Proc. AIAA

Infotech@Aerospace 2007 Conference and Exhibit, Rohnert Park, California, 2007 – T. Shima, S. Rasmussen and D. Gross, “Assigning micro UAVs to task tours in an urban terrain,” IEEE Transactions on Control Systems

Technology, Vol. 15, No. 4, 2007, pp. 601 – 612 – Y.S. Kim, D.W. Gu and I. Postlethwaite, “Real-time optimal mission scheduling and flight path selection, IEEE Transactions on

Automatic Control, Vol. 52, No. 6, 2007, pp. 1119-1123. – B. Alidaee, H. Wang, and F. Landram, “A note on integer programming formulations of the real-time optimal scheduling and flight

selection of UAVS,” IEEE Transactions of Control Systems Technology, Vol. 17, No. 4, 2009, pp.839-843

• Scheduling method for persistent UAV operation – M. Alighanbari and J. P. How, “Decentralized task assignment for unmanned aerial vehicle”, Proceedings of the 44th IEEE Conference

on Decision and Control, and the European Control Conference 2005 seville, spain, december 12-15, 2005

• Battery recharge/exchange methods – J. How, thesis papers at MIT, 2005, 2007 – A.S. Kurt, B.H. Clarence, R.R. Johnhenri, D.W. Richardson, Z.H. White, Q. Elizabeth and G. Anouck, “Autonomous Battery Swapping

System for Small-scale Helicopters”, 2010 IEEE International Conference on Robotics and Automation – R. Godzdanker, M. J. Rutherford and K. P. Valavanis, “ISLANDS: A self-leveling platform for autonomous miniature UAVs”, 2011

IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp 170-175 – A.O.S. Koji, K.F. Paulo and James R. Morrison, “Automatic battery replacement system for UAVs: Analysis and design” Journal of

Intelligent and Robotic Systems, Special Issue on Unmanned Aerial Vehicles (Springer), a Special Volume on Selected Papers from ICUAS’11, Vol. 65, No. 1, pp. 563-586, January 2012. First published online September 9, 2011

– M. Valenti, D. Dale, J. P. How and D. P. de Farias, “Mission health management for 24/7 persistent surveillance operations”, AIAA Guidance, Navigation and Control Conference and Exhibit, 20-23 August 2007, Hilton Head, South Carolina

persistent uav service: overviewxs3d.kaist.ac.kr/lab activity/2014 spring lab... · guidance,...

Documents