Gaussian Process based Subsumption of a Parasitic Control Component

Allan M. Axelrod, Hassan A. Kingravi, Girish V. Chowdhary

Abstract— Many existing control architectures assume thatthe main control system being designed is the only controllerthat governs a system’s actuators. However, with the increasingavailability of off-the shelf controls packages, the number ofinternal unadjustable control systems is increasing. Some ofthese control systems may behave in parasitic way by enforcinga rigid set of behaviors that could disrupt a desired systembehavior. We present a control architecture that can subsumeparasitic control behavior through iteratively shaping the maincontrol command with an intelligent feed-forward term. Ourarchitecture requires very little prior knowledge about thesubsystem whose behavior is to be subsumed, rather it relieson online learned sparsified predictive Gaussian Process (GP)models. We provide rigorous quantifiable bounds relating thesparsification of the GP to the accuracy in estimating andsubsuming the parasitic subsystem. The presented subsumptionarchitecture is realized using a variant of D-Type iterativelearning control (ILC) and is validated through a series of flighttests on a Parrot AR Drone 2.0 quadrotor where the quadrotor’ssonar based altitude control loop’s behavior of maintaining afixed altitude over ground surfaces is subsumed through a maincontroller via a feed-forward term.

I. INTRODUCTION

An inherent assumption in designing controllers for manysystems is that the control system has full authority over theactuators. Indeed, many adaptive, reconfigurable, disturbancerejecting, probabilistic, iterative and optimal robot controlarchitectures assume that the control system being designedis the only system that can adjust the robot’s actuators[4, 6, 7, 12, 17, 18, 20, 22, 25–27, 32, 41]. However, thisassumption does not always hold in practice. For example,several aerial robots today come integrated with CommercialOff The Shelf (COTS) stability augmentation systems thatcommand the same actuators that the designer is wishingto control. Very little information is available about howthese COTS components work, and they often do not outputsufficient information to enable the designer to guess theirinternal state. The prevalent practice has often been to simplyignore the existence of such internal control loops, and buildthe control architecture around them. This approach canfail however if the internal system is preventing the robotfrom achieving some desired or nominal behavior. In thiscase, undesirable system responses may result even when noexternal disturbance or damage is imparted on the system,but the system responds as if there is some perturbation.

A. M. Axelrod, and A. Prof. G. V. Chowdhary are with Mechanicaland Aerospace Eningeering at Oklahoma State University, Stillwater OK,{allanma,girish.chowdhary}@okstate.edu. H.A. Kingravi wasa postdoctoral associate at Oklahoma State University and is now withPindrop security, {hkingravi@yahoo.com}. This work was sponsoredin part by Department of Energy Award Number DE-FE0012173 and theAir Force Office of Scientific Research Award Number FA9550-14-1-0399.

System Actuator

Known Control

Component

= - +

e udsr

uact

x1

x1

y udis

Fig. 1: The difference between a disruption, denoted by thesubscript dsr, and a disturbance, denoted by the subscript dis,is illustrated

Another situation where this may happen is when a Cyberattack is stealthily changing the control response. We termsuch perturbations imparted by parasitic unadjustable controlelements as disruptions. Figure 1 illustrates that while anexternal disturbance affects the forces acting on the system,an internal disruption changes the control command itself.

This paper presents a method to address the relatively un-addressed problem of designing reliable control systems forrobots that have parasitic control loops. One approach totackle this problem can be to attempt to establish control au-thority by rewriting the actuator drivers online. Yet, establish-ing control authority by direct hardware level manipulationis not always necessary, nor practical in an online setting. Analternate approach that is pursued here comes from adjustingthe control commands of the robot’s controller to subsumethe parasitic control input. The main challenge in realizingthis approach is that the behavior of the internal control loopis unknown and can be stochastic [1].

The key contribution of this paper is an online controlarchitecture for subsuming parasitic, uncertain, unadjustable,internal control components through control command shap-ing. Our approach relies on building a predictive model ofthe internal parasitic control component. Therefore, unlikepurely reactive control architectures – such as model freedisturbance rejection [40] or adaptive control [37]– thatwould rely on modifying the control commands only afterthe parasitic control causes a disruption of normal operation,our approach is able to proactively anticipate the change inadvance to shape the control command. To accommodatethe stochasticity in the response of the parasitic component,we model its behavior as a distribution over functions usingthe Gaussian Process (GP) Bayesian Nonparametric (BNP)Prior [39]. Our choice of the GP-BNP is motivated fromthe fact that GP-BNPs are flexible enough to adapt to anunknown operational domain and can inherently handle mea-surement and process noise. To ensure that the architectureis implementable on resource constrained robotic hardware,we employ Csato and Opper’s sparse online GP regression

algorithm which can automatically adjust the kernel set tomeet a computational budget [10]. Another key contributionover past work on control using sparse GPs [9] in our paperis that we provide rigorous bounds on the accuracy of theonline sparsified GP estimate of the underlying stochasticprocess. These bounds provide a quantifiable way to se-lect the computational budget for the presented GP basedsubsumption architecture, and can be directly generalizedto other GP based control architectures [2, 9, 13, 28, 29].The presented architecture is validated through a series offlight tests on the Parrot AR Drone 2.0 quadrotor UnmannedAerial Vehicle (UAV). The AR Drone is a popular and lowcost aerial robot that many researchers are using for indoorflight experimentation. The AR Drone’s internal sonar basedaltitude control loop is designed to keep the vehicle 1 meterabove any underlying surface. However, the user has no wayto directly switch off or control the sonar behavior. Thisinternal control loop can be parasitic if the intended behaviorrequires the UAV to fly at altitudes below 1 meter. Wedemonstrate the effectiveness of the presented architecture tosubsume the quadrotor’s internal, unadjustable, sonar basedaltitude control loop.

II. RELATED WORK

Subsumption architectures deal with creating complexbehaviors by combining simpler behaviors. Existing workin subsumption hopes to decouple the subsystems from oneanother [33, 38], however this is not always possible. Thecoupling problem from subsumption literature motivates ourapproach to negate the commands from the parasitic layer(s)of a control architecture.

There is very little direct work in designing control sys-tems to subsume the behavior of parasitic, unadjustable, andunobservable control subsystems. Cascaded control systemshave been studied [5, 6, 14, 15, 21, 23, 36], however, theunderlying assumption in those works is that the designercontrols the behavior of each subsystem in the cascade.Another relevant body of literature is that of stochasticdisturbance rejection [4, 15, 40]. However, disturbance re-jection assumes sufficient control authority to reject exter-nal forces, while we are concerned with subsumption ofparasitic control competing for the same control authority.Reconfigurable control and fault detection is concerned withcontrol strategies that are robust to sensor and actuator failureas well as structure modification. In most reconfigurablecontrol systems, the control loops are assumed to be knownand adjustable [17, 41]. However, existing reconfigurablecontrol strategies may be ill-suited to detecting and rejectingparasitic control components or a cyber attack, such asStuxnet, since the control code introduced by the cyberattack is unknown and it may not trigger component failuredetection [24]. Our architecture can lead to reconfigurationstrategies in presence of such unknown control components,provided uncorrupted feedback signals.

In Iterative Learning Control (ILC) literature, it is assumedthat the control signal from a previous iteration is known andreproducible [4, 12, 16, 18, 25, 32]. Yet in the presence of

a parasitic control component, only the part of the controlsignal is known and reproducible. However, the unknowncontrol signal may be modeled as a distinct class of distur-bance and rejected using a variant of D-type ILC [35].

III. SUBSUMPTION OF PARASITIC CONTROL

In this section, we formulate a feed-forward term forsubsuming an uncertain, stochastic parasitic subsystem. Thedynamics of the robot are defined as

x(t) = f(t, x(t), u(t)), (1)

where the state is x(t) ∈ Rn, and the signal sent to thesystem actuator is u(t) ∈ Rl. The nominal performance errorof the system, enom(t), is defined in terms of a referencestate, xref(t), and the state of the nominal system, x(t), as

enom(t) = xref(t)− x(t). (2)

For brevity, the time argument t is omitted henceforth. Sup-pose that the system (1), receives a control input composedof a known control component uact(·), and an unknown un-controllable component udsr(·). This unknown unadjustablecomponent is referred to as a parasitic control component.It is assumed that the behavior of the parasitic componentis stochastic and not directly measurable, that is, it is notpossible to get samples of udsr(·). Furthermore, let θnomindicate that the parasitic control component is active, butdoes not cause the system to deviate from the desirednominal performance. The nominal system response u(x),is

u(x) = uact(enom) + udsr(θnom, x). (3)

Subsequently, the parasitic control changes it’s behavior toθdsr which causes the system to deviate from the nominalperformance as

edsrj = xref − x. (4)

and

u(x) = uact(edsr) + udsr(θdsr, x). (5)

Iterative learning control is applied to subsume the unknownparasitic control signal at each iteration τ as

u(xτ+1) = uact(edsrτ+1) + udsr(θdsr, xτ+1) +

τ∑j=1

ushpj ,

(6)

where ushpj is a feed-forward subsuming signal

ushpj = uact(edsrj ) + uact(edsrj − enom), (7)

and where uact(edsrj − enom) is a variant of the D-typeupdate law. The error terms for ushpj are obtained usingGaussian Process regressions as in (13) and (14). Bound onthe predicted errors and a bound on the predicted residualerror edsrj − enom are provided by Theorems 1 and 3,respectively. The Bayesian feed-forward subsuming signalis then

ushpj = (I − Σ21j )(uact(edsrj ) + uact(edsrj − enom)), (8)

where Σ1j is calculated using (11). The predictive uncer-tainty, Σ2

1j , ranges from the zero matrix to I and is used toscale the feedforward control term. A predictive uncertaintyof I indicates an uncertain GP prediction, so then the baselinecontroller and the shaping term for iteration τ is cancelledout. A predictive uncertainty of the zero matrix indicates ahighly confident GP prediction, so then the shaping term iscompletely utilized.

IV. GP-BASED SUBSUMPTION

The form of udsr(θnom(t), x(t)) is potentially problematic;its response is stochastic and the range of values that it mayassume is unknown. An effective strategy for modeling suchstochastic responses is by utilizing a data-driven Bayesiannonparametric method; in particular, Gaussian Processes(GPs) suit this need since they have been theoretically andpragmatically demonstrated to be an effective nonparametricmethod for model reference adaptive control (MRAC) in thepresence of stochasticity. A model of the nominal system isbuilt by training an online GP on enom(t) and a model ofthe disrupted system is built by training an online GP onedsrj (t). In what follows, the data on which the GPs aretrained on is denoted by s(t) = [y(t), ey(t)], where y(t) isthe position in the y component, and ey(t) is the error for theposition in the y component, and s(t) = ez(t), where ez(t)is the position error in the z component. For brevity, we willsometimes drop the time dependence in the equations.

A. GP Regression

A GP is defined as a collection of random variables suchthat every finite subset is jointly Gaussian. The joint Gaussiancondition means that GPs are completely characterized bytheir second order statistics [39]. A GP is a distribution overfunctions, that is, a draw from a GP is a function. For the sakeof clarity of exposition, we will assume that enom(r) ∈ Rand edsrj (r) ∈ R; the extension to the multidimensional caseis straightforward. When a process ∆ follows a Gaussianprocess model, then

∆(·) ∼ GP(m(·), k(·, ·)), (9)

where m(·) is the mean function, and k(·, ·) is a real-valued, positive definite covariance kernel function. UnderGP regression, the mean is assumed to lie in the class offunctions H, a reproducing kernel Hilbert space (RKHS).

Let Rτ = {r1, . . . , rτ} be a set of state measurements,discretely sampled where {1 . . . τ} are indices for the dis-crete sample times {t1, . . . , tτ}. The set defines a covari-ance matrix Kij := k(ri, rj). Given indexed sets A andB, K(A,B) denotes the kernel matrix generated by theevaluations Kij = k(ai, bj) between the two sets, where ai ∈A, bj ∈ B. For each measurement ri, there is an observedoutput s(ri) = m(ri)+εi, where εi ∼ N (0, ω2). The stackedoutputs give s = [s1, . . . , sτ ]T . The most common choice ofcovariance kernel, and the one used here, is the GaussianRBF kernel [39]. It can be shown that the mean function

and posterior variance in GP regression can be computed as

p(sτ+1|Rτ , sτ , rτ+1) ∼ N (mτ+1, Στ+1), (10)

where

mτ+1 = βTτ+1krτ+1, Στ+1 = k∗τ+1 − kTrτ+1

Cτkrτ+1, (11)

are the updated mean and covariance estimates, respectively,and where Cτ := (K(Rτ , Rτ ) + ω2I)−1, βτ+1 := Cτsτ ,krτ+1

= K(rτ+1, Rτ ) and k∗τ+1 = k(rτ+1, rτ+1).Since both Rτ and sτ grow with data, computing the

inverse becomes computationally intractable over time. Thisis less of a problem for traditional GP regression appli-cations, which often involve finite learning samples andoffline learning. However, in an online setting, the lineargrowth in the sample set cardinality degrades computationalperformance. Therefore, the extension of GP regression forcontrol requires an online method to restrict the number ofdata points stored for inference. Since the set R generatesa family of functions FR ⊂ H whose richness characterizesthe quality of the posterior inference, a natural and simpleway to determine whether to add a new point to the subspaceis to check how well it is approximated by the elements in R,using the kernel linear independence test [39]. This restrictedset of selected elements, called the basis vector set, isdenoted by BV . When incorporating a new data point into theGP model, the inverse kernel matrix can be recomputed witha rank-1 update. When the budget is exceeded, a basis vectorelement must be removed prior to adding another element[31]. There are many schemes to remove the basis vector;in our experiments, we rely on a method that efficientlyapproximates the KL divergence between the current GP andthe (t + 1) alternative GPs missing one data point each,then deletes removes the data point with the largest KLdivergence. See [39] for more details.

B. Approximation Error

In this section, review bounds quantifying the effectivenessof the approximate mean m computed by the online GPalgorithm. Later, these bounds will be used to providestronger bounds for the subsuming feed forward term. Notethat when a basis vector is either replaced or added to BV ,it induces a modified GP, whose mean and covariance wedenote by e(r) ∼ GP(mσ(r), k(r, r′)).

Let τ−2Kij := τ−2k(ri, rj) be the kernel matrix asso-ciated to m (assumed normalized), and let |BV| = pmax

Then the approximate mean is associated with a kernelmatrix induced by a quantization operator ϑ : {1, . . . , τ} →{1, . . . , pmax}, such that ri 7→ cϑi, where cϑi ∈ Dx are theset of chosen centers BV . Let mσ(r) =

∑τi=1 αik(cϑ(i), r),

where α =(K + ω2I

)−1y and where Kij := k(cϑi, cϑj).

Theorem 1 [8](Global Approximation Theorem) Letmσ and mσ be defined as above and let ‖s‖∞ ≤Mσ . Then

‖m(r)− mσ(r)‖ ≤ 2κ2Mσ√kmax

ω4+κkmaxM

σ

ω2, (12)

where kmax := maxi∥∥ψ(ri)− ψ(cϑ(i))

∥∥H is the greatest

kernel approximation error.

C. Application to GP-Based Subsumption

We can use the global approximation theorem to quantifythe approximation error of the GPs we train on. Specifically,it’s assumed that

enom(t) ∼ GP(m0(r), k0(r, r′)) (13)edsrj (t) ∼ GP(m1(r), k1(r, r′)). (14)

Equations (13) and (14) allow us to bound the differencebetween the mean outputs of the GPs, thus proving an upperbound on the residual error (i.e. edsrj−enom) in mean squareerror. First, we need the following corollary.

Corollary 2 Let enom(t) and edsrj (t) be defined as in(13) and (14), and let m0(t) and m1(t) be the means of theGP models associated to enom(t) and edsrj (t) trained usingthe online GP algorithm. Define

εσm0(r) := m0(r)− mσ0 (r) (15)

εσm1(r) := m1(r)− mσ1 (r). (16)

Then

‖εσm0(r)‖+ ‖εσm1(r)‖ ≤ 2κ2Mσ′√kmax

ω4+κkmaxM

σ′

ω2,

(17)

where Mσ′:= Mσ

0 + Mσ1 , and these maximum values are

associated to the corresponding GPs.

Proof: Apply Theorem 1 to the GP models (13) and(14) separately to prove the statement.This corollary confirms the intuition that a large basis vectorset results in a smaller error for the approximation, andthat the difference in means between We can now prove atheorem bounding the difference in the actual (ideal) meansof the GPs (13) and (14), in terms of the data we have seenso far.

Theorem 3 Let enom(t) and edsrj (t) be defined as in (13)and (14), and let m0(t) and m1(t) be the means of the GPmodels associated to enom(t) and edsrj (t) trained using theonline GP algorithm. Then

‖m0(r)−m1(r)‖ ≤ C ′ + Mσ′κ

ω2(18)

where C ′ is defined as the error (17) in Corollary 2, andMσ′

is also defined as in 2.

Proof: Using the triangle inequality,

‖m0(r)−m1(r)‖ ≤ ‖m0(r)− mσ0 (r)‖+ ‖mσ

0 (r)−m1(r)‖≤ ‖εσm0(r)‖+ ‖εσm1(r)‖

+ ‖mσ0 (r)− mσ

1 (r)‖≤ C ′ + ‖mσ

0 (r)− mσ1 (r)‖.

To bound the last term, note that both approximate meansshare the same kernel matrix K. We have

mσ0 (r) =

∑τi=1 α0ik(cϑ(i), r), m

σ1 (r) =

∑τi=1 α1ik(cϑ(i), r),

where

α0 :=(τ−2K + ω2I

)−1sσ0 , α1 :=

(τ−2K + ω2I

)−1sσ1 .

Using ideas from the proof of Theorem 1, we have

‖α0 − α1‖ ≤‖sσ0 − sσ1‖

ω2.

Then

|m(r)− mσ(r)| =∥∥∥(αT0 − αT1 )kr

∥∥∥≤∥∥αT0 − αT1 ∥∥∥∥∥kr∥∥∥

≤ ‖sσ0 − sσ1‖κω2√τ

≤ Mσ′κ

ω2.

The above theorem allows us to compute a bound betweenthe means of the GPs modeling the nominal and disruptionsignals, as a function of the observations so far. Thus, wecan predict not only the expected error for the next learningiteration, but we also can predict the expected residual errorin the next learning iteration as well.

V. RESULTS

A. Description of the Experiment

The Parrot AR Drone 2.0 quadrotor [3] is designed to beteleoperated and it is compatible with the Robotics OperatingSystem (ROS) [30]. A ROS python (ROSPY) infrastructurewas developed to enable autonomous flight within a motion-capture testbed. The quadrotor uses a downward-facing sonararray to enforce an operational altitude of 1 meter off theground, and the code to measure or adjust the sonar-basedcontrol component is not accessible through ROSPY. Anominal proportional altitude controller was implementedon ROSPY. The quadrotor’s internal control loop howevercompetes with this nominal controller when tracking trajecto-ries below 1 meter altitude. This behavior of the quadrotor’ssonar based altitude control loop therefore can be consideredparasitic when tracking trajectories below 1 meter over theground or solid objects, resulting in unreliable and oscillatoryquadrotor performance.

The presented architecture was verified using waypointflights from waypoint A to B. The single-dimensional A-to-Btrajectory flight tests were selected since a disruption in thez-direction also affects the translational y-directions of thequadrotor due to couple dynamics. Each of the final data setsare representative of 20 consecutive A-to-B trajectory flighttests. For the nominal data set, the quadrotor executed 20flights over flat ground as an online Guassian Process learnedz-position-error based on the y-position and y-position-error.For the disruption data set, the quadrotor executed the 20

(a) Overlay of the quadrotor con-trol modes in the negative y-direction.

(b) Overlay of the quadrotor con-trol modes in the positive y-direction.

Fig. 2: The GP means and variances of the nominal, disrup-tion, and subsumed responses of the quadrotor are shownas it travels in the negative and positive y-direction as shownin figures 2c and 2d, respectively.

A-to-B trajectory flight tests as an online Guassian Processtrained on the y-position and y-position-error to learn thez-position-error. However, a box was placed with one edgeat -18.4 cm and the other edge at 30.6 cm in terms of thetestbed y-coordinate grid. The box dimensions 72cm x 51cmx 21cm where the 72cm dimension was centered about they-axis of the testbed and the 51cm dimension was alignedparallel to the y-axis. So as the quadrotor passed over thebox, the sonar output changed and the quadrotor altitude wasadversely affected. However, the effect was not immediate assome linear data fusion process in the quadrotor mitigatedthe immediate sonar response of flying over the box [3].

Since Csato and Opper’s sparsified online GP regressionwas not yet executable in python, the quadrotor executed 20A-to-B flight tests using (7) for the subsumed data set, wherethe trajectory was generated offline using Csato and Opper’ssparsified GP regression in MATLAB.

B. Discussion of Results

The quadrotor was commanded to follow a path betweenthe y-position -1 meter and the y-position 1 meter. The 3-dimensional Gaussian Process regressions used are shownfrom a 2-dimensional perspective in figures 3a-3f. Thelarge variance in the data was a result of the selectedplatform. There have been numerous attempts to demonstrateautonomous control of the Parrot AR Drone 2.0 quadrotorand performance issues were common [11, 19, 34]. It isworth noting that the difficulty involved in developing aquality ROS or ROSPY controller for the Parrot AR Drone2.0 motivated the initial platform selection. Thus, a GP wastrained on the batch of 20 nominal runs and another GPwas trained on the batch of 20 disrupted runs. The resultingregressions were used to demonstrate the impact of a singlelearning iteration of the subsuming feed-forward term. Thesubsumed response was recorded across 20 flights.

The performance of the nominal, disrupted and subsumedresponses are compared in figures 2c and 2d. The subsumedresponse shows that the disruption caused by the sonar hasbeen reduced across the batched ILC. More interestingly,the changes in the nominal response due to the sonar arealso mitigated by the subsumed response; i.e., the subsumedresponse outperforms the nominal and disruption responses

(a) Nominal quadrotor controlmode in the negative y-direction.

(b) Nominal quadrotor controlmode in the positive y-direction.

(c) Disruption quadrotor controlmode in the negative y-direction.

(d) Disruption quadrotor controlmode in the positive y-direction.

(e) Subsumed quadrotor controlmode in the negative y-direction.

(f) Subsumed quadrotor controlmode in the positive y-direction.

Fig. 3: The GP means and variances of the response modesof the quadrotor are shown as it travels in the negative andpositive y-direction as shown in figures 3a to 3f, respectively.

before the major sonar disruption is encountered. Had thesubsumed response been separated into nominal and dis-ruption test regimes, this finding would be more evident.However, the gathered experimental data is sufficient todemonstrate the aforementioned result.

VI. CONCLUSION

A novel control architecture for subsuming parasitic con-trol components using a variant of D-type ILC was presented.The architecture leveraged past error terms and knowledgeof the actionable control component to reduce the impactof the parasitic control component on the overall systemperformance. Sparsified Gaussian Processes (GP) were lever-aged to learn a predictive model based on the past errorterms to subsume the parasitic control component for onlineimplementation. Rigorous quantifiable bounds related thesparisification of the GP to the accuracy in estimating andsubsuming the parasitic subsytem.

REFERENCES

[1] R. Allamaraju, B. Reish, and A. Axelrod. Sensor NoiseRejection. ArXiv e-prints, Aug. 2014.

[2] T. Alpcan. Dual control with active learning using gaussianprocess regression. arXiv preprint arXiv:1105.2211, 2011.

[3] P.-J. Bristeau, F. Callou, D. Vissiere, N. Petit, et al. Thenavigation and control technology inside the ar. drone microuav. In 18th IFAC world congress, volume 18, pages 1477–1484, 2011.

[4] M. Butcher, A. Karimi, and R. Longchamp. A statisticalanalysis of certain iterative learning control algorithms. In-ternational Journal of Control, 81(1):156–166, 2008.

[5] A. Chaillet and A. Lorıa. Uniform semiglobal practicalasymptotic stability for non-autonomous cascaded systemsand applications. Automatica, 44(2):337–347, 2008.

[6] C.-S. Chen and L.-Y. Chen. Cross-coupling position commandshaping control in a multi-axis motion system. Mechatronics,21(3):625–632, 2011.

[7] G. Chowdhary, E. N. Johnson, R. Chandramohan, M. S.Kimbrell, and A. Calise. Autonomous guidance and control ofairplanes under actuator failures and severe structural damage.Journal of Guidance Control and Dynamics, 2012.

[8] G. Chowdhary, H. Kingravi, J. How, and P. Vela. Bayesiannonparametric adaptive control using gaussian processes.IEEE Trans. on Neural Networks and Learning Systems, 2014.

[9] G. Chowdhary, H. A. Kingravi, J. P. How, and P. A. Vela.Bayesian nonparametric adaptive control using gaussian pro-cesses. 2013.

[10] L. Csato and M. Opper. Sparse on-line gaussian processes.Neural computation, 14(3):641–668, 2002.

[11] N. Dijkshoorn. Simultaneous localization and mapping withthe ar. drone. PhD diss., Masters thesis, Universiteit vanAmsterdam, 2012.

[12] A. Faust, I. Palunko, P. Cruz, R. Fierro, and L. Tapia. Learningswing-free trajectories for uavs with a suspended load. InRobotics and Automation (ICRA), 2013 IEEE InternationalConference on, pages 4902–4909. IEEE, 2013.

[13] R. C. Grande, G. Chowdhary, and J. P. How. Nonparametricadaptive control using gaussian processes with online hyper-parameter estimation. In Decision and Control (CDC), 2013IEEE 52nd Annual Conference on, pages 861–867. IEEE,2013.

[14] A. Hildebrandt, O. Sawodny, R. Neumann, and A. Hartmann.Cascaded control concept of a robot with two degrees offreedom driven by four artificial pneumatic muscle actuators.In American Control Conference, 2005. Proceedings of the2005, pages 680–685. IEEE, 2005.

[15] Y. Hosseinkhani. Control Methods for Improving TrackingAccuracy and Disturbance Rejection in Ball Screw FeedDrives. PhD thesis, University of Waterloo, 2013.

[16] Z.-S. Hou and Z. Wang. From model-based control to data-driven control: Survey, classification and perspective. Infor-mation Sciences, 235:3–35, 2013.

[17] I. Hwang, S. Kim, Y. Kim, and C. E. Seah. A survey of faultdetection, isolation, and reconfiguration methods. ControlSystems Technology, IEEE Transactions on, 18(3):636–653,2010.

[18] A. Jain, B. Wojcik, T. Joachims, and A. Saxena. Learningtrajectory preferences for manipulators via iterative improve-ment. In Advances in Neural Information Processing Systems,pages 575–583, 2013.

[19] A. G. Jamkhandi, S. Tulpule, A. Chaturvedi, and J.-N.Charvet. Controlling the position and velocity in space of thequad-rotor uav ar. drone using predictive functional controland image processing in open cv. International Proceedingsof Computer Science & Information Technology, 58, 2012.

[20] A. M. Johnson, G. C. Haynes, and D. E. Koditschek. Distur-bance detection, identification, and recovery by gait transitionin legged robots. In Intelligent Robots and Systems (IROS),2010 IEEE/RSJ International Conference on, pages 5347–5353. IEEE, 2010.

[21] E. N. Johnson and S. K. Kannan. Adaptive trajectory controlfor autonomous helicopters. Journal of Guidance, Control,and Dynamics, 28(3):524–538, 2005.

[22] H. Kawano and T. Ura. Dynamics control algorithm ofautonomous underwater vehicle by reinforcement learningand teaching method considering thruster failure under severe

disturbance. In Intelligent Robots and Systems, 2001. Proceed-ings. 2001 IEEE/RSJ International Conference on, volume 2,pages 974–979. IEEE, 2001.

[23] H. K. Khalil and J. Grizzle. Nonlinear systems, volume 3.Prentice hall Upper Saddle River, 2002.

[24] R. Langner. Stuxnet: Dissecting a cyberwarfare weapon.Security & Privacy, IEEE, 9(3):49–51, 2011.

[25] F. L. Mueller, A. P. Schoellig, and R. D’Andrea. Iterativelearning of feed-forward corrections for high-performancetracking. In Intelligent Robots and Systems (IROS), 2012IEEE/RSJ International Conference on, pages 3276–3281.IEEE, 2012.

[26] M. Muhlegg, G. Chowdhary, and F. Holzapfel. Optimizingreference commands for concurrent learning adaptive-optimalcontrol of uncertain dynamical systems.

[27] M. Muhlegg, G. Chowdhary, J. P. How, and F. Holzapfel.Adaptive-optimal control of constrained nonlinear uncertaindynamical systems using concurrent learning model predictivecontrol. 2013.

[28] R. Murray-Smith and D. Sbarbaro. Nonlinear adaptive controlusing non-parametric gaussian process prior models. 2002.

[29] D. Nguyen-Tuong, B. Scholkopf, and J. Peters. Sparseonline model learning for robot control with support vectorregression. In Intelligent Robots and Systems, 2009. IROS2009. IEEE/RSJ International Conference on, pages 3121–3126. IEEE, 2009.

[30] M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs,R. Wheeler, and A. Y. Ng. Ros: an open-source robotoperating system. In ICRA workshop on open source software,volume 3, page 5, 2009.

[31] J. Quinonero-Candela and C. E. Rasmussen. A unifyingview of sparse approximate gaussian process regression. TheJournal of Machine Learning Research, 6:1939–1959, 2005.

[32] M.-B. Radac, R.-E. Precup, E. M. Petriu, S. Preitl, and C.-A.Dragos. Experiment-based approach to reference trajectorytracking. In Control Applications (CCA), 2012 IEEE Interna-tional Conference on, pages 470–475. IEEE, 2012.

[33] J. K. Rosenblatt and D. Payton. A fine-grained alternative tothe subsumption architecture for mobile robot control. In Neu-ral Networks, 1989. IJCNN., International Joint Conferenceon, pages 317–323. IEEE, 1989.

[34] L. K. Scott. Autonomous hover through vision based flightcontrol using the parrot ar drone quadrotor. University of NewSouth Wales at Australian Defence Force Academy, ElectricalEngineering Undergraduate Thesis, 2012.

[35] D. Shen and Y. Wang. Survey on stochastic iterative learningcontrol. Journal of Process Control, 24(12):64–77, 2014.

[36] L. Tang and R. G. Landers. Multiaxis contour control-the stateof the art. IEEE TRANSACTIONS ON CONTROL SYSTEMSTECHNOLOGY, 21(6):1997–2010, 2013.

[37] G. Tao. Adaptive control design and analysis, volume 37.John Wiley & Sons, 2003.

[38] D. Toal, C. Flanagan, and C. Jones. Subsumption architecturefor the control of robots.

[39] C. K. Williams and C. E. Rasmussen. Gaussian processes formachine learning. the MIT Press, 2(3):4, 2006.

[40] B. Xuhui, H. Zhongsheng, and J. Shangtai. A statisticalanalysis of model free adaptive control with measurement dis-turbance. In Control Conference (CCC), 2010 29th Chinese,pages 2175–2181. IEEE, 2010.

[41] Y. Zhang and J. Jiang. Bibliographical review on recon-figurable fault-tolerant control systems. Annual reviews incontrol, 32(2):229–252, 2008.