Learning Generalizable Robot Skills from Demonstrationsin Cluttered Environments

M. Asif Rana, Mustafa Mukadam, S. Reza Ahmadzadeh, Sonia Chernova, and Byron Boots

Abstract— Learning from Demonstration (LfD) is a popu-lar approach to endowing robots with skills without havingto program them by hand. Typically, LfD relies on humandemonstrations in clutter-free environments. This prevents thedemonstrations from being affected by irrelevant objects, whoseinfluence can obfuscate the true intention of the human or theconstraints of the desired skill. However, it is unrealistic toassume that the robot’s environment can always be restructuredto remove clutter when capturing human demonstrations. Tocontend with this problem, we develop an importance weightedbatch and incremental skill learning approach, building on arecent inference-based technique for skill representation andreproduction. Our approach reduces unwanted environmentalinfluences on the learned skill, while still capturing the salienthuman behavior. We provide both batch and incrementalversions of our approach and validate our algorithms on a7-DOF JACO2 manipulator with reaching and placing skills.

I. INTRODUCTION

Intelligent and cooperative robots must be capable ofadapting to novel tasks in dynamic, unstructured environ-ments. This is a challenging problem to address; it requires arobot to possess a diverse set of skills that may be difficult tohand-specify or pre-program. Learning from demonstration(LfD) has proven an effective tool in approaching such prob-lems [1]. To acquire a desired skill, LfD approaches generallyinvolve learning a skill model from a set of demonstrationsprovided by a human. The model can then be queried toreproduce the skill in novel reproduction environments withadditional skill constraints. Common examples of constraintsinclude new start/goal states, or new obstacle configurationsthat constrain the set of possible trajectories. LfD techniquesgenerally differ in the manner in which the skill is repre-sented, learned, and reproduced.

Most prior LfD approaches [2], [3], [4], [5] are based onthe assumption that demonstrations can be performed in un-cluttered, minimally constrained environments. The presenceof clutter in the demonstration environments can introduceadditional constraints on human demonstrations that areunrelated to the target skill or the underlying human intent.If unaccounted for, this can lead to suboptimal skill models.However, restructuring the world to remove clutter is oftenimpractical, which limits the viability of such approaches.

In this work, we tackle the problem of learning skillsfrom a set of demonstrations, which can be partially or fullyinfluenced by the presence of obstacles (see Fig. 1).

All authors are affiliated with the Institute for Robotics andIntelligent Machines (IRIM), Georgia Institute of Technology, Atlanta, GA.Email: {asif.rana, mmukadam3, reza.ahmadzadeh,chernova, bboots}@gatech.edu.

Fig. 1: A human is demonstrating a placing skill, which involvesplacing the (red) cube from the (blue) bowl on the right in to oneof the three bowls on the left. The figure contrasts the demonstratedtrajectory (light blue), which is influenced by an obstacle (drawer)in the environment, with the intended straight-line trajectory (darkblue) in the absence of the obstacle.

To contend with obstacles during training, we presentimportance weighted skill learning. Specifically, we adoptand extend the inference-based view of skill reproduction asproposed by Rana et al. [6] with Combined Learning fromDemonstration And Motion Planning (CLAMP). CLAMPprovides a principled approach for generalizing robot skillsto novel situations, including avoiding unknown obstaclespresent in the reproduction environment. When reproducinga desired skill, trajectories are generated to be optimal withrespect to the demonstrations while remaining feasible in thegiven reproduction environment.

We extend CLAMP to utilize demonstrations from clut-tered environments through importance weighted skill learn-ing (see Fig. 2), which rates the importance of demonstrationtrajectories while learning the skill model. We propose an im-portance weighting function that assigns lower importance toparts of demonstrations that are more likely to be influencedby obstacles. We present batch and incremental versions ofour algorithm: batch learning is useful when the set of initialdemonstrations are sufficient for learning a reasonable skillmodel, while incremental learning is useful in scenarios thatrequire refinement of the skill model as new demonstrationsin new environments become available.

We validate our approach on a 7-DOF JACO2 manipulatorwith reaching and placing skills. In all the experiments,we evaluate the approach by providing demonstrations incluttered environments and then changing the environmentsfor reproduction.

II. RELATED WORK

Many existing approaches to trajectory-based LfD addressthe problem of avoiding obstacles in the reproduction sce-

arX

iv:1

808.

0034

9v2

[cs

.RO

] 4

Aug

201

8

Fig. 2: An overview of our approach. In the demonstration envi-ronment, the human demonstrations and the associated importanceweights are collected. The trajectory prior acts as the skill model.Conditioning this prior on the likelihood of events specified by thereproduction scenario gives the posterior.

nario. Some approaches add obstacle avoidance in the skillreproduction phase as a reactive strategy [7], [8], [9], whileothers carry out motion planning or trajectory optimiza-tion [10], [11], [12], [6]. In all these approaches, the skillmodel is learned from demonstrations that are not affected byobstacles. Any constraints or costs associated with obstaclesare typically present during reproduction only. However, inan obstacle-rich environment, the demonstrations themselvesare likely to be influenced by the presence of obstacles,which could have repercussions during skill reproduction.

There have been a few attempts to address the problemof learning skills from demonstrations in cluttered environ-ments. For example, [13], [14] learn a dynamic movementprimitive (DMP) as well as a coupling term for obstacleavoidance from demonstrations. These approaches sufferfrom two major problems. First, since DMPs follow a singledemonstration, they fail to learn potentially different waysof executing the skill, thereby limiting its robustness in newscenarios. Second, due to the reactive nature of the obstacleavoidance strategy, the reproduced trajectory does not nec-essarily preserve the shape of the motion in the presence ofobstacles. Ghalamzan et al. [15], proposed an approach basedon learning a cost functional from human demonstrations.This cost functional is dependent on two components: thedeviation from the mean of the demonstrations, and the dis-tance from obstacles in the environment. Parameters of boththese components are estimated from human demonstrations.A major drawback of this approach is the assumption thatthe mean of the demonstrations sufficiently expresses thedemonstrated skill. This assumption however stands invalidfor skills which can be executed in multiple ways and hencerequires a more expressive skill model.

Our proposed method is based on learning an underly-ing stochastic dynamical system from demonstrations. De-pending on the part of the state-space the robot lies in,this dynamical system is able to generate different waysof executing a learned skill. We make use of importanceweighting to discount the effect of obstacles that are presentwhen the demonstrations are provided. Specifically, the partsof demonstrations in the vicinity of obstacles are penalizedto account for their deviation from the desired skill or thehuman intention.

III. COMBINED LEARNING FROM DEMONSTRATION ANDMOTION PLANNING

We adopt the probabilistic inference view on learningfrom demonstration which has been previously employed inCLAMP [6].

A. Skill Reproduction as Probabilistic Inference

Skill reproduction using CLAMP is performed by maxi-mum a posteriori (MAP) inference given a trajectory priorand event likelihoods in the reproduction environment.

Trajectory Prior: The trajectory prior or the skill modelrepresents a distribution over robot trajectories. A trajectoryis defined as a finite collection of D-dimensional robot statesxi ∈ RD at time ti, 0 ≤ i ≤ N . The prior is given by ajoint Gaussian distribution over the robot states,

p(x) ∝ exp{−1

2‖x− µ‖2K}, (1)

where,x.= [x0,x1, . . . ,xN ]T

µ.= [µ(t0),µ(t1), . . . ,µ(tN )]T , K .

= [K(ti, tj)]∣∣ij,0≤i,j≤N .

The prior enforces optimality by penalizing the optimaltrajectory on deviating from the mean of the prior duringinference. The trajectory prior is learned from demonstra-tions.

Event Likelihood: The likelihood function encodes theconstraints in the skill reproduction scenario. The constraintsare represented as random events e that the optimal trajectoryshould satisfy thus enforcing feasibility during inferencei.e. reproduction. These events, for example, may includeobstacle avoidance, or a new start/goal state or via-point.The likelihood function is defined as a distribution in theexponential family,

p(e|x) ∝ exp{−1

2‖h(x; e)‖2Σ}, (2)

where h(x; e) is a vector-valued cost function with covari-ance matrix Σ. The reader is referred to [16], [6] for moredetails on these likelihood functions.

MAP Inference: The desired optimal and feasible trajec-tory that reproduces the skill is then given by,

x∗ = argmaxx

{p(x|e)

}= argmax

x

{p(x)p(e|x)

}. (3)

B. Trajectory Prior Formulation

It is assumed that in CLAMP that robot trajectories for adesired skill are governed by an underlying linear stochasticskill dynamics,

xi+1 = Φi+1xi + ui+1 + wi+1, wi+1 ∼ N (0,Qi+1),(4)

where Φi and ui are a time-varying transition matrix anda bias term, respectively, and wi is additive white noisewith time-varying covariance Qi. The trajectory prior can begenerated by taking the first and second order moments of thesolution to this dynamics. This Markovian dynamics yieldsan exactly sparse precision matrix (inverse covariance) [17],[18] inducing structure in the trajectory prior in (1), which

enables efficient learning and inference. The problem oflearning the trajectory prior is equivalent to estimating theunderlying stochastic dynamics.

While learning the trajectory prior, CLAMP assumes allavailable demonstrations are free from external influences,and therefore captures the true human intent or skill con-straints. However, in the presence of such influences, thisassumption no longer holds and the learned prior is subop-timal.

IV. IMPORTANCE WEIGHTED SKILL LEARNING

In this section, we introduce importance weighting whenlearning the prior to exclude the effects of unwanted influ-ences during demonstrations. We seek to estimate the param-eters of the skill dynamics model in (4) from demonstrations.As a preliminary step, lets re-write (4) as follows,

xi+1 = Φ̃i+1x̃i + wi+1, wi+1 ∼ N (0,Qi+1) (5)

where,

x̃i =

[1xi

], Φ̃i+1 =

[ui+1 Φi+1

]We additionally define an importance weighting function asw : Rd 7→ R. The importance weighting function shouldgive higher weights to robot states that are less likely todeviate from the skill constraints or the true human intent.While this importance weighting formulation can be usedin other contexts too, in this paper we define a specificform of importance weighting to account for the influence ofunwanted obstacles in the demonstration environment. Theexact form of this environment-dependent obstacle weightingfunction is presented in Section V.

A. Batch Skill Learning

Let’s assume the availability of K trajectory demon-strations, with the kth demonstration defined as xk =[xk0 ,x

k1 , . . . ,x

kN ]T . For each discrete time interval (ti, ti+1],

the inputs are collected into a matrix X̃i = [x̃1i , x̃

2i , . . . , x̃

Ki ]

while the corresponding targets into a matrix Xi+1 =[x1i+1,x

2i+1, . . . ,x

Ki+1]. Furthermore, the matrix Wi =

diag(w(x1

i ), w(x2i ), . . . , w(x

Ki ))

defines a state-dependentimportance weight matrix.

The batch skill learning formulation seeks to find Φ̃i+1

and Qi+1, which minimize a regularized squared norm overthe provided demonstrations.

Φ̃∗i+1,Q

∗i+1 (6)

= argminΦ̃i+1,Qi+1

{L(Φ̃i+1,Qi+1)

}= argmin

Φ̃i+1,Qi+1

{tr(Q−1i+1Ei+1WiE

Ti+1

)+ λ‖Φ̃i+1‖2F

}where Ei+1 =Xi+1−Φ̃i+1X̃i defines the error matrix, andλ is a regularization coefficient.

The solution to the batch skill learning problem in (6) isgiven by the weighted ridge regression estimate,

Φ̃∗i+1 =XT

i+1WiX̃i

(X̃iWiX̃

Ti + λI

)−1, (7)

Q∗i+1 =

1

zE∗i+1WiE

∗i+1

T , (8)

z =tr(Wi)

2 − tr(WTi Wi)

tr(Wi).

B. Incremental Skill Learning

The batch skill learning procedure assumes that there areenough demonstrations available to learn an optimal skillmodel. However, as more demonstrations are aggregated overtime, possibly in different environments, it is desirable torefine the model since more data provides a better estimateof the skill. To achieve this, we propose incremental weightedskill learning.

Our incremental skill learning procedure is based onBayesian inference. In this formulation, we maintain a jointprobability distribution over the unknown skill dynamicsparameters. Every time a new demonstration is collected,a posterior over the skill dynamics parameters is calculated

p(Φ̃i+1,Qi+1|D1:k)

= p(Dk|Φ̃i+1,Qi+1)p(Φ̃i+1,Qi+1|D1:k−1),(9)

where D1:k = {{x̃1i ,x

1i+1}, {x̃2

i ,x2i+1}, . . . , {x̃ki ,xki+1}}.

At any stage, the mode of the posterior distribution providesan estimate of the unknown parameters.

Skill Dynamics Distribution: The joint probability dis-tribution over the unknown parameters Φ̃i+1 and Qi+1 isgiven by

p(Φ̃i+1,Qi+1) = p(Φ̃i+1|Qi+1)p(Qi+1), (10)

where,

p(Φ̃i+1|Qi+1) =MN (Mi+1,Qi+1,Ri+1), (11)

p(Qi+1) =W−1(Vi+1, νi+1), (12)

MN refers to a matrix-normal distribution with matrix-valued mean Mi+1 and covariances Qi+1 and Ri+1 for therows and columns respectively. W−1 refers to an inverse-Wishart distribution with positive definite scale matrix Vi+1

and νi+1 degrees of freedom. Note that matrix-normaland inverse-Wishart distributions are generalizations of thenormal and inverse-gamma distributions respectively to themultivariate case.

Demonstration Likelihood: The likelihood of observingthe input-target pair from the kth demonstration under thestochastic dynamics (5) is given by

p(Dk|Φ̃i+1,Qi+1).= p(xki+1|x̃ki , Φ̃i+1,Qi+1) (13)

∝ exp

{− 1

2(wki Q

−1i+1e

ki+1e

ki+1

T)

}.

where eki+1 = xki+1 − Φ̃i+1x̃ki and wki = w(xki ). Note that

the likelihood is scaled by the weight in order to incorporatethe importance weighting.

Skill Dynamics Inference: The skill dynamics parametersafter assimilation of k demonstrations is given by the modeof the joint posterior distribution (maximum a posteriori),

Φ̃ki+1,Q

ki+1 = argmax

Φ̃i+1,Qi+1

{p(Φ̃i+1,Qi+1|D1:k)

}. (14)

Due to the properties of matrix-normal and inverse Wishartdistributions, the mode of the joint distribution turns out to beequivalent to the product of the modes of the two conditionaldistributions [19],

Φ̃ki+1 = argmax

Φ̃i+1

{p(Φ̃i+1|Qi+1,D1:k)

}=Mk

i+1 (15)

Qki+1 = argmax

Qi+1

{p(Qi+1|D1:k)} = 1

νki+1 +D + 1V ki+1.

(16)

Furthermore, the parameters of the conditional distributionsare governed by the following update laws,

Rki+1 = wix̃ix̃

Ti +Rk−1

i+1

Mki+1 = (wixi+1x̃

Ti +Mk−1

i+1 Rk−1i+1 )(R

ki+1)

−1

V ki+1 =

V k−1i+1 + wi(xi+1 −Mk

i+1x̃i)(xi+1 −Mki+1x̃i)

T

+ (Mki+1 −Mk−1

i+1 )Rk−1i+1 (M

ki+1 −Mk−1

i+1 )T

νki+1 = 1 + νk−1i+1

The incremental learning procedure is initialized with a priorjoint distribution p(Φ̃i+1,Qi+1|φ). The Gaussian comonentof the joint prior is selected to be the ridge regressionprior, that is, M0

i+1 = 0 and R0i+1 = 1

αI . The inverseWishart component is selected to be an uninformed prior,with V 0

i+1 = 1β I and ν0i+1 = 1

β . Here α and β are positivescalars. In our implementation, we set α = β = 1010. Notethat smaller values of these scalars makes the prior too strict,which restrains the skill model from fitting the data well.

V. ENVIRONMENT-DEPENDENT IMPORTANCE WEIGHTINGFUNCTION

In this section, we define the importance weighting func-tion to enable skill learning from demonstrations, which maybe provided in the presence of obstacles in the environment.The weighting function gives lower importance to the partsof a demonstration which are more likely to be influencedby the presence of an obstacle and therefore deviate fromthe intent of the human.

We hypothesize that the parts of demonstrations closer toobstacles are influenced by the obstacles and therefore failto satisfy the skill constraints. Conversely, partial trajectoriesfarther away from obstacles are more likely to satisfy theskill constraints and should be given more importance. Fora given state xi, we define the importance weight to beequivalent to the likelihood of staying collision-free [16]. Forthis likelihood function, we first define a hinge loss function

c(xi) =

{−d(xi) + ε d(xi) ≤ ε0 d(xi) > ε

,

where d(·) is the signed distance from the closest obstacle inan environment and ε specifies the ‘danger area’ around the

Fig. 3: An illustration of an importance weight function parame-terized by ε = 3 and σobs = 1 (left) and a signed distance field(right). The importance weight levels at 1 outside the danger area,and decays down to zero inside with the slope governed by σobs.

obstacle. With this hinge loss, we assume that an obstacleaffects a state only when it is within the danger area aroundthe obstacle. Outside of this danger area, the obstacle has noinfluence on the state. The importance weight itself is givenby a function in the exponential family,

w(xi) = exp

{− c(xi)

2

2σ2obs

}, (17)

where the parameter σobs dictates the rate of decay of theimportance weight for states within the ’danger area’. Thesmaller the value of σobs, the faster the importance weightwill decay down to zero (see Fig 3).

VI. EXPERIMENTS

We evaluate the performance of our method on twodifferent skills1: 1) the reaching skill, and 2) the placing skill.For both skills, a human provides multiple demonstrationsvia kinesthetic teaching on a 7-DOF JACO2 manipulator. Theend-effector positions are recorded and the correspondinginstantaneous velocities are estimated by fitting a cubicspline to each demonstration and taking its time derivative.Furthermore, the demonstrations are also time-aligned usingdynamic time warping (DTW). To setup the trajectory priorin (1), we define the robot states xi as the vector concate-nation of instantaneous robot positions and velocities.

For the reaching skill, the goal is to reach an object fromdifferent locations. Hence, all the demonstrations share thesame goal state while the initial state varies. In the absence ofany obstacles in the path, a demonstration follows a nearlystraight-line path to the goal. In the presence of obstaclesin the path, the demonstrations deviate from this desiredpath in order to avoid collision with the obstacles. Fig. 4shows the demonstration environment and the correspondingdemonstrations.

In order to learn the trajectory prior for this skill, weuse importance weighted skill learning, as described inSection IV-A. The demonstrations reaching the target fromthe uncluttered part of the environment represent the truehuman intent. Therefore, we expect our trajectory prior tobe biased towards these demonstrations. Fig. 5 shows thetrajectory distributions (i.e. time-evolving state distributions)encoded in the trajectory priors learned with and without im-portance weighting. The trajectory distributions are generated

1Accompanying video: https://youtu.be/03r8Tblhq7k

Fig. 4: Human demonstrations for the reaching skill. All demon-strations reach the bowl from different initial positions in thepresence of three obstacles in the environment. Top: Snapshots of ademonstrations avoiding the obstacles. Bottom: A 3-D plot showingall the demonstrations and the obstacles.

(a) without importance weighting (b) with importance weighting

Fig. 5: Trajectory prior visualization for the reaching skill. Theblue line is the mean of the prior, and the blue shaded region showsone standard deviation around the mean.

by rolling out the stochastic skill dynamics in (5) with aninitial state distribution given by a Gaussian over the initialdemonstration states. The mean of the trajectory distributiongenerated with importance weighting deviates less from theintended straight-line path, exhibiting the true underlyingskill, as compared to the distribution without importanceweighting. To enable this, we empirically selected the pa-rameters of the importance weight function in (17), such thatthe parts of state-space likely to be under obstacle influencecan be successfully downplayed while learning the prior. Avalue of ε = 0.3m and σobs = 0.01m provided sufficientbounding region around the obstacles in most cases.

Fig. 6 shows multiple instances of reproduction for thereaching skill. The skill is reproduced with (3) by con-ditioning the learned trajectory prior on the likelihood ofstarting from a desired initial state and the likelihood ofstaying clear of arbitrarily placed obstacles. We show thetrajectories generated from two different initial states in threedifferent environments. When the obstacles are placed at thesame location as the demonstration phase or displaced, thereproduced trajectories from the prior without importanceweighting take the longer path to the target around theobstacles. This is because the demonstrations on average tooka longer path while avoiding obstacles and the prior shown

Fig. 6: Trajectories generated by conditioning the priors on twoinitial positions in three different environments. Top-left: Environ-ment without obstacles. Top-center: Environment with obstacles atthe same locations as demonstrations. Top-right: Environment withobstacles displaced. Bottom: Trajectory executions on a real robotin the obstacle-free environment.

in Fig. 5(a) forces the reproduced trajectories to exhibit asimilar behavior. For the same reasons, the deviant non-smooth trajectories are also observed when no obstaclesare present in the vicinity of the robot in the reproductionenvironment.

The placing skill involves placing an object at differentlocations on a table. All the demonstrations start from thesame location since the object’s initial location is fixed.The end state of the demonstration varies with the targetplacement location. Initially there is an obstacle present inthe desired path, hence all the demonstrations go above theobstacle causing them to be influenced. Fig. 7 (left) plotsthe human demonstrations provided in this scenario. Sinceonly the influenced demonstrations are available at this stage,the trajectory prior learned from these demonstrations alsoencodes the influence of obstacles which is undesirable.However, as the environment changes and more demonstra-tions are available in a cleaner environment, as shown inFig. 7 (right), the prior is updated using the incrementalweighted learning procedure described in Section IV-B.

Fig. 8 shows the evolution of the prior as demonstrationsare assimilated. The prior initially enforces highly con-strained motion causing the trajectories to avoid the obstacleeven when it is not present. As more demonstrations aremade available in an obstacle-free environment, the highimportance weight relative to the influenced demonstrationsenables adaptation to the desired underlying motion afterjust three updates. On the other hand, when the importanceweighting is not considered in the incremental learningprocedure, the trajectory prior still exhibits the obstacleinfluence even after all the demonstrations are incorporated.This is shown in Fig. 9. The utility of the incrementallearning procedure is high in such scenarios. It is undesirableto keep all the demonstrations and re-learn the prior on arrivalof each new demonstration, since this can be both time-

Fig. 7: Human demonstrations for the placing skill in two differentenvironments. Left: Environment with a large obstacle influencingthe demonstrations. Right: Obstacle-free environment.

Fig. 8: Trajectory priors for the placing skill with importanceweighting. Top-left: Learned from first 3 demonstrations recordedin the presence of obstacle. Top-right: Prior after assimilatingfourth demonstration in clean environment. Bottom-left: Prior afterassimilating fifth demonstration. Bottom-right: Final prior after allthe incremental updates.

Fig. 9: Trajectory priors for the placing skill without importanceweighting. Left: Learned after assimilating first 4 demonstrations.Right: Final prior after all the incremental updates.

consuming as well as memory-intensive.

VII. CONCLUSION

We have presented importance weighted skill learning,which is a novel technique for learning skills from demon-strations in cluttered environments and generalizing them tonew scenarios. Our importance weighting function associateslower weights with parts of demonstrations that are likelyto collide with obstacles. We conjecture that demonstrationswhich are in close proximity to obstacles are more suscepti-ble to not satisfying the constraints of the skill being learned.Hence, those demonstrations should be given lesser impor-tance during the skill learning stage. Our learning approach isalso capable of incrementally updating and refining the skillmodel to incorporate new demonstrations without the need torelearn the model from scratch. Since our learning method isbased on extracting the underlying stochastic skill dynamics,it does not share the same disadvantages as approaches thatassume a mean trajectory to encode the skill. Furthermore,our reproduction method is capable of generalizing the skill

efficiently across various scenarios as demonstrated in theexperiments.

ACKNOWLEDGEMENTS

This research is supported in part by NSF NRI 1637758,NSF CAREER 1750483, NSF IIS 1637562, and ONRN00014-16-1-2844.

REFERENCES

[1] B. D. Argall, S. Chernova, M. Veloso, and B. Browning, “A surveyof robot learning from demonstration,” Robotics and autonomoussystems, vol. 57, no. 5, pp. 469–483, 2009.

[2] A. J. Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor, and S. Schaal,“Dynamical movement primitives: learning attractor models for motorbehaviors,” Neural computation, vol. 25, no. 2, pp. 328–373, 2013.

[3] S. Calinon, F. Guenter, and A. Billard, “On learning, representing,and generalizing a task in a humanoid robot,” IEEE Transactions onSystems, Man, and Cybernetics, vol. 37, no. 2, pp. 286–298, 2007.

[4] S. M. Khansari-Zadeh and A. Billard, “Learning stable nonlineardynamical systems with Gaussian mixture models,” IEEE Transactionson Robotics, vol. 27, no. 5, pp. 943–957, 2011.

[5] A. Paraschos, C. Daniel, J. R. Peters, and G. Neumann, “Probabilisticmovement primitives,” in Advances in neural information processingsystems, 2013, pp. 2616–2624.

[6] M. A. Rana, M. Mukadam, S. R. Ahmadzadeh, S. Chernova, andB. Boots, “Towards robust skill generalization: Unifying learningfrom demonstration and motion planning,” in Conference on RobotLearning, 2017, pp. 109–118.

[7] P. Pastor, H. Hoffmann, T. Asfour, and S. Schaal, “Learning andgeneralization of motor skills by learning from demonstration,” in 2009IEEE International Conference on Robotics and Automation(ICRA).IEEE, 2009, pp. 763–768.

[8] D.-H. Park, H. Hoffmann, P. Pastor, and S. Schaal, “Movement repro-duction and obstacle avoidance with dynamic movement primitivesand potential fields,” in 8th IEEE-RAS International Conference onHumanoid Robots (Humanoids). IEEE, 2008, pp. 91–98.

[9] S. M. Khansari-Zadeh and A. Billard, “A dynamical system approachto realtime obstacle avoidance,” Autonomous Robots, vol. 32, no. 4,pp. 433–454, 2012.

[10] G. Ye and R. Alterovitz, “Demonstration-guided motion planning,” inInternational Symposium on Robotics Research (ISRR), vol. 5, 2011.

[11] T. Osa, A. M. G. Esfahani, R. Stolkin, R. Lioutikov, J. Peters,and G. Neumann, “Guiding trajectory optimization by demonstrateddistributions,” IEEE Robotics and Automation Letters, vol. 2, no. 2,pp. 819–826, 2017.

[12] D. Koert, G. Maeda, R. Lioutikov, G. Neumann, and J. Peters,“Demonstration based trajectory optimization for generalizable robotmotions,” in 2016 IEEE-RAS 16th International Conference on Hu-manoid Robots (Humanoids). IEEE, 2016, pp. 515–522.

[13] A. Rai, F. Meier, A. Ijspeert, and S. Schaal, “Learning coupling termsfor obstacle avoidance,” in Humanoid Robots (Humanoids), 2014 14thIEEE-RAS International Conference on. IEEE, 2014, pp. 512–518.

[14] A. Gams, M. Denisa, and A. Ude, “Learning of parametric couplingterms for robot-environment interaction,” in IEEE-RAS 15th Interna-tional Conference on Humanoid Robots (Humanoids). IEEE, 2015,pp. 304–309.

[15] A. Ghalamzan, C. Paxton, G. D. Hager, and L. Bascetta, “An incre-mental approach to learning generalizable robot tasks from humandemonstration,” in IEEE International Conference on Robotics andAutomation (ICRA). IEEE, 2015, pp. 5616–5621.

[16] M. Mukadam, J. Dong, X. Yan, F. Dellaert, and B. Boots, “Continuous-time Gaussian process motion planning via probabilistic inference,”arXiv preprint arXiv:1707.07383, 2017.

[17] M. Mukadam, X. Yan, and B. Boots, “Gaussian process motionplanning,” in 2016 IEEE International Conference on Robotics andAutomation (ICRA), 2016, pp. 9–15.

[18] T. Barfoot, C. H. Tong, and S. Sarkka, “Batch continuous-timetrajectory estimation as exactly sparse Gaussian process regression,”Proceedings of Robotics: Science and Systems, Berkeley, USA, 2014.

[19] T. Minka, “Bayesian linear regression,” Citeseer, Tech. Rep., 2000.