Toward Real-Time Realistic Humanoid Manipulation Tasks in Changing Environments

by

Xiaoxi Jiang

M.Sc. in Computer Science

National University of Singapore, 2006

B.Eng. in Electrical Engineering

Zhejiang University, 2004

SUBMITTED IN PARTIAL

FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF

MASTER OF SCIENCE

IN ELECTRICAL ENGINEERING AND COMPUTER SCIENCE (EECS)

AT THE

UNIVERSITY OF CALIFORNIA MERCED

2007

Signature of author: ______________________________________________________________________

Accepted by: ___________________________________________________________________________

Assist. Prof. Marcelo Kallmann

EECS, School of Engineering. University of California, Merced

Thesis Advisor

Accepted by: ___________________________________________________________________________

Assist. Prof Miguel Á. Carreira-Perpiñán

EECS, School of Engineering. University of California, Merced

Accepted by: ___________________________________________________________________________

Assist. Prof. Stefano Carpin

EECS, School of Engineering. University of California, Merced

Acknowledgements

I first give my thanks and respects to my thesis advisor, Professor Marcelo Kallmann,

for his invaluable help during my journey in UCMerced. I appreciate the chance

entering the Computer Graphics Lab in UCMerced, being able to further my

knowledge in the field of robotic motion planning, and meanwhile get to know the

exciting computer graphics and animation area. Prof Kallmann did not only spend time

in supporting my research work, but also in teaching me essential ways how to become

a confident and independent researcher. With the guide of him, I gained much

experience and had a lot of fun during my one and a half year study in UCMerced.

I also thank my fiancé Anthony Ip for his patience and support, and all of his commutes

between the bay area and middle California during the year.

Thanks to my thesis committee, Professor Miguel Á. Carreira-Perpiñán and Professor

Stefano Carpin for sharing your time and comments with me.

Toward Real-Time Realistic HumanoidManipulation Tasks in Changing Environments

Xiaoxi Jiang

December 14, 2007

Contents

1 Introduction 21.1 Context Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 Preliminary Background . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.1 Motion Planning for Humanoids . . . . . . . . . . . . . . . . . . 31.2.2 Sampling Based Motion Planners . . . . . . . . . . . . . . . . . 51.2.3 Motion Generation for Virtual Characters . . . . . . . . . . . . . 5

2 Learning Reaching Tasks from Experience 72.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.3 The AGP Planner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.3.1 Problem Formulation and Method Overview . . . . . . . . . . . 102.3.2 Attractor Extraction . . . . . . . . . . . . . . . . . . . . . . . . 112.3.3 Task Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . 122.3.4 Guided Planning . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.4 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3 Learning Manipulation Tasks from Demonstrated Motions 213.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.3 Constraint Detection and Motion Segmentation . . . . . . . . . . . . . . 23

3.3.1 Constraint Type in Object Manipulation . . . . . . . . . . . . . . 233.3.2 Constraint Detection . . . . . . . . . . . . . . . . . . . . . . . . 243.3.3 Parameter Computation . . . . . . . . . . . . . . . . . . . . . . 293.3.4 Motion Segmentation . . . . . . . . . . . . . . . . . . . . . . . . 30

3.4 Attractor-Guided Planning . . . . . . . . . . . . . . . . . . . . . . . . . 313.5 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.5.1 Motion Capture . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.5.2 Constraints Enforced Planning . . . . . . . . . . . . . . . . . . . 32

4 Conclusion and Future Work 35

2

Abstract

A central challenging problem in humanoid robotics is to plan and execute dynamic tasksin changing environments, and at the same time keep the result convincing and realis-tic. Sampling-based online motion planners are particularly powerful for automaticallygenerating collision-free motions in changing environments. However, without learningstrategies, each task still has to be planned from scratch, preventing these algorithms fromgetting closer to realtime performance. Moreover, the nature of the random samplingstrategy employed in these planners also results in extremely non human-like solutions.

This document addresses these two issues by proposing to learn important features frompreviously planned solutions, or from real captured motion to improve both the efficiencyand the solution quality. Our methods work in changing environments, where obstaclescan have different positions in different tasks. However, we assume that obstacles arestatic during the execution of a single task. We first propose the Attractor Guided Planner(AGP), which extends existing motion planners in two simple but important ways. First,it extracts significant attractor points from successful paths as guiding landmarks for newsimilar tasks. Second, it relies on a task comparison metric to decide when previoussolutions should be reused to guide the planning of new tasks. The task comparisonmetric takes into account the task specification and as well environment features whichare relevant to the query.

With combination of motion capture technique, the AGP planner also shows big im-provements towards generating realistic planned motions. We propose a constraint detec-tion method that applies to humanoid manipulation tasks. After recording a performer’sdemonstrated motion, our method will automatically detect important constraints, andthen segment the input motion according to different types of constraints. Attractors areplaced at the connections between each pair of segments and assigned the same con-straints as the previous segment. Then, given a new similar task, the new planning isguided not only toward the locations of the attractors, but also preserving the constraintsof the attractors.

Several experiments are presented with different humanoid reaching examples whereobstacles are differently located for each task. Our results show that the AGP greatlyimproves both the planning time and solution quality, when comparing to traditionalsampling-based motion planners. We also show that with our constraint detection method,the AGP planner can efficiently find a solution that preserves the features of the input mo-tion, making the solution motion coherent with the task being solved and therefore morerealistic. Although our current results are not yet capable of achieving real-time perfor-mance nor overall realistic humanlike motions, we believe that the techniques introducedhere are key for getting closer to these goals.

Chapter 1

Introduction

1.1 Context Overview

During recent years, creating realistic and convincing animations for humanoidrobotics has become a great interest in many fields. For example: indoor servicerobotics, human-computer interactions, virtual humans for training, animation,and computer games.

Motion planning algorithms have been used to automatically compute collision-free motions in obstacle-cluttered environments. Comparing to early motion plan-ners that try to construct explicit representations of the configuration space [1] [2],sampling-based planning algorithms are more suited for changing environments,where obstacles can have different locations in different tasks. However, when ap-plied to realistic scenarios, existing sampling-based motion planning algorithmshave two restrictions. First, due to the complicated humanoid architecture that in-cludes many DoFs, existing planners are not yet capable of computing collision-free motions for humanoids in real time. Second, the nature of employing randomsamplings in the whole configuration space for these planners results in extremelynot human-like solutions. In this document, we will further discuss these twoissues. The organization is as follows:

In the second chapter, we propose to improve the performance of sampling-basedmotion planners by integrating the ability to reuse learned skills on new tasks.Our algorithm, called the Attractor Guided Planner (AGP), extracts significantattractor points from successful paths that were obtained through experience orreal-time demonstration. The attractors are then reused to guide the future plan-ning for similar tasks. Computational efficiency is improved because search is

2

biased towards successful locations in previous paths. In those environments thatare not dramatically changing, our approach is especially beneficial. Experimentsshow that in all the scenarios that we tested, the AGP planner has a significant im-provement over the RRT planner in both computational time and solution quality.

We also propose, in the third chapter, to improve the naturalness of the plannedmotions by enforcing the features from real detected human motions. The use ofmotion capture techniques allows us to obtain natural-looking motion data. How-ever, the real captured motion is restricted to that specific scenario, and needs to beedited when it is applied to a changed environment. We present a method whichis capable of detecting constraints from captured motion data; and then segmentthe motion according to different types of constraints. Attractors are placed atthe connections between each pairs of segments. Future plannings will not onlybe guided by the locations of the attractors, but also by the constraints that havebeen assigned to that attractor. Experiments show that provided certain types oftemplate motion demonstrations, our method is capable of detecting several basictypes of constraints from real captured motions, and partition an input motion intosegments accordingly. We also show that the AGP planner can efficiently find asolution that preserves the features of the input motion.

Then finally, we conclude our work and discuss its limitations. We also presentseveral possible future directions.

1.2 Preliminary Background

1.2.1 Motion Planning for Humanoids

Motion planning is the problem of deciding the set of motions that can take a robotfrom an initial to a final position while avoiding collisions [3]. There have beenmany applications for motion planning algorithms in the robotics field: Surgeryrobotics, museum tour guides, and search and rescue robots just to name a few.

In motion planning problems, robots are represented in the configuration space(C-space). A robot configuration can be seen as a vector that includes all the pa-rameters to specify the physical state of the robot. For example, the configurationfor a 3-dimensional free-flying object is a set of 6 parameters, denoting its posi-tion and orientation. The configuration space is the space of all the robot’s possiblevalid configurations. If the robot has n degrees of freedom, then the configurationspace is a topological space of dimension n.

3

In the field of humanoid robots, solving motion planning problems is especiallychallenging. General motion planning problem has been proven to be very dif-ficult. Comparing to rigid free-flying objects in traditional motion planning puz-zles, a humanoid is a complex structure that typically has 20 or more DOFs todeal with. Due to the high dimensionality of the configuration space, developingpractical motion planning algorithms for humanoids is a daunting task.

Humanoid motion planning problems can be more interesting and challenging.The applications of humanoids in everyday life have placed new requirements onmotion planning algorithms. For example, we expect the humanoids to be able toautomatically generate a sequence of task-level motions, such as "serve a glass ofwater", or "put a book on the shelf". There have been a lot of efforts on solvinga variety of humanoid planning tasks, such as navigations, object grasping andmanipulation, footstep placement, and full-body motions. While these areas areall interesting, we are particularly interested in dealing with object manipulationproblems for humanoids.

Sampling-based motion planning algorithms have been proven flexible enough tosolve humanoid planning problems in realistic scenarios. In general, sampling-based motion planners work by selecting configuration points as samples fromthe configuration space, and then connecting them to construct a graph, whichimplicitly approximate the structure of the configuration space. These algorithms,including roadmap-based planners and tree-based planners, outperform completemotion planning algorithms in both computational time and the ease to implement.We will introduce two famous planners, the PRM planner [4] and the RRT planner[5] in the next section. However, the limit of real computational systems stillleaves a big room for existing motion planners to be used in real-time, especiallyin real-life scenarios, where the dimension of state space can be quite high.

There are many possible directions to improve the performance of sampling-basedplanners: one of them is to improve the sampling strategy. The basis of sampling-based planners is uniform sampling; it implicitly assumes that the configurationspace is uniformly complex. In this document, we propose a planning methodthat employs an experience-guided sampling method. This method is motivatedby observations from human research showing that human arm movements maybasically be described as discrete segments between locations in order to avoidcollision. Therefore, we propose to extract meaningful landmarks from previouslystored solution paths, and use them to guide future planning for similar tasks.

4

1.2.2 Sampling Based Motion Planners

In this section we briefly introduce two important sampling-based planners: Prob-abilistic Roadmaps (PRMs) [4] and Rapidly-exploring Random Trees (RRTs) [5].

A PRM planner starts with randomly generating samples in the configurationspace, tests if they are free and retains the collision-free samples as milestones.It then connects pairs of milestones which are close to each other by simple pathsand keep the collision-free ones as local paths. This step is repeated until a certaincriteria (such as the number of milestones) is reached. The probabilistic roadmapis composed of the milestones and local paths. If the pair of start and goal config-urations of the planning problem can be connected to the same connected compo-nent of the roadmap, then the problem is solved. Figure 1.1(a) shows a constructedroadmap.

The construction of the roadmap can not be done in real-time. For this reason,the PRM planner works in two phases: a pre-processing phase that constructs andrefines the roadmap for the given environment, and a query phase that uses theroadmap to solve planning problems. It is time-consuming to build the roadmap.The advantage is that, once the roadmap is built, it can be used to rapidly solvemultiple planning problems in a static environment, where the obstacles are notmoving.

On the other hand, if only one single planning problem needs to be solved, or ifthe environment is changing, then the efforts to build a static roadmap are wasted.The RRT planner can be employed to address this issue. Beginning with the initialconfiguration (or both initial and goal configurations) of the task, the RRT plannergrows a tree structure in the freespace. Figure 1.1(b) shows one step of expansionof the tree. At each step, a configuration point (qnew) is randomly selected in theconfiguration space, and connected to its nearest point (qnear) in the tree. A pathwith a fixed length (ε) is then created from qnear towards the sample point qnew. Ifno collision is encountered, then the expanded branch ε will be added to the tree.The expansion step repeated until it connects the initial and goal configurationpair, or a stop criteria (such as a time limit) is reached. The RRT planner is veryefficient in solving single-query planning problems, but the result is also difficultto be reused for future plannings.

1.2.3 Motion Generation for Virtual Characters

Humanoids have been widely used in the computer graphics and animation do-mains. They are usually the main characters in video games, cartoons, training

5

Figure 1.1: The PRM planner(a) and the RRT planner(b)

softwares, and so on. However, the development of animation techniques is stillfar behind comparing to the large amount of requirements. In order to get convinc-ing and realistic humanoid movements, animators are still doing most of the workby hand, frame-by-frame. Traditional animation techniques such as keyframing[6] are still widely used in the animation industry. These manual design methodsallow the animators to have most controls in generating character movements, butalso have many limitations: First, it requires a lot of skills and effort for an ani-mator to create expressive and convincing movements. Second, it needs a greatamount of work to change each articulation parameter manually.

The technique of motion capture dramatically alleviates the burdens on animators.Instead of designing a motion from scratch, we can easily reproduce real motionsfrom a performer wearing motion caputre equipments. Motion editing methodsare necessary in order to apply motion capture techniques. First, real-detecteddata contains noise that needs to be filtered. Second, the captured motion is re-stricted to the same scenario under which it was recorded. Without editing, thesame motion can not be reproduced for dynamic tasks in dynamic environments.Existing motion editing methods include Signal Processing [7] [8], Physics-Basedmethods [9] [10], Spacetime Constraints [11] and many more.

We are interested in employing ideas from motion editing techniques to planningnew motions In this thesis, we develop a method which is capable of detectingconstraints from captured manipulation motions, and enforce these constraintsin future planning to get new motions while still retaining realistic features ofthe captured motion. We define three basic types of constraints to identify thecaptured motion as stationary, not rotating or not translating. We also define twoadditional constraints to tell if a translation follows a straight line, or a rotationaround a specific axis. Then, we segment the input motion according to differenttypes of constraints. The segmented motion is then used to guide future planningsof similar manipulation tasks.

6

Chapter 2

Learning Reaching Tasks fromExperience

Figure 2.1: A sequence of attractors (grey spheres) derived from a solution path(red line) generated by a bidirectional RRT planner (green and blue branches).Note that the attractors specifically represent postures that are trying to avoid col-lisions, as shown in the third and fourth images. In addition, our Attractor GuidedPlanning results in less tree expansions and a smoother solution path (last image).

2.1 Introduction

In this chapter, we provide a method designed to learn from several tasks to beplanned in a realistic dynamic environment. Our method learns and reuses attrac-tor points extracted from previously planned reaching paths. The choice of usingattractor points is based on evidence gathered from studies with human subjects

7

showing that human arm movements may basically be described as discrete seg-ments between start point, intermediate points, and end point. In addition, thesesegments appear to be piecewise planar in three-dimensional movements [12]. In-spired by this fact, and based on the observation that sampling-based planners alsoperform straight connections between landmarks, we propose to select meaning-ful landmarks to be reused as attractor points (Figure 2.1), which are then usedto guide subsequent paths in new similar tasks. Note that in most realistic en-vironments, planned solutions for similar tasks often share common structuralproperties. The proposed method is able to learn such structures by learning andreusing the attractor points.

We apply our Attractor Guided Planner (AGP) to enhance a bidirectional Rapidly-Exploring Random Tree (RRT) planner. It proceeds as follows. Every successfulexecuted reaching task is represented by a group of configurations including theinitial point, goal point, attractors, and as well the position of relevant obstacles inthe environment. This information is then stored in a task database. When a newreaching task is being queried, AGP searches in the database for a similar taskpreviously planned. Whenever a similar task in the database is found, the randomsampling strategy of the original RRT is replaced by a biased sampling process,where samples are selected according to dynamic Gaussian distributions aroundthe group of attractors being reused. The process significantly biases the search tothe successful locations in the previous path, preserving similar structures of thesolutions. If a partial set of the attractors is no more valid due to a change in theenvironment, the method gradually changes back to a non-biased random search.

We present experiments simulating several humanoid reaching scenarios. In allthe scenarios, the AGP planner shows a significant improvement over the RRTin both computational time and the solution quality. Results also show that evenwhen a good task comparison method is not available, the AGP can still outper-form traditional planners. Although all the experiments are performed on hu-manoids, we believe that the AGP planner will also be beneficial in several othergeneric motion planning problems.

2.2 Related Work

Sampling-based motion planning algorithms have proven to be useful for ma-nipulators with several degrees of freedom, such as a human-like structures [13][14] [15]. These algorithms are mostly based on Probabilistic Roadmaps (PRMs)[4] and/or Rapidly-exploring Random Trees (RRTs) [5], depending on the re-quirements of the problem. For instance, PRM-like methods are mostly used for

8

multi-query problems but are limited to static environments due to the high costof building and maintaining roadmaps. In contrast, RRT-like methods are usuallyefficient single-query planners, which construct search trees specifically for eachtask and environment. Therefore the trees expanded by the RRT are difficult tobe reused in a different environment. Due to these limitations, current sampling-based planners scale very poorly to task planning in dynamic environments. Wereview in the next paragraphs the main approaches for motion planning in non-static environments.

Some works have proposed to combine the advantages of both the multiple-queryand single-query techniques in order to obtain efficient solutions for multiplequeries in dynamic environments. For instance, dynamic roadmaps have beenproposed for online motion planning in changing environments [16] [17], how-ever with limited performance improvement for humanoid reaching tasks.

Other works have explored improvements to single query planners for makingthem solve multiple queries more efficiently. For instance, the work of [18] usesthe RRT as the local planner of PRM, i.e. RRTs are grown between each pairof milestones in the PRM. The RRF planner introduced in [19], builds a randomforest by merging and pruning multiple RRTs for future use. A roadmap is thenbuilt and maintained for keeping the final set of RRTs concise.

Some other works focus on exploring sampling methods that are more adaptive tothe environment [20] [21] [22]. A model-based motion planning algorithm is alsoproposed in [23]. This method incrementally constructs an approximate model ofthe entire configuration space using a machine learning technique, and then con-centrates the sampling toward difficult regions. While these methods can learnfrom previous successful experiences to improve future plans, they involve build-ing a model for the entire environment, and thus are not well suited to dynamicenvironments.

Our method, in contrast, focuses on learning relevant features from successfullyplanned motions. The learned information is therefore mainly related to the struc-ture of successful paths, capturing not only the structure of the environment butalso specific properties related to the tasks being solved. It is our own experiencethat the maintenance of complex structures (such as roadmaps) [17] may not scalewell to humanoid reaching tasks and that lighter structures can lead to better re-sults [24]. The work proposed in this paper builds on these previous conclusions.

9

2.3 The AGP Planner

2.3.1 Problem Formulation and Method Overview

We represent the humanoid character as an articulated figure composed of hier-archical joints. Let the full configuration of the character be c = (p,q1, ...,qn),where p is the position of the root joint, and qi, i ∈ {1, ...,n}, is the rotationalvalue of the ith joint, in quaternion representation. Assume the character is givena list of random reaching tasks in an environment with a set of moving obstaclesO = (o j

1, ...,ojm). That is to say, for each task, the obstacles can be in different

positions. The variable o ji ∈O here describes the position of the ith obstacle when

the jth task is being executed.

A single query motion planner, such as RRT, solves a reaching task T as follows.Let cinit be the initial character configuration of the reaching task and let cgoal bethe target configuration. Two search trees Trinit and Trgoal are initialized havingcinit and cgoal respectively as root nodes, and are expanded iteratively by addingrandomly generated landmarks that are free of collision. When a valid connectionbetween the two trees can be concluded, a successful motion is found. We denotethis valid path as a sequence of landmark configurations p = (c1, ...,cn).

Different from RRT, our method guides the expansion of the search trees by fol-lowing a set of attractors derived from a similar previous path. The AGP plannerconsists with three main parts:

1. Attractor Extraction. Everytime a successful path p is obtained, we extract asequence of attractors A = (a1, ...,ak) from it, where ai, i ∈ {1, ...,k}, is a configu-ration point on p. These attractors represent important structural features of p andwill be reused for similar tasks.

2. Task comparison. Solved tasks are stored in a task database. Each task Tis represented by its initial configuration, goal configuration, the attractors, andpositions of relevant obstacles, i.e., T = (cinit ,cgoal,A,O). Note that here O is asubset of all the obstacles. Whenever a new task is requested, we find a previouslysolved task from the database that has the largest similarity to serve as an example.If the value of similarity is lower than a threshold, AGP will decide to operate asa non-biased RRT planner.

3. Guided planning. If an example task has been found from the database, itsattractor set A is used to guide the planning for the new task. In order to increasethe probability that regions around previously successful landmarks are sampled,we form a dynamic Gaussian distribution centered at each attractor point a ∈ A.

10

Samples are then generated according to the Gaussian distribution.

Algorithm 1 summarizes how these three parts take place; their details will beexplained in the next sub-sections.

Algorithm 1 AGP Planner(cinit , cgoal , obstacles)1: attractors←DBPICKTASK(cinit ,cgoal,obstacles)2: solution_path←GUIDEDPLANNING(cinit ,cgoal,attractors)3: current_task←ATTRACTOREXTRACTION(solution_path)4: DBINSERTTASK(current_task)5: return solution_path

Line 1 requires selection of an example task from the database that has the largestsimilarity to the current task. The similarity value is computed by our task com-parison function (subsection 2.3.3). By defining a similarity threshold, we ensurethat both the motion query and the environment of the selected example sharesome common features with the new task. Otherwise, the attractor set A is set tonull and the new task will be executed by a non-biased RRT planner.

Line 2 calls the guided planning which returns a solution path. In line 3, thenewly generated path is further investigated by fitting it into approximate straightsegments divided by a sequence of attractors. These attractors are then consideredas a representation of solution path, together with the initial, goal configurationsand obstacle positions, stored into the task database, as shown in line 4. To keepthe database concise, and also make sure the attractors represent general informa-tion of a path, tasks that are generated from scratch always have the priority to bestored in the database. The final step returns the solution path.

2.3.2 Attractor Extraction

As stated before, a set of configurations of the character are extracted as attractorsfrom each solution path. Since the right wrist of the character is the main endeffector for reaching tasks, we fit the path of the right wrist approximately into asequence of piecewise linear segments, and then detect the configuration pointsas attractors where two segments connect. Given a set of 3D points denoting theglobal positions of the right wrist, we employ a variation of the LineTrackingalgorithm [25], which was further experimented and modified by [26]. Note thatour data are configuration points acquired from previously planned path, whichmainly contains straight line segments without a lot of noise. As a result, althoughthe LineTracking algorithm deals with fitting and extracting lines from 2D Laserdata scans, we believe it is also well suited for our 3D data sets.

11

The modified LineTracking algorithm starts by constructing a data window con-taining the first two points, and fits a line with them. Then it adds the next pointto the current line model, and recomputes the new line parameters. If the newparameters satisfy the line condition up to a threshold, the data window incremen-tally moves forward to include the next point. Otherwise, the new included datapoint is detected as an attractor point, and the window restarts with the first twopoints starting from this attractor. Since in our problem the number of configura-tion points contained by a path is always less than a hundred, we limit the lengthof the window by ten.

After a list of attractors are detected from LineTracking, we perform a validationprocedure in order to ensure all the segments connecting the attractors are valid,i.e., collision free. Suppose a collision is detected on a straight segment (ai,ai+1).We find an extra attractor a′, which is the middle point on the original path be-tween ai and ai+1, and insert it to A between ai and ai+1. This attractor insertionprocedure iterates until all of the straight segments are free of collision. The ideais, in addition to LineTracking, the validation procedure will detect extra attrac-tors in difficult regions around obstacles that always cause collisions. If we applythe sequence of valid straight segments to the same task in the same environment,the solution path will be trivially solved by the attractors.

2.3.3 Task Comparison

In order to select a previous path to reuse, we need to define a task comparisonmetric. Consider the case of comparing the query task T with an example taskT ′ from the database. T ′ is considered reusable only when its solution path willshare similar groups of attractor points with the solution of T . The question at thisstage is what information mostly determines the positions of attractor points. Onepossible approach is to define the metric as the distance between the initial andgoal configurations:

dist(T,T ′) = dist(cinit ,c′init)+dist(cgoal,c′goal).

The distance between two configurations is computed in the joint space. Thisnaive metric works well with dynamic queries in a static environment, where allthe obstacles are not moving. When it comes to a dynamic environment, thechange of obstacle positions may largely affect the structure of a solution path,as illustrated in figure 2.2. This motivates us to include the changes of obstaclepositions in the task comparison metric.

12

Figure 2.2: A planning environment with moving obstacles A and B. Task 1 andtask 2 are previously solved (images a and b), and the query task selects an ex-ample from the database. v1, v2 and v3 denote the local coordinates of obstacleA with respect to the three initial configurations. Note that obstacle B is out ofthe local coordinate system range, thus is not included in the comparison metric.Although the initial and goal configurations of task 1 are closer, the query taskchooses task 2 as example because v2 is closer to v3.

When considering obstacles, one important factor is how much influence an ob-stacle has on the solution path. In figure 2.2, obstacle B shows a great change ofposition from task 2 to the query task (images b to c), but since this change haslimited impact on the solution path, we should avoid including this obstacle in thecomparison. Our comparison technique represents obstacles in local coordinatesin respect to the initial and goal configurations.

For each task, we build two coordinate systems with origins respectively locatedat the initial and goal configurations. Obstacles that are too far away from an ori-gin are not included in the respective coordinate system. Our comparison metricaccumulates the distance between each obstacle o from the query task and o′ fromthe example task that is closest to o, computed by their local coordinates. Themetric is described as follows.

dist(T,T ′) = h1∗dist(cinit ,c′init)+h1∗dist(cgoal,c′goal)

+h2∗∑o′mino dist(o,o′), where

o ∈ O from T and o′ ∈ O′ from T ′.

The weights of the motion query distance and the obstacle distance are tuned byheuristic weights h1 and h2.

13

Note that dist(T,T ′) is an asymmetric heuristic; we always use T to denote thequery task and T’ to denote the example task. In this way, we are comparing eachobstacle from the query task to its closet obstacle from the example task. Weare ignoring some obstacles from the example task that are not meaningful to thequery task, which does not affect the planning of the query task.

2.3.4 Guided Planning

Once a set of attractors is decided to be reused, the sampling strategy in the motionplanner is biased toward regions around the attractors. Taking advantage of theattractors is beneficial in two-folds: first, it highly preserves the structure of asuccessful solution path. Second, whenever a moving obstacle collides with partof the solution path, a large portion of the path is still reusable. Note that sinceonly similar tasks are used by the query, we ensure that the environment does notdiffer much.

Algorithm 2 summarizes the AGP guided planning algorithm. Suppose a new taskT with cinit and cgoal has selected a task T ′ = (c′init ,c

′goal,A,O) from database as

example. When planning starts, two search trees Trinit and Trgoal are initialized,and the first and last attractor from set A respectively are selected as samplingguidance, as shown in lines 3 and 4.

We form a dynamic Gaussian distribution around each attractor point in the con-figuration space. It works as a replacement of the random sampling strategy inthe original RRT. The scale of the Gaussian distribution is initialized as zero (line5 and 6), which means we try to sample on the attractor point as a start(line 8).Lines 9 and 10 require searching for the closest configurations in each tree to thesample. If the interpolation between the two configurations is valid (line 11), Rou-tine MakePath is called to connect the tree branch joining root(Trinit) and c1 tothe tree branch joining c2 and root(Trgoal), by inserting the path segment obtainedwith the interpolation between c1 and c2.

The node expansion in line 14 uses crand as growing direction and computes a newconfiguration cexp, as the same as RRT. The difference is, when cexp is returned asvoid, it means the straight segment leading to the current used attractor is not valid.As a result, in line 16, the scale of the Gaussian distribution is increased. Theincreased Gaussian sampler picks samples that are close to the attractor. As timeproceeds, if it remains difficult to find a valid sample close to the current attractor,the dynamic Gaussian distribution gradually decreases the bias on the samplingstrategy. In line 17, when the scale is larger than a threshold, the sampling changesto non-biased as a RRT planner.

14

Algorithm 2 AGPPLANNER(cinit , cgoal , A)1: attractors← A;2: if attractors = null then

return RRT _Planner(cinit , cgoal);3: a1← first attractor;4: a2← last attractor;5: radius1← 0;6: radius2← 0;7: while elapsed time ≤ maximum allowed time do8: crand ← SAMPLE(a1, radius1).9: c1← closest node to csample in Trinit .

10: c2← closest node to csample in Trgoal .11: if INTERPOLATIONVALID (c1,c2) then12: return MAKEPATH(root(Trinit),c1,c2,root(Trgoal)).13: end if14: cexp← NODEEXPANSION(c1, crand , e).15: if cexp = null then16: INCREASE(radius1).17: if radius1 > threshold1 then18: radius1← ∞.19: end if20: end if21: cn← closest node to a1 in T1.22: if DISTANCE(cn,a1) < threshold2 then23: a1← next attractor.24: radius1← 0.25: end if26: Swap Trinit and Trgoal .27: end while28: return failure.

15

Line 21 finds the closet configuration cn in each tree to the current attractor. If thedistance between cn and the attractor is small enough, AGP starts to sample onthe next attractor point. For the tree rooted at the initial configuration, the orderto use the attractors is the same as in the set A, and for the tree started at the goalconfiguration, the order is opposite.

Deciding the best parameters for the guided sampling is not an easy task. Intu-itively, the increase of the sampling radius needs to be small at first and then grad-ually increase. Also, the two values threshold1 and threshold2 need to be tunedsuch as the non-biased sampling will be activated before threshold2 switches thecurrent attractor to the next.

2.4 Experimental Results

We tested a number of different humanoid reaching tasks in dynamic environ-ments to evaluate the use of the attractor guided sampling method within the RRTplanner. Here we are using the basic bidirectional RRT planner as proposed inLavalle’s work [5]. Each test involved planning 200 tasks consecutively. Figure2.3 shows snapshots of some of the test scenarios. In Figure 2.3, scenario (a)represents a single-query problem where every task has the same initial and goalconfigurations, while the environment is cluttered by three randomly moving ob-stacles. In scenario (b) we tested dynamic motion queries in a static environment,where tasks are defined as initial and goal configurations randomly generated inthe free space on both sides of a narrow passage. In scenario (c) and (d) both themotion queries and the positions of obstacles are dynamic: each task consists ofrelocating a book already attached to the character’s hand to a new target locationrandomly chosen.

16

Figure 2.3: Reaching tests scenarios.

While we can conclude from table 1 that AGP outperforms RRT in all the scenar-ios, the amount of improvement varies. It is obvious that in some complicated en-vironments where obstacles occlude in most of the problem-solving region, suchas the narrow passage scenario in Figure 2.3(b), AGP shows a better performancethan others. On the other hand, for simpler tasks such as relocating books on ashelf or a table, obstacles do not collide with the large portion of free space in frontof the shelf, hence the task comparison metric included some information that isnot directly relevant to the motion query. This makes the difference between theinitial configurations and goal configurations less detectable by the task similaritymetric.

17

Table 2.1: Accumulated computation time of AGP (first row) and RRT (secondrow) in seconds for 200 consecutive tasks. The third row shows the number ofentries in the AGP task database.

3(a) 3(b) 3(c) 3(d)

274.058 341.754 226.879 153.342518.133 671.404 302.501 221.329

11 11 13 13

Figure 2.4 shows an example of the solution paths generated by AGP and RRT. Itis possible to observe that due to the limited size of the free reachable space, inmany cases RRT fails to find a solution in a 5-second time allowance. As revealedby figure 2.5, AGP successfully solved 98.5% of the tasks, a higher rate than theratio of success of RRT, which is 94%.

Note that our algorithm uses attractors extracted from solution paths directly gen-erated by RRT, without any path smoothing being applied. We observed in severalexperiments that such set of attractors better represent the structure of the taskand environment. When attractors are extracted from smoothed paths, besidesthe extra computation, they are closer to obstacles and therefore more sensitive toenvironment changes.

18

Figure 2.4: A reaching problem solved by AGP (left) and RRT (right). The redline denotes a successful path, green and blue branches represent the bidirec-tional RRT trees. AGP finds solutions within one second with the help of thetask database, in comparison, RRT fails to find a solution within the maximumtime allowance(5 seconds).

19

Figure 2.5: Computation time (red and blue curved lines) in seconds obtained forsolving 200 consecutive reaching tasks, and task database size of AGP (black solidlines). Note that the red solid curved lines (AGP) predominantly stays lower thanthe blue dashed curved lines (RRT), especially when the database size line staysparallel to the horizontal axis (which means AGP is reusing previous tasks fromthe database). The accumulated computation time is shown in the small rectanglebox.

20

Chapter 3

Learning Manipulation Tasks fromDemonstrated Motions

3.1 Introduction

In the previous chapter, we introduced a method to learn reaching skills from pre-viously planned tasks. In this chapter, we extend this technique and apply it tolearning manipulation skills from real captured motion data. We aim to build apractical framework where the animator can demonstrate highly constrained mo-tions to the virtual character, such as pouring a cup of water. The character is thenable to detect the constraints from the demonstrated motion, and add deformationsto it, making the result motion free of collision while at the same time retainingthe characteristics of the input. The contributions of this chapter are:

1. A constraint detection method for humanoid manipulation motion;

2. The use of AGP that guides the planning not only toward the locations fromthe input motion, but also preserves the characteristics of the input motion.

The characteristics of a motion are usually represented by geometric constraints.For example, to move a cup that is filled with water, it is necessary to make surethat the cup will not perform a rotation around any axis. To pour water from onecup to another, we need the pouring cup to rotate and at the same time its locationremains stationary with respect to the receiving cup.

In previous works, geometric constraints are divided into two subcategories. In-trinsic Constraints are those that are only related to the motion of the character,for example, stationary foot plants or a rotating hand. Interaction constraints, on

21

the hand, represent the interaction between the character and the objects to ma-nipulate, such as the hand is stationary with respect to a bottle, in other words,behaving a solid grasp. In our work, we will only consider the intrinsic con-straints of the character; and the interaction constraints when the end-effector ofthe character and the object to manipulate is stationary to each other. The caseswith more complicated interaction constraints, for example, a pen rotating withrespect to the hand, will be carried on to our future work.

In this chapter, we will propose a constraint-based motion planning technique thatis able to generate collision-free manipulation motions, and meanwhile enforcingthe constraints of the input motion. After a brief overview of related work inconstraints detection, we will present a framework that defines the constraints ofinterest in object manipulations. Then we will present our algorithm for detectingthese constraints from captured input motion in a prioritized order, and how tosegment the input motion regarding to the constraints. Section 3.4 applies AGPto create collision-free motions with the guidance of segmented captured motionexamples. We present experimental results in section 3.5.

3.2 Related Work

In this section, we will introduce several existing works in motion constraint de-tection methods, addressing their success but also their limitations when applyingto humanoid manipulation tasks.

The area of footplants identification has a large demand on constraint detectionmethods. Several methods [27] [28] used specific thresholds on the position andvelocity of the feet to detect footplants. Similarly, [29] extended this approach tobody segments and objects in the environment.

Recent work [11] in constraint detection has built a generic method for detectingintrinsic constraints. It starts with computing all the points that remain stationaryfrom one frame to another. Then, it detects the intersections of these points withbody parts, in order to determine the dimension of the intrinsic constraints. How-ever, this method is unreliable when applied to noisy data obtain from real-worldmotion capture.

Similarly, in [30], the authors propose a method which is more reliable for poten-tially highly noisy data obtained from motion capture. Their method can identifyevents like a character body part being stationary, or rotating around an axis or apoint. Instantaneous constraints are first computed between each pair of framesusing SVD-related methods. Then, they are merged so that the result motion will

22

contain as few constraints as possible. Similar as [11], in their work, instanta-neous constraints are detected based on finding stationary points. While in objectmanipulations, a lot of highly constrained motions do not involve any stationarypoints.

Our work will use a similar approach as in [30] as in finding the displacementmatrix between frames; However, we introduce extra constraints for humanoidmanipulation tasks. Instead of trying to find all the points that remain stationary,we focus on finding the types of transformations that remain stationary or followmeaningful patterns. We define three basic types of constraints that help to iden-tify the motion as: 1) stationary, 2) translation-only or 3) rotation-only. We alsodefine two additional constraints to tell if a translation follows a straight line, ora rotation is around a specific axis. Then, we segment the motion according todifferent types of constraints. The segmented motion is then used to guide futureplannings of similar manipulation tasks.

3.3 Constraint Detection and Motion Segmentation

3.3.1 Constraint Type in Object Manipulation

What kind of constraints are the most crucial in object manipulations? We all havethe experience that it is very difficult to hold an object steadily by hand, such astaking a photograph at night time. Sometimes it is also very important not to be-have some specific motions, as any change in orientation will not be allowed whenrelocating a cup of water. Moreover, the pattern of a movement is also meaningful,for example, whether the right hand of the character is traveling along a straightline on top of a surface, or just along some random paths in the space. In the fol-lowing we provide a definition for these constraints in object manipulation. Notethat in this work, we only consider the cases where the character is manipulatingwith a single object, and only when they are stationary to each other. Thus, thecharacter’s hand and the object can be viewed as one single end-effector.

Stationary This kind of constraint represents the scenarios that both theposition and the orientation of the end-effector remain the same in the worldcoordinate system.

Translation-only This kind of constraint represents the scenarios that theorientation of the end-effector remains the same in the world coordinatesystem.

23

Rotation-only This kind of constraint represents the scenarios that the po-sition of the end-effector remains the same in the world coordinate system.

Patterned-Rotation This kind of constraint may apply if a translation-onlyconstraint is not previously detected. It refers to the scenarios that the end-effector is rotating around an axis or a point.

Patterned-Translation This kind of constraints may apply if a rotation-only constraint is not previously detected. It refers to the scenarios that theend-effector is translating along a straight line.

Before we apply these constraints to our algorithm, it is beneficial to first discussthe relationships between each constraint and the other. We call the first threetypes of constraints: stationary, translation-only and rotation-only as basic types.We can see that the stationary constraint has the top priority. Only when the end-effector does not meet the stationary constraints, then we need to proceed withthe other constraints. We can also see that translation-only constraints can existat the same time with patterned-translation constraints for a motion, but not withpatterned-rotation constraints. Similarly, rotation-only constraints can exist at thesame time with patterned-rotation constraints, but not with patterned-translationconstraints. Based on the above observation, we can build a flowchart to illustratehow the constraints of a motion can be determined, as shown in figure 3.1. Thisdecision graph will attach "tags" to a piece of motion such as "not rotating andtranslating along a straight line", or "not translating but rotating randomly".

3.3.2 Constraint Detection

Our constraint detection algorithm works on an end effector E which consistsof a hand of the character and the object to manipulate with. Given a specifictime interval [t1, tn], we denote motion m of the end-effector as a sequence offrames [ f1, f2, ..., fk, fk+1, ..., fn]. Each frame fk contains geometric informationabout the end effector at that time step, i.e., the position and orientation of the end-effector with respect to the world coordinate system. Instantaneous constraints arefirst detected between each pair of frames. Then, the sequence of instantaneousconstraints will be merged so that the motion m will contain as few constraints aspossible.

24

Figure 3.1: Stages of constraint detection. Boxes: Input or Output motions. Tri-angles: motion constraints. Boxes of Y or N respectively denote whether theconstraints can be detected or not.

Detection of Instantaneous Constraints

An instantaneous constraint Ck is detected between time k and k +1. Given a pairof frames fk and fk+1, the definition of Ck depends on the transformation of theend effector from time k to time k + 1. We want to compute the displacementmatrix Dk of the end effector E with respect to the world coordinate system. Inother words, for a point p on the end effector E, if we use xk (resp. xk+1) to denoteits global homogeneous coordinates at time k (resp. time k + 1), then we want tofind the displacement matrix Dk such that xk+1 = Dkxk. The computation of Dk isillustrated as follows:

Let p be the local homogeneous coordinates of point p. At time k, we have:

25

xk = Wk p

Where Wk is the global transformation matrix of E at time k. Similarly, at timek +1, we can get the coordinates of point p in the world coordinate system as:

xk+1 = Wk+1 p

= Wk+1W−1k Wk p

= Wk+1W−1k xk

so we have:Dk = Wk+1W−1

k

Given the displacement matrix Dk from time k to k +1, we are able to compute ifthe end effector has any kind of movements within this single time step. If E isstationary from time k to k + 1 with respect to the world coordinate system, thenfor all the points located on the end effector, we have

(Dk− I4)xk = 0

Since the solutions of the above equation include all the points on the end effector,we know the elements in matrix (Dk− I4) have very small values, such as:

0 0.000219 −0.000110 0.000486−0.000219 0 0.000228 −0.0008710.000110 −0.000118 0 0.000645

0 0 0 0

Note that here we use centimeter as the unit for translational data, so its value hassimilar order of magnitude with the rotational data. If the we need a smaller unit,it is necessary to scale the range of translational data same as the rotational data,which is [-1,1].

We call the squared sum of all the elements in matrix (Dk− I4) as the "stationaryvalue" of constraint Ck, because this value denotes how much movements the endeffector performed during this time step. If this value is smaller than a stationarythreshold Sth, than we can say that the end effector E is stationary from time k totime k +1. We will introduce how to compute Sth in section 3.3.3.

As the end effector can only perform rigid transformations, we can rewrite Dk as:

Dk =[

Rk tk03 1

]26

Where Rk and tk are the rotational and translational components of Dk. The matrix(Dk− I4) can then be written as:

Dk− I4 =[

Rk− I3 tk03 0

]If E is performing a translation-only movement during this time step, one exampleof matrix (Dk− I4) is:

−0.008967 0.002149 0.019083 1.84927−0.002118 −0.000130 −0.001649 0.062264−0.019086 0.001608 −0.000766 0.010757

0 0 0 0

Similarly, we call the squared sum of all the elements in matrix (Rk− I3) as the"rotational value" of Ck. If it is smaller than a rotational threshold Rth, then themovement of E will be identified as translation-only, which is to say, with norotation involved. Similarly, for a rotation-only movement during one time step,we have matrix Dk− I4 such as

−0.000574 0.008643 0.032758 0.167622−0.005686 −0.004012 −0.089310 0.221429−0.033398 0.089073 −0.004535 0.005759

0 0 0 0

its constraint can be detected by comparing the squared sum of vector tk ("trans-lational value") with translational threshold Tth.

In the following we list the three threshold values used above; the computation ofthese parameters will be provided in section 3.3.3.

Stationary-threshold This is the threshold value to evaluate if the end ef-fector is stationary. We denote it as Sth. For an instantaneous constraintCk, if its stationary value – the squared sum of all the elements in (Dk− I4)– is smaller than this threshold, then we say the end effector is stationarybetween time k and k +1.

Rotational-threshold This is the threshold value to evaluate if the end-effector is rotating. We denote is as Rth. For an instantaneous constraintCk, if its rotational value – the squared sum of all the elements in (Rk− I3)– is smaller than this threshold, then we say the end effector is not rotatingbetween time k and k +1.

Translational-threshold This is the threshold value to evaluate if the end-effector is translating. We denote it as Tth. For an instantaneous constraintCk, if its translational value – the squared sum of all the elements in tk – is

27

smaller than this threshold, then we say the end effector is not translatingbetween time k and k +1.

Now we are able to detect the first three types of constraints: Stationary, Translation-only and Rotation-only. To detect the other two constraints: patterned-rotationand patterned-translation, we need to find a way to merge the transformations thathave the same patterns.

Merging the transformations

We have computed all the transformations between each pair of frames. We knowthat Rk denotes the rotational component of the transformation between time kand k+1. If we write Rk as

Rk =

r00 r01 r02r10 r11 r12r20 r21 r22

Then we can directly extract the rotational axis dk = (r21− r12,r02− r20,r10−r01). Given a sequence of frames [ f1, f2, ..., fk, ..., fn], we can compute all therotational axes [d1,d2, ...,dk,dk+1, ...,dn]. If they can be approximately fitted bya straight line L, then motion m is a rotation around the line L. That is to say,the transformations between time 1 and n can be merged together to denote m.Similarly, if all the translational vectors can be approximately fitting into a straightline M, then we can say the end-effector is translating along the line M. The line-fitting process was implemented same as in section 2.3.2. Again, the computationof the parameters for the line-fitting, represented as Rl f and Tl f will be providedin section 3.3.3.

Since we are dealing with real motion capture data, the existence of noise or hu-man operations will create outliers, i.e., one or more frames that divide one motioninto two parts, while the motion is supposed to be continuous. To deal with out-liers, we create a frame buffer, and store inside every aberrant frame that showsa different motion pattern with previous frames. A new motion will be createdonly when the number of aberrant frames in the frame buffer reaches a limit. Wedefine this buffer limit as additional parameters for detecting patterned-rotationand patterned-translation; the computation will be provided in later sections.

Buffersize-Rotation: This is the threshold value to decide if a patternedrotation has ended. We denote it as Rbu. When the number of inconsistent

28

frames in a patterned rotation is greater than Rbu, this rotation is consideredended and a new motion will be created.

Buffersize-Translation: This is the threshold value to decide if a patternedtranslation has ended. We denote it as Tbu. When the number of inconsis-tent frames in a patterned translation is greater than Tbu, this translation isconsidered ended and a new motion will be created.

Merging the Constraints

After a set of constraints are detected, they need to go through a final filter tobe merged so that the motion does not contain unnecessary segments. Two con-straints can be merged together if they belong to the same constraint category andare close enough to each other. This step accumulates together those constraintsthat belong to the basic constraint types, but not merged into any patterned mo-tions from the last step. For example, if there is a translation-only constraintdetected between time k and k +1, it can be merged with another translation-onlyconstraint between time k+1 and k+2, even if they do not belong to the category"traveling along a straight line".

3.3.3 Parameter Computation

From the previous section, we have a set of parameters that we need to use fordetecting and merging constraints: [ Sth , Rth , Tth , Rl f , Tl f , Rbu , Tbu ]. In orderto perform a robust computation on the constraints, the best way to get the valueof the parameters is through template motions, ideally demonstrated by the sameperformer. We need the performer to demonstrate three motions:

m1: stationary

m2: rotation-only (without translation) around an axis

m3: translation-only (without rotation) along a straight line

From motion m1 we can determine the value of stationary threshold Sth. Supposem1 is denoted as a set of frames [ f1, f2, ..., fk, fk+1, ..., fn]. First, we compute allthe displacement matrix [D1,D2, ...,Dk,Dk+1, ...,Dn−1] according to the previoussection. Let σk be the stationary value of instantaneous constraint Ck, i.e., the

29

squared sum of all the elements in matrix (Dk− I4), then for all the instantaneousconstraints, the set of their stationary values is: SV = [σ1, σ2, ..., σk, σk+1, ..., σn].

It is not correct to take the highest σ value and assign it as the stationary threshold.Due to the presence of noise and human operations in motion capture process,there might be outlier data in set SV . For example, a subtle shake of the humanhand might result a very high σ value. In order to reduce the overestimation ofthe stationary threshold, we need to find and reject outlier data as follows:

First, we find the median number σm from set SV . Then for each σk ∈ SV , (k 6= m),find its squared distance with σm as distk = (σk−σm)2. Let the smallest squareddistance be distmin. We then identify a number σk as an outlier if distk > h∗dmin,where h is a heuristic weight which is bigger than 1. After the outliers are removedfrom the set of the stationary values, we then set the Stationary-threshold Sth asthe highest stationary value from the rest of SV .

We can also compute the Rotational-threshold and the Translational-thresholdsimilarly. Since motion m2 has been tagged as a rotation-only motion withouttranslation, most of its instantaneous constraints should be lower than the transla-tional threshold Tth. We first get the translation vectors [t1, t2, ..., tk, tk+1, ..., tn−1],and compute the set of translational values from them. After removing the out-liers, we can assign Tth as the highest translational value from the set. On the otherhand, motion m3 is translation-only, so we can obtain the rotational threshold Rthfrom it.

The other two parameters that we obtain from m2 and m3 are the line fitting thresh-olds: Rl f and Tl f . Since motion m2 is a rotation around a specific axis, we can fitall the detected instantaneous rotational axes by an approximate straight line. Af-ter removing outliers, Rl f is set as the biggest distance between the instantaneousrotational axes and the fitted line. Tl f can be computed similarly from motion m3.We also assign the maximum number of continuous outliers as Rbu and Rtu.

3.3.4 Motion Segmentation

Our goal is: given any form of captured motion data, we will detect the constraintsfrom it and keep the constraints in future deformations. In previous sections, weintroduced how to detect constraints from stationary motion, meaningful rotationsand translations. However, object manipulation tasks are always complicated andare composed of multiple basic motion skills. For example, a water-pouring taskincludes a translation without rotation, and then followed by a rotation withouttranslation. We propose a motion segmentation method to divide a motion into

30

a sequence of sub-motions according to their constraint types. The connectionsbetween these sub-motions are then represented by attractors.

An attractor at , as stated in chapter 2, is a configuration of the end effector. Whena sub-motion ends another sub-motion with a different constraint type starts, weextract the configuration of the end effector from the connecting frame, and alsorecord the constraint type from the previous motion. This attractor, together withthe constraint type, will be used to guide future plannings for similar tasks.

3.4 Attractor-Guided Planning

Every time when a new manipulation task is given, we will use the segmenteddemonstration motion and the set of attractors to guide the planning. The processof the guided planning is similar with the one explained in section 2.5.3. Thereare several differences to note. The first difference is that the sampling is not onlybiased toward regions around attractors, but also toward the constraints definedwith the attractors. For example, if the attractor is located on a translate-onlymotion, then the samples will only be generated to have the same orientation withthe attractor. This encourages that even in a changing environment, the constraintsof the original motion will still be maintained. Second, the attractors are not anymore configurations of the character, but instead, are configurations of the endeffector.

3.5 Experimental Results

3.5.1 Motion Capture

We used the FASTRAK system in our motion capture process. It includes onetransmitter and two receivers that record the position and orientation of the endeffector. Both of the two receivers can be operated at the update rate 60HZ. Webinded a receiver on the object-to-manipulate, and the other one on the performer’shand. The position and orientation of the two receivers are both recorded fourtimes at each second.

31

Figure 3.2: Complete FASTRAK System.

3.5.2 Constraints Enforced Planning

We first tested our constraint-detection method with a number of different demon-strated motions. After the template motions are processed, our performer per-formed the following motions:

m1: translation - translation

m2: translation - rotation

m3: translation - rotation - translation

m4: translation - "random movements"

Figure 3.3 shows the segmentations of the above four motions. Each segment, Asshown in between each pair of attractors (grey spheres), denotes one single pieceof motion that has different constraint types from its neighbor segments. We cansee that our constraint-detection method is capable of detecting the difference be-tween the translations and rotations, and the "connecting-motions" between them.The "connecting motion" is usually a short piece of motion with the stationaryconstraint, which shows a natural stop when the performer transact between trans-lations and rotations.

Second, we applied the AGP planner to solve a manipulation task: the charac-ter was relocating a cup. We provide a demonstrated motion which involves atranslation without rotation. The demonstrated translation is along a straight linethat connects the initial and goal positions. After recording the demonstration, weplace an obstacle which results in collision with the path of the demonstration.Both the RRT and the AGP are used to plan a new path between the same initial

32

Figure 3.3: The segmentations of four motions. Recorded motions are seg-mented according to different types of constraints. The attractors (denoted bygrey spheres) will be used to guide the planning for future tasks.

and goal positions, but in presence of the obstacle. Figure 3.4 shows the solu-tion path found by the RRT planner; we can clearly see that the orientation of thecup has many variations along the solution path. Figure 3.5 shows that the AGPplanner not only found a shorter path than the RRT planner, but also maintainedthe characteristic of the demonstrated motion: the cup is never rotated during therelocation.

33

Figure 3.4: The solution path found by the RRT planner. An obstacle is denotedby the small cube.

Figure 3.5: The solution path found by the AGP planner. An obstacle is denotedby the small cube.

34

Chapter 4

Conclusion and Future Work

This document presents an attractor guided planning algorithm (AGP) that learns fromprevious experiences and real detected motion data. The algorithm is able to learn featuresand constraints of existing solutions or motion data; store them in a task database andreuse them when solving new tasks. Our experiments show that the method is well suitedfor humanoid reaching problems in changing environments. AGP can also be applied toother generic planning problems, and will show better performance in problems involvingthe repetition of similar tasks.

In summary, the following main contributions are proposed in this thesis:

• The AGP sampling method which avoids exploring spaces that are irrelevant to theplanning query by dynamically biasing the sampling around attractor points.

• A constraint-detection method that is able to detect meaningful constraints fromreal detected motion data.

• The use of a line fitting algorithm for extracting meaningful attractor points from agiven path before being smoothed, or from a segmented input motion.

• A simple but efficient task comparison technique which represents obstacles inlocal coordinates in respect to the initial and goal configurations.

Although our current results are not yet capable of achieving real-time performance noroverall realistic humanlike motions, we believe that the techniques introduced here arekey for getting closer to these goals.

There are some limitations existing in our work. For example, the attractor-guided sam-pling strategy is most suitable in environments that are changing, but not changing dra-matically. If the environment changes dramatically, many of the attractors will lose theireffects, and in the worst case, the AGP planner will have a similar performance as theRRT.

35

Bibliography

[1] J.T.Schwartz and M. Sharir, “On the piano movers’ problem: Ii. general tech-niques for computing topological properties of algebraic manifolds,” Com-munications on Pure and Applied Mathematics, 1983.

[2] J.F.Canny, “The complexity of robot motion planning.” MIT Press, 1988.

[3] J.-C. Latombe, Robot Motion Planning. Kluwer Academic Publisher, De-cember 1990.

[4] L. Kavraki, P. Svestka, J.-C. Latombe, and M. Overmars, “Probabilisticroadmaps for fast path planning in high-dimensional configuration spaces,”IEEE Transactions on Robotics and Automation, vol. 12, pp. 566–580, 1996.

[5] J. J. Kuffner and S. M. LaValle, “Rrt-connect: An efficient approach tosingle-query path planning,” in Proceedings of IEEE Int’l Conference onRobotics and Automation (ICRA), San Francisco, CA, April 2000.

[6] J. Lasseter, “Principles of traditional animation applied to 3d computer ani-mations,” ACM Computer Graphics, vol. 21, no. 4, July 1987.

[7] A. Bruderlin and L. Williams, “Motion signal processing,” Proceedings ofACM SIGGRAPH, 1995.

[8] A. Witkin and Z. Popovic, “Motion warping,” Proceedings of ACM SIG-GRAPH, 1995.

[9] S. Tak, O.-Y. Song, and H.-S. Ko, “Spacetime sweeping: An interactivedynamic constraint solver,” Proceedings of Computer Animation and SocialAgents, 2002.

[10] H. J. Shin, L. Kovar, and M. Gleicher, “Physical touch-up of human mo-tions,” Proceedings of Pacific Graphics, Oct 2003.

[11] C. K. Liu and Z. Popovic, “Synthesis of complex dynamic character motionfrom simple animation,” Proceedings of ACM SIGGRAPH, 2002.

36

[12] S. Schaal, “Arm and hand movement control,” in The handbook of braintheory and neural networks, 2nd ed., M. Arbib, Ed. The MIT Press, 2002,pp. 110–113.

[13] J. J. Kuffner and J.-C. Latombe, “Interactive manipulation planning for ani-mated characters,” in Proceedings of Pacific Graphics’00, Hong Kong, Oc-tober 2000, poster paper.

[14] S. Kagami, J. Kuffner, K. Nishiwaki, S. Kagami, M. Inaba, and H. Inoue,“Humanoid arm motion planning using stereo vision and RRT search,” inJournal of Robotics and Mechatronics, April 2003.

[15] M. Kallmann, “Scalable solutions for interactive virtual humans that can ma-nipulate objects,” in Proceedings of the Artificial Intelligence and InteractiveDigital Entertainment (AIIDE’05), Marina del Rey, CA, June 1-3 2005, pp.69–74.

[16] P. Leven and S. Hutchinson, “Toward real-time path planning in changingenvironments,” in Proceedings of Workshop on Algorithmic Foundations ofRobotics, 2000.

[17] M. Kallmann and M. Mataric’, “Motion planning using dynamic roadmaps,”in Proceedings of the IEEE International Conference on Robotics and Au-tomation (ICRA), New Orleans, Louisana, 2004.

[18] K. E. Bekris, B. Y. Chen, A. M. Ladd, E. Plaku, and L. E. Kavraki,“Multiple query probabilistic roadmap planning using single query planningprimitives,” in International Conference on Intelligent Robots and Systems(IROS), Las Vegas, NV, October 2003, pp. 656–661.

[19] T.-Y. Li and Y.-C. Shie, “An incremental learning approach to motion plan-ning with roadmap management,” Journal of Information Science and Engi-neering, pp. 525–538, 2007.

[20] V. Boor, M. Overmars, and A. Stappen, “The gaussian sampling strategy forprobabilistic roadmap planners,” in Proceedings of IEEE Int’l Conferenceon Robotics and Automation (ICRA), Detroit, Michigan, 1999.

[21] D. Hsu and Z. Sun, “Adaptive hybrid sampling for probabilistic roadmapplanning,” National University of Singapore, Tech. Rep., 2004.

[22] M. Morales, L. Tapia, R. Pearce, S. Rodriguez, and N. Amato, “A machinelearning approach for feature-sensitive motion planning,” in Proceedings ofthe Workshop on the Algorithmic Foundations of Robotics, 2004.

37

[23] B. Burns and O. Brock, “Sampling-based motion planning using predictivemodels,” in Proceedings of the IEEE Int’l Conference on Robotics and Ani-mation (ICRA), 2005.

[24] M. Kallmann, “A simple experiment on the effect of biasing sampling distri-butions for planning reaching motions with RRTs,” University of California,Merced, Tech. Rep., February 2007.

[25] A. Siadat, A. Kaske, S. Klausmann, M. Dufaut, and R. Husson, “An opti-mized segmentation method for a 2d laser-scanner applied to mobile robotnavigation,” in Proceedings of the 3rd IFAC Symposium on Intelligent Com-ponents and Instruments for Control Applications, 1997.

[26] V. T. Nguyen, A. Martinelli, N. Tomatis, and R. Siegwart, “A comparisonof line extraction algorithms using 2d laser rangefinder for indoor mobilerobotics,” in International Conference on Intelligent Robots and Systems(IROS05), Edmonton, Canada, 2005.

[27] L. Kovar, M. Gleicher, and F. H. Pighin, “Motion graphs,” ACM Transactionon Graphics (Proceedings of SIGGRAPH’02), vol. 21, no. 3, pp. 473–482,2002.

[28] S. Menardais, R. Kulpa, F. Multon, and B. Arnaldi, “Synchronization ofinteractively adapted motions,” Proceedings of ACM SIGGRAPH, Aug 2004.

[29] J. Lee, J. Chai, P. S. A. Reitsma, J. K. Hodgins, and N. S. Pollard, “Inter-active control of avatars animated with human motion data,” Proceedings ofACM SIGGRAPH, 2002.

[30] B. L. Callennec and R. Boulic, “Interactive motion deformation with priori-tized constraints,” Graphical Models, 2006.

38