object sensitive grasping of disembodied barrett hand

15
December 18, 2013 Object Sensitive Grasping of Disembodied Barrett Hand Neil Traft and Jolande Fooken University of British Columbia Abstract The proposed goal of this project was to be able to adapt the grasp shape of a robotic hand, based on previous experience, to produce a more stable and dexterous grasp. Following this goal, we looked for dierent ways a robotic hand could build upon its previous grasping experience. We explored the idea of employing the adap- tive grasping algorithms of Humberston and Pai [1], but we found some limitations to implement- ing this work on the Barrett robotic arm and hand. Instead, we opted to try to classify or recognize the object being grasped using mod- ern statistical techniques. We were able to train a neural network to recognize objects in both training and test data sets with a high degree of accuracy (in some cases over 99%). However, when these grasps were repeated on the robot, we were unable to obtain any kind of reliable recognition. The reasons for this may be due to a variety of factors which will be discussed in the following paper. We conclude that the idea of using high-level knowledge about an object to choose strategies for grasping is justified and realizable. However, using neural networks as a tool for encoding this knowledge may not be viable. I Introduction When humans perform simple grasping task in every day life, they depend on a combina- tion of their visual system as well as their sen- sorimotor memory. Hereby, the human hand relies on about 17000 mechanoreceptive tactile units [2] embedded in the hairless skin of the palm that are able to give feedback in response to e.g. touch, pressure or vibrations, constantly adapting fingertip forces and grasping strength. Lifting up an object, such as a cup or a pen, is consequently followed by a cascade of sensory signal generation and processing [3]. In humans, visual information of the objects properties during grasping is important, how- ever not essential [4]. Consequently, a lot of research eort has been put into tactile-driven approaches for robotic grasp control [5] [6]. The main challenge remains yet to find a dexterous robotic grasping technique that can cope with the wide range of dierent grasping contexts. In other words, to mimic natural human grasping behavior as accurately as possible. Conventionally, there are two approaches to developing grasping strategies and algorithms. While the first one uses geometric object mod- els, i.e. calculates a geometry-based, object- specific optimal hand posture, the second ap- proach solely depends on tactile feedback upon contact with the object being grasped. Both

Upload: others

Post on 15-May-2022

6 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Object Sensitive Grasping of Disembodied Barrett Hand

December 18, 2013

Object Sensitive Grasping of Disembodied Barrett Hand

Neil Traft and Jolande Fooken

University of British Columbia

Abstract

The proposed goal of this project was to beable to adapt the grasp shape of a robotic hand,based on previous experience, to produce a morestable and dexterous grasp. Following this goal,we looked for different ways a robotic hand couldbuild upon its previous grasping experience.

We explored the idea of employing the adap-tive grasping algorithms of Humberston and Pai[1], but we found some limitations to implement-ing this work on the Barrett robotic arm andhand. Instead, we opted to try to classify orrecognize the object being grasped using mod-ern statistical techniques. We were able to traina neural network to recognize objects in bothtraining and test data sets with a high degreeof accuracy (in some cases over 99%). However,when these grasps were repeated on the robot,we were unable to obtain any kind of reliablerecognition. The reasons for this may be due toa variety of factors which will be discussed in thefollowing paper.

We conclude that the idea of using high-levelknowledge about an object to choose strategiesfor grasping is justified and realizable. However,using neural networks as a tool for encoding thisknowledge may not be viable.

I Introduction

When humans perform simple grasping taskin every day life, they depend on a combina-tion of their visual system as well as their sen-sorimotor memory. Hereby, the human handrelies on about 17000 mechanoreceptive tactileunits [2] embedded in the hairless skin of thepalm that are able to give feedback in responseto e.g. touch, pressure or vibrations, constantlyadapting fingertip forces and grasping strength.Lifting up an object, such as a cup or a pen,is consequently followed by a cascade of sensorysignal generation and processing [3].

In humans, visual information of the objectsproperties during grasping is important, how-ever not essential [4]. Consequently, a lot ofresearch effort has been put into tactile-drivenapproaches for robotic grasp control [5] [6]. Themain challenge remains yet to find a dexterousrobotic grasping technique that can cope withthe wide range of different grasping contexts. Inother words, to mimic natural human graspingbehavior as accurately as possible.

Conventionally, there are two approaches todeveloping grasping strategies and algorithms.While the first one uses geometric object mod-els, i.e. calculates a geometry-based, object-specific optimal hand posture, the second ap-proach solely depends on tactile feedback uponcontact with the object being grasped. Both

Page 2: Object Sensitive Grasping of Disembodied Barrett Hand

Proactive Grasp Adaptation of Disembodied Barrett Hand 2

approaches have the drawback that each graspwill be performed independently of the previousgrasp experience. In contrast to this, humansuse previous grasping information to preshapetheir grasp. (The simple example of a personlying in bed at night and reaching for a glass ofwater as opposed to a phone or a book illustratesthis.) Accordingly, more recent ideas integratesome kind of grasp experience into the planningof the subsequent grasp [7] [8].

Grasp Preshaping

The main idea of grasp adaptation is to usepreviously acquired grasping knowledge to im-prove future grasping strategies. One possi-ble method we proposed was to try to equalizetime-to-contact across all fingers, based on pre-vious grasp shapes. This idea was inspired bythe adaptive grasping algorithms of Humberstonand Pai [1]. However, in the course of the projectwe found many limitations to adapting this spe-cific algorithm to the Barrett robotic arm andhand.

As the Barrett Hand is equipped with 1-DOFfinger joints, preliminary preshaped grasps werevery similar. Most of the grasping action is gov-erned by the automatic TorqueSwitch™ mecha-nism in each finger [10]. Also, since the objectsbeing grasped were not fixed to the pedestal, thehand had the tendency to push them around un-til all three fingers were making contact simul-taneously.

In addition to the problems inherent in thehardware, we faced temporary technical difficul-ties with respect to the torque sensor data collec-tion. As the torque sensor readouts would havebeen crucial in identifying the different times ofcontact for each finger, we finally discarded theidea of preshaping the Barrett Hand.

Object Recognition

Our other main interest was in how the prop-erties of the object being grasped would influ-ence the sensor output. The Barrett Hand isequipped with a rich set of sensors which coverthree different modalities. This is analogous tothe mechanoreceptors of the human fingertips,which themselves cover at least three differentmodalities: strain, vibration, and rate of change[2]. In our case, the modalities are mass (givenby the force torque sensor), geometric shape (in-ferred from finger joint positions), and pliancy(given by tactile pressure sensors). We expectthat the combination of three such orthogonalmodalities will constitute a fairly unique descrip-tion of an object.

Given such compelling sensor feedback,would the system be able to recognize an objectfrom a predefined trained set? We ultimatelyopted to bring statistical classification to bearon this question. This approach is motivated bythe idea that once the system has more high-level information about the type of object it issensing, it can employ grasps/strategies suitedto that particular type of object. A key part ofusing previous experiences is being able to sortand categorize those experiences.

II Methods

System Overview

The system consists of the 7-DOF BarrettWAM robot arm and 4-DOF Barrett BH-280Hand from Barrett Technology, Inc (compare fig-ure 1). The robot is equipped with one 6-DOFwrist torque sensor, three 1-DOF finger jointtorque sensors, and four 24-DOF tactile pressuresensors, making for a total of 105 independent

Page 3: Object Sensitive Grasping of Disembodied Barrett Hand

Proactive Grasp Adaptation of Disembodied Barrett Hand 3

sensor inputs. Given such rich sensory input,we hoped to obtain feature vectors which ex-hibit statistically significant differences betweendifferent grasp shapes. All the sensors read at125 Hz. Most afferent inputs in humans run atless than 60 Hz, so this rate is sufficient to mimicphysiological driven grasping approaches [9].

Figure 1 7-DOF Barrett WAM robot armand 4-DOF Barrett BH-280 Hand from BarrettTechnology, Inc.

Software Architecture

The software for running our experiments onthe Barrett WAM and Hand is a menu-basedcommand-line program that makes it easy torecord sensor data and test out differentlytrained neural networks. The hand’s homeposition as well as the initial grasp positions arepredefined and can be called individually. Onthe main operational level, the menu lets theuser choose one of 5 different grasp types:

Top-down Prismatic Precision

Top-down Tripod Precision

Side-on Heavy Wrap

Page 4: Object Sensitive Grasping of Disembodied Barrett Hand

Proactive Grasp Adaptation of Disembodied Barrett Hand 4

Side-on Power Grip as well asSide-on Prismatic Precision

Thus, each grasp is a combination of twofactors: the position of the hand and theposition of the fingers. The position of thehand, referred to as the "target position", canbe either from the side (side-on) or from above(top-down). The position of fingers 1 and 2 canbe at 0

� (prism), 30� (tripod), or 180

� (wrap).When the user chooses one of the above grasps,the robot follows a fixed sequence of states:

First Move to preparatory position.

Second Prepare (preshape) the hand for theparticular grasp type (prism, tripod, orwrap).

Third Move to the target position (side-on ortop-down).

Fourth Close the hand on the object.

Fifth Lift the object briefly and return it to thepedestal.

Sixth Release the object and retreat to thepreparatory position.

During steps 3-5, the following sensors arerecorded and logged to disk:

• WAM joint positions

• Finger joint positions (outer link)

• Finger torques

• 3D wrist forces

• Palm and finger tactile pressures

The software is structured such that all menuoptions are executed asynchronously. The useralways retains control and can cancel the cur-rent sequence at any time. Over time work-ing with the robot we also found it necessary toadd various facilities for identifying the name ofthe object currently being grasped, resetting thehand/WAM if they have controller issues, andrecording a failed grasp. We use these annota-tions to sort and label our sensor data samples.

Data Collection

The objects grasped varied in shape, size, sym-metry, texture, weight, pliancy, and firmness.For details see Figure 2. Each object wasgrasped several times with 3 (if possible) differ-ent grip strategies (Top-down Prismatic Preci-sion, Heavy Wrap, and Power Grip).

Data from many trials of grasping these ob-jects were collected into log files. These files werethen imported into Matlab and sorted by ob-ject and grasp type. From these files, we tookonly the interval during which the object wasbeing grasped.

The finger joint positions were used to de-termine the time interval for sensor sampling.Initially, the finger torques were used for thispurpose, but later in the project we encounteredtechnical difficulties in communicating with the

Page 5: Object Sensitive Grasping of Disembodied Barrett Hand

Proactive Grasp Adaptation of Disembodied Barrett Hand 5

Figure 2 Set of grasped objects on which clas-sification was performed. (1) styrofoam ball, (2)soft foam, (3) styrofoam cone, (4) foam square,(5) wood block, (6) plush octopus, (7) foam but-terfly, (8) packaging tape, (9) rattan ball, (10)cookie cutter, (11) wooden egg, (12) football,(13) foam star, (14) drinking bottle, (15) cube,and (16) bean bag.

strain gages, and this method had to be altered.Luckily, the joint position data proved sufficient.As the two positional changes (i.e. the maximumand minimum of the derivative) mark start andend time point of each grasp, the specific timestamps for each trial could be calculated andused to find only the relevant data set for theneural network analysis. Subsequently, each ob-ject was assigned a label to run a classification.

Neural Network

To classify a grasp, the sensor data were nor-malized and used to train a three layer neuralnetwork. We read a total of 103 sensor values,and classified among 16 possible objects. Ad-ditionally, we formed a class ‘Failed Grasp’ towhich we assigned all failed grasps independentof the object, making for a total of 17 classes.

Therefore, the neural network consists of a

103 node input layer, a 25 node hidden layer,and a 17 node output layer. The implementationwas largely based on the one found in AndrewNg’s Machine Learning course [12]. Since thissingle hidden layer with 25 nodes was enough toperform robust character recognition in [12], itwas deemed a satisfactory configuration for ourpurpose as well. Figure 3 depicts the nodes inthe three layers of the neural network.

After the first round of experiments, we usedthe labeled data we collected to train a sepa-rate neural network for each of the three chosengrasps. All 103 features were separately normal-ized before training. For each sensor i, the col-umn vector vi of all samples becomes

vi =vi �mean(vi)

std(vi)

Neural networks were trained using a costfunction J(⇥) similar to K regularized logisticregressions, where K is the number of classes,17. We also added a regularization term with anadjustible weight �. The training algorithm iter-atively finds the parameters ⇥ which minimizethe cost function J(⇥) by computing the costfunction gradient @J(⇥)

@⇥ in the neural network ateach step over all training examples. For fulldetails of the algorithm, see [12].

We produced a variety of different networkswith different regularization weights from � = 1

to � = 1000. 20% of the collected time pointsfrom each grasp were set aside as a test set toverify our results.

At this point a module was added tothe software which predicted the object beinggrasped, given one or more samples of the abovesensor data. The final version of the soft-ware printed out the name of the object whichit “sensed”, while lifting the object from the

Page 6: Object Sensitive Grasping of Disembodied Barrett Hand

Proactive Grasp Adaptation of Disembodied Barrett Hand 6

Figure 3 A three layer neural network forthe classification of grasped objects from fin-ger pose, wrist force, and tactile pressure data.Taken from Programming Exercise 4 in [11].

pedestal. This was implemented by taking asingle time slice of sensor data while graspingan object, and feeding it forward through eachlayer of the neural network. In each layer,

a(i+1)

= sigmoid

�⇥e a(i)

⇤⇥

(i)�

where a(1) 2 Rm x 103

= the sensor samples

a(i) = output of layer i

(i)= parameters of layer i

e = column of all ones

sigmoid(z) =1

1 + e

�z

The final prediction is taken as the labelwhich was assigned the maximum probability bythe output layer:

object = index of max(a(3))

III Results

Sensor Data

The output data of the various sensors may in-dicate different modalities of the object beinggrasped. While the force torque sensor will reactstrongly to the weight of the object, the fingerjoint positions are more sensitive to the shape.Orthogonal to either of these, the tactile pres-sure sensors will give information of the object’scompliance.

If we had obtained finger torque mea-surements, these would have additionally con-tributed to our picture of both the shape andthe compliance of the object. Unfortunately, thedata recording of the finger strain gages causedmajor technical difficulties so that it could notbe done consistently. This was a severe setback,as the finger torques are the most sensitive mea-sure to initial contact with the object. Not onlydid this halt our plans for a grasp preshaping al-gorithm, it also forced us to reprogram part ofour data collection method.

Still, the remaining sensors give us quite afull and diverse description of an object. Let usdiscuss the finger positions. Figure 4 shows thejoint position profile while grasping the cone un-der each of the respective grasp types. The posi-tional change is recorded in radians over the timespan of the grasp. If we examine these graphswe can gain insight into the nature of both thegrip and the object being grip.

Once the grasp is initiated, the position ofeach joint increases steadily until the object isfully enclosed in the hand. The joints remainat this maximal position until the object is re-leased. The most interesting grip in this scenariois the top-down prismatic, where the hand grips

Page 7: Object Sensitive Grasping of Disembodied Barrett Hand

Proactive Grasp Adaptation of Disembodied Barrett Hand 7

the rather thin top of the cone. As the physi-cal gap between fingers 1 and 2 is larger thanthe upper circumference of the cone, we can seethat finger 1 fails to make full contact and thusmoves further than finger 2. As opposed to this,in the side-on grips the fingers just wrap aroundthe base of the cone, thereby only showing smallpositional variations. These differing scenariosplay themselves out clearly in the sensor data.

(a) Top-down Prismatic Precision

(b) Side-on Heavy Wrap

(c) Side-on Power Grip

Figure 4 Finger joint position in radians ver-sus time for the (a) Prismatic Precision Grasp,the (b) Heavy Wrap, and the (c) Power Grip.

We were especially interested in the outputof the tactile pressure sensors. The three Bar-rett fingers as well as the palm are provided witha tactile sensor array, consisting of 24 pressuresensors. The sensors are arranged in an 8x3 ar-ray. In the following, the sensor cell will be ref-erenced according to the enumeration given infigure 5. Note: the distal finger tip is always de-picted at the top of the map, while the bottomcells represent the proximal end of the finger tip.

Figure 5 Sensor arrays in finger tips and palmwith 24 sensors each.

For each cell the mean pressure value duringgrasping was calculated for the respective fin-ger/palm. We then compared different charac-teristics of the material to see which showed themost prominent feature in the pressure maps.Pressure values were recorded in N

cm2 . First, wecompared the object’s shape. Figure 6 showstwo objects of similar weight: an upright squarewood block (object 5) and a round water bottle(object 14 ), gripped with the Heavy Wrap.

The first observation is that for both ob-jects, cells 1 and 4 in finger 1 show signifi-cantly higher pressures than all other cells. Thiswas consistent through all measurements, grips,and objects of that specific data collection. Wetherefore conclude that something was block-ing/triggering these cells leading to faulty dataoutput. Consequently, we did not use these cellsto train the neural network.

Page 8: Object Sensitive Grasping of Disembodied Barrett Hand

Proactive Grasp Adaptation of Disembodied Barrett Hand 8

(a) Side-on Heavy Wrap grip of square wood block (object 5)

(b) Side-on Heavy Wrap grip of round water bottle (object 14)

Figure 6 Pressure maps of fingers 1-3 and palm of Barrett hand. Pressures are recorded in Ncm2 and

plotted as a mean over the grasping trial for each respective cells. The Side-on Heavy Wrap compared fora square and round object.

Page 9: Object Sensitive Grasping of Disembodied Barrett Hand

Proactive Grasp Adaptation of Disembodied Barrett Hand 9

The pressure profiles differ slightly, mainlyfor the palm and for finger 1. As the objectis being grasped from the side-on, the fingerswrap around it tightly. Accordingly, pressuresensors 7-18 are most prominent. Interestingly,the pressure is higher on the finger side than inthe middle, which is most likely due to the bulky(rather square) shape of the Barrett Hand.

Even though the pressure maps show somedistinctive features, the difference is not as strik-ing as one might expect. However, unlike thehuman finger, the Barrett finger only has twophalanges, therefore it is not able to bend at thedistal interphalangeal joint [3]. The sensor arraywill therefore only touch one side of the wood logor the bottle, respectively, making it insensitiveto the object’s shape.

Next, we compared the pressure profiles oftwo objects of the same shape, but differentweight, surface structure, and slightly differentsize (compare figure 7). The two balls (rattanball styrofoam ball) were gripped from top-downwith the prismatic precision grasp. Again, sen-sors 1 and 4 showed faulty pressure data andwere ignored. Unfortunately, sensor 3 of finger1 also began to give unreasonable high feedbackin the course of the data collection and was thusignored.

Again, the pressure maps show similar fea-tures. Most of the feedback is observed for thecells (7-18) in the middle of the finger. The pres-sure in J3 is slightly higher as finger 3 has tocompensate for the two fingers (J1 and J2) grip-ping from the opposite site.

It is interesting that the pressures while grip-ping the lighter styrofoam ball are slightly higherthan the respective pressures while grasping therattan ball. This is most likely due to the smallersize of the styrofoam ball. The fingers can con-

sequently close further around the ball and thustighten the grip. Additionally, the styrofoamball has a smooth surface so that the fingers cantightly wrap around the surface, while the rat-tan ball has a rough surface that impede a tightgrip.

The important factor of compliance becomeseven more apparent, when comparing the pres-sure maps of a soft and hard piece of foam (com-pare figure 8) for a side-on power grip. Thetwo objects, soft foam (object 2) as well as foamsquare (object4), were similar in size, shape, andweight. However, while the soft foam was verycompliant, the foam square was rather firm.

Note that again sensors 1 and 4 of finger 1give faulty feedback and were not taken into ac-count for any analysis. As the fingers approachside-on, they grab the square-shaped foam pieceslongitudinally. The power grip really closedaround the object until the foam was tightlysquished. For the firm foam the two fingers pushthe square into an angled asymmetric position,so that the palm only receives pressure on oneside, while J3 is pushing back hard with the dis-tal end of the finger tip. As opposed to this, thepressure sensors show no response to the softfoam. This was consistent for all grasps of thesoft foam and other soft objects, such as plushoctopus (object 6).

We conclude that the compliance of the ob-ject is the main factor which influences the re-sponse in the pressure sensors. The size ofthe object will influence how tightly it can begripped and thus also show its effect, though in-directly. Grasping small objects such as the cubeor the bean bag with the power grip, finger 1 willnot make contact at all, thus leaving the pressurereadout blank.

Page 10: Object Sensitive Grasping of Disembodied Barrett Hand

Proactive Grasp Adaptation of Disembodied Barrett Hand 10

(a) Top-down Prismatic Precision grip of rattan ball (object 9)

(b) Top-down Prismatic Precision grip of styrofoam ball (object 1)

Figure 7 Pressure maps of fingers 1-3 and palm of Barrett Hand. Pressures are recorded in Ncm2 and

plotted as a mean over the grasping trial for each respective cells. The Top-down Prismatic Precision gripcompared for two round objects with different weights and surface characteristics.

Page 11: Object Sensitive Grasping of Disembodied Barrett Hand

Proactive Grasp Adaptation of Disembodied Barrett Hand 11

(a) Side-on Power Grip of hard foam square (object 4)

(b) Side-on Power Grip of soft foam square (object 2)

Figure 8 Pressure maps of fingers 1-3 and palm of Barrett hand. Pressures are recorded in Ncm2 and

plotted as a mean over the grasping trial for each respective cells. The Side-on Power Grip compared fora soft and a hard square piece of foam.

Page 12: Object Sensitive Grasping of Disembodied Barrett Hand

Proactive Grasp Adaptation of Disembodied Barrett Hand 12

Neural Network

The goal of the neural network analysis was toassign labels to certain objects and train thehand to familiarize itself with the sensor re-sponse generated by each object. In this wayit should be able to recognize an object whilegrasping it, and then plan the next grasp ac-cordingly. We tried to implement this methodand improve its performance in the course of theproject. Each step listed below was taken be-cause the neural network analysis of the previousversion showed no meaningful results (i.e. whenperformed on the robot it was not able to namean object correctly).

• Initially we attempted training on raw, un-normalized sensor data. The network per-formed poorly, attaining only 16% accu-racy even on the training set itself. Wefound that normalization of the data wasabsolutely crucial for training the neuralnetwork to good accuracy.

• After normalization, the data set was splitinto training (80%) and test data (20%)randomly. For both training and valida-tion the neural network analysis reachedsuspiciously high accuracy of more than96%.

• Faulty pressure readouts were detectedand removed. We considered average readouts of over 15 N

cm2 as faulty, especiallywhen identical readings were observed overmany different objects and grasp types.

• The high accuracy on the test set and poorperformance in the real world indicated tous that we had overfit the data. To miti-gate this issue we increased the regulariza-tion weight. This led to a lower accuracy of

training and validation step in the neuralnetwork analysis and seemed to favor cer-tain objects for the respective grasp types.It did not significantly improve real-worldperformance, but did give us some insightto be discussed below.

• As the number of trials for some objectswere significantly higher than for others,we excluded these objects to provide amore balanced data set. This also seemedto afford no improvement.

Despite all our efforts to improve the neural net-work analysis, we were unable to obtain anykind of reliable performance. The fact that per-formance on the validation set nearly alwaysmatches performance on the training set shouldindicate that no overfitting occurred. However,it may be the case that the validation data wasin fact was too similar to the training data, sincethey were acquired as different time slices of thesame grasp, rather than being taken from to-tally different grasp samples. This suspicion isemphasized by the 99% accuracy of the neuralnetwork, a strong indication for overfitting thedata.

Tests on the robot confirmed this. Objectscould not be recognized correctly at all. How-ever, for each grasp type, there were two or threeselect objects which would be identified correctlya majority of the time. The system seemed toprefer these objects and named these repeatedly,independent of which object was being grasped.

Adjusting the regularizer gave a lot of in-sight into this phenomenon. We were able toexpose this behavior in the test data by usingvery heavy regularization. Running the networkwith � = 100 led to a much lower accuracy forthe neural network training—about 36%. When

Page 13: Object Sensitive Grasping of Disembodied Barrett Hand

Proactive Grasp Adaptation of Disembodied Barrett Hand 13

we examined the individual predictions for eachobject, we found that there were a few objectswhich dominated. These objects were repeatedlypredicted, thereby exhibiting 100% recall (truepositives / actual positives) but very low preci-sion (true positives / predicted positives). Theother objects therefore had 0% recall.

To summarize, the neural network analysisdid not work for the given training and valida-tion data. It appears to be too heavily biasedtoward certain objects, though we are still un-sure as to why. This behavior is usually notevident in the test set. Our conclusion is thateither there were not enough individual graspssampled, or the method does not hold for thedesired task.

IV Limitations

Even if our method had performed as well onthe robot as it did on the test set, there weremany limitations to using this approach for ob-ject recognition.

First and foremost, the method is limitedonly to objects which have been observed before.It is designed only to recognize a known object;it does not encode any higher-level characteris-tics which can then be observed in new objects.The method is also highly dependent on objectorientation and size. The neural network needsto have been trained with a grasp of an objectin a particular orientation in order to be able torecognize that object the next time it is observedin that orientation.

Nor is it invariant to the shape of the hand,since we use raw sensor data rather than ex-tracting a feature vector from local keypoints,as is typical in computer vision. Because of thisdependence on hand and finger pose, in order

to perform recognition over all the grasp typesshown in Section II, it was necessary to train aseparate neural network for each type, and usethe network which corresponds to the currentgrasp when performing predictions.

We also consider the method likely to breakdown when number of objects increases. Classi-fication gets significantly harder the more classesyou have to decide between (not to mentiontraining becomes much costlier). There is likelyto be some point at which splitting hairs betweensimilar classes becomes intractable.

Another drawback is the rather inaccuratetactile sensing. Object shapes do not necessarilyshow up in the pressure maps of the tactile sen-sor arrays. Due to the small spatial resolutionof the sensor arrays, localization of shape con-tours is coarse. In addition, the sensors gave re-peatedly erroneous feedback (see figure 9) whichmade reliability doubtful. Due to the tightnessof the grasps, contact surfaces with the fingersare often broad. These particular sensors prob-ably call for a treatment very different from theedge/corner detection of computer vision algo-rithms, and so we are also unsure about theiruse in neural network classification.

Figure 9 Faulty pressure read out exemplaryfor top-down prismatic precision grip. Suchreadout occurred repeatedly for each grasp.

Page 14: Object Sensitive Grasping of Disembodied Barrett Hand

Proactive Grasp Adaptation of Disembodied Barrett Hand 14

V Conclusion and Future Work

The neural network, as we implemented it, didnot yield valid predictions of the object beinggrasped. The question that remains is whetherthe method is not suitable for this specific setupor if the data set was not sufficiently large ordiverse. Other questions we haven’t answered:Why does the network prefer certain objects forcertain grasps? What feature influences the neu-ral response? Was there one feature that dom-inated all other sensor input? Answering thesequestions proves difficult due to the opacity ofthe neural network and the difficulty of under-standing the function of the parameters (⇥).

Possibilities for the future are numerous.Now that the system is up and running, moredata samples could be collected to have differ-ent sets for neural network training and testing.This would hopefully remove the excessive simi-larity between our current training and test sets,and allow us to analyze the performance of theneural network offline, without having to run therobot.

Another important issue would be to fix thelack of finger torque data. In addition to thecurrent setup, a setup with immobile objects(fixed to the workbench) could be further ex-plored. We have observed that if objects are al-lowed to move, the finger torque response is notsignificant until all three fingers are simultane-ously putting pressure on the object. If the ob-jects were fixed, our original idea to implementa preshaping of the hand could then possibly beperformed in a combination of initial contact andobject recognition.

In the course of the project, we becamepainfully aware of the difficulty of collectingsufficient data for statistical techniques to

teach the robot grasp experience. Robots areoften slow, and collecting recordings of theirexperiences in the real time world is timeconsuming and resource-intensive. One majorlesson from all the setbacks we went through isthat there may be more to gain from using whatis known about human motor control, ratherthan unpredictable and black-box statisticaltechniques. Until robots are in widespread use,there may not be enough variety of experiencesfor them to learn from by brute force alone.We should instead start from a known pointusing existing knowledge of human haptics andoptimize from there.

References

[1] B. Humberston and D. Pai, “InteractiveAnimation of Precision Manipulations withForce Feedback,” Draft, 2013.

[2] S. Lederman, Encyclopedia of human biol-ogy, vol. 8. 2 ed., 1997.

[3] G. Tortora and B. Derrickson, Principles ofAnatomy and Physiology. 13 ed., 2011.

[4] P. Jenmalm and R. Johansson, “Visualand Somatosensory Information about Ob-ject Shape Control Manipulative Finter-tip Forces,” The Journal of Neuroscience,vol. 17, pp. 4486–4499, 1997.

[5] M. Lee and H. Nicholls, “Tactile sensing formechatronics – a state of the art survey,”Mechatronics, vol. 9, pp. 1–31, 1999.

[6] H. Yousef, M. Boukallel, and K. Althoe-fer, “Tactile sensing for dexterous in-handmanipulation in robotics – A review,” Sen-

Page 15: Object Sensitive Grasping of Disembodied Barrett Hand

Proactive Grasp Adaptation of Disembodied Barrett Hand 15

sors and Actuators A: Physical, vol. 167,pp. 171–187, 2011.

[7] J. Steffen, R. Haschke, and H. Ritter,“Experience-based and Tactile-driven Dy-namic Grasp Control,” International Con-ference on Intelligent Robots and Systems,vol. 17, pp. 2938–2943, 2007. San Diego.

[8] P. Pastor, L. Righetti, M. Kalakrishnan,and S. Schaal, “Online Movement Adap-tation Based on Previous Sensor Experi-ences,” International Conference on Intel-ligent Robots and Systems, vol. 17, pp. 365–371, 2011. San Francisco.

[9] R. Howe, “Tactile sensing and control ofrobotic manipulation,” Advanced Robotics,vol. 8, pp. 245–261, 1993.

[10] Barrett and Technologies, BH8-Series UserManual Firmware Version 4.4.x.

[11] A. Ng, “Machine Learning ,” 2013.

[12] A. Ng, “Machine Learning ,” 2013.