virtual grasping assessment using 3d digital hand model
TRANSCRIPT
Virtual Grasping Assessment Using 3D Digital Hand Model
Yui Endo, Satoshi Kanai, Takeshi Kishinami
Graduate School of Information Science and Technology, Hokkaido University,
Japan
Natsuki Miyata, Makiko Kouchi, Masaaki Mochimaru
Digital Human Research Center, National Institute of Advanced Industrial Science and Technology,
Japan
Abstract
The purpose of this research is to develop a simulator that can evaluate stability and ease of a person grasping
handheld information appliances such as digital cameras without real subjects and physical mockups. In the
simulator, we integrate 3D digital hand models with the 3D CAD models of the appliance to realize the virtual
grasping assessment.
The simulator features are the following:
1. Geometrically accurate 3D digital hand models with rich Japanese size varieties are used for the assessment,
2. A semi-automatic grasp planning function is installed to efficiently find appropriate grasp posture for the
exterior housings geometries of appliances,
3. "Force-closure" and the "grasp quality" indices to quantitatively evaluate grasp stability for the product, and
4. Ease of grasping can be qualitatively evaluated based on "comfort database" constructed from the PCA for
measurements of finger joint angles of a real subject.
1 Introduction
Recently, as handheld information appliances (e.g. mobile phones, PDAs and digital audio players) have widely
spread to general users, the manufactures of these appliances have had to take more interest in their ergonomic
design. The ergonomic design of the information appliances mainly has to be evaluated from two aspects: physical
and cognitive. The physical aspect of the ergonomic design of these appliances includes ease of grasping the housing,
relevance of arranging positions of the physical components (buttons, switches, dials, displays lumps etc.) of the
user-interface, and the ease of their manipulations.
When manufacturers need to do the assessment from the physical aspect of the ergonomic design (hereafter referred
to “ergonomic assessment”) for the information appliances, user tests are usually done where the subjects for testing
use physical mockups of the appliance and evaluate their design. However, the user test process usually requires the
expensive cost of making physical mockups and takes a long time. These additional costs of user testing for
information appliances occupy a comparatively large part of the development costs, and the manufacturer would like
to decrease them.
On the other hand, the 3-D CAD system has widely spread to the design process of these information appliances.
And the 3-D digital mockup of the housing of the appliances can be easily obtained in the early design stage. So,
there is a strong possibility that we can execute the ergonomic assessments by integrating digital human models,
especially including digital hand models with digital mockups of the handheld information appliances to decrease
the extra time and cost of the user test.
Some simulation software using digital human models have been commercialized [1] and are being used in the
design of automobiles and airplanes. However, the digital hand models included in the digital human models of such
software do not necessarily satisfy with desired accuracy and size variation of human hands when operating the
handheld appliances. Moreover, the software does not have the functions of grasping the digital mockups of the
appliance or of evaluating the ease of grasping the housing and one of the manipulations of the user-interface by the
hand.
Therefore, our research purpose is to develop an automatic ergonomic assessment system for designing handheld
information appliances by integrating the digital hand model with the 3-dimesional product model of the appliance,
as shown in Fig.1.
As shown in Fig.2, in our system, we realize the following feature functions for ergonomic assessment to satisfy our
purpose:
1. Generation of kinematically and geometrically accurate 3-dimensional digital hand models with rich
dimensional variation: We apply a digital hand model called “Dhaiba-Hand” to the ergonomic assessment in
our system. The Dhaiba-Hand was developed by DHRC, AIST, Japan. The Dhaiba-Hand has a precise hand
link structure model which is derived from the kinematic analysis for the measured data obtained from
motion capture and MRI[2,3]. 3-D digital hand models with dimensional variation can be generated by
deforming a generic hand model which has been created by the factor analysis of dimensional
measurements taken from 103 Japanese subjects’ hands [4].
2. Automatic generation and evaluation of the grasp posture: By only a few user-interactions, our system
automatically finds one of the possible grasp postures determined by the product shape and the digital hand
Figure 1. An example of ergonomic assessment by integrating the digital hand model with the product model of
the information appliance
geometry. It also quantitatively evaluates two indices of the grasp: i) grasp stability for the product
geometry and ii) ease of grasping from aspect of finger joint angle configuration. The former i) is calculated
based on the force-closure and the grasp quality used in the grasp planning of robotics. The latter ii) is done
based on “ease of grasping (EOG)” evaluation map which has been constructed from principal component
analysis for finger joint angles in real subjects’ grasp.
3. Automatic evaluation of ease of the finger motions in operating the user interface: The system automatically
moves fingers of the digital hand by following an operation task model of the user-interface (e.g. which
button has to be pushed). It also automatically evaluates ease of finger motions during operation of the user
interface based on the flexion joint angles of fingers.
4. Aiding the designers to redesign the housing shapes and to reallocate the spatial arrangement of the physical
components of the user-interfaces (e.g. buttons, dials, displays) in the digital mockup: By the above two
evaluation indices, the system conducts sensitivity analysis for the redesign of the housing shape. This
analysis finds an optimal combination of some dimensional parameters of the housing shapes so as to
maximize the evaluation indices for the grasp posture.
This automatic ergonomic assessment system is being developed as a part of our government-funded project called
“Sapporo IT Carrozzeria” [5]. Thus far, the above functions 1 and 2 as shown in Fig.2 of our system have been
developed, and are described in this paper.
Figure 2. The overview of our proposed automatic ergonomic assessment system
Digital Hand
Model
Generation of the
Dimensional Variation
of Digital Hand Model
3D Mesh Model UI Operation Task Model
Generation of Grasp Posture
Evaluation of Ease of
UI Operation
Grasp
Posture
Ergonomics Assessment
Finger
Motion
Path
Grasp Stability
Ease of Grasping
Ease of
Operation
Product Model for Information Appliance
Evaluation of Grasp Posture
Aid of
Redesign
(I)
(II)
(III)
(IV)
We also describe the verification results of our system by comparing the estimated grasp postures given from the
system with the ones from experiments, and also by comparing the grasp stability for the products evaluated by our
system with the ones felt by real subjects.
2 RELATED WORKS
In robotics, Miller [6] proposed the well-functioning grasp planning system called “GraspIt!” for a robotic hand.
This system can import a wide variety of robotic hands and product models, and can evaluate the grasp posture
formed by these hands based on the force-closure and the grasp quality. Recently, he proposed to extend the hand
models in this system to the human hand [7]. However, the final goal of this research aims to better understand the
important features for robotic hand design, and does not necessarily aim to discover ergonomic design or provide
assessment of the housing and the user-interface of the information appliances. Moreover, the method to generate
the rich dimensional variation of the digital hand models was not discussed in this research. Miyata [8] developed an
algorithm which can generate realistic grasp postures for cylinders with arbitrary diameter based on their hand link
posture estimator [2,3] from the motion capture data.
On the other hand, much research has been done in applying the digital hand to the virtual reality (VR) environment
[9]. The main purpose of this VR research is to construct a more intuitive and natural interaction environment where
the user can directly operate the virtual object using the digital hand. The grasp motion of the digital hand is
manually instructed through the dataglove, and the geometry of the hand model is usually simplified so as to achieve
real-time interaction. Therefore, in the VR system, we cannot evaluate the grasp stability for the products
corresponding to arbitrary human hands, and cannot apply it to the ergonomic design of the product.
In the area of computer animation, ElKoura proposed a simulator which can generate the motion of the fretting hand
playing a given musical passage on a guitar [10]. This system, called “Handrix”, is aimed to create the precise
animation of guitar fingering for music education and analysis. Kyota [11] also proposed a system to automatically
generate the grasping posture for more general objects. However, the function of the former [10] was only limited to
the guitar-playing motion, while the latter [11] generated too many possible grasping posture candidates to use as a
posture for ergonomic assessment of information appliances design. The precision of the hand model of the latter
[11] was inadequate for ergonomic assessment.
Therefore, the previous related works on the digital hand were not necessarily applicable to the ergonomic
assessment of handheld information appliances.
3 DIGITAL HAND MODEL
3.1 Overview
To perform effective digital ergonomic assessment, it is insufficient to use only one digital hand model with a fixed
dimension because the physical dimensions of appliance users differ from person to person. Therefore, we needed to
generate a digital hand model with possible anthropometric variation. In this purpose, we used a digital hand model
based on the Dhaiba-Hand [4]. The digital hand model used in our system consists of the following four parts:
1. Link structure model: A link structure model approximates to the rotational motion of bones in the hand.
The model was constructed from the measurement by MRI and the motion capture. The model has 17 links,
and each link has two joints at both its ends. These joints can rotate with 1, 3 or 6 degrees of freedom, as
shown in Fig.3(a).
2. Surface skin model: A surface skin model is a 3-dimensional polygonal mesh for the hand surface generated
from CT images. The geometry of the skin model is defined at only one opened posture.
3. Surface skin deformation algorithm: This algorithm defines the deformed geometry of the surface skin
model when the posture of the link structure model is changed.
4. Natural grasping motion generator: The natural grasping motion generator is a function to automatically and
naturally generate a grasping motion path of the hand model from a fully opened state to a clenched one.
This grasping motion reflects the joint angle constraints of the link structure model.
A link structure and a surface skin model are generated by inputting the 82 dimensional parameters of a specified
subject’s hand into the generic hand model which are implemented in the Dhaiba-Hand [4]. On the other hand, a
surface skin deformation algorithm and a natural grasping motion generator were originally developed by us.
Therefore, in the following sections, we describe the latter two functions.
(a)
(b)
Figure 3. The digital hand model of our system. (a) The link structure model, (b) The flow of the deformation of
the surface skin model
IP
MP
CM
DIP
PIP
MPThumb
Index
MiddleRing
Pinky
z
xy
Wrist
Forearm
6 DOF Joint
3 DOF Joint
1 DOF Joint
1f
2f3f
4f
5f
Link
Link Structure
ModelGeneration
Surface Skin
Model Deformation
Surface skin model
in initial hand posture
Link posture
Lengths of links
Forward
kinematics
Mesh deformation algorithm
Deformed
surface skin model
Natural Grasping
Motion Sequencer
Joint rotation angles
Sampled
joint rotation angles
Inverse
Kinematics
Goal point
Joint rotation angles
CCD method
Initial link posture
3.2 Surface Skin Deformation Algorithm
The Dhaiba-Hand can generate a link structure model and surface skin model for the opened hand posture of an
arbitrary subject. However, it does not have a function to deform the surface skin model connected with the joint
rotation of the link structure model. Therefore, we propose a surface skin deformation algorithm where the geometry
of the surface skin is deformed along with the change of the joint angles in the link structure.
For the deformation algorithm, we propose a following deformation method developed by ourselves by taking a hint
from a function of “Poser” [12]:
First, we define j
W T as a homogeneous transformation matrix which represents the posture of a local coordinate
system j of link j of the link structure model with respect to the world coordinate system W . From the
simple geometric relation of the transformation, we obtain
j
jW
j
W TTTT 1
2
1
1
(1)
)()()( 00
1 jjj
j
j T rRPYrRPYtTrans (2)
where j is fixed to the root side (wrist side) of the end of the link j , and links 1j and 1j are the parent
and child of the link j , respectively. j
0t and j
0r are vectors representing relative translation and rotation (roll,
pitch and yaw angles) from 1 j to j at the initial posture of the hand model where all finger joints do not rotate.
We simply describe ),,( zyx tttTrans as )(tTrans , and ),(),(),( xyz rxryrz RotRotRot as )(rRPY .
If vj denotes a position vector of a vertex v in the surface skin model with respect to j , then vW , the
position of a vertex v after the deformation caused by the joint rotation in the link structure model, is represented
as follows:
vrRPY
rRPYrRPYtTransv
j
jk
k
jk
k
j
jjj
j
WW
TT
T
)child(
1
001
)()(
)()()(
(3)
],,[ j
z
j
vz
j
y
j
vy
j
x
j
vx
j rwrwrwr (4)
where ]1,0[j
vzw is a weight value, and indicates how the z-axis-rotation angle of the joint j contributes to the
z-axis-rotation angle applied to the vertex v . The weight j
vzw is calculated by the following two steps, as shown in
(a) (b)
Figure 4. The method to define the weight value of each vertex in surface skin deformation. (a)Step 1, (b)Step 2
x
Mesh of
Surface skin
Joint j
0Vertex v
yD
AB
C
vz
Vertices with
respect to joint j
j
xVertex v
y
Internal ellipsoid
External
ellipsoid
j
Joint j
Fig.4 (x- and y-axis-rotation are also done in the same manner):
STEP 1 For each vertex on the surface skin model, we manually specify which link the vertex belongs to in
advance.
STEP 2 For each vertex v , which belongs to the link j or 1j , we calculate the angle vz between the
vector ]0,,[ y
j
x
j vv and x-axis vector of j . If we specify the four constants DCBA ,,, to define four angle
domains of vz around the joint j , then the weight value j
vzw )1( is defined according to the following equations
as shown in Fig.4:
]),[(
]),[(1
]),0[),2,[(0
]),[(1
)1(
CDvz
DC
Dvz
ABvz
BA
Bvz
DvzAvz
BCvz
j
vzw
(5)
We set DCBA ,,, as
DCBA . These four constants are empirically set so that the domain between
C and B includes most vertices of the link j , the one between
A and D includes most vertices of the
parent link 1j , the one between B and
A includes the compress side and the one between D and
C
includes the tensile side of the finger skin.
STEP 3 We define two ellipsoids fixed to each link j ; the internal ellipsoid and the external ellipsoid as shown
in Fig.4(b). If the vertex v lies in the region inside the internal ellipsoid, then the deformation of v is defined
only by the rotation of the joint j , so we set j
vzw )2( to 1. Otherwise, if the v lies in the region outside the external
ellipsoid, then the deformation of v is not affected by the rotation of the joint j , so we set j
vzw )2( to 0. If the v
lies between the inside of the external ellipsoid and the outside of the internal one, we set j
vzw )2( to a linearly
interpolated value between weights of the two vertices, each of which is the nearest set of points on the internal and
external ellipsoids.
STEP 4 – Finally, we set j
vzw as a product of these two weights:
j
vz
j
vz
j
vz www )2()1( (6)
3.3 Natural Grasping Motion Generator
Lee [13] described the following properties of kinematic constraints for the each finger motion of the human hand:
1. Maximum and minimum angles exist for the rotation of the each finger joint.
2. The rotation angle of the DIP joint relates to one PIP joint in each finger if (except thumb) as
)()( )3/2( i
PIPx
i
DIPx .
3. The lines connecting the position of DIP and PIP joints of each finger (not including thumb) in the clenched
posture converge at a certain point.
To apply these properties to the digital hand model, we use a function called natural grasping motion generator in
our system. This generator automatically rotates the joints of the fingers according to the natural grasping motion
stored in the system, and makes each finger posture from opened to clenched naturally. The natural grasping motion
is obtained by experimentally sampling the joint rotation angles during the actual grasping motion of a human hand.
This generator facilitates the process of estimating the grasp posture of the digital hand.
4 GENERATION OF THE GRASP POSTURE
In this section, we describe how to generate the grasp posture of the digital hand model for the product shape model
in our system. The surfaces of both models are represented by 3-dimensional polygonal meshes in the system.
The overview of the process of grasp posture generation is shown in Fig.5. The process consists of four steps: 1)
selection of the contact point candidates, 2) generation of the rough grasp posture, 3) optional correction of the
contact points and 4) maximization of the number of contact points.
4.1 Selection of Contact Point Candidates
1. As shown in Fig.6, we assume that each finger 52 ,..., ff can be classified into two types when grasping an
object: active finger Activei Ff and passive finger
Passivei Ff . The active finger is the one which tries to
grasp an object actively, while the passive finger is the one which moves passively followed by the motion
of the active finger. In this system, we specify one active finger and three passive fingers. If the part of the
product surface geometry have been originally designed so as to be contacted with the tip of a certain finger
(eg. the shape of the ergonomic mouse), this finger is selected as the active finger.
2. The user of the system interactively chooses a pair of vertices: a vertex Hvpalm on the palm surface of the
digital hand model, and a vertex Pvpalm on the surface of the product model whose position will be identical
to the one of the vertex Hvpalm at the grasp posture.
3. The user of the system interactively chooses another pair of vertices: a vertex Hvactive on the surface of an
active finger, and a vertex Pvactive on the surface of the product model whose position will be identical to the
one of the vertex Hvactive at the grasp posture.
4.2 Contact Detection of the Digital Hand with the Product Model
For generating grasp postures automatically in the system, we need to check whether the surface skin model of the
digital hand contact with the surface of the product model. Actually the surface of a hand is deformed when
contacting the product surface. Generally, deformation of human organs has non-linearly elastic property, and
Figure 5. The algorithm to generate the grasp posture
Final
Grasp Posture
Rough
Grasp Posture
Selecting contact point candidates
Generating rough grasp posture/ Hand transformation and
Natural Grasping Motion
Initial State
Product
Model
Digital Hand Model
Maximizing the num. of
contact points/ Perturbing joint rot. angle
Correcting finger
posture Inputting goal point
/ Inverse Kinematics
simulating the deformation of such non-linear material using FEA relatively needs a large execution time. However,
this is unacceptable to our system where a large number of different grasp postures have to be generated and tested
within a practical time.
Therefore, we substitute simple geometric collision detection for accurately simulating the contact deformation of
the hand. The collision detection classifies each vertex in the surface skin model into the following three states:
1. If a vertex is outside of the surface of the product model, then the vertex is marked as NOT COLLIDE state.
2. If a vertex is inside of the surface of the product model and the distance dv between the vertex and the
product surface along the vertex normal nv is less than the user-defined tolerance as shown in Fig.7,
then the vertex is marked as CONTACT state
3. If a vertex is neither in NOT COLLIDE or CONTACT state, the vertex is marked as COLLIDE state.
If some vertices of the digital hand are in CONTACT state and the others are in NOT COLLIDE state, the system
considers that the hand contacts the product model. If at least one vertex is in COLLIDE state, the system does that
the hand collides the product model. Otherwise, the system does that the hand does not collide the product model. In
all grasp posture, the digital hand must be in CONTACT state.
4.3 Generation of the Rough Grasp Posture
1. The system translates a position of the digital hand model so that the position H
palmv of the vertex Hvpalm is
identical to the position P
palmv of the vertex Pvpalm .
2. For each time step of the natural grasping motion of the active finger Activei Ff , the system minimizes the
distance between H
activev and P
activev , by rotating the hand model around the three axes, passing the point H
palmv . During this minimization, the other fingers remain in the initial opened postures.
Figure 6. The selection of the contact point candidates
3. Starting from the hand posture obtained from the above procedure 2, the system maximizes the number of
the contact points between the digital hand and the product model by rotating the digital hand model around
the axis which connects P
palmv to P
activev .
4. Finally, the system closes each passive finger Passivei Ff and thumb along with the natural grasping
motion until the surface of the digital hand model is in contact with that of the product model.
After these procedures, the system returns one of the following grasp postures of the digital hand model; a) grasp
succeeded, finger could reach to P
activev and its rough grasp posture was obtained, b) grasp failed, finger could not
reach to P
activev , and c) grasp failed, finger could reach to P
activev but the hand model still collided the product
model.
4.4 Optional Correction of the Finger Posture
The rough grasp posture of the digital hand model found by the previous procedure sometimes includes the fingers
with inappropriate postures which are quite different from the desired natural postures. In such cases, the user of the
system can optionally correct these inappropriate postures of the fingers by additionally selecting a goal position to
which the fingertip should move on the surface of product model. The corrected finger posture for the goal position
is found by solving the inverse kinematics using the CCD (Cyclic-Coordinate Descent) method [14].
4.5 Maximizing the Number of the Contact Points
For grasping an object stably, a human hand usually grasps it in order to let the contact area be the maximum.
However, the rough grasp posture found by the previous procedures sometimes does not have such a property. In
that case, our system maximizes the number of contact points by perturbing the joint rotation angles of the each
finger. We perturbed the rotation angles by rotating 16 times with 0.06 [deg] increments for each axis.
5 Evaluation of Grasp Stability
Using the procedures described in the previous sections, our system estimates one of the possible grasp postures
based on the digital hand model and the product model. As the next process, the system automatically evaluates
the grasp stability for the product in this estimated grasp posture. We introduce the force-closure and the grasp
quality into the evaluation of the grasp stability. These two concepts were originally proposed for successful grasp
planning using robotic hands [15].
Figure 7. The method of the collision and contact detection between the surface skin of the digital hand and the
surface of the product model
vnvd
Surface Skin of Digital Hand Model
Surface of
Product Model
v
In our system, the force-closure condition indicates whether the digital hand can grasp the product model stably at
the contact points between the digital hand and the product model. Moreover, if an estimated grasp posture can
satisfy the force-closure, grasp quality can express how stably we can hold the product at this posture. These two
concepts in our system are described in the following sections.
5.1 Force-closure
Originally, in the definition of robotic grasping, a grasp is said to be “force-closure” if it is possible to apply forces
and moments at the contact points such that any external force and moments acting on a grasped object can be
balanced [15]. We adopt this force-closure as a condition for deciding the grasp stability found by the system.
As shown in Fig.8, suppose there are N contact points between the surface of fingers (or a palm) in a digital hand
model and a product model, these contacts are “rigid contact” and their friction follows the Coulomb model with
friction coefficient . We selected 1 , which is the midrange of the friction coefficient between human skin
and an object.
Figure 8. A contact point between the surface skin of digital hand and the surface of the product model
Figure 9. The definition of the grasp quality
Contact point
Surface skin of
digital hand model
Mesh surface of
Product model
Edge vector
Edge
Friction
Pyramid
l)(cd l
c
Contact force vector
0
halfspace
Convex hull of )(cwl
)(cwl
h
Grasp quality
Wrench space
The contact force vector acting at a contact point c must lie within its friction cone so that the finger would not slip
at contact c . The half angle of the cone is 1tan, and we approximate the cone with an L -sided pyramid
(friction pyramid). From this geometric relation between the friction cone and the contact force vector at c , the
contact forces can be represented as a convex combination of L edge vectors of the friction pyramid.
Then a wrench 6)( cw , which represents a combination of the force and the torque acting at contact c , can be
expressed as follows:
L
l
ll ccc1
)()()( ww (7)
)()(
)()(
cc
cc
l
l
ldp
dw (8)
where )(clw is the wrench for an l -th edge of the friction pyramid at c , )(cld is the l -th unit edge vector of
the pyramid, )(cp is the position vector of the contact point at c with respect to a coordinate system fixed at
object gravity center, and )(cl is a nonnegative coefficient.
The force-closure is defined as “the state to resist any external wrench by applying positive forces at the contacts”
[16]. Based on equations (7) and (8), the condition of the force-closure can be expressed as follows:
N
c
L
l
l c1 1
)(ConvexHullInterior w0 (9)
where )(ConvexHull and )(Interior are the convex hull of a set of points and an inner space of a 3D object
respectively, and 0 indicates the coordinate for an origin of the wrench space.
Therefore, from the coordinates of contact points given by the system, we can determine that a grasp posture found
in the system is “stable” when eq.9 holds.
5.2 Grasp Quality
If a grasp posture satisfies the force-closure, then the system must evaluate to what extent this grasp is good. The
grasp quality is used in our system as a quality measure for this purpose.
A grasp quality is defined as “the reciprocal of the sum of magnitudes of contact normal forces required to achieve
the worst case wrench” [16]. As shown in Fig.9, a grasp quality can be calculated as the minimum of distances
between the wrench space origin and each hyperplane Hh constructing the convex hull of eq.9:
),(distmin hHh
0
(10)
where ),(dist ba is the distance between a point a and a plane b .
Figure 10. The result of the grasp posture for a pen-type mouse
(a) Cell-phone
(b) Portable audio player
(c) PDA
Figure 11. The results of generating the grasp posture of a variety of information appliances. All posture satisfy
the force-closure, and the grasp quality is (a) 0.35, (b) 0.51, (c) 0.52. The processing times are (a) 30s, (b) 33s,
(c) 40s. The number of faces of the product surface mesh is (a) 2546, (b) 2170, (c) 2696.
buttons
force-closure hold force-closure doesn’t hold
6 Results on Evaluation and Verification of the Grasp Posture Generation and Grasp Stability
6.1 Evaluation
Figure 10 shows a result of a grasp posture for the product model of a pen-type mouse. This mouse was newly
designed and prototyped by another team in our project [5]. The 3D mesh model of the mouse housing was made by
tessellating a 3D–CAD model. The digital hand geometry was set to the representative Japanese hand [4].
This grasp posture for the pen-type mouse in case of the normal operating situation is shown in Fig 10 (Left). It
satisfied the force-closure. However, in the mouse, a user must release a thumb from the housing surface to operate
the buttons. So we re-evaluated the grasp stability in this released situation, and the grasp posture resulting did not
satisfy the force-closure any longer. From this evaluation, it became clear that this mouse would not be able to be
grasped stably when manipulating the buttons and the position of them has to be redesigned.
Figure 11 shows the other results of grasp postures for a variety of handheld information appliances, each of which
has a similar parallelepiped housing shape with different dimensions. As shown in Fig.11(a)-(c), it was confirmed
that our system could flexibly generate appropriate grasp postures even in case where the dimensions of the product
model differed.
6.2 Verification
We verified the results of the grasp posture and the grasp stability evaluation obtained from our system from the
following two points of view:
1. How equally the digital hand in our system can imitate the real grasp posture in which a human grasps a
physical product, and
2. To what extent a correlation is between subjective evaluations of the grasp stability by real subjects and the
grasp quality indices obtained from our system using the digital hand model.
If these two verification results are satisfactory, then we can conclude that our proposed system is able to estimate
whether the product housing geometry is easy to grasp in the grasp posture assumed by a designer. We describe the
results of the two experiments of the verifications in the following section.
Figure 12 shows the verification results of the grasp posture. We generated a digital hand model for a certain subject
(24 old, a Japanese male), and prepared a cylinder (60mm diameter and 205mm height) and a product model for it as
test models.
Figure 12(a) shows the real grasp posture generated by the subject and the posture generated by our system,
respectively. Finger positions of the real posture were roughly identical to those generated from the system. Figure
12(b) shows contact area maps for these two postures and a superimposed map.
From this experiment, though there are several small and unmatched contact areas between the real (blue) and
digital hand (red) postures at palm and fingertips, the contact areas of the digital hand model could approximate that
of the real subject’s hand at grasping state. The result from the pressure distribution map also supported this
conclusion.
Figure 13 shows the verification result of the grasp stability evaluation. The subjects were 8 Japanese males (ages 22
to 28). The products were a set of 4 cylinders (diameter: 40mm, 60mm, 100mm, 165mm) and 4 square-section
prisms (width: 40mm, 60mm, 75mm, 100mm). The subjects were instructed to put one side of the prism or the
cylinder on the table and hold another side by their no-dominant hand. They were asked to respectively answer their
feeling of stability in their grasp posture for these 8 products on a 4-level subjective evaluation index. The index
classifies the stability as, 0: unable to grasp, 1:a bit hard to grasp, 2: able to grasp commonly, 3:very easy to grasp
firmly. The system also evaluated the grasp stability as grasp quality indices for them. For the cylinders over 60 mm
diameter, the scatter diagram in Fig.13 showed that there was, to some extent, a positive correlation between two
indices. However, under the 40mm, this correlation does not hold, because some subjects felt it easier to grasp the
cylinder with a larger diameter than with a 40mm diameter.
From this experiment, at this point, the grasp quality evaluated by the digital hand model in our system is effective
to roughly indicate averaged subjective feelings for the stability of hand grasp for products with simple geometries
(eg. cylinders or square prisms).
(a) (b)
Figure 12. The verification result of the validity of the grasp posture for a cylinder. (b) Red points show the
contact points between the surface skin of the digital hand model and the surface of the product model. Blue
points show the contact region between the real ones. We mapped these data as distributions for the argument
and the height of the cylinder
Figure 13. The verification results of the validity of the stability evaluation for the grasp posture for the
cylinders (circular points) and square prisms (rectangle points). The graph shows the correlation between the
evaluation by the real subjects (y-axis) and one by our system (x-axis). The values in the parenthesis show the
size of the cylinders or prisms.
0
205
Heig
ht
[mm
]Argument [rad]
2π0
Palm Print
Contact Pts 24yo
male
Real Subject
Digital Hand
0
1
2
3
0 0.2 0.4 0.6 0.8 1
Su
bje
cti
ve
Eva
lua
tio
n
Grasp Quality
(40)(60)
(100)
(165)
(□40)
(□60)
(□75)
(□100)
7 Evaluation and Verification of the Ease of Grasping
When a human is grasping an object, the finger joint angles of the hand affect his/her feeling of ease of grasping the
object. In this section, we describe the method of how to evaluate the ease of grasping the object, which is the
second evaluation index of the grasp.
7.1 Overview of the Evaluation Method for the Ease of Grasping
Figure 14 shows an overview of our evaluation method for the ease of grasping an object. As preprocess of the
method, we make an “EOG (ease of grasping) evaluation map” defined in an M-dimensional space, where a large
number of actual hand postures from the opened state to the grasping state are plotted. This plot is generated by the
following process:
1. For a subject to grasp and an object to be grasped, a sequence of finger joint angles of the subject’s hand
from the opened state to the grasping state are measured.
2. A subject is asked to hold a set of objects including primitive shapes and real daily products.
3. A number of subjects are required to carry out the above experimental process of 1 and 2.
Figure 14. The overview of our evaluation method of the ease of grasping
Dataglove
Real Subjects Grasp Postures of each Subjectfor each Object
EOG Evaluation Map
Generate Virtual Grasp Posture
Generate EOG Evaluation Map
Principal Component
Product Model
Digital Hand
Ease of Grasping
Evaluate Grasp Stability
Evaluate “Ease of Grasping”
Grasp Stability
Grasp Posturefor the Product Model
#Principal Component
Principal Component Analysis
Measure Real Grasp Posture
Subjective Evaluation of “Ease of Grasping”
Eigenvector
Force-ClosureGrasp Quality
]),1[( Ppxp
]),1[( Ppxnp
Sample Objects
],1[ Nn
M
pc
]),1[( Mmznm
Calibration ofDigital Hand→DataGlove
CalibratedGrasp Posture
Posture Data measured by Dataglove
Preprocess
4. All recorded sets of finger joint angles are processed by PCA and the results are plotted as points on the
EOG evaluation map.
In this map, one posture example is indicated as a set of scores of a principal component analysis (PCA) for finger
joint angles of some “real” human hands, measured by a dataglove. We can estimate the ease of grasping which is
generated from a product model and a digital hand by plotting the principal component score for this posture on the
map.
7.2 Principal Component Analysis for Finger Joint Angles of Hand Postures in Grasp Motion
Let’s define a set of P joint angles of a hand posture as },...,2,1|{ Ppxp . P indicates the degree of freedom of the
hand model and is generally defined as more than 30. However, it has been known that finger joint angles are
strongly interrelated with each other during the grasp motion [17]. Therefore, we reduce the degree of freedom of
the hand posture to less than P by PCA.
Suppose joint angles of the hand postures in grasping for N objects are recorded. We define a set of the standardized
measurements of these angles as },...,2,1;,...,2,1|{ PpNnxnp , and define a N x P matrix as ][ npxX
]),1[];,1[;( PpNnxnp . Then we obtain a variance-covariance matrix V and the matrix ][ ijwW
}),...,2,1{,;( Pjiwij as
XXN
V T
1
1
, (11)
] [][ 21 PijwW ccc , (12)
where Pccc ,...,, 21 are eigenvectors of the matrix V which are sorted in decreasing order of eigenvalue.
Suppose the P finger joint angles can be approximated by M ( PM ) principal components, then we approximately
describe the hand posture as the first M principal component scores ),...,( 1 nMn zz as follows:
(a) (b)
Figure 15. Five groups classified by the hand dimension
1
2
3
4
5
P
p
nppmnm NnMmxwz1
),...,2,1,,...,2,1( . (13)
7.3 “Ease of Grasping (EOG)” Evaluation Map
7.3.1 Method for Generating the EOG Evaluation Map
By using the above eq.13, we can evaluate the sequence of the principal component scores nmz of the hand
postures from the opened to grasping states for each sample object with “real” subjects. By plotting the first M
principal component scores of these postures on a M-dimensional space, we can generate the “ease of grasping
(EOG)” evaluation map. At the same time, we also record the two-level subjective evaluation for the grasp postures
(i.e. “easy / not easy to grasp”). By partitioning the EOG evaluation map into the equally spaced M-dimensional
voxel, each voxel can be classified into three categories, which are:
1. EASY TO GRASP –– a voxel which includes grasp postures whose subjective evaluations are “easy to
grasp”
2. ABLE TO GRASP –– a voxel which includes hand postures during the grasping sequence but does not
include final grasp postures
3. CANNOT GRASP –– a voxel which does not include any hand postures
7.3.2 Generation of the Multiple EOG Evaluation Maps Classified by the Hand Dimension
There is a possibility that a large variance in the hand sizes of the subjects causes different results in the EOG
evaluation. To consider this difference, we classify subjects into five groups with respect to their hand dimension
and generate five different EOG evaluation maps for every group. In the experiment for the preprocess, we prepare
nine sheets of the paper where the nine representative Japanese hands shown in Figure 15(b) are printed. By putting
the subject’s hand on these sheets, each subject chooses one representative hand that has the most equivalent hand
dimension. Based on the class of their chosen representative hand, the dimensions of subject’s hand are classified
into five groups, as shown in Figure 15(a).
7.3.3 Results of Generating the EOG Evaluation Maps
Figure 16 shows one of the EOG evaluation maps generated by the above method (the 5th
hand group in Figure
15(a)). The blue/red circular points show the grasp postures that have the subjective evaluation of “easy to grasp/not
Figure 16. The “ease of grasping” evaluation map
1st Principal
Component
Score
2nd Principal
Component Score
Easy to Grasp
Able to Grasp
Cannot Grasp
3rd PCs
1st PCs
φ100
φ60
φ45
φ165φ100
φ60φ45
φ165
(Cylinder)
easy to grasp”. The small gray points show the hand postures in grasping sequence. The EOG evaluation maps are
generated by 8 subjects and 70 sample objects, as shown in Figure 17. The hand postures were measured by the
“CyberGlove”, which has 19 sensors. We determined that the number M of the principal components should be three,
because the cumulative proportion of the first three principal components was more than 80% at every EOG
evaluation map of each hand group.
7.4 The Evaluation and Verification for Ease of Grasping
In the evaluation process in Figure 14, the virtual grasp posture of the digital hand is generated by the method of
section 4, then the principal component scores from eq.13 are calculated and plotted on the EOG evaluation map,
and the ease of grasping from the category of the voxel that includes the posture is evaluated.
We verified the ease of grasping evaluation by plotting some grasp postures of the digital hand on the EOG
evaluation map, as shown in Figure 18. These grasp postures are generated from the digital hand and the product
models which have the same geometry as theose used in generating the EOG evaluation map. The blue/red rectangle
points show the principal component scores of the grasp postures that were estimated as “stable/unstable” grasp
posture by force-closure described in section 5. The two-point pair connected with an arrow shows that these grasp
Figure 17. Sample objects used in the test for generating the EOG evaluation maps
Figure 18. Evaluation and verification results of the ease of grasping
1st Principal
Component
Score
2nd Principal
Component Score
Easy to Grasp
Able to Grasp
Cannot Grasp
3rd PCs
1st PCs
Initial Posture of
Digital Hand
Initial Posture of
Digital Handφ100
φ60
φ45
φ165
φ100
φ60
φ45
φ165
(Cylinder)
postures are for the same object. The “ease of grasping” evaluation results by taking a digital hand and a real
subject’s hand for the same object were fell into the same category for almost all cases. Therefore, the proposed
EOG evaluation map-based method is effective for evaluating the ease of grasping a product model with a digital
hand.
8 CONCLUSIONS
The conclusions of our research summarizes as follows:
1. We proposed a system of automatic ergonomic assessment for handheld information appliances by
integrating the digital hand model with the 3-dimensional product model of the appliances.
2. As a method of generating kinematically and geometrically accurate 3-dimensional digital hand models
with rich dimensional variation, we proposed a digital hand model consisting of a variational link structure
model, variational surface skin model, and a newly proposed deformation algorithm of the surface skin.
3. We proposed a method of semi-automatically generating the grasp posture of the digital hand for the
product model using a natural grasping motion generator, inverse kinematics and maximization of the
contact points. The force-closure and the grasp quality were proposed to automatically evaluate the grasp
stability. We also proposed an evaluation method for ease of grasping the product based on a grasp posture
database of “real” subjects.
4. We verified the grasp posture and the grasp stability obtained from our system by comparing them to those
of real subjects. The results of the verification showed that the grasp posture and the grasp stability
evaluation of our system have a positive correlation to those of real subjects to some extent.
In our future research, we will develop a new function to evaluate the ease of finger operation for the use-interface
of the products, and to aid the designers in redesigning the housing shapes and the user-interfaces in the product
model.
ACKNOWLEDGEMENTS
This work was financially supported by a grant-in-aid of Intelligent Cluster Project (Sapporo IT Carrozzeria) funded
by MEXT.
REFERENCES
[1] For example, Jack, http://www.ugs.com
[2] N.Miyata et al., “Modeling of Human Hand Link Structure from Optical Motion Capture Data,”
IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2129-2135, 2004.
[3] N.Miyata et al., “Method to Derive Hand Link Model with MRI images,” Proceedings of the 21st Annual Conference of the Robotics Society of Japan, 1A11, 2003.
[4] M.Kouchi et al.,“An Analysis of Hand Measurements for Obtaining Representative Japanese
Hand Models,” Proceedings of the 8th Annual Digital Human Modeling for Design and Engineering Symposium, 2005-01-2734, 2005.
[5] Sapporo IT Carozzeria,
http://www.it-cluster.jp/english/
[6] A.Miller et al., “GraspIt!: A Versatile Simulator for Robotic Grasping,” IEEE Robotics and Automation Magazine, 110-122, 2004.
[7] A.Miller et al., “From Robot Hands to Human Hands: A Visualization and Simulation Engine for
Grasping Research,” Industrial Robot, vol.32, no.1, 55-63, 2005.
[8] N.Miyata et al., “Hand Link Modeling and Motion Generation from Motion Capture Data based
on 3D Joint Kinematics,” Proceedings of the 2005 Digital Human Modeling for Design and
Engineering Symposium (CD-ROM), 2005-01-2687, 2005.
[9] H.Wan et al., “Realistic Virtual Hand Modeling with Applications for Virtual Grasping,”
Proceedings of the 2004 ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry, 81-87, 2004.
[10] G.ElKoura et al., “Handrix: Animating the Human Hand”, Proceedings of the 2003 ACM SIGGRAPH / Eurographics Symposium on Computer Animation, 110-119, 2003.
[11] F.Kyota et al., “Detection and Evaluation of Grasping Positions,” Proceedings of the 2005 ACM SIGGRAPH, sketches, 2005.
[12] Poser, http://www.e-frontier.com/
[13] J.Lee et al., “Model-Based Analysis of Hand Posture,” IEEE Computer Graphics and Applications,
vol.15, no.5, 77-86, 1995.
[14] C.Welman, “Inverse Kinematics and Geometric Constraints for Articulated Figure Manipulation,”
Simon Frasier University Master's Thesis, 1993.
[15] V.Nguyen, “Constructing force-closure grasps,” The International Journal of Robotics Research,
vol.7, no.3, 3-16, 1988.
[16] N.S.Pollard, “Closure and Quality Equivalence for Efficient Synthesis of Grasps from Examples,”
The International Journal of Robotics Research, vol.23, no.6, 595-613, 2004.
[17] M. Santello, et al., “Postural Hand Synergies for Tool Use,” The Journal of Neuroscience, vol.18,
no.23, 10105-10115, 1998.