presentación de powerpoint - ujisanzp/irs/autrobmanipula-2011-2.pdf · [email protected] pedro j sanz...

5/9/2011

1

Autonomous Robotic Manipulation (2/4)

[email protected]

Pedro J Sanz

May 2011 Fundamentals of Robotics (UdG) 2

OVERVIEW

mailto:[email protected]

5/9/2011

2

1. Visually-Guided Grasping (2D) 1.1 visually-guided grasping (non dexterity)

1.2 including dynamic scenarios (non dexterity)

1.3 including learning capabilities and dexterity

2. Visually-Guided Grasping (3D)

3. Sensor-based Control Interaction 3.1 planning of physical interaction tasks

3.2 vision-force-tactile integration for robotic

physical interaction

4. The UJI Service Robot: A Case Study

5. Underwater Manipulation

3


1. Visually-Guided Grasping (2D)

1.1 visually-guided grasping (non dexterity) [IMG-04-Sanz]

1.2 including dynamic scenarios (non

dexterity) [IMG-04-Recatala]

1.3 including learning capabilities and

dexterity [IMG-04-Morales]

5/9/2011

3


1.1 visually-

guided grasping

(non dexterity)


The Human hand capabilities

“The Intelligent Pinch”

“Precision Grasp” vs “Power Grasp”

5/9/2011

4


Two-Fingered vs Dexterous Robot Hands

DLR – Institute of robotics University of Karlsruhe

NASA – Robonaut University of Bologna

Industrial gripper:

”UMI RT 100”

A Generic Model


Visually-Guided Grasping

Determination Execution Planning

Perception Reasoning Action

Geometric Knowledge

e. g. symmetry, curvature,... [Bajcsy (93), Arkin (98),...]

Action-Oriented Perception

[Leyton (87), Blake (95),…]

5/9/2011

5


2D Visually-Guided Grasping with Two-Fingered Hands

Global

Local

Grasp Stability

(unknown / unmodeled objects)


Wrap Grip Pinch

• Types of Grasps [Tan & Schlimler, 93]

../Pedro-07-2005/Congresos/UPV-Gandia/Presenta2004/CIRA99.ppt

5/9/2011

6


• Force-Closure [Nguyen, 88]

Iff we can exert, through the set of contacts, arbitrary force

and moment on this object

P1 and P2 are known like “antipodal point grasps”

P 1 2

f 1 n

f 1 t

f 2 n f 2

t

P

DEF

Geometric Interpretation


• Stability Conditions, [Montana, 91]

1. The curvature (from object or fingers).

2. The distance between the grasping points.

3. The viscoelasticity from fingers or object.

4. The existence or not of force feedback.

5/9/2011

7


• Grasp Determination (GloSt)

fn

ft

= µfn

q

Friction cones

Object surface

fn Finger 1

Grasping line

= arctanq ( )

Coulomb friction model


• Grasp Determination (GloSt)

cog1

cog2

cog

2D Image

Camera

Hole

x

y

Optical axis

IMG04

../../../../UJI-Master/Master-UJI2009/MaterialDocente/IMG04-Sanz.ppt

../../UJI-Master/Master-UJI2009/MaterialDocente/IMG04-Sanz.ppt

5/9/2011

8


[Sanz et al., 2005] “Grasping the not-so-

obvious: Vision-Based Object Handling for

Industrial Applications”. IEEE Robotics and

Automation Magazine

The Symmetry Knowledge and the Grasping

Determination Problem

[Li et al., 2008] “Bilateral Symmetry

Detection for Real-time Robotics

Applications”. Int. J. of Robotics Research


Preliminary Conclusions (GloSt)

•CSF permits the quantification of the symmetry degree of

a shape in a simple and efficient manner

• CSF makes easier the geometric reasoning necessary to

seek grasping points from 2D images of real objects

• The global system has proven to work with a broad set

of unknown objects, making real applications feasible

5/9/2011

9


• Grasp Determination (LocSt)

1. Extraction of Grasping Regions

2. Selection of Compatible Regions

3. Grasp Refinement

Main Stages of the LocSt Algorithm



• A “grasping region” is a segment of a

contour which points have a curvature below

the curvature threshold

A grasp region can be described as straight segment. All the

points met the curvature stability condition

This description simplifies the further computation and

reasoning

It reduces the complexity of the problem

5/9/2011

10



GR1

GR2

GR3

GR4

GR1 GR2 GR3GR4

Example-1



GR1GR2

GR3

GR4

GR5

GR6

GR8

GR9

GR10

GR11

GR12

GR13GR14GR15

GR16

GR17

GR7

Example-2

5/9/2011

11



Compatible Regions


GloSt LocSt

All

en w

rench

P

ince

rs

• Experimental Results (GloSt vs LocSt)

5/9/2011

12


• Experimental Results (GloSt vs LocSt)

GloSt

LocSt


• Experimental Results (LocSt)

Types of grasps

Squeezing grasps Expansion grasps

5/9/2011

13



Squeezing grasps Expansion grasps



5/9/2011

14



Fig

ure

by [

Fa

verj

on &

Po

nce

(91)]



Promoting Active Perception?

5/9/2011

15


Conclusions • Fast response in the computation grasping with a state-of-the-art technology has been reached

• This method is able to find solutions, including internal contours or expansion grasps

• Indirect benefits have been obtained that can be applied in other research domains (e.g. the use of in pattern recognition algorithms)

• Ongoing research: – Extension towards dynamic scenarios

– Extension towards dextrous manipulation (e.g. the BarrettHand)

2D Visually-Guided Grasping

with Two-Fingered Hands


1.2 including

dynamic

scenarios

(non dexterity)

5/9/2011

16


• Towards Dynamic Scenarios [Recatalá et al., 2002-04]

“Grasp Tracking”

Gabriel Recatalá

Email: [email protected]


• Towards Dynamic Scenarios

“Grasp Tracking”

video-03


../../../IMRob-Genova(Rezia)-2007/FINAL/video-03.avi

5/9/2011

17


“The Catching Problem”

Example from MIT

The MIT Whole Arm Manipulator (WAM)

The Fast Eye Gimbals (FEGs)

mounted to a ceiling rafter

http://web.mit.edu/nsl/www/



WAM Catching

of a Paper Airplane






5/9/2011

18



MinERVA PROJECT (TUM, Germany)

“Manipulating Experimental Robot with

Visually-Guided Actions”

Looking for human-

robot analogies in

the catching problem



1.3 including

learning

capabilities

and dexterity

5/9/2011

19


• Towards Dexterous Manipulation [Morales et al., 2002-05]

Univ. of Massachusetts

(USA)

Prof. Grupen

Antonio Morales

Email: [email protected]


• Towards Dexterous Manipulation

UMass Humanoid Torso

Two 7 d.o.f. arms.

Pan-tilt head.

Stereo camera system.

Force-torque sensor on fingertips.

Two three-fingered Barrett

hands.

../Pedro-07-2005/Congresos/ICMA2005/Toni/img04-toni.ppt


5/9/2011

20


Goal : Online vision-based grasping of

unmodeled planar objects

1. Process the stereo images of the object

2. Generates a number of feasible candidate grips (See ICRA 2001, IROS 2002, IMG 2004)

3. Selects the grasp to execute

4. Executes the grip



Generation of grasping triplets

Stereo images →Object contour→ Grasping regions

Triplets of regions → triplets of grasping points

5/9/2011

21



video-04


More Results

5/9/2011

22


Which one?

Which one to execute?

Why do prefer one to the others ?

A learning framework Learn through successive experiences the relation

between the reliability of a grasp and its vision-based

description.

• Abstract grasp characterization scheme.

• Practical measurement of reliability.

• A methodology for predicting the reliability of a grasp based on its

similarity to past attempts.

• An active learning technique to select the next grasp to execute with

the purpose of increasing the predictive performance of the

accumulated experience.


5/9/2011

23


Vision-based grasp

characterization Based on nine high-level

features # Feature Name

QFCL Force line (FCL)

QRFD Real focus deviation (RFD)

QFE Finger extension (FE)

QFS Finger spread (FS)

QRFC Real focus centering (RFC)

QFGL Finger limit (FGL)

QPA Point arrangement (PA)

QTS Triangle size (TS)

QCC Contour curvature (CC)

Based on visual information.

Hand constrains included.

Invariant to location and orientation.

Physical meaning.

Reliability and robustness concern.

Object independent.

These features define the G-space (GS)

Siii Gqqg },...,{ 91


Experimental reliability test

CLASS DESCRIPTION

E Couldn't lift the object

D Dropped during 1st sequence

C Dropped during 2nd sequence

B Dropped during 3rd sequence

A Finished the test

= { A, B, C, D, E }

For any grasp g, g IMG’04

../../../../Copia-Seguridad-01-2006/Pedro24JUL2005/Congresos/img04/Presentation/toni/faverjon_fail_short_small.mov

5/9/2011

24


Grasp reliability prediction

Given a grasp gq, computes the probability P of each

reliability class, based on the results of K nearest

neighbors weighted by distance.

)(

)(

)(

)(

)|(

qj

i

qi

gKNNg

j

gKNNgi

qdK

dK

gp

gq QS

KNN(G) : K nearest neighbors of gq QS

K(di): Kernel function, d : distance from gq to gi

Euclidean distance on QS

• KNN classification rule


Experimental database •To obtain experimental data, we have carried out

exhaustive series of grasp trials

• Four Objects

• A wide variety of grasp configurations on each object

• Twelve trials for each grasp (4 times in 3 different orientations)

• More than three hundred executed trials, on four data sets.

5/9/2011

25


Prediction performance

Error

distance Random KNN

0 23.5% 50.8%

1 26.2% 21.9%

2 20.3% 12.8%

3 20.7% 12.0%

4 9.3% 2.5%

e 0.415 0.223

•The prediction error decrease

with the size of training dataset.

• KNN improves the performance

of the random prediction.


Active learning

An “active learning” strategy selects the next action to execute with the aim of acquiring new knowledge of the problem.

next action = next grasp to execute

Exploration rule.

• Uses a KNN prediction function

• Given n candidates,

• Chooses the candidate having a least confident prediction.

gi i [p(i |gi)]

)|(minarg iig gpi

5/9/2011

26


Active learning A Validation framework aimed at emulating the running in the real

world, while measuring the improvement in the prediction performance.

Random selector defined for comparison. • The exploration is executed

several times and the results

are averaged

• The exploration procedure

reach an optimum in a hundred

trials.


Conclusion and main

contributions

We develop a complete learning framework for assessing grasp releiability.

We implement this framework on a working grasping system.

We validate this framework against real experimental data.

presentación de powerpoint - ujisanzp/irs/autrobmanipula-2011-2.pdf · [email protected] pedro j sanz...

Documents