multi-modal robot skins: proximity servoing and its ... · [1] e. cheung and v. lumelsky, “a...

KIT – University of the State of Baden-Wuerttemberg and National Research Center of the Helmholtz Association

INSTITUTE FOR ANTHROPOMATICS AND ROBOTICS - INTELLIGENT PROCESS CONTROL AND ROBOTICS, LAB (IAR-IPR) KARLSRUHE INSTITUTE OF TECHNOLOGY

www.kit.edu

IROS 2015, Hamburg

Multi-Modal Robot Skins: Proximity Servoing and its Applications

Stefan Escaida Navarro and Björn Hein

Workshop “See and Touch: 1st Workshop on multimodal sensor-based robot control for HRI and soft manipulation” at IROS 2015

Karlsruhe Institute of Technology 2

Outline

Introduction State of the Art Proximity Servoing

Preshaping and Grasping

Haptic Exploration

Teleoperation

(Collision Prediction and Detection)

Contour Following/Collision Avoidance

Conclusions and Outlook

Stefan Escaida Navarro -- Multi-Modal Robot Skins: Proximity Servoing and its Applications

Karlsruhe Institute of Technology 3 Stefan Escaida Navarro -- Multi-Modal Robot Skins: Proximity Servoing and its Applications

Introduction


What is Proximity Sensing?

What is the role of proximity sensing in the perception taxonomy? It shares treats with haptics: Perception is “local” and can only be interpreted together with proprioception, i.e. with use of the kinematic chain A spatial resolution is implemented in the skin of the robot (on the robot’s exterior and grippers) As example from nature, the human skin is also locally sensitive to heat radiation



Why Proximity Sensing?

2D and 3D camera systems are inadequate for the detection and modeling of events in the near proximity of the robot

Perception of camera systems is encumbered by occlusions, extraneous light influence and shadows → problematic for safety Tactile perception can only detect the presence of obstacles when the contact has already occurred, aka “too late”

Every touch event has a prior proximity event Good complement All in all: important for safety because it closes a still existing perception gap



Capacitive Proximity Sensing with Spatial Resolution

Advantages: Insensitivity regarding lighting and visibility conditions Insensitivity regarding optical surface properties Wide area around the module is covered (sensing is not oriented)

Disadvantages:

(Sensitivity to object material properties, geometry and size) Can be used to identify object?

Signal to distance non-linearity Signal processing overhead due to demodulation ( Latency)



State of the Art


State of the Art (excerpt)

Cheung and Lumelsky 1992 [1]

Wistort et al. 2008 [2]

Preshaping with adjustment of hand pose and fingers (Hsiao et al., IR, and Wistort et al., capacitive) Collision Avoidance in C-Space (Lumelsky and Cheung, IR) Low latency/high speed preshaping and collision avoidance by Shimojo Labs (IR) Not many of the approaches also consider the tactile modality or all possible DoFs


Hsiao et al., 2009 [3]

Shimojo Labs, 2011 [4]

Shimojo Labs, 2013 [5]


Self-Developed Sensor Modules (2012-2015)


Large Area Module: • 4x4cm • 1 proximity cell • 1 tactile cell • Detection range: ~10cm • Up to 16 modules on

one channel

Sensor array: • 3x16 = 48 sensors • ~25 FPS • ~100ms latency

The arrangement of sensors as a regular grid delivers proximity and tactile images


Proximity Servoing


Proximity Servoing: Closed-loop position or velocity control using proximity sensors

Most common control goal: achieve a configuration where sensor values are equilibrated Use methods from CV to extract features from proximity images

Proximity Servoing is a base for many applications: Preshaping and Grasping, Haptic Exploration, Obstacle Avoidance, etc.

Proximity Servoing



Safe Zone

Proximity Servoing – Challenges

„Rear lights in the fog“-problem (©Franz Heger) Reduced max. velocity due small distances and non-negligible reaction times (latency)


• No signal/ no feedback for control

• Target is too far away

• Signal is weak/noisy • Signal is too close to

the detection threshold

• Signal is ok/good • Distance to target is

safe

• Signal is very good • Distance to target is

not safe

Detection Threshold Distance to Target

Safe Zone

Collision

Some latency

Negligible latency


Applications


Preshaping [8]

Challenge when grasping with tactile-only sensors non-negligible contact force has to be established for perception

Preshaping means to find a suitable configuration for an gripper and its fingers -before touch is established-, that favors robustness when grasping Preshaping is proximity servoing


An industrial gripper equipped with 2x2 CPTS in its jaws


6D Preshaping using CTPS in Gripper

Calibration of the sensors of the two jaws to equal output under same conditions High resolution/low noise is given at small distances (<1mm)



6D Preshaping using CTPS in Gripper

Grouping of measured sensor values of modules for gradient detection: Implementation of closed-loop control “Gradient Descent” results in preshaping Comparison of lowest distance/highest proximity values of the groups

Grouping for translational and rotational adjustments:



6D Preshaping using CPTS in Gripper

Simultaneous adjustment in all 6 DoF Continuous adaption while closing the gripper



6D Preshaping Evaluation with different materials

Alignment always successful Higher deviation with wood and plastic compared to aluminum Good repeat accuracy Low derivation at end of phase two

Wood: Front Back

Alum.: Front Back



Proximity + Haptic Exploration

Haptic Exploration deals with acquiring object and world information through a systematic exploration strategy using touch Combined and complementary tactile- and proximity-based approach

Based on proximity servoing/preshaping



Telemanipulation [9]

Display proximity values acquired inside the gripper as forces on a haptic input device to assist the user in telemanipulation Additionally, assist the user with partial autonomy



Implementation using a cascade-controller Inner loop is a force-position controller

The user opposes full resistance, i.e. maintains effector at 0 pose Full force display and no robot movement User opposes no resistance Effector pose corresponds to proximity gradient and robot moves accordingly

Outer loop is velocity controller like before

Telemanipulation – Cascade Controller



Telemanipulation – Video



Telemanipulation -- Results

Workload and intuitiveness was assessed in a user study against a baseline of “perfect” vision

Workload reduction is possible when operating with partial autonomy Complementary proximity information considered to be useful Small improvement in performance Details: Talk for the paper

Novel approach: Manipulation of moving objects



Contact-Prediction (Depth Tracking) [8]


Contact time given by tactile modality


6D Contour Following/Obstacle Avoidance


Problem statement:

Starting from pose S, follow a path to goal G parallel to an obstacle’s surface first detected at pose D, as long as the obstacle is in the way to the goal. Otherwise, choose a direct path (from S to D and E to G)

An end-effector equipped with an array of proximity sensors is used to demonstrate contour following/obstacle avoidance


Closed-Loop Control and Curvature Prediction


Reactive control approach: Track obstacle tangent plane P over time to plan next increment Get misalignment from gradients in proximity image and use them as feedback for distance and orientation controllers

Predictive control approach:

Track curvature of obstacle (1𝑟𝑟, vertical

and horizontal) over time Use curvature to take a shortcut 𝑇𝑇 that is the chord of the corresponding circle If necessary, pre-align with rotation 𝜑𝜑

1𝑟𝑟 vertical 1

𝑟𝑟 horizontal


Preliminary Results


Curvature prediction should deliver speedup when alignment constraints are tight Correction path corresponds to object shape Implementation of HRI should be straight forward


Conclusions and Outlook


Conclusions

It is purposeful to treat proximity sensing similar to tactile sensing Spatial resolution Image Processing It shares treats with “traditional” haptics

Proximity Servoing has applications in many scenarios Often tactile modality can be used complementarily (Grasping, Haptic Exploration)

Many measurement principles exist with advantages and disadvantages It still is unclear whether there is a “best practice” solution



Outlook, future work

Reduce latency (“Rear lights in the fog”-problem) Grasping smaller skin patches (almost there, work in progress) Big sensor networks challenge for the future

Sensor module design (work in progress)

Modularity: signal processing in each module Increase spatial resolution

If possible, without loosing sensing range Add features to the sensors

Receive mode for better detection of insulating materials, etc. Shear-force measurements (tactile mode)



Thank your for your kind attention! Questions?



References [1] E. Cheung and V. Lumelsky, “A sensitive skin system for motion control of robot arm manipulators,” Robotics and Autonomous Systems, vol. 10, no. 1, pp. 9 – 32, 1992. [2] R. Wistort and J. R. Smith, “Electric field servoing for robotic manipulation,” in Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on, sept. 2008, pp. 494 –499. [3] K. Hsiao, P. Nangeroni, M. Huber, A. Saxena, A. Y Ng, “Reactive Grasping Using Optical Proximity Sensors,” in Intelligent Robots and Systems (IROS), 2009 IEEE/RSJ International Conference on, 2009, pp. 2098–2105. [4] K. Terada, Y. Suzuki, H. Hasegawa, S. Sone, A. Ming, M. Ishikawa, and M. Shimojo, “Development of omni-directional and fastresponsive net-structure proximity sensor,” in International Conference on Intelligent Robots and Systems, 2011. [5] Keisuke Koyama, Hiroaki Hasegawa, Yosuke Suzuki, Aiguo Ming, and Makoto Shimojo, “Pre-shaping for Various Objects by the Robot Hand Equipped with Resistor Network Structure Proximity Sensors,” in Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on, 2013, pp. 4027-4033 [6] D. Göger, M. Blankertz, and H. Wörn, “A Tactile Proximity Sensor,” in IEEE Sensors 2010, 2010, pp. 589 –594. [7] S. Escaida Navarro, M. Marufo, Y. Ding, S. Puls, D. Göger, B. Hein, and H. Wörn, “Methods for Safe Human-Robot-Interaction Using Capacitive Tactile Proximity Sensors,” in Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on, 2013, pp. 1149–1154. [8] S. Escaida Navarro, M. Schonert, B. Hein, and H. Wörn, “6D Proximity Servoing for Preshaping and Haptic Exploration using Capacitive Tactile Proximity Sensors,” in Intelligent Robots and Systems (IROS), 2014 IEEE/RSJ International Conference on, 2014, pp. 7–14. [9] S. Escaida Navarro, F. Heger, F. Putze, T. Beyl, T. Schultz and B. Hein, “Telemanipulation with Force-based Feedback of Proximity Fields,” in Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, 2015



Sensor Circuit and Modules in Our Group


Principled sensor circuit design from D. Göger, M. Blankertz, and H. Wörn: “A Tactile Proximity Sensor,” in IEEE Senors 2010

Proximity Mode (self-capacitance)

Tactile Mode (mutual-capacitance)

multi-modal robot skins: proximity servoing and its ... · [1] e. cheung and v. lumelsky, “a...

Documents