Prof. Emanuele Menegatti

Advancing industrial production with 3D perception for industrial robots

RoboBusiness 2015, Milan, Italy

IAS-Lab (Intelligent Autonomous Systems Laboratory)

University of Padova, Italy

IT+Robotics srl

Spin-off company of University of Padova, Italy

The IAS-Lab@

University of Padova

People

Staff1. Enrico Pagello, Full Professor of Computer

Science

2. Emanuele Menegatti, Associate Professor

3. Michele Moro, Assist. Professor

Students1. Filippo Basso, PhD cand.

2. Mauro Antonello, PhD cand.

3. Riccardo Levorato, PhD cand.

4. Roberto Bortoletto, PhD stud.

5. Elisa Tosello, PhD stud.

6. Nicolò Boscolo, PhD stud.

About 10 Master-level students

Post-Docs1. Luca Tonin, Post Doc

2. Stefano Ghidoni, post-doc

3. Stefano Michieletto, post-doc

4. Matteo Munaro, post-doc

Collaboratorations

• Antonio D’angelo and Claudio Mirolo, Assist. Prof.s at Udine University

• Stefano Carpin, Assoc. Prof. at Univ. of California (USA)

• Hiroshi Ishiguro, Prof. Univ. of Osaka (JAPAN)

•Tamio Arai, Prof. Univ. of Tokyo (Japan)

• Frank Dellaert, Prof. Georgia Teach Inst. (USA)

• Wolfram Burgard, Prof. Univ. of Freiburg (Germany)

•Stefano Soatto, Prof. UCLA

•Jeff Bruke, Assistant Dean UCLA

•Radu Rusu, CEO of Open Perception

Research topics

•Autonomous robotics•Humanoid robotics•Distributed Perceptual Infrastructure•New vision sensors (2D & 3D)

Lab equipment

•3 robot manipulators•9 humanoid robots•5 mobile robots•Distributed network ofintelligent cameras•3D vision cameras

Enhancing robot perception for autonomous robots

IAS-Lab @ UniPD

• EU-H2020 FoF 2014• FOCUS - Factory of the Future Clusters

• EU-FoF 2012• FibreMap - Automatic Mapping of Fibre Orientation for

Draping of Carbon Fibre Parts• EU-FoF 2011

• Thermobot - Autonomous robotic system for thermo-graphic detection of cracks

• EU-RfSME 2010• 3DComplete - Efficient 3D Completeness Inspection

• EU-FSE 2009:• iSP - Innovative Simulation and Programming of robotics

workcells• iDVS2 - Intelligent Distributed Audio and Video Surveillance

System

• EU-FSE 2008:• iDVS: Intelligent Distributed Vision System for surveillance and

quality inspection

• EU-Comenius2 2006:• TERECoP: Teacher Education on Robotics-Enhanced

Constructivist Pedagogical Methods

• University of Padua 2009:• Mobility, Perception, and Coordination for a Team of

Autonomous Robots

Recent Funded Projects @ IAS-Lab

People: 8 employees + 4 collaborators Money: 600.000 Euro annual turnover

10th year anniversary!!

Mapping and Navigation 3D Complete – European Project

• Fully automatic 3D Quality Inspection

• Partners:

• 3 European Company

• 3 European Academic Institutions

Mapping and Navigation Acquisizione e Risultati

• Ricostruzione di un modello 3D

• Aggiunta del colore vero e delle scritte

• Algoritmi di controllo qualità

10

Thermobot• Autonomous robotic system for thermo-graphic detection of cracks

– FP7 European Project– 3 years, start date January 2012– Factory of the Future (FoF.NMP.2011-3)

• Objective:– Detection of cracks using thermography– Automatic definition of process using off-line programming– Online adaptation of process using feedback from camera

Towards semi-autonomous detection of cracks

Arial 8pt

Thermographic image analysis - metallic parts

Crack

Heat transfer across the crack

is reduced

Leads to high temperature gradient

Laser spot

12

FibreMap• Automatic Mapping of Fibre Orientation for Draping of Carbon Fibre

Parts– FP7 European Project– 3 years, expected start date September 2012– Factory of the Future (FoF.NMP.2013-10)

• Objective:– Measurement of fibre angle of 3D carbon fibre parts– Automatic definition of process using off-line programming– Data used to increase simulation accuracy

DEI – UNIPD 2013 /NN DEI – UNIPD 2013 /NN

April2014 Property of Comau - Duplication prohibited

Open Concept

14

The Open approach adds to Robot Control Unit further power from

an External Personal Computer: this helps programming automated

robotic cells and integration with external sensors to simplify the

implementation of complex manufacturing applications

Vision Algorithms:- Look & Move- Visual servoing

Interaction Algorithms:- Force control- Impedance control

CAD Planning:- Surface movements- Complex Interferences

For universities and research centers

Open architecture allows to develop newinnovative motion strategies

IAS-LAB

COMAU C5G integration with ROS-Industrial

RT UDP/IP

PDL2 Program(Moves, etc.)

C4G Open

C4G Open realistic Robot Library

ROS

ROS

ROS ROS

TCP/IP

ROS

ROS

15

IT+Robotics srlIndustrial Products

Artificial Vision for Industrial Robots

Workcell SimulatorThe complete software suite for off-line robot programming.

Software suite for offline programming

1. Create your work cell• Component Editor

Builds the database of your components.

• Workcell EditorBuilds your work cell

2. Simulate it:• Simulator

Defines components behaviors and automatically generate robot trajectories

3. Bring it to real world:• Robot Code Compiler

Translates simulation into working robot code.

Workcell SimulatorThe complete software suite for off-line robot programming.

AutonomousIncreases flexibility of production work cell by decreasing set-up time, allowing reduction

of batch size down to size of one

1. Work cell programming is performed off-line, interacting with an easy-to-use user interface

2. Trajectories are computed automatically, using cutting-edge motion planning engine

3. Simulator can be extended using task planning algorithms tailored for specific simulated process

Workcell SimulatorThe complete software suite for off-line robot programming.

FlexibleCreate a common platform to develop code for any mechanism and any process

Bending Process Palletization

Visual Quality InspectionCNC Simulation

…and much more…

Smart Pick 3DFast and efficient robot guidance for random bin-picking.

“It’s been a desired, but very elusive and unreachable goal for a long time,” Shafi observes. “A lot of people have referred to bin-picking as the Holy Grail of robotic material handling”

February 1, 2006, Automation World

• Robot guide for random bin-picking:• Recognize, precisely locate and grasp

objects randomly placed inside a bin

• Advantages:• Easy integration with existing working

processes

• Easy production change

• More compact than solutions based on conveyor belt

• Reduced cycle time (up to 5 seconds)

= More flexibility

Smart Pick 3DFast and efficient robot guidance for random bin-picking.

FlexibleDifferent products requires different technologies, both software and hardware

Smart Pick 3D Profile Standard camera Conveyor belt Very fast (< 0.5 s)

Smart Pick 3D Solid 3D sensor Bin Complex geometry

Smart Pick 3D Shape Standard camera Bin High reflective objects

Automatic accurate calibration Easy to use GUI to configure products Works with every robot Completely empties the bin

Common features:

Smart Check 3DHigh performance quality control integrated into the production line.

• Automatic visual inspection• Combines of 3D vision and 2D vision with a single sensor

• Focused on completeness inspection, but can be extended to anyvisual inspection check

• Two configurations:• Product on conveyor belt: maximizes cycle time

• Vision system mounted on robot: maximizes flexibility

Smart Check 3DHigh performance quality control integrated into the production line.

FlexibleOne workcell for many products

• Introduction of visual inspection systems is not a matter of technical flexibility but of economic efficiency

• SMEs willing to archive lean production paradigm– Small batch size (down to the size of one)– Frequent change of product’s quality assessment

requirements

• Objectives:– Shift of programming effort to end-user– Use of robot manipulator to increase workcell

flexibility

Smart Check 3DHigh performance quality control integrated into the production line.

Flexibleintegrated 2D and 3D vision systems.

Completeness

…and much more…

3D Measurement

Inclusions

Surface inspection

4th Industrial RevolutionFlexible Robot Production integrated into the production line.

Reactive - Sensorized - Intelligent RobotsProgramming is performed online, robots can see,

robot can behave intelligently