robot vision module dott. emanuele menegatti (emg@dei.unipd.it) intelligent autonomous systems lab...

Robot Vision Module

Dott. Emanuele Menegatti(emg@dei.unipd.it)

Intelligent Autonomous Systems LabUniversity of Padua

ITALY

Based on course notes of Prof. Bob Fisher Edinburgh University - UK

…a Computer Vision example...

Sorting parts on a conveyor belt

... A Robot Vision example... Navigation and Obstable avoidance

?

!! Real Time – Real World !!

Physics of Vision

Illumination

The sensors…

Humans Eyelid Iris Lens Retina Optical Nerve

Robots Shutter Iris Lens CCD TV Cable

The retina…

Horizontal cell

Bipolar cellAmacrine cell

Ganglion cell

Optical Nerve

Light

http://webvision.med.utah.edu/

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Camera Evolution

Pinhole Camera

http://www.kodak.com/global/en/consumer/education//lessonPlans/pinholeCamera/http://www.pinhole.org/about/index.cfm

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Camera Evolution The lens Introduced to collects more

lights

The film Introduced to store the image The CCD Introduced to acquire directly a

digital image (higher performances for certain applications)

How to capture colours

The original (left) image

is split in a beam splitter

Thin Lens terminology

Optical Axis

Lens Axis

Focal point(Secondary)

Focal length

(in a real camera: Focal length = the distance between the equivalent center of the lens

and the image plane)

Ray TracingWe will consider a simple system composed of a single thin lens,

there are simple rules we have to follow to ray-trace:1. Rays travel in straight lines and change directions only when

they encounter a discontinuity of the refractive index

2. It is conventional to have the object on the left of the optical system and the image will form on the right

3. All rays emanating from a single point in space must converge on a single point in the image plane (definition of focus)

4. Any ray entering the lens parallel to the axis on one side goes through the focus point on the other side

5. Any ray entering the lens from the focus point on one side emerges parallel to the axis on the other side

How to look at Vision? Low Level Vision:

Middle Level Vision:

High Level Vision:

Image Level Properties

Properties of World

Properties of Actual Objects

–Feature detection–Lightness–Geometry & Shape

–Stereo & 2½D sketch–Motion & Optical Flow–Shape from Shading

–Object Representation–Object Recognition–Geometric Invariance

Basic Optics

Images’ FileComposed of: Image File Header

Self-description of the image Image dimensions Image type Date of creation Program that created the image

Image data Some image formats can handle only limited types of images (bynary or monocrome), but the current formats are evolving toward Multimedia contents

Formati di ImmagineDue tipi fondamentali di immagini o meglio due modalità grafiche per rappresentare le immagini:

Bitmap L’immagine viene descritta dando il colore pixel per pixel

Vettoriale Oggetti presenti nell’immagine descritti in termini matematici

BitmapCaratteristiche: Risoluzione (ppi) Profondità di colore

Compressione Lossy (JPEG) Lossless (GIF)

Adatto per sfumature di colore

Bitmap (2)

GIF (Lossless)Graphics Interchange Format

JPEG (Lossy)Joint Photographic Expert Group

Bitmap (3)GIF

GIF87 256 colori

GIF89 Trasparenza Interlacciamento Immagini multiple (gif animate)

Created by CompuServe Inc.

JPEG Non ha limiti di colore

Adatto per foto Una sola immagine per file

Header può contenere preview

Immagini Vettoriali

Caratteristiche: Dimensione immagine Descrizione matematica

Compatta Non adatta per foto o immagini ricche di dettagliImmagine vettoriale Bitmap (FACILE)

Bitmap Immagine Vettoriale (DIFFICILE)

PBM (Portable Bit Map) Family of formats:

PBM Supports monochrome bitmaps (1 bit per pixel). PGM Supports greyscale images PPM Supports full-color images PNM Supports content-independent manipulations on

any of the three formats listed above

P3 # magic number# example from the man page 4 4 # cols & rows15 # maxval0 7 0 015 0 15 0 0 0 0 15

TIFF (Tag Image File Format) Supports multiple images with 1 to 214 bit depth

Can be lossy or lossless Very general and very complex Used by scanners Created by Aldus Corp.

PostScript (ps, eps) Store image data using ASCII characters (7-bit ASCII code)

Used for graphics displays and printer

Newer versions include JPEG compression

Used to include graphics or images in a document

MPEG (Motion Picture Expert Group)

Stream-oriented encoding for video Contains video, text, graphics Created by an international group of industry and governments

Uses Spatial and Temporal Redundancy It is evolving:

MPEG-1: 0.25Mbps (audio) 1.25Mbps (video) MPEG-2: 15Mbps good for TV Future version will recognizes objects and generate their images

The Difference Between CCD and CMOS CCD high-quality, low-noise images. CMOS more flexible (every pixel can be read independently) The light sensitivity of a CMOS chip is lower (Many of the

photons hitting the chip hit the transistors instead of the photodiode)

CCDs consume 100 times more power than an equivalent CMOS sensor.

CMOS extremely inexpensive compared to CCD sensors (Chips can be fabricated on any standard silicon production line)

CCD sensors produced for a longer period of time, so they are more mature. They tend to have higher quality pixels, and more of them.

Depth of Field...

… depends on: Shutter Opening Sensitive Element

Focal Length of the lens

Distance of the object

Region of confusion

Hyperfocal distance

Depth of Field

Task: Sort Parts Vision Goal: Describe and Identify

Constraints: Flat Shapes Sit flat Different Areas Opacity Serially delivered

Camera

Synch

Silhouette

Detector

Counter

Size Comparator

Threshold

Threshold