new machin vision

Introduction to Machine vision Machine vision in robotics is nothing but, giving an eye

(generally in the form of a camera) to the robot.

Its main application area is in the industries to describe, the

understanding and interpretation of technically obtained images for

controlling production processes.

It is also mostly & very widely used in the field of medical &

space research works.

SENSORS

• Analogous to human sensory organs: Eyes,

ears, nose, tongue, skin

• Help robot to know its surroundings better

• Improves its actions and decision making

ability

• Provides feedback control.

SENSORS USED IN MACHINE VISION

Generally the optical/light sensors are used in machine vision

system.

From the list of the optical/light sensors, camera is widely used as

a sensor in machine vision system.

CAMERA VISION

CAMERA VISION CONT.

TYPES OF CAMERA VISION USED IN ROBOTICS:

Onboard camera vision Overhead camera vision

DIFFERENT TYPES OF CAMERAS

• CCD Higher quality images with little noise

Typically more pixels

Higher sensitivity to light

• CMOS Lower cost

Easier to build

Lower power consumption

BASIC CONSTRUCTION & WORKING OF A CAMERA

• The image sensor employed by most digital cameras is a charge coupled device (CCD). Some cameras use complementary metal oxide semiconductor (CMOS) technology instead.

• Both CCD and CMOS image sensors convert light into electrons.

BASIC CONSTRUCTION & WORKING OF A CAMERA

• Serial - The earliest digital cameras had a serial interface, but no currentcameras use this since it is so slow.

• USB 1.1 - USB was the first widespread high speed method of data transferfrom cameras. It is theoretically capable of transfer speeds up to 11megabits/second (note megabits not megabytes)

• USB 2.0 - A development of USB but much faster - up to 480megabits/second.

• IEEE 1394 (Firewire) - Though this is an older interface than USB, it wasoriginally only really used much on Apple computers. It's capable of highspeed transfer (400 megabits/second) and it's now found on some PCs orit can be added to them via a plug-in card. More common on digital videocameras than still digital cameras.

ADVANTAGES OF CAMERA VISION

• It is a combination of multiple sensors.

• Much wider view angle.

• Can be employed from a far distance to keep/record the information.

IMAGE PROCESSING TOOLS

EXPLORING MATLAB

• MATLAB is a program for doing numerical computation. It was originally designed for solving linear algebra type problems using matrices.

• It’s name is derived from MATrix LABoratory.

• MATLAB is an interpreter -> not as fast as compilers.

• File extension for MATLAB is .m

INTRODUCTION TO MATLAB

INTRODUCTION TO COMMANDS & FUNCTIONS

• Type the commands by terminating with a semicolon(;) in the command window & see the o/p or result on the same window.

• MATLAB is case sensitive i.e. b and B are different variables.

• % sign in the beginning of any sentence is used to give comment line. Multiline comment is not possible.

• To get help (when keyword is known) type help XXX (where XXX is the keyword).

• Type lookfor (when keyword is unknown).)

VARIABLES:

• Variable names can contain up to 63 characters• Variable names must start with a letter followed by letters, digits,

and underscores.• Variable names are case sensitive• Special variables:- ans Default variable name for results pi Value of π eps Smallest incremental number inf Infinity NaN Not a number e.g. 0/0 Realmin The smallest usable positive real number realmax The largest usable positive real number Disp displays the contents of a variable

INTRODUCTION TO COMMANDS & FUNCTIONS CONT.

INTRODUCTION TO COMMANDS & FUNCTIONS CONT.

• round(X): round to nearest integer, trailing 5 rounds to the nearestinteger away from zero. For example, round(2.5) returns 3; round(-2.5)returns -3.

• fix(X): round to nearest integer toward zero (truncate). For example,fix(2.7) returns 2; fix(-2.7) returns -2

• floor(X): round to the nearest integer toward minus infinity (round tothe nearest integer less than or equal to X). For example, floor(2.7) returns2; floor(-2.3) returns -3.

• ceil(X): round to the nearest integer toward positive infinity (round tothe nearest integer greater than or equal to X); for example, ceil(2.3)returns 3; ceil(-2.7) returns -2

FUNCTIONS:

CONDITIONAL STATEMENTS & LOOPING

• if..elseif..else..end

CONDITIONAL STATEMENTS & LOOPING CONT.• for..end

CONDITIONAL STATEMENTS & LOOPING CONT.• while..end

MATHEMATICAL OPERATIONS

MATHEMATICAL OPERATIONS CONT.

LOGICAL & CONDITIONAL OPERATIONS

< : less than <= : less than or equal to> : greater than>= : greater than or equal to == : equal ~= : not equal

& : and | : or ~ : not

SIGNAL REPRESENTATION

• MATLAB has two inbuilt functions to represent a signal on the display:

• PLOT function is used for continuous domain functions.

• For discrete domain functions, you should use STEM function.

EXAMPLE: > n=-5:0.5:5;> x(n)=exp(j*(0.4*pi*n-0.5*pi));> y(n)=x(n)-x(n-1);> stem(y,n)

SIGNAL REPRESENTATION CONT.

Signal representation example:

fs=100; % sampling frequency

t=0:1/fs:1; % setup time from 0 to 1 second

f=5; % input frequency

x=sin(2*pi*f*t); % input signal

subplot(211);plot(t,x); % plot the signal

title('Continuous-time');

subplot(212);stem(t,x); % plot the signal using stem

title('Discrete-time');

ARRAY OPERATION

The Matlab array arithmetic operations are:

addition (+) subtraction (-)

array multiplication (.*) array division (./)

array power (.^)

These operations act element-wise on the arrays, for example if A is an n by m matrix

and B is an p by q matrix then A.*B is defined only if n=p and m=q, and the (i,j)

element of A.*B is the (i,j) element of A multiplied by the (i,j) element of B.

Example: >> A = [ 1 2; 3 4], B = [5 6; 7 8]

>> A.*B

>> A.^2

>> A./B

ARRAY OPERATION CONT.

• >> size(A); The result gives the size of A, i.e., the number of rows and columns.

• >> sum(A) ; This function calculates the column sums in a rectangular array A and gives the sums in a row vector. (A column sum is the sum of the elements of a column.) If A is a row vector, then the sum of the elements of A is calculated. The function

• >> prod(A); does the same for the product.

Other Array operations are: transpose() < lower than

> greater than == equal to

>= greater than or equal to ~= not equal to

<= lower than or equal to

STRING OPERATION

• A string matrix is like any other, except the elements in it are interpreted as ASCII numbers.

• To create a string variable, enclose a string of characters in ‘single’ quotation marks’

Ex: stg = 'This is a string';

x = ['ab' ; 'cd']

x = ['ab' 'cd']

MATRIX OPERATION

• Matlab treats all variables as matrices. For our purposes amatrix can be thought of as an array, in fact, that is how it isstored.

• Vectors are special forms of matrices and contain only onerow OR one column.

• Scalars are matrices with only one row AND one column.

• A matrix with only one row is called a row vector. A rowvector can be created in Matlab as follows (note thecommas):

» rowvec = [12 , 14 , 63]

rowvec =

12 14 63

MATRIX OPERATION CONT.

• A matrix with only one column is called a column vector.A column vector can be created in MATLAB as follows(note the semicolons):

» colvec = [13 ; 45 ; -2]

colvec =

13

45

-2


• A matrix can be created in Matlab as follows (note thecommas AND semicolons):

• » matrix = [1 , 2 , 3 ; 4 , 5 ,6 ; 7 , 8 , 9]

matrix =

1 2 3

4 5 6

7 8 9


EXTRACTING A SUB-MATRIX:

• A portion of a matrix can be extracted and stored in asmaller matrix by specifying the names of both matricesand the rows and columns to extract. The syntax is:

sub_matrix = matrix ( r1 : r2 , c1 : c2 ) ;

where r1 and r2 specify the beginning and ending

rows and c1 and c2 specify the beginning and

ending columns to be extracted to make the new

matrix.


EXTRACTING A SUB-MATRIX:


Accessing Single Elements of a Matrix

• A(i,j)

Accessing Multiple Elements of a Matrix

• A(1,4) + A(2,4) + A(3,4) + A(4,4) sum(A(1:4,4)) or

• sum(A(:,end))

• The keyword end refers to the last row or column.

Deleting Rows and Columns

• to delete the second column of X, use

• X(:,2) = []

Concatenating Matrices A and B

• C=[A;B]


Colon Operator


MATRIX FUNCTIONS:

• X = ones(r,c) % Creates matrix full with ones

• X = zeros(r,c) % Creates matrix full with zeros

• A = diag(x) % Creates squared matrix with

vector x in diagonal

• [r,c] = size(A) % Return dimensions of matrix A

• + - * / % Standard operations

• .+ .- .* ./ % Wise addition, substraction,…

• v = sum(A) % Vector with sum of columns

MATRIX FUNCTIONS:

• X = A’ % Transposed matrix

• X = inv(A) % Inverse matrix squared matrix

• d = det(A) % Determinant

• [X,D] = eig(A) % Eigenvalues and eigenvectors


MATRIX FUNCTIONS:• ones

ones matrix

• zeroszeros matrix

• sizesize of a matrix

• diagdiagonal elements of a matrix

• inv matrix inverse

• randuniformly distributed random numbers

• randn normally distributed random numbers

• cumprodcumulative product of elements

MATRIX OPERATION CONT.• max

largest component

• min smallest component

• sum sum of elements

• mean average or mean value

• median median value

• std standard deviation

• sort sort in ascending order

• find find indices of nonzero entries

• corrcoefcorrelation coefficients

• covcovariance matrix

M- FILES & FUNCTIONS IN MATLAB• Text files containing MATLAB programs. Can be called form

the command line or from other M-files.

• Present “.m” extension.

• Two kind of M-files:

1. Scripts

2. Functions

• A function file starts with a line declaring the function, its arguments and its outputs. There follow the statements required to produce the outputs from the inputs (arguments). That's it.

• Here is a simple example:

function y = port_val(holdings,prices) y = holdings*prices;

• Note that your function file name should be same as your function name with .m extension.

The main difference between a script and a function isthat a function accepts input from and returns outputto its caller, whereas scripts do not.

FAMILIARIZATION WITH IMAGE PROCESSING

TERMINOLOGIES

Definitions of an image:

• It is a combination of pixel values.

• 0 - means Black & 1- means white.

• Mathematically it is a 2-D function.

FAMILIARIZATION WITH IMAGE PROCESSING

TERMINOLOGIESDizitization

capturing an analog signal in digital form.

Sampling

converting a continuous signal into a

discrete signal.

Quantization

rounding process of samples to a fixed set of numbers (such as integers).

TYPES OF IMAGE

• Monochrome• An image displayed in a single color or shades of a single color.

• Least memory requirement

• Also known as 1-bit image, each pixel is stored as a bit.

• Gray Scale• A grayscale (or graylevel) image is simply one in which the only colors are shades of gray.

• Processing is faster & easier.

• Also known as 8-bit image, each pixel is usually stored as a byte (value between 0 to 255) .

• Colored• A (digital) color image is a digital image that includes color information for each pixel.

• It is the combination of the 3- primary colors (RGB).

• Also known as 24- bit image, each pixel contains 3 values i.e. R, G & B-value.

DIFFERENT PARAMETERS OF AN IMAGE

• PIXEL

Short for Picture Element,

A single point in a graphic image.

• RESOLUTION900 pixels ÷ 300ppi

= 3 inches wide

600 pixels ÷ 300ppi= 2 inches high

• TEXTUREThe feel of a surface or a fabricTexture mapping is a method for adding detailsor color to a computer generated graphic or 3-D model.

DIFFERENT CONCEPTS

• Luminance

The amount of brightness.

• Chrominance

The color information.

• HueThe property of colors by which they can be perceived as ranging

from red through yellow, green, and blue, as determined by the

dominant wavelength of the light.

• Intensity/SaturationThe strength of a color, especially the degree to which it lacks its complementary color.

Same chrominance , different luminance

Same luminance, different chrominance

DIFFERENT COLOUR SPACES

Color Space:

• RGB - Red Green Blue

• CMYK - Cyan Magenta Yellow Black

• HSV - Hue Saturation Value

• YIQ - Luminance Hue Saturation

• YCbCr - Luminance and Chrominance

DIFFERENT COLOUR SPACES CONT.

RGB Color Space:

• Additive Model

• Three primary Colors

• Design Simplification

• Computer Graphics


CMYK Color Space:

• Subtractive Model

• Printing applications


YIQ Color Space:

• Y – luminance

I – Hue (Orange – Blue range)

Q – Saturation (Purple – Green range)

• Grayscale information is separated from color data

• Histogram Equalization (can be applied on Y)

• Defined by NTSC

• Used in United States Televisions


YCbCr Color Space:

• Y - luminance

• Cb & Cr - Chrominance Information

• Widely Used in Digital Video

Y

Cb

Cr

HANDLING IMAGES IN MATLAB

There are two possible ways of handling image: • Using the Image Acquisition Tool box

• It is faster & flexible

• Available on MATLAB7.0 and above

Example: >>imaqhwinfo

>> vid = videoinput(‘winvideo’, 1);

>> preview(vid);

• Using VFM (Video for MATLAB)

• Add-on for all versions of MATLAB

• Once started you have to stop it along with MATLAB

• First of all you have to keep the vfm.m & vfm.dll file in your current directory (work), otherwise it won’t be work.

IMPORTING IMAGE

There are two possible ways of importing image: • Using webcam with Image Acquisition Tool box

Example: >> vid = videoinput(‘winvideo’, 1);

>> image=getsnapshot(vid);

>> imshow(image);

• Using webcam with VFM (Video for MATLAB)

Example: >> image=vfm(‘grab’);

>> imshow(image);

• Using the stored image from the hard disk

>> image=imread(‘ball.jpg’);

>> imshow (image);N.B.: Before using this you have to store ball.jpg in the current directory.

IMPORTING IMAGE CONT.

Graphical Representation of Importing an Image

IMAGE REPRESENTATION

• You can store/save an image by using imwrite();

• An image is stored as a matrix using standard Matlab matrix conventions.

• There are five basic types of images supported by Matlab:

• Grayscale • True color RGB • Indexed • Binary • 8-bit images/Unit8

IMAGE REPRESENTATION CONT.• Functions for conversion between different color spaces:

FINDING PIXEL VALUE

• Use pixval to find out the information about pixels.

• Use the command imview(‘xyz.jpg’) & move the cursor over the image which will show you the pixel value.

• Use the following commands:

• >> i=imread('balls.bmp');

>> p=impixel(i);

>> disp(p);

Write click on image

OPERATIONS USING PIXEL VALUE• The value of a single pixel in an image is indexed as:

p(m,n)

• Multiple pixels can be indexed with a vector as: p([1 3],1)

which selects pixels p(1,1) and p(3,1).

• Multiple pixels also can be indexed using a range. The first row is all pixels with row index 1 and any column index:

p(1,1:N) The notation 1:N indicates all indexes from 1 to N.

• The notation can contain a step value, e.g., 1:2:5 indicates indexes 1, 3, 5.

OPERATIONS USING PIXEL VALUE CONT.• To change the color value of a pixel use:

>> y=imread('balls.bmp');>> y(100,155)=5;>> y(100,154)=5;>> imview(y);

• Filling an ROI

Filling is a process that fills a region of interest (ROI) by interpolating the pixel values from the borders of the region.

>> I = imread('ballgray.bmp');

>> imshow(I);

>> I2 = roifill;

>> imshow(I2);

IMAGE MATRIX• Any image (supported format) can be stored in MATLAB in the form of a matrix.

• To show how an image is being stored in a matrix:

>> I = imread('balls.bmp');

>> size(I)

>> I(1:10,1:10)

• To resize the image, rotate to 45° & crop to original size:


>> I2 = imresize(I, 0.5, 'bil');

>> imshow(I2);

>> I3 = imrotate(I2, 45,'bil', 'crop');

>> imshow(I3);

IMAGE PROCESSING APPROACHES

• Image Processing generally involves extraction of useful information from an image.

• The basic block diagram of image processing system:

Database of images Pre-processing

Image acquisition Feature extraction

Data communication

Decision Making

FUNCTIONS TO ENHANCE IMAGES

• imadjust - Adjust image intensity values>> RGB1 = imread('balls.bmp');>> RGB2 = imadjust(RGB1,[.2 .3 0; .6 .7 1],[]);>> imshow(RGB1), figure, imshow(RGB2)

• histeq - Enhance contrast using histogram equalization>> I = imread('ballgray.bmp');>> J = histeq(I);>> imshow(I)>> figure, imshow(J)

Some other functions: imnoise ‐ Add noise to an image.medfilt2 ‐ Perform 2‐D median filtering.ordfilt2 ‐ Perform 2‐D order‐statistic filtering.stretchlim ‐ Find limits to contrast stretch an image.wiener2 ‐ Perform 2‐D adaptive noise‐removal filtering.

FUNCTIONS TO ENHANCE IMAGES CONT.

• Filtering the image:• Filtering is a technique for modifying or enhancing an image.

• Image processing operations implemented with filtering include smoothing, sharpening, and edge enhancement.


>> h = fspecial('unsharp');

>> I2 = imfilter(I,h);

>> imshow(I), title('Original Image')

>> figure, imshow(I2),title(‘Filtered Image’)

• Other Filters are:‘average’ 'log’

'disk’ 'motion’

'gaussian’ 'prewitt’

'laplacian’ 'sobel’

'unsharp’

THRESHOLDING

• Thresholding is a non-linear operation that converts a gray-scale image into a binary image where the two levels are assigned to pixels that are below or above the specified threshold value.

• You can apply a threshold to data directly from the command line,

e.g., BW = im2bw(I, level)

• Automatically calculate a threshold value using an iterative method. >>

>>x=imread('balls.bmp'); >> x=imread('balls.bmp');

>> BW = im2bw(x,0.4); >> Y = zeros(194,250);

>> imshow(x), figure, imshow(BW) >> for(i=1:194)

>>for(j=1:250)

>>if(x(i,j) > 50)

>>Y(i,j) = 1;

>>end

>>end

>>end

>> imshow(Y);

IMAGE ARITHMATIC

NOISE & NOISE REDUCTION

• Where does noise come from?• Scanner resolution

• Film grain (granularity)

• Hardware (interference patterns)

• Other

• Common type of noise:• Whitenoise (Gaussian)

• Local variance (Gaussian with intensity-dependent

variance)

• Salt and pepper

• Speckle

NOISE & NOISE REDUCTION CONT.

Modeling of noise:

The function imnoise allows different types of noise to be modeled.

Syntax:J = imnoise(I,type,parameters)

I – Image

type – gaussian, localvar, poisson, salt & pepper, speckle

parameters – additional parameters needed

given the type of noise


• The Image Processing Toolbox provides three main methods of filtering:

• Linear filtering

A linear filter computes each output pixel value according to a linear combination of

the input pixel's neighborhood. The basics of linear filtering are done through correlation and convolution.

• Median filtering

When noise causes the pixels to vary greatly from the original value (salt and

pepper), a median filter is more effective in reducing the noise.

• Adaptive filtering

This type of filter is effective in reducing the effects of Gaussian whitenoise.


• Program to add & remove noise in an image:• Adding noise to an image:

>> I = imread('ballgray.bmp');

J = imnoise(I,'salt & pepper',0.05);

figure, imshow(I);

figure, imshow(J);

• Removing noise from an image:

>> I = imread(‘noisyball.bmp');

B = medfilt2(J,[2 3]);

figure, imshow(I);

figure, imshow(B);

EDGE DETECTION

• Edge detection is nothing but a filtering process.

• The edge function of the Image Processing Toolbox makes it easy to perform edge detection.

• Followings are the different edge detection functions:

• sobel

• prewitt

• roberts

• log (laplacian)

• Canny (The perfect one)

• zerocross

EDGE DETECTION CONT.

• Edge information in an image is found by looking at the relationship a pixel has with its neighborhoods.

• If a pixel’s gray-level value is similar to those around it, there is probably not an edge at that point.

• If a pixel’s has neighbors with widely varying gray levels, it may present an edge point.

• Many are implemented with convolution mask and based on discrete approximations to differential operators.

Sobel Mask

Prewitts MaskRoberts Mask

HARDWARE DETAILS

• CMOS webcam:

• Video Format : 24bit RGB

• 30 Frames/sec

• Focus Range : 3 centimeters To Limitless

DATA COMMUNICATION USING MATLAB

• MATLAB provides support to access serial port (also called as COM port) and parallel port (also called as printer port or LPT port) of a PC.

• Serial port:If you have to transmit one byte of data, the serial port will transmit 8 bits as one bit

at a time. The advantage is that a serial port needs only one wire to transmit the 8 bits

(while a parallel port needs 8).

• Parallel port:Parallel port has 25 pins Pins 2-9 are bi-directional data pins (pin 9 gives the most significant bit (MSB)), pins 10-13 and 15 are output pins (status pins), pins 1,14,16,17 are input pins (control pins), while pins 18-25 are Ground pins.

ACCESSING THE PC’s SERIAL PORT

• MATLAB code for accessing the serial port is as follows:

>> ser= serial('COM1','BaudRate',9600,'DataBits',8);

>> fopen(ser);

• To send data through the serial port, the available commands

>> fwrite (ser,1); % for left motion

>> fwrite (ser,2); % for right motion

• You can close the port in case there are other applications using this port using the fclose command.

>> fclose(ser);

ACCESSING THE PC’s PARALLEL PORT

• Parallel port interfacing:• To access the parallel port in MATLAB, define an

object:

>> parport= digitalio('parallel','LPT1');

• You may obtain the port address using,

>> get(parport,'PortAddress')

>> daqhwinfo('parallel'); % To get data acquisition

hardware information

ACCESSING THE PC’s PARALLEL PORT CONT.

• You have to define the pins 2-9 as output pins, by using addline function

>> addline(parport, 0:7, 'out')

• Now put the data which you want to output to the parallel port into a matrix; e.g.

>> dataout = logical([1 0 1 0 1 0 1 1]);

• Now to output these values, use the putvalue function

>> putvalue(parport,dataout);

• Alternatively, you can write the decimal equivalent of the binary data and output it.

>> data = 23;

DEVELOPMENT OF IMAGE PROCESSING ROBOTS

• Amount of data to be processed:If we have each frame to be of size 320x240 pixels for an RGB image the amount of data produced by each bit will be,

• Each color (R,G,B) will be represented by 8 bits.

• Hence, each pixel will be of 8x3=24 bits

• Now, total number of pixels are 320x240=76,800 pixels

• Hence, each frame will be of 76800x24=18,43,200 bits

• Hence, in terms of bytes the volume of data generated will be 18,43,200/8=230KB

• If we need to capture around 15 frames per second, the amount of data that will be generated per second will be approximately 4MB.

DEVELOPMENT OF IMAGE PROCESSING ROBOTS CONT.

• Why conventional microcontrollers can’t be used for image processing?

• Limited throughput

• Low clock frequency (up to around 32Mhz)

• Lack of powerful Instruction set required for handling arrays

• Smaller word length handling capability

• Limited on-chip memory (RAM)

DEVELOPMENT OF IMAGE PROCESSING ROBOTS CONT.

• Analyse the problem statement:

• Overhead

• Onboard

• Finalize the hardware required:

• Processor for image processing & navigation

• Camera/Image acquisition device

• Machine with motors and motor drivers

• Processor to/from machine communication

COLOR DETECTION ROBOT

• Capture the image.• Select which colour to detect.

• Set the threshold value accordingly, that’s it.

• How to set the threshold value?• Capture few live images of the ball

• Using MATLAB (imview) analyze the pixel values

• Take a mean value

• Observe variation in pixel values with changing light conditions

• Light conditions keep on varying with every captured frame

• Try using a color space where color information (chrominance) and light

(luminance) can be separated.

• So which color space to use?

COLOR DETECTION ROBOT CONT.MATLAB code:

clc

clear

while(1)

RGB=vfm('grab');

mode=input('Enter which color to detect: 1-Red, 2-Green, 3-Blue: ');

thres=input('Enter the threshold value: ');

[M,N,t] = size(RGB);

switch (mode)

case 1

I1 = zeros(M,N); I2 = zeros(M,N);

I1( find(RGB(:,:,1) > thres * RGB(:,:,2)) ) = 1;


strTitle = 'Color RED detected (white areas)';

I = I1 .* I2;

COLOR DETECTION ROBOT CONT.case 2




strTitle = 'Color GREEN detected (white areas)';

I = I1 .* I2;

case 3




strTitle = 'Color BLUE detected (white areas)';

I = I1 .* I2;

otherwise

fprintf('Enter a valid choice for color'\n);

return;

end

sub

plo

t(2,1

,1),im

sho

w(R

GB

); title('O

riginal Im

age');

sub

plo

t(2,1

,2),im

sho

w(I,[]); title

(strTitle);

en

d

BALL FOLLOWER ROBOT

• Place the ball in such a distance that centre of ball and centre of frame should coincide.

• According to the color of the ball set the threshold value which will separate the background from the ball.

• Set the values for forward/backward movement through parallel port communication.

BALL FOLLOWER ROBOT CONT.

Ideal position of the ball Distinguishing between BG color & ball color by

setting proper threshold value

• The diameter of the ball when it is at the ideal position is always fixed

Mark all pink pixels as ‘1’ and other pixels as ‘0’


Problematic situation:

• Can

• be tackled by selecting appropriate threshold

• Can be solved by selecting appropriate threshold value.


• Deciding forward/backward & left/right movement:

• Diameter will decide either to go forward/backward.

• The distance between frame center & ball CG will decide right/left turn.

Dia.


• How to prevent the machine from oscillation?

Eg:

• Select a no-action range so that the machine will not oscillate.


• MATLAB code: Flie name: main.mclc

clear

%cb=110;%orange ball

%cr=160;%orange ball

%cb=125;%pink ball

%cr=160;%pink ball

cb=90;%yellow ball

cr=115;%yellow ball

while(1)

image=vfm('grab');

image_ycbcr=rgb2ycbcr(image);

image_cb=image_ycbcr(:,:,2);

image_cr=image_ycbcr(:,:,3);

[r c d]=size(image_ycbcr);

output_image=zeros(r,c);

for i1=1:r

for i2=1:c

if (image_cr(i1,i2)>cr && image_cb(i1,i2)<cb)

output_image(i1,i2)=1;

else

output_image(i1,i2)=0;

end

end

end

imshow(output_image)

BALL FOLLOWER ROBOT CONT.[r_cent c_cent]=centroid1(output_image);

total_pix=sum(sum(output_image));

if(total_pix<700)

stop();

disp('No Ball');

else

if (c_cent<120)

left();

disp('left()');

elseif (c_cent>200)

right();

disp('right()');

else

stop();

disp('stop()');

end

if (r_cent<90)

disp('forward()');

forward();

stop();

forward();

stop();

elseif (r_cent>150)

disp('back()');

back();

stop();

back();

stop();

else

stop();

disp('stop');

end

end

end


• File name: centroid1.m:

function [r_cent c_cent]=centroid1(arg)

[r_index c_index]=find(arg);

r_cent=mean(r_index);

c_cent=mean(c_index);

• File name: forward.m

function [ ]=forward()

parport=digitalio('parallel','LPT1');

line=addline(parport,0:7,0,'out');

pval=[1 0 1 0 0 0 0 0];

putvalue(parport,pval);

• File name: delay.mfunction[ ]=delay(x)

%tic

for i=1:500

for j=1:500

for k=1:x %if u give x=420 u will get a delay of 1 second.

end

end

end

%toc

LINE FOLLOWER ROBOT

• All the steps discussed above for ball follower is applicable here.

• But here the only change you have to make is for forward/backward movement, you have to take care of the changes in the x- direction.

• You have to control the speed of the robot by generating PWM signal.

• Use right spin, left spin instead of right turn, left turn.

LINE FOLLOWER ROBOT CONT.• MATLAB code:

clcclearcb=127;cr=0;while(1)

image=vfm('grab');image_ycbcr=rgb2ycbcr(image);image_cb=image_ycbcr(:,:,2);image_cr=image_ycbcr(:,:,3);[r c d]=size(image_ycbcr);output_image=zeros(r,c);for i1=1:r

for i2=1:cif (image_cr(i1,i2)>cr &&

image_cb(i1,i2)<cb)output_image(i1,i2)=1;

elseoutput_image(i1,i2)=0;

endend

end

imshow(output_image)[r_cent c_cent]=centroid1(output_image);

total_pix=sum(sum(output_image));disp(total_pix);if(total_pix<7000)

% stop();disp('No Line');

elseif (c_cent<120)

left();left();disp(‘left()');disp(c_cent);disp(r_cent);

LINE FOLLOWER ROBOT CONT.• MATLAB code:

elseif (c_cent>200)

right();right();disp(‘right()');disp(c_cent);disp(r_cent);

else

end

if (r_cent<90)back();disp('back()');disp(c_cent);disp(r_cent);

elseif (r_cent>120)forward();stop();forward();stop();disp('forward()');disp(c_cent);disp(r_cent);

elsedisp('stop()');disp(c_cent);disp(r_cent);

endend

end

new machin vision

Documents