bt 0080 fundamentals of algorithms contents · mr. deepak shetty mr. vinayak g pai assistant...
Post on 31-Jul-2020
2 Views
Preview:
TRANSCRIPT
BT 0080
Fundamentals of Algorithms
Contents
Unit 1
Elementary Algorithms 1
Unit 2
Mathematical Functions and Notations 30
Unit 3
Divide and Conquer 79
Unit 4
Greedy Method 97
Unit 5
Dynamic Programming 113
Unit 6
Backtracking 123
Unit 7
Branch and Bound 138
Unit 8
Additional Features 148
Unit 9
Graphs 163
Unit 10
Trees and Algorithms 177
Edition: Spring 2009
BKID – B1092 9th
Oct. 2009
Unit 11
Traversability 206
Unit 12
Representation of Graphs 216
Unit 13
Directed Graphs 231
Unit 14
Complexity of Algorithms 250
Acknowledgement, References and Suggested Readings 266
Director & Dean Directorate of Distance Education Sikkim Manipal University of Health, Medical & Technological Sciences (SMU-DDE)
Board of Studies Dr. U. B. Pavanaja (Chairman) Mr. Nirmal Kumar Nigam General Manager – Academics HOP – IT Manipal Universal Learning Pvt. Ltd. SMU-DDE, Manipal. Bangalore.
Prof. Bhushan Patwardhan Dr. A. Kumaran Chief Academics Research Manager (Multilingual) Manipal Education Microsoft Research Labs India Bangalore. Bangalore.
Dr. Harishchandra Hebbar Mr. Ravindranath P. S. Director Director (Quality) Manipal Centre for Info. Sciences, Bangalore Yahoo India, Bangalore
Dr. N. V. Subba Reddy Dr. Ashok Kallarakkal HOD-CSE Vice President Manipal Institute of Technology, Manipal IBM India, Bangalore
Dr. Ashok Hegde Mr. H. Hiriyannaiah Vice President Group Manager MindTree Consulting Ltd., Bangalore EDS Mphasis, Bangalore
Dr. Ramprasad Varadachar Director, Computer Studies Dayanand Sagar College of Engg., Bangalore.
Content Preparation Team
Content Writing Content Editing Mr. Deepak Shetty Mr. Vinayak G Pai Assistant Professor – Mathematics Assistant Professor, Dept. of IT SMU-DDE, Manipal. SMU-DDE, Manipal.
Instructional Design Language Editing Mr. Kulwinder Pal Ms. R. K. Rao Assistant Professor – Education Lecturer - English SMU-DDE, Manipal. Upendra Pai Memorial College, Udupi.
Edition: Spring 2009
This book is a distance education module comprising a collection of learning material for our students. All rights reserved. No part of this work may be reproduced in any form by any means without permission in writing from Sikkim Manipal University of Health, Medical and Technological Sciences, Gangtok, Sikkim. Printed and published on behalf of Sikkim Manipal University of Health, Medical and Technological Sciences, Gangtok, Sikkim by Mr.Rajkumar Mascreen, GM, Manipal Universal Learning Pvt. Ltd., Manipal – 576 104. Printed at Manipal Press Limited, Manipal.
SUBJECT INTRODUCTION
The subject, Fundamentals of Algorithms(BT 0080), of four credits, contains
the various algorithms and approaches to solve problems in different
techniques. The time complexity for various algorithms is also considered
and explained in a simple manner. The different models for executing
algorithms is explained with well illustrated examples wherever necessary.
This book has been split into 14 units, the overview of which is given below:
Unit 1: Elementary Algorithms
This unit gives the different ways of expressing algorithms, the building
blocks of algorithms and their characteristics is explained in a simple
manner.
Unit 2: Mathematical Functions and Notations
This unit explains in brief the different types of functions and their notations.
The different types of Asymptotic functions and their notations is well
explained with standard examples.
Unit 3: Divide and Conquer
In this unit, we learn the divide and conquer method. Using this method, we
find the Algorithm for Binary Search, Max and Min, Merge Sort, and Quick
Sort.
Unit 4: Greedy Method
In this unit, we use the Greedy Method for Optimistic storage on tapes,
Knapsack Problem, Job Sequencing with dead lines, Optimal Merge Pattern
and Single Source Shortlist Paths.
Unit 5: Dynamic Programming
This unit deals with Dynamic Programming approach to multistage graphs,
All Pair Shortlist Paths and Travelling Salesman Problem.
Unit 6: Backtracking
In this unit, we study about Back Tracking method. In the search for
fundamental principles of algorithm design, backtracking represents one of
the most general techniques. We use Backtracking approach to solve 8-
queen problem, Sum of Subsets problem and Knapsack problem.
Unit 7: Branch and Bound
In this unit, we deal with branch and bound method and use this approach to
solve O/I Knapsack problem and Travelling Salesman problem. The Branch
and Bound refers to all state space search methods in which all children of
the E-node are generated before any other live node can become the E-
node.
Unit 8: Additional Features
In this unit, we discuss the lower boundary theory and its implications. The
concept of NP Hard and NP Complete is briefly discussed here. The
different types of Approximate Algorithms for NP Hard Problems are studied
here.
Unit 9: Graphs
In this unit, we study the Preliminary definitions of graphs. The different
Notations used in graphs are also discussed here. The concept of trees is
explained in a simple manner.
Unit 10: Trees and Algorithms
In this unit, we study the Characterization of Trees. The Rooted Trees and
its applications is discussed here with simple examples. Spanning Trees
and the algorithms is briefly discussed here with necessary steps.
Unit 11: Traversability
In this unit, we study the concept of Eulerian Graphs with application to
Hamiltonian Paths and circuits. The travelling salesman problem is briefly
discussed with application to traversability.
Unit 12: Representations of Graphs
In this unit, we discuss the concept of Adjacency and Incidence Matrix. The
application of the above concepts to Circuit Matrix and Path Matrix is
illustrated here.
Unit 13: Directed Graphs
In this unit, we study different types of Directed Graph. The concept of
Euler’s Digraphs and Matrix Representation of digraphs is illustrated with
some simple examples.
Unit 14: Complexity of Algorithms
In this unit, we discuss the Notations for the Growth Rates of Functions. We
study here the different methods of establishing NP-Completeness of
Problems.
Objectives of studying the subject:
After studying this subject you should be able to:
write the procedures in a simple manner
use a number of Mathematical Notations like , , , , mod, log, e etc
apply divide and conquer strategy to practical problems
define the Job Sequencing problem using Greedy Method
apply Dynamic Programming in Multistage graphs
analyze the 8-queen problem
solve the O/I Knapsack Problem
apply the Lower Boundary theory
use the definition of trees and properties of trees
apply the concept of trees in real life situations
identify an Euler graph
write the matrix representations of a given graph
apply the properties to flows, network and traffic problem
define a number of problems which are known to be NP-Complete
problems.
The different examples for the algorithms given in this book will be given in
the Edunxt Portal.
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 1
Unit 1 Elementary Algorithms
Structure
1.1 Introduction
Objectives
1.2 Notation for Expressing Algorithms
1.3 Role and Notation for Comments
1.4 Example of an Algorithm
1.5 Problems and Instances
1.6 Characteristics of an Algorithm
1.7 Building Blocks of Algorithms
1.8 Procedure and Recursion
Procedure
Recursion
1.9 Outline of Algorithmics
1.10 Specification Methods for Algorithms
1.11 Summary
1.12 Terminal Questions
1.13 Answers
1.1 Introduction
In this unit, you are going to study about the concepts that are basically
needed to frame an elementary algorithm, which you have already
discussed in your earlier classes.
Problem may be a state of mind of a living being, of not being satisfied with
some situation. However, for our purpose, we may take the unsatisfactory /
unacceptable / undesirable situation itself, as a problem.
One way of looking at a possible solution of a problem is, as a sequence of
activities (if such a sequence exists at all), that if carried out using allowed
available tools, leads us from the unsatisfactory (initial) position to an
acceptable, satisfactory or desirable position. For example, the solution of
the problem of baking delicious pudding may be thought of as a sequence of
activities, that when carried out, gives us the pudding (the desired state)
from the raw materials that may include sugar, flour and water (constituting
the initial position) using cooking gas, oven and some utensil etc. (the tools).
The sequence of activities when carried out gives rise to a process.
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 2
Technically, the statement or description in some notation of the process is
called an algorithm, the raw materials are called the inputs and the resulting
entity (in the above case, the pudding) is called the output. In view of the
importance of the concept of algorithm, we repeat.
An algorithm is a description or statement of a sequence of activities that
constitute a process of getting the desired outputs from the given inputs.
An algorithm, when expressed in a notation that can be understood and
executed by a computer system is called a computer program or simply a
program. We should be clear about the distinction among the terms viz., a
process, a program and an algorithm.
A process is a sequence of activities actually being carried out to solve a
problem. But algorithm and programs are just description of a process in
some notation. Further, a program is an algorithm in a notation that can be
understood and be executed by a computer system.
It may be noted that for some problems and the available tools, there may
not exist any algorithm that should give the desired output. For example, the
problem of baking delicious pudding may not be solvable, if no cooking gas
or any other heating substance is available. Similarly, the problem of
reaching the moon is unsolvable, if no spaceship is available for the
purpose.
Objectives
After studying this unit, you should be able to:
use notations for expressing algorithms
write the comments wherever necessary
solve the problems and instances connected to algorithms
apply the different characteristics of an algorithm
use the different building blocks in algorithms in an effective manner
write the procedures in a simple manner
use the concept of recursion effectively in mathematical concepts
use the different outlines of algorithms wherever necessary.
1.2 Notation for Expressing Algorithms
This issue of notation for representations of algorithms will be discussed in
some detail, later. However, mainly, some combinations of mathematical
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 3
symbols, English phrases and sentences, and some sort of pseudo – high –
level language notations, shall be used for the purpose.
The symbol ‘ ’ is used for assignment. For example, x y + 3, means that
3 is added to the value of the variable y and the resultant value becomes the
new value of the variable x. However, the value of y remains unchanged.
If in an algorithm, more than one variables are required to store values of
the same type, notation of the form A [1..n] is used to denote n variables
A[1], A[2], …, A[n].
In general, for the integers m, n with m<n, A[m, n] is used to denote the variables
A[m], A[m+1], …., A[n]. However, we must note that another similar notation
A[m, n] is used to indicate the element of the matrix (or two – dimensional
array) A, which is in mth row and nth column.
1.3 Role and Notation for Comments
Comments help the human reader of the algorithm to better understand the
algorithm. In different programming languages, there are different notations
for incorporating comments in algorithms. We use the convention of putting
comments between pair of braces i.e., { }. The comments may be inserted at
any place within an algorithm. For example, if an algorithm finds roots of a
quadratic equation, then we may add the following comments, somewhere
in the beginning of the algorithm, to tell what the algorithm does.
{this algorithm finds the roots of a quadratic equation in which the coefficient
of x2 is assumed to be non – zero}.
Self Assessment Questions
1. Comments help the human reader of the algorithm to better understand
the ––––––––––––.
2. The symbol –––––––––––– is used for assignment.
1.4 Example of an Algorithm
Before going in to the detail of problem - solving with algorithms, just to have
an idea of what an algorithm is, we consider a well – known algorithm for
finding Greatest Common Divisor (G.C.D) of two natural numbers and also
mention some related historical facts. First, the algorithm is expressed in
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 4
English. Then, we express the algorithm in a notation resembling a
programming language.
Euclid‘s Algorithm for finding G.C.D of two Natural Numbers m and n
El: {Find remainder}. Divide m by n and let r be the (new) remainder
{we have 0 < r < n}
E2: {is r zero ?} If r = 0, the algorithm terminates and n is the answer.
Otherwise,
E3: {Interchange} Let the new value of m be the current value of n and the
new value of n be the current value of r. Go back to Step E1.
The termination of the above method is guaranteed, as m and n must
reduce in each iteration and r must become zero in finite number of
repetitions of steps E1, E2, and E3.
The great Greek mathematician Euclid sometimes between fourth and third
century BC, at least knew and may be the first to suggest, the above
algorithm. This algorithm is considered as the first among non – trivial
algorithms. However, the word ‗algorithm‘ itself came into usage quite late.
The word is derived form the name of the Persian mathematician
Mohammed al – Khwarizmi who lived during the ninth century A.D. The
word ‗al – khowarizmi‘ when written in Latin became ‗algorismus‘, from
which ‗algorithm‘ is a small step away.
In order to familiarize ourselves with the notation usually used to express
algorithms, next, we express the Euclid‘s algorithm in a pseudo – code
notation which is closer to a programming language.
Algorithm GCD – Euclid (m, n)
{This algorithm computes the greatest common divisor of two given positive
integer}
begin {of algorithm}
while n 0 do
begin {of while loop}
r m mod n;
{a new variable is used to store the remainder which is obtained by dividing
m by n, with 0 < r < m}
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 5
m n;
{the value of n is assigned as new value of m; but at this stage, value of n
remains unchanged}
n r,
{the value of r becomes the new value of n and the value of r remains
unchanged}
end {of while loop}
return (n)
end; {of algorithm}
Self Assessment Question
3. ―A new variable is used to store the remainder which is obtained by
dividing m by n, with 0 < r < m”. This is denoted by ––––––––––––– .
1.5 Problems and Instances
The difference between the two concepts viz., ‗problem‘ and ‗instance‘, can
be understood in terms of the following examples. An instance of a problem
is also called a question‘. We know that the roots of a general quadratic
equation.
0cbxax2 a 0 (1)
are given by the equation
a2
ac4bbx
2
(2)
Where a, b, c, may be any real numbers except the restriction that a 0.
Now, if we take a = 3, b = 4 and c = 1,
we get the particular equation
01x4x3 2 (3)
Using (2), the roots of (3) are given by
,6
24
32
13444 2
1or3
1x
With reference to the above discussion, the issue of finding roots of the
general quadratic equation ax2 + bx + c = 0 with a 0 is called a problem,
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 6
whereas the issue of finding the roots of the particular equation 3x2 + 4x + 1 = 0
is called a question or instance of the (general) problem.
In general, a problem may have a large, possibly infinite, number of
instances. The above mentioned problem of finding the roots of the
quadratic equation.
ax2 + bx + c = 0 with a 0, b and c as real numbers, has infinitely many
instances, each obtained by giving some specific real values to a, b and c,
taking care that the value assigned to a is not zero. However, all problems
may not be of generic nature. For some problems, there may be only one
instance / question corresponding to each of the problems. For example, the
problem of finding out the largest integer that can be stored or can be
arithmetically operated in a given computer is a single-instance problem.
Many of the interesting problems like the ones given below, are just single-
instance problems.
Problem (i): Crossing the river in a boat which can carry at one time, along
with the boatman only one wolf, a horse and a bundle of grass, in such a
way that neither wolf harms, nor horse eats grass. In the presence of the
boatman, neither wolf attacks horse, nor horse attempts to eat grass.
Problem (ii): The Four-Colour Problem which requires us to find out
whether a political map of the world can be drawn using only four colours,
so that no two adjacent countries get the same colour.
Coming back to the problem of finding the roots of a quadratic equation, it
can be easily seen that in finding roots of a quadratic equation, the only
operations that have been used are plus, minus, multiplication and division
of numbers along with the operation of finding out the square root of a
number Using only these operations, it is also possible through step by step
method, to find the roots of a cubic equation over the numbers, which, in
general is of the form
0dcxbxax 23
where, a 0, b, c and d are real numbers.
Further, using only the set of operations mentioned above, it is also
possible, through a step by step method to solve a biquadratic equation over
real numbers, which in general, is of the form
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 7
0edxcxbxax 234
Where, a 0, b, c, d and e are real numbers.
However, the problem of finding the roots of a general equation of degree
five or more, cannot be solved using only the operations mentioned above,
through a step by step method i.e., can not be solved algorithmically.
In such cases, we may attempt some non-algorithmic methods including
solutions based on numerical methods which may not be exact but will be
some good approximate solutions to such problems. Or we may just use
some hit and trial method, e.g. consisting of guessing a possible root and
then verifying the guess as a possible solution, by actually substituting the
guessed root in the given equation. A hit and trial method is not an
algorithm, because we cannot guarantee the termination of the method, as
discussed later. Termination is one of the characteristic properties of an
algorithm.
It may be noted that a (general) problem, like finding the roots of an
equation of degree 5 or more, may not be solvable algorithmically, i.e.,
through some step-by-step method, still it is possible for some (particular)
instances of the problem to have algorithmic solutions. For example, the
roots of the equation 032x5 are easily available through a step by step
method. Also, the roots of the equation 01x3x2 36 can be easily
found through a method, in which, to begin with, we may take 3xy .
Self Assessment Question
4. Four-Colour Problem requires us to find out whether a political map of
the world, can be drawn using only ––––––––––––.
1.6 Characteristics of an Algorithm
Next, we consider the concept of algorithm in more detail. While designing
an algorithm as a solution to a given problem, we must take care of the
following five important characteristics of an algorithm.
Finiteness
An algorithm must terminate after a finite number of steps and further each
step must be executable in finite amount of time. In order to establish a
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 8
sequence of steps as an algorithm, it should be established that it
terminates (in finite number of steps) on all allowed inputs.
Definiteness (no ambiguity)
Each step of an algorithm must be precisely defined; the action to be carried
out must be rigorously and unambiguously specified for each case.
However, the method is not definite, as two different executions may yield
different outputs.
Inputs
An algorithm has zero or more but only finite, number of inputs.
Examples of algorithm requiring zero inputs:
i) Print the largest integer, say MAX, representable in the computer
system being used.
ii) Print the ASCII code of each of the letters in the alphabet of the
computer system being used.
iii) Find the sum S of the form I + 2 + 3 +……… Where S is the largest
integer less than or equal to MAX defined in example (i) above.
Outputs
An algorithm has one or more outputs. The requirement of at least one
output is obviously essential. Otherwise we can not know the answer/
solution provided by the algorithm.
The outputs have specific relation to the inputs, where the relation is by the
algorithm.
Effectiveness
An algorithm should be effective. This means that each of the operations to
be performed in an algorithm must be sufficiently basic that it can, in
principle, be done exactly and in a finite length of time, by a person using
pencil and paper. It may be noted that the ‗FINITENESS‘ condition is a
special case of ‗EFFECTIVENESS‘. If a sequence of steps is not finite, then
it cannot be effective also.
A method may be designed which is a definite sequence of actions but is
not finite (and hence not effective).
Example: If the following instruction is a part of an algorithm: Find exact
value of e using the following formula.
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 9
.....!3
1
!2
1
!1
11e and add it to x.
Then, the algorithm is not effective, because as per instruction, computation of e
requires computation of infinitely many terms of the form!n
1 for n =1, 2, 3, ……,
which is not possible/ effective.
However, the instruction is definite as it is easily seen that computation of
each of the term !n
1 is definite (at least for a given machine).
Self Assessment Question
5. An algorithm has one or more ––––––––––––
1.7 Building Blocks of Algorithms
Next, we enumerate the basic actions and corresponding instructions used
in a computer system based on Von Neumann architecture. We may recall
that an instruction is a notation for an action and a sequence of instructions
defines a program whereas a sequence of actions constitutes a process. An
instruction is also called a statement.
The following three basic actions and corresponding instructions form the
basis of any imperative language. For the purpose of explanations, the
notation similar to that of a high — level programming language is used.
Basic Actions & Instructions
i) Assignment of a value to a variable is denoted by
Variable expression;
Where the expression is composed from variable and constant operands
using familiar operators like +, –, * etc.
Assignment action includes evaluation of the expression on the R.H.S.
An example of assignment instruction/ statement is
j 2 * i + j – r;
It is assumed that each of the variables occurring on R.H.S of the above
statement, has a value associated with it before the execution of the
above statement. The association of a value to a variable, whether
occurring on L.H.S or on R.H.S, is made according to the following rule.
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 10
For each variable name, say i, there is a unique location, say loc 1 (i), in
the main memory. Each location loc (i), at any point of time contains a
unique value say v (i). Thus the value v (i) is associated to variable i.
Using these values, the expression on R.H.S is evaluated. The value so
obtained is the new value of the variable on L.H.S. This value is then
stored as a new value of the variable (in this case, j) on L.H.S. It may be
noted that the variable on L.H.S (in this case, j) may also occur on R.H.S
of the assignment symbol.
In such cases, the value corresponding to the occurrence on R.H.S (of j,
in this case) is finally replaced by a new value obtained by evaluating
the expression on R.H.S (in this case, 2 * i + j – r)
The values of the other variables, viz., i and r remain unchanged due to
assignment statement.
ii) The next basic action is read values of variables i, j, etc. from some
secondary storage device, the identity of which is (implicitly) assumed here,
by a statement of the form read (i j, );
The values corresponding to variables i, j, ... in the read statement, are,
due to read statement, stored in the corresponding locations loc (i), loc(j)
...., in the main memory. The values are supplied either, by default,
through the keyboard by the user or from some secondary external
storage. In the latter case, the identity of the secondary external storage
is also specified in the read statement.
iii) The last of the three basic actions is to deliver/ write values of some
variables say i, j etc. to the monitor or to an external secondary storage
by a statement of the form
Write (i, j,….);
The values in the locations loc (i), loc(j),……. corresponding to the variables
i, j….., in the write statement are copied to the monitory or a secondary
storage. Generally, values are written to the monitor by default. In case, the
values are to be written to a secondary storage, then identity of the
secondary storage is also specified in the write statement. Further, if the
argument in a write statement is some sequence of characters enclosed
within quotes then the sequence of characters as such, but without quotes,
is given as output. For example, corresponding to the write statement.
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 11
Write (‗This equation has no real roots‘)
The algorithm gives the following output:
This equation has no real roots.
In addition to the types of instructions corresponding to the above
mentioned actions, there are other non – executable instructions which
include the ones that are used to define the structure of the data to be used
in an algorithm.
Control Mechanism and Control structures
In order to understand and to express an algorithm for solving a problem, it
is not enough to know just the basic actions viz., assignment, reads and
writes. In addition we must know and understand the control mechanisms.
These are the mechanisms by which the human beings and the executing
system become aware of the next instruction to be executed after finishing
the one currently in execution. The sequence of execution of instructions
need not be the same as the sequence in which the instruction occurs in
program text. First, we consider three basic control mechanisms or
structuring rules, before considering more complicated ones.
(i) Direct sequencing: When the sequence of execution of instructions is
to be the same as the sequence in which instruction are written in
program text, the control mechanism is called direct sequencing. Control
structure, (i.e.,) the notation for the control mechanism for direct
sequencing is obtained by writing off the instructions,
One after the other on successive lines, or even on the same line if
there is enough space on a line, and
Separated by some statement separator, say semi-colons, and in the
order of intended execution.
For example, the sequence of the next lines
A;
B;
C;
D;
denotes that the execution of A is to be followed by execution of B, to be
inturn followed by execution of C and finally by that of D.
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 12
When the composite action consisting of actions denoted by A, B, C and
D, in this order is to be treated as single component of some larger
structure, brackets such as ‗begin.... end‘ may be introduced i.e., in this
case we may use the structure
Begin A ; B ; C ; D end
Then the above is also called a (composite/ compound) statement
consisting of four (components) statement viz A, B, C and D.
(ii) Selection: In many situations, we intend to carry out some action A if
condition Q is satisfied and some other action B if condition Q is not
satisfied. This intention can be denoted by
If Q then do A else do B,
Where A and B are instructions, which may be even composite
instructions obtained by applying these structuring rules recursively to
the other instructions.
Further, in some situations B is null, i.e., if Q is false, then no action is
stated. This new situation may be denoted by
If Q then do A
In this case, if Q is true, A is executed. If Q is not true, then the
remaining part of the instruction is ignored, and the next instruction, if
any, in the program is considered for execution.
Also, there are situation when Q is not just a Boolean variable i.e., a
variable which can assume either a true or a false value only. Rather Q
is some variable capable of assuming some finite number of values say,
a, b, c, d, e, f. Further, suppose depending upon the value of Q, the
corresponding intended action is as given by following table:
Value Action
a A b A c B d NO ACTION e D f NO ACTION
The above intention can be expressed through the following notation:
Case Q of
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 13
a, b: A;
c: B;
e: D;
end;
Example: We are to write a program segment that converts % of marks
to grades as follows
% of marks (M) Grades (G)
M 80 A
60 M < 60 B
50 M < 60 C
40 M < 50 D M < 40 F
Then the corresponding notation may be:
Case M of
80 .. 100: ‘A’
60 .. 79 : ‘B’
50 .. 59 : ‘C’
40 .. 49 : ‘D’
0 .. 39 : ‘F’
Where M is an integer variable
(iii) Repetition: Iterative repetitive execution of a sequence of actions is the
basis of expressing long processes by comparatively small number of
instructions. The repeated execution of the sequence of actions has to
be terminated. The termination may be achieved either through some
condition Q or by stating in advance the number of times the sequence
is intended to be executed.
When we intend to execute a sequence S of actions repeatedly, while
condition Q holds, the following notation may be used for the purpose:
While (Q) do begins S end;
Example: We are required to find out of the sum (SUM) of first n natural
numbers. Let a variable x be used to store an integer less than or equal
to n, then the algorithm for the purpose may be of the form.
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 14
Algorithm Sum_First_N_1
begin
read (n); {assuming value of n is an integer 1}
x 1; SUM 1;
while (x<n) do ……………. ( 1)
begin
x x + 1;
SUM SUM + x
end; {of while Ioop} ………………………….. ( 1)
Write (‗The sum of the first‘, n, ‗natural numbers is‘ SUM)
end. {of algorithm}
Explanation of the algorithm Sum_First_N_1:
Initially, an integer value of the variable n is read. Just to simplify the
argument, we assume that the integer n 1. The next two statements
assign value 1 to each of the variables x and SUM. Next, we get the
execution of the while – loop. The while – loop extends from the statement
( 1) to ( 1) both inclusive. Whenever we enter the loop, the condition x < n
is (always) tested. If the condition x < n is true then the whole of the
remaining portion upto (inclusive) is executed. However, if the condition is
false then all the remaining statement of the while – loop, i.e., all statements
upto and including ( 1) are skipped.
Suppose we read 3 as value of n, and (initially) x equal 1, because of x 1.
Therefore, as 1< 3, therefore the condition x < n is true. Hence the following
portion of the while loop is executed:
begin
x x + 1;
SUM SUM + x;
end
and as a consequence of execution of this composite statement
The value of x becomes 1 and
the value of SUM becomes 3
As soon as the word end is encountered by the meaning of the while – loop,
the whole of the while – loop between ( 1) and ( 1),( including ( 1) and ( 1)
is again executed).
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 15
By our assumption n = 3 and it did not change since the initial assumption
about n; however, x has become 2. Therefore, x < n is again satisfied. Again
the rest of the while loop is executed. Because of the execution of the rest of
the loop, x becomes 3 and SUM becomes the algorithm comes to the
execution of first statement of while – loop, i.e., while (x < n) do, which tests
whether x < n. At this stage x = 3 and n = 3. Therefore, x < n is false.
Therefore, all statement upto, and including ( 1), are x < n and skipped.
Then the algorithm executes the next statement, viz, write (‗The sum of the
first‘ n, ‗natural number is‘ sum). As, at this stage, the value of SUM is 6, the
following statement is prompted, on the monitor:
The sum of the first 3 natural numbers is 6.
It may be noticed that in the statement write (‗ , ‗n‘ ‗ ‗ , SUM) the
variables n and SUM are not within the quotes and hence, the values of n
and SUM viz 3 and 6 just before the write statement are given as output.
Some variations of the ‗while ... do‘ notation are used in different
programming languages. For example, if S is to be executed at least once,
then the programming language C uses the following statement:
Do S while (Q)
Here S is called the body of the ‗do while‘ loop. It may be noted that here S
is not surrounded by the brackets, viz., begin and end. It is because of the
fact do and while enclose S.
Again consider the example given above, of finding out the sum of first n
natural numbers.
The above instruction is denoted in the programming language Pascal by
Repeat S until (not Q)
Example:
Algorithm Sum_First_N _2
Begin {of algorithm}
Read (n);
x 0; Sum 0 ……….( 2)
do
x x + 1
Sum Sum + x
While (x < n)…………… ( 2)
end {of algorithm}
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 16
If S is executed n number of times, the following notation may be used:
For v varying from I to (i + n – 1) do begin S end;
OR
For v I to (i + n – 1) do begin S end;
where v is some integer variable assuming initial value i and increasing by 1
after each execution of S and final execution of S takes place after
v assumes the value i + n – 1
Then the execution of the for-loop is terminated. Again begin S, do is called
the body of the for -loop. The variable x is called index variable of the for-loop.
Example: Again, consider the problem of finding the sum of first n natural
numbers, algorithm using ‗for ...‘ may be of the form.
algorithm Sum_First_N_3
begin
Read (n);
SUM 0
for x 1 to n do ( 3)
begin
SUM SUM + ( 3)
end:
write (‗The of the first‘, n, natural numbers‘, SUM) end {of the algorithm}
In the algorithm Sum First-N-3, there is only one statement in the body of
the for-loop. Therefore, the bracket words begin and end not will be used in
the for – loop. In this algorithm, also, it may be noted that only the variable
SUM is initialized. The variable x is not initialized explicitly. The variable x is
implicit initialized to 1 through the construct ‘for x varying from 1 to n do‘.
And, after each execution of the body of the ‗for – loop, x is implicitly
incremented by 1.
A noticeable feature of the structuring rules viz., sequencing, selection and
iteration, is that each defines a control structure with a single entry point and
single exit point. It is this property that makes them simple but powerful
building blocks for more complex control structures.
Self Assessment Question
6. The variable x is called ––––––––––– of the for-loop
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 17
1.8 Procedure and Recursion
Though the above mentioned three control structure, viz., direct sequencing,
selection and repetition, are sufficient to express any algorithm, yet the
following two advanced control structures have proved to be quite useful in
facilitating the expression of complex algorithms viz.
i) Procedure
ii) Recursion
First we consider the advanced control structure procedure.
1.8.1 Procedure
Among a number of terms that are used, instead of procedure, are
subprogram and even function. These terms may have shades of
differences in their usage in different programming languages. However, the
basic idea behind these terms is the same.
It may happen that a sequence frequently occurs either in the same
algorithm or repeatedly in different parts of the algorithm or may occur in
different algorithms. In such cases, writing repeatedly the same sequence,
is a wasteful activity. Procedure is a mechanism that provides a method of
checking this wastage.
Under this mechanism, the sequence of instructions expected to be
repeatedly used in one or more algorithms, is written only once, outside and
independent of the algorithms of which the sequence could have been a
part otherwise. There may be many such sequences and hence, there is
need for an identification of each of such sequences. For this purpose, each
sequence is prefaced by statements in the following format.
Where < name >, parameter – list > and other expressions with in the
angular brackets as first and last symbols, are place – holders for suitable
values that are to be substituted in their places. For example, suppose
finding the sum of squares of two variables is a frequently required activity,
(1)
Procedure < Name > (< parameter — list >) [:< type >1]
<declarations>
<sequence of instructions expected to occur
repeatedly
end;
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 18
then we may write the code for this activity independent of the algorithm of
which it would otherwise have formed a part. And then, in (1), <name> may
be replaced by ‘sum — square’ and <parameter – list> by the two element
sequence x, y. The variables like x when used in the definition of an
algorithm are called formal parameters or simply parameters. Further,
whenever the code which now forms a part of a procedure, say sum –
square is required at any place in an algorithm, then in place of the indented
code, a statement of the form
sum – square (a, b); (2)
is written, where values of a and b are defined before the location of the
statement under (2) within the algorithm.
Further, the pair of brackets in [ : < type > ] indicates that ‘ < type >’ is
optional. If the procedure passes some value computed by it to the calling
program, then, ‘ ;< type>, is used and then <type> in (1) is replaced by the
type of the value to be passed, in this case integer.
In cases of procedure which pass a value to the calling program another
basic construct (In addition to assignment, read and write) viz., return (x) is
used, where x is a variable used for the value to be passed by the
procedure.
There are various mechanisms by which values of a and b are respectively
associated with or transferred to x and y. The variables like a and b, defined
in the calling algorithm to pass data to the procedure (i.e. the called
algorithm), which the procedure may use in solving the particular instance of
problem, are called actual parameters or arguments.
Also, there are different mechanisms by which statement of the form sum –
square (a, b) of an algorithm is associated with the code, of the procedure
for which the statement is used. However, all these mechanisms are named
as ‗calling the procedure‘. The main algorithm may be called as the ‗calling
algorithm‘ and the procedure may be called the ‗the called algorithm‘.
In order to explain the involved ideas, let us consider the following simple
examples of a procedure and a program that calls the procedure. In order to
simplify the discussion, in the following, we assume that the inputs etc., are
always of the required types only, and make other simplifying assumptions.
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 19
Example:
Procedure sum-square (a, b: integer) integer;
{denoted the inputs a and b integers and the output is also an integer}
S: integer;
{to store the required number}
begin
22 baS
Return (S)
end;
Program diagonal – Length
{the program finds lengths of diagonals of the sides of right – angled
triangles whose lengths are given as integers. The program terminates
when the length of any side is not positive integer}
L1, L2 integer; {given side lengths}
D: real;
{to store diagonal length}
read (L1, L2)
while (L1 > 0 and L2 > 0) do
begin
D square – root (sum – square (L1, L2))
Write (‗For sides of given length‘, L1, L2, ‗the required diagonal length is‘ D);
read (L1, L2)
end
In order to explain how diagonal length of a right-angled triangle is
computed by the program diagonal-Length using the procedure sum-square,
let us consider the side lengths being given as 4 and 5.
First Step: In program Diagonal-length through the statement read (L1, L2),
we read L1 as 4 and L2 as 5. As L1 > 0 and L2 > 0. Therefore, the program
enters the while-loop. Next the program, in order to compute the value of the
diagonal calls the procedure sum-square by associating with a the value of
L1 as 4 and with b the value of L2 as 5. After these associations, the
procedure sum -square takes control of the computations. The procedure
computes S as 41 = 16 + 25. The procedure returns 41 to the program. At
this point the program again takes control of further execution. The program
uses the value 41 in place of sum-square (L1, L2). The program calls the
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 20
procedure square-root, which is supposed to be built in the computer
system, which temporarily takes control of execution. The procedure
square-root returns value 41and also returns control of execution to the
program Diagonal-Length which in turn assigns this value to D and prints
the statement.
For sides of given lengths 4 and 5, the required diagonal length is 41 .
The program under while-loop again expects values of L1and L2 from the
user. If the values supplied by the user are positive integers, whole process
is repeated after entering the while loop. However, if either L1 0 (say –34)
or L2 0, then while loop is not entered and the program terminates.
We can summarize as
A procedure is a self-contained algorithm which is written for the purpose of
connecting into another algorithm, but is written independent of the
algorithm into which it may be connected.
1.8.2 Recursion
Next, we consider another important control structure namely recursion. In
order to facilitate the discussion, we recall from Mathematics, one of the
ways in which the factorial of a natural number n is defined.
factorial (1) = 1
factorial (n) = n * factorial (n – 1)
For those who are familiar with recursive definitions like the one given above
for factorial, it is easy to understand how the value of (n!) is obtained from
the above definition of factorial of a natural number. However, for those who
are not familiar with recursive definitions, let us compute 4 using the
above definition.
By definition
factorial (4) = 4 * factorial (3)
Again by the definition
factorial (3) = 3 * factorial (2)
Similarly,
factorial (2) = 2 * factorial (1)
And by definition
factorial (1) = 1
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 21
Substituting back values of factorial (1), factorial (2) etc, we get
Factorial (4) = 4.3.2.1 = 24, as desired.
This definition suggests the following procedure/ algorithm for computing the
factorial of a natural number n:
In the following procedure factorial (n), let ‗fac‘ be the variable which is used
to pass the value by the procedure factorial to a calling program. The
variable ‗fac‘ is initially assigned value 1, which is the value of factorial (1),
Procedure factorial (n)
fac; integer:
begin
fac 1
if n equals 1 then return fac
else begin
fac n * factorial (n – 1)
return (fac)
end;
end;
In order to compute factorial (n – 1), procedure factorial is called by itself,
but this time with (simpler) argument (n – 1). The repeated calls with simpler
arguments continue until factorial is called with argument 1. Successive
multiplications of partial results with 2, 3, …… up to n finally deliver the
desired result.
Definition: A procedure, which can call itself, is said to be recursive
procedure/ algorithm. For successful implementation of the concept of
recursive procedure, the following conditions should be satisfied.
i) There must be an in-built mechanism in the computer system that
supports the calling of a procedure by itself, eg., there may be an in-
built stack operations on a set of stack registers.
ii) There must be conditions within the definition of a recursive procedure
under which, after finite number of calls, the procedure is terminated.
iii) The arguments in successive calls should be simpler in the sense that
each succeeding argument takes us towards the conditions mentioned
in (ii).
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 22
In view of the significance of the concept of procedure, and specially of the
concept of recursive procedure, in solving some complex problems, we
discuss another recursive algorithm for the problem of finding the sum of
first n natural numbers, discussed earlier. For the discussion, we assume n
is a non-negative integer.
Procedure SUM (n : integer: integer)
s : integer:
If n = 0 then, return (0)
Begin s n + SUM (n – 1)
end;
end;
Self Assessment Questions
7. –––––––––––– is a self-contained algorithm which is written for the
purpose of connecting into another algorithm.
8. A procedure, which can call itself, is said to be –––––––––––––
1.9 Outline of Algorithmics
The problem, which has at least one algorithmic solution, is called
algorithmic or computable problem. Also, we should note that there is no
systematic method (i.e., algorithm) for designing algorithms even for
algorithmic problems. In fact, designing an algorithm for a general
algorithmic/ computable problem is a difficult intellectual exercise. It requires
creativity and insight and no general rules can be formulated in this respect.
As a consequence, a discipline called algorithmics has emerged that
comprises large literature about tools, techniques and discussion of various
issues like efficiency etc. related to the design of algorithms. In the rest of
our study, we shall be explaining and practising algorithms. We see below
some well-known techniques which have been found useful in designing
algorithms.
i) Divide and conquer
ii) Dynamic programming
iii) The greedy approach
iv) Bracketing
v) Branch and bound
vi) Searches and traversals.
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 23
In view of the difficulty of solving algorithmically even the computationally
solvable problems, some of the problem types, enumerated below, have
been more rigorously studied
i) Sorting problems
ii) Searching problems
iii) Linear - programming problems
iv) Number - theory problems
v) String processing problems
vi) Graph problems
vii) Geometric problems
viii) Numerical problems.
Understanding the problem
Understanding allows appropriate action. This step forms the basis of the
other steps to be discussed. For understanding the problem, we should read
the statement of the problem, if required, a number of times. Then we
should find out
i) The type of problem, so that if a method of solving problems of the type,
is already known, then the known method may be applied to solve the
problem under consideration.
ii) The type of inputs and the type of expected/ desired outputs, specially,
the illegal inputs, i.e., inputs which are not acceptable, are characterized
at this stage. For example, in a problem of calculating income-tax, the
income cannot be non-numeric character strings. The range of inputs,
for those inputs which are from ordered sets. For example, in the
problem of finding whether a large number is prime or not, we cannot
give as input a number greater than the maximum number that the
computer system used for the purpose can store and arithmetically
operate upon. For still larger numbers, some other representation
mechanism has to be adopted.
iii) Special cases of the problem, which may need different treatment for
solving the problem. For example, if for an expected quadratic equation
ax2 + bx + c = 0, a the coefficient of x2, happened to be zero then usual
method of solving quadratic equations, cannot be used for the purpose.
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 24
Analyzing the problem
This step is useful in determining the characteristics of the problem under
consideration, which may help in solving the problem. Some of the
characteristics in this regard are discussed below:
i) Whether the problem is decomposable into independent smaller or
easier sub problems, so that programming facilities like procedure and
recursion etc. may be used for designing a solution of the problem. For
example, the problem of evaluating
dxxcosxsinx5 222
can be decomposed into smaller and simpler problem viz.,
dxxcosxSinanddxx5 222
ii) Whether steps in a proposed solution or solution strategy of the
problem, may or may not be ignorable, recoverable or inherently
irrecoverable, i.e., irrecoverable by the (very) nature of the problem.
Depending upon the nature of the problem, the solution strategy has to
be decided or modified.
For example,
a) While proving a theorem, if an unrequired lemma is proved, we may
ignore it. The only loss is the loss of efforts in proving the lemma.
Such a problem is called ignorable step problem.
b) Suppose we are interested in solving 8 — puzzle problem of
reaching from some initial state, say,
2 8 7
1 3 5
6 4
To some final state, say,
1 2 3
8 4
7 6 5
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 25
By sliding, any one of the digits from a cell adjacent to the blank cell,
to the blank cell. Then a wrong cannot be ignored but has to be
recovered. By recoverable, we mean that we are allowed to move
back to the earlier state from which we came to the current state, if
current state seems to be less desirable than the earlier state. The 8-
puzzle problem has recoverable steps, we may say the problem is a
recoverable problem.
c) However if, we are playing chess, then a wrong step may not be
even recoverable. In other words, we may be in a position, because
of the adversary‘s move back to earlier state. Such a problem is
called an irrecoverable step problem.
Depending on the nature of the problem as ignorable-step,
recoverable-step or irrecoverable-step problem, we have to choose
out tools, techniques and strategies for solving the problem.
For example, for ignorable-step problem, simple control structures
for sequencing and iteration may be sufficient. However, if the
problem additionally has recoverable-step possibilities then facilities
like back-tracking, as are available in the programming language
PROLOG, may have to be used. Further, if the problem additionally
has irrecoverable-step possibilities then planning tools should be
analyzed in advance to find out where the sequence may lead to,
before the first step is actually taken.
There are many other characteristics of a problem viz,
Whether the problem has certain outcome or uncertain outcome
Whether a good solution is absolute or relative
Whether the solution is a state or a path
Whether the problem essentially requires interaction with the
user etc.
Which can be known through analyzing the problem under
consideration, and the knowledge of which, in turn, may help us in
determining or guessing a correct sequence of actions for solving the
problem under consideration.
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 26
Capabilities of the computer system
We know that because of change in computational capabilities, a totally
different algorithm has to be designed to solve the problem (eg. that of
multiplying two natural numbers)
Most of the computer systems used for educational purposes are PCs
based on Von-Neumann architecture. Algorithms, that are designed to be
executed on such machines are called sequential algorithms.
However, new powerful machines based on parallel distributed
architectures, are also increasingly becoming available. Algorithms, that use
such additional facilities, are called parallel distributed; such parallel
distributed algorithms, may not have much resemblance to the
corresponding sequential algorithms for the same problem.
Approximate vs Exact Solution
For some problems like finding the square root of a given natural number n,
it may not be possible to find exact value for all n‘s (eg., n = 3). We have to
determine in advance what approximation is acceptable, eg., in this case,
the acceptable error may be, say, less than 01.
Also, there are problems, for which finding the exact solutions may be
possible, but the cost may be too much.
In the case of such problems, unless it is absolutely essential, it is better to
use an alternative algorithm which gives reasonably approximate solution,
which otherwise may not be exact. For example, consider the Traveling
Salesperson Problem: A salesperson has a list of, say n cities, each of
which he must visit exactly once. There are direct roads between each pair
of the given cities. Find the shortest possible route that takes the
salesperson on the round trip starting and finishing in any one of the n cities
and visiting other cities exactly once.
In order to find the shortest paths, one should find the cost of covering each
of the n! different paths covering the n given cities. Even for a problem for
visiting 10 cities, n!, the number of possible distinct paths is more than 3
million. In a country like India, a traveling salesperson may be expected to
visit even more than 10 cities. To find out exact solution in such cases,
though possible, is very time consuming. In such case, a reasonably good
approximate solution may be more desirable.
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 27
Self Assessment Question
9. In order to find the shortest paths, one should find the cost of covering
each of the ––––––––––– different paths covering the n given cities
1.10 Specification Methods for Algorithms
An algorithm is a description statement of a sequence of activities that
constitute a process of getting desired output from the given inputs. Such
description or statement needs to be specified in some notation or
language. We briefly mentioned about some possible notations for the
purpose. Three well — known notations/ languages used mostly for the
purpose, are enumerated below.
i) Natural Language (NL): An NL is highly expressive in the sense that it
can express algorithms of all types of computable problems. However,
main problem with an NL is the inherent ambiguity, i.e., a statement or
description in NL may have many meaning, which, may be unintended
and misleading.
ii) A Pseudo code notation is a sort of dialect obtained by mixing some
programming language constructs with natural language descriptions of
algorithms. The pseudo-code method of notation is the frequently used
one for expressing algorithms. However, there is no uniform/ standard
pseudo-code notation used for the purpose, though, most pseudo-code
notations are quite similar to each other.
iii) Flow chart is a method of expressing algorithms by a collection of
geometric shapes with imbedded descriptions of algorithmic steps.
However, the technique is found to be too cumbersome, specially, to
express complex algorithms.
Analyzing an Algorithm
Generally, the resources that are taken into consideration for analyzing
algorithms include
i) Time expected to be taken in executing the instances of the problem
generally as a function of the size of the instance.
ii) Memory space expected to be required by the computer system, in
executing the instances of the problem, generally as a function of the
size of the instances.
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 28
iii) Also sometimes, the man-hour or man-month taken by the team
developing the program, is also taken as a resources for the purpose.
Self Assessment Question
10. A ––––––––––––– notation is a sort of dialect obtained by mixing some
programming language constructs with natural language descriptions of
algorithms
1.11 Summary
In this unit we have studied
If in an algorithm, more than one variables are required to store values
of the same type, notation of the form A [1..n] is used to denote n
variables A[1], A[2], …, A[n]
The comments may be inserted at any place within an algorithm
The difference between the two concepts viz., ‗problem‘ and ‗instance‘
The characteristics of an algorithm is very important to design it
The following two building blocks
i) Basic actions and instructions
ii) Control Mechanism and control structure
are very much necessary to frame an algorithm
The following two advanced control structures
i) Procedure
ii) Recursion
have proved to be quite useful in facilitating the expression of
complex algorithms
The different outline of Algorithms which is very much necessary to
solve a problem effectively
The different specification methods and the resources that are taken into
consideration for analyzing algorithms
1.12 Terminal Questions
1. What is the use of comments in an Algorithm?
2. Briefly explain the concept of ―Problems and Instances‖
3. Give the different characteristics of an algorithm
4. Explain the different control mechanisms and control structures in an
Algorithm
Fundamentals of Algorithms Unit 1
Sikkim Manipal University Page No. 29
5. Explain the concept of procedure in a Complex Algorithm
6. Explain the concept of ―Analyzing the Problem‖ in Algorithms
7. Explain ―Specification Methods‖ in an Algorithm
8. What do you mean by ―Recursion in Algorithms‖? Explain?
1.13 Answers
Self Assessment Questions
1. Algorithm
2.
3. r m mod n
4. Four colours
5. Outputs
6. Index varialbe
7. Procedure
8. Recursive procedure/algorithm
9. n!
10. Pseudo code
Terminal Questions
1. Comments help the human reader of the algorithm to better understand
the algorithm. In different programming languages, there are different
notations for incorporating comments in algorithms. (For details refer
section 1.3)
2. A (general) problem, like finding the roots of an equation of degree 5 or
more, may not be solvable algorithmically, i.e., through some step-by-
step method, still it is possible for some (particular) instances of the
problem to have algorithmic solutions. (For details refer to section 1.5)
3. Refer to section 1.6
4. Refer to section 1.7
5. Refer to section 1.8
6. Refer to section 1.10
7. Refer to section 1.11
8. Refer to section 1.9
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 30
Unit 2 Mathematical Functions and Notations Structure
2.1 Introduction
Objectives
2.2 Functions & Notations
2.3 Modular Arithmetic/Mod Function
2.4 Mathematical Expectation in Average Case Analysis
2.5 Efficiency of an Algorithm
2.6 Well Known Asymptotic Functions & Notations
2.7 Analysis of Algorithms – Simple Examples
2.8 Well known Sorting Algorithms
Insertion sort
Bubble sort
Selection sort
Shell Sort
Heap Sort
2.9 Summary
2.10 Terminal Questions
2.11 Answers
2.1 Introduction
You know that number of mathematical and statistical tools, techniques and
notations form an essential part in the concept of analysis of algorithms. In
this unit, we will study a number of well known approximation functions.
These functions will calculate approximate values of quantities, and these
quantities are useful in many situations and among these some of the
quantities are calculated just for comparison with each other.
Objectives
After studying this unit, you should be able to:
use a number of Mathematical Notations like , , , , mod, log, e etc.
apply concepts of Mathematical Expectation in Algorithms
use of principle of Mathematical Induction in establishing truth of
infinitely many Statements
analyze the efficiency of an Algorithm
use Asymptotic Functions and Notations
apply different techniques in writing algorithms.
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 31
2.2 Functions & Notations
To start with, we recall the following notations and definitions.
Here,
N = {1, 2, 3, ….}
I = {…, –2, –1, 0, 1, 2, …..}
R = set of Real numbers.
Notation: If a1, a2, …. an are n real variables/ numbers then
(i) Summation
The expression a1 + a2 + …..+ ai + ….+ an may be denoted in shorthand
as n
1i
ia
(ii) Product
The expression a1 a2 …. x ai ….. an is denoted as n
1i
ia
Function
For two given sets A and B (which need not be different , i.e., A may be the
same as B) a rule f which associates with each element of A, a unique
element of B, is called a function from A to B. If f is a function from a set A to
a set B, then we denote it by f: A B. Also, for x A, f(x) is called image of
x in B. Then, A is called the domain of f and B is called the co domain of f.
Example:
Let f: I I be defined such that f(x) = x2 for all x I then
f maps 4 to 16
f maps 0 to 0
f maps 5 to 25
Remark
We may note the following:
i) If f: x y is a function, then there may be more than one element, say
x1 and x2 such that f(x1) = f(x2)
In the above example
f(2) = f(–2) = 4
By putting restriction that f(x) f(y) if x y, we get special functions,
called 1 – 1 or injective functions.
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 32
ii) Though for each element x X, there must be at least one element
y Y, such that f(x) = y. However it is not necessary that for each
element y Y there must be at least one element x X, such that
f(x) = y, For example, for y = – 3 Y there is no x X such that
f(x) = x2 = –3. By putting the restriction on the function f, such that for
each y Y, there must be at least one element x of X such that f(x) = y,
we get special functions called onto or surjective functions.
Definition of some functions:
1 – 1 or Injective Function: A function f: A B is said to be 1 – 1 or
injective function if for x, y A, if f(x) = f(y) then x = y
We have already seen that the function defined in above example is not
1 – 1. However, by changing the domain, though defined by the same rule,
f becomes a 1 – 1 function.
Example: In this particular case, if we change the domain to
N = {1, 2, 3 …..}, then we can easily check that function.
f: N N defined as f(x) = x2 for all x N is 1 – 1 because, in this case, for
each x N its negative –x N. Hence for f(x) = f(y) implies x = y. For
example, If f(x) = 4 then there is only one value of x = 2 such that f(2) = 4.
Onto/surjective function: A function f: X Y is said to be onto, or
subjective if to every element of, the co domain of f, there is an element
x X such that f(x) = y.
We have already seen that the function defined in above example is not
onto.
However, by changing the co domain y or changing the rule, (or both) we
can make f as onto.
Example: (Changing the domain)
Let X = I = {….. –3, –2, –1, 0, 1, 2, 3 ……} but, if we change Y as
Y = {0, 1, 4, 9 …..} = {y | y = n2 for n X }
Then it can be seen that f: X Y defined by f(x) = x2 for all x X is onto
Example: (changing the rule)
Here, we change the rule so that X = Y = { …….–3, –2, –1, 0, 1, 2, 3 …….}
But f: X Y is defined as f(x) = x + 3 for x X.
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 33
Then we apply the definition to show that f is onto
If y Y, then, by definition, for f to be onto, there must exist an x X such
that f(x) = y. So the problem is to find out x X such that f(x) = y .
Let us assume that x X exists such that f(x) = y
i.e., x + 3 = y
i.e., x = y – 3
But, as y is given, x is known through the above equation. Hence f is onto.
Monotonic Functions: For the definition of monotonic functions, we
consider only functions
f: R R where, R is the set of real numbers.
A function f: R R is said to be monotonically increasing if for x, y R and
x y we have f(x) f(y)
Further, f is said to be strictly monotonically increasing if x < y then
f(x) < f(y)
Example:
Let f: R R be defined as f(x) = x + 3, for x R
Then, for x1, x2 R, the domain, if x1 x2 then x1 + 3 x2 + 3, (by using
monotone property of addition), which implies f(x1) f(x2). Hence, f is
monotonically increasing.
A function f: R R is said to be monotonically decreasing if, for x, y R
and x y then
f(x) f(y)
In other words, as x increases, value of its image decreases.
Further, f is said to be strictly monotonically decreasing, if x < y then
f(x) > f(y).
Example:
Let f: R R be defined as
f(x) = – x + 3
If x1 x2 then –x1 –x2 implying – x1 + 3 –x2 + 3 which further implies
that f(x1) f(x2) hence, f is monotonically decreasing.
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 34
Suppose that a computer can execute one billion steps per second. Table
2.1 shows the time that the computer takes to execute f(n) steps. Note that
1 s = 10–6 Seconds and 1 ms = 10–3 seconds.
Table No. 2.1: Table showing the time
N F(n) = n f(n) = log2n f(n) = nlog2n f(n) = n2
f(n) = 2n
10 0.01 s 0.003 s 0.033 s 0.1 s 1 ms
20 0.02 s 0.004 s 0.086 s 0.4 s 1 ms
30 0.03 s 0.005 s 0.147 s 0.9 s 1 sec
50 0.05 s 0.006 s 0.282 s 2.5 s 13 days
100 0.10 s 0.007 s 0.664 s 10 s 4 – 1013
years
106 1 ms 0.020 s 19.93 s 16.7 min
Floor Function: maps each real number x to the integer, which is the
greatest of all integers less than or equal to x. Then the image of x is
denoted by x
For example: 2.5 = 2, –2.5 = –3, 6 = 6
Ceiling Function: maps each real number x to the integer, which is the
least of all integers greater than or equal to x. Then the image of x is
denoted by x .
For example: 2.5 =3 –2.5 = –2, 6 = 6
Note: For every real number x, we have
x – 1 < x x x < x + 1
Example:
Each of the floor function and ceiling function is a monotonically increasing
function but not strictly monotonically increasing function. Because, for real
numbers x and y.
If x y then y = x + k for some k 0.
y = x + k = integer part of (x + k) integer part of x = x
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 35
Similarly,
y = x + k = least integer greater than or equal to x + k least integer
greater than or equal to x = x
But, each of floor and ceiling function is not strictly increasing, because
2.5 = 2.7 = 2.9 = 2 and 2.5 = 2.7 = 2.9 = 3
Self Assessment Questions
1. A function f: R R is said to be monotonically increasing if for x, y R
and x y we have ––––––––––––.
2. –––––––––––––– maps each real number x to the integer, which is the
least of all integers greater than or equal to x.
2.3 Modular Arithmetic/ Mod Function
Consider the following
i) If, we are following 12 – hour clock, and if it is 11 O‘clock now then after
3 hours, it will be 2 O‘clock and not 14 O‘clock (whenever the number of
o‘clock exceeds 12, we subtract n = 12 from the number)
ii) If, it is 5th day (i.e., Friday) of a week, after 4 days, it will be 2nd day
(i.e., Tuesday) and not 9th day, of course of another week (whenever the
number of the day exceeds 7, we subtract n = 7 from the number, we
are taking here Sunday as 7th day. Instead of 0th day).
iii) If, it is 6th month (i.e., June) of a year, then after 8 months, it will be
2nd month (i.e., February) of course another year (whenever, the number
of the month exceeds 12, 23 subtract n = 12) Thus, we have
Definitions
b mod n: If n is a given positive integer and b is any integer, then
b mod n = r where 0 r < n and b = k * n + r
In other words, r is obtained by subtracting multiplies of n so that the
remainder r lies between 0 and (n – 1).
For example: If b = 42 and n = 11 then
b mod n = – 42 mod 11 = 2 (– 42 = (–4) 11 + 2)
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 36
Mod function can also expressed in terms of the floor function as follows:
b(mod n) = nn
bb
Factorial: For N = {1, 2, 3, ….}, the factorial function
factorial: 0N0N
given by n = n. n – 1 = n factorial (n – 1)
Exponentiation Function (Exp): is a function of two variables x and n
where x is any non – negative real number and n is an integer (though n can
be taken as non integer also, but we restrict to integers only)
Exp (x n) denoted by xn, is defined recursively as follows:
For n = 0
Exp (x, 0) = x0 = 1
For n > 0
Exp (x, n) = x. Exp (x, n – 1)
i.e.,
xn = x . xn–1
For n < 0, let n = –m where m > 0
m
mn
x
1xx
In xn, n is also called the exponent/ power of x.
For example: If x = 1.5, n = 3, then
Also, Exp (1.5, 3) = (1.5)3 = (1.5) [(1.5)2] = [(1.5) [1.5 (1.5)1]
= 1.5 [(1.5 (1.5 (1.5)0))]
= 1.5 [(1.5 (1.5 1))] = 1.5 [(1.5 1.5)]
1.5 [2.25] = 3.375
Exp (1.5, – 3) = (1.5)–3 = 375.3
1
5.1
13
Further, the following rules apply to exponential function.
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 37
mnnmbb
mnnm bb
nmnm bb.b
For b 1 and for all n, the function bn is monotonically increasing in n. In
other words, if n1 n2 then .1bifbb 2n1n
Polynomial: A polynomial in n of degree k, where k is a non – negative
integer, over R, the set of real numbers, denoted by P(n), is of the form
011k
1kk
kk ana......nananP
Where, 0ak and ,Rai i = 0, 1, …….,K
Using the summation notation
Ra,0ananP ik
k
0i
iik
Each of ii na is called a term.
Generally, the suffix k in Pk (n) is dropped and instead of Pk(n) we write P(n)
only.
We may note that P(n) = nk = 1. nk for any k, is a single term polynomial. If
k 0 then P(n) = nk is monotonically increasing. Further, if k 0 then
p(n) = nk is monotonically decreasing.
Result: Though 0 = 1. The following is a very useful result relating the
exponentials and polynomials.
For any constants b and c with b > 1
0b
nLim
n
c
n
The result, in non – mathematical terms, states that for any given constants
b and c, but with
b > 1, the terms in sequence ,........b
k,.....,
b
3,
b
2,
b
1k
c
3
c
2
c
1
c
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 38
gradually decrease and approaches zero. Which further means that for
constants b and c, and integer variable n, the exponential term bn, for b > 1,
increases at a much faster rate than the polynomial term nc.
Exponential Function: The letter e is used to denote the quantity
......!3
1
!2
1
!1
11
and is taken as the base of natural logarithm function, then for all real
numbers x, we define the exponential function
0i
i32x
!i
x.........
!3
x
!2
xx1e
For all real numbers, we have
Further, if | x | 1 then 2x xx1ex1
The following is another useful result, which we state without proof:
Result: xn
ne
n
x1Lim
Logarithm: The concept of logarithm is defined indirectly by the definition of
exponential defined earlier. If a > 0, b > 0 and c > 0 are three real numbers,
such that
c = ab
Then clogb a (read as log of c to the base a)
Then a is called the base of the logarithm.
For example: if 26 = 64, then log2 64 = 6,
i.e., 2 raised to power 6 gives 64.
Generally, two bases, viz, 2 and 3 are very common in scientific and
computing fields and hence, the following special notations for these bases
are used.
(i) Ig n denotes log2n n (base 2)
(ii) In n denoted logen (base e);
Where the letter n denotes logarithm and the letter n denotes natural.
Result: Given below are some important properties of logarithms without
proof.
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 39
For n, a natural number and real numbers a, b and c all greater than 0, the
following identities are true
i) clogblogbclog aaa
ii) blognblog an
a
iii) blogb
1log aa
iv) alog
1blog
ba
v) ablogcblog
ca
Self Assessment Questions
3. ––––––––––––– is a function of two variables x and n where x is any
non – negative real number and n is an integer.
4. 0i
i32
!i
x.........
!3
x
!2
xx1
2.4 Mathematical Expectation in Average Case Analysis
The concept of mathematical expectation is needed in average – case
analysis of algorithms. In order to understand the concept better, consider
the following example.
Example: Suppose, the students of MCA, who completed all the courses in
the year 2005, had the following distribution of marks:
Range of marks Percentage of students who
scored in the range
0% to 20% 08
20% to 40% 20
40% to 60% 57
60% to 80% 09
80% to 100% 06
If a student is picked up randomly from the set of students under
consideration, what is the % of marks expected of such a student? After
scanning the table given above, we intuitively expect the student to score
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 40
around the 40% to 60% class, because, more than half of the students have
scored marks in and around this class.
Assuming that marks within a class are uniformly scored by the students in
the class, the above table may be approximated by the following more
compact table.
% marks Percentage of students
scoring the marks
10* 08
30 20
50 57
70 09
90 06
As explained earlier, we expect a student picked up randomly, to score
around 50% because more than half of the students have scored marks
around 50%
This informal idea of expectation may be formalized by giving to each
percentage of marks, weight in proportion to the number of students scoring
the particular percentage of marks in the above table.
Thus, we assign weight 100
8 to the score 10% (8, out of 100 students
score on the average 10% marks); 100
20 to the score 30% and so on.
Thus,
Expected % of marks
47100
690
100
970
100
5750
100
2030
100
810
The final calculation of expected marks of 47 is roughly equal to our intuition
of the expected marks, according to our intuition, to be around 50.
We generalize and formalize these ideas in the form of the following
definition.
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 41
Mathematical Expectation
For a given set S of items, let to each item, one of the n values, say
v1, v2 ….., vn, be associated. Let the probability of occurrence of an item with
value vi be pi. If an item is picked up at random, then its expected value E(v)
is given by
nn2211i
n
1i
i vp.....vpvpvp)v(E
Self Assessment Question
5. The concept of –––––––––– is needed in average – case analysis of
algorithms
2.5 Efficiency of an Algorithm
If a problem is algorithmically solvable then it may have more than one
algorithmic solutions. In order to choose the best out of the available
solutions, there are criteria for making such a choice. The
complexity/efficiency measures or criteria are based on requirement of
computer resources by each of the available solutions. The solution which
takes least resources is taken as the best solution and is generally chosen
for solving problem. However, it is very difficult to even enumerate all
possible computer resources e.g., time taken by the designer of the solution
and the time taken by the programmer to encode the algorithm.
Mainly, the two computer resources taken into consideration for efficiency
measures, are time and space requirements for executing the program
corresponding to the solution/algorithm. We will restrict to only time
complexities of algorithms of the problems.
In order to understand the complexity/efficiency of an algorithm, it is very
important to understand the notion of the size of an instance of the problem
under consideration and the role of size in determining complexity of the
solution.
It is easy to realize that given an algorithm for multiplying two n x n matrices,
the time required by the algorithm for finding the product of two
2 x 2 matrices, is expected to take much less time than the time taken by
the same algorithm for multiplying, say, two 100 x 100 matrices. This
explains intuitively the notion of the size and instances of a problem and
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 42
also the role of size in determining the (time) complexity of algorithm. If the
size of general instance is n then time complexity of the algorithm solving
the problem under consideration is some function of n.
In view of the above explanation, the notion of size of an instance of a
problem plays an important role in determining the complexity of an
algorithm for solving the problem under consideration. However, it is difficult
to define precisely the concept of size in general, for all problems that may
be attempted for algorithmic solutions.
However, for all types of problems, this does not serve properly the purpose
for which the notion of size is taken into consideration. Hence, different
measures of size of an instance of a problem are used for different types of
problem. For example,
i) In sorting and searching problems, the number of elements, which are to
be sorted or are considered for searching, is taken as the size of the
instance of the problem of sorting/ searching.
ii) In the case of solving polynomial equations i.e., dealing with the algebra
of polynomials, the degrees of polynomial instances, may be taken as
the sizes of the corresponding instances.
There are two approaches for determining complexity (or time required) for
executing an algorithm, viz.,
i) empirical (or a posterior)
ii) theoretical (or priori)
In the empirical approach (the programmed) algorithm is actually executed
on various instances of the problem and the size (s) and time (t) of
execution for each instance is noted. And then by some numerical or other
technique, t is determined as a function of s. This function then, is taken as
complexity of the algorithm under consideration.
In the theoretical approach, we mathematically determine the time needed
by the algorithm, for a general instance of size, say, n of the problem under
consideration. In this approach, generally, each of the basic instructions like
assignment, read and write and each of the basic operation like ‘+’,
comparison of pair of integers etc. is assumed to take one or more, but
some constant number of, (basic) units of time for execution. Time for
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 43
execution for each structuring rule is assumed to be some function of the
time required for constituent of the structure.
Thus starting from the basic instructions and operations and using
structuring rules, we can calculate the time complexity of a problem or an
algorithm.
The theoretical approach has a number of advantages over the empirical
approach as listed below:
i) The approach does not depend on the programming language in which
the algorithm is coded and on how it is coded in the language.
ii) The approach does not depend on the computer system used for
executing (a programmed version of) the algorithm.
iii) In case of a comparatively inefficient algorithm, which ultimately is to be
rejected, the computer resources and programming efforts which
otherwise would have been required and wasted, will be saved.
iv) Instead of applying the algorithm to many different – sized instances, the
approach can be applied for a general size, say, n of an arbitrary
instance of the problem under consideration. In the case of theoretical
approach, the size n may be arbitrary large. However, in empirical
approach, because of practical considerations, only the instances of
moderate sizes may be considered.
Self Assessment Question
6. Starting from the basic instructions and operations and using structuring
rules, we can calculate the ––––––––– of a problem.
2.6 Well known Asymptotic Function & Notations
We often want to know a quantity only approximately and not necessarily
exactly, just to compare with another quantity. And, in many situations,
correct comparison may be possible even with approximate values of the
quantities. The advantage of the possibility of correct comparisons through
even approximate value of quantities is that the time required to find
approximate values may be much less than the time required to find exact
values. We will introduce five approximation functions and their notations.
The purpose of these asymptotic growth rate functions to be introduced, is
to facilitate the recognition of essential character of a complexity function
through some simpler functions delivered by these notations. For example a
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 44
complexity function f(n) = 5004 n3 + 83 n2 + 19 n + 408, has essentially
same behavior as that of g(n) = n3 as the problem size n becomes larger
and larger. But g(n) = n3 is much more understtandable and its value easier
to compute than the function f(n)
Given below are the five well-known approximation functions.
1) O : (O (n2) is pronounced as big-oh of n2, or sometimes just as oh of n2)
2) : ( (n2) is pronounced as ‗big-omega of n2 or sometimes just as
omega of n2)
3) : ( (n2) is pronounced as ‗theta of n2)
4) o : (o (n2) is pronounced as ‗little-oh h2)
5) : ( (n2) is pronounced as ‗little-omega of n2)
These approximations denote relations from functions to functions.
f, g: N N are given by
f(n) = n2 – 5n and g(n) = n2 then O(f(n)) = g(n) or O(n2 – 5n) = n2
To be more precise, each of these notations is a mapping that associates a
set of functions to each function under consideration. For example, if f(n) is
a polynomial of degree k then the set O(f(n)) includes all polynomials of
degree less than or equal to k.
Remarks
In the discussion of any one of the five notations, generally two functions,
say, f and g are involved. The functions have their domains and co domains
as N, the set of natural numbers, i.e., f : N N
g : N N
These functions may also be considered as having domain and co domain
as R.
1) The notation O
Provides asymptotic upper bound for a given function. Let f(x) and g(x) be
two functions each from the set of natural numbers or set of positive real
numbers.
Then f(x) is said to be O(g(x)) (pronounced as big-oh of g of x) if there exists
two positive integers/real number constants C and K such that
f(x) C g(x) for all x k ……….(A)
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 45
Note: (The restriction of being positive on integers/ real is justified as all
complexities are positive numbers)
Example: For the function defined by
f(x) = 2x3 + 3x2 + 1 show that
i) F(x) = O(x3)
ii) f(x) = O(x4)
iii) x3 = O (f(x))
iv) x4 O(f(x))
v) F(x) O(x2)
Solution:
i) Consider
f(x) = 2x2 + 3x2 + 1
2x3 + 3x3 + 1x3 = 6x3 for all x 1
(by replacing each term xi by the highest degree term x3)
there exist C = 6 and k = 1 such that
f(x) C, x3 for all x k
Thus we have found the required constants C and k. Hence f(x) is O(x3)
ii) As, above, we can show that
f(x) 6x4 for all x 1
However, we may also, by computing some value of f(x) and x4, find
C and k as follows
f(1) = 2 + 3 + 1 = 6 ; (1)4 = 1
f(2) = 2.23 + 3.22 + 1 = 29 ; (2)4 = 16
f(3) = 2.33 + 3.32 + 1 = 82 ; (3)4 = 81
for C = 2 and k = 3 we have
f(x) 2. x4 for all x k
Hence f(x) is O(x4)
iii) for C = 1 and k = 1 we get
x3 C (2x3 + 3x2 + 1) for all x k
iv) We prove the result by contradiction. Let there exist positive constants C
and k such that
x4 C (2x3 + 3x2 + 1) for all x k
x4 C (2x3 +3x3 + x3) = 6 Cx3 for x k
x4 6 Cx3 for all x k
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 46
Implying x 6C for all x k
But for x = max {6C + 1, k}, the previous statement is not true.
Hence the proof.
v) Again we establish the result by contradiction.
Let O(2x3 + 3x2 + 1) = x2
Then for some positive numbers C and k
2x3 + 3x2 + 1 Cx2 for all x k,
Implying
x3 C x2 for all x k (x3 2x3 + 3x2 + 1 for all x 1)
implying
x C for x k
Again for x = max {C + 1, k}
The last inequality does not hold. Hence the result.
2) The Notation
The function f(n)= (g(n)) (read as ―f of n is omega of g of n‖) if there exist
positive constants c and no such that f(n) =c g(n) for all n, n no.
Eg: The function 3n+2= (n) as 3n+2 3n for n 1
The function 3n+3= (n) as 3n+3 3n for n 1
The function 10n2+4n+2= (n2) as 10n2+4n+2 n2 for n 1
The statement f(n)= (g(n)) states that g(n) is only a lower bound on f(n).
3) The Notation
This notation provides simultaneously both asymptotic lower bound and
asymptotic upper bound for a given function.
Let f(x) and g(x) be two functions, each from the set of natural numbers or
positive real numbers to positive real numbers. Then, f(x) is said to be
(g(x)) (pronounced as big-theta of g of x) if, there exists positive constants
C1, C2 and k such that C2 g(x) f(x) C1g(x) for all x k.
(Note the last inequalities represent two conditions to be satisfied
simultaneously viz.,
C2 g(x) f(x) and f(x) C1g(x))
Now we state the following theorem without proof which relates the three
functions O
*
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 47
Theorem: For any two functions f(x) and g(x), f(x) = (g(x)) if and only if
f (x) = 0 (g(x)) and f(x) = (g(x)) where f(x) and g(x) are non-negative.
Example: Let f(n): 1 + 2 + .... + n, n 1. Show that f(n) = (n2).
Solution: First, we find an upper bound for the sum. For n 1, consider
1 + 2 + …… + n n + n + ….. + n = n. n = n2
This implies that f(n) = O(n2). Next, we obtain a lower bound for the sum.
We have
n.....2
n...21n...21
n.....2
n
2
n..
2
n
2
n
2
n.
2
n
.4
n2
This proves that f(n) = (n2). Thus by the above. Theorem f(n) = (n2).
4) The Notation
The asymptotic upper bound provided by big-oh notation may or may not be
tight in the sense that if f(x) = 2x3 + 3x2 + 1. Then for f(x) = O(x3), though
there exist C and k such that f(x) C(x3) for all x k yet there may also be
some values for which the following equality also holds
f(x) = C(x3) for x k However, if we consider f(x) = O(x4) then there can not
exist positive integer C such that f(x) = Cx4 for all x k.
The case of f(x) = O(x4) provides an example for the notation of small-oh
i.e., notation o
Let f(x) and g(x) be two functions, each from the set of natural numbers or
positive real numbers to positive real numbers.
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 48
Further, let C > 0 be any number, then f(x) = o (g(x)) (pronounced as little oh
of g of x) if there exists natural number k satisfying
f(x) < Cg (x) for all x k 1 ………..(B)
Here we get the following points
i) In the case of little-oh the constant C does not depend on the two
functions f(x) and g(x). instead, we can arbitrarily choose C > 0
ii) The inequality (B) is strict whereas the inequality (A) of big — oh is not
necessarily strict.
Examples:
For f(x) = 2x3 + 3x2 + 1, we have
i) f(x) = o(xn) for any n 4
ii) f(x) o(xn) for any n 3
Solutions:
i) Let C > 0 be given and to find out k satisfying the requirement of little-
oh.
Consider
2x3 + 3x2 + 1 < C xn 3n
3Cx
x
1
x
32
When n = 4
Then, above inequality becomes
Cxx
1
x
32
3
If we take 1,C
7maxk
Then,
2x3 + 3x2 + 1 < C x4 for n k. In general, as xn > x4 for n 4, therefore,
2x3 + 3x2 + 1 < C xn for n 4.For all x k with 1,C
7maxk
ii) We prove the result by contradiction Let, if possible, f(x) = 0 (x) for n 3.
Then there exist positive constants C and k such that 2x3 + 3x2 + I < C xn
for all x k.
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 49
Dividing by x3 throughout, we get
3n
2xC
x
1
x
32
As C is arbitrary, we take C = 1, then the above inequality reduces to
3n
2xC
x
1
x
32 for n 3 and x k 1
Also, it can be easily seen that xn–3 1 for n 3 and x k 1.
1x
1
x
32
2 for n 3
However, the last inequality is not true. Therefore, the proof by
contradiction. Generalizing the above example, we get the results
f(x) is a polynomial of degree m and g(x) is a polynomial of degree n. Then
f(x) = o(g(x)) if and only if n > m
We state below the two results which can be useful in finding small-oh upper
bound for a given function (without proof).
More generally, we have
Theorem: Let f(x) and g (x) be functions in definition of small — oh notation.
Then f(x) = o(g (x)) if and only if
0xg
xfLimx
Next, we introduce the last asymptotic notation, namely, small-omega. The
relation of small -omega to big-omega is similar to what is the relation of
small-oh to big oh.
5) The Notation
Again the asymptotic lower bound may or may not be tight. However, the
asymptotic bound cannot be tight. The definition of is as follows:
Let f(x) and g(x) be two functions each from the set of natural numbers or
the set of positive real numbers to set of positive real numbers.
Further, let C > 0 be any number, then, f(x) = (g(x)) if there exists a
positive integer k Such that f(x) > C g(x) for all x k.
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 50
Example:
Lf f(x) = 2x3 + 3x2 + 1 then f(x) = and also f(x) = (x2)
Solution:
Let C be any positive constant
Consider 2x3 + 3x2 + 1 > Cx
To find out k 1 satisfying the Conditions of the bound .
Cx
1x3x2 2 (dividing through out by x)
Let k be integer with k C + 1 then for all x k
1CkCC3C2K3k2x3x2x
1x3x2 12222
f(x) = (x)
Again, consider for any C > 0, 2x3 + 3x2 + I > C > x2 then
Cx
13x2
2 Let k be integer with k C + 1
Then, for x k we have C3C23k23x2x
13x2
2
Hence
f(x) = (x2)
In general, we have the following two theorems
Theorem: If f(x) is a Polynomial of degree n, and g(x) is a polynomial of
degree n, then
f(x) = (g(x)) if and only if m >n.
Theorem: Let f(x) and g(x) be functions in the definitions of little-omega.
Then f(x) = (g(x)) if and only if
xg
xflimx
0xf
xglimx
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 51
Self Assessment Questions
7. The ––––––––– provides asymptotic upper bound for a given function.
8. Let f(x) and g (x) be functions in definition of small-oh notation. Then f(x)
= o(g (x)) if and only if ––––––––––––.
2.7 Analysis of Algorithms — Simple Examples
Now we study the different types of analysis of Algorithms with simple
illustrations.
Compute Prefix Average: For a given array A[1 …n] of numbers, the
problem is concerned with finding an array B[1…n] such that
B [1] = A[1]
B [2] = average of first two entries = 2
2A1A
B [3]=average of first 3 entries=3
3A2A1Aand in general for
1 i n,
B[ i ] = average of first i entries in the array A [1…n] = i
iA....2A1A
Algorithm first-prefix-Average (A [1…n])
begin (of algorithm)
for i 1 to n do
begin (first for-loop)
Sum 0;
{sum stores the sum of first i terms, obtained in different iterations of
for-loop}
for j 1 to i do
begin {of second for-loop}
Sum Sum + A [ j ];
end {of second for loop}
B [ i ] Sum / i
End {of the first for — loop}
end {of algorithm}
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 52
Analysis of First-Averages
The different steps involved in the analysis of first averages is discussed
below
Step 1: Initialization step for setting up of the array A [1. .n] takes constant
time, say, C1, in view of the fact that for the purpose, only address of A (or of
A[1] is to be passed. Also after all the values of B [1...n] are computed, then
returning the array B [1…n] also takes constant time say C2, again for the
same reason.
Step 2: The body of the algorithm has two nested for-loops, the outer one,
called the first for loop is controlled by i and is executed n times. Hence, the
second for-loop along with its body, which form a part of the first for-loop, is
executed n times. Further, each construct within second for-loop controlled
by j, is executed i times just because of the iteration of the second for-loop.
However, the second for-loop itself is being executed n times because of the
first for-loop. Hence, each instruction within the second for-loop is executed
(n, i) times for each values of
i = 1, 2, . . .,n.
Step 3: In addition, each controlling variable i and j is incremented by 1 after
each iteration of i or j as the case may be. Also, after each increment in the
control variable, it is compared with the (upper limit + 1) of the loop to stop
the further execution of the for-loop.
Thus, the first-for-loop makes n additions (to reach (n + 1) and n
comparisons with (n + 1)).
The second for-loop makes, for each value of i = 1, 2,……n, one addition
and one comparison. Thus, total number of each of additions and
comparisons done just for controlling variable j
2
1nnn......21
Step 4: Using the explanation of steps 2, we count below the number of
times the various operations are executed.
i) Sum 0 is executed n times, once for each value of i from 1 to n.
ii) On the similar lines of how we counted the number of additions and
comparisons for the control variable j, it can be seen that the number of
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 53
additions (sum sum + A [ j ] and divisions (B [ i ] sum / i) is
2
1nn
Self Assessment Question
9. 1 + 2 + ……….. + n = ––––––––
2.8 Well known Sorting Algorithms
The following are the well-known Sorting Algorithms:
1) Insertion sort
2) Bubble sort
3) Selection sort
4) Shell sort
5) Heap sort
6) Merge sort
7) Quick sort
In this section, we will discuss some of the well-known algorithms for sorting
a given list of numbers:
Ordered set: Any set S with a relation, say, , is said to be ordered if for
any two elements x and y of S, either x y or y x is true. Then, we may
also say that (S, ) is an ordered set.
2.8.1 Insertion sort
The insertion sort, algorithm for sorting a list L of n numbers represented by
an array A [1... n] proceeds by picking up the numbers in the array from left
one by one and each newly picked up number is placed at its relative
position, w.r.t. the sorting order, among the earlier ordered ones. The
process is repeated till each element of the list is placed at its correct
relative position i.e., when the list is sorted.
Example:
A[1] A[2] A[3] A[4] A[5] A[6]
80 32 31 110 50 41 { given list, initially}
{Pick up the left — most number 80 from the list, we get the sorted sublist}
80 {pick up the next number 32 from the list and place it at correct position
relative to 80, so that the sublist considered so far is sorted}
32 80
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 54
{We may note in respect of the above sorted sublist, that in order to insert
32 before 80, we have to shift 80 from first position to second and then
insert 32 in first position.
The task can be accomplished as follows:
i) First 32 is copied in a location, say, m
ii) 80 is copied in the location A [2] = 32 so that we have
A[1] A[2] m
80 80 32
32 is copied in A [1] from m so that we have
A[1] A[2] m
32 80 32
{thus we get the sorted sublist given above}
32 80
{next number 31 is picked up, compared first with 80 and then (if required)
with 32, in order to insert 31 before 32 and 80, we have to shift 80 to third
position and then 32 to second position and then 31 is placed in the first
position}
The task can be accomplished as follows:
1) First 31 is copied in a location, say, m
2) 80 is copied in the location A[3] = 31 so that we have
A[1] A[2] A[3] m
32 80 80 31
3) 32 is copied in A[2] from A[1] so that we have
A[1] A[2] A[3] m
32 32 80 31
4) the value 31 from m is copied in A[1] so that we get
A[1] A[2] A[3] m
31 32 80 31
Thus, we get the sorted sublist
31 32 80
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 55
{Next, 110 is picked up, compared with 80. As 110 > 80, therefore,
no shifting and no more comparisons. 110 is placed in the first position
after 80}
31 32 80 110
{next, number 50 is picked up, First compared with 110, found less; next
compared with 80, again found less, again compared with 32. The
correct position for 50 between 32 and 80 in the sublist given above.
Thus, each of 110 and 80 is shifted one place to make space for 50 and
then 50 is placed over there}
The task can be accomplished as follows:
1) First 50 is copied in a location say m
2) 110 is copied in the location A [5] = 50 so that we have
A [1] A [2] A [3] A [4] A [5] m
31 32 80 110 110 50
3) 80 is copied in A [4] from A [3] so that we have
A [1] A [2] A [3] A [4] A [5] m
31 32 80 80 110 50
4) the value 50 from m is copied in A [3] so that we get
A [1] A [2] A [3] A [4] A [5] m
31 32 50 80 110 50
Thus, we get the following sorted sublist}
31 32 50 80 110
{next in order to place 40 after 32 and before 50, each of the values 50, 80
and 110 need to be shifted one place to the right as explained above.}
{However, values 31 and 32 are not to be shifted. The process of inserting
40 at correct place is similar to the ones explained earlier}
31 32 40 50 80 110
Algorithm: The insertion sort:
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 56
The idea of insertion sort as explained above, may be implemented through
procedure insertion -sort given below. It is assumed that the numbers to be
sorted are stored in an array A [1...n].
procedure insertion-Sort (A [1…..n]: real)
begin {of procedure}
if n = 1
then
write (‗A list of one element is already sorted‘)
else
begin {of the case when n 2}
for j 2 to n do
{to find out the correct relative position for A [j] and insert it there among the
already sorted elements A [1] to A [j — 1]}
begin {of for loop}
If A [ j ] < A [j — 1] then begin
{we shift entries only if A [j] < A [j — 1])
i j – 1; m A [ j ]
{In order to find correct relative position we store A [ j ] in m and start with
the last element
A [j – 1] of the already sorted part. If m is less than A [j – 1], then we move
towards left, compare again with the new element of the array or we reach
the left-most element A [1] }
While (I > 0 and m < A [ i ] ) do
begin {of while loop}
A [ i + 1] A [ i ]
i i – 1
end
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 57
{after finding the correct relative position, we move all the elements of the
array found to be greater than m = A [ j ], one place to the right so as to
make a vacancy at correct relative position for A [ j ] }
end; {of while loop}
A [i + 1) m
(i.e., m = A [ j ] is stored at the correct relative position}
end {if}
end ; { of for loop }
end ; (of else part)
end; {of procedure}
2.8.2 Bubble sort
The Bubble Sort algorithm for sorting of n numbers, represented by an array
A [1..n], proceeds by scanning the array from left to right. At each stage,
compares adjacent pairs of numbers at positions A [ i ] and A [i + 1] and
whenever a pair of adjacent numbers is found to be out of order, then the
positions of the numbers are exchanged. The algorithm repeats the process
for numbers at positions A [i + 1] and A [i + 2]
Thus, in the first pass after scanning once all the numbers in the given list,
the largest number will reach its destination, but other numbers in the array,
may not be in order. In each subsequent pass, one more number reaches
its destination.
Example:
In the following, in each line, pairs of adjacent numbers, shown in bold, are
compared. And if the pair of numbers are not found in proper order, then the
positions of these numbers are exchanged.
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 58
The list to be sorted has n = 6 as shown in the first row below:
Iteration number i = 1
80 32 31 110 50 40 (j = 1)
32 80 31 110 50 40 (j = 2)
32 31 80 110 50 40
32 31 80 110 50 40
32 31 80 50 110 40 (j = 5)
32 31 81 50 40 110 (j = 1)
remove from further
consideration
In the first pass traced above, the maximum number 110 of the list reaches
the rightmost position (i.e., 6th position). In the next pass, only the list of
remaining (n – 1) = 5 elements, as shown in the first row below, is taken into
consideration. Again pairs of numbers in bold, in a row, are compared and
exchanged, if required.
Iteration number i = 2
31 32 81 50 40 (j = 2)
31 32 81 50 40 (j = 3)
31 32 50 81 40 (j = 4)
31 32 50 40 81 (j = 1)
remove from further
consideration
In the second pass, the next to maximum element of the list viz., 81,
reaches the 5th position from left. In the next pass, the list of remaining
(n – 2) = 4 elements are taken into consideration.
Iteration Number i = 3
31 32 50 40 (j = 2)
31 32 50 40 (j = 3)
31 32 40 50 (j = 1)
remove from further
consideration
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 59
In the next iteration, only (n – 3) = 3 elements are taken into consideration
31 32 40
31 32 40
In the next iteration, only n – 4 = 2 elements are considered
31 32
These elements are compared and found in proper order. The process
terminates.
Procedure bubblesort (A[1... n])
begin
for i 1 to n-1 do
for j 1 to (n-i)
{In each new iteration, as explained above, one less number of elements is
taken into consideration. This is why j varies upto only (n- i)}
If A[ j ] > A[j + 1] then interchange A[ j ] and A [j + 1]
end
{A [1.. n] is in increasing order}
2.8.3 Selection sort
Selection Sort for sorting a list L of n numbers, represented by an array
A [1.. n], proceeds by finding the maximum element of the array and placing
it in the last position of the array representing the list. Then repeat the
process on the sub array representing the sublist obtained from the list
excluding the current maximum element.
The difference between Bubbles sort and selection sort is that, in selection
sort to find the maximum number in the array, a new variable MAX is used
to keep maximum of all the values scanned upto a particular stage. On the
other hand, in Bubble sort, the maximum number in the array under
consideration is found by comparing adjacent pairs of numbers and by
keeping larger of the two in the position at the right. Thus, after scanning the
whole array once, the maximum number reaches the right-most position of
the array under consideration.
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 60
The following steps constitute the selection sort algorithm:
Step 1: Create a variable MAX to store the maximum of the values scanned
upto a particular stage. Also create another variable say Max-POS which
keeps track of the position of such maximum values.
Step 2: In each iteration, the whole list / array under consideration is
scanned once find out the position of the current maximum through the
variable MAX and we have to find out the position of the current maximum
through MAX-POS.
Step 3: At the end of iteration, the value in last position in the current array
and the (maximum) value in the position Max-POS are exchanged.
Step 4: For further consideration, replace the list L by L ~ {MAX} {and the
array A by the corresponding sub array} and go to step 1.
Example
80 32 31 110 50 40 ( given initially)
Initially, MAX 80 MAX-POS 1
After one iteration, finally, MAX 110, MAX-POS = 4
Numbers 110 and 40 are exchanged to get
80 32 31 40 50 110
New list, after one iteration, to be sorted is given by:
80 32 31 40 50
Initially, MAX 80, MAX-POS 1
Finally, also Max 80, MAX-POS 1
entries 80 and 50 are exchanged to get
50 32 31 40 80
New List, after one iteration, to be sorted is given by:
50 32 31 40
Initially, MAX 50, MAX0-POS 1
Finally, also Max 50, MAX-POS 1
entries 50 and 40 are exchanged to get
40 32 31 50
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 61
New List, after third iteration, to be sorted is given by:
40 32 31
Initially & finally
Max 40; MAX-POS 1
Therefore, entries 40 and 31 are exchanged to get
31 32 40
New List, after fourth iteration, to be sorted is given by:
31 32
Max 32 and MAX-POS 2
no exchange, as 32 is the last entry
New List after fifth iteration, to be sorted given by: 31
This is a single element list. Hence, no more iterations. The algorithm
terminates. This completes the sorting of the given list.
Next, we formalize the method used in Sorting the list
Procedure Selection Sort: (A [1.. n])
begin {of procedure}
for i 1to (n – 1)do
begin {of i for loop}
MAX A[ i ];
MAX-POS i
for j i + 1 to n do
begin {j for-loop}
if MAX < A[j] then
begin
MAX-POS j
MAX A[j]
end {of if}
end {of j for-loop}
A[MAX-POS] A[n – i + 1]
A[n – i + 1] MAX
{the ith maximum value is stored in the ith position from right or equivalent in
(n – i + 1) the position of the array
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 62
end {of i loop}
end {of procedure}.
2.8.4 Shell Sort
The sorting algorithm is named so in honour of D.L Short (1959), who
suggested the algorithm. Shell sort is also called diminishing-increment sort.
The essential idea behind Shell-Sort is to apply any of the other sorting
algorithm (generally Insertion Sort) to each of the several interleaving
sublists of the given list of numbers to be sorted. In successive iteration, the
sublists are formed by stepping through the file with an increment INCi taken
from some pre-defined decreasing sequence of step-sizes INC1 > INC2 >
………> INCi >…………...>1, which must terminate in 1.
Examples: Let the list of numbers to be sorted, be represented by the next
row.
13 3 4 12 14 10 5 1 8 2 7 9 11 6 (n = 14)
Initially, we take INC as 5, i.e.,
Taking sublist of elements at 1st, 6th and 11th positions, viz., sublsit of values
13, 10 and 7. After sorting these values we get the sorted sublist
7 10 13
Taking sublist of elements at 2nd, 7th and 12th positions, viz, sublist of values
3, 5 and 9. After sorting these values we get the sorted sublist.
3 5 9
Taking sublist of elements at 3rd, 8th and 13th positions, viz, sublist of values
4, 1 and 11. After sorting these values we get the sorted sublist
1 4 11
Similarly, we get sorted sublist
6 8 12
Similarly, we get sorted sublist
2 14
{Note that, in this case, sublist has only two elements it is 5th sublist and
n – 14 is less than 5INCINC
14 where INC = 5}
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 63
After merging or interleaving the entries from the sublists, while maintaining
the initial relative positions, we get the New List:
7 3 1 6 2 10 5 4 8 14 13 9 11 12
Next, take INC = 3 and repeat the process, we get sorted sublists:
5 6 7 11 14,
2 3 4 12 13 and
1 8 9 10
After merging the entries from the sublists, while maintaining the initial
relative positions, we get the New List
New List
5 2 1 6 3 8 7 4 9 11 12 10 14 13
Taking INC = 1 and repeat the process, we get the sorted list
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Note: Sublists should not be chosen at distances which are multiples of
each other e.g., 8, 4, 2 etc. Otherwise, same elements may be compared
again and again.
Procedure Shell — Sort (A [1..n] : real)
K: integer
{to store value of the number of increments}
INC [1..k] :integer
{to store the values of various increments}
begin {of procedure}
read (k)
for i 1 to (k — 1) do
read (INC [ i ]
INC [k] 1
{last increment must be 1}
For i 1 to k do
{this i has no relation with previous occurrence of i)
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 64
Begin
j INC [ i ]
r [n / j];
for t 1 to k do
{for selection of sublist starting at position t}
begin
if n < r * j + t
then s r — 1
else s r
Insertio-Sort (A[ t… (t + s*j)])
end {of t-for-loop}
end {i for-loop}
end { of procedure}
2.8.5 Heap Sort
In order to discuss Heap-Sort algorithms we recall the following definitions:
Binary Tree: A tree is called a binary tree, if it is either empty, or it consists of a
node called the root together with two binary trees called the left subtree
and a right subtree. Now , we make the following observations.
1. It may be noted that the above definition is a recursive definition, in the
sense that definition of binary tree is given in its own terms (i.e. binary tree).
2. The following are all distinct and the only binary trees are having two
nodes.
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 65
The following are all distinct and only binary trees are having three
nodes
Heap: is defined as a binary tree with keys assigned to its nodes (one key
per node) such that the following conditions are satisfied.
i) The binary tree is essentially complete (or simply complete), i.e., all its
levels are full except possibly the last where only some rightmost leaves
may be missing.
ii) The key at each node is greater than or equal to the key at its children.
The following binary tree is Heap
10
5 7
4 2 1
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 66
However, the following is not a heap because the value 6 in a child node is
more than the value 5 in the parent node.
Also, the following is not a heap, because some leaves (e.g., right child
of 5), in between two other leaves (viz 4 and 1), are missing.
Alternative definition of Heap: Heap is an array H [1.. n] in which every
element in position i (the parent) in the first half of the array is greater than
or equal to elements in positions 2i and 2i + 1 (the children).
HEAP SORT is a three-step algorithm as discussed below:
i) Heap Construction for a given array.
ii) (Maximum deletion) Copy the root value (which is maximum of all values
in the Heap) to right-most yet-to-be occupied location of the array used
to store the sorted values and copy the value in the last node of the tree
(or of the corresponding array) to the root.
iii) Consider the binary tree (which is not necessarily a Heap now) obtained
from the Heap through the modification through step (ii) above and by
removing currently the last node from further consideration. Convert the
binary tree into a Heap by suitable modifications.
10
5 7
6 2 1
10
5 7
4 1
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 67
Example:
Let us consider applying Heap Sort of the list 80 32 31 110 50 40
120 represented by an array A [1... 7]
Step 1: Construction of a Heap for the given list
First, we create the tree having the root as the only node with node-value 80
as shown below:
Next, value 32 is attached as left child of the root, as shown below
As 32 < 80, therefore, heap property is satisfied. Hence, no modification of
the tree. Next, value 31 is attached as right child of the node 80, as shown
below
Again, as 31 < 80, heap property is not disturbed. Therefore, no modification
of the tree.
80
32
80
32 31
80
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 68
Next, value 110 is attached as left child of 32; as shown below:
However, 110 > 32, the value in child node is more than the value in the
parent node. Hence the tree is modified by exchanging the values in the two
nodes so that, we get the following tree.
Again, as 110 > 80, the value in child node is more than the value in the
parent node. Hence, the tree is modified by exchanging the values in the
two nodes so that, we get the following tree
80
32 31
110
110
80 31
32
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 69
Next, number 50 is attached as right child of 80, so that the new tree is as
given below
As the tree satisfies all the conditions of a Heap, we insert the next number
40 as left child of 31 to get the tree
As the new insertion violates the condition of Heap, the values 40 and 31
are exchanged to get the tree which is a heap
110
80 31
32 50
110
80 31
32 50 40
110
80 40
32 50 31
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 70
Next, we insert the last value 120 as right child of 40 to get the tree
The last insertion violates the conditions for a Heap. Hence, 40 and 120 are
exchanged to get the tree.
Again, due to movement of 120 upwards, the Heap property is disturbed at
nodes 110 and 120. Again, 120 is moved up to get the following tree which
is a heap
After having constructed the Heap through Step 1 above, we consider
110
80 40
32 50 31 120
110
80 120
32 50 31 40
120
80 110
32 50 31 40
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 71
Step 2 & 3: Consists of a sequence of actions viz, (i) deleting the value of
the root ii) moving the last entry to the root and (iii) then readjusting the
heap. The root of a heap is always the maximum of all the values in the
nodes of the tree. The value 120 currently in the root is saved in the last
location B [n] in our case B [7] of the sorted array, say, B[1. .n] in which the
values are to be stored after sorting in increasing order.
Next, value 40 is moved to the following binary tree, which is not a heap
In order to restore the above tree as a heap, the value 40 is exchanged with
the maximum of the values of its two children. Thus, 40 and 110 are
exchanged to get the tree which is a heap.
40
80 110
32 50 31
110
80 40
32 50 31
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 72
Again, 110 copied to B[6] and 31, the last value of the tree is shifted to the
root and last node is removed from further consideration to get the following
tree, which is not a heap
Again, the root value is exchanged with the value which is maximum of its
children‘s value i.e., exchanged with value 80 to get the following tree, which
again is not a Heap.
Again, the value 31 is exchanged with the maximum of the values of its
children, i.e., with 50, to get the tree which is a heap.
80
50 40
32 31
31
80 40
32 50
80
31 40
32 50
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 73
Again, 80 is copied in B[5] and 31, the value of the node replaces 80 in the
root and the last node is removed from further consideration to get the tree
which is not a heap.
Again, 50, the maximum of the two children‘s values is exchanged with the
value of the root 31 to get tree, which is not a heap.
Again, 31 and 32 are exchanged to get the heap
Next, 50 is copied in B [4]
50
32 40
31
50
31 40
32
31
50 40
32
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 74
The entry 31 in the last node replaces the value in the root and the last node
is deleted to get the following tree which is not a Heap
Again, 40, the maximum of the values of children is exchanged with 31, the
value in the root. We get the Heap
Again, 40 is copied in B [3]. The value in the last node of the tree viz 31,
replaces the value in the root and the last node is removed from further
consideration to get the tree, which is not a heap.
Again, 32, the value of its only child is exchanged with the value of the root
to get the Heap
32
31
40
32 31
31
32 40
31
32
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 75
Next, 32 is copied in B [2] and 31, the value in the last node is copied in the
root and the last node is deleted, to get the tree which is a heap.
This value is copied in s and the Heap Sort algorithm terminates.
The list L of n numbers to be sorted is represented by an array A [1..n]
The following procedure reads one by one the values from the given to-be-
sorted array A[1...n] and gradually builds a heap. For this purpose, it calls
the procedure Build-Heap. For building the heap, an array H [1..n] is used
for storing the elements of the heap. Once the Heap is built, the elements of
A [1..n] are already sorted in H [1..n] and hence A may be used for sorting
the elements of finally sorted list from which we used the array B. Then, the
following three steps are repeated n times (n, the number of elements in the
array), in the ith iteration.
i) The root element H [1] is copied in A [n — i + 1] location of the given
array A. The first time, root element is stored in A [n]. The next time, root
element is stored in A [n – 1] and so on.
ii) The last element of the array H [n – i + 1] is copied in the root of the
heap, i.e., in H [1] and H [n – i + 1] is removed from further
consideration. That is in the next iteration, only the array H [1 ... (n – i)]
(which may not be a heap) is taken into consideration.
iii) The procedure Build-Heap is called to build the array H [1.. (n – i)] into a
heap.
Procedure Heap-Sort (A [ 1..n]: real)
begin {of the procedure}
H[1..n] Build – Heap (A [1..n])
For i 1 to n do
begin {for loop}
A[n – i + 1] H [1]
H[1] H [n – i + 1]
Build – Heap (H[1 ... (n — i)])
end {for – loop}
end {procedure}
31
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 76
The following procedure takes an array B[1...m] of size m, which is to be
sorted, and builds it into a heap
Procedure Build-Heap (B [1.. m] : real)
begin {of procedure}
for j = 2 to m do
begin {of for-loop}
location j
while (location > 1) do
begin
parent {location / 2]
If A [location] A [parent] then return {i.e., quite while loop]
else
(i.e., of A [location] > A [parent] then}
begin {to exchange A [parent] and A [location]}
temp A [parent]
A [parent] A [location]
A[location] temp
end {of if .. then ..else}
location parent
end {while loop}
end {for loop}
end { of procedure}
Self Assessment Questions
10. The –––––––––– algorithm for sorting of n numbers, represented by an
array A [1..n], proceeds by scanning the array from left to right.
11. The essential idea behind ––––––––– is to apply any of the other sorting
algorithm to each of the several interleaving sublists of the given list of
numbers to be sorted.
2.9 Summary
We Studied the different types of functions and Notations with well
illustrated example.
The concept of Modular arithmetic/ Mod function is well defined with
simple examples.
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 77
The concept of Mathematical Expectation and its use in average case
analysis of algorithms is well illustrated.
In this unit, we discussed that if a problem is algorithmically solvable
then it may have more than one algorithmic solutions. In order to choose
the best out of the available solutions, there are criteria for making such
a choice. The complexity/efficiency measures or criteria are based on
requirement of computer resources by each of the available solutions.
In this unit, we also studied the different types of Asymptotic functions
and their Notations.
The different methods to analyse the algorithm is studied step by step in
this unit .
2.10 Terminal Questions
1. Define an injective function
2. Define monotonic functions
3. Define exponential function with an example
4. Write the advantages of theoretical approach over the empirical
approach
5. For the function f(x) = 2x3 + 3x2 + 1,
show that
i) f(x) = (x3)
ii) f(x) (x2)
iii) f(x) (x4)
2.11 Answers
Self Assessment Questions
1. f(x) f(y)
2. Ceiling function
3. Exponentiation function
4. ex
5. Mathematical Expectation
6. time complexity
7. Notation O
Fundamentals of Algorithms Unit 2
Sikkim Manipal University Page No. 78
8. 0)x(g
)x(fLimx
9. 2
)1n(n
10. Bubble Sort
11. Shell-Sort
Terminal Questions
1. Refer Section 2.2.2
2. Refer Section 2.2.4.
3. Refer Section 2:3.5
4. Refer Section 2.6
5. i) for C1 =3, C2 = 1 and k = 4 i.e. 1. C2 x3 f(x) C1 x
3 for all x k
ii) We can show by contradiction that no C1 exists. Let, if possible for
some positive integers k and C1, we have 2x3 + 3x2 + 1 C1. x2 for
all x k hence x3 C1 x2 for all x k
i.e.,x C1 for all x k
But for x = max {C1 + 1, k}
The last inequality is not true.
Therefore, x = max {C1 + 1, k}
iii) f(x) (x4)
we can show by contradiction that there does not exist C2
such that
C2 x4 (2x3 + 3x2 + 1)
If such a C2 exists for some k then C2 x4 2x3 + 3x2 + 1 6x3 for all
x 1, k 1, implying C2 x 6 for all x k
But for 16
2Cx
The above inequality is false. Hence, proof by contradiction.
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 79
Unit 3 Divide and Conquer Structure
3.1 Introduction
Objectives
3.2 Divide and Conquer Strategy
3.3 Binary Search
3.4 Max. and Min.
3.5 Merge Sort
3.6 Quick Sort
3.7 Summary
3.8 Terminal Question
3.9 Answers
3.1 Introduction
After studying the different concepts in mathematical functions and notaions,
we will discuss the concept of divide and conquer.
Given a function to compute n inputs, the divide-and-conquer strategy
suggests splitting the inputs into K distinct subsets, 1< K n yielding K
subproblems. These subproblems must be solved and then a method must
be found to combine subsolutions into a solution as the whole. If the
subproblems are still relatively large, then the divide-and-conquer strategy
can be possibly re-applied.
Often, the subproblems resulting from a divide-and-conquer design are of
the same type as the original problem. For those cases, the reapplication of
the divide-and-conquer principle is naturally expressed by a recursive
algorithm. Now, smaller and smaller subproblems of the same kind are
generated until eventually subproblems that are small enough to be solved
without splitting are produced.
Objectives
After studying this unit, you should be able to:
apply divide and conquer strategy to practical problems
analyze the binary search algorithm and compute the time for successful
and unsuccessful search
analyze Max and Min algorithm using divide-and-conquer method
analyze the time for the Merge Sort Algorithm
use the working of Quick Sort Algorithm.
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 80
3.2 Divide and Conquer Strategy
Given a function to compute on n inputs, the divide-and-conquer strategy
suggests splitting the inputs into K distinct subsets, 1<K<n, yielding K
subproblems. These subproblems must be solved and then a method must
be found to combine subsolutions into a solution as a whole. If the
subproblems are still relatively large, then the divide-and-conquer strategy
can be possibly reapplied.
We can write a control abstraction that mirrors the way an algorithm based
on divide-and-conquer will look. By a control abstractions we mean a
procedure whose flow of control is clear but whose primary operations are
specified by other procedures whose precise meanings are left undefined.
Algorithm 3.1
1.procedure DANDC (p,q)
2.global integer n, A(1:n)
3.integer p,q; //1 p q n//
4.local integer m;
5.if small (p,q)
6.then return (G(p,q))
7.else m divide (p,q) //p < m < q
8.return (combine(DANDC(p,m), DANDC(m + 1,q)))
9.endif
10.end DANDC
In Algorithm 3.1, procedure is called by DANDC (1,n)
– Small (p,q) is a Boolean function and will return true or false. True if
input size is small enough that the answer can be computed without
splitting. If so, function G is invoked.
– Divide (p,q), divides into (p, m) and (m+1, q)
– Combine function combines the results of p,m and m+1, q. Combine is a
function that determines the solution to p using the solutions to the K
sub problems.
If the size of p is n and the sizes of the K subproblems are n1, n2…….., nk
respectively, then the computing time of DANDC is described by the
recurrence relation.
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 81
otherwise
smallisnif
nfnT.....nTnT
ngnT
k21
otherwise
smallnif
nfn/22T
ngnTe.
i.
Where, T(n) is the time for DANDC on any input of size n & g (n) is the time
to compute the answer directly for small inputs. The function f(n) is the time
for dividing P and combining the solutions to subproblems.
The complexity of many divide-and-conquer algorithms is given by
recurrences of the form.
1nnfb/naT
1n1TnT
Where, a and b are known constants. We assume that T(1) is known and n
is a power of b (i.e. n = bk)
Self Assessment Question
1. The complexity of many divide-and-conquer algorithms is given by
recurrences of the form –––––––––––––.
3.3 Binary Search
Let ai, 1 < i < n be a list of elements that are sorted in non-decreasing order.
Consider the problem of determining whether a given element x is present in
the list. If x is present we are to determine a value j such that aj = x. If x is
not present in the list, then j is to be set to zero.
Let P = (n, ai, ….., a1, x) denote an arbitrary instance of this search problem,
where,
n is the number of element in the list, ai ……….. a1 is the list of elements
and
x is the element searched for.
We use divide-and-conquer strategy to solve this problem. Let small (p, q)
be true for n = 1.
In this case, G(p, q) will take the value i if x = ai, otherwise it will take the
value 0.
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 82
Then, g(1) = (1), if P has more than one element, it can be divided (or
reduced) into a new subproblem as follows:
Pick an index q in the range [i, I] and compare x with aq. There are 3
possibilities.
1) x = aq: In this case, the problem P is immediately stored.
2) x < aq: In this case, x has to be searched in the sublist (ai, ai+1, ….., aq–1)
P reduces to (q – i, a1, aq–1, x)
3) x > aq: In this case, x has to be searched in the sublist (aq+1, ……….. , ai)
P reduces to (i – q,aq+1, …………..,ai, x)
Any given problem, P gets divided (reduced) into one new subproblem. This
division takes only (1) time. After a comparison with aq, the instance
remaining to be solved (if any) can be solved by using divide-and-conquer
rule.
If q is always chosen such that aq is the middle element (i.e. q = (n+1)/ 2),
then, the resulting search algorithm is known as binary search.
Algorithm 3.2 shows how binary search works for n elements where n 0.
Algorithm 3.2
1. procedure BINSRCH (x, j)
2. integer low, mid, high, x,j;
3. low 1, high n;
4. while low high do
5. mid (low + high)/2
6. case
7. :x<A(mid):high mid — 1
8. :x>A(mid): low mid + 1
9. :else:j mid; return
10. end case
11. repeat
12. j o
13. endBINSRCH
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 83
Suppose there are n elements, then, for successful search there is n
comparisons and for unsuccessful search there is n + 1 comparisons.
To determine the average behaviour, we need to look more closely at the
binary decision tree and equate its size to the number of element
comparison in the algorithm.
The distance of a node from the root is one less than its level.
The internal path length l is the sum of the distances of all internal nodes
from the root.
The external path length E is the sum of the distances of all external
nodes from the root.
Let s(n) = average number of comparisons in a successful search.
u(n) = average number of comparisons in an unsuccessful search.
For successful search
Total distance of all node to the root = l
Average distance of a node from root = n
l
Average number of element comparison for successful search = 1n
l
n
l1nS ––––––––––– (1)
For unsuccessful search
Total distance from node = E
Average distance of any node from root
1n
Elnu
––––––––––– (2)
E (n + 1)
In other words, E n(n + 1)
Substituting in (2) nlognu
Self Assessment Question
2. Average number of element comparison for successful search –––––––.
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 84
3.4 Max. and Min.
The problem is to find the maximum and minimum items in a set of n
elements.
Algorithm 3.3
1. procedure SMAXMIN(max, min)
2. integer i, max, min;
3. max min A(1);
4. for I 2 to n do
5. if A(i)>max then max A(i)
6. else if A(i)<min then min A(i)
7. endif
8. endif
9. repeat
10. end SMAXMIN
Best Case: If we have n element, (n – 1) element comparison.
Worst Case: If n elements are in descending order, 2 comparison will take
place.
We have 2(n – 1) element comparison.
Average Case: 1n2
3 element comparison
Algorithm 3.4 MaxMin is a recursive algorithm that finds the maximum and
the minimum of set of elements {a(i),a(i+1)….,a(j)}. The situation of set sizes
one (i = j) and two (i = j – 1) are handled separately. For sets containing
more than two elements, the midpoint is determined (just as in binary
search) and two new subprograms are generated. When the maxima and
minima of these subproblems are determined, the two maxima are
compared and two minima are compared to achieve the solution for the
entire set.
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 85
Algorithm 3.4
1. Algorithm MaxMin (i,j, max, min)
2. // a[1:n] is a global array Parameter i and j are integers,
3. // 1 i j < n. The effect is to set max and min to the
4. // largest and smallest values in a [i, j], respectively.
5. {
6. if (i=j) then max:=min:=a[i]; //small(P)
7. else if (i = j – 1) then //Another case of Small (P)
8. {
9. if a [i] < a[j] then
10. {
11. max:=a[j]; min:=a[i];
12. }
13. else
14. {
15. max:=a[i]; min:=a[j]
16. }
17. }
18. else
19. {
20. // If P is not small, divide P into subproblems
21. // Find where to split the set
22. mid:= (i+j)/2;
23. //Solve the subproblems
24. Maxmin (i, mid, max, min);
25. MaxMin (mid+i, j, max1, min1);
26. // Combine the solutions
27. if (max<max 1) then max:=max1;
28. if(min>min1) then min:=min1;
29. }
30. }
Algorithm 3.4 Recursively finds the maximum and minimum
The procedure is initially invoked by the statement MaxMin(1,n,x,y)
Suppose, we simulate MaxMin on the following nine elements:
A: [1] [2] [3] [4] [5] [6] [7] [8] [9]
22 13 –5 –8 15 60 17 31 47
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 86
A good way of keeping track of recursive calls is to build a tree by adding a
node each time a new call is made. For this algorithm, each node has four
items of information: i,j, max and min. On the array a [ ] above, the tree of
Fig. 3.1 is produced.
Fig. 3.1: Trees of recursive calls of MaxMin
Examining Fig 3.1, we see that the root node contains 1 and 9 as the values
of i and j corresponding to the initial call to MaxMin. This execution produces
two new calls to MaxMin, where i and j have the values 1, 5 and 6, 9
respectively and thus split the set into two subsets of approximately the
same size. From the tree we can immediately see that the maximum depth
of recursion is four (including the first call). The circled numbers in the upper
left corner of each node represent the orders in which max and min are
assigned values.
If (n) represents this number, then the resulting recurrence relation is
1n0
2n1
2n22/nT2/nT
nT
When n is a power of two, n=2k for some positive integer k, then
22
nT2)n(T
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 87
224
nT22
244
nT4
…
….
1ki1
i1k 2)2(T2
22
n3222 k1k
Note that 22
n3 is the best, average and worst-case number of
comparisons when n is a power of two.
Self Assessment Question
3. A good way of keeping track of –––––––– is to build a tree by adding a
node each time a new call is made.
3.5 Merge Sort
We assume throughout that the elements are to be sorted in non-decreasing
order. Given a sequence of n elements (also called keys) a[1], ……., a[n],
the general idea is to imagine them split into two sets a[1], …….. a[n/2]
and a[n/2+1],………..,a[n]. Each set is individually sorted and the resulting
sorted sequences are merged to produce a single sorted sequence of n
elements.
Thus, we have another ideal example of the divide-and-conquer strategy in
which the splitting is into two equal-sized sets and the combining operation
is the merging of two sorted sets into one.
Mergesort (Algorithm 3.5) describes this process very succinctly using
recursion and a function Merge (Algorithm 3.6) which merges two sorted
sets. Before executing MergeSort, the n elements should be placed in a
[1:n]. Then MergeSort (1, n) causes the keys to be rearranged into the non-
decreasing order in a
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 88
Algorithm 3.5
1. Algorithm MergeSort (low, high)
2. //a[low:high] is a global array to be sorted.
3. //small (P) is true if there is only one element
4. // to sort. In this case the list is already sorted.
5. {
6. if (low<high> then //If there are more than one element
7. {
8. //Divide P into subproblems
9. //Find where to split the set
10. mid:= (low+high)/2
11. //Solve the subproblems
12. MergeSort (low,mid);
13. MergeSort(mid+1, high);
14. //combine the solutions
15. Merge(low, mid, high)
16. }
17. }
Algorithms 3.6
Algorithm Merge (low, mid, high)
//a[low:high] is a global array containing two sorted
//subsets in a [low:mid} and in a [mid+1 :high]. The
//goal is to merge these two sets into a single set residing
// in a [low:high],b[ ] is an auxiliary global array.
{
h:=low; i:=low; j:=mid+1;
while ((h mid) and (j high)) do
{
If (a[h] a[j] then
{
b[i]: a[h]; h:=h+1;
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 89
}
Else
{
b[i]:=a[j]; j:=j+1;
}
i:i+1;
}
If (h>mid) then
for k = j to high do
{
b[i]:=a[k]; i:=i+1;
}
else
For k:=h to mid do
{
b[i]:=a[k]; i:=i+1;
}
for k:=low to high do a[k]:=b[k];
}
Fig. 3.2: Tree of calls of MergeSort (1, 10)
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 90
Fig. 3.3: Tree of calls of Merge
Fig. 3.2 is a tree that represents the sequence of recursive calls that are
produced by MergeSort when it is applied to ten elements. The pair of values in
each node are the values of the parameters low and high. Notice how the
Splitting continues until sets containing a single element are produced. Fig. 3.3
is a tree representing the calls to produce Merge by MergeSort.
For example, the node containing 1, 2 and 3 represent the merging of [1:2]
with a[3].
If the time for the merging operation is proportional to ‘n’, then the
computing for Mergesort is described by the recurrence relation.
ttanconsaisc,1ncn2/naT
ttanconsisa,1nanT
When n is a power of 2, n=2k, we can solve this equation by successive
substitutions
cn2
nc
4
nT22)n(T
cn24
nT4
cn24
cn
8
nT24
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 91
……
…….
kcn)1(T2k
nlogcnan
It is easy to see that if 2k < n 2k+1, then
T(n) T(2k+1)
Therefore, nlognOnT
Self Assessment Question
4. If the time for the merging operation is proportional to ‘n’, then the
computing for Mergesort is described by the recurrence relation ––––––.
3.6 Quick Sort
In merge sort, the file a[1:n] was divided at its midpoint into subarrays which
were independently sorted and later merged.
In quick sort, the division into two subarrays is made so that the sorted
subarrays do not need to be merged later. This is accomplished by
rearranging the elements in a [1:n] such that a[i] a[j] for all i between 1 and
m and all j between m+1 and n for some m, 1 m n. Thus, the elements in
a [1:m] and a[m+1:n] can be independently sorted. No merge is needed.
The rearrangement of the elements is accomplished by picking some
element of a[ ], say, t=a[s] and then reordering the other elements so that all
elements appearing before t in a[1:n] are less than or equal to t and all
elements appearing after t are greater than or equal to t. This rearranging is
referred to as partitioning.
Function Partition of Algorithm 3.7 accomplishes an in-place partitioning of
the elements of a[m:p – 1]. It is assumed that a[p] a[m] and that a[m] is the
partitioning element. If m = 1 and p – 1 = n, then a[n + 1] must be defined
and must be greater than or equal to all elements in a[1:n]. The assumption
that a[m] is the partition element is merely for convenience; other choices for
the partitioning element than the first item in the set are better in practice.
The function, interchange (a, i, j) exchanges a[i] with a [j].
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 92
Algorithm 3.7
1. Algorithm Partition (a,m,p)
2. //within a[m], a[m+1], …., a[p-1] the elements are
3. //rearranged in such a manner that if initially t=a[m],
4. //then after completion a[q]=t for some q between m
5. // and p-1, a[k] t for m k q, and a[k] t
6. //for q<k<p. q is returned. Set a[p]=
7. {
8. V:=a[m]; i:=m; j:=p;
9. repeat
10. {
11. repeat
12. i:=i+1
13. until (a[i]v);
14. repeat
15. j:=j-1;
16. until (a[j]v);
17. if (i<j) then Interchange (a, i, j);
18. } until (ij);
19. a[m]:=a[j]; a[j]:=v; return j;
20. }
1. Algorithm Interchange (a,i,j)
2. //Exchange a[i] with a[j]
3. {
4. p:=a[i];
5. a[i]:=a[j]; a[j]:= p;
6. }
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 93
Algorithm 3.8
1. Algorithm QuickSort(p,q)
2. //Sorts the elements a[p],……,a[q] which reside in
3. //global array a[1:n] into ascending order; a[n+1] is
4. //considered to be defined and must be all the
5. // elements in a [1:n]
6. {
7. if (p<q) then //If there are more than one element
8. {
9. // divide p into two subproblems
10. j:=Partition(a, p, q+1);
11. //j is the position of the partitioning element
12. // Solve the subproblems
13. QuickSort(p, j-1);
14. QuickSort(j+1, q);
15. // There is no need for combining solutions.
16. }
17. }
In analyzing QuickSort, we count only the number of element comparisons
C(n). It is easy to see that the frequency count of other operations is of the
same order as C(n). We make the following assumptions: the n elements to
be sorted are distinct, and the input distribution is such that the partition
element v=a[m] in the call to Partition (a,m,p) has an equal probability of
being the ith smallest element, 1 i p-m, in a[m:p-1].
First, let us obtain the worst-case value Cw(n) of C(n). The number of
element comparisons in each call of Partition is at most p – m + 1. Let r be
the total number of elements in all the calls to Partition at any level of
recursion. At level one only one call, Partition (a, 1, n+1), is made and
r = n; at level two at most two calls are made and r = n – 1; and so on. At
each level of recursion, O(r) element comparisons are made by Partition. At
each level, r is atleast one less than the r at the previous level, as the
partitioning elements of the previous level are eliminated. Hence, Cw(n) is
the sum on r, as r varies from 2 to n, or O(n2).
The average value CA(n) of C(n) is much less than Cw(n).
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 94
Even though the worst-case time is O(n2), the average time is only O(n log
n). Let us now look at the stack space needed by the recursion. In the worst
case the maximum depth of recursion may be n – 1. This happens, for
example, when the partition element on each call to Partition is the smallest
value in a[m:p-1]. The amount of stack space needed can be reduced to
O(log n) by using an iterative version of Quick Sort in which the smaller of
the two subarrays a[p : j – 1] and a[j+1:q] is always sorted first. Also, the
second recursive call can be replaced by some assignment statements and
a jump to the beginning of the algorithm.
Algorithm 3.9
1. Algorithm QuickSort 2(p, q)
2. //Sorts the elements in a [p: q]
3. {
4. //stack is a stack of size 2 log (n)
5. repeat
6. {
7. while (p<q) do
8. {
9. j:=Partition (a,p,q+1):
10. if (j-p)<(q-j) then
11. {
12. Add (j+1); //Add j+1 to stack
13. Add(q); q:=j-1; // Add q to stack
14. }
15. else
16. {
17. Add(p; //Add p to stack)
18. Add (j-p); p:=j+1; //Add j-1 to stack
19. }
20. }// Sort the smaller subfile
21. if stack is empty then return:
22. Delete (q); Delete (p); //Delete q and p from stack
23. } until (false);
24. }
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 95
We can now verify that the maximum stack space needed is O(log n)
Let S(n) be the maximum stack space needed. Then it follows that
1n0
1n2
1nS2
nS which is less than 2 log n
Self Assessment Question
5. The average value CA(n) of C(n) is much less than ––––––––––––.
3.7 Summary
In this unit, we discussed that the complexity of many divide-and-
conquer algorithms is given by recurrences of the form
1n)n(f
b
naT
1n1T
nT (where a and b are known constants).
For successful binary search, n
l1nS and for unsuccessful binary
search, 1n
EnU
.
For Max and Min, 22
n3
is the best, average and worst-case number
of comparisons where n is a power of two.
For Merge sort , we have T(n) = O(n logn)
For quicksort, the worst case time is O(n2), the average time is only
O(n logn).
3.8 Terminal Questions
1. Briefly explain the Binary search Method.
2. Write the Algorithm to find the maximum and minimum items in a set of n
element.
3. Explain the concept of merge sort.
4. Sketch the Algorithm of Quick sort.
5. Write the Algorithm for sorting by Partitioning.
Fundamentals of Algorithms Unit 3
Sikkim Manipal University Page No. 96
3.9 Answers
Self Assessment Questions
1.
1nnfb/naT
1n1TnT
2. 1n
l
3. Recursive calls
4.
ttanconsaisc,1ncn2/naT
ttanconsisa,1nanT
5. Cw(n).
Terminal Questions
1. Let ai, 1 < i < n be a list of elements that are sorted in non-decreasing
order. Consider the problem of determining whether a given element x is
present in the list. If x is present, we are to determine a value j such that
aj = x. If x is not present in the list, then j is to be set to zero.
Let P = (n, ai, ….., a1, x) denote an arbitrary instance of this search
problem, where n is the number of element in the list, ai ……….. a1 is
the list of elements and x is the element searched for. (Refer section
3.3)
2. Refer section 3.4
3. Refer section 3.5
4. Refer section 3.6
5. Refer section 3.6
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 97
Unit 4 Greedy Method
Structure
4.1 Introduction
Objectives
4.2 Greedy Method Strategy
4.3 Optimistic Storage on Tapes
4.4 Knapsack Problem
4.5 Job Sequencing with Deadlines
4.6 Optimal Merge Pattern
4.7 Single Source Shortlist Paths
4.8 Summary
4.9 Terminal Questions
4.10 Answers
4.1 Introduction
In the previous unit, we have discussed the different techniques of
algorithms. Now we will study a well known method known as the greedy
method. The greedy method is the most straightforward design technique,
and it can be applied to a wide variety of problems. Most of these problems
have n inputs and require us to obtain a subset that satisfies some
constraints. Any subset that satisfies these constraints is called a feasible
solution. We need to find a feasible solution that either maximizes or
minimizes a given objective function. A feasible solution that does this is
called an optimal solution. There is usually a simple way to determine a
feasible solution but not necessarily an optimal solution.
Objectives
After studying this unit, you should be able to:
describe the greedy method approach
apply the greedy method for optimistic storage on tapes, knapsack
problem
define the job sequencing problem using greedy method.
4.2 Greedy Method Strategy
The greedy method suggests that one can device an algorithm that works in
stages, considering one input at a time. At each stage, a decision is made
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 98
regarding whether a particular input is in an optimal solution. This is done by
considering the inputs in an order determined by some selection procedure.
If the inclusion of the next input into the partially constructed optimal solution
will result in an infeasible solution, then this input is not added to the partial
solution. Otherwise, it is added. The selection procedure itself is based on
some optimization measure. This measure may be the objective function.
This version of the greedy technique is called the subset paradigm.
We can describe the subset paradigm abstractly, but more precisely than
above, by considering the control abstraction in Algorithm 4.1
Algorithm 4.1
1. Algorithm Greedy (a,n)
2. //a[1:n] contains the n inputs
3. {
4. Solution: = ;//initialize the solution
5. for i:=1 to n do
6. {
7. x: =select (a);
8. if feasible (solution, x) then
9. solution:=Union(solution, x);
10. }
11. return solution.
12. }
The function Select selects an input from a [ ] and removes it. The selected
input’s value is assigned to x. Feasible is a Boolean-valued function that
determines whether x can be included into the solution vector. The function
union combines x with the solution and updates the objective function. The
function Greedy describes the essential way that a greedy algorithm will
look, once a particular problem is chosen and the functions select, feasible
and union are properly implemented.
Problems that do not call for the selection of an optimal subset, in the
greedy method we make decisions by considering the inputs in some order.
Each decision is made using an optimization criterion that can be computed
using decisions already made. This version of the greedy method is called
the ordering paradigm.
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 99
Self Assessment Question
1. ––––––––– is a Boolean-valued function that determines whether x can
be included into the solution vector.
4.3 Optimistic Storage on Tapes
There are n programs that are to be stored on a computer tape of length l.
Associated with each program i is a length li, 1 i n. Clearly, all programs
can be stored on the tape if and only if the sum of the length of the program
is at most l.
We assume that whenever a program is to be retrieved from this tape, the
tape is initially positioned at the front. Hence, if the programs are stored in
the order I=i1, i2 -----in, the time tj needed to retrieve program ij is proportional
to .Ijk1
ik If all programs are retrieved equally often, then the expected or
mean retrieval time (MRT) is nj1
jtn
1. In the optimal storage on tape
problem, we are required to find a permutation for the n programs so that
when they are sorted on the tape in this order the MRT is minimized. This
problem fits the ordering paradigm.
Minimizing the MRT is equivalent to minimizing ik
jk1nj1
lld
A greedy approach to building the required permutation would choose the
next program on the basis of some optimization measure. One possible
measure would be the d value of the permutation constructed so far. The
next program to be stored on the tape would be one that minimizes the
increase in d. If we have already constructed the permutation i1, i2, ….. ir,
then appending program j gives the permutation i1, i2,….., ir, ir+1=j. This
increases the d value by nj1
jik .ll Since jk1
ikl is fixed and
independent of j, we trivially observe that the increase in d is minimized if
the next program chosen is the one with the least length form among the
remaining programs.
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 100
The greedy method simply requires us to store the programs in non-
decreasing order of their lengths. This ordering can be carried out in
O(n log n).
Theorem: If l1 l2 …… ln, then the ordering ij=j, 1 j n, minimizes
k
1j
ij
n
1k
l over all possible permutations of the ij.
Proof: Let l=i1,i2,…..,in be any permutation of the index set {1, 2, …..,n}.
Then
n
1k
jk
k
1j
ij
n
1k
l1knlId
If there exist a and b such that a<b and lia > Iib, then interchanging ia and ib
results in a permutation I with
iaib
bk
ik l1bnl1anl1knId
akk
Subtracting Id from Id ,we obtain
iaibibia ll1bnll1anIdId
ibia llab
> 0
Hence, no permutation that is not in non-decreasing order of the li’s can
have minimum d. It is easy to see that all permutations in non-decreasing
order of the li’s have the same d value. Hence, the ordering defined by ij=j,
1 j n minimizes the d value.
The tape storage problem can be extended to several tapes. If there are
m>1, tapes, T0, ….., Tm-1, then the programs are to be distributed over these
tapes. For each tape a storage permutation is to be provided If lj is the
storage permutation for the subset of programs on tape j, then d(lj) is as
defined earlier. The total time 1mj0
jldisTD . The objective is to store the
programs in such a way as to minimize TD.
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 101
The obvious generalization of the solution for the one-tape case is to
consider the program in non-decreasing order of li’s. The program currently
being considered is placed on the tape that results in the minimum increase
in TD. This tape will be the one with the least amount of tape used so far. If
there is more than one tape with this property, then the one with the
smallest index can be used. If the jobs are initially ordered so that
,l.....ll n21 then the first m programs are assigned to tapes T0,…..,Tm-1
respectively. The next m programs will be assigned to tapes T0,…….., Tm-1
respectively.
The General rule is that program i is stored on tape T i mod m• On any given
tape the programs are stored in non-decreasing order of their lengths.
Algorithm 4.2 presents this rule in pseudocode. It assumes that the
programs are ordered as above. It has a computing time of (n) and does
not need to know the program lengths.
Algorithm 4.2
1. Algorithm Store (n,m)
2. //n is the number of programs and m the number
3. // of tapes
4. {
5. j:=0; //Next tape to store on
6. for i:=1 to n do
7. {
8. write(”append program”, i,”
9. to permutation for tape”,j);
10. j:= (j+1)mod m,
11. }
12. }
Self Assessment Questions
2. A greedy approach to building the required permutation would choose
the next program on the basis of some–––––––––––––.
3. The ordering defined by ij=j, 1 j n, –––––––––– the d value.
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 102
4.4 Knapsack Problem
Given n objects and a knapsack or bag. Object i has a weight wi, and the
knapsack has a capacity m. If a fraction xi, 0 xi 1, of object i is placed into
the knapsack, then a profit of Pi xi is earned. The objective is to obtain a filling of
the knapsack that maximizes the total profit earned. Since the knapsack
capacity is m, we require the total weight of all chosen objects to be atmost m.
Formally, the problem can be stated as
Maxmise ni1
ii xP -------------------- 1
Subject to ni1
ii mxw ------------- 2
And ni1,1x0 i ------------------ 3
The profits and weights are positive numbers. A feasible solution (or filling)
is any set (x1, …., xn) satisfying 2 and 3 above. An optimal solution is a
feasible solution for which 1 is maximized.
The Greedy knapsack algorithm requires O(n) time. The Greedy knapsack
algorithm is given below.
Algorithm 4.3
Algorithm Greedyknapsack (m,n)
//p[1:n} and w[1:n] contain the profits and weights respectively
//of the n objects ordered such that p[i]/w[i] p[i+1]/w[i+1]
//m is the knapsack size and x[1:n] is the solution vector
{
for i:=1 to n do x[i]:=0.0;//Initialize x
u:=m;
for i:=1 to n do
{
if (w[i]>u) then break;
x[i]:=1.0; U:=U-w[i];
}
if (i n) then x[i]:=u/w[i];
}
Self Assessment Question
4. The profits and weights are –––––––– numbers.
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 103
4.5 Job Sequencing with Deadlines
We are given a set of n jobs. Associated with job i is an integer deadline
di 0 and a profit pi > 0. For any job i the profit pi is earned iff the job in
completed by its deadline. To complete a job, one has to process the job on
a machine for one unit of time. Only one machine is available for processing
jobs. A feasible solution for this problem is a subset J of jobs such that each
job in this subset can be completed by its deadline. The value of a feasible
solution J is the sum of the profits of the jobs in J, or
Ii
ip . An optimal
solution is a feasible solution with maximum value. Since the problem
involves the identification of a subset, it fits the subset paradigm.
Algorithm 4.4 demonstrates the greedy algorithm for sequencing unit time
jobs with deadlines and profits
Algorithm 4.4 1. Algorithm JS (d,j,n)
2. //d[i] 1, 1 i n are the deadlines, n 1. The jobs
3. // are ordered such that p[1] p[2] .... p[n].
4. //J[i] is the ith job in the optimal solution, 1 i k.
5. //Also, at termination d [J[i]] d[J[i+1]], 1 i k.
6. {
7. d[0]:=J[0]:=0;//lnitialize
8. J[1]:=1 ;// include job 1
9. K:=1;
10. for i:=2 to n do
11. {
12. //Consider jobs in non-increasing order of p[i]
13. //Find position for i and check feasibility for insertion
14. r:=k;
15. While((d[J[r]]>d[i]) and (d[J[r]] r)) dp r:= r – 1;
16. if ((d[J[r]] d[i]) and (d[i]>r)) then
17. {
18. // Insert i into J[ ].
19. for q:=k to (r+1) step – 1 do J[q+1]:=J[q];
20. J[r+1]:=i; k:=k+1;
21. }
22. }
23. return k;
24. }
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 104
Theorem: If p1/w1 p2/w2 ….. pn/wn. Then, Greedy knapsack algorithm
generates an optimal solution to the given instance of the knapsack
problem.
Proof: Let x = (x1, …….,xn) be the solution generated by Greedy knapsack
algorithm. If all the xi equal one, then clearly the solution is optimal. So, let j
be the least index such that xj 1. From the algorithm it follows that xi=1 for
1 i<j, xi = 0 for j<i n and 0 xj<1. Let y=(y1, ……, yn) be an optimal solution
we can assume that wjyi = m. Let k be the least index such that yk xk.
Clearly, such a k must exist. It also follows that yk<xk. To see this, consider
the three possibilities k<j, k=j or k>j.
1. If k<j, then xk=1, But, yk xk, and so yk < xk.
2. k=j, then since wiyi = m and yi = xi for 1 i<j. it follow that either yk<xk
or wiyi = m
3. k > j, then wiyi = m, and this is not possible.
Now, suppose we increase yk to xk and decrease as many of (yk+1,……., yn)
as necessary so that the total capacity used is still on. This results in a new
solution z=(z1, …….., zn) with zi=xi, 1 i k and k<i nwi (yi - zi) = wk(zk-yk),
Then, for z we have
ni1 ni1
iik
k
nik
iikkkii
nik i
iiiikkk
ni1
ii
ni1
ii
ypw
pzywyzyp
w
pwzywYzypzp
If iiii ypzp then y could not have been an optimal solution. If these
sums are equal, then either z=x and x is optimal, or z x. In the latter case,
repeated use of the above argument will either show that y is not optimal or
transform y into x and thus show that x too is optimal.
Greedy algorithm for sequencing unit time jobs with deadlines and profits
For JS, there are two possible parameters in terms of which its complexity
can be measured. We can use n, number of jobs and s, the number of jobs
included in the solution J. The while loop of line 15 in Algorithm 4.4 is
iterated at most K times. Each iteration takes (1) time. If the condition; of
line 16 is true, then lines 19 and 20 are executed. These lines require (k-r)
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 105
time to insert job i. Hence, the total time for each iteration of the for loop of
line 10 is (k). This loop is iterated n-1 times. If s is the final value of k, that
is, S is the number of jobs in the final solution, then the total time needed by
algorithm JS is (Sn). Since S n, the worst-case time, as a function of n
alone is (n2). If we consider the job set pi=di=n — i + 1, 1 i n, then
algorithm JS takes (n2) time to determine J. Hence. The worst-case
computing time for JS is (n2) In addition to the space needed for d, JS
needs (S) amount of space for J.
Self Assessment Question
5. If––––––––––, then Greedy knapsack algorithm generates an optimal
solution to the given instance of the knapsack problem.
4.6 Optimal Merge Pattern
When more than two sorted files are to be merged together, the merge can
be accomplished by repeatedly merging sorted files in pairs. Thus, if files x1,
x2, x3, and x4 are to be merged, we could first merge x1 and x2 to get a file yi.
then we could merge yi and x3 to get y2. Finally, we could merge y2 and x4 to
get the desired sorted file. Alternatively, we could first merge x1 and x2
getting yi, then merge x3 and x4 and get y2, and finally merge y1 ad y2 and
get the desired sorted file. Given n sorted files, there are many ways in
which we can pairwise merge them into a single sorted file. Different
pairings require differing amounts of computing time. The problem we
address ourselves to now is that of determining of an optimal way to
pairwise merge n sorted files. Since this problem calls for an ordering
among the pairs to be merged, it fits the ordering paradigm.
A greedy attempt to obtain an optimal merge path. Since merging of an n-
record file and an m-record file requires possibly n + m record moves, the
obvious choice for a selection criterion is at each step merge the two
smallest size files together. Thus, if we have files (x1,…….,x5) with sizes (20,
30, 10, 5, 30), our greedy rule would generate the following merge pattern:
merge x4 and x3 to get z1(lz1I=15), merge z1 and x1 to get z2(|z2| = 35) merge
x2 and x5 to get z3 (|z3|=60) and merge z2 and z3 to get the answer z4. The
total number of record moves is 205. One can verify that this is an optimal
merge pattern for the given problem instance.
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 106
The merge pattern such as the one just described will be referred to as a
two way merge pattern (each merge step involves the merging of two files).
The two-way merge pattern can be represented by binary merge trees. Fig.
4.1 shows a binary merge tree representing the optimal merge pattern
obtained for the above five files.
Fig. 4.1: Binary merge tree representing a merge pattern
The leaf nodes are drawn as squares and represent the given five files.
These nodes are called external nodes. The remaining nodes are drawn as
circles and are called internal nodes. Each internal node has exactly two
children, and it represents the file obtained by merging the files represented
by its two children. The number in each nodes is length (no. of records) of
the file represented by that node.
The external node x4 is at a distance of 3 from the root z4 (a node at level i is
at a distance of i – 1 from the root) Hence, the records of files x4 are moved
3 times, once to get z1, once again to get z2, and finally, one more time to
get z4. If di is distance from the root to the external node for file xi and qi, the
length of xi is then the total number of record moves, for this binary merge
tree it is n
1i
ii qd
This sum is called the weighted external path length of the tree.
tree node = record{
treenode*lchild; treenode * rchild;
integer weight;
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 107
In the algorithm below we discuss how to generate a two-way merge tree
Algorithm 4.5
1. Algorithm Tree (n)
2. //list is a global list of n single node
3. //binary trees as described above
4. {
5. for i:=1 to n–1 do
6. {
7. pt:=new treenode; //Get a new tree node
8. (pt /child):=Least(list);//Merge two trees with
9. (pt. /child) :=Least(list) ;//smallest lengths
10. (pt weight):=((pt lchild) weight)
11. +((pt rchild) weight);
12. Insert (list,pt):
13. }
14. return Least (list); //Tree left in list is the merge tree
15. }
An optimal two-way merge pattern corresponds to a binary merge tree with
minimum weighted external path length. The function Tree of Algorithm 4.5
uses the greedy rule stated earlier to obtain a two-way merge tree for n files.
The algorithm has as input a list of n trees.
Each node in a tree has three fields, /child, rchild and weight. Initially, each
tree in list has exactly one node. This node is an external node and has
lchild and rchild fields zero whereas weight is the length of one of n files to
be merged. During the course of the algorithm, for any tree in list with root
node t, t weight is the length of the merged file it represents (t weight
equals the sum of the lengths of the external nodes in tree t). Function Tree
uses two functions, Least (list) and Insert (list, t). Least (list) finds a tree in
list whose root has least weight and returns a pointer to these trees. These
trees are removed from list. Insert (list, t) inserts the tree with root t into list.
The main for loop in Algorithm 4.5 is executed n – 1 times. If list is kept in
non-decreasing order according to the weight value in the roots, then Least
(list) requires only O(1) time and Insert (list, t) can be done in O(n) time.
Hence, the total time taken is O(n2).
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 108
Computing time for Tree is O(n log n). Some speedup may be obtained by
combining the Insert of line 12 with the Least of line 9.
Self Assessment Questions
6. Each node in a tree has three fields ––––, ––––– and ––––
4.7 Single Source Shortlist Paths
We are given a directed graph G = (V, E), a weighting function cost for the
edges in a source vertex v0. The problem is to determine the shortest paths
from v0 to all the remaining vertices of G. It is assumed that all the weights are
positive. The shortest path between v0 and some other node v is an ordering
among a subset of the edges. Hence, this problem fits the ordering paradigm.
To formulate a greedy-based algorithm to generate the shortest paths, we
must conceive of a multistage solution to the problem and also an
optimization measure. One possibility is to build the shortest paths one by
one. As an optimization measure, we can use the sum of the lengths of all
paths so far generated. For this measure to be minimized, each individual
path must be of minimum length. If we have already constructed i shortest
path, then using this optimization measure, the next path to be constructed
should be the next shortest minimum length path. The greedy way to
generate the shortest paths from v0 to the remaining vertices is to generate
these paths in non-decreasing order of path length. First, a shortest path to
the nearest vertex is generated. Then a shortest path to the second nearest
vertex is generated and so on.
Fig. 4.2: Graph & Shortest paths from vertex 1 to all destinations
For the graph of fig 4.2 the nearest vertex to v0 = 1 is 4 (cost [1,4] =10). The
path 1, 4 is the first path generated. The second nearest vertex to node 1 is
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 109
5 and the distance between 1 and 5 is 25. The path 1, 4, 5 is the next path
generated. In order to generate the shortest paths in this order, we need to
be able to determine.
1) The next vertex to which a shortest path must be generated and
2) A shortest path to this vertex
Let S denote the set of vertices (including v0) to which the shortest paths
have already been generated. For w not in S, let dist[w] be the length of the
shortest path starting from v0, going through only those vertices that are in S
and ending at w. We observe that:
1. If the next shortest path is to vertex u, then the path begins at v0, and
goes through only those vertices that are in S. To prove this, we must
show that all the intermediate vertices on the shortest path to u are in S.
Assume there is a vertex w on this path that is not in S. Then, the v0 to u
path also contains a path from v0 to w that is of length less than the v0 to
u path. By assumption, the shortest paths are being generated in non
decreasing order of path length, and so the shorter path v0 to w must
already have been generated. Hence, there can be no intermediate
vertex that is not in S.
2. The destination of the next path generated must be that of vertex u
which has the minimum distance, dist [u], among all vertices not in S.
This follows from the definition of dist and observation 1. In case there
are several vertices not in S with the same dist, then any of these may
be selected.
3. Having selected a vertex u in observation 2 and generated the shortest
v0 to u path, vertex u becomes a member of S. At this point, the length of
the shortest paths starting at v0, going through vertices only in S, and
ending at a vertex w not in S may decrease; that is, the value of dist [w]
may change. If it does change, then it must be due to a shorter path
starting at v0 and going to u and then to w. The intermediate vertices on
the v0 to u path and the u to w path must all be in S. Further, the v0 to u
path must be the shortest such path; otherwise dist [w] is not defined
properly. Also, the u to w path can be chosen so as not to contain any
intermediate vertices. Therefore, we can conclude that if dist [w] is to
change, then it is because of a path from v0 to u to w, where, the path
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 110
from v0 to u is the shortest path and the path from u to w is the edge
(u, w). The length of this path is dist [u] + cost [u, w].
The above observations lead to a simple algorithm 4.6 for the single source
shortest path problem. These algorithms (known as Dijkstra’s algorithm)
only determines the lengths of the shortest path from v0 to all other vertices
in G.
Algorithm 4.6
1. Algorithm shortest paths (v, cost dist, n)
2. // dist [j], 1 j n, is set to the length of the shortest
3. // path from vertex v to vertex in a digraph G with n
4. // vertices dist [v] is set to zero. G is represented by its
5. // cost adjacency matrix cost [1:n, 1:n]
6. {
7. for i: = 1 to n do
8. { // Initialise S
9. S [i] : = false; dist [i] : = cost [v, i];
10. }
11. S[v]: = true; dist[v] : = 0.0; //put v in s
12. for num:=2 to n – 1 do
13. {
14. // Determine n –1 paths from v
15. choose u from among those vertices not
16. in S such that dist [u] is minimum;
17. S[u] : = true; //put u in S
18. for (each w adjacent to u with S[w] = false) do
19. // update distances
20. if (dist [w] > (dist [u] + cost [u, w])) then
21. dist [W]: = dist [u] + cost [u, W];
22. }
23. }
Above algorithm is also known as Greedy algorithm to generate shortest
paths.
In the function shortest paths (Algorithm 4.6) it is assumed that the n
vertices of G are numbered 1 through n. The set S is maintained as a bit
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 111
array with S [i] = 0 if vertex I is not in S and S [i] = 1. It is assumed that the
graph itself is represented by its cost adjacency matrix with cost [i, j]’s being
the weight of the edge <i, j>. The weight cost [i, I] is set to some large
number, , in case the edge <i, j> is not in E (G). For i = j, cost [i, j] can be
set to any non negative number without affecting the outcome of the
algorithm.
The time taken by the algorithm on graph with n vertices is O(n2). To see
this, note that the for loop of line 7 in Algorithm 4.6 takes (n) time. The for
loop of line 12 is executed n – 2 times. Each execution of this loop requires
O(n) time at lines 15 and 16 to select the next vertex and again at the for
loop at line 18 to update dist. So the total time for this loop is O(n2). In case
a list t of vertices currently not in S is maintained, then the number of nodes
on this list would at any time be n-num. This would speed up lines 15 and 16
and the for loop of line 18, but the asymptotic time would remain O(n2).
Self Assessment Questions
7. The shortest path between v0 and some other node v is an –––––––
among a subset of the edges.
8. The time taken by the algorithm on graph with n vertices is –––––––––.
4.8 Summary
In this unit we studied that the greedy method for optimally storage on
tapes algorithm takes (n) computing time.
The greedy method for knapsack problem takes (n2) time to determine
the solution.
The greedy method for optimal merge pattern has O(nlogn) computing
time for tree
The greedy method for single source shortlist path problem has (n2)
asymptotic time.
4.9 Terminal Questions
1. Explain the Greedy Method
2. Explain the concept of Optimal Storage on Tapes
3. Briefly write in your own words the concept of Knapsack Problem
4. Explain the technique involved in Optimal Merge Pattern
Fundamentals of Algorithms Unit 4
Sikkim Manipal University Page No. 112
5. Do you agree that the single source shortlist paths problem fits the
ordering paradigm ? Justify your answer
6. Write an algorithm for single source shortest path problem
7. Write the Greedy-Knapsack algorithm
4.10 Answers
Self Assessment Questions
1. Feasible
2. Optimization measure
3. Minimizes
4. Positive
5. p1/w1 p2/w2 ….. pn/wn
6. lchild, rchild and weight.
7. ordering
8. O(n2)
Terminal Questions
1. The greedy method suggests that one can device an algorithm that
works in stages, considering one input at a time. At each stage, a
decision is made regarding whether a particular input is in an optimal
solution. This is done by considering the inputs in an order determined
by some selection procedure. (Refer Section 4.2)
2. We assume that whenever a program is to be retrieved from this tape,
the tape is initially positioned at the front. Hence, if the programs are
stored in the order I=i1, i2 -----in, the time tj needed to retrieve program ij is
proportional to .Ijk1
ik If all programs are retrieved equally often, then
the expected or mean retrieval time (MRT) is nj1
jtn
1. (Refer
Section 4.3)
3. Refer Section 4.4
4. Refer Section 4.6
5. Refer Section 4.7
6. Refer Section 4.7
7. Refer Section 4.4
Fundamentals of Algorithms Unit 5
Sikkim Manipal University Page No. 113
Unit 5 Dynamic Programming Structure
5.1 Introduction
Objectives
5.2 Dynamic Programming Strategy
5.3 Multistage Graphs
5.4 All Pair Shortest Paths
5.5 Traveling Salesman Problems
5.6 Summary
5.7 Terminal Questions
5.8 Answers
5.1 Introduction
In the previous unit, we discussed the concept of greedy method but when
the optimal solutions to subproblems are to be retained in order to avoid
recomputing their values, we use dynamic programming. Dynamic
programming is an algorithm design method that can be used when the
solution to problem can be viewed as the result of sequence of decisions.
The principle of optimality states that an optimal sequence of decisions has
the property that whatever the initial state and decision are, the remaining
decisions must constitute an optimal decision sequence with regard to the
state resulting from the first decision.
The essential difference between the greedy method and dynamic
programming is that in the greedy method only one decision sequence is
ever generated. In dynamic programming, many decision sequences may
be generated. However, sequences containing suboptimal subsequences
cannot be optimal and so not be generated.
Objectives
After studying this unit, you should be able to:
apply Dynamic Programming in Multistage graphs
solve all pair shortlist paths using Dynamic Programming
solve Traveling Salesman Problems using Dynamic Programming.
Fundamentals of Algorithms Unit 5
Sikkim Manipal University Page No. 114
5.2 Dynamic Programming Strategy
Because of the use of the principle of optimality, decision sequences
containing subsequences that are suboptimal are not considered. Although
the total number of different decision sequences is exponential to the
number of decisions (if there are d choices for each of the n decisions to be
made then there are dn possible decision sequences). Dynamic
programming algorithms often have a polynomial complexity.
Another important feature of the dynamic programming approach is that
optimal solutions to subproblems are retained so as to avoid recomputing
their values. The use of these tabulated values makes it natural to recast the
recursive equations into an iterative algorithm.
Self Assessment Question
1. Dynamic programming algorithms often have a ––––––––––––
5.3 Multistage Graphs
A multistage graph G = (V,E) is a directed graph in which the vertices are
partitioned into K 2 disjoint sets Vi, 1 i k. In addition, if <u, v> is an
edge in E, then u vi, and v vi+1 for some i, 1 i < k. The sets V1 and Vk
are such that iVV k1 Let s and t, respectively, be the vertices in V1
and Vk. The vertex s is the source, and t the link. Let c(i, j) be the cost of
edge <i, j>. The cost of a path from s to t is the sum of the costs of the
edges on the path. The multistage graph problem is to find a minimum-cost
path from s to t. Each set Vi defines a stage in the graph. Because of the
constraints, on E, every path from s to t starts in stage 1, goes to, stage 2,
then to stage 3, then to stage 4, and so on, and eventually terminates in
stage K.
Fundamentals of Algorithms Unit 5
Sikkim Manipal University Page No. 115
Fig. 5.1: A minimum-cost s to t path is indicated by the broken edges.
Many problems can be formulated as multistage graph problems. We give
only one example. Consider a resource allocation problem in which n units
of resource are to be allocated to r projects. If j 0 j n units of the resource
are allocated to project i, then the resulting net profit is N(i,j). The problem is
to allocate the resource to the r projects in such a way as to maximize total
net profit. This problem can be formulated as an r+1 stage graph problem as
follows:
Stage i, 1 i r, represents project i. There are n+1 vertices V(i, j), 0 j n,
associated with stage i, 2 i r. Stages 1 and r+1 each have one vertex,
V(1,0) = s and V(r+1,n)=t, respectively. Vertex V(i,j), 2 i r, represents the
state in which a total of j units of resource have been allocated to projects
1,2, ….. ,i –1. The edges in G are of the form <V(i,j),V(i+1, l)> for all j l and
1 i < r. The edge <V(i,j, V(i+1, l)>, j<l, is assigned a weight or cost of N(i, l-j)
and corresponds to allocating l-j units of resource to project i, 1 i < r. In
addition, G has edges of the type <v(r,j), v(r+1,n)>. Each such edge is
assigned a weight of )}p,r(N{maxjnp0
. It should be easy to see that an
optimal allocation of resources is defined by a maximum cost s to t path.
This is easily converted into a minimum-cost problem by changing the sign
of all the edge costs.
A dynamic programming formulation for a k-stage graph problem is obtained
by first noticing that every s to t path is the result of a sequence of k – 2
Fundamentals of Algorithms Unit 5
Sikkim Manipal University Page No. 116
decisions. The ith decision involves determining which vertex in Vi+1, 1 i k
– 2, is to be on the path. It is easy to see that the principle of optimality
holds. Let p(i,j) be a minimum-cost path from vertex j in Vi of vertex t. Let
cost(i,j) be the cost of this path. Then, using the forward approach, we
obtain
)}l,1i(tcos)l,j(c{min)j,i(tcos
El,j1iVl
…………(1)
Before writing an algorithm for a general k-stage graph, let us impose an
ordering on the vertices in V. This ordering makes it easier to write the
algorithm. We require that the n vertices in V are indexed 1 through n.
Indices are assigned in order of stages. First, s is assigned index 1, then
vertices in V2 are assigned indices, then vertices from V3, and so on. Vertex
t has index n. Hence, indices assigned to vertices in Vi. As a result of this
indexing scheme, cost and d can be computed in the order n-1, n-2..., 1.
The first subscript in cost, p and d only identifies the stage number and is
omitted in the algorithm. The resulting algorithm in pseudocode is F Graph,
given in Algorithm 5.1
Algorithm 5.1
1. Algorithm FGraph (G,k,n,p)
2. //The input is a K-stage graph G=(V,E) with n vertices
3. //indexed in order of stages. E is a set of edges and c[i,j]
4. //is the cost of <i, j> p[1:k] is a minimum-cost path.
5. {
6. cost[n] :=0.0;
7. for j:=n-1 to 1 step – 1do
8. {//compute cost[j]
9. Let r be a vertex such that <j,r> is an edge
10. of G and c[j,r]+cost[r] is minimum:
11. cost[j] :=c[j, r]+cost[r]:
12. d[j]:r;
13. }
14. //Find a minimum-cost path
15. p[i]:=1; p[k]:=n;
16. for j:=2 to k-1 do p[j]:=d [p-1]];
17. }
Fundamentals of Algorithms Unit 5
Sikkim Manipal University Page No. 117
In the above algorithm, we study multistage graph pseudocode
corresponding for the forward approach.
The complexity analysis of the function F Graph is fairly straightforward. If G
is represented by its adjacency lists, then r in line 9 of Algorithm 5.1 can be
found in time proportional to the degree of vertex j, Hence, if G has IEl
edges, then time for the for loop of line 7 is (lVl+lEI). The time for the loop
of line 16 is (K). Hence, the total time is (IVI+lEl). In addition to the space
needed for input, space is needed for cost[ ], d[ ], and p[ ].
The multistage graph problem can also be solved using the backward
approach. Let bp(i,j) be a minimum-cost path from vertex s to a vertex j in Vi.
Let bcost (I,j) be the cost bp(i,j). From the backward approach we obtain.
)}j,l(c)l,1i(tcosb{min)j,i(tcosb
Ej,l1iVl
………….5.2
Algorithm 5.2
1. Algorithm Bgraph (G, k, n, p)
2. // same function as Fgraph
3. {
4. bcost[1]:= 0.0;
5. for j:=2 to n do
6. {// compute bcost [j]
7. let r be such that <r,j> is an edge of
8. G and bcost [r] + c [r,j] is minimum;
9. bcost [j]: = bcost [r] + c[r,j];
10. d[j]:=r;
11. }
12. // Find a minimum-cost path
13. p[1]:=1;p[k]:=n;
14. for j:=k-1 to 2 do p[j]:=d[p [j+1]];
15. }
In the above algorithm we study, the multistage graph pseudocode
complexity to backwards approach
Fundamentals of Algorithms Unit 5
Sikkim Manipal University Page No. 118
This algorithm has the same complexity as F graph provided G is now
represented by its inverse adjacency lists (i.e., for each vertex v, we have a
list of vertices w such that <w,v> E).
It should be easy to see that both F graph and B graph work correctly even
on a more generalized version of multistage graphs. In this generalization,
the graph is permitted to have edges <u.v> such that u Vi, V V1 and i<j.
Self Assessment Question
2. A multistage graph G = (V,E) is a directed graph in which the vertices
are ––––––––––––into K 2 disjoint sets Vi, 1 i k.
5.4 All Pair Shortest Paths
Let G = (V, E) be a directed graph with n vertices and let cost be a cost
adjacency matrix for G such that cost (i, i) = 0, 1 i n. Then cost (I,j) is the
length (or cost) of edge <i,j> if <i,j> E(G) and cost (i,j) = if i j and <i,j>
E(G). The all pairs shortest path problem is to determine a matrix A such
that A (i,j) is the length of a shortest path from i to j.
Let us examine a shortest i to j path in Gi j. This path originates at vertex i
and goes through some intermediate vertices (possibly none) and
terminates at vertex j. We can assume that this path contains no cycles for if
there is a cycle, then this can be deleted without increasing the path length.
If k is an intermediate vertex on this shortest path, then the subpaths from i
to k and from k to j must be shortest paths from i to k and k to j respectively.
Otherwise, the i to j path is not of minimum length. So the principle of
optimality holds. This alerts us to the prospect of using dynamic
programming. If k is the intermediate vertex with highest index, then the i to
k path is a shortest i to k path in G going through no vertex with index
greater than k–1. Similarly, the k to j path is a shortest k to j path in G going
through no vertex of index greater than k–1. We can regard the construction
of a shortest i to j path as first requiring a decision as to which is the highest
indexed intermediate vertex k. once this decision has been made, we need
to find two shortest paths, one from i to k and the other from k to j. Neither of
these may go through a vertex with index greater than k–1.Using Ak(i,j) to
represent the length of a shortest path from i to j going through no vertex of
index greater than k, we obtain
Fundamentals of Algorithms Unit 5
Sikkim Manipal University Page No. 119
)j,i(tcos),j,k(A)k,i(A{minmin)j,i(A 1k1k
nk1……….5.3
Clearly, A°(i,j) = cost (i, j), 1 i n, 1 j n. We can obtain a recurrence for
Ak(j,j) using argument similar to that used before. A shortest path from i to j
going through no vertex higher than k either goes through vertex k or it does
not. If it does, Ak (i,j) = Ak–1 (i,k) + Ak–1 (k, j). If it does not, then no
intermediate vertex has index greater than k-1. Hence,
Ak(i,j)=Ak–1 (i, j). Combining, we get
A(i,j)=min{A k-1(i, j), Ak-1 (i,k)+Ak-1(k,,j)},k>1 --------------- (5.4)
Recurrence (5,4) can be solved for An by first computing A1, then A2, then A3
and so on. Since there is no vertex in G with index greater than n,
A (i,j) = A(i,j)
Algorithm Allpaths (cost, A, n)
Algorithm 5.3
1. //cost[1:n, 1:n] is the cost adjacency matrix of a graph
2. //with n vertices; A[I, j] is the cost of a shortest path from
3. //vertex i to vertex j. cost [I, j] = 0.0, for 1 i n.
4. {
5. for i:=1 to n do
6. for j:=1 to n do
7. A[i, j]:=cost [i, j]; //copy cost into A.
8. for k:=1 to n do
9. for i:=1 to n do
10. for j:=1 to n do
11. A[i, j]:=min(A[i, j], A[i, k]+A[k, j];)
12. }
Above algorithm functions to compute the length of shortest paths
Function All paths computer An (i.j). The computation is done in place so the
superscript on A is not needed. The reason this computation can be carried
out in-place is that Ak (i,k) = Ak-1 (i, j) and Ak (k, j) = Ak-1 (k, j). Hence, when
Ak is formed, the kth column and row do not change. Consequently, when
Ak(i,j) is computed in line II of Algorithm 5.3, A (i,k) = Ak-l (i,k) = Ak (i, k) and
A(k,j) = Ak-l (kJ) = Ak (k,j). So, the old values on which the new values are
based do not change on this iteration.
Fundamentals of Algorithms Unit 5
Sikkim Manipal University Page No. 120
Let M = max {cost (i,j)l<i,j> E (G)}. It is easy to see that An (i,j) (n–1)M.
From the working of all paths, it is clear that if <i,j> E (G) and i j, then we
can initialize cost (i,,j) > (n-I) M,. If, at termination, A(i,j)>(n-1), then there is
no directed path from i to j in G.
The time needed by All paths (Algorithm 5.3) is especially easy to determine
because the looping is independent of the data in the matrix A. Line 11 is
iterated n3 times, and so the time for all paths is (n3).
Self Assessment Question
3. The time needed by All paths is easy to determine, because the looping
is –––––––––––– of the data in the matrix .
5.5 Travelling Salesman Problem
A tour of G is a directed simple cycle that includes every vertex in V. The
cost of the tour is the sum of the cost of the edges on the tour. The travelling
salesperson problem is to find a tour of minimum cost.
Let G=(V,E) be a directed graph with edges cost cij. The variable cij is
defined such that cij = 0, cij> o for all i and j and cij = if < i,j> E. Let lVl = n
and assume n>1. A tour is a simple path that starts and ends at vertex 1.
Every tour consists of an edge < 1, k> for some k V — {1} and a path from
vertex k to vertex 1. The path from vertex k to vertex 1 goes through each
vertex in V-{1,k} exactly once. It easy to see that if the tour is optimal, then
the path from k to 1 must be a shortest k to 1 path going through all vertices
in V– {1,k). Hence, the principle of optimality holds. Let g (i,s) be the length
of a shortest path starting at vertex i, going through all vertices in S, and
terminating at vertex 1. The function g(1, V-{1}) is the length of an optimal
salesperson tour. From the principal of optimality if follows that
})}k,j{V,k(gc{min})1{V,1(g k1nk2
……….5.5
Generalizing (5.5), we obtain (for i s)
})}j{S,j(gc{min)S,i(g ijSj
…………..5.6
Equation 5.5 can be solved for g (1, V- {1}) if we know g (k, V-{1,k}) for all
choices of k. The g values can be obtained by using (5.6). Clearly, g (i, ) =
Ci1, i 1 n.
Fundamentals of Algorithms Unit 5
Sikkim Manipal University Page No. 121
Hence, we can use (5.6) to obtain g(i,S) for all S of size 1. Then, we can
obtain g (i, s) for S with |S|=2, and so on. When |S|< n-1, the values of i and
S for which (i, S) is needed are such that
i 1, 1 S, and i S.
Let N be the number of g (i, S)’s that have to be computed before (5.6) can
be used to compute g (i,v-{i}). For each value of |S| there are n-1 choices for
i. The number of distinct sets S of size k not including l and i is (kn-2). Hence,
2n
0k
2n2n 21nk1nN
An algorithm that proceeds to find an optimal tour by using (5.5) and (5.6)
will require (n22n) time by using computation of g(i,s) with |S|=k ,require k-1
comparisons when solving (5.6). This is better than enumerating all n!
different tours to find best one. The most serious drawback of this dynamic
programming solution is, O(n2n). This is too large even for modest values
of n.
Self Assessment Question
4. A ––––––– of G is a directed simple cycle that includes every vertex in
V.
5.6 Summary
In this unit, we studied that Dynamic programming is an algorithm
design method that can be used when the solution to a problem can be
viewed as the result of a sequence decisions.
Optimal sequence of decisions has the property that whatever the initial
state and decision are, the remaining decisions must constitute an
optimal decision sequence with regard to the state resulting from the first
decision, this is called as the principle of optimality.
We also studied the concept of travelling salesmen problems using the
concept of graphs.
5.7 Terminal Questions
1. Explain the concept of Multistage Graphs
2. Explain the Concept of travelling Salesman Problem
3. Write an Algorithm to obtain a minimum cost s-t path
Fundamentals of Algorithms Unit 5
Sikkim Manipal University Page No. 122
5.8 Answers
Self Assessment Questions
1. Polynomial complexity
2. partitioned
3. independent
4. tour
Terminal Questions
1. A multistage graph G = (V,E) is a directed graph in which the vertices
are partitioned into K 2 disjoint sets Vi, 1 i k. In addition, if <u, v> is
an edge in E, then u vi, and v vi+1 for some i, 1 i < k. The sets V1
and Vk are such that iVV k1 Let s and t, respectively, be the
vertices in V1 and Vk. The vertex s is the source, and t the link. Let c(i, j)
be the cost of edge <i, j>. The cost of a path from s to t is the sum of the
costs of the edges on the path. The multistage graph problem is to find a
minimum-cost path from s to t. Each set Vi defines a stage in the graph.
(Refer Section 5.3)
2. Refer Section 5.5
3. Refer Section 5.3
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 123
Unit 6 Backtracking Structure
6.1 Introduction
Objectives
6.2 Backtracking Strategy
6.3 8-Queens Problem
6.4 Sum of Subsets
6.5 Knapsack Problem
6.6 Summary
6.7 Terminal Questions
6.8 Answers
6.1 Introduction
In the previous unit, we discussed the concept of dynamic programming and
now in the search for fundamental principles of algorithm design, we will
discuss about backtracking, which represents one of the most general
techniques. Many problems which deal with searching for a set of solutions
or which ask for an operational solution satisfying some constraints can be
solved using the backtracking formulation. The name backtrack was first
coined by D.H Lehmer in the 1950s. Early workers who studied the process
were R.J. Woler, who gave an algorithmic account of it in 1960, and S.
Golomb and L.Barmert who presented a very general description of it in the
form of applications.
Objectives
After studying this unit, you should be able to:
analyze the 8-queen problem
solve the problem of the Sum of subsets
use the applications of the Knapsack problem.
6.2 Backtracking Strategy
In many applications of the backtrack method, the desired solution is
expressible as an ntuple (x1,……..,xn) where the xis are chosen from some
finite set si often the problem to be solved calls for finding one vector that
maximizes a criterion function p(x1,....xn). Sometimes it seeks all vectors that
satisfy P. For example, sorting the array of integers in a [1:n] is a problem
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 124
whose solution is expressible by an n-tuple, were xi is the index in a of ith
smallest element. The criterion function P is the inequality a[xi] a[xi+1] for
1 i < n. The set Si is finite and includes the integer 1-through n. Through
sorting is not usually one of the problems solved by backtracking, it is one
example of a familiar problem whose solution can be formulated as an n-
tuple. In this unit we study a collection of problems whose solution are best
done using backtracking.
Suppose mi is the size of set Si. Then there are m=m1 m2 ……. mn n-tuples
that are possible candidates for satisfying the function P. The brute force
approach would be to form all these n-tuples, evaluate each one with P and
save those which yield the optimum. The backtrack algorithm has its virtue,
the ability to yield the same answer, for fewer in trials. Its basic idea is to
build up the solution vector on component at a time and to use modified
criterion functions Pi{x1, ….. xi) (some times called bound functions) to test
whether the vector being formed has any chance of success. The major
advantage of this method is that if it is realized that the partial vector
(x1, x2, …. xi) can in no way lead to an optimal solution, then mi+1…..mn
possible test vectors can be ignored entirely. Many of the problems we solve
using backtracking requires that all the solution satisfy a complex set of
constraints. For any problem, these constraints can be divided into two
categories; explicit and implicit.
Definition
Explicit constraints: Are rules that restrict each xi, to take a value only
from a given set,
common examples of explicit constraints
are xi > 0 or Si = {all non negative real no}
xi = 0 or 1 or Si = {0, 1}
di xi < ui or Si = {a: li a ui}
The explicit constraints depend on the particular instance I of the problem
being solved. All tuples that satisfy the explicit constraints define a possible
solution space for I.
Definition
The implicit constraints are tuples that determine which of the tuples in the
solution space of I satisfy the criterion function. Thus implicit constraints
describe the way in which the xi must relate to each other.
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 125
Self Assessment Question
1. –––––––––––– are rules that restrict each xi, to take a value only from a
given set.
6.3 8-Queens Problem
A classic combinational problem is to place eight queens on 8x8 chessboard
so that no two “attack” that is, so that no two of them are on the same row,
colours or diagonal. Let us number the rows and columns of the chessboard
1 through 8 (Fig 6.1). The queens can also be numbered 1 through 8. Since
each queen must be on a different row, we can without loss of generality
assume that queen i is to be placed on row i. All solutions to the 8-queens
problem can therefore be represented as 8-tuples (x1…… x8), where xi is the
column on which queen i is placed. The explicit constraints using this
formulation are Si = {1, 2, 3, 4, 5, 6, 7, 8}; 1 i 8. Therefore, the solution
space consists of 88-tuples. The implicit constraints for this problem are that
no two queens can be on the same diagonal. The first of these two
constraints implies that all solutions are permutations of the 8-tuple
(1, 2, 3, 4, 5, 6, 7, 8). This realization reduces the size of the solution space
from 88 tuples to 8! Tuples. We see later how to formulate the second
constraint in terms of the xi. Expressed as an 8-tuple, the solution in Fig. 6.1
is (4, 6, 8, 2, 7, 1, 3, 5).
1 2 3 4 5 6 7 8
1 Q
2 Q
3 Q
4 Q
5 Q
6 Q
7 Q
8 Q
Fig. 6.1: 8-Queens Problem
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 126
Algorithm 6.1
Algorithm Place (k, i)
// Returns true if a queen can be placed
// in kth row and ith columns otherwise
// it returns false, x[ ] is a global array
// whose first (k – 1) values have been set
// Abs (r) returns the absolute value of r
{
For (i:=1) to k – l) do
if ((x [j] – l // Two in the same column
or (Abs (x [j] = Abs (j – k)))
// or in the same diagonal
Then return false.
Return true:
}
Algorithm NQueens (k, n)
// Using backtracking this procedure prints all
// possible placements of n queens on an
// n x n chessboard so that they are
// non attacking
{
for I : = 1 to n do
{
if Place (k,j) then
{
x [ k] : = i;
if (k=n) then write (x[1:n])
else Nqueen (k+1, n);
}
}
}
At this point, we can see how effective function Nqueen is over the brute
force approach. For an 8x8 chessboard, there are 8
64 possible ways to
place 8 pieces or approximately 4.4 billion 8 tuples to examine. However, by
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 127
allowing only placements of queens on distinct rows and columns, we
require the examination of at most 8! or only 40,3208-tuples.
We can estimate the number of nodes that will be generated by NQueens.
The bounding function is static and no change is made to the function as the
search proceeds. In addition, all nodes on the same level, of the separate
space level, state space tree have the same degree.
As required placement of each queen on the chessboard was chosen
randomly with each choice we kept track of the number of columns a queen
could legitimately be placed on. These numbers are listed beneath each
chessboard. The number following the vector represents the value that
function estimate would produce from these sizes. The coverage of these
five trials is 1625.
So that estimated no. of unbounded nodes is only 2.34% of the total no of
nodes in the 8 queen state space tree.
Self Assessment Questions
2. The total no of nodes in the 8-queen state space is –––––––––––.
3. For an 8x8 chessboard there are ––––––––––possible ways to place 8
pieces.
6.4 Sum of Subsets
Given positive numbers wi, 1 i n, and m, this problem calls for finding all
subsets of the wi whose sums are m. For example, if n = 4, (w1, w2, w3, w4) =
(11, 13, 24, 7), and m = 31 then the desired subsets are (11, 13, 7) and
(24, 7) rather than represent the solution vector by the wi which sum to m,
we could represent the solution as described by the vector (1,2,4) and (3, 4).
In general, all solutions are k-tuples (x1, x2…….. xk), 1 k n, and different
solutions may have different sized tuples. The explicit constraints xj {jlj is
an integer and 1 j n}. The implicit requires that no two be the same and
that the sum of the corresponding will be m. Since we wish to avoid
generating multiple instance of the same subject (e.g (1, 2, 4) and (1, 4, 2)
represent the same subject), another implicit constraints that is imposed is
xi<xi+1, I i < k.
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 128
In another formulation of the sum of subsets problem, each solution subject
is represented by n-tuple (x1, x2, ….. xn) such that xi {0,1}, 1 i n. Then,
xi = 0, if wi is not chosen then the above instance are (1, 1, 0, 1) and
(0, 0, 1, 1). The formulation express all solutions using fixed sized tuple.
Thus, we conclude that there may be several ways to formulate a problem
so that all solutions are tuples that satisfy some constraints. One can verify
that for both of the above formulations, the solutions space consists of 2n
distinct tuples.
Backtracking algorithms determine problem solutions by systematically
searching the solution space for the given problem instance. This search is
facilitated by using a tree organization for the solution space. For a given
solution space, many tree organizations may be possible. The next two
examples examine some of the ways to organize a solution into a tree.
We have two possible formulations of the solutions space for the sum of
subsets problem. Fig 6.2 and Fig 6.3 show a possible tree organization for
each of these formulations for the case n=4. The tree of Fig 6.2 corresponds
to the variable tuple size formulation. The edges are labeled such that an
edge from a level i node to a level i + 1 node represents a value for xi. At
each node the solution space is partitioned into solution space. The solution
space is defined by all paths from the root node to any node in the tree,
since any such path corresponds to a subset satisfying the explicit
constraints. The possible paths are ( ) (this corresponds to the empty path
from the root to itself), (1), (1,2), (1,2,3), (1,2,3,4), (1,2,4), (1,3,4), (2), (2,3)
and so on. Thus, left most subtree defines all subsets containing wi, the next
subtree defines all subsets containing w2 but not w1, and so on.
The tree of Fig 6.3 corresponds to the fixed tuples size formulation. Edges
from level i nodes to level i + I nodes are labeled with the value of xi, which
is either zero or one. All paths from the root to a leaf node define the
solution space. The left subtree of the root defines all subsets containing wl,
the right subtree defines all subsets not containing wl and so on. Now there
are 24 leaf nodes which represent 16 possible tuples.
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 129
Fig. 6.2: A possible space organization for the sum of subsets problem.
Nodes are numbered as in breath first search.
At this point, it is useful to develop some terminology regarding tree
organization of solution spaces. Each node in this tree defines a problem
state. All paths from the root to other nodes define the state space of the
problem. Solution states are those problem states s for which the path from
the root to s defines a tuple in the solution space. In the tree Fig 6.2, all
nodes are solutions states whereas in the tree Fig 6.3, only leaf nodes are
solution states. Answer states are those solution states s for which the path
from the root to s defines a tuple that is a member of the set of solutions (i.e.
it satisfies the implicit constraints) of the problem.
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 130
The tree organization of the solution space is referred to as the state space
tree.
Fig. 6.3: Corresponds to the fixed tuples size formulation.
Suppose we are given distinct positive number and we desire to find all
combinations of these numbers whose sums are m. This is called the sum
of subsets problem. The example in above shows how we could formulate
this problem using either fixed or variable sized tuples. We consider a back
tracking solution using the fixed tuple size strategy. In this case, the element
xi of the solution vector is either one or zero depending on whether the
weight wi, is included or not.
The children of any node in Fig 6.3 are easily generated. For a node at level
i, the left child corresponds to xi = 1 and the right to xi = 0.
A simple choice for the bounding functions Bk (x1, ……. xi) = true if n
ki
i
k
i
ii mwxw11
Clearly, x1, ……..,xk cannot lead to an answer node if this condition is not
satisfied. The bounding functions can be strengthened if we assure that the
wi’s are initially in non-decreasing order. In this case, x1…… xk cannot lead
to an answer node if
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 131
mwxw k
k
i
ii 1
1
The bounding functions we use are therefore Bk (x1, …………….,xk) = true iff
mWxwandmwxw k
k
i
ii
n
ki
i
k
i
ii 1
111
Since our algorithms will not make use of Bn, we need not be concerned by
the appearance of Wn+1 in this function. Although we have now specified all
that is needed is to directly use either of the backtracking scheme, a simpler
algorithms results if we tailor either of these schemes to the problem at
hand. This simplification results from the realization that if xk = 1, then
n
1ki
i
k
1i
ii mwxw
For simplicity, we redefine the recursive scheme. The resulting algorithms is
SumOfSub (Algorithm 6.2)
Algorithm 6.2
Algorithm SumOfSub (s,k,r)
// Find all subsets of w(k....n) that sum to m
// The values of x[j], 1 j < k, have already
// been determined 1k
1j
]j[X]j[wS
//and n
kj
]j[wr . The w[j]’s are in
// non decreasing order It is assumed that
// w[l] m and n
1i
m]i[w
{
// Generate left child. Note: S+tw[j] m
// since Bk–1 is true.
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 132
x [k] : = 1;
If (s + w[k] = m) then write (x [1: k]);
// sub set found there is no recursive call
// here as w[j] > 0, 1 j n
else if (s + w[k] + w [k + 1] m)
then SumOfSub (s + w[k], k + 1, r – w[k]);
// Generate right child and evaluate Bk
if(s + r – w [k] m) and (s + w [k + 1] m ) then
{
x[k]:=0
SumOfSub (s, k + 1, r — w [k]):
}
}
Algorithm SumOfSub avoids computing n
1ki
i
k
1i
ii wandkw each time by
keeping those values in variables s and r respectively. The algorithm
assumes n
1i
ii .mwandmw The initial call is Sum Of Sub n
1i
iw,1,0 .
It is interesting to note that the algorithm does not explicitly use the test k> n
to terminate the recursion. This test is not needed as an entry to the
algorithm s m and s+r m. Hence, r 0 and so k can be no greater than n.
Also note that in the else if statement, since s + wk < m and s + r m, it
follows that r wk and hence k + 1 n, observe also that if s + wk = m, then
xk+1,……………….xn must be zero. These zeros are omitted from the output.
In next statement we do not test for n
1ki
i
k
1i
ii mwxw as we already
know s + r m and xk + 1.
Self Assessment Questions
4. A simple choice for the bounding functions Bk (x1, ……. xi) is true
if ––––––––––
5. Given positive numbers wi, 1 i n, and m, the problem calls for finding
all –––––––––––of the wi whose sums are m
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 133
6.5 Knapsack Problem
Given n positive weights wi, n positive profits Pi, and a positive number m
that is the knapsack capacity, this problem calls for choosing a subset of the
weights such that
ni1
ii
ni1
ii xPandmxw is minimized.
The xi’s constitute a zero-one-valued vector.
The solution space for this problem consists of the 2n distinct ways to assign
zero or one values to the xi’s. Thus, the solution source is the same as that
for the sum of subset problem. Two possible tree organizations are possible,
One corresponds to the fixed tuple size formulation and the other to the
variable tuple size formulation. Backtracking algorithms for the knapsack
problem can be arrived at using either of these two state space trees.
Regardless of which is used, bounding functions are needed to help kill
some live nodes without expanding them. A good bounding function for this
problem is obtained by using an upper bound on the value of the best
feasible solution.
Obtainable by expanding the given live node and any of its descendants, if
this upper bound is not higher than the value of the best solution determined
so far, then that live node can be killed.
We continued the discussion using the fixed tuple size formulation, If at
node z, the values of xi, 1 i k have already been determined, then an
upper bound for z can be obtained by relaxing the requirement xi = 0 on 1 to
0 xi 1 for k + 1 i n and using the greedy algorithm to solve the
relaxed problem. Function Bound (cp, cw, k) determines an upper bound on
the best solution obtained by expanding any zone z at level of the state
space tree. The object weights and profits are w[i] and p[i]. It is assumed
that ni1;)1i(w
)1i(p
]i[w
)i(p
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 134
Algorithm 6.3
Algorithm Bound (cp, cw, k)
// cp is the current profit, cw is the current
// weight total k is the index of the last removed
// item : and m is the knapsack size
{
b:=cp; c:= cw;
for i = k +1 to n do
{
c: = c + w [i]
if (c < m) then b:= b + p [i]
else return b + (cI – (c-m)) / w[i] * p [i];
} return b:
{
From bound it follows that the bound for a feasible left child of a node z is
the same as that for z. Hence, the bounding function need not be used
whenever the backtracking algorithm makes a move to the left child of a
node. The resulting algorithm is Bknap. It was obtained from the recursive
backtracking scheme, Initially set fp: = -1: This algorithm is invoked as
Bknap (1, 0, 0):
When fp —1, x[i], 1 i n is such that n
1i
p .fixip The path y [i], 1 i
k, is the path to the current node. The current weight
1k
1i
p
1k
1i
w iyipcandiyiwc
Algorithm 6.4
Algorithm Bknap (k, cp, cw)
// m is the size of the knapsack; is the
// no of weights and profits w[ ] and p [ ]
// are the weights and profits
// p [i] /w [i] p[i +1] fw wf]1i[piw
ipis the final
// weight of knapsack: fp is the final
// maximum profit x[k] = 0 if w[k] is not
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 135
// the knapsack else x [k] = 1
{
// Generate left child
If (cw + w [k] m ) then
{
y [k] = 1:
if (k<n) then Bknap (k+1, cp+p [k], cw + w [k]
if (cp + p[k] > fp) and (k= n) then
{
fp: = cp + p[k]; fw: = cw + w[k]
for j: = 1 to k do x [j]: = y[j]:
}
}
// Generate right child
if (Bound (cp, cw, k) fp) then
{
y[k]: = 0; if (k < n) then Bknop (k + 1, cp, cw):
If ((cp > fp) and (k = n) then
{
fp : = cp : fw = cw
for j : = 1 to k do
x[j] : = y[j]:
}
}
}
Self Assessment Question
6. If the object weights and profits are w[i] and p[i], It is assumed
that ––––––
6.6 Summary
In this unit, we studied that any problem can be divided into two
categories explicit and implicit.
We also studied in this unit a classic combinational problem called as
8-queens problem.
The concept of sum of subsets was discussed with suitable examples.
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 136
The Knapsack problem and its algorithm was also discussed with
suitable explanation
6.7 Terminal Questions
1. Write briefly the concept of 8-Queens Problem
2. Write the Algorithm of sum of Subsets
3. Briefly explain the Knapsack Problem
6.8 Answers
Self Assessment Questions
1. Explicit constraints
2. 281,69)8(17
0 0j
j
i
i
3. 8
64
4. n
ki
i
k
i
ii mwxw11
5. subsets
6. ni1;)1i(w
)1i(p
]i[w
)i(p
Terminal Questions
1. A classic combinational problem is to place eight queens on 8x8
chessboard so that no two “attack” that is, not two of them are on the
same row, colours or diagonal. Let us number the rows and columns of
the chessboard 1 through 8 (Fig 6.1). The queens can also be
numbered 1 through 8. Since each queen must be on a different row, we
can without loss of generality assume that queen i is to be placed on row
i. All solutions to the 8-queens problem can therefore be represented as
8-tuples (x1…… x8), where xi is the column on which queen i is placed.
The explicit constraints using this formulation are Si = {1, 2, 3, 4, 5, 6, 7,
8}; 1 i 8. Therefore the solution space consists of 88-tuples. The implicit
constraints for this problem are that no two queens can be on the same
diagonal. The first of these two constraints implies that all solutions are
Fundamentals of Algorithms Unit 6
Sikkim Manipal University Page No. 137
permutations of the 8-tuple (1, 2, 3, 4, 5, 6, 7, 8). This realization
reduces the size of the solution space from 88 tuples to 8! Tuples. (Refer
Section 6.3)
2. Refer Section 6.4
3. Refer Section 6.5
Fundamentals of Algorithms Unit 7
Sikkim Manipal University Page No. 138
Unit 7 Branch and Bound Structure
7.1 Introduction
Objectives
7.2 Branch and Bound Strategy
7.3 O/I Knapsack Problem
7.4 Travelling Salesman Problem
7.5 Summary
7.6 Terminal Questions
7.7 Answers
7.1 Introduction
In the previous units we discussed about the concept of backtracking and
now we will study a new concept known as branch and bound. The term
branch-and-bound refers to all state space search methods in which all
children of the E-node are generated before any other live node can
become the E-node.
As in the case of backtracking, bounding functions are used to help avoid
the generation of subtrees that do not contain an answer node.
Objectives
After studying this unit, you should be able to:
solve the O/I Knapsack Problem
find the solution using Traveling Salesman Problem.
7.2 Branch and Bound Strategy
The term branch-and-bound refers to all state space search methods in
which all children of the E-node are generated before any other live node
can become the E-node.
In branch-and-bound terminology, a BFS-like state space search will be
called FIFO (First In First Out) search as the list of live nodes is first-in-first-
out list (or queue). A D-search-like state space search will be called LIFO
(Last In First Out) search as the list of live nodes is a last-in-first-out list
(or stack). As in the case of backtracking, bounding functions are used to
help avoid the generation of subtrees that do not contain an answer node.
Fundamentals of Algorithms Unit 7
Sikkim Manipal University Page No. 139
Self Assessment Question
1. The term branch-and-bound refers to all state space search methods in
which all children of the E-node are generated before any other live
node can become the––––––––––––.
7.3 O/l Knapsack Problem
To use the branch-and-bound technique to solve any problem, it is first
necessary to conceive of a state space tree for the problem.
The objective function ii xp is replaced by the function .xp ii Clearly,
ii xp is maximized iff – ii xp is minimized. This modified knapsack
problem is stated as given below
n
1i
ii xpMinimize
n
1i
ii mxwtoSubject
xi = 0 or 1, 1 i n ------- (7.1)
We continue the discussion assuming a fixed tuple size formulation for the
solution space. The discussion is easily extended to the variable tuple size
formulation. Every leaf node in the state space tree represents an
assignment for which ni1
ii mxw is an answer (or solution) node. All
other leaf nodes are infeasible. For a minimum-cost answer node to
correspond to any optional solution, we need to define ni1
ii xpxc for
every answer node x. The cost c(x) = for infeasible leaf nodes. For non-
leaf nodes, c(x) is recursively defined to be min{c(Ichild(x)), c(rchild(x))}
Fundamentals of Algorithms Unit 7
Sikkim Manipal University Page No. 140
Algorithm 7.1
1. Algorithm U Bound(cp, cw, k, m)
2. // cp is the current profit total, cw is the current
3. // weight total; k is the index of the last removed
4. // item; and m is the knapsack size
5. // w[i] and p[i] are respectively the weight and profit
6. // of the ith object
7. {
8. b:=cp; c:=cw;
9. for I:=k+1 to n do
10. {
11. if (c+w[i] m) then
12. {
13. c:c+w[i]; b:=b-p[i];
14. }
15. }
16. return b;
17. }
We now need two functions c (x) and u(x) such that c (x) c(x) u(x) for
every node x. The cost c (.) and u(.) satisfying this requirement may be
obtained as follows. Let x be a node at level j, 1 j n+1. At node x,
assignments have already been made to xi, 1 i < j. The cost of these
assignments is ji1
ii xp . So, ji1
ii xpxc and we may use
ji1
ii .xpxu If ji1
ii xpq then an improved upper bound function u(x) is
u(x) = Ubound ,m,1j,xw,qji1
ii where Ubond is defined in Algorithm
Self Assessment Question
2. To use the branch-and-bound technique to solve any problem, it is first
necessary to conceive of a –––– ––––– –––––for the problem.
Fundamentals of Algorithms Unit 7
Sikkim Manipal University Page No. 141
7.4 Travelling Salesman Problem
An O(n22n) dynamic programming algorithm for the travelling sales person
was arrived at in Unit 5. We now investigate branch-and-bound algorithms
for this problem. Although the worst-case complexity of these algorithms will
not be any better than O(n22n), the use of the good bounding functions will
enable these branch and bound algorithms to solve some problem instances
in much time less than that required by the dynamic programming algorithm.
Let G=(V,E) be a directed graph defining an instance of the travelling sales
person problem. Let cij equal the cost of edge <i, j>,cij = if < i, j > E, and
let |V|=n. Without loss of generality, we can assume that every tour starts
and ends at vertex 1. So, the solution space S is given by S = {1, , 1| is a
permutation of (2,3 ,n)}. Then, | s |= (n — 1)!. The size of S can be reduced
by restricting S so that (1, i1,i2,……in–1,1) S iff < ij, ij+1 > E, 0 j n — 1,
and i0 = in = 1.
To search the travelling salesperson state space tree, we need to define a
cost function c (.) and two other functions c (.) and u(.) such that c (r) c(r)
u(r) for all nodes r. The cost c(.) is such that the solution node with least
c(.) corresponds to a shortest tour in G. One choice for c(.) is
c(A) = length of tour defined by the path from the root to A, if A is a leaf
and
c(A) = cost of minimum-cost leaf in the subtrees A if A is not a leaf
l(i) = length of the path defined at node i
Fig. 7.1: State space tree for the travelling sales person problem with n=4 and
i0 = i4 = 1
Fundamentals of Algorithms Unit 7
Sikkim Manipal University Page No. 142
120011
012315
2030
021112
101710
167416
318619
4253
241615
11103020
a) Lost matrix b) Reduced cost matrix L=25
Fig. 7.2: An example of cost matrix and reduced cost matrix
A simple c (.) such that c (A) c(A) for all A is obtained by defining c (A) to
be the length of the path defined at node A. A better c (.) can be obtained
by using the reduced cost matrix corresponding to G. A row (column) is said
to be reduced if it contains atleast one zero and all remaining entries are
non-negative. A matrix is reduced iff every row and column is reduced. As
an example of how to reduce the cost matrix of a given graph G, consider
the matrix of fig. 7.2(a). This corresponds to a graph with five vertices. Since
every tour on this graph includes exactly one edge <i, j> with i=k, 1 k 5,
and exactly one edge <i, j> with j=k, 1 k 5, subtracting a constant t from
every entry in one column or one row of the cost matrix reduces the length
of every tour by exactly t. A minimum-cost tour remains a minimum-cost tour
following this subtraction operation. If t is chosen to be the minimum entry in
row i (column j), then subtracting it from all entries in row i (column j)
introduces a zero into row i (column j) Repeating this as often as needed,
the cost matrix can be reduced. The total amount subtracted from the
columns and rows is a lower bound on the length of a minimum-cost tour
and can be used as the c value for the root of the state space tree.
Subtracting 10, 2, 2, 3, 4, 1 and 3 from rows 1, 2, 3, 4 and 5 and Column 1
and 3 respectively of the matrix of fig 7.2(a), yields the reduced matrix of fig.
7.2(b). The total amount subtracted is 25. Hence, all tours in the original
graph have a length of at least 25.
We can associate a reduced cost matrix with every node in the travelling
salesperson state space tree. Let A be the reduced cost matrix for node R.
Let S be a child of R such that the tree edge (R, S) Corresponds to including
edge <i,j> in the tour. If S is not a leaf, then the reduced cost matrix for S
may be obtained as follows:
Fundamentals of Algorithms Unit 7
Sikkim Manipal University Page No. 143
1) Change all entries in row i and column j and A to . This prevents the
use of any more edge leaving vertex i or entering vertex j.
2) Set A(j,1) to . This prevents the use of edges <j,1>.
3) Reduce all rows and columns in the resulting matrix except for rows and
columns containing only .
Let the resulting matrix be B.
Step (1) and (2) are valid as no tour in the subtree S can contain edges of
the type <i,k> or <k,j> or <j,1> (except for edge <i,j>). If r is the total amount
subtracted in step (3) then,
c (S) = c (R) c + A(i, j)+r. For leaf nodes, (.)= c( c ) is easily computed as
each leaf defines a unique tour. For the upper bound function u, we can use
u( R)= for all nodes R.
Self Assessment Question
3. We can associate a –––– –––– ––––– with every node in the travelling
salesperson state space tree
7.5 Summary
The term branch-and-bound refers to all state space search method in
which all children of the E-node are generated before any other live
node can become the E-node.
The Knapsack problem using branch-and-bound technique states as
n
1i
ii xpMinimize
n
1i
ii mxwtoSubject
xi = 0 or 1, 1 i n.
In travelling salesperson problem, one choice for c(.) is
leafanotisAifA,subtreestheinleafcostminimumofcost
leafaisA,torootthefrompaththebydefinedtourlengthAc
Fundamentals of Algorithms Unit 7
Sikkim Manipal University Page No. 144
7.6 Terminal Questions
1. Briefly explain the O/I Knapsack Problem
2. Explain the travelling Salesman Problem
3. Write the Algorithm for Knapsack Problem
4. Write the general method for Branch and Bound
5. What do you mean by Bounding?
7.7 Answers
Self Assessment Questions
1. E-node
2. state space tree
3. reduced cost matrix
Terminal Questions
1. To use the branch-and-bound technique to solve any problem, it is first
necessary to conceive of a state space tree for the problem.
The objective function ii xp is replaced by the function .xp ii Clearly
ii xp is maximized iff – ii xp is minimized. This modified knapsack
problem is stated as given below
n
1i
ii xpMinimize
n
1i
ii mxwtoSubject
xi = 0 or 1, 1 i n. (Refer Section 7.3)
2. Let G=(V,E) be a directed graph defining an instance of the traveling
sales person problem. Let cij equal the cost of edge <i, j>,cij = if < i, j >
E, and let |V|=n. Without loss of generality, we can assume that every
tour starts and ends at vertex 1. So, the solution space S is given by S =
{1, , 1| is a permutation of (2,3 ,n)}. Then | s |= (n — 1)!. The size of S
can be reduced by restricting S so that (1, i1,i2,……in–1,1) S iff < ij, ij+1 >
E, 0 j n — 1, and i0 = in = 1.
Fundamentals of Algorithms Unit 7
Sikkim Manipal University Page No. 145
To search the traveling salesperson state space tree, we need to define
a cost function c (.) and two other functions c (.) and u(.) such that c (r)
c(r) u(r) for all nodes r. The cost c(.) is such that the solution node
with least c(.) corresponds to a shortest tour in G. One choice for c(.) is
c(A) = length of tour defined by the path from the root to A, if A is a leaf
and c(A) = cost of minimum-cost leaf in the subtrees A and if A is not a
leaf l(i) = length of the path defined at node i. (Refer Section 7.4).
3. To use the branch-and-bound technique to solve any problem, it is first
necessary to conceive of a state space tree for the problem.
The objective function ii xp is replaced by the function .xp ii
Clearly, ii xp is maximized iff – ii xp is minimized. This modified
knapsack problem is stated as given below
n
1i
ii xpMinimize
n
1i
ii mxwtoSubject
xi = 0 or 1, 1 i n ------- (7.1)
We continue the discussion assuming a fixed tuple size formulation for
the solution space. The discussion is easily extended to the variable
tuple size formulation. Every leaf node in the state space tree represents
an assignment for which ni1
ii mxw is an answer (or solution) node.
All other leaf nodes are infeasible. For a minimum-cost answer node to
correspond to any optional solution, we need to define ni1
ii xpxc
for every answer node x. The cost c(x) = for infeasible leaf nodes. For
non-leaf nodes, c(x) is recursively defined to be min{c(Ichild(x)),
c(rchild(x))}
Fundamentals of Algorithms Unit 7
Sikkim Manipal University Page No. 146
Algorithm 7.1
18. Algorithm U Bound(cp, cw, k, m)
19. // cp is the current profit total, cw is the current
20. // weight total; k is the index of the last removed
21. // item; and m is the knapsack size
22. // w[i] and p[i] are respectively the weight and profit
23. // of the ith object
24. {
25. b:=cp; c:=cw;
26. for I:=k+1 to n do
27. {
28. if (c+w[i] m) then
29. {
30. c:c+w[i]; b:=b-p[i];
31. }
32. }
33. return b;
34. }
Now we need two functions c (x) and u(x) such that c (x) c(x) u(x)
for every node x. The cost c (.) and u(.) satisfying this requirement may
be obtained as follows. Let x be a node at level j, 1 j n+1. At node x,
assignments have already been made to xi, 1 i < j. The cost of these
assignments is ji1
ii xp . So, ji1
ii xpxc and we may use
ji1
ii .xpxu If ji1
ii xpq then, an improved upper bound function
u(x) is u(x) = Ubound ,m,1j,xw,qji1
ii where Ubond is defined in
Algorithm.
4. The general method is
1. In branch and bound method a state space tree is built and all the
children of E nodes ( a live node whose children are currently being
Fundamentals of Algorithms Unit 7
Sikkim Manipal University Page No. 147
generated) are generated before any other node can become a live
node.
2. For exploring new nodes either a BFS or D-Search technique can be
used.
3. In Branch and bound technique, BFS-like state space search will be
called FIFO (First in First Out) search. This is because the list of live
node is First In First Out list (queue). On the other hand, the D-
search like state space search will be called LIFO search because
the list of live node is Last in First out list (stack).
4. In this method, a space tree of possible solutions is generated. Then
partitioning (called as branching) is done at each node of the tree.
We compute lower bound and upper bound at each node. This
computation leads to selection of answer node.
5. Bounding functions are used to avoid the generation of subtrees that
do not contain an answer node.
5. Bounding is having the following characteristics
1. As we know that the bounding functions are used to avoid the
generation of sub trees that do not contain the answer nodes. In
bounding lower bounds and upper bounds are generated at each
node.
2. A cost function c (x) is such that c (x) c (x) is used to provide the
lower bounds on solution obtained from node x.
3. Let upper is an upper bound on cost of minimum-cost solution. In
that case, all the live nodes with c (x) upper can be killed.
4. At the start, the upper is usually set to . After generating the
children of current E-node, upper can be updated by minimum cost
answer node. Each time a new answer node can be obtained.
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 148
Unit 8 Additional Features Structure
8.1 Introduction
Objectives
8.2 Lower Boundary Theory
8.3 NP Hard and NP Complete Problems
8.4 Approximate Algorithms for NP Hard Problems
Absolute approximation
- Approximation
Polynomial Time Approximation Schemes
8.5 Summary
8.6 Terminal Questions
8.7 Answers
8.1 Introduction
We have studied in the previous units the different techniques of algorithms.
Now, the purpose of this unit is to expose you to some techniques that have
been used to establish that a given algorithm is the most efficient possible.
The way this is done is by discovering a function g(n) that is a lower bound
on the time that any algorithm must take to solve the given problem. If we
have an algorithm whose computing time is the same order as g(n), then we
know that asymptotically we can do no better.
Objectives
After studying this unit, you should be able to:
apply the Lower Boundary theory
use Comparison tree for searching and sorting
apply Approximate Algorithms for NP hard problems
use Approximation in Bin Packing
analyse Polynomial time approximation schemes.
8.2 Lower Boundary Theory
If f(n) is the time for some algorithm, then we write f(n)= (g(n)) to mean that
g(n) is a lower bound for f(n). Formally, this equation can be written if there
exist positive constants C and no such that |f(n)| C|g (n)| for all n>no. In
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 149
addition to developing lower bounds to within a constant factor, we are also
concerned with determining more exact bounds whenever this is possible.
For many problems it is possible to easily observe that a lower bound
identical to n exists where n is the number of input to the problem. Consider
all algorithms that find the maximum of an unordered set of n integers.
Clearly, every integer must be examined at least once, so (n) is a lower
bound for any algorithm that solves this problem.
Comparison Trees for Searching and Sorting
Below, we will study some of the comparison trees used for searching and
sorting
Ordered Searching:
In obtaining the lower bound for the ordered searching problem, we consider
only those comparison based algorithms in which every comparison
between two elements of S is of the type “compare x and A [i]”. The
progress of any searching algorithm that satisfies this restriction can be
described by a path in a binary tree. Each internal node in this tree
represents a comparison: between x and A[i], There are three possible
outcome of this comparison:
x<A[i],x=A[i], or x>A[i]. We can assume that if x=A[i], the algorithm
terminates. The left branch is taken if x<A[i], and the right branch is taken
x>A[i]. If the algorithm terminates, following a left or right branch, then no i
has been such that x=A[i] and the algorithm must declare the search
unsuccessful.
Theorem: Let a[1:n],n 1, contain n distinct elements ordered so that
a[1]<……<a[n]. Let FIND (n) be the minimum number of comparisons
needed, in the worst case, by any comparison-based algorithm to recognize
whether x A[1:n]. Then FIND (n) [log (n+1)].
Proof: Consider all possible comparison trees that model algorithm to solve
the searching problem. The value of FIND (n) is bound below by the
distance of the longest path from the root to a leaf in such a tree. There
must be n internal nodes in all these trees corresponding to the n possible
successful occurrences of x in A. If all internal nodes of a binary tree are at
levels less than or equal to k, then there are at most 2k–1 internal modes.
Thus n<2k–1 and FIND (n)=k [log (n+1)].
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 150
Sorting:
Now, let’s consider the sorting problem. We can describe any sorting
algorithm that satisfies the restrictions of the comparison tree model by a
binary tree. Consider the case in which the n numbers A [1:n] to be sorted
are distinct. Now, any comparison between A[i] and A[j] must result in one
of two possibilities:
Either A [i]<A [j] or A [i] > A[j]. So, the comparison tree is a binary tree in
which each internal node is labeled by the pair i : j which represents the
comparison of A[i] with A[j]. If A[i] is less than A[j], then the algorithm
proceeds down the left branch of the tree; otherwise, it proceeds down the
right branch. The external nodes represent termination of the algorithm
associated with every path from the root to an external node is a unique
permutation. To see that this permutation is unique, note that the algorithms
we allow are only permitted to move data and make comparisons. The data
movement on any path from the root to an external node is the same no
matter what the initial input values are. As there are n! different possible
permutations of n items, and any one of these might legitimately be the only
correct answer for the sorting problem on a given instance, the comparison
tree must have at least n! external nodes.
Fig 8.1 shows a comparison tree for sorting three items. The first
comparison is A[1]:A[2]. If A[1] is less than A[2], then the next comparison
is A[2] with A[3]. If A[2] is less than A[3], then the left branch leads to an
external nodes containing left branch leads to an external node containing
1,2,3. This implies that the original set was already sorted for
A[1]<A[2]<A[3]. The other five external nodes correspond to the other
possible orderings that could yield a sorted set.
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 151
Fig. 8.1: A Comparison Tree for sorting three items
We consider the worst case for all comparison based sorting algorithms. Let
T (n) be the minimum number of comparisons that are sufficient to sort n
items in the worst case. Using our knowledge of binary trees once again, if
all internal nodes are at levels less than k, then there are at most 2k external
nodes. Therefore, if we let k = T (n)
n! 2T(n)
Since T(n) is an integer, we get the lower bound
T (n) [log n!]
Self Assessment Question
1. In obtaining the lower bound for the ordered searching problem, we
consider only those comparison based algorithms in which every
comparison between two elements of S is of the type ––––––––––– .
8.3 NP Hard and NP Complete Problems
Basic Concepts
In this unit, we are concerned with the distinction between problems that can
be solved by a polynomial time algorithm and problems for which no
polynomial time algorithm is known. It is an unexplained phenomenon that
for many of the problems we know and study, the best algorithms for their
solutions have computing times that cluster into two groups. The first group
consists of problems whose solution are bounded by polynomials of small
degree.
1:2
2:3 2:3
1:3 1:3 1, 2, 3
3, 2, 1
1, 3, 2 3, 1, 2 2, 1, 3 2, 3, 1
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 152
The second group is made up to problems whose best-known algorithms
are non-polynomial. Examples we have seen include the traveling
salesperson and the knapsack problems for which the best algorithms given
in this text have complexities O (n22n) and O (2n/2) respectively.
In the quest to develop efficient algorithms, no one has been able to develop
a polynomial time algorithm for any problem in the second group. This is
very important because algorithms whose computing times are greater then,
polynomial very quickly require such vast amounts of time to execute that
even moderate-size problems cannot be solved.
The theory of NP-completeness, which we present here, does not provide a
method of obtaining polynomial time algorithms for problems in the second
group. Nor does it say that algorithms of this complexity do not exist.
Instead, what we do is show that many of the problems for which there are
no known polynomial time algorithms are computationally related. In fact, we
establish two classes of problems. These are given names, NP-hard and
NP-Complete.
A problem that is NP-Complete has the property that it can be solved in
polynomial time if and only if all other NP-Complete problems can also be
solved in polynomial time.
If an NP-hard problem can be solved in polynomial time, then all NP-
Complete problems can be solved in polynomial time. All NP complete
problems are NP-hard, but some NP-hard problems are not known to be
NP-complete.
Fig. 8.2: Venn diagram
NP Complete
NP
P
NP hard
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 153
Although one can define many distinct problem classes having the
properties stated above for the NP-hard and NP-complete classes, the
classes we study are related to non-deterministic computations. The
relationship of these classes to non deterministic computations together with
the apparent power of non- determinism lead to the intuitive conclusion that
no NP- complete or NP- hard problem is polynomially solvable.
Cook’s Theorem
Cook’s theorem states that satisfiability is in P if and only if P=NP. We now
prove this important theorem. We know that satisfiability in NP. Hence, if P
= NP, then satisfiability is in P. It remains to be shown that if satisfiability is
in P, then P = NP. To do this, we show how to obtain from any polynomial
time non- deterministic decision algorithm A and input I a formula Q (A, I)
such that Q is satisfiable if A has a successful termination with input I. If the
length of I – is n and the time complexity of A is p (n) for some polynomial
p(), then the length of Q is O (p3(n)log n) = O(p4(n)). The time needed to
construct Q is also O (p3(n) log n). A deterministic algorithm Z to determine
the out come of A on any input I can be easily obtained. Algorithm Z simply
computes Q and then uses a deterministic algorithm for the satisfiability
problem to determine whether Q is satisfiable. If O (q (m)) is the time
needed to determine whether a formula of length m is satisfiable, then the
complexity of Z is O (p3(n) log n+q (p3(n)log n)). If satisfiability is in P, then q
(m) is a polynomial function of m and the complexity of Z becomes O(r (n))
for some polynomial r(). Hence, if satisfiability is in P, then for every non-
deterministic algorithm A in NP we can obtain a deterministic Z in P. So, the
above construction shows that if satisfiability is in P, then P=NP.
Some Theorems
Definition: A K- clique in a graph G is a complete sub graph of G with K-
vertices.
CNF product of sums
Theorem 1: CNF satisfiability is polynomially transformable to the clique
problem. Therefore, the clique problem is NP complete.
Proof: Let F=F1 F2…….Fq be an expression in CNF, where the Fi’ s are the
factors, each Fi is of the form
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 154
(xi1+xi2+…..xik) where xij is a literal. Construct an undirected graph G=(V,E)
whose vertices are pairs of integers [i,j] for 1 i q and 1 j k.
First component represents a factor and the 2nd component represents a
literal within the factor since there are 2n combinations. It is a polynomial
And it is NP-complete.
Definition: Let G=(V,E) be an undirected graph. A vertex cover of G is a
subset SÇV such that each edge of G is incident upon some vertex in S.
Theorem 2: The clique problem is polynomially transformable to the vertex
cover problem. Therefore, the vertex problem is NP-Complete.
Proof: Given an undirected graph G=(V,E), consider the complement graph
G=(V,E)
Where E {(v, w)|v,w V,v w and (v,w) E}
A set S V is a clique in G iff V-S is a vertex cover of G. If S is a clique in
G, no edge in G connects 2 vertices in S. Thus, every edge in G is incident
upon atleast one vertex in V-S, implying V-S is a vertex cover of G.
Similarly, if V-S is a vertex cover of G, then every edge of G is incident upon
atleast one vertex of V-S. Thus, no edge of G connects 2 vertices in S.
Therefore, every pair of vertices of S is connected in G, and S is a clique in
G.
To find whether there exists a clique of size k, we construct G and
determine whether it has a vertex cover of size ||v||-k. Given a standard
representation of G=(V,E) and K, we find a representation of G and ||v||-k in
time which is a polynomial in the length of the representation for G and K.
Self Assessment Questions
2. first group consists of problems whose solution times are bounded by
polynomials of ––––––––––.
3. A K- clique in a graph G is a complete sub graph of G with ––––––––.
8.4 Aproximate Algorithms for NP Hard Problems
Many NP-hard optimization problems have great practical importance and it
is desirable to solve large instances of these problems in a reasonable
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 155
amount of time. The best-known algorithms for NP-hard problems have a
worst-case complexity that is exponential in the number of inputs.
A feasible solution with value close to the value of an optimal solution is
called an approximate solution. An approximate algorithm for P is an
algorithm that generates approximate solution for P
Let A be an algorithm that generates a feasible solution to every instance I
of a problem P.
Let F*(I) be the value of an optimal solution to I. Let F(I) be the value of the
feasible solution generated by A.
Definition: A is an absolute approximation algorithm for problem P if and
only if for every instance I of P, | F*(I) – F(I) | K for some constant k.
Definition: An -approximate algorithm is an approximate algorithm for
which f(n) for some constant .
Definition: A ( ) is an approximation scheme if and only if for every given
>O and problem instance I, A( ) generates a feasible solution such that
|F*(I) – F(I)|/F*(I) . Again, we assume F* (I) > O
Definition: An approximation scheme is a polynomial time approximation
scheme if and only if for every fixed >O it has a computing time that is
polynomial in the problem size.
8.4.1 Absolute Approximation
Below, we study some of the absolute approximations along with their
algorithms
Planar Graph Coloring
There are very few NP-hard optimization problems for which polynomial
time absolute approximation algorithms are known. One problem is that of
determining the minimum number of colors needed to color a planar graph
G= (V,E). It is known that every planar graph is four colorable. It is zero
colorable iff V=ø It is one colorable iff E=ø It is two colorable if it is bipartite.
Determining whether a planar graph is three colorable is NP – hard.
However, all planar graphs are four colorable. An absolute approximation
algorithm with |F*(I)-F(I)| 1 is easy to obtain. Algorithm 7.1 is such an
algorithm. It finds an exact answer when the graph can be colored using at
^
^
^
^
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 156
most two colors. Since we can determine whether a graph is bipartite in
time O (|V|+|E|), the complexity of the algorithm is to 0(|V|+|E|).
Below, we discuss an algorithm for approximate coloring
Algorithm 8.1
1. Algorithm A color (V,E,)
2. // Determine an approximation to the minimum number of colors.
3. {
4. if V=Ø then return 0;
5. else if E=Ø then return 1;
6. else if G is bipartite then return 2;
7. else return 4;
8. }
Maximum Programs Stored Problem
Assume that we have n programs and two storage devices (say disks or
tapes). We assume the devices are disks and our discussion applied to any
kind of storage device. Let li be the amount of storage needed to store the
ith program. Let L be the storage capacity of each disk. Determine the
maximum number of these n program over the disks) is NP-hard.
Theorem: Partition varies as the maximum programs stored.
Proof: Let {a1, a2,……an}, define an instance of the partition problem. We
can assume ai = 2T. Define an instance of the maximum programs stored
problem as follows: L=T and li=ai, 1 i . Clearly {a1,…..an} has a partition
if and only if n programs can be stored on the two disks.
By considering programs in order of non-decreasing storage requirement li,
we can obtain a polynomial time absolute approximation algorithm.
Function Pstore (Algorithm 8.2) assumes l1 l 2 …. l n and assigns
programs to disk 1 so long as enough space remains on this tape. Then it
begins assigning programs to disk 2. In addition to the time needed to
initially sort the programs into non-decreasing order of ll, O (n) time is
needed to obtain the storage assignment.
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 157
Below we discuss approximation Algorithm to store programs
Algorithm 8.2
1) Algorithm Pstore(l, n, L)
2) //Assume that l [i] l [i+1],1 i n.
3) {
4) i:=1;
5) for j:=1 to 2 do
6) {
7) sum:=0;//Amount of disk j already assigned
8) while (sum + l [i]) L do
9) {
10) write(“Store program “,i,”on disk”,j);
11) sum :=sum+l[i];i :=i+1 ;
12) if i>n then return;
13) }
14) }
15) }
Example: Let L=10, n=4, and (l 1, l 2, l 3, l 4)=(2,3,4,5). Function Pstore will
store programs 1 and 2 on disk 1 and only program 3 on disk 2. An optimal
storage scheme stores all four programs. One way to do this is to store
programs 1 and 4 on disk 1 and the other two on disk 2.
8.4.2 Approximation
Below, we study some Approximation
Scheduling Independent Tasks
Obtaining minimum finish time schedules on m, m 2, identical processors
is NP-hard. There exists a very simple scheduling rule that generates
schedules with a finish time very close to that of an optimal schedule. An
instance I of the scheduling problem is defined by a set of n task times ti, 1
i n, and m, the number of processors. The scheduling rule we are about to
describe is known as the LPT. An LPT schedule is a schedule that results
from this rule.
Definition: An LPT schedule is one that is the result of an algorithm that,
whenever a processor becomes free, assigns to that processor a task
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 158
whose time is the largest of those tasks not yet assigned. Ties are broken
in an arbitrary manner.
Example: Let m=3, n=6, and (t1, t2,t3,t4,t5,t6)=(8,7,6,5,4,3).In an LPT
schedule tasks 1, 2 and 3 are assigned to processor 1, 2 and 3 respectively.
Tasks 4, 5 and 6 are respectively assigned to process 3, 2 and 1. The finish
time is 11. Since ti/3 =11,the schedule is also optimal.
6 7 8 11
P1
P2
P3
1 6
2 5
3 4
Fig. 8.3: LPT schedule for the example
Bin Packing
In this problem, we are given n object that have to be placed in bin s of
equal capacity L. Object i requires li units of bin capacity. The objective is to
determine the minimum number of bins needed to accommodate all n
objects. No object may be placed partly in one bin and partly in another.
Example: Let L=10,n=6,and (l 1, l 2, l 3, l 4, l 5, l 6)=(5, 6, 3, 7, 5, 4). Fig 8.3
shows a packing of the six objects using three bins. Numbers in bins are
object indices. Obviously, at least three bins are needed.
The bin packing problem can be regarded as a variation of the scheduling
problem. The bins represent processors and L is the time by which all tasks
must be completed. The variable li, is the processing requirement of task i.
The problem is to determine the minimum number of processor needed to
accomplish this. An alternative interpretation is to regard the bins as tapes.
The variable L is the length of a tape, and li, the tape length needed to store
program i. The problem is to determine the minimum number of tapes
needed to store all n programs. Clearly, many interpretations exist for this
problem.
Theorem: The bin packing problem is NP-hard.
Proof: To see this, consider the partition problem Let {a1, a2…..an} be an
instance of the partition problem. Define an instance of the bin packing
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 159
problem as li = ai, 1 i n, and L= ai/2. Clearly, the minimum number of
bins needed is two if and only if there is a partition for {a1, a2……..an}.
(a) (b) (c)
Fig. 8.4: Optimal Packing for the example
8.4.3 Polynomial Time Approximation Schemes
The LPT rule leads to a (1/3-1/(3m))-approximate algorithm for the problem
obtaining an m processor schedule for n task . A polynomial time
approximation scheme is also known for this problem. This scheme relied
on the following scheduling rule Let k be some specified and fixed integer.
Obtain an optimal schedule for the K longest tasks. Schedule the remaining
n-k tasks using the LPT rule
Example: Let m = 2, n = 6, (t1, t2, t3, t4, t5, t6) = (8, 6, 5, 4, 4, 1), k = 4. The
four longest tasks have task times 8, 6, 5 and 4 respectively. An optional
schedule for these has finish time 12 (Fig. 8.4(a)). When the remaining two
task are scheduled using the LPT rule, the schedule of Fig. (8.4 (b)) results.
This has finish time 15. Fig. (8.4 (c)) shows as optimal schedule. This has
finish time 14.
Theorem: Let I be an m-processor instance of the scheduling problem. Let
F*(I) be the finish time of an optimal schedule for I and let F (I) be the length
of the schedule generated by the above scheduling rule. Then,
Proof: Let r be the finish time of an optimal schedule for the k longest tasks.
If F (I)=r, the F*(I)= F (I) and the theorem is proved. So, assume F (I)>r. Let
ti, 1 i n, be the task times of the n tasks of I.
Without loss of generality, we can assume ti ti+1, 1 i<n, and n>k. Also, we
can assume n>m. Let j, j>k, be such that task j has finish time F(I). Then, no
5 1
6 2
6 2
]m/k[1
m/11
)I(*F
|)I(F)I(*F|
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 160
processor is idle in the interval [0, F (I)–tj]. Since tk+1 tj, it follows that no
processor is idle in the interval [0, F )I)–tk+1].
Hence
1k1k1i i t)t)I(F(mt
and so, F*(I) 1kn
1 i tm
1m)I(Ft
m
1
or |F*(I)– F (I)| 1ktm
1m
Since ti tk+1, 1 i k+1 and at least one processor must execute at least
1+[k/m] of these k+1 tasks, it follows that
1kmk t1)I(*F
Combining these two inequalities, we obtain
mk
m1
mk 1
1
1
m/)1m(
)I(*F
|)I(F)I(*F|
Using the result of theorem, we can construct a polynomial time –
approximation scheme for the scheduling problem. This scheme has as
an input variable. For any input , it computes an integer k such that (1-1/m)/(1+[k/m]). This defines the k to be used in the scheduling rule described
above. Solving for k, we obtain that any integer k, k>(m – 1)/ – m,
guarantees -approximate schedules. The time required to obtain such
schedules, however, depends mainly on the time needed to obtain an
optimal schedule for k tasks on m machines. Using a branch and bound
algorithm this time is O (mk). The time needed to arrange the tasks so that ti
t i+1 and also to obtain the LPT schedule for the remaining n – k tasks is
O(n log n). Hence the total time needed by the -approximate scheme is
O(n log n+mk) = O(n log n)+m[(m-1)/ – m]).Since this time is not polynomial in
1/ (it is exponential in1/ ), this approximation scheme is not a fully
polynomial time approximation scheme. It is a polynomial time
approximation scheme (for any fixed m) as the computing time is polynomial
in the number of task n.
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 161
Self Assessment Questions
4. A feasible solution with value close to the value of an optimal solution is
called an ––––––––––.
5. An -approximate algorithm is an approximate algorithm for which ––––.
6. –––––––– are broken in an arbitrary manner.
7. The bin packing problem is –––––––––.
8.5 Summary
In this unit, we studied the following
f(n) = (g(n)) is the lower bound on time for any algorithm.
Cooks theorem states that satisfiability is in P if and only if P=NP.
A k-clique in a graph G is a complete sub-graph of G with k-vertices.
A vertex cover of G, where G=(V, E) is a undirected graph, is a subset S
V such that each edge of G is incident upon some vertex in X.
The bin packing problem is NP-hard.
8.6 Terminal Questions
1. What do you mean by Order Searching
2. Explain Cook’s Theorem.
8.7 Answers
Self Assessment Questions
1. compare x and A [i]
2. small degree
3. K-vertices
4. approximate solution
5. f(n) for some constant
6. Ties
7. NP-hard
Fundamentals of Algorithms Unit 8
Sikkim Manipal University Page No. 162
Terminal Questions
1. In the ordered searching problem, we consider only those comparison
based algorithms in which every comparison between two elements of S
is of the type “compare x and A [i]”. The progress of any searching
algorithm that satisfies this restriction can be described by a path in a
binary tree. Each internal node in this tree represents a comparison:
between x and A[i]. (Refer Section 8.2).
2. Cook’s theorem states that satisfiability is in P if and only if P=NP. We
now prove this important theorem. We know that satisfiability in NP.
Hence, if P = NP, then satisfiability is in P. It remains to be shown that if
satisfiability is in P, then P = NP. To do this, we show how to obtain from
any polynomial time non- deterministic decision algorithm A and input I a
formula Q (A, I) such that Q is satisfiable if A has a successful
termination with input I. (Refer Section 8.3).
Fundamentals of Algorithms Unit 9
Sikkim Manipal University Page No. 163
Unit 9 Graphs Structure
9.1 Introduction
Objectives
9.2 Preliminary Definitions and Notations
9.3 Trees
9.4 Summary
9.5 Terminal Questions
9.6 Answers
9.1 Introduction
In the previous units we discussed the different techniques of Algorithms. In
order to use these techniques in a proper way, you should be knowing the
concepts of graph. Because of its inherent simplicity, graph theory has a
very wide range of applications in engineering, in physical, social and
biological sciences, in linguistics and in numerous other areas. A graph can
be used to represent almost any physical situation involving discrete objects
and a relationship among them.
Objectives
After studying this unit, you should be able to:
use the definition of graph with related terminologies such as vertex,
edge etc.
apply the concept of directed graph
use the terminologies such as degree, complete graph, bipartite graph
and connectivity
use the definition of trees and properties of trees.
9.2 Preliminary Definitions and Notations
Definition: A linear graph (or simply a graph) G = (V,E) consists of a set of
objects V = {v1,v2…..} called vertices, and another set E = {e1,e2…}, whose
elements are called edges, such that each edge ek is identified with an
unordered pair (vi, vj) of vertices. The vertices vi, vj associated with edge ek
are called the end vertices of ek. The most common representation of a
graph is by means of a diagram in which the vertices are represented as
points and each edge as a line segment joining its end vertices. Often, this
Fundamentals of Algorithms Unit 9
Sikkim Manipal University Page No. 164
diagram itself is referred to as the graph. The object shown in Fig. 9.1 for
instance, is a graph.
Fig. 9.1: Graph with five vertices and seven edges
An edge having the same vertex as both its end vertices called a self loop.
Edge e1 in Fig. 9.1 is a self loop. Also, note that the definition allows more
than one edge associated with a given pair of vertices, for example, edge e4
and e5 in Fig. 9.1. Such edges are referred to as parallel edges.
A graph that has neither self-loops nor parallel edges is called a simple
graph. Some authors use the term general graph to emphasize that parallel
edges and self-loops are allowed.
A graph is also called a linear complex, a 1 – complex, or a one-dimensional
complex. A vertex is also referred to as a node, a function, a point, o-cell or
an O-simplex. Other terms used for an edge are a branch, a line, an
element, a 1-cell, an arc, and a – simplex. In this unit, we shall generally use
the terms graph, vertex, and edge.
Definition: A directed graph G consists of a set of vertices
V = {v1,v2,……}, a set of edges E = {e1, e2…..}, and a mapping that maps
every edge onto some ordered pair of vertices (vi,vj). As in the case of
undirected graph, a vertex is represented by a point and an edge by a line
segment between vi and vj with an arrow directed from vi to vj.
Definition: A walk is defined as a finite alternating sequence of vertices and
edges, beginning and ending with vertices, such that edge is incident with
the vertices preceding and following it. No edge appears more than once in
Fundamentals of Algorithms Unit 9
Sikkim Manipal University Page No. 165
a walk. A vertex, however, may appear more than once. A walk is also
referred to as an edge Train or a chain. The set of vertices and edges
constituting a given walk in a graph G is clearly a sub graph of G.
Vertices with which a walk begins and ends are called its terminal vertices,
Vertices v1 and v5 are the terminal vertices of the walk shown in Fig. 9.2. It is
possible for a walk to begin and end at the same vertex. Such a walk is
called a closed walk. A walk that is not closed (i.e. Terminal vertices are
distinct) is called an open walk (Fig. 9.2).
Fig. 9.2: An open Walk
An open walk in which no vertex appears more than once is called a path
(or a simple path or an elementary path). In the graph, v1a v2b v3d v4 is a
path where as v1a v2b v3d v4e v2f v5 is not a path. In other words, a path
does not intersect itself.
The number of edges in a path is called the length of a path. It immediately
follows, then, that an edge which is not a self-loop is a path of length one. It
should also be noted that a self-loop can be included in a walk but not a
path.
Definition: A closed walk in which no vertex (except the initial and the final
vertex) appear more than once is called a circuit. That is, a circuit is a
closed, nonintersecting walk. In Fig. 9.2., v2b v3d v4e v2 is, for example, a
circuit. Three different circuits are shown in Fig. 9.3 A. Clearly, every vertex
Fundamentals of Algorithms Unit 9
Sikkim Manipal University Page No. 166
in a circuit is of degree two; again, if the circuit is a sub graph, one must
count degrees contributed by the edges in the circuit only.
Fig. 9.3 A: Three different circuits
A circuit is also called a cycle, elementary cycle, circular path and polygon.
In electrical engineering a circuit is sometime refered to as a loop.
The definitions in this section are summarized in Fig. 9.3 B the arrows are in
the direction of increasing restriction.
Fig. 9.3 B: Walks, Paths and Circuits as Subgraphs
Definition: The number of edges incident on a vertex vi with self-loops
counted twice, is called the degree, d(vi), of vertex vi. In Fig. 9.4, for
example,
d(v1) = d(v3) = d(v4) = 3, d(v2) = 4, and d(v5) = 1. The degree of a vertex is
sometimes also referred to as its valency.
Sub graph of G
Walk in G
Path in G Circuit in G
Any collection of edges in G
A non-edge-retracing sequence of edge of G
A non-intersecting closed walk in G
A non-intersecting open walk in G
Fundamentals of Algorithms Unit 9
Sikkim Manipal University Page No. 167
Fig. 9.4: A Graph with five vertices and seven edges.
Definition: A simple graph in which there exists an edge between every pair
of vertices is called a complete graph. Complete graphs of two, three, four
and five vertices are shown in Fig. 9.5. A complete graph is sometimes also
referred to as a universal graph or a clique.
Fig. 9.5: Complete Graphs of two, three, four and five vertices.
Since every vertex is joined with every other vertex through one edge, the
degrees of every vertex is n –1 in a complete graph G of n vertices. Also the
total number of edge in G is 2
)1n(n.
Components of a graph: Let G = (V, E) be a disconnected graph.
We define a relation ~ on the set of vertices as follows:
v ~ u there is a walk from v to u.
Then this relation ~ is an equivalence relation.
Let {Vi}i be the collection of all equivalence classes.
Fundamentals of Algorithms Unit 9
Sikkim Manipal University Page No. 168
Now, V = i
iV . Write Ei = {e E / an end point of e is in Vi} for
each i. Then (Vi, Ei) is a connected subgraph of G for every i . This
connected subgraph (Vi, Ei) of G is called a connected component (or
component) of G for every i .
The collection {(Vi, Ei)}i of subgraphs of G is the collection of all connected
components of G.
If G is a connected graph, then G contains only one connected component
and it is equal to G. Now, suppose that G is a disconnected graph. Consider
a vertex v in G. If each vertex of G is joined by some path to v, then the
graph is connected, a contradiction. So, there exists at least one vertex
which is not joined by any path to v. The vertex v and all the vertices of G
that are joined by some paths to v together with all the edges incident on
them form a component (G1, say). To find another component, take a vertex
u (from G) which is not in G1. The vertex u and all the vertices of G that are
joined by some paths to u together with all the edges incident on them form
a component (G2, say). Continue this procedure to find the components.
Since the graph is a finite graph, the procedure will stop at a finite stage. In
this way, we can find all the connected components of G. It is clear that, a
component itself is a graph.
Observations:
i. If G is a connected graph, then G is only the connected component of
G. So if G is a connected graph, then G contains only one connected
component.
ii. A disconnected graph G consists of two or more connected
components.
iii. Connected component of a graph G is a maximal connected subgraph
of G.
iv. Consider the graph given.
Fig 9.6: A graph
Fundamentals of Algorithms Unit 9
Sikkim Manipal University Page No. 169
This graph is a disconnected graph with two components.
Definition: A graph G is called bipartite if its vertex set V can be
decomposed into two disjoint subsets V1 and V2 such that every edge in G
joins a vertex in V1 with a vertex in V2. Thus, every tree is a bipartite graph.
A bipartite graph can have no self-loop. A set of parallel edges between a
pair of vertices can all be replaced with one edge without affecting
bipartiteness of a graph.
Definition: A graph G is said to be connected if there is at least one path
between every pair of vertex in G. Otherwise, G is disconnected. For
instance the graph in Fig. 9.4 is connected but in Fig. 9.7 is disconnected A
null graph of more than one vertex is disconnected.
Fig. 9.7: A Disconnected Graph with two Components
It is easy to see that a disconnected graph consists of two or more
connected graphs. Each of these connected sub graphs is called a
component. The graphic in the above figure consists of two components.
Another way of looking at the components is as follows: Consider a vertex vi
in a disconnected graph G. By definition, not all vertices of G are joined by
paths to vi, Vertex vi and all the vertices of G that have paths to vi, together
with all the edges incident on them, form a component, obviously, a
component itself is a graph.
Definition: An edge that has not acquired any takes is a bridge. (A bridge is
an edge whose removal disconnects the graph).
Fundamentals of Algorithms Unit 9
Sikkim Manipal University Page No. 170
Definition
A separable graph consists of two or more non-separable sub graphs. Each
of the largest non-separable sub graphs is called a block. The graphs in
Fig. 9.8 has five blocks (and three cut-vertices a, b, and c) ; each block is
shown enclosed by a broken line. Note that a non-separable connected
graph consists of just one block.
Fig. 9.8: Separate graph with three cut vertices and five blocks
Self Assessment Questions
1. A –––––– is also called a linear complex.
2. A –––––––– is defined as a finite alternating sequence of vertices and
edges.
3. A closed walk in which no vertex (except the initial and the final vertex)
appear more than once is called a ––––––––– .
4. A simple graph in which there exists an edge between every pair of
vertices is called a –––––––––––– .
5. A graph G is called ––––––––––– if its vertex set V can be decomposed
into two disjoint subsets.
6. An edge that has not acquired any takes is a –––––––– .
9.3 Trees
A tree is a connected graph without any circuits. The graph in
Fig. 9.9 A, for instance, is a tree. Trees with one, two, three and four
vertices are shown in Fig. 9.9 B. A graph must have at least one vertex, and
therefore so must a tree. Some authors allow the null tree, a tree without
any vertices.
It follows immediately from the definition that a tree has to be a simple
graph, that is, having neither a self-loop nor parallel edges (because they
both form circuits).
Fundamentals of Algorithms Unit 9
Sikkim Manipal University Page No. 171
Trees appear in numerous instances. The genealogy of a family is often
represented by means of tree (in fact the term tree comes from family tree)
A river with its tributaries and subtributaries can be represented by a tree.
The sorting of mail according to zip code and the punched cards is done
according to a tree (called decision tree or sorting tree).
Fig. 9.9 A: Tree
Fig. 9.9 B: Tree with one, two, three, and four vertical
Fundamentals of Algorithms Unit 9
Sikkim Manipal University Page No. 172
Fig. 9.9 C: Decision Tree
Fig. 9.9 C might represent the flow of mail, all the mail arrives at some local
office, vertex N. The most significant digit in the zip code is read at N, and
the mail is divided into 10 piles N1, N2, … N9 and N0 on the most significant
digit. Each pile is further divided into 10 piles according to second most
significant digit, and so on, till the mail is sub divided into 105 possible piles,
each representing a unique five-digit zip code.
Sub-graphs: A graph g is said to be a sub-graph of graphs G if all the
vertices and all the edges of g are in G, and each edge of g has the same
end vertices in g as in G. For instance, the graph in Fig. 9.10 B is a
sub-graph of one in Fig. 9.10 A. The concept of sub-graphs is akin to the
concept of subset in set theory. A sub-graph can be thought of as being
contained in another graph. The symbol from set theory, g h is used in
stating “g is a sub-graph of G”.
Fundamentals of Algorithms Unit 9
Sikkim Manipal University Page No. 173
(A) (B)
Fig. 9.10: Graph(A) and one of its Sub-graph(B)
The following observations can be made immediately :
1) Every graph is its own sub-graph
2) A sub-graph of a sub-graph of G is a sub-graph of G
3) A single vertex in a graph G is a sub-graph of G
4) A single edge in G, together with its end vertices, is also a sub-graph
of G.
Definition: Two sub-graphs g1 and g2 of graph G are said to be edge disjoint if
g1 and g2 do not have any edges in common, note that although edge disjoint
graphs do not have any edge in common, they may have vertices in common.
Sub-graphs that do not even have vertices in common are said to be vertex
disjoint.
Definition: A tree T is said to be a spanning tree of a connected graph G, if
T is a sub-graph of G and T contains all vertices of G. For instance, the
sub-graphs in heavy lines in Fig. 9.11 is a spanning tree of the graph shown
Fundamentals of Algorithms Unit 9
Sikkim Manipal University Page No. 174
Fig. 9.11: Spanning Tree
Since the vertices of G are barely hanging together in a spanning tree, it is a
sort of skeleton of the original graph G. This is why a spanning tree is the
largest (with maximum number of edges) tree among all trees in G. It is also
quite appropriate to call spanning tree a maximal tree sub-graph or maximal
tree of G.
It is to be noted that spanning tree is defined only for a connected graph,
because a tree is always connected, and in a disconnected graph of n
vertices we cannot find a connected sub-graphs with n vertices. Each
component of a disconnected graph, however, does have a spanning tree.
Thus, a disconnected graph with K component has a spanning forest
consisting of K spanning trees (A collection of trees is called a forest).
Self Assessment Questions
7. A ––––––––is a connected graph without any circuits.
8. A graph g is said to be a ––––––––of graphs G if all the vertices and all
the edges of g are in G
9. A ––––––––––vertex in a graph G is a sub-graph of G
10. A tree T is said to be a ––––––––– tree of a connected graph G if T is a
sub-graph of G and T contains all vertices of G
Fundamentals of Algorithms Unit 9
Sikkim Manipal University Page No. 175
9.3 Summary
In this unit, we studied
Some basic concepts of graph theory and some elementary results
based on these concepts.
That graphs can be used to represent almost any problem involving
discrete arrangements of objects, where concern is not with the internal
properties of these objects but with the relationship among them.
The concepts of trees with basic applications to its different properties.
9.4 Terminal Questions
1. In a ––––– an edge is not only incident on a vertex, but is also incident
out of a vertex and incident into a vertex.
2. –––––– are digraphs in which for every edge(a, b) there is also an edge
(a, b) there is also an edge (b, a).
3. A digraph is said to be –––––– if for every vertex vi the in-degree equals
the out-degree.
4. A closed walk in which no vertex appear more than once is called a
___________.
5. A graph G is said to be –––––– if there is at least one path between
every pair of vertices in G.
6. A connected graph without any circuit in ––––––––––––.
7. Give the definition of Walk.
9.5 Answers
Self Assessment Questions
1. graph
2. walk
3. circuit
4. complete graph
5. bipartite
6. bridge.
7. tree
8. sub-graph
9. single
10. spanning
Fundamentals of Algorithms Unit 9
Sikkim Manipal University Page No. 176
Terminal Questions
1. Graph
2. Symmetric digraphs
3. Balanced
4. Circuit
5. Complete
6. Spanning tree
7. A walk is defined as a finite alternating sequence of vertices and edges,
beginning and ending with vertices, such that edge is incident with the
vertices preceding and following it. No edge appears more than once in
a walk. A vertex, however, may appear more than once. A walk is also
referred to as an edge Train or a chain. The set of vertices and edges
constituting a given walk in a graph G is clearly a sub graph of G
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 177
Unit 10 Trees and Algorithms Structure
10.1 Introduction
Objectives
10.2 Characterization of Trees
10.3 Rooted Trees and Applications
10.4 Spanning Trees
10.5 Algorithms for Spanning Trees
Kruskal Algorithm
Prim‟s Algorithm
10.6 Summary
10.7 Terminal Questions
10.8 Answers
10.1 Introduction
After studying the concept of graphs in the previous unit, now, you will study
the concept of trees. Trees are extensively used as models in area like
computer science, chemistry and in search procedures. These are also
useful in design of wide range of algorithms. In this unit, we will provide
some characterizations of trees with suitable illustrations. We will also
introduce a special type of trees namely rooted trees and binary trees. We
will give few applications of these trees. The spanning trees will be
introduced and the algorithms for spanning trees will also be obtained.
Objectives
After studying this unit, you should be able to:
describe the connected graphs without circuits (called trees) with their
properties
characterize a tree from a connected graph
apply the concept of trees in real life situations
analyze and write algorithms for spanning trees.
10.2 Characterization of Trees
The concept of a „tree‟ plays a vital role in the theory of graphs. First, we
introduce the concept of „tree‟, study its properties and some of its
applications. Later, we introduce the concept of „spanning tree‟, and study
the relationships among circuits and trees.
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 178
Definition: A connected graph without circuits is called a tree. A collection
of trees is called a forest.
Example:
Trees with one, two, three and four vertices are given in the Fig. 10.1
Fig. 10.1
Observations
i) A tree contains at least one vertex.
ii) A tree without any edge is referred to as a null tree.
iii) The trees considered are finite and a tree is always a simple graph.
Some natural examples
i) The list of the ancestors of a family may be represented by a tree. This
tree referred to as a family tree.
ii) A river with its tributaries and sub tributaries may be represented by a
tree. This tree is referred to as a river tree.
iii) The sorting of mail according to zip code are done according to a tree.
This tree is called decision tree (or) sorting tree.
Theorem: In a tree T, there is one and only one path between every pair of
vertices.
Proof: Suppose T is a tree.
By definition, T is a connected graph and contains no circuits.
Since T is connected, there exists at least one path between every pair of
vertices in T. Suppose that between two vertices a and b of T, there are two
distinct paths. Now, the union of these two paths will contain a circuit in T, a
contradiction (since T contains no circuits).
This shows that there exists one and only one path between a given pair of
vertices in T.
Theorem: If there is one and only one path between every pair of vertices in
G, then G is a tree.
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 179
Proof: Assume that there is one and only one path between every pair of
vertices in a graph G. Clearly G is connected.
If possible, suppose that G contains a circuit. Then, there is at least one pair
of vertices a, b such that there are two distinct paths between a and b,
which is a contradiction to our assumption. So G contains no circuits. Thus
G is a tree.
Theorem: A tree G with „n‟ vertices has (n - 1) edges.
Proof: (Use induction on n):
Step (i): If n = 1, then G contains only one vertex and no edge.
So the number of edges in G is n -1 = 1 - 1 = 0.
Step (ii): Induction hypothesis: The statement is true for all trees with less
than „n‟ vertices.
Step (iii): Let us consider a tree with „n‟ vertices.
Let „ek‟ be any edge in T whose end vertices are vi and vj.
Since T is a tree, there is no other path between vi and vj.
So by removing ek from T, we get a disconnected graph.
Also, T- ek consists of exactly two components(say T1 and T2).
Since T is a tree, there were no circuits in T and so there were no circuits in
T1 and T2. Therefore, T1 and T2 are also trees.
It is clear that |V(T1)| + |V(T2)| = |V(T)| where V(T) denotes the set of
vertices in T.
Also, |V(T1)| and |V(T2)| are less than n.
Therefore, by the induction hypothesis, we have
|E(T1)| = |V(T1)| - 1 and |E(T2)| = |V(T2)| - 1.
Now |E(T)| - 1 = |E(T1)| + |E(T2)| = |V(T1)| - 1 + |V(T2)| - 1
Thus |E(T)| = |V(T1)| + |V(T2)| - 1 = |V(T)| - 1 = n-1.
Theorem: Any connected graph with „n‟ vertices and n -1 edges is a tree.
Proof: Let „G‟ be a connected graph with n vertices and n - 1 edges.
It suffices to show that G contains no circuits.
If possible, suppose that G contains a circuit. Let „e‟ be an edge in that
circuit.
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 180
Since ‘e‟ in a circuit, we have that G - e is still connected.
Now G - e is connected with „n‟ vertices, and so |G - e| n -1 edges, a
contradiction (to the fact that G - e contain only (n-2) edges, and so G
contains no circuits. Therefore, G is a tree.
Definition: A connected graph is said to be minimally connected if the
removal of any one edge from the graph provides a disconnected graph.
Example:
i) Graph given in Fig. 10.2 A is not minimally connected.
ii) Graph given in Fig10.2 B is minimally connected.
iii) Every circuit is not minimally connected.
iv) Every tree is minimally connected.
Fig. 10.2 A Fig. 10.2 B
Problem: A graph G is a tree if and only if it is minimally connected.
Solution: Part (i): Assume that G is a tree. If G is not minimally connected,
then there exists an edge „e‟ such that G - e is connected. That is, e is in
some circuit, which implies G is not a tree, a contradiction. Hence G
minimally connected.
Part (ii): Suppose that G is minimally connected.
If G contains a circuit, then by removing one of the edges in the circuit, we
get a connected graph, a contradiction (to the supposition). This shows that
G contains no circuits. Thus G is a tree.
Observation: To join „n‟ given distinct points, the minimum number of line
segments needed is n -1.
Problem: If a graph G contains n vertices, n -1 edges and no circuits, then
G is a connected graph.
Proof: Let G be a graph with „n‟ vertices, n – 1 edges and contains no circuits.
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 181
Suppose that (in a contrary way) G is disconnected.
G consists of two or more circuitless components (say, g1, g2, , gk) (here k 2).
Choose a vertex vi in gi, for 1 i k.
Add new edges e1, e2, …, ek-1 where ei = 1iivv to get a new graph G*.
It is clear that G* contains no circuits and connected, and so G* is a tree.
Now G* contains n vertices and
(n - 1) + (k - 1) = (n + k - 2) n edges, a contradiction (since a tree
contains (n - 1) edges). This shows that G is connected.
Characterization for Tree
For a given graph G, the following conditions are equivalent:
i) G is connected and is circuit less,
ii) G is connected and has n -1 edges,
iii) G is circuitless and has n -1 edges,
iv) There is exactly one path between every pair of vertices in G,
v) G is a minimally connected graph,
vi) G is a tree.
Problem: If G is a tree (with minimum two vertices), then there exists at
least two pendant vertices.
Proof: Let G be a tree with |V| 2.
Let v0e1v1e2v2e3 . . . vn-1envn be a longest path in G (since G is finite graph,
it is possible to find a longest path).
Now we show that d(v0) = 1 = d(vn).
If d(v0) > 1, then there exists at least one edge e with end point v0 such
that e e1. If e {e1, e2, … en}, then e = ei for some i 1.
So either vi-1 = v0 or vi = v0 v0 repeated in the path, a contradiction.
Therefore, e {e1, e2, e3 , … en} and so e, e1, e2, e3 , … en is a path of length
n + 1, a contradiction.
Thus, d(v0) = 1. In a similar way, we can show that d(vn) = 1. Hence v0, vn
are two pendant vertices.
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 182
Definition: Let G be a connected graph. The distance between two vertices
v and u is denoted by d(v , u) and is defined as the length of the shortest
path [that is, the number of edges in the shortest path] between v and u. In
a connected graph, we can find the distance between any two given
vertices.
Example:
i) Consider the connected graph given in Fig. 10.3A. Here some of the
paths between v1 and v2 are (a, e), (a, b, f), (b, c, e), (b, f), (b, g, h), (b,
g, i, j), (b, g, i, k). Here there are two shortest paths (a, e) and (b, f)
each of length 2. Hence, d(v1, v2 ) = 2.
Fig. 10.3 A
ii) Consider the tree given in the Fig. 10.3 B. Here d(a, b) = 1, d(a, c) =
2, d(a, d) = 2, d(b, d) = 1.
Fig. 10.3 B
Self Assessment Questions
1. Any connected graph with „n‟ vertices and n -1 edges is a –––––––– .
2. A graph G is a tree if and only if it is –––––––––––– .
10.3 Rooted Trees and Applications
Rooted trees are extensively used in the computer search methods, binary
identification problems, and variable length binary codes.
j
b
d
c
v2
v1 i
h
k
g
e
a
f
a
b c d
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 183
Definition: A tree in which one vertex (called the root) is distinguished from
all the other vertices, is called a rooted tree. In a rooted tree, the root is
generally marked in a small triangle (or small circle).
Example:
Distinct rooted trees with four vertices, were given in Fig. 10.4
Fig. 10.4
Generally, the term „tree‟ means trees without any root. However, they are
sometimes called free trees (or) non-rooted trees. A variety of rooted trees
(called the Binary rooted trees) is of particular interest.
Definition: A tree in which there is exactly one vertex of degree 2, and all
other remaining vertices are of degree one or three, is called a binary tree.
Fig-.10.5
v3
v2
v4
v6
v8
v5
v11
v7
v1
v10
v13
v9
v12
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 184
i) The Fig. 10.5 represents a binary tree (since the only vertex „v1‟ is of
degree 2, and all other vertices are of degree either 1 or 3).
ii) The vertex of degree 2 (that is, v1) is distinct from all other vertices, this
vertex v1 is the root.
iii) In a binary tree, the vertex with degree 2 serves as a root. So, every
binary tree is a rooted tree.
Properties of Binary trees
Property (i): The number of vertices n, in a binary tree is always odd.
Property (ii): The number of pendent vertices is 2
1n.
Property (iii): Number of vertices of degree 3 is
= n - p - 1 = n - (2
1n) - 1 =
2
3n.
Example:
In the graph given in Fig. 10.5, we have that n = 13, p =2
1n =
2
113 =
2
14=7. Therefore, number of vertices of degree 3 is
2
3n=
2
313 = 5.
Definition:
A non-pendent vertex in a tree is called an internal vertex.
Observation:
i) The number of internal vertices in a Binary tree is
2
1n = (p - 1) where p = the number of pendent vertices.
ii) In the binary tree given in Fig. 10.5, the internal vertices are v1, v3, v4,
v5, v6, v9. These are 6 (=7 - 1 = p - 1) in number.
Definition: Let v be a vertex in a binary tree. Then v is said to be at level I if
v is at a distance of I from the root.
Example:
A 13-vertex, 4-level binary tree is given in Fig. 10.6
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 185
Fig. 10.6
Here, the number of vertices at levels 0, 1, 2, 3, 4 are 1, 2, 2, 4 and
4 respectively.
Definition: The sum of path lengths from the root to all pendent vertices is
called the path length (or) external path length of a tree.
Example:
i) The path length of the binary tree given in Fig. 10.6 is:
1 + 3 + 3 + 4 + 4 + 4+ 4 = 23.
ii) In the Figures 10.7 A and B, there are two 11-vertex binary trees.
Fig. 10.7 A
The path length of graph (fig. 10.7 A): 2 + 2 + 3 + 3 + 3 + 3 = 16.
level -1
level - 0
level -2
level -3
level 1
level 4
level 0
level 2
level 3
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 186
Fig. 10.7 B
The path length of graph (fig. 10.7 B): 1 + 2 + 3 + 4 + 5 + 5 = 20.
Search procedures:
Each vertex of a binary tree represents a test with two possible outcomes.
We start at the root. The outcome of the test at the root sends us to one of
the two vertices at the next level, where further tests are made and so on.
Reaching a specified pendent vertex (that vertex which represents the goal
of the search), terminates the search.
For such search procedures, it is often important to construct a binary tree in
which, for a given number of vertices n, the vertex frothiest from the root in
as close to the root as possible.
i) There can be only one vertex (the root) at level 0. Number of vertices
at level one is at most 2. Number of vertices at level two is at most 22
and so on. So the maximum number of vertices possible in a k -level
binary tree is 20 + 21 + 22 + . . . + 2k.
So n 20 + 21 + 22 + . . . + 2k
ii) The maximum number among the levels of the vertices in a binary tree
is called height of the tree. So height = max { level of a vertex v /
v V}.
This height is denoted by l max .
iii) To construct a binary tree for a given n such that the farthest vertex
is as far as possible from the root, we must have exactly two vertices at
each level, except at the 0 level. So max l max = 2
1n.
level 0
level 5
level 1
level 2
level 4
level 3
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 187
An Application: (Coke Machine Problem): The machine is said to have a
sequence of tests (for example, it should be capable of identifying the coin
that is put into the machine). We suppose that five rupees coin, two rupees
coin, one rupee coin and fifty paise coin can go through the slot. So the
machine can identify only these four coins. Every coin put in, is to be tested
by the machine. Each test got the effect of partitioning the coins into two
complementary sets. [Suppose a coin is put into the machine. It should test
whether the coin is “five rupee coin”. If it is not a five rupees coin, then it
should test whether it is a two rupees coin and so on]. We suppose the time
taken for each test is –
Test Pattern-1: One type of testing pattern was shown in Graph-(i), given
in Fig. 10.8 A.
Suppose the statistical data tells that
w1 = probability of putting a Rs 5 coin = 0.5
w2 = probability of putting a Rs 2 coin = 0.2
w3 = probability of putting a Rs 1 coin = 0.2
w4 = probability of putting a Rs 0.5 coin = 0.1
Now
)v(lw ii = )v(lw 11 + )v(lw 22 + )v(lw 33 + )v(lw 44
= (0.5) (1) + (0.2) (2) + (0.2) (3) + (0.1) (4) = 1.9
Fig. 10.8 A
not Rs 0.5 coin
Rs 0.5 coin
Rs 1 coin
Rs 5 coin
v3 0.2
v1
v2
v4 0.1
0.2
0.5
not Rs 1 coin
not Rs 2 coin
not Rs 5 coin
Rs 2 coin
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 188
So expected time to be taken by the machine for testing one coin is 1.9t.
Thus, if the machine follows (for its testing pattern) the binary tree given in
Graph-(a), then the expected time for testing one coin is equal to 1.9t.
Test Pattern-2: Another type of testing pattern was given in the
Fig. 10.8 B.
)v(lw ii = )v(lw 11 + )v(lw 22 + )v(lw 33 + )v(lw 44
= (0.5) (2) + (0.2) (2) + (0.2) (2) + (0.1) (2) = 2.0
Fig. 10.8 B
So, here, the expected time to be taken by the machine for testing one coin
is 2t.
Thus, if the machine follows (for its testing pattern) the binary tree given in
graph 10.8 B then the expected time for testing one coin is 2t.
Definition: A graph G in which every vertex is assigned a unique name or
label (that is, no two vertices have the same label) is called a labeled graph.
Otherwise is said to be an unlabeled graph.
Example:
If n = 4, then there are 16 trees, shown below.
Take the vertex set V = {A, B, C, D}.
v1 v2 v3 v4
0.1 0.2 0.2 0.5
Rs 5 or Rs 2
Rs 5 Rs 2
Rs 1 Rs 0.50
Rs 1 or Rs 0.50
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 189
The following are the 16 trees of four labeled vertices
B
C
A
D
A
D C
B A
C D
B
C
B A
D
A B
D C D C
B A
A B
C D
A B
C D
A
C
B
D
B
D
A
C
A B
C D
A B
C D
A B
C D
A B
C D
A B
C D
A B
C D
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 190
Observations:
i) When we count the number of different graphs, the distinction between
labeled and unlabeled graphs is important.
ii) Consider the graphs in above example the 5th, 6th, 7th and 8th, are
counted as four different trees (even through they are isomorphic),
because they are labeled. If there is no distinction between A, B, C,
D, then these four trees counted as one.
iii) A careful inspection of the 16 graphs will reveal that the number of
unlabeled trees with four vertices (no distinction made between A, B,
C, D), is two. These two graphs were.
Cayley’s Theorem:
The number of labeled trees with n vertices (n 2) is n(n-2) .
Example:
Part-(i): Suppose the given tree is T1 (Fig. 10.9 A). a1 = the pendent vertex
with smallest label. So, a1 is the vertex 2. Now b1 = 1. After removing a1
and the edge (a1, b1), in the remaining graph, a2 = the pendent vertex with
smallest index = 4.
Fig. 10.9 A
a1
1
2
6
5
3
7
9
4
8
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 191
Now b2 = 1. a3 = 1, b3 = 3, a4 = 3, b4 = 5, a5 = 6, b5 = 5, a6 = 7, b6 = 5, a7 = 5,
b7 = 9.
Therefore, we have the sequence (b1, b2, b3, b4, b5, b6, b7)=(1, 1, 3, 5, 5, 5, 9).
Part (ii): (converse of part (i)): we have to construct a tree with n = 9 vertices.
Consider 1, 2, 3, 4, 5, 6, 7, 8, 9 . . . . . . . . . (i)
Given (n - 2)-tuple is (1, 1, 3, 5, 5, 5, 9) . . . . . . . (ii)
Fig. 10.9 B
Observe the sequence in (ii). First join 1 and 3 , 3 and 5, 5 and 9.
Then, we get the graph given in Fig. 10.9 B. Now the least number in (i)
which is not in (ii), is 2. So we join 2 and 1. Next least in (i) which is not in (ii)
is 4. So we join 4 and 1. Cancel 3 in (i) and 3 in (ii). Also cancel 5 in (i) and 5
in (ii) (in (ii), canceling of only one 5 is allowed). The next least which is not
in (ii) is 6. So we join 5 and 6. The next least which is not in (ii) is 7. So we
join 7 and 5. The next least which is not in (ii) is 8. The remaining number
available in (ii) is 9. So, we join 8 and 9.
Fig. 10.9 C
1 3
5
9
8
1
2
6
5
3
7
9
4
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 192
Now, we will study the tree as a subgraph of another graph. A given graph
may have numerous subgraphs. If e is the number of edges in G, then
there are 2e distinct subgraphs possible. Obviously, some of these
subgraphs will be trees. Out of these trees, we are particularly interested in
certain type of trees, called spanning trees.
Self Assessment Questions
3. A tree in which one vertex (called the root) is distinguished from all the
other vertices, is called a ––––––––––––– .
4. The number of pendent vertices is –––––––––.
10.4 Spanning Trees
Definition: A tree T is said to be a spanning tree of a connected graph G
if T is a subgraph of G and T contains all the vertices of G.
Example:
Consider the graph G given in Fig. 10.10 A. Graph T (given in Fig. 10.10B)
is a spanning tree of G.
Fig. 10.10 A: Graph-G Fig. 10.10 B: Graph-T
v3
v1
v4 v2
v5
v6
v7
b2
b3
b1
b6
b4
c5
c6
c8
c7
c1
b5
c2
c3
c4
v7
v1
v2
v3
v4
v5
v6
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 193
Observations:
i) Spanning trees are the largest (with the maximum number of edges)
trees among all trees in G. Spanning tree is also called a maximal tree
subgraph or maximal tree of G.
ii) Spanning is defined only for a connected graph (since a tree is always
connected).
iii) Each component of a disconnected graph, does have a spanning tree.
Thus, a disconnected graph with k components contains a spanning
forest consisting of k spanning trees.
Theorem: Every connected graph has at least one spanning tree.
Definition:
i) An edge in a spanning tree T is called a branch of T.
ii) An edge of G that is not in a given spanning tree T is called a
chord. In electrical engineering, chord some times referred to as tie or a
link.
iii) Let T be any spanning tree of a connected graph G, and T1 is the
complement of T in G. Then, each edge in T is called a branch (with
respect to T), and the set of all edges in T is called the branch set.
Each edge in T1 is called a chord (with respect to T), and the set of
edges in T1 is called the chord set (or) Tie set. T1 is called as the
cotree. We may write T instead of T1.
Observations:
i) Branches and chords are defined only with respect to a given spanning
tree.
ii) An edge that is a branch with respect to one spanning tree T1 (of G)
may be a chord with respect to another spanning tree T2.
Theorem: With respect to any of its spanning trees, a connected graph of
„n‟ vertices and „e‟ edges has „n-1‟ tree branches and e - n + 1 chords.
Proof: Let G be any connected graph on „n‟ vertices and e edges. Let T be
any spanning tree in G. Since every spanning tree of G contains all vertices
of G, we have that |V(T)| = n and so |E(T)| = |V(T)| - 1 = n - 1. Since
every edge of a spanning tree T is called a branch of T, we have that G
contains n - 1 branches. Since the number of edges in G is e , we have
that the number of chords of T is e - (n - 1) = e - n + 1.
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 194
Example:
There is a form consisting of six walled plots of land as shown in the fig and
these plots are full of water. We find the minimum number of walls are to be
broken so that all the water can be drained out.
Consider the wall joints as vertices, and walls as edges. Then, we can
consider it as a graph. In this graph, the number of vertices is n = 10, and
the number of edges is e = 15.
Fig. 10.11
If there exists a circuit, then the water inside the circuit cannot be drained
out. So, we have to remove minimum number of edges so that the graph do
not contain circuits. To have this, we should have a spanning tree with
(n - 1) edges. Hence, we have to break e - (n - 1) = e - n + 1 = 15 - 10
+ 1 = 6 edges (walls) so that all the water can be drained out.
If we add an edge between any two vertices of a tree, then a circuit is
created. This is because, there already exists one path between any two
vertices of a tree, adding an edge in between, creates an additional path,
and hence a circuit. The concept of a fundamental circuit has an enormous
significance in electrical network analysis.
Definition:
Let T be any spanning tree of a connected graph G. Adding any one chord
to T will create exactly one circuit. Such a circuit formed by adding a chord
to a spanning tree, is called a fundamental circuit.
Example:
Consider the graph G (given in Fig. 10.12A), and its spanning tree T (given
in Fig. 10.12 B) of G. Now, if we add the chord c1 to T, we get a circuit
„b1 b2 b3 b5 c1‟ which is called as fundamental circuit (given in Fig. 10.12C).
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 195
Fig. 10.12 A Fig. 10.12 B Fig. 10.13 C
Graph-G Spanning tree-T Fundamental circuit-F
Note: A connected graph G is a tree if and only if adding an edge
between any two vertices in G creates exactly one circuit.
Note: For a tree G is a tree, by adding a new edge between any two
vertices in G creates exactly one fundamental circuit.
Self Assessment Question
5. Every connected graph has at least one –––––––––––––––.
10.5 Algorithms for Spanning Trees
Definition:
i) A graph G is said to be a weighted graph if all the edges e of G were
assigned by a corresponding real number (e) (read as the weight of e).
ii) Let T be any spanning tree of a connected graph G. then the weight of
T, that is, (T) is defined as the sum of weights of all branches in T.
iii) A spanning tree with the smallest weight in a weighted graph is called a
shortest spanning tree (or) minimal spanning tree (or) shortest-distance
spanning tree.
Observation:
i) Different spanning trees of G may have different weights. Among all the
spanning trees of G, one with the smallest weight is of practical
significance.
c6
c5
c7
c8
c4
c3
Fig. 10.10 B Graph-T
c1
b2
b1
b4
b5 b3 b6
c2 b1
b2
b3
b4
b6 b5
c1 b1
b2
b3
b4
b6 b5
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 196
ii) Let G be a graph on n vertices in which every edge has a unit weight .
Then all the spanning trees have the same weight of (n - 1) units.
Applications:
i) Suppose that we are to connect n cities v1, v2, …, vn through a network
of roads.
ii) The cost Cij of building a direct road between vi and vj is given for
pairs of cities vi and vj where roads can be built. (Note that there may
be pairs of cities between which no direct road can be built).
iii) Now the problem is: To find the least expensive network that connects
all „n‟ cities together. That is, to find a shortest spanning tree in a
connected weighted graph of „n‟ vertices.
iv) Here the connected network must be a tree. Otherwise, we can always
remove some edges and get a connected graph with smaller weight.
Thus „the problem of connecting „n‟ cities with a least expensive
network is the problem of finding a shortest spanning tree in a
connected weighted graph of „n‟ vertices.
10.5.1 Kruskal Algorithm:
This algorithm is used in finding shortest spanning tree
Step-(i): List all the edges of G in order of non-decreasing weight. Now we
select an edge e1 of G such that (e1) is as small as possible and „e1‟ is not
a loop.
Step-(ii): Select next smallest edge from the set of all remaining edges of
G such that the selected edge do not form a circuit with the edges that
have already been chosen.
Step-(iii): We continue this process of taking smallest edges among those
not already chosen, provided no circuit is formed with those, that have been
chosen already.
[If edges e1, e2, …, ei have been chosen, then chose ei+1 from E -
{e1, e2, …, ei} in such way that graph with {e1, e2, …, ei+1} is acyclic and
(ei+1) is as small as possible].
Step-(iv): If a graph G has „n‟ vertices, then we will stop this process
after choosing n - 1 edges. These edges form a subgraph T, which is not
cyclic. (Thus T is a shortest spanning tree of G).
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 197
Example: Consider the connected weighted graph G. Listing:
)1(AB , )1(AD , )2(BC , )2(CD , )3(AE , )3(FE , )3(AF , )4(EB ,
)4(AC , )4(ED , )5(FD .
Step-(i): Consider a spanning subgraph of G without edges.
Select the edge with minimum weight. For this graph the edge AB is of
minimum weight.
Add this edge e1 = AB to the spanning null graph. Then, we get the
subgraph-G1.
Step(ii): Select the edge e2 such that G1 + e2 contains no circuits and e2
got minimum weight among such edges. For this graph e2 = AD. Now, add
e2 = AD to G1. Then we get the subgraph-G2.
Step-(iii): We continue this process. For this example, we add e3 = BC to
G2 to get the subgraph-G3.
Graph-G Subgraph-G1
Next, we add e4 = AE to G3 to get the subgraph-G4. Next, we add
e5 = EF to get the subgraph-G5. If we add any edge to G5, we get a circuit.
So, the process is to be stopped here.
C
2
5 1
D
4
A
B
E
F
1
2 3 4
3
3
4
A
B 1 F
E
D
C
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 198
Subgraph-G2 Subgraph-G3
Subgraph-G4 Graph G5 or T
The final graph G5 is a subgraph of G and contains no cycles. Thus G5 is
a shortest spanning tree of G. Write T = G5. Now (T) = sum of the weights
of the branches in T = 1 + 2 + 1 + 3 + 3 = 10
10.5.2 Prim's Algorithm:
This algorithm is used to find the shortest spanning tree
Step- (i): Choose any vertex v1 in G.
Step-(ii): Choose an edge e1 = v1v2 of G such that v1 v2 and „e1‟ has
the smallest weight among the edges of G incident with v1. [For
convenience, we can form a table indicating the weights].
Step-(iii): If the edges e1, e2, …, ei have been already chosen involving
end vertices v1, v2, …, vi+1 , choose an edge „ei+1‟, where ei+1 = vjvk with
vj { v1, v2, …,vi+1}, and vk {v1, …, vi+1} such that ei+1 has the smallest
weight among the edges of G with precisely one end in {v1, v2, …, vi+1}.
A
B
E
1 F
1 2
D C
3
3
F
A
B
E
1
1 2
D C
3
F
E
A
B
1
1 2
D C C
A
B
1
1
D
F
E
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 199
Note that after adding ei+1, the graph should be a cyclic.
Step-(iv): The process will stop after choosing the n-1 edges (otherwise,
repeat the step (iii)).
Illustration: Consider the connected weighted graph G. Let us start with C.
Step-(i): Take the spanning null subgraph G0 of G.
Let us choose v1 = A. The edge AD is incident on v1 and has the smallest
weight among the edges incident on v1 = A. Write G1 = (G0 + AD)
Step-(ii): In this step we select DC and write G2 = (G1 + DC).
Step-(iii): We select AB and write G3 = (G2 + AB).
Step-(iv): We select AF and write G4 = (G3 + AF).
Step-(v): We select FE and write G5 = (G4 + FE).
Step-(vi): Since G5 is a spanning tree, the process stops here.
Write T = G5.
5 1
D
4 2
A
C
B
E
F
1
2 3 4
3
3
4
Given graph G Graph-G1
D
A
F
E
B
C
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 200
Graph-G2 Graph-G3 Graph-G4 Graph-G5 = T
Hence, the shortest spanning tree of G is G5 = T.
Now, (T) = 1 + 1 + 2 + 3 + 3 = 10.
Self Assessment Questions
6. Draw trees with 5 vertices.
7. Find all spanning trees of the graph.
8. Find all spanning trees of the following grpah.
9. The complete graph on „n‟ vertices Kn: has 2nn different spanning
trees. Give all the spanning trees of K4.
A A
B
E
1
F
1
2
D
C
3
3
B
1
F
1
2
D
C
3
E
A
B
1
1
2
D
C
F
E
A
1
1
D
C
F
E
B
o o o
o
o
o o o
o o
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 201
10. Find a spanning tree for each of the following graphs
11. Which connected simple graphs have exactly one spanning tree ?
12. Use Prim‟s Algorithm to find a minimal spanning tree for the following
weighted graph.
10.6 Summary
This unit dealt with a special type of graphs namely trees, which are
extensively used in search procedures and design of computer algorithms.
You got an idea to find the different spanning trees from a given connected
graph.
Different characterizations and properties of trees were given. Also, some
applications of binary trees were discussed.
This unit highlights that spanning trees play a vital role in multicasting over
internet protocol net works.
(i) o
o
o
o o
o o
o o
(ii)
o
o
o
o
(iii)
o
o
o o
o
(iv)
o
o
o o o
o
o o
o
8
A
3
6
4 7
2
4
6
8 6
C
B E
D
o
o
o
o
o
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 202
10.7 Terminal Questions
1. Draw two different binary trees with five vertices having maximum
number of leaves.
2. Draw a graph with the given specification. In case, if not possible, then
explain why no such graphs exist.
i) full binary tree, five internal vertices
ii) full binary tree, five internal vertices, seven terminal vertices
iii) full binary tree, twelve vertices
iv) full binary tree, nine vertices.
3. If T1 and T2 are two spanning trees in a graph, then show that
i) The number of edges in T1 not in T2 is equal to the number of
edges in T2 not in T1. (ii) d(T1, T2) = d(T2, T1).
4. Let N(g) = the number of edges in a graph g. Then N(Ti Tj) = the
number of edges in Ti Tj , and d(Ti , Tj) = The distance between Ti
and Tj. Show that d(T1, T2) = 2
1N(T1 T2).
5. Consider the table of airline distances in miles between six of largest cities
in the world: London, Mexico, New York, Paris, Peking and Tokyo.
L MC NY Pa Pe T
L - 5558 3469 214 5074 5959
MC - 2090 5725 7753 7035
NY - 3636 6844 6757
Pa - 5120 6053
Pe - 1307
T -
Write the spanning tree for the above
10.8 Answers
Self Assessment Questions
1. Tree
2. Minimally connected
3. Rooted tree
4. 2
1n
5. Spanning tree
L
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 203
6.
7. The given graph has four vertices and so each spanning tree must
have 4 – 1 = 3 edges. They are
8.
9. K4 has 244 = 16 different spanning trees. Each having three edges.
10. 100.
o
o
o
o
o
o o
o
o
o
o
o
o
o
o
o o
o
o
o
o o o o o o
o
o o
o
o
o
o
o o o o o
o o
o o o o o o
o
o o
o
o
o o o o o
o o o o o
o o o
o
(i) o
o o
o
o
o
o
o
o
(ii) o
o o o
(iii) o
o
o
o
o
(iv)
o
o o
o
o
o o
o
o
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 204
11. Tree.
12. Choose edges AE, AC, DC, AB.
Terminal Questions
1.
2. i.
ii. Not possible, since any full binary tree with five internal vertices have
six terminal vertices, but not seven.
iii. No, a full binary tree has 2k + 1 vertices where „k‟ is the number of
internal vertices but 2k + 1 is odd.
iv.
3. i) Let T1, T2 be two spanning trees. Now |E(T1)| = n - 1 = |E(T2)|.
Suppose T1 contains m edges which are not in T2 . Now the number
of edges in T2 not in T1 is
(n - 1) - [( n - 1) - m] = m. follows from (i).
b
c
a
e
f g
j k
o
o
o o
o
o
o
d
h i o o
o o
o
o
o
o
o
o
o
o o
o
o o
o o o
o
o o
o
Fundamentals of Algorithms Unit 10
Sikkim Manipal University Page No: 205
4. N(T1 T2) = number of edges in T1 T2 = (number of edges in T1 but
not in T2) + (number of edges in T1 but not in T2) = d(T1, T2) + d(T2, T1)
= 2 d(T1, T2) d(T1, T2) = 2
1N(T1 T2)
5.
Fig. A Fig.B: Shortest spanning tree
6757
5074
1307
T
Pe
Pa
214
2090
3469
5074
1307
M
C
NY
T
L
Pa
Pe 5725
6053
7753
3469
5120 3636
5558
2090
595
9
MC
NY
7035
6844
214
Fundamentals of Algorithms Unit 11
Sikkim Manipal University Page No. 206
Unit 11 Traversability Structure
11.1 Introduction
Objectives
11.2 Eulerian Graphs
11.3 Hamiltonian Circuits and Paths
11.4 Travelling Salesman problem
11.5 Summary
11.6 Terminal Questions
11.7 Answers
11.1 Introduction
In the previous unit we, studied Trees and their algorithms. Now, in this unit,
we will discuss Euler graphs, Hamiltonian paths and Hamiltonian circuits in
a graph G with special properties. A given graph G can be characterized
and studied in terms of the presence or absence of these sub-graphs. Many
physical problems can be represented by graphs and solved by observing
the relevant properties of the corresponding graphs.
Objectives
After studying this unit, you should be able to:
identify a Euler graph
use the characteristics of Euler graphs
recognize Hamiltonian paths and circuits in Graphs
use Travelling Sales man problem in Hamiltonian circuits .
11.2 Eulerian Graphs
Graph theory was born in 1736 with Euler’s famous paper. In the same
paper, Euler posed (and then solved) a more general problem.
We state the following results before giving the formal definitions of Euler’s
graphs.
A graph G is disconnected its vertex set V can be partitioned into two
non-empty disjoint subsets V1 and V2 such that there exists no edge in
G, whose one end-vertex is in V1 and the other end vertex is in V2.
Fundamentals of Algorithms Unit 11
Sikkim Manipal University Page No. 207
If a graph (either connected or disconnected) has exactly two vertices of
odd degree, then, there exists a path joining these two vertices.
A simple graph with ‘n’ vertices and ‘k’ components can have at most (n
- k) (n -k + 1)/2 edges. [or the maximum number of edges in a simple
graph with ‘n’ vertices and ‘k’ components is (n - k)(n - k + 1)/2].
Definition: Let G be a graph. A closed walk running through every edge of
the graph G exactly once is called an Euler line. A graph ‘G’ that contains
Euler line is called an Euler graph.
Example: The graphs given in the Fig. 11.1A A and fig 11.1 B are Euler
graphs.
Fig. 11.1 A Fig. 11.1 B
i. Consider Fig-A, 123456789(10)(11)(12)13579(11)1 is an Euler line.
ii. Consider Fig-B, 23456789(10)12(11)357(11)8(10)2 is an Euler line.
A Euler graph may contain isolated vertices. If G is Euler graph and it
contains no isolated vertices, then it is connected. Hereafter, we consider
only those Euler graphs that do not contain isolated vertices. So, the Euler
graphs that we consider are connected.
Theorem: A given connected graph G is a Euler graph all the vertices of
G are of even degree.
Proof: Suppose G is a Euler graph. Then G contains a Euler line. So, there
exists a closed walk running through all the edges of G exactly once. Let v
V be a vertex of G. Now, in tracing the walk, it goes through two incident
edges on v with one entered v and the other exited.
8
1
2
3
5
7 9 10
11
12
4
6
10
4
8 6 9
3
5
1 2
11
7
Fundamentals of Algorithms Unit 11
Sikkim Manipal University Page No. 208
Fig. 11.2
This is true not only for all the intermediate vertices of the walk, but also true
for the terminal vertex because we had the exit and entered at the same
vertex at the beginning and ending of the walk. Therefore, if v occurs k times
in the Euler line, then d(v) = 2k. Thus, if G is a Euler graph, then the
degree of each vertex is even.
Converse: Suppose all the vertices of G are of even degree. Now, to show
that G is a Euler graph, we have to construct a closed walk starting at an
arbitrary vertex v and running through all the edges of G, exactly once. To
find a closed walk, let us start from the vertex v. Since every vertex is of
even degree, we can exist from every vertex we entered, the tracing cannot
stop at any vertex but at ‘v’. Since ‘v’ is also of even degree, we shall
eventually reach ‘v’ when the tracing comes to an end. If this closed walk
(h, say) includes all the edges of G, then G is an Euler graph. Suppose the
closed walk h does not include all the edges. Then, the remaining edges
form a subgraph h1 of G. Since both G and h have all their vertices of even
degree, the degrees of the vertices of h1 are also even. Moreover, h1 must
touch h at least one vertex ‘a’ because G is connected. Starting from ‘a’ we
can again construct a new walk in graph h1. Since all the vertices of h1 are
of even degree, and this walk in h1 must terminates at the vertex ‘a’. This
walk in h1 combined with h forms a new walk which starts and ends at vertex
v and has more edges than those are in ‘h’.
We repeat this process until we obtain a closed walk that travels through all
the edges of G. In this way, one can get an Euler line. Thus, G is a Euler
Graph.
Konigsberg Bridges Problem
In the graph of the Konigsberg bridges problem, there exist vertices of odd
degree. So all the vertices are not of even degree. Hence by the above
Theorem, we conclude that "the graph representing the Konigsberg bridges
v
Fundamentals of Algorithms Unit 11
Sikkim Manipal University Page No. 209
problem" is not a Euler graph. So we conclude that it is not possible to walk
over each of the seven bridges exactly once and return to the starting point.
Fig. 11.3: Konigsberg bridge
Theorem: A connected graph G is a Euler graph if and only if it can be
decomposed into circuits.
Proof: Suppose graph G can be decomposed into circuits; that is, G is a
union of edge-disjoint circuits. Since the degree of every vertex in a circuit
is two the degree of every vertex in G is even. Hence G is an Euler graph.
Conversely, let G be an Euler graph. Consider a vertex v1. There are atleast
two edges incident at v1. Let one of these edges be between v1 and v2.
Since vertex v2 is also of even degree, it must have at least another edge,
say, between v2 and v3. Proceeding in this fashion, we eventually arrive at a
vertex that has previously been traversed, thus forming a circuit . Let us
remove from G. All vertices in the remaining graph (not necessarily
connected) must also be of even degree. From the remaining graph remove
another circuit in exactly the same way as we removed from G. Continue
this process until no edges are left. Hence the theorem.
Arbitrarily Traceable Graphs:
Consider the graph in Fig. 11.4, which is a Euler graph. Suppose that we
start from vertex ‘a’ and trace the path a b c.
A
B
D C
Fundamentals of Algorithms Unit 11
Sikkim Manipal University Page No. 210
Fig. 11.4: Arbitrarily traceable graph from c
Now at c we have the choice of going to a, d or e. If we took the first choice,
we would only trace the circuit a b c a, which is not an Euler line. Thus,
starting from a, we cannot trace the entire Euler line simply by moving along
any edge that has not already been traversed.
Such a graph is called an arbitrarily traceable graph from vertex v. For
instance, the Euler graph in Fig. 11.4 is an arbitrarily traceable graph from
vertex C, but not from any other vertex.
A Euler graph G is arbitrarily traceable from vertex v in G if and only if
every circuit in G contains v.
Self Assessment Questions
1. A simple graph with ‘n’ vertices and ‘k’ components can have at most
–––––––––– edges.
2. A connected graph G is an Euler graph if and only if it can be
decomposed into –––––––––––––.
11.3 Hamiltonian Circuit and Path
A Euler line of a connected graph was characterized by the property of
being a closed walk that traverses every edge of the graph (exactly once).
Hamiltonian Circuit: A Hamiltonian circuit in a connected graph is defined
as a closed walk that traverses every vertex of G exactly once, except of
course, the starting vertex, at which the walk also terminates.
A circuit in a connected graph G is said to be Hamiltonian as it includes
every vertex of G. Hence a Hamiltonian circuit in a graph of n vertices
consists of exactly n edges. Obviously, not every connected graph has a
Hamiltonian circuit.
a b
c
d e
Fundamentals of Algorithms Unit 11
Sikkim Manipal University Page No. 211
The resemblance between the problem of as Euler line and that of a
Hamiltonian circuit is deceptive. The latter is infinitely more complex.
Although one can find Hamiltonian circuits in many specific graphs.
Hamiltonian Path: If we remove any one edge from a Hamiltonian circuit,
we are left with a path. This path is called a Hamiltonian path. Clearly, a
Hamiltonian path in a graph G traverses every vertex of G. Since a
Hamiltonian path is a sub-graph of a Hamiltonian circuit (which in turn is a
subgraph of another graph) every graph that has a Hamiltonian circuit also
has a Hamiltonian path. There are many graph with Hamiltonian paths that
have no Hamiltonian circuits. The length of a Hamiltonian path (if it exists) in
a connected graph of n vertices is n–1.
In considering the existence of a Hamiltonian circuit (or path), we need only
consider simple graphs. This is because a Hamiltonian circuit (or path)
traverses every vertex exactly once. Hence it cannot include a self-loop or a
set of parallel edges. Thus a general graph may be made simple by removing
parallel edges and self-loops before looking for a Hamiltonian circuit in it.
A given graph may contain more than one Hamiltonian circuit. Of interest
are all the edge-disjoint Hamiltonian circuits in a graph. The determination
of the exact number of edge-disjoint Hamiltonian circuits (or paths) in a
graph in general is also an unsolved problem. However, the number of
edge-disjoint Hamiltonian circuits in a complete graph with odd number of
vertices is an unsolved problem.
Self Assessment Questions
3. A circuit in a connected graph G is said to be ––––––––– it includes
every vertex of G.
4. A Hamiltonian path in a graph G –––––––– every vertex of G.
11.4 Travelling Salesman Problem
A problem closely related to the question of Hamiltonian circuits is the
travelling-salesman problem, stated as follows: A salesman is required to
visit a number of cities during a trip. Given the distances between the cities,
in what order should he travel so as to visit every city precisely once and
return home, with the minimum mileage travelled?
Fundamentals of Algorithms Unit 11
Sikkim Manipal University Page No. 212
Representing the cities by vertices and the road between them by edges,
we get a graph. In this graph, with every edge ei there is associated a real
number (the distances in miles, say), w(ei). Such a graph is called weighted
graph; w(ei) being the weight of edge eI .
In our problem, if each of the cities has a road to every other city, we have a
complete weighted graph. This graph has numerous Hamiltonian circuits, and
we are to pick the one that has the smallest sum of distances (or weights).
The total number of different Hamiltonian circuits in a complete graph of n
vertices can be shown to be 2
)!1n(. This follows from the fact that starting
from any vertex we have n–1 edges to choose from the first vertex, n–2 from
the second, n–3 from the third and so on. These being independent choices,
we get (n–1)! possible number of choices. This number is, however, divided
by 2, because each Hamiltonian circuit has been counted twice.
Theoretically, the problem of the travelling salesman can always be solved
by enumerating all (n–1)!/2 Hamiltonian circuits, calculating the distance
traveled in each, and then picking the shortest one. However, for a large
value of n, the labor involved is too great even for a digital computer.
Self Assessment Questions
5. The problem of the travelling salesman can always be solved by
enumerating all –––––––– Hamiltonian circuits
6. State True or False.
a) A given connected graph G is an Euler graph if and only if all
vertices of G are of odd degree.
b) A connected graph G is an Euler graph if and only if it can be
decomposed into circuits.
c) If we remove any one vertex from Hamiltonian circuit, we are left with
a path. This path is called Hamiltonian path.
11.5 Summary
If some closed walk in a graph contains all the edges of the graph, then
the walk is called an Euler line and the graph an Euler graph.
A connected graph G is a Euler graph if and only if it can be
decomposed into circuits.
Fundamentals of Algorithms Unit 11
Sikkim Manipal University Page No. 213
A Hamiltonian circuit in a connected graphs is defined in a closed walk
that traverses every vertex of G exactly once.
11.6 Terminal Questions
1. Prove that a connected graph G is a Euler graph if and only if it can be
decomposed into circuits
2. Write short notes on Travelling Salesman Problem
3. Prove that a given connected graph G is a Euler graph if and only if all
the vertices of G are of even degree
11.7 Answers
Self Assessment Questions
1. (n - k) (n -k + 1)/2
2. Circuits
3. Hamiltonian
4. Traverses
5. (n–1)!/2
6. a) False
b) True
c) True
Terminal Questions
1. Suppose graph G can be decomposed into circuits; that is, G is a union
of edge-disjoint circuits. Since the degree of every vertex in a circuit is
two, the degree of every vertex in G is even. Hence, G is an Euler
graph.
Conversely, let G be a Euler graph. Consider a vertex v1. There are
atleast two edges incident at v1. Let one of these edges be between v1
and v2. Since vertex v2 is also of even degree, it must have at least
another edge, say between v2 and v3. Proceeding in this fashion, we
eventually arrive at a vertex that has previously been traversed, thus
forming a circuit . Let us remove from G. All vertices in the remaining
graph (not necessarily connected) must also be of even degree. From
the remaining graph remove another circuit in exactly the same way as
we removed from G. Continue this process until no edges are left.
Hence the theorem.
Fundamentals of Algorithms Unit 11
Sikkim Manipal University Page No. 214
2. Representing the cities by vertices and the road between them by
edges, we get a graph. In this graph, with every edge ei there is
associated a real number (the distances in miles, say), w(ei). Such a
graph is called weighted graph; w(ei) being the weight of edge eI .
In our problem, if each of the cities has a road to every other city, we have a
complete weighted graph. This graph has numerous Hamiltonian circuits,
and we are to pick the one that has the smallest sum of distances (or
weights).
The total number of different Hamiltonian circuits in a complete graph of
n vertices can be shown to be 2
)!1n(. This follows from the fact that
starting from any vertex we have n–1 edges to choose from the first
vertex, n–2 from the second, n–3 from the third and so on. These being
independent choices, we get (n–1)! possible number of choices. This
number is, however, divided by 2, because each Hamiltonian circuit has
been counted twice.
3. Suppose, G is a Euler graph. Then, G contains an Euler line. So there
exists a closed walk running through all the edges of G exactly once. Let
v V be a vertex of G. Now in tracing the walk it goes through two
incident edges on v with one entered v and the other exited.
This is true not only for all the intermediate vertices of the walk, but also
true for the terminal vertex because we exited and entered at the same
vertex at the beginning and ending of the walk. Therefore if v occurs k
times in the Euler line, then d(v) = 2k. Thus if G is an Euler graph, then
the degree of each vertex is even.
Converse: Suppose all the vertices of G are of even degree. Now, to
show that G is a Euler graph, we have to construct a closed walk
starting at an arbitrary vertex v and running through all the edges of G,
exactly once. To find a closed walk, let us start from the vertex v. Since
every vertex is of even degree, we can exist from every vertex we
entered, the tracing cannot stop at any vertex but at ‘v’. Since ‘v’ is also
of even degree, we shall eventually reach ‘v’ when the tracing comes to
an end. If this closed walk (h, say) includes all the edges of G, then G
is a Euler graph. Suppose, the closed walk h does not include all the
edges. Then the remaining edges form a subgraph h1 of G. Since both
Fundamentals of Algorithms Unit 11
Sikkim Manipal University Page No. 215
G and h have all their vertices of even degree, the degrees of the
vertices of h1 are also even. Moreover, h1 must touch h at least one
vertex ‘a’ because G is connected. Starting from ‘a’ we can again
construct a new walk in graph h1. Since all the vertices of h1 are of even
degree, and this walk in h1 must terminates at the vertex ‘a’. This walk in
h1 combined with h forms a new walk which starts and ends at vertex v
and has more edges than those are in ‘h’.
We repeat this process until we obtain a closed walk that travels through
all the edges of G. In this way, one can get a Euler line. Thus, G is a
Euler Graph.
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 216
Unit 12 Representations of Graphs Structure
12.1 Introduction
Objectives
12.2 Adjacency and Incidence Matrix
12.3 Circuit Matrix
12.4 Path Matrix
12.5 Summary
12.6 Terminal Questions
12.7 Answers
12.1 Introduction
Even though, a pictorial representation of a graph is very convenient for a
visual study, other representations are better for computer processing and
constructing graphical algorithms. Many results of matrix algebra can be
applied to study the structural properties of graphs. In this unit, we will study
the adjacency, incidence, circuit, and path matrices with illustrations. We
will observe properties of each of these matrices. We will also discuss
correspondence between some graph theoretic operations (properties) and
matrix properties.
Objectives
After studying this unit, you should be able to:
write the matrix representations of a given graph
give different matrix representations of graphs
explain the algebraic properties of matrices of graphs
apply the properties to switching networks
12.2 Adjacency and Incidence Matrix
Definition: Let G be a graph with n vertices v1, v2, …, vn and no self-loops
and no parallel edges. We define an n n adjacency matrix A = [ ija ]n n, as
follows:
aij = 1 if vi is adjacent to vj;
= 0 other wise.
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 217
Fig. 12.1
6x6001000
001011
110010
000010
011101
010010
v
v
v
v
v
v
vvvvvv
6
5
4
3
2
1
654321
Matrix
Observations:
i) The entries along the principal diagonal of the adjacency matrix are all
0’s (since it has no self loops).
ii) The degree of a vertex equals the number of 1’s in the corresponding
row (or a column).
iii) Given any square, symmetric, binary matrix X of order n, one can
always construct a graph G of n vertices (and no parallel edges) such
that X is the adjacency matrix of G.
Definition: Let G be a graph with n vertices, e edges and no self-loops. We
define an n e matrix I = [ ija ]n e, whose n rows correspond to the n
vertices, and the e columns correspond to the e edges, as follows:
aij = 1 if jth edge ej is incident on ith vertex vi;
= 0 other wise.
v1
v2
v3
v4
v5
v6
d
f g
a
c
e
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 218
This matrix I is said to be the vertex-edge incidence matrix (or) simply
incidence matrix. This matrix I (for the graph G) may be denoted as I(G).
Example
Consider the graph G given in Fig. 12.2
Fig. 12.2
The incidence matrix I(G) is an n e = 6 8 matrix as given below:
I(G) =
8x600000011
01001100
00010111
10000000
11110000
00101000
v
v
v
v
v
v
hgfedcba
6
5
4
3
2
1
Matrix
A matrix whose entries are only either 0 or 1, is called a binary matrix (or)
(0, 1)-matrix. We can treat 0 and 1 as two elements from GF(2), the Galois
field modulo 2.
Observations:
The following are some observations on the incidence matrix I of a graph G.
i) Each column of I contains exactly two 1’s, since every edge is incident
on exactly two vertices
ii) The number of 1’s in each row is equal to the degree of the
corresponding vertex.
iii) A row contains only 0’s, denote an isolated vertex.
iv) If two edges a, b are parallel edges (in the graph G), then the
corresponding columns (in the incidence matrix I) are identical.
v1
v2
v3
v4
v5
v6
d
f g
a
b
c
e
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 219
v) If the given graph G is disconnected and consists of two components
g1 and g2, then (we can reorder the list of edges and the list of vertices,
if necessary) then the incidence matrix I(G) of G can be written in a
block-diagonal form as given below:
I(G) = 1
2
I(g ) 0
0 I(g ) where, I(g1) and I(g2) are the incidence matrices
of the components g1 and g2 respectively. (This observation follows
from the fact that no edge in g1 is incident on a vertex of g2, and vice
versa).
[For this, suppose v1, v2, …, vm are vertices and e1, e2, …, en are edges of
G; the subgraphs g1, g2 are two components of G; the vertices v1, v2, …,
vs are the vertices in g1; the vertices vs+1, ..., vm are vertices in g2; the
edges e1, e2, …, ek are in g1; the edges ek+1, …, en are in g2. If we take
vertices and edges in this order, then we get that I(G) = 1
2
I(g ) 0
0 I(g )]
Theorem: Two graphs G1 and G2 are isomorphic if and only if their
incidence matrices I(G1) and I(G2) differ only by permutations of rows and
columns.
Note: If I(G) is an incidence matrix of a connected graph G with n vertices,
then the rank of I(G) is n -1. [Note that the entries 0, 1 are elements from
ℤ2 or the elements of GF(2)].
Note:
i) If we extend the content of the above note for a disconnected graph G
with k components, then the rank of I(G) is n - k. This number n - k is
called as the rank of the graph G with k components.
ii) If we remove any one row from the incidence matrix of a connected
graph, then the remaining (n - 1) e submatrix is of rank n - 1 .
(That is, the remaining n - 1 vectors are linearly independent).
iii) Such (n - 1) e submatrix of I is called a Reduced incidence matrix
and it is denoted by Af. The vertex corresponding to the deleted row
(to get Af) is called the reference vertex.
iv) Any vertex of a connected graph can be made the reference vertex.
Note: The reduced incidence matrix of a tree is non-singular.
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 220
Example: In a disconnected graph, there exists vertices which cannot be
made as the reference vertices. To observe this, consider the graph G given
in Fig. 12.3. This graph is a disconnected graph with two components
(that is, k = 2), n = 5 vertices and n - 1 = 4 edges.
Fig. 12.3
Now the incidence matrix is given by
I(G) =
1001
1100
0110
0011
0000
v
v
v
v
v
dcba
5
4
3
2
1
If =
1001
1100
0110
0011
[after removing the 1st row].
If we add the rows, we get 0 (mod 2). So this matrix is a singular matrix (that
is, det (If) = 0). Note that this is not a reduced matrix.
Definition: Let g be a subgraph of a graph G. Suppose I(g) and I(G) are
the incidence matrices of g and G respectively. Then I(g) is called a
submatrix of I(G). (So permutation of rows (or) columns are permitted).
Note: There is a one-to-one correspondence between each n k submatrix
of I(G) and a subgraph of G with k edges, where k being any positive
integer less than e, and n being the number of vertices in G.
Note: Let I(G) be an incidence matrix of a connected graph G with n
vertices.
v1
v2
v3 v4
v5
c
b d
a
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 221
An (n - 1) (n - 1) submatrix of I(G) is non singular if and only if the n - 1
edges corresponding to the n - 1 columns of this matrix constitute a
spanning tree in G.
Definition: Each largest non-separable subgraph is called a block.
A separable graph consists of two or more non-separable subgraphs.
Example: Consider the graph given in Fig. 12. 4, which contains two blocks.
Fig. 12.4
Self Assessment Questions
1. The entries along the ––––––––– of the adjacency matrix are all 0’s
2. The number of 1’s in each row is equal to the degree of the
corresponding ––––––––––––– .
3. The reduced incidence matrix of a tree is ––––––––––––– .
4. Each largest non-separable subgraph is called a –––––––––––––.
12.3 Circuit Matrix
Definition: Let the number of edges in G be e and the number of circuits in
G be q. Then, a circuit matrix
B = ij q eb , (0, 1)-matrix defined as follows:
bij = 1 if the ith circuit includes the jth edge;
Block-1 Block-2
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 222
= 0 otherwise.
The circuit B of G may be denoted by B(G).
Example: Consider the graph given in Fig. 12.5
Fig. 12.5
For this graph, n = number of vertices = 6
e = number of edges = 8
Here, the graph has 4 different circuits:
{a, b}, {c, d, f, e}, {d, f, g}, and {c, e, g}.
Therefore, the circuit matrix B(G) is a 4 8, (0, 1)-matrix as given below.
B(G) =
8x400111100
01101000
01010100
00000011
4
3
2
1
hgfedcba
Matrix
Observations:
i) If a column of B(G) contains all zeros, then the related edge does not
belong to any circuit. An edge that does not lie in any circuit is said to
be a non-circuit edge.
ii) Each row of B(G) corresponds to a circuit. So each row may be called
as a circuit vector.
iii) If a row contains only one "1", then the related edge is a self-loop.
iv) The number of 1’s in the ith-row is equal to the number of the edges in
the ith circuit.
h
v1
v2
v3
v4
v5
v6
d
f g
a
b
c
e
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 223
v) If a graph G is separable (or disconnected) and consists of two blocks
(or components) g1 and g2, then the circuit matrix B(G) can be written
in a block-diagonal form as B(G) = )(0
0)(
2
1
gB
gB where, B(g1)
and B(g2) are the circuit matrices of g1 and g2 respectively.
Theorem: Let I and B be the incidence matrix and circuit matrix of a (self-
loop free) graph whose columns are arranged with the same ordering of
edges. Then every row of B is orthogonal to every row of I. That is, IBT = BIT
= 0 (mod 2) where IT, BT are the transpose matrices of I and B respectively.
Example: Consider the graph given in Fig. 12.6
Fig. 12.6
The incidence matrix I(G) is given by
I (G) = I =
8x600000011
01001100
00010111
10000000
11110000
00101000
v
v
v
v
v
v
hgfedcba
6
5
4
3
2
1
h
v1
v2
v3
v4
v5
v6
d
f g
a
b
c
e
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 224
B(G) = B =
8x400111100
01101000
01010100
00000011
4c
3c
2c
1c
hgfedcba
The list of all circuits (of the graph) used in getting B(G) is given below: c-1
= {a, b}, c-2 = {c, e, g}, c-3 = {d, f, g}, c-4 = {c, d, f, e}
Now, BT (transpose of B) =
480000
0110
1100
1010
1100
1010
0001
0001
Therefore, IBT =
00000011
01001100
00010111
10000000
11110000
00101000
0000
0110
1100
1010
1100
1010
0001
0001
=
0000
0000
0000
0000
0000
0000
= 0 (mod
2)
Note:
Let G be a connected graph with n vertices, e edges and T a spanning tree.
Then, T contains (n - 1) edges. The remaining e - (n - 1) = e - n + 1 edges
are chords with respect to T. By joining a chord, we get a fundamental
circuit. So, there are e-n+1 fundamental circuits with respect to T.
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 225
Fundamental circuit matrix:
A submatrix (of a circuit matrix) in which all rows corresponds to a set of
fundamental circuits, is denoted by Bf.
Example:
Consider the graph G and its spanning tree T given in Figure 12.7A and
12.7B respectively. The fundamental circuit matrix with respect to the
spanning tree T is given below
Fig. 12.7 A Graph G Fig. 12.7 B Spanning tree T
Bf =
1100100
1000010
1011001
Matrix
Here, 1, 2, 3 are the circuits corresponds to the chords e2 , e3, e6 respectively.
Self Assessment Question
5. An edge that does not lie in any circuit is said to be a –––––––––––––.
12.4 Path matrix
This matrix is used in communications and transport net works.
This is also a (0,1) matrix, which is defined for a specific pair of vertices in a
graph, (x, y), and it is denoted by P(x, y). The rows in P(x, y) correspond to
different paths between vertices x and y, and the columns correspond to the
edges in G.
e4
e1
e2
e3 e5
e6
e7
e1
e4
e5 e7
Chrods
1
2
3
e2 e3 e6 e1 e4 e5 e6
chords
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 226
Definition: P(x, y) = (Pij)p q, where, p = number of paths between x and y
and
q = total number of edges.
th th
ij
1, if j edge lies in i path, andP =
0, otherwise
Observations:
i) A column of all 0’s corresponds to an edge that does not lie in any path
between x and y.
ii) A column of all 1’s corresponds to an edge that lies in every path
between x and y.
iii) There is no row with all 0’s.
Theorem:
If the edges of a connected graph are arranged in the same order for the
columns of the incidence matrix I and the path matrix P(x, y), then the
product (mod 2)
I PT(x, y) = M
Where, the matrix M has 1’s in two rows x and y, and the rest of the n-2
rows are all 0’s.
Self Assessment Question
6. The matrix used in communications and transport net works is –––––– .
12.5 Summary
In this unit, we discussed representation of a graph in several matrix
forms.
A matrix representation is a convenient and useful way of representing a
graph to a computer.
The algebra of matrices can be used to identify certain properties of
graphs.
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 227
12.6 Terminal Questions
1. Draw a graph for each of the following adjacency.
i) ii) iii)
0101
1011
0100
1100
0110
1001
1001
0110
010
101
010
2. Write adjacent matrix for each of the following graphs.
i) 3. Write adjacency matrix to represent the graphs shown below. i) 4. Draw the digraph corresponding to the following adjacency matrix.
0011
1011
0100
1100
v
2
v
4
v
3
v
5
v
1
o o
o o o
iii) v
2
v
5
v
3
v
4
v
1 o
o o
o
o
ii)
v
1 v
2
v
3
v
4
o
o o
o
o o
o
ii)
o
o o
o
iii)
o
o o
o
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 228
5. Verify whether or not the following graphs isomorphic.
6. Find the incidence matrices of the following graphs.
12.7 Answers
Self Assessment Questions
1. Principal Diagonal
2. Vertex
3. Non-Singular
4. Block
5. Non-Circuit Edge
6. Path Matrix
G: v5
v2
v3
v4
v1
o o o
o
o 1
1v 1
3v
1
4v
1
2v
1
5v
o
o
o
o o
G1:
e2
v1
e1
e5
e3
e4
v2 v3
v4
o
o
o
o
i) (ii)
e1
e2
e4
e7 e6 e3
e5
e8 d f
e a
b
c
o o
o
o o
o
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 229
Terminal Questions
1. 2.
i)
01000
10110
01000
01001
00110
ii)
1210
2012
1103
0230
iii)
00110
00111
11000
11001
01010
3.
i)
0101
1011
1101
1110
ii)
020
211
010
iii)
0001
1000
0100
0110
4.
5. Each of the graphs G and G1 have the adjacency matrix:
I(G) = I(G1) =
01110
10101
11010
10101
01010
Therefore, G and G1 are isomorphic.
o
o
o
o
i)
o
o
o
o ii)
o
o o
o
iii) o
o o
Fundamentals of Algorithms Unit 12
Sikkim Manipal University Page No. 230
6.
i)
11010
01100
00111
10001
e1 e 2 e3 e4 e5
v1
v2
v3
v4
ii)
00000011
01001100
00010111
10000000
11110000
00101000
a
b
c
d
e
f
e1 e2 e3 e4 e5 e6 e7 e8
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 231
Unit 13 Directed Graphs Structure
13.1 Introduction
Objectives
13.2 Types of Directed Graphs
13.3 Binary Relation As a Digraph
13.4 Euler’s Digraphs
13.5 Matrix Representation of Digraphs
13.6 Summary
13.7 Terminal Questions
13.8 Answers
13.1 Introduction
The graphs so far studied are undirected graphs (no direction was assigned to
the edge in a graph). In this unit, we shall consider directed graphs (graphs in
which edges have directions). Many physical situations require directed graphs.
The applications of directed graphs are in the street map of city with one-way
streets, flow networks with valves in the pipes, and electrical networks.
Directed graphs in the form of signal flow graphs are used for system analysis
in control theory. Most of the concepts and terminology of undirected graphs
are also applicable to directed graphs. In this unit we will discuss some of the
properties of directed graphs which are not shared by undirected graphs.
Objectives
After studying this unit, you should be able to:
write the indegree and outdegree of a vertex
explain the tournaments and Euler’s digraphs
give the different matrix representations of digraphs
apply the properties to flows, network and traffic problem.
13.2 Types of Directed Graphs
The different types of directed graphs are discussed below:
Definition: A directed graph (or) a digraph D consists of a non-empty set V
(the elements of V are normally denoted by v1, v2, …) and a set E (the
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 232
elements of E are normally denoted by e1, e2, ….) and a mapping that
maps every element of E onto an ordered pair (vi, vj) of elements from V.
The elements of V are called as vertices or nodes or points. The elements
of E are called as edges or arcs or lines. If e E and vi, vj V such that
(e) = (vi, vj), then we write e = jivv . In this case, we say that e is an edge
between vi and vj. (We also say that e is an edge from vi to vj). (We also say
that e originates at vi and terminates at vj). (An edge from vi to vj is denoted
by a line segment with an arrow directed from vi to vj).
Note:
i) A directed graph is also refered to as an oriented graph.
ii) Let D be a directed graph and e = vu .
Then, we say that e is incident out of the vertex v, and incident into the
vertex u. In this case, we say that the vertex v is called the initial vertex and
the vertex u is called the terminal vertex of e.
Example: The graph given in Figure-13.1 is a digraph with 5 vertices and
ten edges.
Fig .13.1
Here, v5 is the initial vertex and v4 is the terminal vertex for the edge e7. The
edge e5 is a self-loop.
Definition:
i) The number of edges incident out of a vertex v is called the out degree
(or) out-valence of v. The out degree of a vertex v is denoted by d+(v).
ii) The number of edges incident into v is called the indegree (or) in-
e5
e9
v2
e10
e7
v1
e8
e6
v4
v3
v5
e1 e4
e3 e2
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 233
valance of v. The indegree of a vertex v is denoted by d-(v). Note that
the degree of v is equal to the sum of indegree and out degree of v, for
any vertex v in a graph. (In symbols, we can write as d(v) = d+(v) + d-(v)
for all vertices v).
Example: Consider the graph given in Fig. 13.1
i) Here, d+(v1) = 3, d+(v2) = 1, d+(v3) = 1, d+(v4) = 1, d+(v5) = 4. d - (v1) = 1,
d - (v2) = 2, d - (v3) = 4, d -(v4) = 3, d - (v5) = 0.
ii) d(v1) = 4 = 3 + 1 = d+(v1) + d-(v1) ; d(v2) = 3 = 1 + 2 = d+(v2) + d-(v2),
and so on.
Problem: Let D be a directed graph. Show that the sum of all in-degrees of
the vertices of D, is equal to the sum of all out-degrees of the vertices of D;
and each sum is equal to the number of edges in D.
[In other words,
n
1i
i )v(d =
n
1i
i )v(d = q where q is the number of
edges in D, and {v1, v2, …, vn} is the set of all vertices in D].
Solution: Let D be any digraph with n vertices and q edges. When the
indegrees of vertices are counted, each edge is counted exactly once.
[This is because every edge goes to exactly one vertex].
So
n
1i
i )v(d = q ... (i)
Similarly, when the out-degrees of vertices are counted, each edge is
counted exactly once. [This is, because each edge goes out of exactly one
vertex].
So
n
1i
i )v(d = q …. (ii).
From (i) and (ii), we get that
n
1i
i )v(d =
n
1i
i )v(d = q.
Example: Consider the digraph in Fig. 13.1 Here, the number of vertices
n = 5, and the number of edges q = 10.
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 234
Now,
n
1i
i )v(d = d+(v1) + d+(v2) + d+(v3) + d+(v4) + d+(v5) = 3 + 1 + 1 + 1
+ 4 = 10 = q
Also,
n
1i
i )v(d = d -(v1) + d -(v2) + d -(v3) + d -(v4) + d -(v5) = 1 + 2 + 4
+ 3 + 0 = 10 = q.
Therefore,
n
1i
i )v(d =
n
1i
i )v(d = q = the number of edges.
Definitions:
i) A vertex v is said to be an isolated vertex if the out degree of v and the
indegree of v are equal to zero. (In symbols, d+(v) = 0 = d-(v)).
ii) A vertex v in a digraph D is said to be a pendent vertex if it is of degree
1. (In other words, a vertex is said to be a pendent vertex if the degree
of v = d+(v) + d -(v) = 1).
iii) Two directed edges are said to be parallel edges if they are mapped
onto the same ordered pair of vertices.
iv) In other words, two directed edges e and f are said to be parallel edges
if both e and f originates from the same vertex, and also terminates at
the same vertex. (In the graph given in Fig. 13.1, the edges e8, e9, and
e10 are parallel edges. The edges e2 and e3 are not parallel).
Note:
i) Let D be directed graph. If we disregard the orientation (that is,
direction) of every edge in D, then we get an undirected graph. The
undirected graph obtained (in this way) from D is called the undirected
graph corresponding to D.
ii) Suppose, H is an undirected graph. To each edge of H we can assign a
direction. Then the digraph obtained is called an orientation of H. (or a
directed graph associated with H).
Definition: Let D1 and D2 be two digraphs. These two graphs are said to be
isomorphic if
i) the corresponding undirected graphs are isomorphic, and
ii) the directions of corresponding edges must be same.
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 235
Example:
i) Consider the directed graphs D1 and D2. The two digraphs D1 and D2
are not isomorphic (because the direction of the edge e4 in D1 is
different from the direction of the edge e4 in D2).
Fig. 13.2: Graph D1 Graph D2
ii) Note that the undirected graphs corresponding to D1 and D2 are
isomorphic.
Definition: A digraph that has neither self-loops nor parallel edges is called
a simple digraph.
In other words, a directed graph D is said to be a simple digraph if the
corresponding undirected graph is a simple graph. The two digraphs given
in Fig. 13.2 are simple directed graphs.
Definitions:
i) Digraphs that have at most one directed edge between any pair of
vertices, but are allowed to have self-loops are called the asymmetric
(or) anti-symmetric digraphs.
ii) A digraph D is said to be a symmetric digraph if for every edge (a, b) in
D there is also an edge (b, a) in D.
iii) A digraph that is both simple and symmetric is called a simple
symmetric digraph.
iv) A digraph that is both simple and asymmetric is called simple
asymmetric digraph.
v) A simple digraph is said to be a complete symmetric digraph if it
satisfies the following condition: "Given any two vertices v1 and v2,
there corresponds exactly one edge directed from v1 to v2".
vi) A complete asymmetric digraph (or) tournament (or) a complete
tournament. is an asymmetric digraph in which there is exactly one
edge between every pair of vertices.
e1 e4
e5
e2
e3
e6
e1 e4
e5
e2
e3
e6
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 236
Example: In the following,
i) The graph G1 is not asymmetric since there exist two directed edges
between v1 and v2.
ii) The graph G2 is asymmetric.
iii) Any null graph is asymmetric.
Fig. 13.3 G1 G2 G3
iv) The digraph G3 is a complete symmetric digraph.
Observation:
i) A tournament (means, complete asymmetric digraph) of n vertices
contains 2
1nn edges.
The directed graph given in fig. 13.2 is a tournament. In this graph,
the number of vertices is n = 4.
The number edges is 6 = 2
144 =
2
1nn.
ii) A complete symmetric graph of n vertices contains n(n - 1) edges. The
digraph given in fig. 13.3, G3 is a complete symmetric graph on
n = 4 vertices.
In this graph, the number of edges is 12 = 4(4 -1) = n (n - 1).
Definition: A digraph D is said to be balanced if the in-degree of v is equal to
the out-degree of v for every vertex v in D. In other words, a digraph D is said to
be balanced if d+(v) = d-(v) for all vertices v in the digraph D. A balanced
digraph is also known as pseudo symmetric digraph (or) an isograph.
v2 v3
v1
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 237
A balanced digraph is said to be a regular digraph if it satisfies the following
two conditions:
i) The in-degrees of all the vertices of D are equal; and
ii) The out-degrees of all the vertices of D are equal.
Self Assessment Questions
1. A directed graph is also refered to as an –––––––––––– .
2. The number of edges incident out of a vertex v is called the––––––––––.
3. A vertex v is said to be an isolated vertex if the out degree of v and the
indegree of v are equal to––––––––––––––.
13.3 Binary Relation As a Digraph
Let X be a set. Represent the elements of X by the symbols x1, x2, …
Suppose that R is a relation on X. We represent the elements of X as
vertices and draw a directed edge from xi to xj if (xi, xj) R. Then, we get a
directed graph which represents the given relation R on X.
Example: Consider the set X = {3, 4, 5, 7, 8} and the relation (R, >) on X.
Then, R = {(8, 3), (8, 4), (8, 5), (8, 7), (7, 3), (7, 4), (7, 5), (5, 4), (4, 3)}
= {(x, y) / x > y, x, y X}.
The graph D given below represents this relation R on X.
Fig 13.4
It is clear that, every binary relation on a finite set can be represented by a
directed graph without parallel edges. Conversely, if a directed graph D
without parallel edges was given, then there corresponds a binary relation
on the set of vertices.
3
4
8
5 7
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 238
Definition:
i) Let X be a set, and R a relation on X (that is, R X X). If (x, y) R,
then we also write xRy. The relation R is said to be reflexive relation if
xRx for all x X.
(Note that the digraphs of a reflexive relation have a self-loop at every
vertex).
We call a directed graph representing a reflexive binary relation on its
vertex set as reflexive digraph”.
A digraph in which no vertex has a self-loop is called an irreflexive
digraph.
ii) Let X be a set, and R a relation on X. We say that R is symmetric if a,
b X, aRb bRa. Note that a directed graph representing a
symmetric relation is a symmetric digraph.
Example: The graph given in G1 is the digraph of a relation which is
symmetric but not reflexive. This relation is on the set {x1, x2, x3, x4}. Some
authors represent this relation by the undirected graph given in G2. Note that
in the graph G2, only one undirected edge is taken between a pair of
vertices (that are related under the relation R).
Fig. 13.5: Graph G1 Graph G2
Definition: Let X be a set, and R a relation on X. The relation R is said to be
transitive if a, b, c R, aRb, bRc aRc. A digraph representing a transitive
relation is called a transitive digraph. [Observe that the graph given in 13.4
is a transitive digraph].
Let X be a set, and R a relation on X.
i) The relation R is said to be an equivalence relation if it is reflexive,
symmetric and transitive.
x1 x2
x3 x4
x1 x2
x4 x3
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 239
ii) A digraph representing an equivalence relation is called an equivalence
digraph.
Example: Consider the binary relation R (= “Congruent modulo 3”) defined
on the set X = {10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20}.
We can observe that this relation R on X is an equivalence relation. The
related equivalence graph was given. We are using undirected graph here.
Observe that the set of vertices is divided into three disjoint equivalence
classes. Each of these sets form separate components. Each component is
an undirected subgraph (due to symmetry, we are using undirected graph
with a self-loop at each vertex). Also, note that (since any two elements in
an equivalence class are related) any two vertices inside the component
were joined by an edge.
Definition: Let X be a set, and R is a binary relation on X. This relation R on X
may be represented by a matrix. This matrix is called as relation matrix. It is a
(0,1)-n n-matrix, where n is the number of elements in the set X = {x1, x2, …,
xn}.
The relation matrix ( ija ) is defined as follows:
ija = 1 if xiRxj,
= 0 otherwise.
11 14
17 20
)3(mod2
10 13
16 19
)3(mod1
15
12
18
)3(mod0
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 240
Example: The relation matrix of the relation R (" usual grater than") on the
set
X = {3, 4, 5, 7, 8}, is the Matrix
01111
00011
01011
00001
00000
8
5
7
4
3
85743
Definitions:
i) A path v0e1v1e2v2 … envn is said to be a directed path if ek is oriented
from vk-1 to vk for each 1 k n. A path which is not a directed path
is called a semi-path. In a directed graph, the word "path" refers to
either a directed path or a semi-path.
ii) Let D be a directed graph. A directed walk (in D) from a vertex v to u is
an alternating sequence of vertices and edges beginning with v and
ending with u such that each one of the edges is oriented (or directed)
from the vertex preceding it to the vertex following it. [In other words, a
walk v0e1v1e2v2 … envn in the undirected graph D is said to be a
directed walk if ek is oriented from vk-1 to vk for all 1 k n].
Example: Consider the graph given in the Fig-13.1
i) The path v5 e8 v3 e6 v4 e3 v1 is a directed path from v5 to v1.
ii) If we disregard the orientation in D, then v5 e7 v4 e6 v3 e1v1 is a path.
Observe that it is not a directed path. So it is a "semi-path".
Since a directed walk is a walk, no edge appears more than once (but a
vertex may appear more than once).
Definitions: Let D be a digraph.
i) A walk which is not a directed walk is called as semi-walk. We use the
term "walk" to mean either a directed walk (or) a semi-walk.
ii) A circuit (in the corresponding undirected graph) v0e1v1e2v2 … envn is said
to be a directed circuit if ek is oriented from vk-1 to vk for all 1 k n.
iii) A circuit which is not a directed circuit is a semi-circuit.
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 241
Example: Consider the digraph G given 13.1
i) The circuit v1e1v3e6v4e3v1 is a directed circuit.
ii) The circuit v1e1v3e6v4e2v1 is not a directed circuit. So it is a semi-circuit.
In undirected graphs, we define the connectedness by using the notion "path".
As there are two different types of paths (namely directed path, semi-path) in
digraphs, we have two different types of connectedness in the digraphs.
Definitions: Let D be a digraph.
i) D is said to be strongly connected if there exists at least one directed
path from every vertex to every other vertex.
ii) D is said to be weakly connected if its corresponding undirected graph
is connected, but D is not strongly connected.
iii) We say that D is connected if the undirected graph corresponding to D
is connected. So the word "connected digraph" refers to both strongly
connected digraph and weakly connected digraph.
iv) If D is not connected, then we say that D is a disconnected graph.
Example: Consider the digraphs D1 and D2 given in the following.
i) The graph-D1 is a strongly connected graph.
ii) In the graph-D2, there is no directed path from v1 to v4 so it is a weakly
connected digraph.
Graph-D1 Graph-D2
Since there are two types of connectedness in a digraph, we define two
types of components.
Definitions: Let D be a directed graph.
i) Each maximal connected (weakly or strongly) subgraph of a digraph
D, is said to be a component of D.
ii) Within each component of D, the maximal strongly connected
subgraphs are said to be the fragments (or) strongly connected
fragments of D.
e2 e2
v4
v1
e1 e4
e5
e3
e6
e1 e4
e5
e3
e6
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 242
Example: Consider the digraph given in the Fig-13.6.
This graph consists of two components H1 and H2.
Fig-13.6
i) The component H1 contains three fragments: {e1, e2}; {e5, e6, e7, e8};
and {e10}.
ii) The component H2 contains a fragment {e11, e12, e13}.
iii) It can be observed that e3, e4 and e9 do not appear in any fragment of
H1.
Definitions: Let D be a digraph. The condensation Dc of D is a digraph
obtained by using the following method:
i) Each strongly connected fragment is to be replaced by a new vertex, and
ii) For any two strongly connected components c1 and c2, the set of all
the directed edges from c1 to c2 is to be replaced by a single directed
edge (from c1 to c2).
iii) Let D be a directed graph and v, u are vertices in D. Then v said to be
accessible (or) reachable from the vertex u, if there is a directed path
from u to v.
Example: Consider the graph D given in the Fig-13.6
i) The components of D are H1, H2.
ii) The fragments of D are C1 = {e1, e2};
C2 = {e5, e6, e7, e8}; C3 = {e10}; and C4 = {e11, e12, e13}.
e1
e3
e8
H1
v1
e9
e13
e12
e14
e11
e10
e2
e5
e4
e6 e7
H2
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 243
iii) The fragments C1, C2, C3 and C4 are to be considered as vertices.
iv) There are two edges e3 and e4 from C1 to C2.
These two edges e3 and e4 are to be replaced by an edge (say f1)
from the vertices C1 to C2.
v) There is only one edge e9 from the fragment C2 to C3. So this edge
e9 is to be drawn from the vertex C2 to C3.
vi) There is only one edge (e14) from the fragment C4 to the vertex v1.
So, this edge is to be drawn (in the condensation) from the vertex C4
to v1.
vii) Finally, we get the condensation Dc that is given in the Fig-13.7
Fig. 13.7
Observations:
i) The condensation Dc of a strongly connected digraph D is a vertex.
ii) The condensation Dc of a digraph D contains no directed circuits.
It is clear that a digraph D is strongly connected "v is accessible from u"
for all vertices v and u in D.
Self Assessment Questions
4. Let X be a set, and R a relation on X. The relation R is said to be –––––
if a, b, c R, aRb, bRc aRc.
5. Each maximal connected subgraph of a digraph D, is said to be a –––––
of D
13.4 Euler’s Digraphs
Definitions: Let D be a directed graph.
i) A directed walk that starts and ends at the same vertex is called a
closed directed walk.
C3
e9
v1
C4
e14
C2
f1
C1
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 244
ii) A closed directed walk which traverses every edge of D exactly once,
is called a directed Euler line.
iii) D is said to be a Euler digraph if it contains a directed Euler line.
Example: Consider the graph given below
Fig. 13.8
The edge sequence f, a, b, c, d, e forms a directed Euler line. Hence, this
graph is a Euler digraph.
Theorem: Let D be a digraph. Then D is an Euler digraph if and only if D is
connected and balanced.
[that is, d+(v) = d-(v) for every vertex v in D].
Definition: A connected digraph containing no circuit (neither a directed
circuit nor a semi-circuit) is said to be a tree.
Note:
i) A tree of n vertices contain n - 1 edges (directed).
ii) Trees in digraphs have additional properties (than those in undirected
graphs) and variations resulting from the relative orientations of the
edges.
Definitions: A digraph D is said to be an arborescence if it satisfies the
following two conditions:
i) D contains no circuit (neither a directed circuit nor a semi-circuit).
ii) There exists exactly one vertex v of zero in-degree (this vertex v is
called the root of the arborescence).
a
fc
e
c
1
2
b
d
c
4
3
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 245
Example: The graph given here is arborescence.
Fig. 13.9
Self Assessment Questions
6. A closed directed walk which traverses every edge of D exactly once, is
called a –––––––––––.
7. A tree of n vertices contains –––––––– edges.
13.5 Matrix Representation of Digraphs
Definition: Let D be a digraph with p vertices. The adjacency matrix of D is
a p p matrix (aij) with
aij = i j1 if v v is an arc of D
0 otherwise
.
It is denoted by A(D).
Fig. 13.10
v (root)
v2
e5
e7
v1
e8
e6
v4
v3
v5
e1
e4
e3 e2
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 246
The adjacency matrix for the digraph given in figure13.10 is
01110
01000
00010
10000
00110
.
The sum of the ith row entries of the adjacency matrix gives d+(vi) and the
sum of the ith column entries gives d-(vi) for every i.
The powers of A(D) give the number of walks from one point to another.
Definitions:
i) Let D be digraph with p vertices. The reachability matrix R = (rij) is the
p p matrix with
.itselffromreachableisvertexeachthatassumew e,otherw ise0
vfromreachableisvif1r ijij
ii) The distance matrix is the p p matrix whose (i,j)th entry gives the
distance from the point vi to the point vj and is infinity if there is no path
from vi to vj.
iii) The detour matrix is the p p matrix whose (i,j)th entry is the length of
any longest vi-vj path and infinity if there is no such path.
Example:
The reachable matrix, distance matrix and the detour matrix respectively for
the matrix of given of 13.10 is given below.
11110 1 1 1 1 1 2
01000
10110 , 2 3 3 1 and 2 3 3 1 .
11110 1 2 2 2 1 2 2 3
11110 2 3 3 1 2 3 3 2
Self Assessment Question
8. Let D be a digraph with p vertices. The adjacency matrix of D is a ––––
matrix
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 247
13.6 Summary
In this unit, we have investigated the fundamental features of the
directed graphs.
We have observed that the properties of digraphs are similar to some
types of undirected graphs. The close relation ship between binary
relation and digraphs was given in the unit.
Directed graphs have interesting applications in telecommunications,
flows and networks routing problems.
13.7 Terminal Questions
1. Find the in-degree and out-degree of the following digraphs
2. Find the spanning tree of the digraph given in 1 (ii).
3. Verify whether or not the following digraphs are isomorphic? If so, write
the isomorphisms.
4. Draw the digraphs for the digraph D = {a, b, c, d, e} where the arcs
represented by {(a, c), (a, d), (b, e), (e, c), (d, c)}. Write the in-degree
and out-degree of D.
(i)
c
b a
o
o o
c
a
b
d
e
(ii)
o
o
o o
o
1
2
3
4 o
o
o
o
G:
a
b
c d
G1:
o o
o
o
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 248
Find also the converse D1 (i.e., reversing the each arc direction in D)
and find the indegree and outdegree of each point in D1.
5. Give an example of a directed tree with five vertices.
6. Find in-degree and out-degree of each vertex in the following directed
graphs.
13.8 Answers
Self Assessment Questions
1. Oriented Graph
2. Out-Degree
3. Zero
4. Transitive
5. Component
6. Directed Euler line
7. n - 1
8. p p
Terminal Questions
1. i) In-degree a = 1
out-degree a = 2
Similarly id (b) = 1, Od (b) = 1
id (c) = 2, Od (c) = 1
ii) id (a) = 1 id (b) = 2 id (c) = 3 id (d) = 1 id (e) = 1
Od (a) = 1 Od ( b) = 2 Od (c) = 2 Od (d) = 1 Od (e) = 1
(i)
v1 v2
v3 v4
v5
o
o
o
o o
(ii)
d
a b
c
o o
o o
Fundamentals of Algorithms Unit 13
Sikkim Manipal University Page No. 249
2.
3. D and D1 are isomorphic under the isomorphism f(1) = a, f(2) = b, f(3) =
c, f(4) = d.
4. The digraph representing the given arcs is
The converse digraph also can be found in a similar way.
5.
6. i) in-degree v1 =2, out-degree v1 = 1
in-degree v2 = 2, out-degree v2 = 2
Similar for other vertices. (ii). Similar to (i).
c
e
b o
o
o o
o b
d
a
d
e
c a b
o
o
o
o o
a
b c
e
d
aa
a
o
o o
o
o
o
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 250
Unit 14 Complexity of Algorithms
Structure
14.1 Introduction
Objectives
14.2 Notations for the Growth Rates of Functions
14.3 Classification of Problems
14.4 Reduction, NP-Complete and NP-Hard Problems
14.5 Establishing NP-Completeness of Problems
14.6 Summary
14.7 Terminal Questions
14.8 Answers
14.1 Introduction
In the previous units, we discussed a number of problems which cannot be
solved by algorithmic means and also discussed a number of issues about
such problems.
In this unit, we will discuss the issue of efficiency of computation of an
algorithm in terms of the amount of time used in its execution. On the basis
of analysis of an algorithm, the amount of time that is estimated to be
required in executing an algorithm will be referred to as the time complexity
of the algorithm. The time complexity of an algorithm is measured in terms
of some (basic) time unit (not second or nano-second). Generally, time
taken in executing one move of a TM, is taken as (basic) time unit for the
purpose. Or, alternatively, time taken in executing some elementary
operation like addition, is taken as one unit. More complex operations like
multiplication etc., are assumed to require an integral number of basic units.
As mentioned earlier, given many algorithms (solutions) for solving a
problem, we would like to choose the most efficient algorithm from amongst
the available ones. For comparing efficiencies of algorithms, that solve a
particular problem, time complexities of algorithms are considered as
functions of the sizes of the problems (to be discussed). The time
complexity functions of the algorithms are compared in terms of their growth
rates (to be defined), as growth rates are considered important measures of
comparative efficiencies.
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 251
Objectives
After studying this unit, you should be able to:
explain the concepts of time complexity, size of a problem and growth
rate of a function
define and explain the well-known notations for growth rates of
functions, viz., O, , , ,
explain criteria for classification of problems into undefinable, definable
but not solvable, solvable but not feasible, P, NP, NP-hard and
NP- Complete etc.
define a number of problems which are known to be NP-Complete
problems
14.2 Notations for the Growth Rates of Functions
The time required by a solution or an algorithm for solving a (solvable)
problem, depends not only on the size of the problem/input and the number
of operations that the algorithm/solution uses, but also on the hardware and
software used to execute the solution. However, the effect of change/
improvement in hardware and software on the time required may be closely
approximated by a constant.
Suppose, a supercomputer executes instructions one million times faster
than another computer. Then irrespective of the size of a (solvable) problem
and the solution used to solve it, the supercomputer solves the problem
roughly million times faster than the computer, if the same solution is used
on both the machines to solve the problem. Thus, we conclude that the time
requirement for execution of a solution, changes roughly by a constant
factor on change in hardware, software and environmental factors.
The Constant Factor in Complexity Measure
An important consequence of the above discussion is that if the time taken
by one machine in executing a solution of a problem is a polynomial
(or exponential) function in the size of the problem, then time taken by every
machine is a polynomial (or exponential) function respectively, in the size of
the problem. Thus, functions differing from each other by constant factors,
when treated as time complexities should not be treated as different, i.e.,
should be treated as complexity-wise equivalent.
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 252
Asymptotic Considerations
Computers are generally used to solve problems involving complex
solutions. The complexity of solutions may be either because of the large
number of involved computational steps and / or large size of input data.
The plausibility of the claim apparently follows from the fact that, when
required computers are used generally, not to find the product of two 2 x 2
matrices but to find the product of two n x n matrices for large n running into
hundreds or even thousands.
Similarly, computers, when required, are generally used not to find roots of
quadratic equations but for finding roots of complex equations including
polynomial equations of degrees more than hundreds or sometimes even
thousands.
The above discussion leads to the conclusion that when considering time
complexities f1(n) and f2(n) of (computer) solutions of a problem of size n,
we need to consider and compare the behaviours of the two functions only
for large values of n. If the relative behaviours of two functions for smaller
values conflict with the relative behaviours for larger values, then we may
ignore the conflicting behaviour for smaller values. For example, if the
earlier considered two functions
f1(n) = 1000 n2 and
f2(n) = 5n4
represent time complexities of two solutions of a problem of size n, then
despite the fact that
f1(n) f2(n) for n 14,
we would still prefer the solution having f1(n) as time complexity because
f1 (n) f2(n) for all n 15.
Self Assessment Question
1. Time requirement for execution of a solution, changes roughly by a
constant factor on change in –––––––––– and–––––––––––––––––.
14.3 Classification of Problems
The fact of being engaged in solving problems may be the only sure
indication of a living entity being alive (though, the distinction between
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 253
entities being alive and not being alive is getting fuzzier day by day). The
problems, attempted to be solved, may be due to the need for survival in a
hostile and Competitive environment or may be because of intellectual
curiosity of knowing more and more of the nature. In the previous unit, we
studied a number of problems which are not solvable by computational
means. We can go still further and categorize the problems, which we
encounter or may encounter, into the following broad classes:
i) Problems which cannot even be defined formally.
By a formal definition of a problem, we mean expressing in terms of
mathematical entities like sets, relations and functions etc., the
information concerning the problem, in respect of at least
a) Possible inputs
b) Possible outcomes
c) Entities occurring and operations on these entities in the (dynamic)
problem domains.
In this sense of definition of a problem, how to talk of solving, most of
the problems can not even be defined. Think of the following problems.
a) Why the economy is not doing well?
b) Why there is hunger, illiteracy and suffering despite international
efforts to eradicate these?
c) Why some people indulge in corrupt practices despite being
economically well?
These are some of the problems, the definition of each of which
require enumeration of potentially infinite parameters, and hence are
almost impossible to define.
ii) Problems which can be formally defined but can not be solved by
computational means. We discussed some of these problems in the
previous unit.
iii) Problems which, though theoretically can be solved by computational
means, yet are infeasible, i.e., these problems require so large amount
of computational resources that practically is not feasible to solve
these problems by computational means. These problems are called
intractable or infeasible problems. The distinguishing feature of the
problems is that for each of these problems any solution has time
complexity which is exponential, or atleast non-polynomial, function of
the problem size.
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 254
iv) Problems that are called feasible or theoretically not difficult to solve
by computational means. The distinguishing feature of the problems is
that for each instance of any of these problems, there exists a
Deterministic Turing Machine that solves the problem having time-
complexity as a polynomial function of the size of the problem. The
class of problem is denoted by P.
v) Last, but probably most interesting class includes large number of
problems, for each of which, it is not known whether it is in P or not
in P.
These problems fall somewhere between class Ill and class IV given
above. However, for each of the problems in the class, it is known that
it is in NP, i.e., each can be solved by at least one Non-Deterministic
Turing Machine, the time complexity of which is a polynomial function
of the size of the problem.
A problem from the class NP can equivalently but in more intuitive
way, be defined as one for which a potential solution, if given, can be
verified in polynomial time whether the potential solution is actually a
solution or not.
The problems in this class are called NP-Complete problems (to be formally
defined later). More explicitly, a problem is NP-complete if it is in NP and for
which no polynomial-time Deterministic TM solution is known so far.
Most interesting aspect of NP-complete problems, is that for each of these
problems neither, so far, it has been possible to design a Deterministic
polynomial-time TM solving the problem nor it has been possible to show
that Deterministic polynomial-time TM solution can not exist.
The idea of NP-completeness was introduced by Stephen Cook* in 1971
and the satisfiability problem defined below is the first problem that was
proved to be NP-complete, of course, by S. Cook.
Next, we enumerate some of the NP-complete problems without justifying
why these problems have been placed in the class.
Problem: Hamiltonian circuit problem (HCP) given an undirected graph
G = (V, E), does G contain a Hamiltonian circuit?
Further Explanation: A Hamiltonian circuit of a graph G = (V, E) is a set of
edges that connects the nodes into a single cycle, with each node appearing
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 255
exactly once. We may note that the number of edges on a Hamiltonian
circuit must equal the number of nodes in the graph.
Further, it may be noted that HCP is a special case of TSP in which the cost
between pairs of nodes is the same, say, 1.
Example: Consider the graph
Fig. 14.1
Then, the above graph has one Hamiltonian circuit viz., (1, 2, 4, 3, 1)
Problem: The vertex cover problem (VCP) (also known as Node cover
problem). Given a graph G = (V, E) and an integer k, is there a vertex cover
for G with k vertices?
Explanation: A vertex cover for a graph G is a set C of vertices so that
each edge of G has an endpoint in G. For example, for the graph shown
above, {1, 2, 3} is a vertex cover. It can be easily seen that every superset
of a vertex cover of a graph is also a vertex cover of the graph.
Problem: K-Colourability Problem: Given a graph G and a positive integer
k, is there a k-colouring of G?
Explanation: A k-colouring of G is an assignment to each vertex of one of
the k colours so that no two adjacent vertices have the same color. It may
be recalled that two vertices in a graph are adjacent if there is an edge
between the two vertices.
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 256
Fig. 14.2
As the vertices 1, Z 3 are mutually adjacent therefore, we require atleast
three colours for k-colouring problem.
Problem: The complete sub-graph problem (CSP Complete Sub) or clique
Problem: Given a graph G and positive integer k, does G have a complete
sub-graph with k vertices?
Explanation: For a given graph G = (V, E), two vertices v1 and v2 are said
to be adjacent if there is an edge connecting the two vertices in the graph. A
sub-graph H = (V1, E1,) of a graph G = (V, E) is a graph such that V1 V
and E1 E. In other words, each vertex of the sub-graph is a vertex of the
graph and each edge of the sub-graph is an edge of the graph.
Complete Sub-graph of a given graph G is a sub-graph in which every pair
of vertices is adjacent in the graph.
For example, in the above graph, the sub-graph containing the vertices
{1, 2, 3} and the edges (1, 2), (1, 3), (2, 3) is a complete sub-graph or a
clique of the graph. However, the whole graph is not a clique as there is no
edge between vertices 1 and 4.
Problem: Independent set problem: Given a graph G = (V, E) and a positive
integer k, is there an independent set of vertices with atleast k elements?
Explanation: A subset V1 of the set of vertices V of graph G is said to be
independent, if no two distinct vertices in V1 are adjacent. For example, in
the above graph V1 = {1, 4} is an independent set.
Problem: The sub-graph isomorphism problem: Given graph G1 and G2,
does G1 contain a copy of G2 as a subgraph?
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 257
Explanation: Two graphs H1 = (V1, E1) and H2 = (V2, E2) are said to be
isomorphic if we can rename the vertices in V2 in such a manner that after
renaming, the graph H1 and H2 look identical (not necessarily pictorially, but
as ordered pairs of sets)
Fig. 14.3
are isomorphic graphs because after mapping 1 a, 2 b, 3 c and
4 d, the two graphs become identical.
Problem: Given a graph g and a positive integer k, does G have an ―edge
cover‖ of k edges?
Explanation: For a given graph G = (V, E), a subset E1 of the set of edges
E of the graph, is said to be an edge cover of G, if every vertex is an end of
at least one of the edges in E1.
Example: For the graph
Fig. 14.4
The two-edge set {(1, 4), (2, 3)} is an edge cover for the graph.
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 258
Example: Let S = {1, 2,……, 10}
and P = {S1, S2, S3, S4, S5} such that
S1 = {1, 3, 5}
S2 = {2, 4, 6}
S3 = {1, 2, 3, 4}
S4 = {5, 6, 7, 9, 10}
S5 = {7, 12, 9, 10}
Then Q = {S1, S2, S5} is a set cover for S.
Self Assessment Questions
2. Given a graph G and a positive integer k, is there a –––––––––––– of G.
3. Complete Sub-graph of a given graph G is a sub-graph in which every
pair of vertices is ––––––––– in the graph.
14.4 Reduction, NP-Complete and NP-Hard Problems
We know that a problem is called NP-Complete if P has atleast one Non-
Deterministic Polynomial-time solution.
In this section we formally define the concept and then describe a general
technique of establishing the NP-Completeness of problems and finally
apply the technique to show some of the problems as NP-complete.
For the formal definition of NP-completeness, polynomial-time reduction, as
defined below, plays a very important role.
We have already discussed in previous units the reduction technique to
establish some of the problems as undecidable. The method that was used
for establishing undecidability of a language using the technique of
reduction, may be briefly described as follows:
Let P1 be a problem which is already known to be undecidable. We want to
check whether a problem P2 is undecidable or not. If we are able to design
an algorithm which transforms or constructs an instance of P2 for each
instance of P1, then P2 is also undecidable.
The process of transformation of the instances of the problem already
known to the undecidable to instances of the problem, (the undecidability is
to be checked), is called reduction.
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 259
Some-what similar, but, slightly different, rather special, reduction called
polynomial-time reduction is used to establish NP-Completeness of
problems.
A Polynomial-time reduction is a polynomial-time algorithm which constructs
the instances of a problem P2 from the instances of some other problems
P1.
A method of establishing the NP-Completeness (to be formally defined later)
of a problem P2 constitutes of designing a Polynomial time reduction that
constructs an instance of P2 for each instance of P1, where P1 is already
known to be NP-Complete.
The direction of the mapping must be clearly understood as shown below.
Reduction
PtimePolynomial
P 21
(Problem already known to be undecidable) (Problem whose NP-
completeness is to be
established)
Self Assessment Questions
4. ––––––––––– –––––––––– is a polynomial-time algorithm which
constructs the instances of a problem P2 from the instances of some
other problems P1.
5. A problem L is said to be –––––––––––––– if for any problem L1 in NP,
there is a polynomial-time reduction of L1 to L.
14.5 Establishing NP-Completeness of Problems
In general, the process of establishing a problem as NP-Complete is a two
step process. The first step, which in most of the cases is quite simple,
constitutes of guessing possible solutions of the instances, one instance at a
time, of the problem and then verifying whether the guess actually is a
solution or not.
The second step involves designing a polynomial-time algorithm which
reduces instances of an already known NP-Complete problem to instances
of the problem, which is intended to be shown as NP-Complete.
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 260
However, to begin with, there is a major hurdle in execution of the second
step. The above technique of reduction can not be applied unless we
already have established atleast one problem as NP-Complete. Therefore,
for the first NP-Complete problem, the NP-Completeness has to be
established in a different manner.
As mentioned earlier, Stephen Cook (1971) established Satisfiability as the
first NP-Complete problem. The proof was based on explicit reduction of the
language of any non-deterministic, polynomial-time TM to the satisfiability
problem.
The proof of satisfiability problem as the first NP-Complete problem, is quite
lengthy and we skip the proof. Interested readers may consult any of the
texts given in the reference.
Assuming the satisfiability problem as NP-Complete, the rest of the
problems that we establish as NP-Complete, are established by reduction
method as explained above. A diagrammatic notation of the form.
Fig. 14.5
The block diagram in 14.5 indicates that , assuming P as already
established to be NP-Complete, the NP- Completeness of Q is established
by a polynomial-time reduction from P to Q.
A scheme for establishing NP-Completeness of some the problems is
suggested by means of Figure 14.6 given below:
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 261
Fig. 14.6: A scheme for establishing NP-Completeness of
some the problems
Example: Show that the Clique problem is an NP-Complete problem.
Proof: The verification of whether every pairs of vertices is connected by an
edge in E, is done for different pairs of vertices by a Non-deterministic TM,
i.e., in parallel. Hence, it takes only polynomial time because for each of n
vertices we need to verify atmost n (n + 1) /2 edges, the maximum number
of edges in a graph with n vertices.
Next, we show that 3-CNF-SAT problem can be transformed to clique
problem in polynomial time.
Take an instance of 3-CNF-SAT. An instance of 3CNF-SAT consists of a set
of n clauses, each consisting of exactly 3 literal, each being either a variable
or negated variable. It is satisfiable if we can choose literals in such a way
that:
• Atleast one literal from each clause is chosen
• If literal of form x is chosen, no literal of form x is considered.
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 262
Fig. 14.7: Graph
For each of the literals, create a graph node, and connect each node to
every node in other clauses, except those with the same variable but
different sign.
Example: Show that the Vertex cover problem is an NP-Complete.
A vertex cover of an undirected graph G = (V, E) is a subset V’ of the
vertices of the graph which contains atleast one of the two endpoints of
each edge.
Fig. 14.8: Graph showing vertex cover problem
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 263
The vertex cover problem is the optimization problem of finding a vertex
cover of minimum size in a graph. The problem can also be stated decision
problem:
VERTEX-COV = {<G, k> /graph G has a vertex cover of size k}.
A deterministic algorithm to find a vertex cover in a graph is to list all
subsets of vertices of size k and check each one to see whether it forms a
vertex cover. This algorithm is exponential in k.
Proof: To show that Vertex cover problem NP, for a given graph
G= (V, E), we take V’ V and verifies to see if it forms a vertex cover.
Verification can be done by checking for each edge (u, v) E whether u V’
or v V’. This verification can be done in polynomial time.
Now, we show that clique problem can be transformed to vertex cover
problem in polynomial time. This transformation is based on the notion of
the complement of a graph G. Given an undirected graph G = (V, E), we
define the complement of G as G’ = (V, E’), where E’={(u, v)/(u, v) E}. i.e.,
G’ is the graph containing exactly those edges that are not in G. The
transformation takes a graph G and k of the clique problem. It computes the
complement G’ which can be done in polynomial time.
To complete the proof, we can show that this transformation is indeed
reduction: the graph has a clique of size k if and only if the graph G’ has a
vertex cover of size |V| – k.
Suppose that G’ has a clique V’ V with |V’| = k. We claim that V — V’ is a
vertex cover in G’ Let (u, v) be any edge in E’. Then, (u, v) E, which
implies that atleast one of u or v does not belong to V’, since every pair of
vertices in V’ is connected by an edge of E. Equivalently, at least one of u or
v is in V — V’, which means that edge (u, v) is covered by V — V’. Since (u,
v) was chosen arbitrarily from E’, every edge of E’ is covered by a vertex in
V — V’. Hence, the set V — V’, which has size |V| - k, forms a vertex cover
for G’.
Conversely, suppose that G’ has a vertex cover V’ V, where |V’| = |V| — k.
Then, for all u, v V, if (u, v) E’, then u V’ or v V’ or both. The contra
positive of this implication is that for all u, v V, if u V’ and v V’, then (u,
v) E. In other words, V— V’ is a clique, and it has size |V| - |V’| = k. For
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 264
example, the graph G(V, E) has a clique {A, B, E}. The complement of graph
G is given by G’ and have independent set given by {C, D, F}.
This proves that finding the vertex cover is NP-Complete.
In short, problems are designated as NP-complete if their solutions can be
quickly checked for corrections, and if the same solving algorithm used can
solve all other NP problems. They are the most difficult problems in NP in
the sense that a deterministic, polynomial-time solution to any NP complete
problem would provide a solution to every other problem in NP. Conversely,
if any one of them provably lacks a deterministic polynomial-time solution
then none of them has one. One example of an NP complete problem is the
subset sum problem which is: given a finite set of integers, determine
whether any non-empty subset of them sums to zero. A supposed answer is
very easy to verify for correctness, but there is no known efficient algorithm
to find an answer, that is, all known algorithms are impractically slow for
large sets of integers.
Self Assessment Questions
6. The process of establishing a problem as –––––––––is a two step
process
7. Example of an NP complete problem is the––––– ––––– ––––––.
12.6 Summary
P denotes the class of all problems, for each of which there is atleast
one known polynomial time Deterministic TM solving it.
The notation O provides asymptotic upper bound for a given function.
The notation provides an asymptotic lower bound for a given function
The Notation provides simultaneously both asymptotic lower bound
and asymptotic upper bound for a given function.
A Polynomial-time reduction is a polynomial-time algorithm which
constructs the instances of a problem P2 from the instances of some
other problems P1.
12.7 Terminal Questions
1. Explain the Constant Factor in Complexity Measure
2. Explain the Different Category of Problems
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 265
12.8 Answers
Self Assessment Questions
1. Hardware, software, environmental factors
2. k-colouring
3. Adjacent
4. A Polynomial-time reduction
5. NP-Hard
6. NP-Complete
7. Subset sum problem
Terminal Questions
1. If the time taken by one machine in executing a solution of a problem is
a polynomial (or exponential) function in the size of the problem, then
time taken by every machine is a polynomial (or exponential) function
respectively, in the size of the problem. Thus, functions differing from
each other by constant factors, when treated as time complexities
should not be treated as different, i.e., should be treated as complexity-
wise equivalent
2. Problems which cannot even be defined formally.
By a formal definition of a problem, we mean expressing in terms of
mathematical entities like sets, relations and functions etc., the
information concerning the problem, in respect of at least
a) Possible inputs
b) Possible outcomes
c) Entitles occurring and operations on these entities in the (dynamic)
problem domains. (Refer Section 14.3)
Fundamentals of Algorithms Unit 14
Sikkim Manipal University Page No. 266
Acknowledgement, References and Suggested Readings
1. Coremen. T. H., Leiserson. C.E. & Stein. C, (2002), Introduction to
Algorithms (second edition), Prentice-Hall of India.
2. Deo Narsing, 1986, ―Graph Theory with Applications to Engineering and
Computer Science‖, Prentice Hall India, New Delhi.
3. Horowitz. E. & Sahni. S., Fundamentals of Computer Algorithms,
(Galgotia Publications)
4. Kolman Bernard, Busby. R.C., Ross Sharon, 1999, ―Discrete
Mathematical Structures‖ PHI,.
5. Levis. H.R. & Papadimitriou. C.H., (1981), Elements of the Theory of
computation, PHI,
6. www.wikipedia.com.
top related