Algorithms and their Applications

CS2004 (2012-2013)

Dr Stephen Swift

1.2 Introduction to Algorithms

Introduction to Algorithms Slide 2

IntroductionIn this topic we will look at:

What is a computer? What is computer science? What is an algorithm?

Other concepts we should already know will also be discussed:

SoftwareProgramming LanguageImplementation

Introduction to Algorithms Slide 3

Where Did it all Start? – Part 1How did the computer come about?In ~1900, a very famous German Mathematician called David Hilbert (1862-1943) listed 23 unsolved mathematical problemsHe later refined problem number ten to ask “is there a way of determining if an arbitrary first-order symbolic logic statement is true or false?”(Do not worry what this means!)

Introduction to Algorithms Slide 4

Where Did it all Start? – Part 2This became known as the “Entscheidungsproblem” or “Decision Problem”In 1936 Alan Turing (1912-1954) created two concepts called a “Turing Machine” and a “Universal Turing Machine”These were used to show that the “Decision Problem” was impossible to solve

Introduction to Algorithms Slide 5

Where Did it all Start? – Part 3In 1939 Alan Turing used his ideas to create a machine called “The Bombe” which helped crack the German enigma codeIn 1948, the first computer (called the Baby) and associated computer program were developed at Manchester University, UK, based on Turing’s ideas

Introduction to Algorithms Slide 6

Computer ScienceComputer science is the study of computation

A discipline that studies computable problems and computational structures A discipline that involves the understanding and design of computers and computational processes

Is it a mathematical, scientific or engineering discipline?

Interdisciplinary

Introduction to Algorithms Slide 7

What is ComputationComputation is the procedure of calculating i.e. determining something by mathematical or logical methods

“In natural science, Nature has given us a world and we’re just to discover its laws. In computers, we can stuff laws into it and create a world”, by A. Kay

Computability is about what computers can do and cannot doCharacterises problems that can be solved AlgorithmicallyExhibit problems that cannot be solved (e.g. the proof of the insolvability of the “halting problem”)

Introduction to Algorithms Slide 8

So What is a Computer and Computer Program?A computer is a device for carrying out certain types of algorithmsAn algorithm (in our context) is a set of steps for solving a problemA computer programming language is a “language” for describing algorithms to computersA computer program is (usually) a description of a single algorithm and/or problemComputer software is a collection of programs that carry out a similar task or related

E.g. Windows

Introduction to Algorithms Slide 9

Why use a Computer?A computer can perform many mundane tasks much faster than people,

E.g. adding up a large number of figuresSome tasks can only be done by computers

E.g. high precision roboticsHowever there are many tasks that computer’s are no good at

E.g. critically appraising a film

Introduction to Algorithms Slide 10

What is an Algorithm?

Input OutputAlgorithm

An algorithm is any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values as output

An algorithm can be seen as a tool for solving a well-specified computational problem

Introduction to Algorithms Slide 11

What is Implementation?

Implementation is the task of turning an algorithm into a computer programDepending on the level of abstraction of the description of the algorithm, implementation can be anything between a straightforward translating task and a lengthy, difficult job

Introduction to Algorithms Slide 12

Programming Languages – Part 1

Computers really just understand binary, which is just sequences of zeros and onesA computer language is a method of allowing us (the user) to describe to a computer what we want it to do

I.e. The specification of an AlgorithmIt is then translated (by the computer!) into binary so the computer can then understand it

Introduction to Algorithms Slide 13

Programming Languages – Part 2

Computer programming languages are similar to spoken languages

They have structure and grammar They have words and phrases A incorrectly written program will not

be understood by the computer and will result in an error

Programmers usually use previously written programs if possible and combine them to solve new problems

I.e. They try not to “reinvent the wheel”!

Introduction to Algorithms Slide 14

Why Study Algorithms? – Part 1So why do we want to study algorithms?Some algorithms that solve the same problem can differ massively in their efficiencyComputers may be getting faster but they are not infinitely fast!

For example, sorting a list of billions of numbers could take a computer a very long time

If we do not make the algorithms we write efficient then they may take too long to run

If we do not analyse our algorithms we do not know how long they will take

I am currently trying to make a program more efficient (by changing the algorithms and developing new algorithms) which will take about 4 years to run in its current state! I am hoping to get this down to around two weeks!!!

Introduction to Algorithms Slide 15

Why Study Algorithms? – Part 2What makes the difference between a truly skilled programmer and an average one?

Strong algorithmic knowledge and techniqueIt might be that you can perform certain programming tasks without knowing much about algorithmsHowever with a good grounding in algorithms, you can accomplish much more

Introduction to Algorithms Slide 16

What is the Analysis of an Algorithm?

This is an indication of the time a computer will take to solve a problem given the size of the problemTime is abstract, and is usually measured in operations

Multiplications, additions, swaps This allows us to compare two algorithms that

try and solve the same problem Independent of the speed of a computer

Also known as the computational complexity of an algorithm

Introduction to Algorithms Slide 17

Algorithm Example – Part 1

You have 70 coins (N=70) which are all “supposed” to be gold coins of the same weight, although you know that one coin is fake and weighs less than the othersYou have a balance scale; you can put any number of coins on each side of the scale at one time, and it will tell you if the two sides weigh the same, or which side is lighter if they don’t weigh the same

Introduction to Algorithms Slide 18

Algorithm Example – Part 2

Outline an algorithm for finding the fake coinHow many times do you have to weigh?Easiest approach is not necessarily the best approach!Ideally, we want to design an algorithm that will take the shortest period of timeWe want an algorithm that is efficient, for example, what if we are then given 10,000 (N=10,000) coins and told that one is fake?

Introduction to Algorithms Slide 19

Example Solution One – Part 1

Take two coins at a time, one on each side of the scales:

If weight(Left) = weight(Right) Then Take the next two coinsElse Coin at the lighter side is fakeEnd IfRepeat the above until either the fake coin is found or you have checked all the coins

Introduction to Algorithms Slide 20

Example Solution One – Part 2

In the worst case, e.g. the fake coin is the last or the last but one, the IF Then-Else statement will have to run N/2 (35) timesIn the best case, e.g. the fake coin is the first or second, the same statement will only have to run once

Introduction to Algorithms Slide 21

Example Solution Two – Part 1

Divide-and-Conquer:

Split the set of coins into two subsets of equal size; N/2 go to the left side of the scale, N/2 go to the right sideIf weight(left) ≠ weight(right) Then

The lighter side contains the fake coinSplit the lighter N/2 coins …

ElseNeither side contains the fake coin

End If

Introduction to Algorithms Slide 22

Example Solution Two – Part 2

Essentially this is a form of Binary Search

This will be discussed later in the moduleThe number of splits is equivalent to the number of levels (k) in the corresponding complete binary tree (we will cover this later)

Since there is only one comparison on each level, the worst case is to conduct log2(N+1) comparisonsNote: 2x=y, log2(y) = x, e.g. log2(8) = 3

So when N=70 this will be less than 7 in total

Introduction to Algorithms Slide 23

Example Solution ComparisonSolution 1: F1(N) = N/2, F1 ∈ O(N)

Solution 2: F2(N) = log2(N+1), F2 ∈ O(log2(N))

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 100001

10

100

1000

10000

F1F2

N

Num

ber

of

Com

pari

sons

Introduction to Algorithms Slide 24

The Analysis of Algorithms

So there can be many competing algorithms for solving a given problemThis module will enable you to gain experience in analysing and comparing different algorithmsLater on in the module we will discuss that there are some problems that are impossible to solve computationally

Introduction to Algorithms Slide 25

Next TopicLecture

We will next look at algorithm analysis in more detail

LaboratoryThere are no formal laboratories this weekThis weeks laboratory is a worksheet for you to attempt in your own time (practice)