Recursion, Complexity, and Sorting

By Andrew Zeng

Table of Contents

Quicksort Mergesort Recursion Algorithms and Complexity Review Questions

Overview

In this slideshow we will cover:– The implementation of sorts– Speed of the sorts– The benefits and disadvantages of recursion– The analysis of algorithms– Big-O notation– Review problems

Algorithms and Complexity

Big-O notation– It is an abstract function used to represent time

complexity of an algorithm or another function with parameter size n

– Examples are O(n3),O(log n), and O(an). (Note than a can be different values, such as 2 and 3, and be different complexities because they differ by a non constant factor of O(1.5n). Also note n2 is the same as n2/100)

Algorithms and Complexity cont.

Big-O analysis– Simple assignment actions are O(1)– The running time of a function calls is the Big-O of

its body.– Method parameters are O(1)– Logic tests such as in if/else statements are in

O(1). The complexity of the if/else chain is the complexity of the worst case

– Add together all of the complexities or multiply if in a loop to get the Big-O notation

Recursion

In computer science, recursion is when a method or function calls itself, much like a recursive series in mathematics.

This requires stack memory to keep track of all the methods that have been called

Recursion must have a base case, which will stop the function from infinitely calling itself.

For example, a selection sort could call itself to sort from 0 size – 1, then call itself to sort from 0 size – 2, then find the greatest and put that on the top of the sorted array

Pros and cons of recursion

Imagine the sort previously mentioned. – If the size was 1 billion, it would

require 1 billion method calls, so the sort would run out of memory

Therefore it is almost always better to use iterative solutions whenever possible.

Recursion can be useful when– It is difficult to divide the problem into

an iterative approach– It is as fast and as memory consuming

as an iterative approach because it will probably make code clearer

Now imagine a stack a millions times as big.

Example of recursion

Click to continue Click this if you are bored of clicking

Not having a base case is like falling forever without an edge to hang on to in Portal.

Quicksort

Its complexity is O(n2) in its worst case because, by choosing the worst pivot every time (the highest element value), it basically turns into a bubble sort

On average it is O(n log n)

Quicksort cont.

Generally what happens is you choose a pivot. Then all the values smaller than the pivot are moved to the left and the bigger ones to the right.

This is the most simple version. The algorithm can be modified also to sort items ascending or descending.

Merge Sort

Background– Developed by the mathematician John von

Neumann in 1945, who invented the computer– Based on the divide and conquer algorithm

idea– Its complexity is O(n log n) in both best and

worst case, but it is a slower O(n log n) on average than that of quicksort. Sometimes, though, Mergesort can be tied with quicksort

Merge Sort cont.

The general code for the sort is

– If the list size is less than 2, it is sorted and return the sublist

– Otherwise divide the list in two half the size lists and recursively sort the sublists

– Merge the sublists together and return the new list Here is a visual sample of Mergesort

Pros and Cons

It has good speed and is very consistent Not really much, except that with a huge array, memory

problems can arise from an overly large stack– Its total recursive calls is O(2n-1)compared with the

quicksort n It also makes fewer comparisons than quicksort except in rare

cases

Summary

We have found in this presentation that: – Recursion should be used only when necessary.– The speed of the sorts Quicksort and Mergesort,

among others.– How to analyze a algorithm, recursive or iterative.

Review Questions

#1 – Analyze the Big-O of the following code:

A: O(n2 log n) B: O(n*n!) C: O(nn) D: O(n3)

Review Questions

#2 – Which of the following is the fastest algorithm?

– A: Quicksorting an array of size 5n– B: Mergesorting an array of size 5n– C: Bubble sorting an array of size n– D: It depends

#3 – Analyze the result of the following code on the left for a = 3. (Spaces = line feeds)

– A: c++ c++ c++ .net .net .net– B: c++ c++ c++ .net .net– C: c++ c++ c++ .net c++ c++ c+

+ .net .net– D: None of the above

Review Answers

#1 – Start with sort(), because that is the first method called, it calls inorder() and negates it which is O(1).

inorder() is clearly O(n), because there a single equality test done at most n times so its O(n * 1 = n).

shuffle() is O(n) because all of the action in the loop are O(1) and the shuffle is done n times.

Add loop overhead of inorder() and shuffle() and you get O(n) because the O(1) is of little consequence as n gets big.

Now analyze the while loop. The probability that all the elements are in order is 1/n * 1/(n-1) * … ½ * 1. So it has probability 1/n! it is right, so the expected amount of times it takes on average is n!, so the complexity is O(n*n!). In this case we do not consider the worst case because it may never randomly get to the answer.

Review Answers cont.

#2 – The answer is D: it depends. One way to approach this is that for all

cases, Quicksort on average faster than Mergesort, but can be slower.

On the other hand, bubble sort is slower for the same size array as the other two sorts, but the size is smaller so for small n it is faster.

Review Answers cont.

#3 – The answer is C. For a = 3, the method is recursively called for a = 2, which then calls a = 1, and then a = 0.

At a = 0, the method tests to see the value of a and breaks out of the recursive sequence.

Then at a = 1, c++ is printed once in the 1st loop and .net 0 times in the 2nd.

Then the method finishes and a = 2 executes, and acts almost like a = 1 except for the number of times the words are printed, which both increase by 1 as a goes up by 1.

This is also true for a = 3.

Code writing exercise

Write a recursive and iterative algorithm to create anagrams of a word. Imagine a word as collection or characters. Let n = size

Assume that there are no repeats of a letter.– Recursive:

Starting with the whole collection do this size of word times– Create anagrams of the sub-word (the whole word excluding the

first letter, then display the word when if it has reached a size of 2 (the base case is size 1, because there are no permutations except itself)

– Then rotate the word by moving the first letter to the end– Iterative:

Starting at element 0, swap adjacent elements until you reach the end Repeat this process size! / (size -1) times.