session 11: data structures and collections

FEN 2011-02-05 UCN T&B: IT Technology 1

Session 11: Data Structures and Collections

Lists (Array based, linked)

Sorting and SearchingHashing

TreesSystem.Collections.Generic

Lists• A data structure where elements are

organised by position (index).• ArrayList (List) and LinkedList• Sometimes lists are called sequences.


One fixed size segment in memory.

Each element has a reference to the next element. Hence elements may

be allocated at different memory locations.

numList

ArrayList

• Array-based:– Fixed size (statically allocated).– Always occupies maximum memory.– May grow or shrink dynamically, but that requires halting the

application and allocation of a new array. • Direct access to elements by position (index), otherwise

searching is required.• Inserting and deleting in the middle of the list requires moving

(many) elements.


Linked Lists (LinkedList)• A linked list consists of nodes representing elements.• Each node contains a value (or value reference) and

a reference (pointer) to the next element:


• The list it self is represented by a reference to the first element, often called head

• The next-reference of the last element is usually null• The linked list is dynamic in size: it grows and shrinks

as needed.• Access by position is slow (may require traversing

the hole list).

• See this Java Example.


Linked Lists (LinkedList)

Figure 4.1a) A linked list of integers; b) insertion; c) deletion


Implementation

private class Node { private object val; private Node next; public Node(object v, Node n) { val= v; next= n; }


public object Val { get{return val;} set{val= value;} } public Node Next { get{return next;} set{next= value;} }}

Class Node

Linked Implementation of ADT list

class LinkedList{ private class Node //… Node head,tail; int n;//number of elements public LinkedList() { head= null; tail= null; n= 0; } public int Count { get { return n; } }


public void AddFront(object o){ Node tmp = new Node(o, null); if (Count == 0)//list is empty tail = tmp; else tmp.Next = head; head = tmp; n++;}

public void Print(){//for debugging... Node p = head; //start of list while (p != null) //while not end of list {

Console.WriteLine(p.Val); //print current valuep = p.Next; //set p to next element of the list

}}


Traversing a Linked List

tail

head p

public int FindPos(object o){ //Returns the position of o in the list (counting from 0). //If o is not contained, -1 is return. bool found = false; int i = 0; Node p = head; while (!found && p != null){ if (p.Val.Equals(o)) found = true; else{ p = p.Next; i++; } } if (found) return i; else return -1;}


Finding a Position in a Linked List

Dynamic vs. Static Data Structures

• Array-Based Lists:– Fixed (static) size (waste of memory).– May be able to grown and shrink (ArrayList), but this is very

expensive in running time (O(n))– Provides direct access to elements from index (O(1))– May be sorted. Hence binary search gives fast access (O(log n))

• Linked List Implementations:– Uses only the necessary space (grows and shrinks as needed).– Overhead to references and memory allocation– Only sequential access: access by index requires searching

(expensive: O(n))


numList

Linked List - Variants• Using a tail-reference


• Using a dummy head node


• Circular


Doubly Linked List


…operations become more complicated …


The Full Monty….(LinkedList)


Search Trees:Dynamic Data Structures with Fast Search

• Binary Trees• Binary Search Trees• General Trees (Composite Pattern)• Balanced Search Trees (2-3 Trees etc.)• B- Trees (external, database index)



Terminology

• General trees:– leaf/external node/terminal– root– internal node– siblings, children, parents, ancestors, descendents– sub trees – the depth or height of a node = number of ancestors– the depth or height of a tree = max depth/height for

any leaf


Binary Trees

• A binary tree can be defined recursively by– Either the tree is empty– Or the tree is composed by a root with left

and right sub trees, which are binary trees themselves

• Note: contrary to general trees binary trees– have ordered sub trees (left and right)– may be empty


Reference Based Implementation


Figure 10.9Traversals of a binary tree: a) preorder; b) inorder; c) postorder


Binary Search Trees• Value based container:

– The search tree property:• For any internal node: the value in the root is greater

than the value in the left child• For any internal node: the value in the root is less than

the value in the right child– Note the recursive nature of this definition:

• It implies that all sub trees themselves are search trees• Every operation must ensure that the search tree

property is maintained (invariant)


Example:A Binary Search Tree Holding Names


Balance Problems (skewed tree):

• Values are inserted in sorted order


InOrder:Traversal Visits Nodes in Sorted Order


Efficiency• insert• retrieve• delete

– All depends on the depth of the tree

– If insertions and deletions are uniformly distributed, then the tree will eventually grow skewed

• O(log n) / O(n)• O(log n) / O(n)• O(log n) / O(n)


Solution:Balanced Search Trees

• Trading time for space:– In worst case additional space in

O(n) is required; but:– retrieve, insert and delete in

O(log n) – also w.c..• Principle:

– A node may hold several keys (n) and has several children (n+1)

– A node must be at least half filled (n/2 keys)

– Insert and delete can be performed, so the tree is kept balanced in O(logn)

2-3-tree:k = 2

FEN 2011-02-05 UCN T&B: IT Technology29

2-3-Trees (n=2)

FEN 2011-02-05UCN T&B: IT Technology

30

Retrieve

• Search using the same principle as in binary search trees:– Search the root– If not found, the search recursively in

the appropriate sub tree– Performance is proportional to the

height of the tree– Since the tree is balanced: O(log n)


31

Insertion

• The insert algorithm must ensure that the 2-3-tree properties are conserved. It goes like this:– Search down through the tree to the appropriate leaf node and

insert– If there is room in the leaf, then we are done– Otherwise split the leaf node into two new leafs and move the

middle value up into the parent node– If there is no room in the parent, then continue recursive until a

node with room is reached, or– Eventually the root is reached. If there is no room in the root,

then a new root is created, and the height of the tree is increased

– Performance depends on the height of the tree (searching down through the tree + in worst case a trip from the leaf to the root rebalancing on the way up)

– That is: O(log n)


32

Inserting 39 (there is room)


33

Inserting 38 (there is no room in the leaf)

• Insert any way,• Split leaf and • Move middle value up


34

Inserting 37 (there is room)


35

Inserting 36 (there is no room)

Split and move up

Split and move up


36

Inserting 35 , 34 and 33 (there is room)


37

Deletion• Like insertion – just the other way around:-)

– find the node with the value to be deleted– If this is not a leaf, the swap with its inorder

successor (which is always a leaf - why?), and remove the value

– If there now is too few values (< n/2) in the leaf, then merge the node with a sibling and pull down a value from the parent node

– If there now is too few values in the parent, then continue recursively until there are enough values or the root is reached

– If the root becomes empty, the remove it and the height of the tree is decreased

– Performance: once again: down and up through the tree : O(log n)


38

Balanced Search Trees• Variants:

– 2-3-trees– 2-3-4-trees– Red-Black-trees– AVL-trees– Splay-trees….

• Is among other used for realisation of the map/dictionary/table ADT

• In Java.Collections: TreeMap and TreeSet

An Alternative to Sorting and Searching: Hashing

• Keys are converted to indices in an array.

• A hash function, h maps a key to an integer, the hash code.

• The hash code is divided by the array size and the remainder is used as index

• If two or more keys gives the same index, we have a collision.


39

Collision Handling

• Avoiding collisions:– Use a prime as the size of the array:

• Trying to store keys with hash codes 200, 205, 210, 215, 220,.., 595 in an array of size 100 yields three collisions for each key.

• But an array with size 101 results in no collision.– Choose a good hash function:

• this is a (mathematical) discipline of its own


40

Collision Handling• Probing is searching for a near by free slot in the array.• Probing may be:

– Linear(h(x)+1, +2, +3, +4,…)

– Quadratic(h(x)+1, +2, +4, +8,…)

– Double hashing– …


41

Chaining• The array doesn’t hold the element itself, but a reference to a

collection (a linked list for instance) of all colliding elements.• On search that list must be traversed


42

Efficiency of Hashing• Worst case (maximum collisions):

– retrieve, insert, delete all O(n)• Average number of collisions depends on the load

factor, λ, not on table sizeλ = (number of used entries)/(table size)

– But not on n.• Typically (linear probing):

numberOfCollisionsavg = 1/(1 - λ)• Example: 75% of the table entries in use:

– λ = 0.75:1/(1-0.75) = 4 collisions in average

(independent of the table size).FEN 2011-02-05UCN T&B: IT

Technology43

When Hashing Is Inefficient

• Traversing in key order.• Find smallest/largest key.• Range-search (Find all keys

between high and low).• Searching on something else than

the designated primary key.


44

See this Java Example


.NET 2:System.Collections.Generics

ICollection<T>

IList<T> LinkedList<T> IDictionary<TKey, TValue>

List<T>Dictionary

<TKey, TValue>SortedDictionary<TKey, TValue>

Index ableArray-based Balanced

search tree Hashtabel

(key, value) -pair

interface:

(i.e. Dictionary)

Specification

class Appl{

----

IDictionary d;

-----

m= new XXXDictionary();

Application

class:

Dictionary

SortedDictionary

----

ADT Data Structures and Algorithms

Select and use ADT, i.e.:

Dictionary

Select and use data structure, i.e. SortedDictionary

Knowledge of.

Read and write (use)

specifications

Learning Goals


Exercises• Consider some of our programmes (Banking, Forest,

AndersenAndAsp, for instance).

• Would it be better to use some other collection instead of List?

• Try to chance the implementation in one or more of your programs, so, for instance a hash table is used.

• Implement InsertAt(int index, object element) and RemoveAt(int index) on the linked list.


47

48

Time Complexity – Big-”O”

• Investigation of the use of time and/or space of an algorithm

• Normally one looks at– Worst-case (easer to determine)– Only growth rates – not exact measures– Counts the number of some “basic

operations” (a computation, a comparison of to elements etc.).

FEN 2011-02-05 UCN T&B: IT Technology

49

Big-O notation:

• The complexity of an algorithm is notated with “Big-O”– O(f(n)), n is the size of the problem (number of input

elements, for instance), f is a function that indicates the efficiency of the algorithm, for instance n (the running time is linear in problem size)

– Big-O: is asymptotic (only holds for large values of n)– Big-O: only regards most significant term– Big-O: ignores constants


50

Examples public int sum (int a, b) {

int sum;sum = a + b;return sum;

}

What is the basic operation

?

public int sum (int[] a) {

int sum= 0;for(int i= 0; i<a.length; i++)

sum= sum+a[i];return sum;

}

What is the basic

operation?

O(1)

O(n)


51

Searching

• Linear search in a sequence with n elements: O(n) (why?)

• Binary search in a sorted sequence with n elements: O(log n) (why?)

• What about sweep algorithms?• Complexity O(n)


52

Constant and Linear complexity

• Consider an algorithm working on a sequence of length n:– If running time is independent of n, then the time

complexity is constant or O(1)– If we (in worst case) has to do some thing to every

element, then the time complexity is linear or O(n)– There are other possibilities:

• Quadratic O(n2) (some sorting algorithms), O(nlogn) (better sorting algorithms, logarithmic O(log n) (binary search), exponential O(2n) (“difficult” problems like the Towers of Hanoi – more on 3rd semester)


53

Does it matter…?

“år” means “year”

“døgn” means “day”

NOTE

Assuming one basic operation in 1 ns (one billion operations pr. sec. – GHz)


54

A Rule of Thumb• For each nested loop the complexity must be

multiplied with a factor n:

for(int i = 0; i < n; i++) O(n){…}

for(int i = 0; i < n; i++) { for(int j = 0; j < n; j++) O(n2)

{…}}


55

O(1)

public add(int n) {lastIndex++;data[lastIndex] = n;

}

Both statements are basic and their performanceis independent of the size of the array


56

O(n)

public void insert(int i, int newInt) {// make room for newIntfor(int j = data.length; j > i; j++)

data[j] = data[j-1];data[i] = newInt;//insert newInt

}

The for-loop indicates a time complexity of O(n)


57

O(n2)

public void sort() { for (int j = 0; j < numbers.size(); j++){ for (int i = 0; i < numbers.size()-1; i++){ if (numbers.get(i) > numbers.get(i+1)) swap(i,i+1);//swaps elements i and i+1 }//end for }//end for }//end sort

Nested for-loops suggestO(n2)


session 11: data structures and collections

Documents