AMORTIZED ANALYSIS

Ajinkya Nene, Lynbrook CS Club. 04/21/2014

Definition

• Allows you to figure the worst-case bound for the performance of an algorithm (most useful for usaco)o In usaco, analyzing algorithms is usually just looking at number of calls

or for loops

• The average running time is sometimes different from the worst case running time when an algorithm uses randomization o For this case you use randomized analysis

• You can also get a best case running time (really only see these in computer science papers)

Worst Case Analysis

• Big O Notation. This represents the Worst Case Running time for some algorithmo O (n2) – represents that the algorithm will grow in quadratic time as n

increaseso O(n) – the algorithm will grow in linear timeo O(logn) – the algorithm’s worst case grows logarithmically (usually in

terms of base 2)o O (n*logn) – grows a bit faster than linear. Usually good in practice

• If you had a O (n^2 + n) you can say this is O(n^2) as n^2 grows much faster

For Loop Examples

for (int i = 0; i < n; i++) {

for (int ii = 0; ii < m; ii++) {

// do something

}

}

For each outer loop run, the inner loop will run m times. The outer loop runs n times. This is O(n*m)

for (int i = 0; i < n - 1; i++) {for (int ii = i + 1; ii < n; ii++) {

//do something}

}

Note the sequence of costs: (Assuming “do something” is O(1))(n-1) + (n-2) + … + (1) = (n-1)(n)/2 = (n2-n)/2.O((n2-n)/2) is O(n2)

Binary Search Example/Master Theoremint binary_search(int A[], int key, int imin, int imax) {

if (imax < imin) {

return KEY_NOT_FOUND;

else {

int imid = midpoint(imin, imax);

if (A[imid] > key)

return binary_search(A, key, imin, imid-1);

else if (A[imid] < key)

return binary_search(A, key, imid+1, imax);

else

return imid;

}

}

Proof that Binary Search is log(n)

• The recurrence relation for binary search is:o T(n) = T(n/2) + O(1) (this is the worst case scenario)

• Master Theorem: T(n) = aT(n/b) + f(n)o N is the size of the problem

o A is number of subrecursions

o n/b is size of subproblems

o F(n) cost of work done outside recursive calls

• In this case a = 1, b = 2 and f(n) = O(1) = O(nlog_b(a)).

• The master theorem also states that:o If f(n) = O(nlog_b(a) * logk(n)) (in this case k = 0), then T(n) = O(n log_b(a) * logk+1(n))

o So, in this case T(n) = O(log0+1 n) = O(log n)

METHODS FOR AMORTIZED ANALYSIS

First Method: Averaging Method

• Determines Upper Bound T(n) on the total cost of a sequence of n operations, then calculates the amortized cost to be T(n) / n

Example: Consider an Empty StackAnd a sequence of n Push, Pop, MULTIPOP operations

MULTIPOP(S, k)1.while not Stack-Empty(S) and k > 02. Pop(S);3. K k-1

• Worst Case Analysis: For N Operations Worst Case is O(n)

• Averaging Method Analysis: The upper bound is O(n) as the each operation takes O(1) time and there are n operations: O(n)/n = O(1). At most 1 pop/push is used each operation.

Second Method: Accounting Method• We assign artificial charges to different method

• Any overcharge for an operation on an item is stored (in an bank account) reserved for that item.

• Later, a different operation on that item can pay for its cost with the credit for that item.

• The balanced in the (bank) account  is not allowed to become negative.

• The sum of the amortized cost for any sequence of operations is an upper bound for the actual total cost of these operations.

• The amortized cost of each operation must be chosen wisely in order to pay for each operation at or before the cost is incurred.

Example: Accounting MethodExample: Consider an Empty StackAnd a sequence of n Push, Pop, MULTIPOP operations

MULTIPOP(S, k)1.while not Stack-Empty(S) and k > 02. Pop(S);3. K k-1

Actual Costs: Amortized Costs:Push: 1 Push: 2Pop : 1 Pop: 0Multipop : min (k, s) Multipop: 0

• Suppose $1 represents a unit cost.• When pushing a plate, use one dollar to pay the

actual cost of the push and leave one dollar on the plate as credit.

• Whenever POPing a plate, the one dollar on the plate is used to pay the actual cost of the POP. (same for MULTIPOP).

• By charging PUSH a little more, do not charge POP or MULTIPOP.

• The amortized cost is thus O(n) as it is O(1) per operation• The base condition holds that the balance never goes

negative.

Case Study: Union Find

• Find: Determine which subset a particular element is in

• Union: Join two subsets together into one

• To represent this structure (usually set a fixed representative element for each set)

• Sets are stored as trees. Root is representative (start of with N separate sets)

• Union: merges two trees. Each element has parent (root’s are there own parents.

• Need to use path compression and union by rank for faster worst-case time

Ackerman Function

• Ao (x) = x + 1

• Ak + 1 (x) = Axk + 1 (x)

• Where Aki is the the i-fold composition of Ak.

• Basically Aki = Ak …. Ak i times

Approach

• Define the rank of a node: rank (u) = 2 + height (Tm(u)) where Ti(u) represents the subtree rooted at u at time i in the execution m union and find instructions

• Define a new function &(u) for which it is defined to be the greatest value k for which rank (parent (u)) >= Ak (rank(u))

• For n >= 5: the maximum value &(u) can have is the inverse Ackerman function, α(x) – 1. If &(u) = ko n > floor of (log (n)) + 2 >= rank (parent (u)) >= Ak (rank (u)) >= Ak (2).

o Thus, α(n) > k

Continued…

• Outline: All union functions are constant time giving O(m). Next, tally the separate the number of time units apportioned to the vertices and to the find instructions and show that in each case the total is O((m + n) α(n)).

• To get O(m α(n)): note that there are most α(n) units charged to find instructions. This is because at most one unit is charged for the α(n) possible values of &() since for each k the last vertex is on &(k) = x, and only it gives it time unit charged to fine. There are at most m find instructions giving O(mα(n)..

• To get the O(nα(n)) part you count the number of charges to some vertex x over the entire computation.