Incremental Computation
AJ Shankar
CS265 Spring 2003
Expert Topic
The idea
Sometimes a given computation is performed many times in succession on inputs that differ only slightly from each other
Let’s optimize this situation by trying to use the previous output of the computation in computing the current one
Iteration and recursion are special cases of “repeated computation” – so lots of gains to be had
A simple exampleCompute the successive sums of each m-element window in an n-element array (with m < n)
for (int i = 0 ; i < n-m ; i++) { for (int j = i ; j < m ; j++) {
sum[i] += ary[j]; }
}
This is an O(n2) algorithm.
A simple example, continued
5
8
3
1
6
9
5
4
Consider a four-element window. The naïve approach adds up each group of four numbers from scratch.
However, all we need to do to compute a new window is subtract the number that is leaving the window and add the number that is entering it!
A simple example, continued
So we can incrementally compute each successive window as follows:
// compute the first window from scratchfor (int j = 0 ; j < m ; j++) {
sum[0] += ary[j]; } // incrementally compute each successive windowfor (int i = 1 ; i < n-m ; i++) {
sum[i] = sum[i-1] - ary[i-1] + ary[i+m-1]; }
This new algorithm is O(n)!
Our goal
Let the user write natural, maintainable code
Discover repeated computations that can be “incrementalized”
Generate an incremental version of these computations that is faster than the original
Do this as automatically as possible
What this problem is not
Incremental algorithms: algorithms explicitly designed to accommodate incremental changes to their inputs. We’d like to study the automated incrementalization of existing code.
Incremental model of computation: code is rendered incremental at run-time via function caching, etc. Relies on a run-time mechanism and therefore never explicitly constructs incremental code that can be run by conventional means.
Some history
First introduced by Early in 1974 as ‘iterator inversion’
Explored further by Paige, Schwartz, and Koenig (1977, 1982) Very high level (sets) Called ‘formal differentiation’ For the first time, discussed the possible
automation of the algorithm Strength reduction
More specific notion of replacing costly operations with cheap ones (say * by +)
Incremental computation
Described in a series of papers by Yanhong Annie Liu et. al. through the 90s
First systematic approach to incrementalizing programs written in a common functional language
Proof of correctness (not covered here)
Formal definitionLet f be a program and let x be an input to f.
Let y be a change in the value of x, and let be a change operation that combines x and y to produce a new input value xy.
Let r = f(x): the result of executing f on x.
Let f’(x,y,r) be a program that computes f(xy) such that
1. f’ computes faster than f for almost all x and y
2. f’ makes non-trivial use of r
Then f’ is an incremental version of f.
An illustrative case Let sort(x) be selection sort; it takes
O(n2) time for x of length n Let sort’(x,y,r) – where r is sort(x) –
compute sort(cons(x,y)) by running merge sort on cons(x,y); takes O(n log n) time. Not incremental!
Let sort’’(x,y,r) compute sort(cons(x,y)) by inserting y in r in O(n) time. Non-trivial use of r; hence, incremental.
A general systematic transformational approach Given f and , derive an incremental
program that computes f(xy) using1. The value of f(x)
(Liu, Teitelbaum 1993)2. The intermediate results of f(x)
(Liu, Teitelbaum 1995)3. Auxiliary information of f(x)
(Liu, Stoller, Teitelbaum 1996) Each successive class of information allows
for greater incrementality than the previous one
P1. Using the previous result
1. Given f(x), introduce f’(x,y,r)2. Unfold
• Expand f using the definition of the operator
3. Simplify• Use basic rewrite rules like car(cons(a,b)) = a
4. Replace using cached result• Substitute r when we see f(x) in the expanded
function
5. Eliminate dead code
An examplesum(x) =
if null(x) then 0else car(x) + sum(cdr(x))
x y = cons(y, x)
1. Introduce f’
2. Unfold
3. Simplify
4. Replace
5. Eliminate
sum’(x,y,r) = sum(cons(y,x))
x y = cons(y, x)
sum’(x,y,r) = if null(cons(y,x)) then 0else car(cons(y,x)) + sum(cdr(cons(y,x)))
x y = cons(y, x)
sum’(x,y,r) = if (false) then 0else y + sum(x)
x y = cons(y, x)
sum’(x,y,r) = y + r
x y = cons(y, x)
sum’(y,r) = y + r
x y = cons(y, x)
sum(cons(y,x)) takes O(n) time
sum’(y,r) takes O(1) time and one unit of space
P2. Using intermediate results
In computing f(x), we might calculate some intermediate results that would be useful in computing f(xy) but are not retrievable from r
Recall the successive sums problem: f(2..6) = 6 + f(2..5) // intermediate result from f(1..5)
So let’s keep track of all the intermediate results
…But there might be a ton of them!
The cache-and-prune method
Stage 1: Construct f* that extends f to return r and all intermediate results
Stage 2: Incrementalize f* to get f*’ as per P1
Stage 3: Figure out which results are necessary and prune out the rest from f*’, yielding f^’
Our old friend, Fibonaccifib(x) = if x 1 then 1
else fib(x-1) + fib(x-2)
Fibonacci, continued
Note that the standard P1 method will not work – we still have fib(x-2)
So, cache-and-prune: Stage 1: Construct a function that, if run,
would return an exponential-sized tree Stage 2: Incrementalize this function,
noticing that both fib(x-1) and fib(x-2) can be retrieved from the cached tree
Stage 3: Remove the other unnecessary results (the rest of the tree)
Fibonacci, continuedThe final incrementalized version of fib(x) is
fib^’(x) = if x 1 then <1,0> // pairelse if x = 2 then <2,1>else let r = fib^’(x-1) in
<1st(r) + 2nd(r), 1st(r)>
The old fib(x) took O(2n) time, whereas this version takes O(n) time.
P3. Auxiliary information
Let’s go even further and discover information that would be useful for computing f(xy) but that is never computed in f(x)
Two-phase method: Identify computations in f(xy) done only on x that
cannot be retrieved from any existing cached data Determine whether such information would aid in
the efficient computation of f(xy); if so, compute and store it
Most of this can be done using techniques from P1 and P2, respectively
A (complicated) examplecmp(x) = sum(odd(x)) prod(even(x))
x y = cons(y, x)
When we add y, the odd and even sublists are swapped – we must now take the product of what we used to sum and vice-versa
Therefore, the results (even intermediate ones) of the previous computation are useless!
So we really want to compute and save the values of sum(even(x)) and prod(odd(x)) too, which can be done with a single addition or multiplication each
This is auxiliary information (see board)
Unrolling cmpcmp(x) = sum(odd(x)) <= prod(even(x))odd(x) = if null(x) then nil else cons(car(x), even(cdr(x)))even(x) = if null(x) then nil else odd(cdr(x))
Unroll…cmp(cons(y,x)) = sum(odd(cons(y,x))) prod(even(cons(y,x)))
cmp(cons(y,x)) = sum(
if null(cons(y,x)) nil else cons(car(cons(y,x)), even(cdr(cons(y,x))))
) prod(if null(cons(y,x)) then nilelse odd(cdr(cons(y,x)))
)
Simplify…cmp(cons(y,x)) =
sum(cons(y,even(x)) prod(odd(x))
Identify even(x) and odd(x) as computations that only depend on x that can be incrementalized.
cmp*(x) = let v1 = odd(x), u1 = sum(v1),
v2 = even(x), u2 = prod(v2) in
<u1 u2, u1, u2, sum(v2), prod(v1)>
cmp’(y,r) = <y + 4th(r) 5th(r), y + 4th(r), 5th(r), 2nd(r), y * 3rd(r)>
Optimizing cmp, continued
res sum(odd) prod(even) sum(even) prod(odd)
Further work
CACHET (1996) An interactive programming environment
that derives incremental programs from non-incremental ones
Transformations directly manipulate the program tree
Use annotations to preserve user-specified stuff and to give direction to the optimizer
Basically a proof of concept
Further work
Using incrementalization to transform general recursion into iteration (1999) Find base and recursive cases of f For each recursive case, identify an input
increment (f(x) = 2*f(x-1))and derive an incremental version
Form the iterative program using some generic iteration constructs as appropriate
Recursion to iteration, con’t The tail recursion optimization may in fact
produce slower code than the original recursive function! Multiplying small numbers is faster than
multiplying large ones, etc. So far we can generate an additional function
that computes an incremental result given a previous result r
This work handles the inlining of the iterative computations, including the hairy bits with multiple base and recursive cases Use associativity, loop contraction, redundant test
elimination, pointer reversal