backward thinking confessions of a numerical analyst keith evan schubert
TRANSCRIPT
Backward Thinking
Confessions of a Numerical Analyst
Keith Evan Schubert
Simple Problem
Consider the problem ax=b
The resulting x value is
A1.50 1.50
1.01 0.990
b
3.00
2.00
x 1.00
1.00
Simple Problem 2
Consider the problem ax=b
The resulting x value is
A1.5 1.5
1.0 0.99
b
3.0
2.0
x 2.0
0.0
What’s Up?
The condition number (sensitivity to perturbations) is about 400.
A condition number of 1 is perfect. Perturbation is 0.01, so 0.01*400=4. Components of x can vary by this much!
What Can We Do?
Rather than solve it the standard way• X=a\b
• X=(ATA)-1atb
Consider the following:• X=(ATA+i)-1atb =.01
Then:x
1.0
1.0
Lucky Guess?
-1 -0.5 0 0.5 10
0.5
1
1.5
2
x 2
-1 -0.5 0 0.5 10
0.5
1
1.5
2
x 1
Does It Always Work?
No Consider X0 Consider i
2 (i is singular value of A)
X± Picking the wrong value can get junk
Skyline
Consider a 1 dimensional picture Use height instead of color Result looks like the silhouette of a city’s
skyline Have smog which blurs and softens Don’t know exactly how much blur Want to get sharp edges
Getting Garbage
Getting Improvement
Why Backward? Forward errors
• Explicitly account for each error source• (X+1)(y+2)=xy+(y1+x2+12)
Backward errors• Check that my algorithm acting on data will give me
a solution that is “near” to the actual system acting on a nearby set of data
• I.E. My algorithm with good data should do about as well as a perfect calculation on ok data
•
error Ax bA x b
Picture Please!
ActualData (x)
NearbyData (x*)
Perfect Calculations
My Algorithm
Inherent errorsin A
b
Errors due toalgorithm
b*
best
Least Squares
Usually we don’t have an invertible matrix Need to find an estimated solution Criterion: minimize ||ax-b|| Normal equation
• ATA x = ATb
Solution• X = (ATA)-1atb
Backward Error
Criterion: minimize ||Ax-b||/(||A|| ||x||+||b||) Normal Equations
•
Solution:•
ATA I xATb
A Ax b
x A x b
x ATA I 1ATb
Non Convex
Finding The Root
g A Ax b
x A x b
Look At Critical Region
Informal Algorithm
Get (A,b) svd(A) [u1 u2],,v
U1b b1
Use rootfinder (bisection, Newton, etc.) to get in [-n
2,0]
vT(2- I)-1 b1 x
What You GetBlurred Image
Least SquaresLeast Squares Solution
Total Least SquaresTotal Least Squares Solution
TikhonovTikhonov Solution
Backward ErrorMin Max Backward Error Solution
OriginalActual Image
ComparisonActual Image Blurred Image
Least Squares Solution Tikhonov Solution
Degenerate Min Min Solution Min Max Solution
Min Max Backward Error Solution Total Least Squares Solution
Final Thoughts
BE is always optimistic in that it presumes that the real system is “better”
Even with this it is “robust” There is a perturbed version of this
algorithm which can be either optimistic or pessimistic
That version is not fully proven