Download - Simplicity Truth Zen
![Page 1: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/1.jpg)
SimplicityTruthZen
Kevin T. Kelly Conor Mayo-Wilson
Hanti LinDepartment of Philosophy
Carnegie Mellon Universityhttp://www.andrew.cmu.edu/user/kk3n/ockham/Ockham.htm
![Page 2: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/2.jpg)
Theoretical Underdetermination
???
I dare you to choose one!
![Page 3: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/3.jpg)
Western Epistemology The demon undermines science, so
defeat him.
I smite thee with scientific rationality!
R
![Page 4: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/4.jpg)
Western Epistemology The demon undermines science, so
defeat him. Portray him as weak.
You can’t fool me!
R
![Page 5: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/5.jpg)
Zen Epistemology The demon justifies science with his
strength. Stronger demons justify weaker
inferences!
![Page 6: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/6.jpg)
Zen Epistemology
How is this supposed to work, eh?
![Page 7: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/7.jpg)
Zen Epistemology• If you could do better, you should. • But because of me, you can’t. • So you are optimal and, hence, justified.
![Page 8: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/8.jpg)
Zen EpistemologyIsn’t that how computer scientists routinely evaluate algorithms?
Yes.
![Page 9: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/9.jpg)
Zen EpistemologyBut computing is deduction. Isn’t induction a completely different animal?
No. That’s what formal learning theory is about.
![Page 10: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/10.jpg)
Zen EpistemologyCool. But how does it apply to science?
![Page 11: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/11.jpg)
Ockham’s Razor
![Page 12: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/12.jpg)
Planetary retrograde motion
MarsEarth
Sun
Astronomy 1543
Only at solar opposition
![Page 13: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/13.jpg)
Ptolemaic Explanation
q
Retrograde path is due to an epicycle.Can happen regardless of the position of the sun
epicycle
![Page 14: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/14.jpg)
Copernican Explanation“Lapped” competitor appears to backtrack against the bleachers.
lappingq
epicycle
![Page 15: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/15.jpg)
Copernicus VictoriousExplains the data rather than merely accommodating them.
Has fewer adjustable parameters.Is more falsifiable (failure of correlation would rule it out).
Is more unified.Is simpler.
![Page 16: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/16.jpg)
More Victories for Simplicity
universal gravitation vs. divided cosmos
wave theory of light vs. particles and aether
oxygen vs. phlogistonnatural selection vs. special creationspecial relativity vs. classical electrodynamics
Dirac’s equationchaos vs. infinite series of random variables
![Page 17: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/17.jpg)
Puzzle An indicator must be sensitive to
what it indicates.
simple
![Page 18: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/18.jpg)
Puzzle An indicator must be sensitive to
what it indicates.
complex
![Page 19: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/19.jpg)
Puzzle But Ockham’s razor always points
at simplicity.
simple
![Page 20: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/20.jpg)
Puzzle But Ockham’s razor always points
at simplicity.
complex
![Page 21: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/21.jpg)
Puzzle How can a broken compass help
you find something unless you already know where it is?
complex
![Page 22: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/22.jpg)
Metaphysicians for Ockham
Somehow, I don’t like the sound of that…
![Page 23: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/23.jpg)
Information Channel
Simplicity bias Simple reality
Mystical visionNatural selection??
![Page 24: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/24.jpg)
Pre-established Harmony
Simplicity bias Simple reality
G
![Page 25: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/25.jpg)
Idealism
Simplicity bias Simple reality
![Page 26: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/26.jpg)
Metaphysicians for Ockham
So before you can use your razor to eliminateMetaphysical causes, you have to assume metaphysical causes.
![Page 27: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/27.jpg)
Theoretical “Virtues”Simpler theories:1. Are more unified;2. Explain better;3. Are more falsifiable.
But the truth might not be virtuous.Wishful thinking to assume
otherwise.
![Page 28: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/28.jpg)
Statistical Explanations1. Prior Simplicity Bias
Bayes, BIC, MDL, MML, etc.
2. Risk MinimizationSRM, AIC, cross-validation, etc.
![Page 29: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/29.jpg)
Prior Simplicity Bias
I am inclined to believe the simpler theory because I am inclined to believe that the world is simple.
![Page 30: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/30.jpg)
More Subtle VersionSimple data are a miracle in the complex theory but not in the simple theory.
P C
Has to be!
![Page 31: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/31.jpg)
However… correlation would not be a miracle given P(q);
Why not this?
P C
q
![Page 32: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/32.jpg)
The Real MiracleIgnorance about model: p(C) p(P);
+ Ignorance about parameter setting: p’(P(q) | P) p(P(q’ ) | P).
= Knowledge about worlds.p(P(q)) << p(C).
CP
qqqqqqqq
Ignorance is knowledge.
![Page 33: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/33.jpg)
= Paradox of IndifferenceIgnorance of red vs. not-red
+ Ignorance over not-red: = Knowledge about red vs. white.
Knognorance = All the priveleges of knowledgeWith none of the responsibilitiesI’m for it!
![Page 34: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/34.jpg)
In Any Event
The coherentist foundations of Bayesianism have nothing to do with short-run truth-conduciveness.Not so loud!
![Page 35: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/35.jpg)
Ockham Bayesians Converge
T1
T2
T3
![Page 36: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/36.jpg)
Ockham Bayesians Converge
T1
T2
T3
![Page 37: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/37.jpg)
Ockham Bayesians Converge
T1
T2
T3
![Page 38: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/38.jpg)
Ockham Bayesians Converge
T1
T2
T3
![Page 39: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/39.jpg)
Ockham Bayesians Converge
T1
T2
T3
Oops!
![Page 40: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/40.jpg)
Ockham Bayesians Converge
T1
T2
T3
![Page 41: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/41.jpg)
Ockham Bayesians Converge
T1
T2
T3
![Page 42: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/42.jpg)
Ockham Bayesians Converge
T1
T2
T3
![Page 43: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/43.jpg)
Ockham Bayesians Converge
T1
T2
T3
Oops!
![Page 44: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/44.jpg)
Ockham Bayesians Converge
T1
T2
T3
Oops!
![Page 45: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/45.jpg)
But So Would Other Methods
T1
T2
T3
Oops!
![Page 46: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/46.jpg)
Summary of Bayesian Approach
Prior-based explanations of Ockham’s razor are circular.
Convergence-based explanations of Ockham’s razor fail to single out Ockham’s razor.
![Page 47: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/47.jpg)
2. Risk Minimization Ockham’s razor minimizes
expected distance of empirical estimates from the true value.
Truth
![Page 48: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/48.jpg)
Unconstrained Estimates
are Centered on truth but spread around it.
Pop!Pop!Pop!Pop!
Unconstrained aim
![Page 49: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/49.jpg)
Off-center but less spread.
Clamped aim
Truth
Constrained Estimates
![Page 50: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/50.jpg)
Off-center but less spread Overall improvement in expected
distance from truth…
Truth
Pop!Pop!Pop!Pop!
Constrained Estimates
Clamped aim
![Page 51: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/51.jpg)
Doesn’t Find True Theory
False theories that aren’t too false theory typically predict more accurately than the true theory.
Four eyes!
Clamped aim
![Page 52: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/52.jpg)
When Theory Doesn’t Matter
Predicting lung cancer from ash trays.
![Page 53: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/53.jpg)
When Theory Does Matter
Predicting effectiveness of a ban on ash trays.
![Page 54: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/54.jpg)
Correlation does imply causation if there are multiple variables, some of which are common effects. [Pearl, Spirtes, Glymour and Scheines, etc.]
Great News
Protein A
Protein BProtein C Cancer protein
![Page 55: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/55.jpg)
Joint distribution p is causally compatible with causal network G iff:
Causal Markov Condition: each variable X is independent of its non-effects given its immediate causes.
Faithfulness Condition: no other conditional independence relations hold in p.
Core assumptions
![Page 56: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/56.jpg)
F1 F2
Tell-tale CorrelationsH C
FGiven F, H gives some info about C(Faithfulness)
C
Given C, F1 gives no further info about F2(Markov)
![Page 57: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/57.jpg)
Standard Applications Linear Causal Case: each variable
X is a linear function of its parents and a normally distributed hidden variable called an “error term”. The error terms are mutually independent.
Discrete Multinomial Case: each variable X takes on a finite range of values.
![Page 58: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/58.jpg)
The CatchProtein A
Protein BProtein C Cancer protein
It’s full of protein C!
English Breakfast?
![Page 59: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/59.jpg)
As the Sample Increases…
Protein A
Protein BProtein C Cancer proteinweak
Protein D
I do! Bon apetit!
This situation empiricallyapproximates the last one. So who cares?
![Page 60: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/60.jpg)
As the Sample Increases Again…
Protein A
Protein BProtein C Cancer protein
Wasn’t that last approximationto the truth good enough?
weak
Protein D
Aaack!
Protein Eweaker
weaker
![Page 61: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/61.jpg)
Causal Flipping Theorem
For each convergent M and standard (G, p), there exists standard (G’, p’ ) such that:
p’ is indistinguishable from p at the current sample size and
consistent M flips the orientation of causal arrow X Y in (G’, p’ ) at least n times as sample size increases.
oops
I meant oops
oopsI meant
I meant
![Page 62: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/62.jpg)
Simulation Using CPC Algorithm
Proportion of outputs out of 1000 trials at each sample size.
![Page 63: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/63.jpg)
Zen Response Many explanations have been
offered to make sense of the here-today-gone-tomorrow nature of medical wisdom — what we are advised with confidence one year is reversed the next — but the simplest one is that it is the natural rhythm of science.
(Do We Really Know What Makes us Healthy, NY Times Magazine, Sept. 16, 2007).
![Page 64: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/64.jpg)
Pursuit of Truth Short-run Tracking
Too strong to be feasible when theory matters.
![Page 65: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/65.jpg)
Pursuit of Truth Long-run Convergence
Too weak to single out Ockham’s razor
![Page 66: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/66.jpg)
The Middle Way Straightest Convergence
Just right?
![Page 67: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/67.jpg)
Ancient Roots"Living in the midst of ignorance and considering themselves intelligent and enlightened, the senseless people go round and round, following crooked courses, just like the blind led by the blind." Katha Upanishad, I. ii. 5, c. 600 BCE.
![Page 68: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/68.jpg)
II. Navigation by Broken Compass
simple
![Page 69: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/69.jpg)
Asking for Directions
Where’s …
![Page 70: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/70.jpg)
Asking for Directions
Turn around. The freeway ramp is on the left.
![Page 71: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/71.jpg)
Asking for Directions
Goal
![Page 72: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/72.jpg)
Best Route
Goal
![Page 73: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/73.jpg)
Best Route to Any Goal
![Page 74: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/74.jpg)
Disregarding Advice is Bad
Extra U-turn
![Page 75: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/75.jpg)
Helpful A Priori Advice
…so fixed advice can help you reach a hidden goalwithout circles, evasions, or magic.
![Page 76: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/76.jpg)
Polynomial Laws Data = open intervals around Y
at rational values of X.
![Page 77: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/77.jpg)
Polynomial Laws No effects:
![Page 78: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/78.jpg)
Polynomial Laws First-order effect:
![Page 79: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/79.jpg)
Polynomial Laws Second-order effect:
![Page 80: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/80.jpg)
In Step with the Demon
ConstantLinear
QuadraticCubic
There yet?Maybe.
![Page 81: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/81.jpg)
In Step with the Demon
ConstantLinear
QuadraticCubic
There yet?Maybe.
![Page 82: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/82.jpg)
In Step with the Demon
ConstantLinear
QuadraticCubic
There yet?Maybe.
![Page 83: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/83.jpg)
In Step with the Demon
ConstantLinear
QuadraticCubic
There yet?Maybe.
![Page 84: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/84.jpg)
Passing the Demon
ConstantLinear
QuadraticCubic
There yet?Maybe.
![Page 85: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/85.jpg)
Passing the Demon
ConstantLinear
QuadraticCubic
I know you’re coming!
![Page 86: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/86.jpg)
Passing the Demon
ConstantLinear
QuadraticCubic
Maybe.
![Page 87: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/87.jpg)
Passing the Demon
ConstantLinear
QuadraticCubic
!!!
Hmm, it’s quite nice here…
![Page 88: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/88.jpg)
Passing the Demon
ConstantLinear
QuadraticCubic
You’re back!Learned your lesson?
![Page 89: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/89.jpg)
Ockham Violator’s Path
ConstantLinear
QuadraticCubic
See, you shouldn’t run aheadEven if you are right!
![Page 90: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/90.jpg)
Ockham Path
ConstantLinear
QuadraticCubic
![Page 91: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/91.jpg)
Empirical Problems
T1 T2 T3
Set K of infinite input sequences. Partition of K into alternative
theories.K
![Page 92: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/92.jpg)
Empirical Methods
T1 T2 T3
Map finite input sequences to theories or to “?”.
K
T3
e
![Page 93: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/93.jpg)
Method Choice
T1 T2 T3
e1 e2 e3 e4
Input history
Output historyAt each stage, scientist can choose a new method (agreeing with past theory choices).
![Page 94: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/94.jpg)
Aim: Converge to the Truth
T1 T2 T3
K
T3 ? T2 ? T1 T1 T1 T1 . . .T1 T1 T1
![Page 95: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/95.jpg)
Retraction Choosing T and then not choosing
T next
T’T
?
![Page 96: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/96.jpg)
Aim: Eliminate Needless Retractions
Truth
![Page 97: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/97.jpg)
Aim: Eliminate Needless Retractions
Truth
![Page 98: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/98.jpg)
Aim: Eliminate Needless Delays to Retractions
theory
![Page 99: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/99.jpg)
applicationapplicationapplication
applicationcorollary
applicationtheory
applicationapplicationcorollary applicationcorollary
Aim: Eliminate Needless Delays to Retractions
![Page 100: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/100.jpg)
An Epistemic Motive1. Future retraction is Gettier
situation even if current belief is true.
2. If Gettier situations are bad, more of them are worse.
![Page 101: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/101.jpg)
Easy Retraction Time Comparisons
T1 T1 T2 T2
T1 T1 T2 T2 T3 T3T2 T4 T4
T2 T2
Method 1
Method 2
T4 T4 T4
. . .
. . .
at least as many at least as late
![Page 102: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/102.jpg)
Worst-case Retraction Time Bounds
T1 T2
Output sequences
T1 T2
T1 T2
T4
T3
T3
T3
T3
T3 T3
T4
T4
T4
T4 T4
. . .
(1, 2, ∞)
. . .
. . .
. . .. . .. . .
T4
T4
T4
T1 T2 T3 T3 T3 T4T3 . . .
![Page 103: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/103.jpg)
IV. Ockham Without Circles, Evasions,
or Magic
![Page 104: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/104.jpg)
Empirical Effects
![Page 105: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/105.jpg)
Empirical Effects
![Page 106: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/106.jpg)
Empirical Effects
May take arbitrarily long to discover
![Page 107: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/107.jpg)
Empirical Effects
May take arbitrarily long to discover
![Page 108: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/108.jpg)
Empirical Effects
May take arbitrarily long to discover
![Page 109: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/109.jpg)
Empirical Effects
May take arbitrarily long to discover
![Page 110: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/110.jpg)
Empirical Effects
May take arbitrarily long to discover
![Page 111: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/111.jpg)
Empirical Effects
May take arbitrarily long to discover
![Page 112: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/112.jpg)
Empirical Effects
May take arbitrarily long to discover
![Page 113: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/113.jpg)
Empirical Theories True theory determined by which
effects appear.
![Page 114: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/114.jpg)
Empirical Complexity
More complex
![Page 115: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/115.jpg)
Assume No Short PathsWeaker results if some path is shorter than some other path.
![Page 116: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/116.jpg)
Ockham’s Razor Don’t select a theory unless it is
uniquely simplest in light of experience.
![Page 117: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/117.jpg)
Stalwartness Don’t retract your answer while it is
uniquely simplest.
![Page 118: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/118.jpg)
Stalwartness Don’t retract your answer while it is
uniquely simplest.
![Page 119: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/119.jpg)
Timed Retraction Bounds
r(M, e, n) = the least timed retraction bound covering the total timed retractions of M along input streams of complexity n that extend e
Empirical Complexity 0 1 2 3 . . .
. . .
M
![Page 120: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/120.jpg)
Efficiency of Method M at e
M converges to the truth no matter what;
For each convergent M’ that agrees with M up to the end of e, and for each n: r(M, e, n) r(M’, e, n)
Empirical Complexity 0 1 2 3 . . .
. . .
M M’
![Page 121: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/121.jpg)
M is Beaten at e There exists convergent M’ that
agrees with M up to the end of e, such that r(M, e, n) > r(M’, e, n), for each n.
Empirical Complexity 0 1 2 3 . . .
. . .
M M’
![Page 122: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/122.jpg)
Basic Idea Ockham efficiency: Nature can
force arbitary, convergent M to produce the successive answers down an effect path arbitrarily late, so stalwart, Ockham solutions are efficient.
![Page 123: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/123.jpg)
Basic Idea Unique Ockham efficiency: A
violator of Ockham’s razor or stalwartness can be forced into an extra retraction or a late retraction at the time of the violation, so the violator is beaten by each stalwart, Ockham solution.
![Page 124: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/124.jpg)
Ockham Efficiency Theorem
Let M be a solution. The following are equivalent:
M is henceforth Ockham and stalwart;
M is henceforth efficient; M is henceforth never beaten.
![Page 125: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/125.jpg)
Some Applications Polynomial laws Conservation laws Causal networks
![Page 126: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/126.jpg)
Concrete RecommendationsExtra retraction by PC
algorithm
![Page 127: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/127.jpg)
Generalizations Generalized Simplicity Bayesian retractions Mixed Strategies
Times of retractions in chance Expected retraction times
Drop the no short paths assumption…
Retractions weighted by content…
Statistical inference…
![Page 128: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/128.jpg)
Conclusions Ockham’s razor is necessary for
staying on the straightest path to the true theory but does not point at the true theory.
A priori justification---no evasions or circles are required.
Analogous to computational complexity theory.
![Page 129: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/129.jpg)
IV. Simplicity
![Page 130: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/130.jpg)
ApproachEmpirical complexity reflects
nested problems of induction posed by the problem.
Hence, simplicity is question-relative but topologically invariant.
![Page 131: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/131.jpg)
Empirical Problems
T1 T2 T3
Topological space (W, T) Countable partition Q of W into
alternative theories.
W
![Page 132: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/132.jpg)
Why Topology? Topology is the pure mathematics of
(ideal) verifiability.
Proof: The tautology and contradiction are
verifiable; If A, B are verifiable, A & B is verifiable; If A1, A2, … are verifiable, A1 v A2 v… is
verifiable.
![Page 133: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/133.jpg)
Simplicity Concepts
c = 0
c = 1
![Page 134: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/134.jpg)
Simplicity Concepts
c = 0
c = 1
c = 2
![Page 135: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/135.jpg)
Simplicity Concepts
c = 0
c = 1
c = 2
![Page 136: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/136.jpg)
Simplicity Concepts A simplicity concept (without short
paths) for (W, T, Q) is just an assignment c of natural numbers to worlds such that:
1. The condition c < n is closed;2. Each answer is clopen given that c =
n;3. the condition A & (c = n) is in the
boundary of the condition not-A & (c = n + 1) , for each answer A and non-terminal complexity n.
![Page 137: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/137.jpg)
Virtues Can still prove the Ockham efficiency
theorem. Grounds simplicity in the logic of the
question. Topologically invariant---grue-proof
relative to question. Generalizes dimension---works for
continuous and discrete spaces.
![Page 138: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/138.jpg)
AGM Belief Revision No induction without refutation. After refutation, almost anything
goes. Greedy---leaps to new theory
immediately. No concern with truth-
conduciveness. Not much like science or even
standard Bayesians
![Page 139: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/139.jpg)
Stalwart Ockham Method Can delay expansion to new
Ockham theory after a surprise, as a Bayesian would.
Converges to truth with optimal efficiency.
Scientifically natural. More like Bayesian with standard
prior.
![Page 140: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/140.jpg)
V. Bayesian Efficiency
![Page 141: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/141.jpg)
Anti-Simplicity Bayesian
T1
T2
T3
![Page 142: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/142.jpg)
Simplicity-biased Bayesian
T1
T2
T3
![Page 143: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/143.jpg)
Greedy Ockham
T1
T2
T3
![Page 144: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/144.jpg)
Focus on Retractions Euclidean distance measures
retractions + expansions. Charge for retractions only.
![Page 145: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/145.jpg)
Not Entropy Increase
T1
T2
T3
Retractions can’tbe increase in ascalar field.
0
![Page 146: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/146.jpg)
Not Kullback Leibler Divergence
T1
T2
T3
Still charges forexpansions.
> 0
![Page 147: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/147.jpg)
Classical Logic
pq
r
p v q q v rp v r
T
p v q v r
![Page 148: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/148.jpg)
Logical Retractions
1
1
![Page 149: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/149.jpg)
Code Disjunctions
(1,0,0) (0,0,1)(0,1,0)
(0,0,0)
(0,1,1)(1,1,0) (1,0,1)
(1,1,1)
![Page 150: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/150.jpg)
Fuzzy Disjunctions
p v q v r = (1, .8, .2). Valuation in assignment (0, 1, 0) is
just the inner product: (0, 1, 0) . (1, .8, .2) = .8.
Entailment: p entails q iff b . q > b . p, for all Boolean b.
![Page 151: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/151.jpg)
Geologic on Euclidean Cube
(1, .8, .2)
![Page 152: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/152.jpg)
Geological Retractions
(.5, 1, 0) (0, 1, .5).5 0
![Page 153: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/153.jpg)
Probability Lives in Geologic
p(q) = p . qp
q
![Page 154: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/154.jpg)
Classical Principle of Indifference= perspective projection of corners
![Page 155: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/155.jpg)
General Principle of Indifference= perspective projection in general
f
![Page 156: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/156.jpg)
Bayesian Retractions= shadow of geological retractions
f
![Page 157: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/157.jpg)
Bayesian Retractions
0
1
1
0
![Page 158: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/158.jpg)
“Retraction Cones”
Retraction-free
Pure retractionMixedMixed
![Page 159: Simplicity Truth Zen](https://reader035.vdocuments.us/reader035/viewer/2022062501/5681663f550346895dd9a9f4/html5/thumbnails/159.jpg)
Efficient Bayesian
0
0
<1
0<1
0