what we didn’t cover: the present and...
Post on 22-Jul-2020
6 Views
Preview:
TRANSCRIPT
What We Didn’t Cover:The Present and Future
Reading for this lecture:
Lecture Notes and cited papers.
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
1Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Course narrative:
01
1
1 0
...001011101000...
ModellerSystem
A
B
C
ProcessInstrument
!
"
# $1
0
0
00
0
1
1 11
Forms of Chaos: Deterministic sources of novelty Mechanisms that produce unpredictability Sensitive dependence on initial condition Sensitive dependence on parameter
How random?
How structured?
Measurement Theory: Partitions Optimal Instrument:
max
{P}hµ
min{P}
Cµ
Universal model:
Pattern defined Causal Architecture Intrinsic Computation
! ! Machine
Cµ,E,T,�,⌅
�max
, H(L), hµ,
G,R
2Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
What we didn’t cover:
ε-Machine Enumeration Infinite ε-Machines & Generalized Hidden Markov Models Hierarchical ε-Machine Reconstruction Quantum Processes & Quantum Machines Rate Distortion Theory & Optimal Causal Inference Cellular Automata Computational Mechanics Spin Systems in 1D and 2D Optimal Instrument Design Complexity-Entropy Diagrams
3Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
What we didn’t cover ... ε-Machine Enumeration
Every process has its ε-machine presentationSpace of processes = Space of ε-Machinesε-Machines can be exactly enumerated
B. D. Johnson, J. P. Crutchfield, C. J. Ellison, and C. S. McTague, "Enumerating Finitary Processes", (2010) submitted.Santa Fe Institute Working Paper 10-11-027. arXiv:1011.0036 [cs.FL].
n\k 2 3 4 5 61 3 7 15 31 632 7 141 1,873 20,925 213,9973 78 15,598 1,658,606 136,146,5904 1,388 3,625,6385 35,1866 1,132,6137 43,997,4268 1,993,473,480
States
Alphabet
4Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
What we didn’t cover ... ε-Machine Enumeration
Every process has its ε-machine presentationSpace of processes = Space of ε-Machinesε-Machines can be exactly enumerated
B. D. Johnson, J. P. Crutchfield, C. J. Ellison, and C. S. McTague, "Enumerating Finitary Processes", (2010) submitted.Santa Fe Institute Working Paper 10-11-027. arXiv:1011.0036 [cs.FL].
n\k 2 3 4 5 61 3 7 15 31 632 7 141 1,873 20,925 213,9973 78 15,598 1,658,606 136,146,5904 1,388 3,625,6385 35,1866 1,132,6137 43,997,4268 1,993,473,480
States
Alphabet
Unknown unknowns!
4Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
What we didn’t cover ... Infinite ε-Machines & Generalized Hidden Markov Models
D. Upper, Ph.D. Dissertation (Berkeley 2007).J. P. Crutchfield, "Calculi of Emergence:", Computation, Dynamics, and Induction", Physica D 75 (1994) 11-54.
5Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
What we didn’t cover ... Hierarchical ε-Machine Reconstruction
J. P. Crutchfield, "Calculi of Emergence:", Computation, Dynamics, and Induction", Physica D 75 (1994) 11-54.
6Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
What we didn’t cover ...
Quantum Processes & Quantum Machines
K. Wiesner and J. P. Crutchfield, "Computation in Finitary Stochastic and Quantum Processes", Physica D 237:9 (2008) 1173-1195.
7Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
What we didn’t cover ... Rate Distortion Theory & Optimal Causal Inference
I[ !X 2;"X 5 R]
I[" X5 ;R]
0.0
3.5
0.5
3.02.52.01.51.0
0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14|
P(00)
P(01)P(10)
P(11)
P(!X "x )|
3 states4 states
6 states
2 states
5 states
1 stateInfeasible
Feasible
2 5
minPr(R|
�X)
�I[�X;R] + �I[
�X;⇥X |R]
⇥
S. Still, J. P. Crutchfield, and C. J. Ellison "Optimal Causal Inference: Estimating Stored Information and Approximating Causal Architecture", CHAOS 20:3 (2010) 037111.
8Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
What we didn’t cover ... Cellular Automata Computational Mechanics:
0
Time
99286Site0
0
Time
99286Site0
J. E. Hanson and J. P. Crutchfield, "Computational Mechanics of Cellular Automata: An Example", Physica D 103 (1997) 169-189.
9Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
What we didn’t cover ... Spin Systems in 1D and 2D
: GeneralizedOrder Parameter!ED. P. Feldman and J. P. Crutchfield, "Structural Information in Two-Dimensional Patterns:
Entropy Convergence and Excess Entropy", Physical Review E 67 (2003) 051104.
10Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
What we didn’t cover ... Optimal Instrument Design
C. C. Strelioff and J. P. Crutchfield, "Optimal Instruments and Models for Noisy Chaos", CHAOS 17 (2007) 043127.
11Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Analogous to Thermodynamic Phase Diagram (gas, liquid, solid). But uses only intrinsic computation properties.A wide diversity of Complexity-Entropy Diagrams.
RandomnessEntropy
StructuralComplexity
Predictable Unpredictable
Simple
Complex
D. P. Feldman, Carl S. McTague, and J. P. Crutchfield, "The Organization of Intrinsic Computation: Complexity-Entropy Diagrams and the Diversity of Natural Information Processing", CHAOS 18:4 (2008) 53-73.
What we didn’t cover ... Complexity-Entropy Diagrams: Analyze a class of processes
12Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Logistic Map
What we didn’t cover ... Complexity-Entropy Diagrams: Analyze a class of processes
13Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Logistic Map
0
1
2
3
4
5
6
0 0.2 0.4 0.6 0.8 1
Exce
ss E
ntro
py E
Entropy Rate hµ
One-Band RegionTwo-Band Region
What we didn’t cover ... Complexity-Entropy Diagrams: Analyze a class of processes
14Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0 0.2 0.4 0.6 0.8 1
Exce
ss E
ntro
py E
I
Entropy Density hµ
2D Ising Spin System
What we didn’t cover ... Complexity-Entropy Diagrams: Analyze a class of processes
15Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Complexity-Entropy Diagram: Analyze a class of processes ...
0.0 0.2 0.4 0.6 0.8 1.0 1.2hµ
0.0
0.5
1.0
1.5
2.0
2.5
E
E vs hµ
N = 5N = 4N = 3N = 2N = 1
ε-Machines
16Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Applications by Others:
Chaotic Dynamical SystemsSymbolic DynamicsStatistical Mechanical Models: Spin systems (Ising, glasses, ...)Cellular AutomataHidden Markov ModelsChaotic CrystallographyHydrodynamics: Dripping faucet, turbulenceQuantum Dynamical SystemsSingle Molecule Dynamics
17Sunday, May 18, 14
Molecular Dynamics Spectroscopy:
C.-B. Li, H. Yang, & T. Komatsuzaki, Proc. Natl. Acad. Sci USA 105:2 (2008) 536–541.
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Topics & Applications...
18Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research, Ongoing:
Novel materialsQuantum chaotic dynamics & measurement effectsContinuous processesSpatiotemporal processesInteractive learningNetwork dynamicsNeurobiological processesMultiagent systemsDistributed roboticsEvolutionary dynamics...
19Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research ... Complex Materials & ε-Machine Spectral Reconstruction
ε-MSR
D. P. Varn, G. S. Canright, and J. P. Crutchfield, "ε-Machine spectral reconstruction theory: A direct method for inferring planar disorder and structure from X-ray diffraction studies", Acta Cryst. Sec. A 69:2 (2013) 197-206.
20Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research ...
Novel materials: Use eMSR to design Run eMSR backwards
Hypothesis: Structure key to properties.But we can now analyze structure.
E.g., Design polytypes = Exotic semiconductors
21Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research ...Spectral Decomposition of Intrinsic Computation:
Exact, closed-form expression of all information measures using ε-machine presentation.
How?
Mixed states + Eigenspectrum of ε-machine.
J. P. Crutchfield, C. J. Ellison, P. Riechers in preparation.
22Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research ...
hµ(L) = H⇥XL�1|XL�1
0 , S0 ⇠ µ0(�)⇤
= H⇥XL�1|
�RL�1|R0 = µ0(�)
�⇤
Recall mixed state expression:
Track analytically using spectral decomposition:
J. P. Crutchfield, C. J. Ellison, P. Riechers in preparation.
Need only track evolution of mixed states via .
TL =X
�2⇤T
�LT�
Projection operators:
TL
T� =Y
⇣2⇤T⇣ 6=�
T � ⇣I
�� ⇣
Eigenvalues: ⇤T ⌘ {� 2 C : det(�I � T ) = 0}
Spectral decomposition of intrinsic computation ...
23Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research ...Spectral decomposition of intrinsic computation ...
Excess entropy:
J. P. Crutchfield, C. J. Ellison, P. Riechers in preparation.
Synchronization information:
Transient information:T ⌘1X
L=1
L [hµ(L)� hµ]
=X
�2⇤T|�|<1
1
(1� �)2h�⇡|T�|H({T (s)})i
S =
(P�2⇤T|�|<1
11�� h�⇡|T�|H[⌘]i if h⇡T |H[⌘]i = 0
1 otherwise
E ⌘1X
L=1
[hµ(L)� hµ]
=X
�2⇤T|�|<1
1
1� �h�⇡|T�|H({T (s)})i
24Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research ...
ε-Transducers: Input-output functions! ε-Machines: Behaviors and distributions over them Now, mappings on stochastic behaviors Optimal, minimal, unique presentation of mappingsNoisy communication channel:
Input process
Output process
Joint process
N. Barnett, C. J. Ellison, J. P. Crutchfield, in preparation.
. . . , (X,Y )�1, (X,Y )0, (X,Y )1, . . .
. . . , X�1, X0, X1, . . .
. . . , Y�1, Y0, Y1, . . .
Input OutputChannel
25Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research ...ε-Transducers ...
Transitions: x|p|y x is input symbol, y is output symbol, and p is probability
Identity channel.
All-to-fair channel.
Memoryless channels:
26Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research ...ε-Transducers ...
Delay-by-1 Channel:Input is order-1 Markov, output nonsynchronizing.
Memoryful Channel:
Memoryless channels ...
Z Channel: Fuzz out ones, transmit zeros.
27Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research ...ε-Transducers ...
Even-to-Fair: Map Even Process to Fair Coin.Random bits from Even Process, but fuzz out pairs of 1s.
Feedback-NOR: NOR of the previous output & current input.Strictly sophic on input, nonsynchronizing on output, but order-1 Markov on joint symbols.
Memoryful channels ...
28Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research ... Interactive Learning:
World
Agent
Model Policy
ActionsObservations
The Feedback Loop
29Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research ...
The Evolution of Language:
Game of Telephone:
M0 ↵N0 = a0 a1 · · · aN
M1 ↵N1 = a0 a1 · · · aN
M2 · · ·
Generatealleles
Generatealleles
Infer✏-machine
Infer✏-machine
J. P. Crutchfield and S. Whalen, "Structural Drift: The Population Dynamics of Sequential Learning", PLoS Computational Biology 8:6 (2012) e1002510.
‘‘Send Three- and Four-Pence, We’re Going to a Dance’’
‘‘Send reinforcements we’re going to advance’’
Heard over radio during WWI:
Intended message:
Mathematical Theory of Sequential Learning
30Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research ...
The Evolution of Language ...Mathematical Theory of Sequential Learning
Genetic Drift
t1(p0) = � 1
p0[4Ne(1� p0) log(1� p0)]
0 0.2 0.4 0.6 0.8 1
0
10
20
30
40
Initial Probability p
Time(Generations)
Fixation (MC) Fixation (t1 Theory)
Deletion (t0 Theory)
0 N = 10
(Theory: Kimura)
J. P. Crutchfield and S. Whalen, "Structural Drift: The Population Dynamics of Sequential Learning", PLoS Computational Biology 8:6 (2012) e1002510.
31Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research:
The Evolution of Language ...
Structural Drift
GMP SubspaceAlternating
Process
M0 = GMP
Semantic Creation
0 50 100 150 200 250
0.4
0.6
0.8
1
Time (Generations)
Pr[Heads]
Golden Mean Even Process
Biased Coin
BA1 |H
1 |T
A
1 |H
Sequential learning as a diffusion in space of ε-Machines
32Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research:
The Evolution of Language ...
translationparty.com
33Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research:Emergence of Evolutionary Selection
Self-Organizing NetworksEvolutionary Theory: From the Primordial Soup to Natural Selection
TR
TB TA
TC
TD
J. P. Crutchfield and O. Goernerup, “Objects That Make Objects: The Population Dynamics of Structural Complexity”, J. Royal Society Interface 3 (2006) 345-349.
34Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Computational Mechanics Research:Emergence of Evolutionary Selection
Self-Organizing NetworksEvolutionary Theory: From the Primordial Soup to Natural Selection
TR
TB TA
TC
TD
J. P. Crutchfield and O. Goernerup, “Objects That Make Objects: The Population Dynamics of Structural Complexity”, J. Royal Society Interface 3 (2006) 345-349.
1
2
3
4
6
8
7
Decom
position
9
10
Com
positon
|M|
�
5
34Sunday, May 18, 14
Topics & Applications...
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
The Message:
01
1
1 0
...001011101000...
ModellerSystem
A
B
C
ProcessInstrument
!
"
# $1
0
0
00
0
1
1 11
How Nature is Organized is How Nature Computes
35Sunday, May 18, 14
Further:
Computational Mechanics Archive: http://csc.ucdavis.edu/~cmg/
Dynamics of Learning Research Group Mail List: http://lists.csc.ucdavis.edu/mailman/listinfo/dynlearn
Last Lecture: Natural Computation & Self-Organization, Physics 256 (Spring 2014); Jim Crutchfield
Topics & Applications...
36Sunday, May 18, 14
top related