complexity, parallel computation and statistical...
TRANSCRIPT
![Page 1: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/1.jpg)
Complexity, ParallelComputation and Statistical
Physics
Jon MachtaUniversity of Massachusetts Amherst
Supported by the National Science Foundation
![Page 2: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/2.jpg)
Outline
• Parallel computing and computational complexity
• Complexity of models in statistical physical
• Physical complexity and computational complexity
Complexity Journal (to appear) and cond-mat/0510809
![Page 3: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/3.jpg)
Collaborators
• Ray Greenlaw, Armstrong Atlantic University• Cris Moore, University of New Mexico, SFI
• Ken Moriarty, UMass• Xuenan Li, UMass
• Dan Tillberg, UMass• Ben Machta, Brown University
![Page 4: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/4.jpg)
Computational Complexity•How do computational resources scale with the size ofthe problem?
TimeHardware
•Equivalent models of computation.Turing machineParallel random access machineBoolean circuit familyFormal logic
![Page 5: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/5.jpg)
Parallel Random AccessMachine
1 2 3 m
Controller
Global Memory
Processors
PRAM•Each processor runs thesame program but has adistinct label
•Each processorcommunicates with anymemory cell in a single timestep.
•Primary resources:Parallel time
Number of processors
![Page 6: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/6.jpg)
Boolean Circuit Family
NOR NOR
NOR NOR NOR
NOR NOR
INPUT INPUT INPUT
OUTPUT OUTPUT
width
depth
•Gates evaluated one levelat a time from input tooutput with no feedback.•One hardwired circuit foreach problem size.•Primary resources
Depth=number of levels≈ parallel time
Width=maximum numberof gates in a level
≈number of processorsWork=total number of gates
![Page 7: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/7.jpg)
Parallel ComputingAdding n numbers can be carried out in O(log n) steps
using O(n) processors.
+
X1 X2
+
X3 X4
+
X5 X6
+
X7 X8
+ +
+
ΣXi
log n
Connected components of a graph can be found inO(log2n) steps using n2 processors.
![Page 8: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/8.jpg)
Complexity Classes andP-completeness
•P is the class of feasible problems: solvable with
polynomial work.
•NC is the class of problems efficiently solved inparallel (polylog depth and polynomial work, NC ⊆ P).
•Are there feasible problems that cannot be solvedefficiently in parallel (P≠NC)?
•P-complete problems are the hardest problems in P to
solve in parallel. It is believed they are inherently
sequential: not solvable in polylog depth.
•The Circuit Value Problem is P-complete.
NOR NOR
NOR NOR NOR
NOR
TRUE FALSE TRUE
?
![Page 9: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/9.jpg)
Statistical PhysicsThe study of the emergent properties of many
particle system using probabilistic methods.
Objects of study are statistical ensembles of
system states or histories:•Equilibrium states--Gibbs distribution:
•Non-equilibrium states--stochastic dynamics
Computational statistical physics: sample theseensembles.
![Page 10: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/10.jpg)
Sampling Complexity
•Monte Carlo simulationsconvert random bits intodescriptions of a typicalsystem states.•What is the depth of theshallowest feasible circuit(running time of the fastestPRAM program) thatgenerates typical states?
NOR NOR
NOR NOR NOR
NOR NOR
Typical System State
depthRandom Bits
Depth is a property of systems in statistical physics
![Page 11: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/11.jpg)
Diffusion Limited Aggregation
•Particles added one at a timewith sticking probabilitiesgiven by the solution ofLaplace’s equation.•Self-organized fractal object
df=1.715… (2D)•Physical systems:
Fluid flow in porous mediaElectrodepositionBacterial colonies
Witten and Sander, PRL 47, 1400 (1981)
![Page 12: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/12.jpg)
Random Walk Dynamics for DLA
![Page 13: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/13.jpg)
![Page 14: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/14.jpg)
#1#3
#2 #1 #3#2
Parallel dynamics ignoresinterference between 1 and 3
Sequential dynamics
The Problem with Parallelizing DLA
![Page 15: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/15.jpg)
Depth of DLATheorem: Determining the shape of an aggregate from therandom walks of the constituent particles is a P-completeproblem.
Proof sketch: Reduce the Circuit Value Problem to DLA dynamics.
Caveats:1. P≠NC not proven2. Average case may be easier than worst case3. Alternative dynamics may be faster than random walk dynamics
a b c
input 1 input 2
power
output
d
![Page 16: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/16.jpg)
132
tentative cluster, step N tentative cluster, step N+1
1. Start with seed particle at the origin and N walk trajectories2. In parallel move all particles along their trajectories to tentative sticking
points on tentative cluster, which is initially the seed particle at the origin.3. New tentative cluster obtained by removing all particles that interfere with
earlier particles.4. Continue until all particles are correctly placed.
Parallel Algorithm for DLAD. Tillberg and JM, PRE 69, 051403 (2004)
2 1
![Page 17: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/17.jpg)
Efficiency of the Algorithm•DLA is a tree whosestructural depth, Ds scales asthe radius of the cluster.•The running time, T of thealgorithm is asymptoticallyproportional to the structuraldepth.
Ds/T
T
![Page 18: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/18.jpg)
Internal DLAParticles start at the origin, random walk and stick where theyfirst leaves the cluster.
•Shape approaches a circle with logarithmic fluctuations.•P-completeness proof fails.
![Page 19: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/19.jpg)
Parallel Algorithm for IDLA1. Start with seed particle at the origin and N walk trajectories2. Place particles at expected positions along their trajectories.3. Iteratively move particles until holes and multiple occupancies are
eliminated
Average parallel timepolylogarthmic orpossibly a small powerin N.
Cluster of 2500 particles made in 6 parallel steps.
1
2 4
3 5
6
C. Moore and JM, J. Stat. Phys. 99, 661 (2000)
![Page 20: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/20.jpg)
Hierarchy of Depth vs Size
Mandelbrot percolation DLA
constant polylog polynomial
T>Tc Ising T=Tc IsingEden growth
Invasion percolation
Eden growthInvasion percolationScale free networksBallistic depositionBak-Sneppen model
Internal DLA
SandpilesGame of Life
![Page 21: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/21.jpg)
What is Physical Complexity?
• “I shall not today attempt further to define the kinds ofmaterial I understand to be embraced with thatshorthand description. ... But I know it when I see it.”– Justice Potter Stewart on pornography
< ≤ <<
![Page 22: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/22.jpg)
History and Complexity
• The emergence of a complex system fromsimple initial conditions requires a long history.
• History can be quantified in terms of thecomputational complexity of simulating statesof the system.
–Charles Bennett
Depth is the appropriate measure to quantify history.
![Page 23: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/23.jpg)
The depth of a physical system isthe depth of a Boolean circuit (orparallel time on a PRAM) tosimulate a typical system statewith polynomial hardware usingthe most efficient algorithm.
Depth of Physical Systems
![Page 24: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/24.jpg)
+ ≈
Maximal Property of Depth
Follows immediately from parallelism.
For a system AB composed of independent subsystems A andB, the depth of the whole is the maximum over subsystems:
![Page 25: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/25.jpg)
Depth is Maximal
Follows immediately from parallelism.
For a system AB composed of independent subsystems A andB, the depth of the whole is the maximum over subsystems:
≈Depth is intensive (nearlyindependent of size) forhomogeneous systems with shortrange correlations.
![Page 26: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/26.jpg)
Depth is greatest at the boundary betweenorder and disorder.
In equilibrium systems with continuousphase transitions, such as the Ising model,depth is greatest at the critical point,separating order and disorder (criticalslowing down).
![Page 27: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/27.jpg)
DLA ComputesTheorem: Determining the shape of an aggregate from therandom walks of the constituent particles is a P-completeproblem.
Proof sketch: Reduce the Circuit Value Problem to DLA dynamics.
DLA dynamics can embody arbitrary computation
a b c
input 1 input 2
power
output
d
![Page 28: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/28.jpg)
Properties of Depth
• Defined for any system in the framework ofstatistical physics.
• For a system of nearly independent subsystemsgiven by the maximum over the subsystems.
• Greatest at the boundary between order anddisorder.
• Systems that solve (P-hard) computationalproblems are deep.
…and physical complexity
![Page 29: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/29.jpg)
Conclusions
Parallel computational complexity theory providesinteresting perspectives on model systems in statisticalphysics.Depth, defined as the minimum number of parallel stepsneeded to simulate a system, is a prerequisite for physicalcomplexity and shares many of its properties.
![Page 30: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/30.jpg)
Discount Communication
≈
Complexity emerges from interactions, notfrom signal propagation.
![Page 31: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/31.jpg)
Choose the Fastest Algorithm
Swendsen-Wang z≈0
Metropolis z≈2
![Page 32: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/32.jpg)
The physical depth, D(A) of a systemA is the average parallel time neededto generate a typical system stateusing the most efficient, feasibleMonte Carlo algorithm for A
Depth of Physical Systems
![Page 33: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/33.jpg)
Models of Computation
![Page 34: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/34.jpg)
Ising Model
!
H = " si< i, j>
# s j
si = ± 1
Aligned spins lower the energy: Two ground states:
or
![Page 35: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/35.jpg)
Statistical Mechanics (T>0)
!
H = " si< i, j>
# s j
!
P[si] = e
"H [si]/kT
Z
T=TcΤ→0 T→∞
Gibbs DistributionCritical point
![Page 36: Complexity, Parallel Computation and Statistical …people.umass.edu/machta/talks/complexity_6-06.pdfOutline •Parallel computing and computational complexity •Complexity of models](https://reader033.vdocuments.us/reader033/viewer/2022042611/5ada97457f8b9a6d318cfacf/html5/thumbnails/36.jpg)
Depth of the Ising Model
Sampling time for the Ising model, isgreatest at the critical point, separatingordered and disordered states (criticalslowing down).
Swendsen-Wang z≈0
Metropolis z≈2