computing distributions using random walks on graphs guy kindler guy kindler dimacs dan romik dan...

Computing Computing

Distributions using Distributions using

Random Walks on Random Walks on

GraphsGraphs

Computing Computing

Distributions using Distributions using

Random Walks on Random Walks on

GraphsGraphsGuy KindlerGuy Kindler

DIMACSDIMACS

Dan RomikDan Romik

Weizmann Institute Weizmann Institute

of Scienceof Science

Computing distributionsComputing distributionsComputing distributionsComputing distributions

[Knuth, Yao 76][Knuth, Yao 76]

Given a Given a source source of random bits,of random bits,

output a sample with givenoutput a sample with given distribution distribution DD. .

11

FormalizationFormalizationFormalizationFormalization

Example:Example: ComputeCompute D={(“0”,1/2), (“1”,1/4), D={(“0”,1/2), (“1”,1/4),

(“2”,1/4)}(“2”,1/4)} “H”=right “T”=left

1122“1”“1”

00Interpretation:Interpretation:

Computing a distribution using a Computing a distribution using a

random walk on a binary treerandom walk on a binary tree..

Infinite trees are sometimes Infinite trees are sometimes neededneeded

Infinite trees are sometimes Infinite trees are sometimes neededneeded

D={(“0”,2/3), (“1”,1/3)}D={(“0”,2/3), (“1”,1/3)}

00

00

11

o Requirement:Requirement:

Output is reached with Output is reached with

probability probability 11

o [Knuth, Yao 76][Knuth, Yao 76] Output can be Output can be

reached in expected timereached in expected time

Ent(D)+O(1)Ent(D)+O(1)

Tight!Tight!Tight!Tight!

Some other modelsSome other modelsSome other modelsSome other models

o [Romik ’99][Romik ’99] Generate dist. Generate dist. BB from dist. from dist. AA in optimal in optimal time.time.

o [von Neumann ’51][von Neumann ’51] Generate unbiased coins from Generate unbiased coins from biased ones (when bias is unknown).biased ones (when bias is unknown).

o [Keane & O’Brien ’94][Keane & O’Brien ’94] Generate Generate f(p)f(p)-biased coins -biased coins from from pp-biased ones.-biased ones.

o [Peres & Nacu ’03][Peres & Nacu ’03] Generate Generate f(p)f(p)-biased in “good -biased in “good time”.time”.

o [Mossel & Peres ’03][Mossel & Peres ’03] Generate Generate f(p)f(p)-biased coins -biased coins from from pp-biased, using a finite graph.-biased, using a finite graph.

Finite state generatorsFinite state generatorsFinite state generatorsFinite state generators

“0”“0”

“1”“1” “0”“0”

“1”“1”

“0”“0”

Output:Output:110…

[Knuth+Yao]:[Knuth+Yao]:

Finite state generatorsFinite state generatorsFinite state generatorsFinite state generators

“0”“0”

“1”“1” “0”“0”

“1”“1”

“0”“0”

o Interpretation – binary representation:Interpretation – binary representation:

Generating a random variable Generating a random variable

on on [0,1][0,1] using a using a random walk random walk

on a graphon a graph

o Definition:Definition: A distribution function A distribution function

is computable, if it is the is computable, if it is the

output distribution of some f.s.g.output distribution of some f.s.g.

o Question [Knuth+Yao]:Question [Knuth+Yao]: which which

distributions are computable?distributions are computable?smooth/analyticsmooth/analytic

Output:Output:110…

History of the problemHistory of the problemHistory of the problemHistory of the problem

o [Knuth+Yao ’76][Knuth+Yao ’76] Computable analytic density Computable analytic density

functions must be polynomials with rational functions must be polynomials with rational

coefficientscoefficients

o [Yao ’84] [Yao ’84] The roots of such functions must be The roots of such functions must be

rationalrationalo [this work] [this work]

1.1. All functions with above properties can be All functions with above properties can be

computedcomputed

2.2. Allowing smooth functions does not add Allowing smooth functions does not add

computable functions.computable functions.

We’ll discuss…We’ll discuss…We’ll discuss…We’ll discuss…

[Theorem][Theorem] Let Let DD be a be a

distribution with density distribution with density

function function ff. If. If

o ff is a non-negative is a non-negative

polynomiapolynomiall

o with rational coefficientswith rational coefficients

o and no irrational roots in and no irrational roots in

[0,1][0,1],,

then then DD is computable. is computable.

“0”“0”

“1”“1” “0”“0”

“1”“1”

“0”“0”

Generating some Generating some distributionsdistributions

Generating some Generating some distributionsdistributions

uniform distribution:uniform distribution:

Generating Generating max(X,Y)max(X,Y)::

o run two f.s.g’s “in run two f.s.g’s “in

parallel”parallel”

o output the maximumoutput the maximum

“0”“0”“1”“1”o All distributions with All distributions with

density of the form:density of the form:

f(x)=c xf(x)=c xmm(1-x)(1-x)nn

[Knuth+Yao ’76][Knuth+Yao ’76]

o All order statistics All order statistics

of independent of independent

uniform variablesuniform variables

More distributionsMore distributionsMore distributionsMore distributions

[Knuth+Yao ’76]:[Knuth+Yao ’76]:

uniform on uniform on [a,b][a,b], for , for aa,,bb

rational.rational.

All distributions with All distributions with

density of the form:density of the form:

f(x)=c (x-a)f(x)=c (x-a)mm(b-(b-

x)x)nn11[a,b][a,b](x)(x)Generating Generating max(X,Y)max(X,Y)::

o run two f.s.g’s “in run two f.s.g’s “in

parallel”parallel”

o output the maximumoutput the maximum

o easy:easy: if if ff1,..,1,..,ffkk are are

computable, then so iscomputable, then so is

aa11ff11+…a+…akkffkk

(for (for aaii rational) rational)

All distributionsAll distributionsAll distributionsAll distributions

Answer:Answer: all polynomialsall polynomialswith rational with rational coefficients, and no coefficients, and no irrational roots in irrational roots in

[0,1][0,1] !!

Answer:Answer: all polynomialsall polynomialswith rational with rational coefficients, and no coefficients, and no irrational roots in irrational roots in

[0,1][0,1] !!

All distributions with All distributions with

density of the form:density of the form:

f(x)=c (x-a)f(x)=c (x-a)mm(b-(b-

x)x)nn11[a,b][a,b](x)(x)

Question:Question: what is the what is the set of rational set of rational mixtures of such mixtures of such functions ?functions ?

Question:Question: what is the what is the set of rational set of rational mixtures of such mixtures of such functions ?functions ?

Proof:Proof:

1.1. Geometric in natureGeometric in nature

2.2. Non-constructiveNon-constructive

Proof:Proof:

1.1. Geometric in natureGeometric in nature

2.2. Non-constructiveNon-constructive

Q.E.D. !

Q.E.D. !

Q.E.D. !

Q.E.D. !

ConclusionsConclusionsConclusionsConclusions

o We solved the computability problem in the f.s.g. We solved the computability problem in the f.s.g. model, for smooth functions.model, for smooth functions.

o We have no good bounds on complexity (size of We have no good bounds on complexity (size of graph) in this model.graph) in this model.

Open problemsOpen problemsOpen problemsOpen problemso Solve for other computational models (stack Solve for other computational models (stack

automaton? automaton? [Yao84][Yao84]))o Solve the general computablity question (no Solve the general computablity question (no

smoothness restriction) smoothness restriction) o Solve the complexity questionSolve the complexity question

The End…The End…

The End…The End…