15-252 Spring 2021 : Lecture 4

Kolmogorov complexity

How"

intrinsically complex"

or

" random"is

an object ?✓

Strings over a finitealphabet ⇐

* )

A - B3333.3333 Which string is most

-random ?

B =2718281828

C = 1764212643Which is most complex ?

A is a simple string

BIC look more random /complex .

'

Turns out B also has a simpledescriptions

B - first to digitsof e

C - I wrotewhat came to my

mind (sampled arandom

(o digitnumber!

Kolmogorov - Chaitin or descriptive complexB

aims to measure the complexity of a Anglobred

Complexity should berelated to the shortest

'description of th object.

"How compressible is the string"?

Intuition natural - but meaningful defy is

subtle & important to get

Beware of Berry's paradoxMsht

Need to define valid description method .

Description : partial function f : s*→s*

To describe x,we use

(⇐ 101137

a description p s -t fcp) = a

Complexity of a under this desc. method f

Cf Gc) : = min { Ipl : ftp)=x }

What is f ? what to pick I€ wanted complexity ofx to be intrinsic tox

Cf Cx) depends on f .

Imagines f(g) = 0210110110010110111f-Hock x # ogling IM

So can arbitrarily favor certain strip- watt

short descriptions .-3 Not GOOD-

GGIo.li. Develop a"

universal descriptions method"

L

-

↳"Best

' ' desc . method that givesthe

minimum description length to all strings .

Defy (Minimization) API"

f.no. EE, z*

minorities a partial fn g : E*→s* *

of F constant a set forced sees

Cf Cx) s agent t c

God , Minimal universal desc.

method U

which minorizes everyother potentillaHoned

description method .

Define Kolmogorov Complexity

Kcal := glad

#it?• What

'

does such a U look like ?

If we allow all partial fas then

no such U can exist .

Ulp,)=X , Ukip Hr.

. Utpal - kn .

. .

P, L Bec . . Lpn h . .

infinite. Seg .

Subsequence 't>

Pi ,s Pne L. .

. pin

Pi , > 21PM hlp , )= Ulpi,)

Piz > 21h21 hbhlz ? (Pid??Infinitely many strip Whare descriptions

of shalt the length under h compacduto

.

Description : partial recursive fuchsincomputable)

Z TM M s - t on input p forwhich

fb) is defined,Mon inputp

halts

with ftp) written on output tape .

Computable wayto map descriptors p

to the strings X=fCp)

theorem : F partial recursive fg U thatMinories every other partial seaman

' th.

Proof : U = a Universal Turing Machine

U on input (M ,w>

Simulates Mon wand does whatever

M does on w

K¥064 := min l l LM.ws/ : TM M

halt oninput

WI outputsx }

Note u ( LM ,w>) =x

Let's argue U minories everyother

partial recursive fa , say g.

Let TM N compute of .

For xEE* 8 let p be ashortest story .

Sit g (phx ,ie N Cpl = K

N on input phe159outputs x

How to describe x under U ?

Clearly U ( LN , p >) =x

KN , p> I = CN + Ipl

↳ const . dependsonN -

CTO encode pain ka, by, repeateach bit of a

twice, add 01 , thenwrite b - Ka,by a 2) at -141-12

8. FK Cola ) s ont Ipl= cent Cg Ge) ④

Deth (Kolmogorov Complexity) of a

string x is defined as

Kcal Could= mins KM.ws/ a TM M

halt

on input w& outputs x }

Note M can runon w fora long time,

doesn't factor intoKhed

=

Defy:(Incompressible ship )

A string xis said to be incompressible on

Kolmogorovrandom if Kcx) > la I

Note . Tx ,K Gc) I felt 04 )

(Justify this ? )

Use"

print x"

as the program

Lenya : F incompressible strip of every

length .

Pf . There are 2"

binary strings oflength n .

but only 211 binary strip of

length L n.

ID

Examples :

KCon ) s log n toll)( Specify n in binary 8 a

program that on input n'

in biliary

outputs on )K ( first n digits of e ) ← login toll)

K ( yon-M D = In ± OLD (

Exercise)

→ y is an incompressible spy'

og

length in

theorem :

↳ { a 1 kcal 31×1 }is undecidable .

[It is undecidable to tell ifa

Sony is compressible )

PI : Suppose 8 TM p that decides

path if a shiny is incompressible .large

enough .

Consider TM p'

: :

--Onn input Ln >:

• Russ P onall strings of length

is

in lexicographic order .

• For the first Amy? thatP reports to

be incompressible , outputZ

.

Lip ! Cns ) is a description of z

Ktp! an>71=04)+ log n

OTOH , KEE) z n .

Cog P says

z isincompressible,

→ n s log n told

a contradiction cos nis largeenough )

can use Kolmogorov complexity to prove

Godel first incompleteness theorem

For Any sufficiently expressive

consistent mathematical theory

Fai:m7:÷tm'T:mExpress statements like 4kcal > L

"

Using similar Berry paradox like

argument , can proc:

IL St Fx"Kcal > L

'

is not provable .

OTOH Ix at Ktx > L

(by counting )

⇒ A true statementthat can't

be

proud in a theorythatcan

express"Kfc)>L

"

type statements.