universal laws (and architectures): networks, bugs, brains, dance, art, music, literature, fashion...

Universal laws (and architectures):networks, bugs, brains, dance, art,

music, literature, fashion

John Doyle

John G Braun Professor

Control and Dynamical Systems, EE, & BioE

C a #l t e c h

and

zombi

es

accessibleaccountableaccurateadaptableadministrableaffordableauditableautonomyavailablecredibleprocess

capablecompatiblecomposable configurablecorrectnesscustomizabledebugabledegradabledeterminabledemonstrable

dependabledeployablediscoverable distributabledurableeffectiveefficientevolvableextensiblefail transparentfastfault-tolerantfidelityflexibleinspectableinstallableIntegrityinterchangeableinteroperable learnablemaintainable

manageablemobilemodifiablemodularnomadicoperableorthogonalityportableprecisionpredictableproducibleprovablerecoverablerelevantreliablerepeatablereproducibleresilientresponsivereusable robust

safety scalableseamlessself-sustainableserviceablesupportablesecurablesimplicitystablestandards

compliantsurvivablesustainabletailorabletestabletimelytraceableubiquitousunderstandableupgradableusable

Requirements on systems and architectures

accessibleaccountableaccurateadaptableadministrableaffordableauditableautonomyavailablecredibleprocess

capablecompatiblecomposable configurablecorrectnesscustomizabledebugabledegradabledeterminabledemonstrable

dependabledeployablediscoverable distributabledurableeffectiveefficientevolvableextensiblefail transparentfastfault-tolerantfidelityflexibleinspectableinstallableIntegrityinterchangeableinteroperable learnablemaintainable

manageablemobilemodifiablemodularnomadicoperableorthogonalityportableprecisionpredictableproducibleprovablerecoverablerelevantreliablerepeatablereproducibleresilientresponsivereusable robust

safety scalableseamlessself-sustainableserviceablesupportablesecurablesimplicitystablestandards

compliantsurvivablesustainabletailorabletestabletimelytraceableubiquitousunderstandableupgradableusable

Requirements on systems and architectures

When concepts fail, words arise. Mephistopheles, Faust, Goethe

Mephistopheles. …Enter the templed hall of Certainty.

Student. Yet in each word some concept there must be.

Mephistopheles. Quite true!

But don't torment yourself too anxiously;For at the point where concepts fail,At the right time a word is thrust in there…

When concepts fail, words arise. Mephistopheles, Faust, Goethe

Mephistopheles. …Enter the templed hall of Certainty.Student. Yet in each word some concept there must be.Mephistopheles. Quite true!

But don't torment yourself too anxiously;For at the point where concepts fail,At the right time a word is thrust in there…

• Concrete case studies• Theorems

• Neuroscience

• Tech nets

• Cell biology

• Medical physiology

• Smartgrid, cyber-phys

• Wildfire ecology

• Earthquakes

• Lots of aerospace

• Physics:

– turbulence,

– stat mech (QM?)

• “Toy”:

– Lego

– clothing, fashion

• Buildings, cities

• Synesthesia

Case studies

• Neuroscience

• Tech nets

• Cell biology

• Medical physiology

• Smartgrid, cyber-phys

• Wildfire ecology

• Earthquakes

• Lots of aerospace

• Physics:

– turbulence,

– stat mech (QM?)

• “Toy”:

– Lego

– clothing, fashion

• Buildings, cities

• Synesthesia

Case studies

Cloth

Thread

Fiber

Diverse Garments

Xform

Xform

Xform

Diverse outfits

Constraints that

deconstrainProtocols

The layered architecture of clothing

This paper aims to bridge progress in neuroscience involving sophisticated quantitative analysis of behavior, including the use of robust control, with other relevant conceptual and theoretical frameworks from systems engineering, systems biology, and mathematics.

Doyle, Csete, Proc Nat Acad Sci USA, JULY 25 2011

Most accessibleNo math

Architecture case studies comparison

Bacteria Internet BrainUnderstood? By scientists? Live demos?!?

Who cares? Design quality?

Math?

Fast

Slow

Flexible Inflexible

VOR

d = hand

-vision

d = head

e=d-u

Act

uslowdelay

VOR

fast

vision

Speed and flexibility tradeoffs

Vestibular Ocular Reflex (VOR)

Vestibular Ocular Reflex (VOR)A handwaving explanation illustrating

fundamental tradeoffs

eye

head hand

vision

eye

Head and hand motion

Compensating eye movement

eye

head hand

vision

eye

Easy Hard

Why?

Still Move

Still Easy Easy

Move Hard Hardest

Head

HandAccident or necessity?

d =disturbance

= hand

-visionerror=d-u

Act

u

u =eye position

eyevision

hand

eye

d = hand

-visione=d-u

Act

uslowdelay

eye

hand

vision

slow eye

Hard

-vision

d = head

e=d-u

Act

u

eye

head

vision

eye

-vision

d = head

e=d-u

Act

uVOR

fast

Easy

eye

head

vision

eyefast

-

d = head

e=d-u

Act

uVOR

fast

Easy

eye

head

eyefast

Doesn’t depend on vision

Works in the dark

-vision

d = head

e=d-u

Act

uVOR

fast

Easy

eye

head

vision

eyefast

d = hand

-vision

d = head

e=d-u

Act

uslow

delay

VOR

fast

eye

head hand

vision

eyeslowfast

Standing and seeing

2 legs 1 leg

eyes open Easiest Harder

eyes shut Easy Hardest

Why?

Vision is important in balance?

eyes 2 legs 1 leg

open Easiest Harder

shut Easy Hardest

Fast

Slow

Flexible Inflexible

VOR

vision

Better at low frequencies

Better at high frequencies

Laws and architectures: lessons from VOR and vision

• Robust control– nested, diverse feedbacks– hidden, automatic, unconscious

• Speed vs flexibility tradeoffs (laws?)• Good architectures manage tradeoffs• Evolution: necessity vs accident?• Universal?

efficient

Fast

Slow

Flexible Inflexible

Global Local

(Overly) Simple dichotomous tradeoff pairs

accessibleaccountableaccurateadaptableadministrableaffordableauditableautonomyavailablecompatiblecomposable configurablecorrectnesscustomizabledebugabledegradabledeterminabledemonstrable

dependabledeployablediscoverable distributabledurableeffective

evolvableextensiblefail transparentfastfault-tolerantfidelityflexibleinspectableinstallableIntegrityinterchangeableinteroperable learnablemaintainable

manageablemobilemodifiablemodularnomadicoperableorthogonalityportableprecisionpredictableproducibleprovablerecoverablerelevantreliablerepeatablereproducibleresilientresponsivereusable

safety scalableseamlessself-sustainableserviceablesupportablesecurablesimplestablestandardssurvivable

tailorabletestabletimelytraceableubiquitousunderstandableupgradableusable

efficient

robust

sustainableFast

Slow

Flexible Inflexible

Global Local

Simple dichotomous

tradeoff pairs

PCA Principal Concept Analysis

• Formalize architecture as constraints• Good architecture = “constraints that

deconstrain” (G&K)• Most effective architectures are layered

• Constraints on system and components– System level function and uncertainty– Component level capability and uncertainty

• Laws, hard limits, tradeoffs• Protocols (are the essence of constraints

that deconstrain)

Slow

Fast

Flexible Inflexible

Impossible

Architecture Architecture (constraints that

deconstrain)

General Special

Want fast, flexible, and general

Fast

Slow

Flexible Inflexible

Speed and flexibility tradeoffs

both

vision

VOR

Combining controls

Fast

Slow

Flexible Inflexible

bothlaws? (constraints)

Speed and flexibility tradeoffs

Accident or necessity?vision

VOR

Sensing fast and slow

• Applies to vision and hearing?

• For action (fast, luminance)• For “perception” (slow, includes color)

Stare at the intersection

This is pretty good.

Universal tradeoffs?

Fast

Slow

Flexible Inflexible

Learning

Evolution

SensoryMotor

Prefrontal

Striatum

SlowFlexible

Learning

Ashby & Crossley

Slow

Reflex(Fastest,

LeastFlexible)

SensoryMotor

Prefrontal

Striatum

Fast

FastInflexible

SlowFlexible

Learning

Reflex(Fastest,

LeastFlexible)

Ashby & Crossley

Learning can be very slow.

Sense

Universal tradeoffs?

Fast

Slow

Flexible Inflexible

Motor

Prefrontal

Fast

Learn

Reflex

Evolve

Learning can be very slow.

Evolution is even slower.

Sense

Universal architectures

Motor

Prefrontal

Fast

Reflex

Evolution on

long timescales

LearnE

volve

Evolution on long timescales

accessibleaccountableaccurate

administrableaffordableauditableautonomyavailablecredibleprocess

capablecompatiblecomposable configurablecorrectnesscustomizabledebugabledegradabledeterminabledemonstrable

dependabledeployablediscoverable distributabledurableeffectiveefficient

extensiblefail transparent

fault-tolerantfidelity

inspectableinstallableIntegrityinterchangeableinteroperable learnablemaintainable

manageablemobilemodifiablemodularnomadicoperableorthogonalityportableprecisionpredictableproducibleprovablerecoverablerelevant

repeatablereproducible

responsivereusable

safety scalableseamlessself-sustainableserviceablesupportablesecurablesimplicity

standards compliant

survivablesustainabletailorabletestabletimelytraceableubiquitousunderstandableupgradableusable

Many more dimensions

robust

resilient

stable

adaptable

evolvable

reliableflexible

fast

Universal laws (constraints)

Well known from physics and chemistry:• Classical: gravity, energy, carbon,…• Modern: speed of light, “Heisenberg”, …

Not so much:• Robustness

– Critical to study of complex systems– Unknown outside narrow technical disciplines– Theorems, not mere metaphors

Modern Turing tradeoffs: PC, smartphone, router, etc

AppsOS

HW

Digital

Lumped

Distrib.

OSHW

Digital

Lumped

Distrib.

DigitalLumped

Distrib.

LumpedDistrib. Distrib.

Fast

Slow

Flexible Inflexible

Accident or necessity?

General Special

SensoryMotor

Prefrontal

Striatum

Reflex

Learning

Catabolism

AA

RNA

transl.Proteins

xRNA transc.P

recursors

Nucl.AA

DNARepl.Gene

ATP

ATPRibosome

RNAp

DNAp

Software

Hardware

Digital

Analog

Flexible/ Adaptable/Evolvable

Horizontal Gene

Transfer

Horizontal App

Transfer

Horizontal Meme

Transfer

Depends crucially on

layered architecture

Catabolism

AA

RNA

transl.Proteins

xRNA transc.P

recursors

Nucl.AA

DNARepl.Gene

ATP

ATPRibosome

RNAp

DNAp

Horizontal Gene

Transfer

Sequence ~100 E Coli (not chosen randomly)• ~ 4K genes per cell• ~20K different genes in total• ~ 1K universally shared genes • ~ 300 essential (minimal) genes

See slides on bacterial

biosphere

Mechanisms in molecular biology

Think of this as a “protocol stack”

0. HGT (Horizontal Gene Transfer)1. DNA replication2. DNA repair3. Mutation4. Transcription5. Translation6. Metabolism7. Signal transduction8. …

Think of this as a “protocol stack”

0. HGT1. DNA replication2. DNA repair3. Mutation4. Transcription5. Translation6. Metabolism7. Signal transduction8. …

Highly controlled

Control 1.0

Think of this as a “protocol stack”

0. HGT1. DNA replication2. DNA repair3. Mutation4. Transcription5. Translation6. Metabolism7. Signal transduction8. …

Highly controlled

Highly controlled

?!?

Control 2.0

AppsOSHW

DigitalLumpedDistrib.

OSHW

DigitalLumpedDistrib.

DigitalLumpedDistrib.

LumpedDistrib. Distrib.

FastCostly

SlowCheap

Flexible Inflexible

General Special

HGTDNA replication

DNA repair Mutation

Transcription Translation

MetabolismSignal…

Prosen-cephalon

Telen-cephalon

Rhinencephalon, Amygdala, Hippocampus, Neocortex,

Basal ganglia, Lateral ventricles

Dien-cephalon

Epithalamus, Thalamus, Hypothalamus,

Subthalamus, Pituitary gland, Pineal gland, Third

ventricle

Brain stem

Mesen-cephalon

Tectum, Cerebral peduncle, Pretectum, Mesencephalic

duct

Rhomb-encephalon

Meten-cephalon

Pons, Cerebellum

Myelen-cephalon

Medulla oblongata

Spinal cord

CNS HW “stack”

Brain

Slow

Fast

Flexible Inflexible

Undecidable

Universal Turing

MachineArchitecture

(constraints that deconstrain)

General Special

Laws and architectures(Turing, 1936)

Turing 1912-1954

Fast

Slow

Flexible/General

Inflexible/Specific

Undecidable NP P

hard limits

Really slow

Computational complexity

Decidable

Flexible/General

Inflexible/Specific

NP P

Decidable

Computational complexity

PSPACENPPNLPSPACE ≠ NL

NLPSPACE

Space is powerful and/or cheap.

Fast

Slow

Flexible/General

Inflexible/Specific

Undecidable NP P

hard limits

Really slow

These are hard limits on the intrinsic computational complexity of problems.

Decidable

Must still seek algorithms that achieve the limits, and architectures that support this process.

Control, OR

CommsCompute

Physics

Shannon

Bode

Turing

Gödel

EinsteinHeisenberg

Carnot

Boltzmann

Theory?Deep, but fragmented, incoherent, incomplete

Nash

Von Neumann

Control, OR

CommunicateCompute

Physics

Shannon

Bode

Turing

Einstein

Heisenberg

Carnot

Boltzmann

Delay and risk are

most important

Delay and risk are

least important

Control, OR

Compute

Bode

Turing

Delay and risk are

most important

• Worst-case (risk)• Time complexity (delay)

• Worst-case (risk)• Delay severely degrades

robust performance

Communicate

Physics

Shannon

Einstein

Heisenberg

Carnot

Boltzmann

Delay and risk are

least important

• Average case (risk neutral)• Random ensembles• Asymptotic (infinite delay)

• Space complexity

Control, OR

CommunicateCompute

Physics

Shannon

Bode

Turing

Delay and risk are

most important

Delay and risk are

least important

New progress!

Slow

Fast

Flexible Inflexible

Impossible

Architecture Architecture (constraints that

deconstrain)

General Special

How general is this picture?

simple tech

complex tech

How general is this picture?

wasteful

fragile

efficient

robust

Implications for human evolution?Cognition?Technology?Basic sciences?

Chandra, Buzi, and Doyle

UG biochem, math, control theory

Most important paper so far.

simple enzyme

Fragility

Enzyme amount

complex enzyme

lnz p

z p

2 20

1ln ln

z z pS j d

z z p

Theorem!

z and p functions of enzyme complexity

and amount

Savageaumics

k

z p

z p

10-1

100

10110

0

101

Simple, but too fragile

complex

No tradeoff

Hard tradeoff in glycolysis is• robustness vs efficiency• absent without autocatalysis• too fragile with simple control• plausibly robust with complex control

expensive

fragile

2 20

1ln

ln

zS j d

z

z p

z p

hard harder hardest!

Easy to prove using simple models.

What is sensed matters.

Why?

l

This is a cartoon, but can be made precise.

Frag

ility change

length

lnz p

z p

2 20

2 20

1 2ln ln

1 2ln ln

z z pS j d

z z p

p z pT j d

p z p

L

change sense

down

1

1 r

1

1

z p

z p

l

I recently found this paper, a rare example of exploring an explicit tradeoff between robustness and efficiency.

This seems like an important paper but it is rarely cited.

1m

Bacteria

Phage

Multiply

Survive

Phage lifecycle

InfectLyse

slow

fragile

fast

robust

Survive

Multiply

thinsmall

Good architectures?Hard limits?

thickbig

CapsidGenome

Universal architectures

What can go wrong?

Horizontal Bad Gene Transfer

Horizontal Bad App Transfer

Horizontal Bad Meme

Transfer

Parasites & Hijacking

Fragility?

Exploiting layered

architecture

Virus

Virus

Unfortunately, not intelligent design

Ouch.

Why?

left recurrent laryngeal nerve

Why? Building humans from fish parts.

Fish parts

It could be worse.

Original design challenge?

TCP/IP

Deconstrained(Hardware)

Deconstrained(Applications)

Constrained • Expensive mainframes• Trusted end systems• Homogeneous• Sender centric• Unreliable comms

Facilitated wild evolutionCreated

• whole new ecosystem• completely opposite

Networked OS

CPU/Mem

Dev2CPU/

Mem

Dev CPU/

Mem

Dev2

Dev2

App AppIPC

Global and direct access to

physical address!

DNS

IP addresses interfaces

(not nodes)

caltech.edu?

131.215.9.49

CPU/Mem

Dev2CPU/

Mem

Dev CPU/

Mem

Dev2

Dev2

App AppIPC

Global and direct access to

physical address!

Robust?• Secure• Scalable• Verifiable• Evolvable• Maintainable• Designable• …

DNS

IP addresses interfaces

(not nodes)

Physical

IP

TCP

Application

Naming and addressing need to have scope and • resolved within layer• translated between layers• not exposed outside of layer

Related “issues”• VPNs• NATS• Firewalls• Multihoming• Mobility• Routing table size• Overlays• …

?

Deconstrained(Hardware)

Deconstrained(Applications)

Next layered architectures

Constrained Control, share, virtualize, and manage resources

CommsMemory, storageLatencyProcessingCyber-physical

Few global variables

Don’t cross layers

Every layer has

different diverse graphs.

Architecture is least graph topology.

Architecture facilitates arbitrary graphs.

Persistent errors and confusion (“network science”)

Physical

IP

TCP

Application