universal laws (and architectures): networks, bugs, brains, dance, art, music, literature, fashion...
TRANSCRIPT
Universal laws (and architectures):networks, bugs, brains, dance, art,
music, literature, fashion
John Doyle
John G Braun Professor
Control and Dynamical Systems, EE, & BioE
C a #l t e c h
and
zombi
es
accessibleaccountableaccurateadaptableadministrableaffordableauditableautonomyavailablecredibleprocess
capablecompatiblecomposable configurablecorrectnesscustomizabledebugabledegradabledeterminabledemonstrable
dependabledeployablediscoverable distributabledurableeffectiveefficientevolvableextensiblefail transparentfastfault-tolerantfidelityflexibleinspectableinstallableIntegrityinterchangeableinteroperable learnablemaintainable
manageablemobilemodifiablemodularnomadicoperableorthogonalityportableprecisionpredictableproducibleprovablerecoverablerelevantreliablerepeatablereproducibleresilientresponsivereusable robust
safety scalableseamlessself-sustainableserviceablesupportablesecurablesimplicitystablestandards
compliantsurvivablesustainabletailorabletestabletimelytraceableubiquitousunderstandableupgradableusable
Requirements on systems and architectures
accessibleaccountableaccurateadaptableadministrableaffordableauditableautonomyavailablecredibleprocess
capablecompatiblecomposable configurablecorrectnesscustomizabledebugabledegradabledeterminabledemonstrable
dependabledeployablediscoverable distributabledurableeffectiveefficientevolvableextensiblefail transparentfastfault-tolerantfidelityflexibleinspectableinstallableIntegrityinterchangeableinteroperable learnablemaintainable
manageablemobilemodifiablemodularnomadicoperableorthogonalityportableprecisionpredictableproducibleprovablerecoverablerelevantreliablerepeatablereproducibleresilientresponsivereusable robust
safety scalableseamlessself-sustainableserviceablesupportablesecurablesimplicitystablestandards
compliantsurvivablesustainabletailorabletestabletimelytraceableubiquitousunderstandableupgradableusable
Requirements on systems and architectures
When concepts fail, words arise. Mephistopheles, Faust, Goethe
Mephistopheles. …Enter the templed hall of Certainty.
Student. Yet in each word some concept there must be.
Mephistopheles. Quite true!
But don't torment yourself too anxiously;For at the point where concepts fail,At the right time a word is thrust in there…
When concepts fail, words arise. Mephistopheles, Faust, Goethe
Mephistopheles. …Enter the templed hall of Certainty.Student. Yet in each word some concept there must be.Mephistopheles. Quite true!
But don't torment yourself too anxiously;For at the point where concepts fail,At the right time a word is thrust in there…
• Concrete case studies• Theorems
• Neuroscience
• Tech nets
• Cell biology
• Medical physiology
• Smartgrid, cyber-phys
• Wildfire ecology
• Earthquakes
• Lots of aerospace
• Physics:
– turbulence,
– stat mech (QM?)
• “Toy”:
– Lego
– clothing, fashion
• Buildings, cities
• Synesthesia
Case studies
• Neuroscience
• Tech nets
• Cell biology
• Medical physiology
• Smartgrid, cyber-phys
• Wildfire ecology
• Earthquakes
• Lots of aerospace
• Physics:
– turbulence,
– stat mech (QM?)
• “Toy”:
– Lego
– clothing, fashion
• Buildings, cities
• Synesthesia
Case studies
Cloth
Thread
Fiber
Diverse Garments
Xform
Xform
Xform
Diverse outfits
Constraints that
deconstrainProtocols
The layered architecture of clothing
This paper aims to bridge progress in neuroscience involving sophisticated quantitative analysis of behavior, including the use of robust control, with other relevant conceptual and theoretical frameworks from systems engineering, systems biology, and mathematics.
Doyle, Csete, Proc Nat Acad Sci USA, JULY 25 2011
Most accessibleNo math
Architecture case studies comparison
Bacteria Internet BrainUnderstood? By scientists? Live demos?!?
Who cares? Design quality?
Math?
Fast
Slow
Flexible Inflexible
VOR
d = hand
-vision
d = head
e=d-u
Act
uslowdelay
VOR
fast
vision
Speed and flexibility tradeoffs
Vestibular Ocular Reflex (VOR)
Vestibular Ocular Reflex (VOR)A handwaving explanation illustrating
fundamental tradeoffs
eye
head hand
vision
eye
Head and hand motion
Compensating eye movement
eye
head hand
vision
eye
Easy Hard
Why?
Still Move
Still Easy Easy
Move Hard Hardest
Head
HandAccident or necessity?
d =disturbance
= hand
-visionerror=d-u
Act
u
u =eye position
eyevision
hand
eye
d = hand
-visione=d-u
Act
uslowdelay
eye
hand
vision
slow eye
Hard
-vision
d = head
e=d-u
Act
u
eye
head
vision
eye
-vision
d = head
e=d-u
Act
uVOR
fast
Easy
eye
head
vision
eyefast
-
d = head
e=d-u
Act
uVOR
fast
Easy
eye
head
eyefast
Doesn’t depend on vision
Works in the dark
-vision
d = head
e=d-u
Act
uVOR
fast
Easy
eye
head
vision
eyefast
d = hand
-vision
d = head
e=d-u
Act
uslow
delay
VOR
fast
eye
head hand
vision
eyeslowfast
Standing and seeing
2 legs 1 leg
eyes open Easiest Harder
eyes shut Easy Hardest
Why?
Vision is important in balance?
eyes 2 legs 1 leg
open Easiest Harder
shut Easy Hardest
Fast
Slow
Flexible Inflexible
VOR
vision
Better at low frequencies
Better at high frequencies
Laws and architectures: lessons from VOR and vision
• Robust control– nested, diverse feedbacks– hidden, automatic, unconscious
• Speed vs flexibility tradeoffs (laws?)• Good architectures manage tradeoffs• Evolution: necessity vs accident?• Universal?
efficient
Fast
Slow
Flexible Inflexible
Global Local
(Overly) Simple dichotomous tradeoff pairs
accessibleaccountableaccurateadaptableadministrableaffordableauditableautonomyavailablecompatiblecomposable configurablecorrectnesscustomizabledebugabledegradabledeterminabledemonstrable
dependabledeployablediscoverable distributabledurableeffective
evolvableextensiblefail transparentfastfault-tolerantfidelityflexibleinspectableinstallableIntegrityinterchangeableinteroperable learnablemaintainable
manageablemobilemodifiablemodularnomadicoperableorthogonalityportableprecisionpredictableproducibleprovablerecoverablerelevantreliablerepeatablereproducibleresilientresponsivereusable
safety scalableseamlessself-sustainableserviceablesupportablesecurablesimplestablestandardssurvivable
tailorabletestabletimelytraceableubiquitousunderstandableupgradableusable
efficient
robust
sustainableFast
Slow
Flexible Inflexible
Global Local
Simple dichotomous
tradeoff pairs
PCA Principal Concept Analysis
• Formalize architecture as constraints• Good architecture = “constraints that
deconstrain” (G&K)• Most effective architectures are layered
• Constraints on system and components– System level function and uncertainty– Component level capability and uncertainty
• Laws, hard limits, tradeoffs• Protocols (are the essence of constraints
that deconstrain)
Slow
Fast
Flexible Inflexible
Impossible
Architecture Architecture (constraints that
deconstrain)
General Special
Want fast, flexible, and general
Fast
Slow
Flexible Inflexible
Speed and flexibility tradeoffs
both
vision
VOR
Combining controls
Fast
Slow
Flexible Inflexible
bothlaws? (constraints)
Speed and flexibility tradeoffs
Accident or necessity?vision
VOR
Sensing fast and slow
• Applies to vision and hearing?
• For action (fast, luminance)• For “perception” (slow, includes color)
Stare at the intersection
This is pretty good.
Universal tradeoffs?
Fast
Slow
Flexible Inflexible
Learning
Evolution
SensoryMotor
Prefrontal
Striatum
SlowFlexible
Learning
Ashby & Crossley
Slow
Reflex(Fastest,
LeastFlexible)
SensoryMotor
Prefrontal
Striatum
Fast
FastInflexible
SlowFlexible
Learning
Reflex(Fastest,
LeastFlexible)
Ashby & Crossley
Learning can be very slow.
Sense
Universal tradeoffs?
Fast
Slow
Flexible Inflexible
Motor
Prefrontal
Fast
Learn
Reflex
Evolve
Learning can be very slow.
Evolution is even slower.
Sense
Universal architectures
Motor
Prefrontal
Fast
Reflex
Evolution on
long timescales
LearnE
volve
Evolution on long timescales
accessibleaccountableaccurate
administrableaffordableauditableautonomyavailablecredibleprocess
capablecompatiblecomposable configurablecorrectnesscustomizabledebugabledegradabledeterminabledemonstrable
dependabledeployablediscoverable distributabledurableeffectiveefficient
extensiblefail transparent
fault-tolerantfidelity
inspectableinstallableIntegrityinterchangeableinteroperable learnablemaintainable
manageablemobilemodifiablemodularnomadicoperableorthogonalityportableprecisionpredictableproducibleprovablerecoverablerelevant
repeatablereproducible
responsivereusable
safety scalableseamlessself-sustainableserviceablesupportablesecurablesimplicity
standards compliant
survivablesustainabletailorabletestabletimelytraceableubiquitousunderstandableupgradableusable
Many more dimensions
robust
resilient
stable
adaptable
evolvable
reliableflexible
fast
Universal laws (constraints)
Well known from physics and chemistry:• Classical: gravity, energy, carbon,…• Modern: speed of light, “Heisenberg”, …
Not so much:• Robustness
– Critical to study of complex systems– Unknown outside narrow technical disciplines– Theorems, not mere metaphors
Modern Turing tradeoffs: PC, smartphone, router, etc
AppsOS
HW
Digital
Lumped
Distrib.
OSHW
Digital
Lumped
Distrib.
DigitalLumped
Distrib.
LumpedDistrib. Distrib.
Fast
Slow
Flexible Inflexible
Accident or necessity?
General Special
SensoryMotor
Prefrontal
Striatum
Reflex
Learning
Catabolism
AA
RNA
transl.Proteins
xRNA transc.P
recursors
Nucl.AA
DNARepl.Gene
ATP
ATPRibosome
RNAp
DNAp
Software
Hardware
Digital
Analog
Flexible/ Adaptable/Evolvable
Horizontal Gene
Transfer
Horizontal App
Transfer
Horizontal Meme
Transfer
Depends crucially on
layered architecture
Catabolism
AA
RNA
transl.Proteins
xRNA transc.P
recursors
Nucl.AA
DNARepl.Gene
ATP
ATPRibosome
RNAp
DNAp
Horizontal Gene
Transfer
Sequence ~100 E Coli (not chosen randomly)• ~ 4K genes per cell• ~20K different genes in total• ~ 1K universally shared genes • ~ 300 essential (minimal) genes
See slides on bacterial
biosphere
Mechanisms in molecular biology
Think of this as a “protocol stack”
0. HGT (Horizontal Gene Transfer)1. DNA replication2. DNA repair3. Mutation4. Transcription5. Translation6. Metabolism7. Signal transduction8. …
Think of this as a “protocol stack”
0. HGT1. DNA replication2. DNA repair3. Mutation4. Transcription5. Translation6. Metabolism7. Signal transduction8. …
Highly controlled
Control 1.0
Think of this as a “protocol stack”
0. HGT1. DNA replication2. DNA repair3. Mutation4. Transcription5. Translation6. Metabolism7. Signal transduction8. …
Highly controlled
Highly controlled
?!?
Control 2.0
AppsOSHW
DigitalLumpedDistrib.
OSHW
DigitalLumpedDistrib.
DigitalLumpedDistrib.
LumpedDistrib. Distrib.
FastCostly
SlowCheap
Flexible Inflexible
General Special
HGTDNA replication
DNA repair Mutation
Transcription Translation
MetabolismSignal…
Prosen-cephalon
Telen-cephalon
Rhinencephalon, Amygdala, Hippocampus, Neocortex,
Basal ganglia, Lateral ventricles
Dien-cephalon
Epithalamus, Thalamus, Hypothalamus,
Subthalamus, Pituitary gland, Pineal gland, Third
ventricle
Brain stem
Mesen-cephalon
Tectum, Cerebral peduncle, Pretectum, Mesencephalic
duct
Rhomb-encephalon
Meten-cephalon
Pons, Cerebellum
Myelen-cephalon
Medulla oblongata
Spinal cord
CNS HW “stack”
Brain
Slow
Fast
Flexible Inflexible
Undecidable
Universal Turing
MachineArchitecture
(constraints that deconstrain)
General Special
Laws and architectures(Turing, 1936)
Turing 1912-1954
Fast
Slow
Flexible/General
Inflexible/Specific
Undecidable NP P
hard limits
Really slow
Computational complexity
Decidable
Flexible/General
Inflexible/Specific
NP P
Decidable
Computational complexity
PSPACENPPNLPSPACE ≠ NL
NLPSPACE
Space is powerful and/or cheap.
Fast
Slow
Flexible/General
Inflexible/Specific
Undecidable NP P
hard limits
Really slow
These are hard limits on the intrinsic computational complexity of problems.
Decidable
Must still seek algorithms that achieve the limits, and architectures that support this process.
Control, OR
CommsCompute
Physics
Shannon
Bode
Turing
Gödel
EinsteinHeisenberg
Carnot
Boltzmann
Theory?Deep, but fragmented, incoherent, incomplete
Nash
Von Neumann
Control, OR
CommunicateCompute
Physics
Shannon
Bode
Turing
Einstein
Heisenberg
Carnot
Boltzmann
Delay and risk are
most important
Delay and risk are
least important
Control, OR
Compute
Bode
Turing
Delay and risk are
most important
• Worst-case (risk)• Time complexity (delay)
• Worst-case (risk)• Delay severely degrades
robust performance
Communicate
Physics
Shannon
Einstein
Heisenberg
Carnot
Boltzmann
Delay and risk are
least important
• Average case (risk neutral)• Random ensembles• Asymptotic (infinite delay)
• Space complexity
Control, OR
CommunicateCompute
Physics
Shannon
Bode
Turing
Delay and risk are
most important
Delay and risk are
least important
New progress!
Slow
Fast
Flexible Inflexible
Impossible
Architecture Architecture (constraints that
deconstrain)
General Special
How general is this picture?
simple tech
complex tech
How general is this picture?
wasteful
fragile
efficient
robust
Implications for human evolution?Cognition?Technology?Basic sciences?
Chandra, Buzi, and Doyle
UG biochem, math, control theory
Most important paper so far.
simple enzyme
Fragility
Enzyme amount
complex enzyme
lnz p
z p
2 20
1ln ln
z z pS j d
z z p
Theorem!
z and p functions of enzyme complexity
and amount
Savageaumics
k
z p
z p
10-1
100
10110
0
101
Simple, but too fragile
complex
No tradeoff
Hard tradeoff in glycolysis is• robustness vs efficiency• absent without autocatalysis• too fragile with simple control• plausibly robust with complex control
expensive
fragile
2 20
1ln
ln
zS j d
z
z p
z p
hard harder hardest!
Easy to prove using simple models.
What is sensed matters.
Why?
l
This is a cartoon, but can be made precise.
Frag
ility change
length
lnz p
z p
2 20
2 20
1 2ln ln
1 2ln ln
z z pS j d
z z p
p z pT j d
p z p
L
change sense
down
1
1 r
1
1
z p
z p
l
I recently found this paper, a rare example of exploring an explicit tradeoff between robustness and efficiency.
This seems like an important paper but it is rarely cited.
1m
Bacteria
Phage
Multiply
Survive
Phage lifecycle
InfectLyse
slow
fragile
fast
robust
Survive
Multiply
thinsmall
Good architectures?Hard limits?
thickbig
CapsidGenome
Universal architectures
What can go wrong?
Horizontal Bad Gene Transfer
Horizontal Bad App Transfer
Horizontal Bad Meme
Transfer
Parasites & Hijacking
Fragility?
Exploiting layered
architecture
Virus
Virus
Unfortunately, not intelligent design
Ouch.
Why?
left recurrent laryngeal nerve
Why? Building humans from fish parts.
Fish parts
It could be worse.
Original design challenge?
TCP/IP
Deconstrained(Hardware)
Deconstrained(Applications)
Constrained • Expensive mainframes• Trusted end systems• Homogeneous• Sender centric• Unreliable comms
Facilitated wild evolutionCreated
• whole new ecosystem• completely opposite
Networked OS
CPU/Mem
Dev2CPU/
Mem
Dev CPU/
Mem
Dev2
Dev2
App AppIPC
Global and direct access to
physical address!
DNS
IP addresses interfaces
(not nodes)
caltech.edu?
131.215.9.49
CPU/Mem
Dev2CPU/
Mem
Dev CPU/
Mem
Dev2
Dev2
App AppIPC
Global and direct access to
physical address!
Robust?• Secure• Scalable• Verifiable• Evolvable• Maintainable• Designable• …
DNS
IP addresses interfaces
(not nodes)
Physical
IP
TCP
Application
Naming and addressing need to have scope and • resolved within layer• translated between layers• not exposed outside of layer
Related “issues”• VPNs• NATS• Firewalls• Multihoming• Mobility• Routing table size• Overlays• …
?
Deconstrained(Hardware)
Deconstrained(Applications)
Next layered architectures
Constrained Control, share, virtualize, and manage resources
CommsMemory, storageLatencyProcessingCyber-physical
Few global variables
Don’t cross layers
Every layer has
different diverse graphs.
Architecture is least graph topology.
Architecture facilitates arbitrary graphs.
Persistent errors and confusion (“network science”)
Physical
IP
TCP
Application
