Fat tails, hard limits, thin layers John Doyle, Caltech

Fat tails, hard limits, thin layers John Doyle, Caltech
Rethinking fundamentals* Parameter estimation and goodness of fit measures for fat tail distributions Hard limits on robust, efficient networks integrating comms, controls, energy, materials Essentials of layered architectures, naming and address, pub-sub, control, coding, latency, and implications for wireless Implications for control over networks Try to get you to read some papers you might otherwise not *really simple so we can move fast

Systems Fundamentals! A rant
A series of “obvious” observations (hopefully)

Case studies Networking and clean slate architectures
wireless end systems info or content centric application layer integrate routing, control, scheduling, coding, caching control of cyber-physical Lots from cell biology glycolytic oscillations for hard limits bacterial layering for architecture Neuroscience PC, OS, VLSI, etc (IT components) Earthquakes Medical physiology Smartgrid, cyber-phys Physics: turbulence, stat mech (QM?) Wildfire ecology Lots of aerospace Fundamentals!

Fix? Existing design frameworks Sophisticated components
Poor integration Limited theoretical framework Fix? Meta-layers Physiology Organs Prediction Goals Actions errors Cortex Fast, Limited scope Slow, Broad scope Comms Disturbance Plant Remote Sensor Actuator Interface Control

Layered architectures
Meta-layers Physiology Organs Prediction Goals Actions errors Cortex Fast, Limited scope Slow, Broad scope Comms Disturbance Plant Remote Sensor Actuator Interface Control

Doyle and Csete, Proc Nat Acad Sci USA, online JULY 25 2011
This paper aims to bridge progress in neuroscience involving sophisticated quantitative analysis of behavior, including the use of robust control, with other relevant conceptual and theoretical frameworks from systems engineering, systems biology, and mathematics. Doyle and Csete, Proc Nat Acad Sci USA, online JULY

Requirements on systems and architectures
accessible accountable accurate adaptable administrable affordable auditable autonomy available credible process capable compatible composable configurable correctness customizable debugable degradable determinable demonstrable dependable deployable discoverable distributable durable effective efficient evolvable extensible failure transparent fault-tolerant fidelity flexible inspectable installable Integrity interchangeable interoperable learnable maintainable manageable mobile modifiable modular nomadic operable orthogonality portable precision predictable producible provable recoverable relevant reliable repeatable reproducible resilient responsive reusable robust safety scalable seamless self-sustainable serviceable supportable securable simplicity stable standards compliant survivable sustainable tailorable testable timely traceable ubiquitous understandable upgradable usable

Simplified, minimal requirements
accessible accountable accurate adaptable administrable affordable auditable autonomy available credible process capable compatible composable configurable correctness customizable debugable degradable determinable demonstrable dependable deployable discoverable distributable durable effective efficient evolvable extensible failure transparent fault-tolerant fidelity flexible inspectable installable Integrity interchangeable interoperable learnable maintainable manageable mobile modifiable modular nomadic operable orthogonality portable precision predictable producible provable recoverable relevant reliable repeatable reproducible resilient responsive reusable robust safety scalable seamless self-sustainable serviceable supportable securable simplicity stable standards compliant survivable sustainable tailorable testable timely traceable ubiquitous understandable upgradable usable

Requirements on systems and architectures
accessible accountable accurate adaptable administrable affordable auditable autonomy available credible process capable compatible composable configurable correctness customizable debugable degradable determinable demonstrable dependable deployable discoverable distributable durable effective efficient evolvable extensible failure transparent fault-tolerant fidelity flexible inspectable installable Integrity interchangeable interoperable learnable maintainable manageable mobile modifiable modular nomadic operable orthogonality portable precision predictable producible provable recoverable relevant reliable repeatable reproducible resilient responsive reusable robust safety scalable seamless self-sustainable serviceable supportable securable simplicity stable standards compliant survivable sustainable tailorable testable timely traceable ubiquitous understandable upgradable usable fragile robust wasteful efficient

Robust to uncertainty in environment and components
Efficient in use of real physical resources fragile Want robust efficiency robust efficient wasteful

complex simple 2.5d space of systems and architectures fragile robust
wasteful efficient

Want to understand the space of systems/architectures
Case studies? fragile Strategies? Hard limits on robust efficiency? Architectures? Want robust and efficient systems and architectures robust efficient wasteful

Deep, but fragmented, incoherent, incomplete
Control, OR Comms Kalman Pontryagin Shannon Bode Nash Theory? Deep, but fragmented, incoherent, incomplete Von Neumann Carnot Boltzmann Turing Godel Heisenberg Compute Einstein Physics

? fragile? wasteful? Control Comms Each theory  one dimension
Shannon Bode Each theory  one dimension Tradeoffs across dimensions Assume architectures a priori Progress is encouraging, but… fragile? slow? ? wasteful? Carnot Boltzmann Turing Godel Heisenberg Compute Physics Einstein

Technology? fragile At best we get one robust efficient wasteful

??? fragile Often neither robust efficient wasteful

??? ? ? Bad architectures? gap? Bad theory? fragile robust efficient
wasteful

Conservation “laws”? fragile Case studies Sharpen hard bounds
Hard limit Case studies wasteful

Control Comms Shannon Bode fragile? slow? wasteful?

- - Control Comms Familiar pictures e=d-u d e=d-u d Entropy
delay Disturbance - - u Plant u Capacity C Sense channel Sense/ Encode Decode Control Entropy Sensitivity Circa 1950?  Bode Shannon Assume “favorable” delays

- Shannon e=d-u d Capacity CS Decode Entropy
delay Disturbance - Capacity CS Sense channel Sense/ Encode Decode Entropy Assume “favorable” delays

-  Bode e=d-u d unstable pole p Sensitivity u Plant Control
Simplified and dropped constants, slight differences between cont and disc time

- - Control Comms Circa 1950? So, anything happened since? e=d-u d
delay Disturbance - - u Plant u Capacity C Sense channel Sense/ Encode Decode Control Circa 1950? So, anything happened since?

Diverse applications TCP IP MAC MAC MAC Switch Pt to Pt Pt to Pt Physical

Doyle and Csete, Proc Nat Acad Sci USA, online JULY 25 2011
This paper aims to bridge progress in neuroscience involving sophisticated quantitative analysis of behavior, including the use of robust control, with other relevant conceptual and theoretical frameworks from systems engineering, systems biology, and mathematics. Doyle and Csete, Proc Nat Acad Sci USA, online JULY

Proceedings of the IEEE, Jan 2007
Chang, Low, Calderbank, and Doyle

Too clever? Diverse applications TCP IP Diverse Physical

Deconstrained (Applications) TCP Constrained Networks IP Deconstrained (Hardware) “constraints that deconstrain” (Gerhart and Kirschner)

Original design challenge?
Deconstrained (Applications) Networked OS Trusted end systems Unreliable hardware TCP/ IP Constrained Facilitated wild evolution Created whole new ecosystem complete opposite Deconstrained (Hardware)

Essentials Deconstrained (Applications) Few global variables Don’t cross layers Control, share, virtualize, and manage resources Constrained OS Processing Memory I/O Deconstrained (Hardware)

Layered architectures Bacterial biosphere
Deconstrained (diverse) Environments Architecture = Constraints that Deconstrain Catabolism AA Ribosome RNA RNAp transl. Proteins xRNA transc. Precursors Nucl. DNA DNAp Repl. Gene ATP Enzymes Building Blocks Shared protocols Deconstrained (diverse) Genomes

almost Inside every cell ATP Macro-layers Precursors Catabolism
AA Enzymes Nucl. Macro-layers Building Blocks AA transl. Proteins ATP Ribosome RNA transc. xRNA Crosslayer autocatalysis RNAp DNA Repl. Gene DNAp

What makes the bacterial biosphere so adaptable?
Environment Deconstrained phenotype Action Core conserved constraints facilitate tradeoffs Catabolism AA Ribosome RNA RNAp transl. Proteins xRNA transc. Precursors Nucl. DNA DNAp Repl. Gene ATP Enzymes Building Blocks Layered architecture Active control of the genome (facilitated variation) Deconstrained genome

Deconstrained (Applications) Few global variables Don’t cross layers Direct access to physical memory? Control, share, virtualize, and manage resources Constrained OS Processing Memory I/O Deconstrained (Hardware)

Few global variables Don’t cross layers Bacterial biosphere
Deconstrained (diverse) Environments Few global variables Architecture = Constraints that Deconstrain Catabolism AA Ribosome RNA RNAp transl. Proteins xRNA transc. Precursors Nucl. DNA DNAp Repl. Gene ATP Enzymes Building Blocks Shared protocols Don’t cross layers Deconstrained (diverse) Genomes

Problems with leaky layering
Modularity benefits are lost Global variables? Poor portability of applications Insecurity of physical address space Fragile to application crashes No scalability of virtual/real addressing Limits optimization/control by duality?

Fragilities of layering/virtualization
Hijacking, parasitism, predation Universals are vulnerable Universals are valuable Breakdowns/failures/unintended/… not transparent Hyper-evolvable but with frozen core

Original design challenge?
Deconstrained (Applications) Networked OS Trusted end systems Unreliable hardware TCP/ IP Constrained Facilitated wild evolution Created whole new ecosystem complete opposite Deconstrained (Hardware)

Deconstrained (Applications) Few global variables? Don’t cross layers? TCP/ IP Control, share, virtualize, and manage resources Constrained I/O Comms Latency? Storage? Processing? Deconstrained (Hardware)

IP addresses interfaces (not nodes)
IPC App App DNS Global and direct access to physical address! Robust? Secure Scalable Verifiable Evolvable Maintainable Designable … IP addresses interfaces (not nodes) Dev2 Dev2 Dev CPU/Mem Dev2 CPU/Mem CPU/Mem

Naming and addressing need to be resolved within layer
translated between layers not exposed outside of layer Physical IP TCP Application Related “issues” VPNs NATS Firewalls Multihoming Mobility Routing table size Overlays …

? Next layered architectures Deconstrained (Applications)
Few global variables Don’t cross layers ? Control, share, virtualize, and manage resources Constrained Comms Memory, storage Latency Processing Cyber-physical Deconstrained (Hardware)

Persistent errors and confusion (“network science”)
Architecture is least graph topology. Every layer has different diverse graphs. Application Architecture facilitates arbitrary graphs. TCP IP Physical

- What happened to this picture? Source d Decode Source coding Code
Hard limits Achievability Decomposition/Layering Channel coding Decode Channel Code

Decoupled Hides details Virtualizes channel Decode Code Channel coding
Under certain assumptions Decoupled Hides details Virtualizes channel Decode Code Channel coding Physical layer Rcv Channel Xmit

- Source d Decomp Source coding Compress Decoupled Hides details
Virtualizes source Under certain assumptions

gap? Hard tradeoffs R Rate distortion (backwards) error
Architecture= separation + coding Hard tradeoffs data rate R

gap? delay? Hard tradeoffs R Rate distortion (backwards) error
Architecture= separation + coding Hard tradeoffs data rate R

Internet= Distributed OS
Source - Decode Channel Code d Compress Decomp Link layer Rcv Xmit Internet= Distributed OS Layered architecture Control theory Data compression

Control/optimization theory needed for
Source - d Compress Decomp Application layer Control/optimization theory needed for Routing Congestion control Scheduling Caching? Distributed control of cyber-physical Still incomplete, needs more integration, with OS, languages Info theory, particularly for wireless Decode Channel Code Physical layer Rcv Xmit

- - What next? Control Comms e=d-u d e=d-u d Entropy Sensitivity
delay Disturbance - - u Plant u Capacity C Sense channel Sense/ Encode Decode Control Entropy Sensitivity Circa 1950? What next?

- costs  Bode e=d-u d  unstable pole p u Plant Control causality
stabilize benefits costs unstable pole p

- costs  Bode e=d-u d  unstable pole p u Plant Control causality
benefits stabilize costs unstable pole p

- costs  Bode e=d-u d unstable pole p u Plant Control causality
benefits stabilize costs unstable pole p

- costs e=d-u d Control Channel Plant Control remote control benefits
Martins and Dahleh, IEEE TAC, 2008 - e=d-u d Control Channel Plant Control remote control benefits stabilize max feedback costs

- Shannon e=d-u d Capacity CS Entropy Assume “favorable” delays delay
Disturbance - Capacity CS Sense channel Sense/ Encode Decode Entropy Assume “favorable” delays

- CS -CS Shannon e=d-u d delay Disturbance Sense/ Decode Encode
benefits -CS

- CS -CS costs delay Disturbance e=d-u d Plant Control Sense/ Encode
remote control benefits stabilize -CS Martins, Dahleh, Doyle IEEE TAC, 2007 causality costs

-CS -CS costs delay Disturbance d delay Benefits? ? ? benefits
delay Benefits? ? ? -CS benefits stabilize -CS causality costs

Physical implementation?
delay Disturbance - e=d-u d Physical implementation? Plant Sense/ Encode Control CS Abstract models of resource use Foundations, origins of noise dissipation amplification catalysis

Chandra, Buzi, and Doyle

CSTR, yeast extracts Experiments
K Nielsen, PG Sorensen, F Hynne, H-G Busse. Sustained oscillations in glycolysis: an experimental and theoretical study of chaotic and complex periodic behavior and of quenching of simple oscillations. Biophys Chem 72:49-62 (1998).

“Standard” Simulation
20 40 60 80 100 120 140 160 180 200 1 2 3 4 v=0.03 0.5 1.5 v=0.1 0.2 0.4 0.6 0.8 v=0.2 Figure S4. Simulation of two state model (S7.1) qualitatively recapitulates experimental observation from CSTR studies [5] and [12]. As the flow of material in/out of the system is increased, the system enters a limit cycle and then stabilizes again. For this simulation, we take q=a=Vm=1, k=0.2, g=1, u=0.01, h=2.5.

Why? Experiments Simulation
20 40 60 80 100 120 140 160 180 200 1 2 3 4 v=0.03 0.5 1.5 v=0.1 0.2 0.4 0.6 0.8 v=0.2 Experiments Simulation Why? Figure S4. Simulation of two state model (S7.1) qualitatively recapitulates experimental observation from CSTR studies [5] and [12]. As the flow of material in/out of the system is increased, the system enters a limit cycle and then stabilizes again. For this simulation, we take q=a=Vm=1, k=0.2, g=1, u=0.01, h=2.5.

Glycolytic “circuit” and oscillations
Most studied, persistent mystery in cell dynamics End of an old story (why oscillations) side effect of hard robustness/efficiency tradeoffs no purpose per se just needed a theorem Beginning of a new one robustness/efficiency tradeoffs complexity and architecture need more theorems and applications

x? y? autocatalytic? a? fragile? h? g? Robust
PFK? fragile? Tradeoffs? Hard limit? h? x? control? y? g? Robust =maintain energy charge w/fluctuating cell demand PK? Rest? rate k? robust? efficient? wasteful? Efficient=minimize metabolic overhead

z and p functions of enzyme complexity and amount
Theorem! z and p functions of enzyme complexity and amount Fragility simple enzyme complex enzyme Enzyme amount Savageaumics

almost Inside every cell ATP Macro-layers Precursors Catabolism
AA Enzymes Nucl. Macro-layers Building Blocks AA transl. Proteins ATP Ribosome RNA transc. xRNA Crosslayer autocatalysis RNAp DNA Repl. Gene DNAp

Energy Protein biosyn ATP Macro-layers Precursors Catabolism Enzymes
AA Enzymes Nucl. Macro-layers Building Blocks AA transl. Proteins ATP Protein biosyn Ribosome RNA transc. xRNA Crosslayer autocatalysis RNAp DNA Repl. Gene DNAp

Energy ATP Precursors Catabolism

x Metabolic flux ATP ATP Reaction 1 (“PFK”) energy Rest of cell
Minimal model Metabolic flux Reaction 1 (“PFK”) energy Rest of cell intermediate metabolite ATP ATP x Reaction 2 (“PK”) metabolic overhead Efficient

enzymes catalyze reactions
Reaction 1 (“PFK”) Rest of cell enzymes enzymes Protein biosyn Reaction 2 (“PK”) enzymes metabolic overhead  enzyme amount Efficient

Fluorescence histogram (fluorescence vs
Fluorescence histogram (fluorescence vs. cell count) of GFP-tagged Glyceraldehyde-3-phosphate dehydrogenase (TDH3). Cells grown in ethanol has lower mean and median of fluorescence, and also higher variability.

1 2 3 10 10 10 Metabolic Overhead

Fragility highly variable g=0 is implausibly fragile g=0 g=1
10 g=0 1 Fragility g=1 g=0 is implausibly fragile -1 10 -1 1 10 10 10 Metabolic Overhead

autocatalytic feedback: energy
Reaction 1 (“PFK”) energy Rest of cell ATP enzymes Protein biosyn Reaction 2 (“PK”) enzymes enzymes metabolic overhead  enzyme amount Efficient

x Metabolic flux ATP ATP energy Reaction 1 (“PFK”) Rest of cell
Inherently unstable ATP ATP x Reaction 2 (“PK”) metabolic overhead  enzyme amount Efficient

x control feedback control ATP ATP Robust = Maintain energy
disturbance Reaction 1 (“PFK”) Fragile h energy control ATP ATP x Rest of cell g Reaction 2 (“PK”) Robust Robust = Maintain energy despite demand fluctuation

x control ATP ATP disturbance Reaction 1 (“PFK”) Fragile h energy
Rest of cell g Reaction 2 (“PK”) Robust metabolic overhead  enzyme amount Efficient

Theorem! Fragile Robust metabolic overhead Efficient  enzyme amount x
ATP Rest of cell energy x h g control Reaction 2 (“PK”) Reaction 1 (“PFK”) disturbance Fragile Robust metabolic overhead  enzyme amount Efficient

Theorem! Fragile simple enzyme complex enzyme Robust
ATP Rest of cell energy x h g control Reaction 2 (“PK”) Reaction 1 (“PFK”) disturbance Fragile simple enzyme complex enzyme Robust metabolic overhead  enzyme amount Efficient

Fragile complex enzyme Robust metabolic overhead Efficient
ATP Rest of cell energy x h g control Reaction 2 (“PK”) Reaction 1 (“PFK”) disturbance Fragile complex enzyme Robust metabolic overhead  enzyme amount Efficient

y = WS  WS h H [P W]  y=ATP “weighed sensitivity” x ATP ATP control
disturbance Reaction 1 (“PFK”) h energy H [P W]  x ATP ATP Rest of cell g Reaction 2 (“PK”) y=ATP control

Architecture Good architectures allow for effective tradeoffs fragile
Alternative systems with shared architecture “Conservation laws” wasteful

Theorem z and p are functions of enzyme complexity and amount
standard biochemistry models phenomenological first principles?

? Fragility hard limits General Rigorous First principle simple
complex ? Overhead, waste Domain specific Ad hoc Phenomenological Plugging in domain details

? Control Comms Fundamental multiscale physics Foundations, origins of
Wiener Comms Bode Shannon robust control Kalman Fundamental multiscale physics Foundations, origins of noise dissipation amplification catalysis General Rigorous First principle ? Carnot Boltzmann Heisenberg Physics

“New sciences” of complexity and networks
Alderson &Doyle, Contrasting Views of Complexity and Their Implications for Network-Centric Infrastructure, IEEE TRANS ON SMC, JULY 2010 Control Comms Complex networks Compute doesn’t work “New sciences” of complexity and networks edge of chaos, self-organized criticality, scale-free,… Stat physics Carnot Boltzmann Heisenberg Physics

Control Comms Complex networks Compute Stat physics Physics
Alderson &Doyle, Contrasting Views of Complexity and Their Implications for Network-Centric Infrastructure, IEEE TRANS ON SMC, JULY 2010 Control Comms Complex networks Compute doesn’t work Sandberg, Delvenne, & Doyle, On Lossless Approximations, the Fluctuation-Dissipation Theorem, and Limitations of Measurement, IEEE TRANS ON AC, FEBRUARY, 2011 Stat physics Carnot Boltzmann Heisenberg Physics

Control Comms Complex networks Compute “orthophysics” Stat physics,
Sandberg, Delvenne, & Doyle, On Lossless Approximations, the Fluctuation-Dissipation Theorem, and Limitations of Measurement, IEEE TRANS ON AC, FEBRUARY, 2011 Stat physics, fluids, QM Carnot Boltzmann Heisenberg Physics

Turbulence and drag? J. Fluid Mech (2010) Flow Streamlined
Laminar Flow Transition to Turbulence Increasing Drag, Fuel/Energy Use and Cost Turbulent Flow

Coherent structures and turbulent drag
Physics of Fluids (2011) y y Flow z x z x Coherent structures and turbulent drag high-speed region 3D coupling upflow Blunted turbulent velocity profile low speed streak downflow Turbulent Laminar

? Laminar Control? Turbulent Turbulent Laminar fragile robust
efficient wasteful

Foundations, origins of noise dissipation amplification catalysis
Control Wiener Comms Bode Shannon robust control Kalman Foundations, origins of noise dissipation amplification catalysis General Rigorous First principle Carnot Boltzmann Heisenberg Physics

Smart Antennas (Javad Lavaei)
Security, co-channel interference, power consumption Conventional Antenna Smart Antenna Smart Antennas 1) Multiple active elements: Easy to program Hard to implement 2) Multiple passive elements: Easy to implement Hard to program

Passively Controllable Smart (PCS) Antenna
PCS Antenna: One active element, reflectors and several parasitic elements This type of antenna is easy to program and easy to implement but “hard” to solve (e.g. 4 weeks offline computation). We solved the problem in 1 sec with huge improvement:

Decentralized control Partial bibliography (Lamperski)
Triaged today Great topic

In press, available online
Really fat tails How big is “big”?

 “characteristic earthquake”???
Note: rare example (in science) involving power laws that isn’t obviously ridiculous smaller Persistent controversy Gutenberg-Richter log(rank)  “characteristic earthquake”??? slope = -1 ? “bump” largest magnitude  log(power)

Randomly sampled G-R magnitudes
Distribution of max event No bump still might look like a “bump”

Tail stays highly variable
Order statistics P( (k of n) > x) p=- dP/dx More data, but… Tail stays highly variable More samples

Fault traces and epicenters

155 faults Synthetic GR Mag of largest quake
Number of earthquakes per fault 155 faults Synthetic GR Number of earthquakes per fault Mag of largest quake

3 biggest quakes Mag of largest quake

3 biggest quakes p=.03 p=.01 p=.002

Magnitude binned by distance to faults
3.6

Randomly sampled G-R magnitudes
Distribution of max event No bump still might look like a “bump”

Los Padres National Forest
Truncated pareto model •P(>x) 3 10 Wildfires 2 10 1 10 LPNF data (decimated) 10 -1 1 2 3 10 10 10 10 10

Comparison of model n•P(>x) with cumulative raw data (decimated).
3 3 10 10 10 2 10 2 1 1 10 LPNF data (decimated) 10 10 10 10 -1 10 10 1 10 2 10 3 10 -2 10 -1 10 10 1 10 2

LPNF data (black) plus 4 pseudo-random samples from P(>x) (colors).
Comparison of variations in LPNF data versus that of pseudo-random samples. LPNF data (black) plus 4 pseudo-random samples from P(>x) (colors). LPNF data and pseudo-random samples have similar variations. 3 3 10 10 10 2 10 2 10 1 10 1 10 10 10 -1 10 10 1 10 2 10 3 10 -2 10 -1 10 10 1 10 2

MLE as WLS Exponential Pareto

MLE as WLS Exponential

Pareto Distribution α=1 n=100 K-S p-value: 0.34

Pareto Distribution MATLAB’s ksstest function with the null hypothesis Pareto alpha=1 α=1 n=100 p=0.34 2 4 6 8 10 10 10 10 10 x

α=1 n=100 p=0.34!!! makes no difference
MATLAB’s ksstest function with the null hypothesis Pareto alpha=1 10 2 Test Distribution Null Distribution α=1 n=100 p=0.34!!! Rank 1 10 makes no difference need weighted KS, but what weight? 10 2 4 6 8 10 10 10 10 10 x

Order statistics P( (k of n) > x) p=- dP/dx
But this is so easy and is (apparently) advocated by many statisticians. More samples

Fat tails, hard limits, thin layers John Doyle, Caltech

Similar presentations

Presentation on theme: "Fat tails, hard limits, thin layers John Doyle, Caltech"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Fat tails, hard limits, thin layers John Doyle, Caltech

Similar presentations

Presentation on theme: "Fat tails, hard limits, thin layers John Doyle, Caltech"— Presentation transcript:

Similar presentations

About project

Feedback