1. Dynamics of Learning & Distributed Adaptation PI: James P. Crutchfield, Santa Fe Institute Agent-Based Computing Grantee Meeting 3-5 October 2000 Dynamics of Learning: Single-agent learning theory Emergence of Distributed Adaptation: Agent-collective learning theory Related Projects: – Computational Mechanics www.santafe.edu/projects/CompMech – Evolutionary Dynamics www.santafe.edu/~evca – Network Dynamics Program discuss.santafe.edu/dynamics – Coyote: Multiprocessor for Agent-based Computing Dynamics of Learning: Single-agent learning theory Emergence of Distributed Adaptation: Agent-collective learning theory Related Projects: – Computational Mechanics www.santafe.edu/projects/CompMech – Evolutionary Dynamics www.santafe.edu/~evca – Network Dynamics Program discuss.santafe.edu/dynamics – Coyote: Multiprocessor for Agent-based Computing

2. Computational Mechanics: The Learning Channel TLC: Adaptation of Communication Channel What are fundamental constraints on learning? – How to measure environmental structure? – How to measure “cognitive” capacity of learning agents? – How much data for a given complexity of inferred model? TLC: Adaptation of Communication Channel What are fundamental constraints on learning? – How to measure environmental structure? – How to measure “cognitive” capacity of learning agents? – How much data for a given complexity of inferred model?

3. Computational Mechanics: Preliminaries Observations: s  = s  s  Past  Future: … s -L s -L+1 …s -1 s 0 |s 1 …s L-1 s L … Probabilities: Pr(s  ), Pr(s  ), Pr(s  ) Uncertainty: Entropy H[P] = -  i p i log p i [bits] Prediction error: Entropy Rate h  = H[Pr(s i |s i-1 s i-2 s i-3 …)] Information transmitted to future: Excess Entropy E = H[Pr(s  )/ (Pr(s  )Pr(s  ))] Measure of independence: Is Pr(s  )=Pr(s  )Pr(s  )? Describes information in “raw” sequence blocks Observations: s  = s  s  Past  Future: … s -L s -L+1 …s -1 s 0 |s 1 …s L-1 s L … Probabilities: Pr(s  ), Pr(s  ), Pr(s  ) Uncertainty: Entropy H[P] = -  i p i log p i [bits] Prediction error: Entropy Rate h  = H[Pr(s i |s i-1 s i-2 s i-3 …)] Information transmitted to future: Excess Entropy E = H[Pr(s  )/ (Pr(s  )Pr(s  ))] Measure of independence: Is Pr(s  )=Pr(s  )Pr(s  )? Describes information in “raw” sequence blocks

4. Computational Mechanics: Mathematical Foundations Casual state = Condition of knowledge about future  -Machines = {Causal states, Transitions} Optimality Theorem:  -Machines are optimal predictors of environment. Minimality Theorem: Of the optimal predictors,  -Machines are smallest. Uniqueness Theorem: Up to isomorphism, an  -Machine is unique. The Point: Discovering an  -Machine is the goal for any learning process. Practicalities may preclude this, but this is the goal. (w/ DP Feldman/CR Shalizi) Casual state = Condition of knowledge about future  -Machines = {Causal states, Transitions} Optimality Theorem:  -Machines are optimal predictors of environment. Minimality Theorem: Of the optimal predictors,  -Machines are smallest. Uniqueness Theorem: Up to isomorphism, an  -Machine is unique. The Point: Discovering an  -Machine is the goal for any learning process. Practicalities may preclude this, but this is the goal. (w/ DP Feldman/CR Shalizi)

5. Computational Mechanics: Why Model? Structural Complexity of Information Source C  = H[Pr( S )], S = {Casual states} Uses: – Environ’l complexity: Amount/kind of relevant structure – Agent’s inferential capacity: Sophistication of models? Theorem: E  C  Conclusion: Build models vs. storing only E bits of history. – Raw sequence blocks do not allow optimal prediction, only E bits of mutual information in blocks. – Optimal prediction requires larger model: 2 C , not 2 E. – Explicit: 1D Range-R Ising spin system: C  = E +R h . Structural Complexity of Information Source C  = H[Pr( S )], S = {Casual states} Uses: – Environ’l complexity: Amount/kind of relevant structure – Agent’s inferential capacity: Sophistication of models? Theorem: E  C  Conclusion: Build models vs. storing only E bits of history. – Raw sequence blocks do not allow optimal prediction, only E bits of mutual information in blocks. – Optimal prediction requires larger model: 2 C , not 2 E. – Explicit: 1D Range-R Ising spin system: C  = E +R h .

6. Dynamics of Learning: The Aha! Effect Learning complex environments (w/ C Douglas) Learning paradigm Three phases – Memorization – Aha! – Refinement Learning complex environments (w/ C Douglas) Learning paradigm Three phases – Memorization – Aha! – Refinement

7. Dynamics of Learning: Hierarchical Modeling Computation at the Onset of Chaos Onset of chaos leads to infinite  -machine Learn the higher level representation Go from series of DFAs to Stack Automaton

8. Dynamics of Learning: Some Open Questions Learning agents – Dynamical systems view of learning as a process whose behavior is predictive model building – Define and measure agent “cognitive” abilities – Development math’lly analyzable and simulatable models – What state-space structures are responsible for learning? E.g., Basins = robust memories; bifurcations = adaptation; models = attractor-basin portrait in subspace; … Robot collectives – Group versus individual function – Define and measure degree of cooperation – Agent collective functioning versus communication topologies Learning agents – Dynamical systems view of learning as a process whose behavior is predictive model building – Define and measure agent “cognitive” abilities – Development math’lly analyzable and simulatable models – What state-space structures are responsible for learning? E.g., Basins = robust memories; bifurcations = adaptation; models = attractor-basin portrait in subspace; … Robot collectives – Group versus individual function – Define and measure degree of cooperation – Agent collective functioning versus communication topologies

9. Evolutionary Dynamics Research – Mathematical Analysis Epochal evolution Fitness barrier crossing: neutral paths v. fitness valleys? Optimal evolutionary search w/ E van Nimwegen (Dissn@SFI, Fall ‘99) – Evolving Cellular Automata Population dynamics Embedded particle computation in CAs w/ W Hordijk (Dissn@SFI, Fall ‘99), M Mitchell, L Pagie, C Shalizi Research – Mathematical Analysis Epochal evolution Fitness barrier crossing: neutral paths v. fitness valleys? Optimal evolutionary search w/ E van Nimwegen (Dissn@SFI, Fall ‘99) – Evolving Cellular Automata Population dynamics Embedded particle computation in CAs w/ W Hordijk (Dissn@SFI, Fall ‘99), M Mitchell, L Pagie, C Shalizi

10. Structure and Dynamics in Complex Interactive Networks Research Areas – Network Structure – Network Dynamics – Hierarchical and Heterarchical Networks – Components: Annual Workshop SFI-Intel Post-Doctoral Fellow Visitor Program Multiprocessor JPC-DW Individual Research Intel (BusNet) 3 years (JPC & DW) Research Areas – Network Structure – Network Dynamics – Hierarchical and Heterarchical Networks – Components: Annual Workshop SFI-Intel Post-Doctoral Fellow Visitor Program Multiprocessor JPC-DW Individual Research Intel (BusNet) 3 years (JPC & DW)

11. Coyote: SFI’s Beowulf A Supercomputer for Complex Adaptive Systems Cheap Off-the-Shelf Technology (“Piles of PCs”) 64 Compute Nodes (128 CPUs), expandable Fast Network Interconnect (Cisco Gigabit switch) Physical: Gatehouse room (power/cooling retrofit) General Availability: Summer Y2K Team: –Lolly: Cluster Administration/Maintenance –JPC: Coordination, System Architecture –Tim: Node and Network Architecture –Alex: Parallel, Distributed Code Development