1. Ec1818 Economics of Discontinuous Change Section 2 [Lectures 5-7]
Wei Huang
Harvard University
(Preliminary and subject to revisions)
Previous TA section PPTs are used

2. Lec. 5. Landscape and Kauffman’s N-K Model
Some concepts:
Fitness landscape is a geometry mapping strategies to profitability.
A fitness landscape model is a representation of theory/view of how to boil a set of outcomes into a single metric and of the factors affecting it and how easy or hard it is to change those factors.
A landscape is not necessarily concave or single-peaked in the world.
Search algorithms explore landscapes.
Path is important; uncertainty; no best algorithm for all spaces on a rugged landscape

3. Lec. 5. Landscape and Kauffman’s N-K Model
There are N-dimensional vectors.
N = # factors that define your policy.
The profitability of each element depends on itself and other K elements.
K = # of other elements affecting the profitability of any policy/person; parameter that tunes the ruggedness of landscape.
When K=0, it is a concave landscape, single-peaked.
When K=1,...,N-2, it is a correlated landscape.
When K=N-1, it is a random landscape, possible to have many local maxima.
Propositions and Applications (In lec notes, understand them)

4. Lec 6. Genetic Algorithms
GA seeks to discover/create/evolve new strategies that produce better solutions by mimicking evolutionary processes.
A search algorithm for correlated but not single-peaked landscapes imitating the evolutionary process.
To do a GA,
Each strategy is defined by 0/1 for different attributes,
The profit is determined by the strategy.

5. Lec 6. How GA works? Prepare a present population with N strategies.
Compute the profit by each strategy in the present population.
Create strategies by the following methods
Copy: just copy one of the strategies.  
Cross-over: take two strategies, split them and cross over. (1001 and 0111 → 1011 and 0101)
Mutation: take one strategy and randomly change some of the attributes. (0101 → 0100)
Inversion: take one strategy and switch two attributes. (1001 → 1010)
Continue 3 until the next population is filled by N strategies.
Replace the present population with the next population.
Repeat 2-5 for G times and output a strategy with the highest profit as a solution.

6. Lec 6. Schemata and Fundamental Theorem of GA
A schema is a generalized strategy which includes ∗ (this can be 0 or 1).
Consider 3 attributes. Then 23 = 8 strategies and 33 = 27 schemata.
For example, ∗10 is a schema for 010 and 110.
1∗∗ is a schema for 100, 101, 110 and 111.
Average fitness for a schema tells us what combination of 0/1 does well in general.
Fundamental Theorem of GA: A GA using fitness-proportionate reproduction and cross-over and mutation produces a population where representatives of schema S grow exponentially proportionate to its fitness relative to the average fitness.

7. Lec 6. Merits and Problems with GA
Can be used in the optimization of non single-peaked and non-differentiable functions;
Easy to program, already many applications.
Problems:
It does not necessarily find the global max;
GA gets stuck at the local max if we start from a small group of good (but not the best!) strategies;
Need to modify the first population and the probability of mutation.
Royal Road Clunker.

8. Lec 6. Genetic Algorithms and Stimulated Annealing
Simulated annealing is an algorithm for going in the wrong direction to get off a local peak and find something better.
The probability of going in the wrong direction is P(s) = exp(−δ/t) where
δ is the cost (big δ ⇒ small P(s)), and
t is the temperature, willingness to go the wrong way (big t ⇒ P(s) close to 1).

9. Lec 7. Search Models and Stopping Rules
There are many reasons why it is impossible to search the entire landscape and find the best one.
Cost of Search; You cannot go back to the choice you rejected.
Analyze the following models:
Reservation wage model.
Optimal stopping problems.

10. Lec 7. Reservation Wage Model
Assumptions:
You can choose the best among what you searched, but there is a cost of search.
Marginal cost of an additional search is given.
Procedures:
Compute marginal benefit for each search and find the optimal number of searches n with which
MB for nth search ≥ MC for nth search, and
MB for n+1th search ≤ MC for n+1th search.
If MC is constant (or increasing) and MB is decreasing in n, there is a unique optimal number n∗.

11. Lec 7. Secretary Problem Assumptions: Question: Answer: Why?
No explicit search cost, but you cannot go back to choices you rejected.
You know the number of choices, but do not know the distribution of values from them.
Question:
What is the algorithm that maximizes the probability of choosing the one with the highest value?
Answer:
skipping the first 0.38 ∗ n choices and select the first one whose value is higher than the maximum of the values of the first 0.38 ∗ n choice (for large n).
Why?
The probability of winning by R discovery searches is 1/n [1 + R/(R+1) + R/(R+2) + … + R/(n-1)];
Max it with respective to R. R* = (n-1)/e

12. Lec. 7 Thomas Bruss’s Odds-Algorithm for Last-success Problem
Throw die 12 times and declare “this is the last 4.” Win if it is and lose if it’s not.
What is the best strategy for maximizing the probability of winning?
The odds algorithm is one of the solutions.