1. The University of Chicago Graduate School of Business
Dynamic Portfolio Optimization using Decomposition and Finite Element Methods
John R. Birge
Quantstar and
The University of Chicago Graduate School of Business
Quantstar
QuantDay2007-New York

2. Theme Models for Dynamic Portfolio Optimization are:
big (exponential growth in time and state)
general (can model many situations)
structured (useful properties somewhere)
Some hope for solution by:
modeling the “right” way
using structure wisely
approximating (with some guarantees/bounds)
Quantstar
QuantDay2007-New York

3. Outline General Model – Observations
Dynamic Model Construction and Motivation
Overview of approaches
Decomposition
Lagrangian and ADP methods
Finite-Element Approach
Conclusions
Quantstar
QuantDay2007-New York

4. Why Model Dynamically? Three potential reasons: Example Market timing
Reduce transaction costs (taxes) over time
Maximize wealth-dependent objectives
Example
Suppose major goal is $100MM to pay pension liability in 2 years
Start with $82MM; Invest in stock (annual vol=18.75%, annual exp. Return=7.75%); bond (Treasury, annual vol=0; return=3%)
Can we meet liability (without corporate contribution)?
How likely is a surplus?
Quantstar
QuantDay2007-New York

5. Alternatives Markowitz (mean-variance) – Fixed Mix
Pick a portfolio on the efficient frontier
Maintain the ratio of stock to bonds to minimize expected shortfall
Buy-and-hold (Minimize expected loss)
Invest in stock and bonds and hold for 2 years
Dynamic (stochastic program)
Allow trading before 2 years that might change the mix of stock and bonds
Quantstar
QuantDay2007-New York

6. Efficient Frontier Some mix of risk-less and risky asset
For 2-year returns:
Quantstar
QuantDay2007-New York

7. Best Fixed Mix and Buy-and-Hold
Fixed Mix: 27% in stock
Meet the liability 25% of time (with binomial model)
Buy-and-Hold: 25% in stock
Meet the liability 25% of time
Quantstar
QuantDay2007-New York

8. Best Dynamic Strategy Start with 57% in stock
If stocks go up in 1 year, shift to 0% in bond
If stocks go down in 1 year, shift to 91% in stock
Meet the liability 75% of time
Stocks Up
Stocks Down
Quantstar
QuantDay2007-New York

9. Advantages of Dynamic Mix
Able to lock in gains
Take on more risk when necessary to meet targets
Respond to individual utility that depends on level of wealth
Target
Shortfall
Quantstar
QuantDay2007-New York

10. Approaches for Dynamic Portfolios
Static extensions
Can re-solve (but hard to maintain consistent objective)
Solutions can vary greatly
Transaction costs difficult to include
Dynamic programming policies
Approximation
Restricted policies (optimal – feasible?)
Portfolio replication (duration match)
General methods (stochastic programs)
Can include wide variety
Computational (and modeling) challenges
Quantstar
QuantDay2007-New York

11. Dynamic Programming Approach
State: xt corresponding to positions in each asset (and possibly price, economic, other factors)
Value function: Vt (xt)
Actions: ut
Possible events st, probability pst
Find:
Vt (xt) = max –ct ut + Σst pstVt+1 (xt+1(xt,ut,st))
Advantages: general, dynamic, can limit types of policies
Disadvantages: Dimensionality, approximation of V at some point needed, limited policy set may be needed, accuracy hard to judge
Consistency questions: Policies optimal? Policies feasible? Consistent future value?
Quantstar
QuantDay2007-New York

12. Other Restricted Policy Approaches
Kusy-Ziemba ALM model for Vancouver Credit Union
Idea: assume an expected liability mix with variation around it; minimize penalty to meet the variation
Formulation:
min Σi ci xi + Σst pst(qst+ yst+ + qst- yst-)
s.t. Σi fits xi + yst+ - yst- = lts all t and s; xi y >= 0, i = 1…n
Problems: Similar to liability matching.
Consistency questions: Possible to purchase insurance at cost of penalties? Best possible policy?
Quantstar
QuantDay2007-New York

13. General Methods Basic Framework: Stochastic Programming
Model Formulation:
Advantages:
General model, can handle transaction costs, include tax lots, etc.
Disadvantages: Size of model, insight
Consistency questions: Price dynamics appropriate? objective appropriate? Solution method consistent?
max  p(U(W(  , T) )
s.t. (for all ): k x(k,1, ) = W(o) (initial)
k r(k,t-1, ) x(k,t-1, ) - k x(k,t, ) = 0 , all t >1;
k r(k,T-1, ) x(k,T-1, ) - W(  , T) = 0, (final);
x(k,t, ) >= 0, all k,t;
Nonanticipativity:
x(k,t, ’) - x(k,t, ) = 0 if ’, Sti for all t, i, ’, 
This says decision cannot depend on future.
Quantstar
QuantDay2007-New York

14. Model Consistency Price dynamics may have inherent arbitrage
Example: model includes option in formulation that is not the present value of future values in model (in risk-neutral prob.)
Does not include all market securities available
Policy inconsistency
May not have inherent arbitrage but inclusion of market instrument may create arbitrage opportunity
Skews results to follow policy constraints
Lack of extreme cases
Limited set of policies may avoid extreme cases that drive solutions
Quantstar
QuantDay2007-New York

15. Objective Consistency
Examples with non-coherent objectives
Value-at-Risk
Probability of beating benchmark
Coherent measures of risk
Can lead to piecewise linear utility function forms
Expected shortfall, downside risk, or conditional value-at-risk (Uryasiev and Rockafellar)
Quantstar
QuantDay2007-New York

16. Model and Method Difficulties
Model Difficulties
Arbitrage in tree
Loss of extreme cases
Inconsistent utilities
Method Difficulties
Deterministic incapable on large problems
Stochastic methods have bias difficulties
Particularly for decomposition methods
Discrete time approximations
Stopping rules and time hard to judge
Quantstar
QuantDay2007-New York

17. Resolving Inconsistencies
Objective: Coherent measures (& good estimation)
Model resolutions
Construction of no-arbitrage trees (e.g., Klaassen)
Extreme cases (Generalized moment problems and fitting with existing price observations)
Method resolutions
Use structure for consistent bound estimates
Decompose for efficient solution
Quantstar
QuantDay2007-New York

18. General Form in Discrete Time
Find x=(x1,x2,…,xT) and p (allows for “robust formulation”) to
minimize Ep [ t=1Tft(xt,xt+1,p) ]
s.t xt 2 Xt, xt nonanticipative, p2 P (distribution class)
P[ ht (xt,xt+1,pt,) <= 0 ] >= a (chance constraint)
General Approaches:
Simplify distribution (e.g., sample) and form a mathematical program:
Solve step-by-step (dynamic program)
Solve as single large-scale optimization problem
Use iterative procedure of sampling and optimization steps
Quantstar
QuantDay2007-New York

19. What about Continuous Time?
Sometimes very useful to develop overall structure of value function
May help to identify a policy that can be explored in discrete time (e.g., portfolio no-trade region)
Analysis can become complex for multiple state variables
Possible bounding results for discrete approximations (e.g., FEM approach)
Quantstar
QuantDay2007-New York

20. Simplified Finite Sample Model
Assume p is fixed and random variables represented by sample xit for t=1,2,..,T, i=1,…,Nt with probabilities pit ,a(i) an ancestor of i, then model becomes (no chance constraints):
minimize St=1T Si=1Nt pit ft(xa(i)t,xit+1, xit)
s.t xit Î Xit
Observations?
Problems for different i are similar – solving one may help to solve others
Problems may decompose across i and across t yielding
smaller problems (that may scale linearly in size)
opportunities for parallel computation.
Quantstar
QuantDay2007-New York

21. Outline General Model – Observations
Dynamic Model Construction and Motivation
Overview of approaches
Decomposition
Lagrangian and ADP methods
Finite Element Methods
Conclusions .
Quantstar
QuantDay2007-New York

22. Solving As Large-scale Mathematical Program
Principles:
Discretization leads to mathematical program but large-scale
Use standard methods but exploit structure
Direct methods
Take advantage of sparsity structure
Some efficiencies
Use similar subproblem structure
Greater efficiency
Size
Unlimited (infinite numbers of variables)
Still solvable (caution on claims)
Quantstar
QuantDay2007-New York

23. Standard Approaches Sparsity structure advantage
Partitioning
Basis factorization
Interior point factorization
Similar/small problem advantage
DP approaches
Decomposition:
Benders, l-shaped (Van Slyke – Wets)
Dantzig-Wolfe (primal version)
Regularized (Ruszczynski)
Various sampling schemes (Higle/Sen stochastic decomposition, abridged nested decomposition)
Approximate DP (Bertsekas, Tsitsiklis, Van Roy..)
Lagrangian methods
Quantstar
QuantDay2007-New York

24. Outline General Model – Observations Overview Decomposition
Lagrangian and ADP methods
Finite Element Methods
Conclusions
Quantstar
QuantDay2007-New York

25. Similar/Small Problem Structure: Dynamic Programming View
Stages: t=1,...,T
States: xt -> Btxt (or other transformation)
Value function:
Vt(xt) = E[Vt(xt,xt)] where
xt is the random element and
Vt(xt,xt) = min ft(xt,xt+1,xt) +Vt+1(xt+1)
s.t. xt+1 Î Xt+1t(,xt) xt given
Solve : iterate from T to 1
Quantstar
QuantDay2007-New York

26. Linear Model Structure
VN+1(xN) = 0, for all xN,
-Vt,k(xt-1,a(k)) is a piecewise linear, convex function of xt-1,a(k)
Quantstar
QuantDay2007-New York 3

27. Decomposition Methods
Benders idea
Form an outer linearization of -Vt
Add cuts on function :
Feasible region
(feasibility cuts)
-Vt
new cut
(optimality cut)
min at k : < -Vt
LINEARIZATION AT ITERATION k
Quantstar
QuantDay2007-New York

28. Nested Decomposition
In each subproblem, replace expected recourse function Vt,k(xt-1,a(k)) with unrestricted variable t,k
Forward Pass:
Starting at the root node and proceeding forward through the scenario tree, solve each node subproblem
Add feasibility cuts as infeasibilities arise
Backward Pass
Starting in top node of Stage t = N-1, use optimal dual values in descendant Stage t+1 nodes to construct new optimality cut. Repeat for all nodes in Stage t, resolve all Stage t nodes, then t t-1.
Convergence achieved when
Quantstar
QuantDay2007-New York 6

29. Sample Results Example performance LOG (CPUS) 4 Standard LP
NESTED DECOMP. 3 2 1
LOG (NO. OF VARIABLES)
PARALLEL: 60-80% EFFICIENCY IN SPEEDUP
Other problems: similar results
Quantstar
QuantDay2007-New York

30. Decomposition Enhancements
Optimal basis repetition
Take advantage of having solved one problem to solve others
Use bunching to solve multiple problems from root basis
Share bases across levels of the scenario tree
Use solution of single scenario as hot start
Multicuts
Create cuts for each descendant scenario
Regularization
Add quadratic term to keep close to previous solution
Sampling
Stochastic decomposition (Higle/Sen)
Importance sampling (Infanger/Dantzig/Glynn)
Multistage (Pereira/Pinto, Abridged ND)
Quantstar
QuantDay2007-New York

31. Abridged Nested Decomposition
Donohue/JRB 2006
Incorporates sampling into the general framework of Nested Decomposition
Assumes relatively complete recourse and serial independence
Samples both the sub-problems to solve and the solutions to continue from in the forward pass through sample-path tree
Quantstar
QuantDay2007-New York 10

32. Outline General Model – Observations Overview of approaches
Decomposition
Lagrangian and ADP methods
Finite Element Methods
Conclusions
Quantstar
QuantDay2007-New York

33. Lagrangian-based Approaches
General idea:
Relax nonanticipativity (or perhaps other constraints)
Place in objective
Separable problems
MIN E [ St=1T ft(xt,xt+1) ]
xt Î Xt
+ E[w,x] + r/2||x-x||2
MIN E [ St=1T ft(xt,xt+1) ]
s.t xt Î Xt
xt nonanticipative
Update: wt; Project: x into N - nonanticipative space as x
Convergence: Convex problems - Progressive Hedging Alg.
(Rockafellar and Wets)
Advantage: Maintain problem structure (e.g., network)
Quantstar
QuantDay2007-New York

34. Approximate Dynamic Programming: Infinite Horizon
Use LP solution of dynamic (Bellman) equation:
max (d,V) s.t. TV ¸ V for distribution d on x
Approximate V with finite set of basis functions j, weights j
LP for finite set becomes: Find  to
max (d,) s.t. T¸ 
Quantstar
QuantDay2007-New York

35. Solving ADP Form Bounds available (Van Roy, De Farias)
Discretizations:
Discrete state space x
Use structure to reduce constraint set
Use Duality:
Dual Form:
min max (d,) + (,T-)
Can combine with outer approximation
Quantstar
QuantDay2007-New York

36. Outline General Model – Observations Overview of approaches
Decomposition
Lagrangian and ADP methods
Finite element methods
Conclusions
Quantstar
QuantDay2007-New York

37. Continuous-Time Setup
Suppose Vt(xt)=maxx2 X E[stT fu(xu|xt) du]
Questions:
Can the form of Vt provide insight into the effects of time discretization?
When does Vt have useful structural properties?
Can different methods of discretization provide better results than others?
Quantstar
QuantDay2007-New York

38. Portfolio in Continuous Time
Setup:
Value function, u
PDE with no trade, ut + Gu – ru =0, u(T,x) given
Define Mu(t,x)=supx’|x’2 Y(t,x) u(t,x’)
where Y(t,x) is the set of attainable portfolios from x at t
Quantstar
QuantDay2007-New York

39. Variational Inequality Form
General variational inequality form
ut+Gu-ru· 0, u-Mu¸ 0, (ut+Gu-ru)(u-Mu)=0
Computational approach:
Apply high-order FEM methods for the continuous regime
Use a sequential optimization process to determine the free boundary (which then effectively determines the no-trade region)
Quantstar
QuantDay2007-New York

40. FEM Approach Local Discontinuous Galerkin Method
Decomposes by time
Parallel implementation
Can achieve high order of accuracy
Use Legendre polynomials as basis functions that then approximate the value function
General result (Liu/JRB): ||u – uh||· C hk+1/2
for kth order polynomials
Quantstar
QuantDay2007-New York

41. Extensions
Increase the complexity of portfolio examples in higher dimensions
Extend approach to other models governed by smooth dynamics plus non-smooth impulse-type controls
Provide generalizations for other forms of stochastic programs
Quantstar
QuantDay2007-New York

42. Conclusions
Use of structure in solving large-scale dynamic portfolio problems:
Repeated problems
Nonzero pattern for sparsity
Use of decomposition and sampling ideas
Potential for high-accuracy methods with FEM
Computational results
Structure accelerates solution (and allows additional complexity: asset types/transaction costs/etc)
Speedups possible in orders of magnitude over standard software implementations
Quantstar
QuantDay2007-New York