1. Simultaneous Graphical Dynamic Models

2. Simultaneous graphical dynamic models: SGDLMs
e.g. 500 stocks S&P, daily
Interest in:
Scale up: large
“Decoupled” univariate series/models
Series-specific independent predictors
Series-specific states, evolutions, discount factors, volatilities
Sequential analysis: Fast? Parallelisation?
Cross-series / multivariate structure : Sensitive co-volatility modelling
Critical: coherent joint forecast distributions
Computationally fast

3. Simultaneous models: Coupled set of univariate DLMs
Multiple univariate models
“decoupled”
in parallel
Independent
predictors
Simultaneous
“parents”
Simultaneous parents
*Current* values of (some) related series as predictors
*Independent* across j: residuals & univariate volatilities:

4. Implied coherent multivariate model
dynamic regressions
& precision/volatility
*Sparse* Cholesky or spectral-style MSV
Cross-talk in dynamic regressions
Sparse
Non-zeros in .
Notation: extend to row j and pad with 0 entries
General conclusions from this example

5. Dynamic graphical model structure induced
precision/volatility
Non-zeros in .
zero precision: conditional independence
Non-zeros in .
General conclusions from this example
Links to & from parents
Links to & from parents-of-parents

6. Side notes: Critical modelling interpretation by example
Simplest special example:
No time
Joint distribution has (complete) conditionals:
Simultaneous models
Simultaneous models are NOT complete conditionals
Just one specification of the joint
Joint:
General conclusions from this example

7. Parental-induced cross-talk in dynamic regressions
Non-zeros in .
Non-zeros in .
General conclusions from this example
parents
grandparents
…….
great-grandparents

8. Simultaneous models: Coupled set of univariate DLMs
Multiple univariate models
“decoupled”
in parallel
Parallel states - independent evolutions

9. Normal pdf in simultaneous model j
Critical result: Coupled joint model at each time
Critical result:
absolute value of determinant
Normal pdf in simultaneous model j
Set of parallel, univariate DLMs
Coherence: “Complicating factor”

10. Summary concepts: Simultaneous graphical DLMs
- Multiple univariate models: decoupled, in parallel
- Conducive to on-line sequential learning: Analytic, fast, parallel
- Simultaneous parental sets define *sparse* multivariate stochastic volatility matrix
- New dynamic graphical models for MSV: evolutions of
- NOT dependent on choice of order of named series j=1:m
- Recoupling for coherent posterior and predictive inferences?
: determinantal “complicating” factors :

11. Dynamic Dependency Network Models
Special cases:
Dynamic Dependency Network Models
- DDNMs -
? Dependent on order of series
? Scale-up in dimension
? 400 S&P stock price series

12. Simultaneous Graphical Dynamic Models
General class of
Simultaneous Graphical Dynamic Models
- SGDLMs -

13. Simultaneous parents of j
General class of SGDLMs
Simultaneous parents of j
Joint pdf …
likelihood for states and volatilities:
General conclusions from this example
Non-zeros in .
“almost” decoupled!
- coupling/coherence: “complicating” determinant

14. Decouple/Recouple: Sequential analysis of SGDLM
Basis:
Increasingly sparse for larger m
“minor” correction to likelihood
- Triangular: compositional models:
- Partly triangular, sparse:
General conclusions from this example
Strategy:
- exploit DLM analytics in decoupled models
- simulation for forecasting & state inference
Recouple: importance sampling
Decouple: variational Bayes
Non-zeros in .

15. Decouple/Recouple: (i) Time t-1 predictions
Decoupled parallel models: Conjugate state priors
- Evolve : Recouple : Forecast - C T
EBAY
BOA GE
Synthetic future
states
General conclusions from this example
Recouple
Joint predictions
Synthetic futures

16. = { decoupled conjugate forms }
Decouple/Recouple: (ii) Time t updating
Full posterior
= { decoupled conjugate forms }
Importance sampler
IS weights *
Conjugate
state posteriors
Decouple ?
Simulation-based
full posterior
SPX C
EBAY
BOA GE
General conclusions from this example
Observe

17. Decoupling/recoupling: Forecasting at time t
Parallel, normal/gamma priors:
Directly simulate and combine samples:
Transform and recouple for prediction:
Simulate the future: Monte Carlo sample
General conclusions from this example
Predict further as needed e.g.

18. Decoupling/recoupling: Updating at time t
Exact posterior:
“Complicating” term
Parallel, normal/gamma
“Most” of the posterior
Full posterior inferences: Monte Carlo importance sampling
Importance weights:
General conclusions from this example
inference using weighted samples
may resample for uniform weights
very large samples: trivial computations
… decoupled conjugate posteriors in parallel

19. Decoupling posteriors at time t
Posterior “almost” decoupled normal-inverse gammas
Decoupling: match a product form with the full posterior importance sample
Variational Bayes (mean field approximation)
Normal-gamma parameters:
General conclusions from this example
min Kullback-Leibler divergence:
Decoupled - normal-gamma posteriors:

20. Decoupling/recoupling: Evolution to time t+1
Parallel states - independent evolutions
Kullback-Leibler divergence: Decreases through evolutions
Time t+1: Parallel, normal/inverse-gamma priors:
Time t+1: Decoupled priors
Recouple to forecast
Update
Decouple to evolve ….
General conclusions from this example
Completes the t : t+1 forecast/update/evolve sequence
… Continue ...

21. Side note: KL divergence and importance sampling
Kullback-Leibler (KL) divergence (of q from p)
Importance sample (IS)
target, “true”
approximation
- “good” approximation?
- divergence scale?
IS accuracy measures:
Effective sample size (ESS):
- uniform weights:
- cost of IS cf. independent samples
Calibrate KL to ESS scale:
General conclusions from this example
Entropy of weights relative to uniform:

22. Example: SGDLM for 400 S&P stock price series
m=400 stocks & S&P index :
daily closing prices
1/2002 – 10/2013
Model (log-difference) returns
Training data : 1/2002 – 12/2006
Test/prediction/portfolio decisions :
1/2007 – 10/2013

23. S&P example – Models for comparison
W: Traditional Wishart discount DLM
- common predictor models only
- “oversmooths” volatilities with “many” series
Various other models based on:
Lagged &/or changes of
- TNX: 10-year T-bill rate
- VIX: implied 30-day volatility index
- S&P index

24. S&P example – Simultaneous parental sets
Bayesian selection, model averaging?
Stochastic search and selection of candidate sp(j)?
Many sp(j) “exchangable”; market/industry segment representatives
Names of parents? - Potentially for interventions, interpretations
- Important for multivariate volatility characterization here?
Decision problem, or inference problem?
Dimension: Hard problem – conceptually and computationally

25. S&P example – Simultaneous parental set specification
Simple (ad-hoc) example: Bayesian “hotspot”
choose 20 parents per series
Update/refresh periodically (end of each year, 252 days)
How?
Run Wishart discount DLM in parallel to SGDLM
Quick simulation at any time point:
- 20 “most important” parents
- Posterior ranks on abs(precision matrix entries)

26. S&P SGDLMs : 1-step forecast insights
data
1-step forecasts volatilities
co-volatilities
data
1-step forecasts volatilities
Empirical (60-day MA) - Wishart DLM - SGDLM

27. S&P example – Simultaneous parental set turnover
sp(3M)

28. Realized 1-step forecast CDF transform : U(0,1) is perfect
S&P SGDLM : Can we ignore recoupling?
Realized 1-step forecast CDF transform : U(0,1) is perfect
No recoupling
Full recoupling

29. S&P SGDLMs and portfolios
Traditional Bayesian/Markowitz portfolios:
target return
minimize risk (portfolio SD)
subject to constraints
Daily trading
Hypothetical trading cost: 10bp

30. S&P SGDLMs – Sharpe ratios

31. S&P SGDLMs and portfolios: Cumulative returns
SGDLM: M1
Wishart: W1
Trading costs adjusted returns
Uniform improvements (Sharpe ratios, etc) over WDLMs

32. S&P example – Monitoring decoupling/recoupling
I=5,000 Monte Carlo sample size
[ recoupling ]
ESS
Entropy
[ decoupling ]
Potential to intervene- “signal” of deterioration of ESS:
Increase sample size &/or modify model: parental sets, discounts, etc

33. St Louis Fed “Financial Markets Stress Index”
8/07
Subprime loan PR
Northern Rock bailout
10/08
US loans buy-back
NES Act
3/10
Eurozone crisis
EU/ECB 1st responses
8/11
US credit rating downgrade
Sharp drops in global markets

34. SGDLM decoupling/recoupling and computations
- Flexible, customisable univariate models
- Flexible, sparse stochastic volatility matrix model
- No series-order dependence
- Scale-up : conceptual & computational
- On-line sequential learning: Analytics + cheap simulation + VB
… parallelisable within each time step
- GPU computation
- Recouple: CPU for coherence
- Decouple/Recouple IS&VB: theory, monitoring per series

35. People & Links www.stat.duke.edu/~mw Lutz Gruber PhD (TUM 2015)
QuantCo, Boston & UN Lincoln
GPU-accelerated Bayesian learning in SGDLMs, Bayesian Analysis, 2016
Bayesian forecasting and portfolio decisions using SGDLMs, EcoStat 2017