1. Parameter Redundancy and Identifiability Diana Cole and Byron Morgan University of Kent Initial work supported by an EPSRC grant to the National Centre for Statistical Ecology Herring GullWandering Albatross Striped Sea Bass Great Crested Newts

2. Introduction If a model is parameter redundant you cannot estimate all the parameters in the model. Parameter redundancy equivalent to non-identifiability of the parameters. A model that is not parameter redundant will be identifiable somewhere (could be globally or locally identifiable). Parameter redundancy can be detected by symbolic algebra. Ecological models and models in other areas are getting more complex – then computers cannot do the symbolic algebra and numerical methods are used instead. In this talk we show some of the tools that can be used to overcome this problem.

3. Example 1- Cormack Jolly Seber (CJS) Model Capture-Recature Herring Gulls (Larus argentatus) capture-recapture data for 1983 to 1986 (Lebreton, et al 1995) Numbers Ringed: Numbers Recaptured:  83  84  85 Ringing yr 83  84  85  Recapture yr 84 85 86

4. Example 1- Cormack Jolly Seber (CJS) Model  i – probability a bird survives from occasion i to i+1 p i – probability a bird is recaptured on occasion i  = [  1,  2,  3, p 2, p 3, p 4 ] recapture probabilities Can only ever estimate  3 p 4 - model is parameter redundant or non-identifiable.

5. Derivative Method (Catchpole and Morgan, 1997) Calculate the derivative matrix D rank(D) = 5 rank(D) = 5 Number estimable parameters = rank(D). Deficiency = p – rank(D) no. est. pars = 5, deficiency = 6 – 5 = 1

6. Derivative or Jacobian Rank Test Jacobian is the transpose of the derivative matrix, so two are interchangeable. Uses of rank test: – Catchpole and Morgan (1997) exponential family models, mostly used in ecological statistics. – Rothenberg (1971) original general use, examples econometrics. – Goodman (1974) latent class models. – Sharpio (1986) non-linear regression models. – Pohjanpalo (1982) first use for compartment models.

7. Derivative or Jacobian Rank Test The key to the symbolic method for detecting parameter redundancy is to find a derivative matrix and its rank. Models are getting more complex. The derivative matrix is therefore structurally more complex. Maple runs out of memory calculating the rank. Examples: Hunter and Caswell (2009), Jiang et al (2007) How do you proceed? – Numerically – but only valid for specific value of parameters. But can’t find combinations of parameters you can estimate. Not possible to generalise results. – Symbolically – involves extending the theory, again it involves a derivative matrix and its rank, but the derivative matrix is structurally simpler. Wandering Albatross Multi-state models for sea birds Striped Sea Bass Age-dependent tag-return models for fish

8. Exhaustive Summaries An exhaustive summary, , is a vector that uniquely defines the model (Walter and Lecoutier, 1982). Derivative matrix r = Rank(D) is the number of estimable parameters in a model. p parameters; d = p – r is the deficiency of the model (how many parameters you cannot estimate). If d = 0 model is full rank (not parameter redundant, identifiable somewhere). If d > 0 model is parameter redundant (non-identifiable). More than one exhaustive summary exists for a model CJS Example:

9. Exhaustive Summaries Choosing a simpler exhaustive summary will simplify the derivative matrix. CJS Example: Computer packages, such as Maple can find the symbolic rank of the derivative matrix if it is structurally simple. Exhaustive summaries can be simplified by any one-one transformation such as multiplying by a constant, taking logs, and removing repeated terms. A simpler exhaustive summary can also be found using reparameterisation.

10. Methods For Use With Exhaustive Summaries What can you estimate? (Catchpole and Morgan, 1998, developed separately for compartment models in Chappell and Gunn,1998 and Evans and Chappell, 2000 extended to exhaustive summaries in Cole and Morgan, 2009a) A model: p parameters, rank r, deficiency d = p – r There will be d nonzero solutions to  T D = 0. Zeros in  s indicate estimable parameters. Example: CJS, regardless of which exhaustive summary is used Solve PDEs to find full set of estimable pars. Example: CJS, PDE: Can estimate:  1,  2, p 2, p 3 and  3 p 4

11. Methods For Use With Exhaustive Summaries Extension Theorem (Catchpole and Morgan, 1997 extended to exhaustive summaries in Cole and Morgan, 2009a) Suppose a model has exhaustive summary  1 and parameters  1. Now extend that model by adding extra exhaustive summary terms  2, and extra parameters  2. (eg. add more years of ringing/recovery) New model’s exhaustive summary is  = [  1  2 ] T and parameters are  = [  1  2 ] T. If D 1 is full rank and D 2 is full rank, the extended model will be full rank. The result can be further generalised by induction. Method can also be used for parameter redundant models by first rewriting the model in terms of its estimable set of parameters.

12. Methods For Use With Exhaustive Summaries The PLUR decomposition Write derivative matrix which is full rank r as D = PLUR (P is a square permutation matrix, L is a lower diagonal square matrix, with 1’s on the diagonal, U is an upper triangular square matrix, R is a matrix in reduced echelon form). If Det(U) = 0 at any point, model is parameter redundant at that point (as long as R is defined). The deficiency of U evaluated at that point is the deficiency of that nested model (Cole and Morgan, 2009a). Example 2: Ring-recovery model: Rank(D) = 5 Therefore nested model is parameter redundant with deficiency 1

13. Finding simpler exhaustive summaries Reparameterisation 1.Choose a reparameterisation, s, that simplifies the model structure. CJS Model (revisited): 2.Reparameterise the exhaustive summary. Rewrite the exhaustive summary,  (  ), in terms of the reparameterisation -  (s).

14. Reparameterisation 3.Calculate the derivative matrix D s. 4.The no. of estimable parameters = rank(D s ) rank(D s ) = 5, no. est. pars = 5 5.If D s is full rank ( Rank(D s ) = Dim(s) ) s = s re is a reduced-form exhaustive summary. If D s is not full rank solve set of PDE to find a reduced-form exhaustive summary, s re. There are 5 s i and the Rank(D s ) = 5, so D s is full rank. s is a reduced-form exhaustive summary.

15. Reparameterisation 6.Use s re as an exhaustive summary. A reduced-form exhaustive summary is Rank(D 2 ) = 5; 5 estimable parameters. Solve PDEs: estimable parameters are  1,  2, p 2, p 3 and  3 p 4

16. Reparameterisation Example 2 Hunter and Caswell (2009) examine parameter redundancy of multi- state mark-recapture models, but cannot evaluate the symbolic rank of the derivative matrix (developed numerical method). 4 state breeding success model: survival breeding given survival successful breedingrecapture Wandering Albatross 1 3 2 4 1 success 2 = failure 3 post-success 4 = post-failure

17. Reparameterisation 1.Choose a reparameterisation, s, that simplifies the model structure. 2.Rewrite the exhaustive summary,  (  ), in terms of the reparameterisation -  (s).

18. Reparameterisation 3.Calculate the derivative matrix D s. 4.The no. of estimable parameters =rank(D s ) rank(D s ) = 12, no. est. pars = 12, deficiency = 14 – 12 = 2 5.If D s is full rank s = s re is a reduced-form exhaustive summary. If D s is not full rank solve set of PDE to find a reduced-form exhaustive summary, s re.

19. Reparameterisation Method 6.Use s re as an exhaustive summary. Breeding Constraint Survival Constraint  1 =  2 =  3 =  4  1 =  3,  2 =  4  1 =  2,  3 =  4  1,  2,  3,  4  1 =  2 =  3 =  4 0 (8)0 (9)1 (9)1 (11)  1 =  3,  2 =  4 0 (9)0 (10) 2 (12)  1 =  2,  3 =  4 0 (9)0 (10)1 (10)1 (12) 1,2,3,41,2,3,4 0 (11)0 (12) 2 (14)

20. Reparameterisation Method Further Examples Multi-state models - general exhaustive summary has been developed if there is more than one observable state (Cole, 2009). Maple procedures for finding this exhaustive summary and the derivative matrix. Able to come up with general rules. Jiang et al (2007) age-dependent fisheries model is more complex, but essentially uses reparameterisation method (Cole and Morgan, 2009b). Able to give general results, whereas Jiang et al (2007) result only applies for 3 years of data. Wandering Albatross Striped Sea Bass

21. Reparameterisation Method Further Examples Multi-state analysis of Great Crested Newts. The parameter redundancy of the more complex models can be examined using the reparameterisation method to find a simpler exhaustive summary. This example consists of 2 states, one observable and one unobservable, so required development of another simpler exhaustive summary (McCrea and Cole work in progress). Parameter redundancy in Pledger et al (2009)'s stopover models (Matechou and Cole unpublished work). Clint – a male great crested newt Sandpiper

22. Reparameterisation Method Further Examples Audoly et al (1998) consider a linear compartment model :

23. Reparameterisation Method Further Examples In Catchpole and Morgan (2009a) we use the reparameterisation method to show that the model is not redundant.

24. Reparameterisation Method Further Examples We show that an exhaustive summary is: By solving s i (  ) = b i i = 1,...,11 we find there is a unique solution with k 21 = b 9, k 12 = b 5 /b 9,...,V 1 = b 11. Hence the model is globally identifiable.

25. Reparameterisation Method Further Examples Dochain et al (1995) examine the identifiability of models for the activated sludge process using a non-linear compartment model. Symbolic method possible for k=1. However for k=2 model too complex. Using the reparameterisation method with the extension theorem Cole and Morgan (2009a) show that for any k there are 3k estimable parameters (out of 4k+1) of the form

26. Conclusion Exhaustive summaries offer a more general framework for symbolic detection of parameter redundancy. Parameter redundancy can be investigated symbolically by examining a derivative matrix and its rank. In the symbolic method we can find the estimable parameter combinations (via PDEs). The symbolic method can easily be generalised using the extension theorem. Parameter redundant nested models can be found using a PLUR decomposition of any full rank derivative matrix. The use of reparameterisation allows us to produce structurally much simpler exhaustive summaries, allowing us to examine parameter redundancy of much more complex models symbolically. Methods are general and can in theory be applied to any parametric model.

27. References – Audoly, S. D’Angio, L., Saccomani, M. P. and Cobelli, C. (1998) IEEE Transactions on Biomedical Engineering 45, 36-47. – Catchpole, E. A. and Morgan, B. J. T. (1997) Biometrika, 84, 187-196 – Catchpole, E. A., Morgan, B. J. T. and Freeman, S. N. (1998) Biometrika, 85, 462-468 – Chappell, M. J. and Gunn, R. N. (1998) Mathematical Biosciences, 148 21-41. – Dochain, D., Vanrolleghem, P. A. and Van Dale, M. (1995) Water Research, 29, 2571-2578. – Evans, N. D. and Chappell, M. J. (2000) Mathematical Biosciences 168, 137-159. – Goodman, L. A. (1974) Biometrika, 61, 215-231. – Hunter, C.M. and Caswell, H. (2009). Ecological and Environmental Statistics Volume 3. 797-825 – Jiang, H. Pollock, K. H., Brownie, C., et al (2007) JABES, 12, 177-194 – Lebreton, J. Morgan, B. J. T., Pradel R. and Freeman, S. N. (1995). Biometrics, 51, 1418- 1428. – Pledger, S., Efford, M. Pollock, K., Collazo, J. and Lyons, J. (2009) Ecological and Environmental Statistics Series: Volume 3. – Pohjanpalo, H. (1982) Technical Research Centre of Finland Research Report No. 56. – Rothenberg, T. J. (1971) Econometrica, 39, 577-591. – Shapiro, A. (1986) Journal of the American Statistical Association, 81, 142-149. – Walter, E. and Lecoutier, Y (1982) Mathematics and Computers in Simulations, 24, 472-482

28. References Recent Work – Cole, D. J. (2009) Determining Parameter Redundancy of Multi-state Mark- Recapture Models for Sea Birds. Presented at Eurings 2009 to appear in Journal of Ornithology. – Cole, D. J. and Morgan, B. J. T (2009a) Determining the Parametric Structure of Non-Linear Models IMSAS, University of Kent Technical report UKC/IMS/09/005 – Cole, D. J. and Morgan, B. J. T. (2009b) A note on determining parameter redundancy in age-dependent tag return models for estimating fishing mortality, natural mortality and selectivity. IMSAS, University of Kent Technical report UKC/IMS/09/003 (To appear in JABES) – See http://www.kent.ac.uk/ims/personal/djc24/parameterredundancy.htm for papers and Maple codehttp://www.kent.ac.uk/ims/personal/djc24/parameterredundancy.htm