1. Choosing Between Diversity Indices
James A. Danoff-Burg
Dept. Ecol., Evol., & Envir. Biol.
Columbia University

2. Alpha Diversity Indices
A diversity of diversities
Log Alpha
Log-Normal Lambda
Q-Statistic
Simpson
McIntosh
Berger-Parker
Shannon-Wiener
Brillouin
How to choose between these?
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

3. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Bases for Choice
Appropriateness of each index for your data
Discriminant ability of the index
Statistical Comparability
Widespread utility of the index
Your Question
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

4. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Bases for Choice
Appropriateness of each index for your data
Discriminant ability of the index
Statistical Comparability
Widespread utility of the index
Your Question
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

5. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Appropriateness
Index assumptions need to be met
Abundance model of data
Sensitivity to sample size
Each index needs to be considered for all of these aspects
determines whether can be used for your data
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

6. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Assumptions
Alpha diversity indices do not make many assumptions
No assumptions made about species abundance distributions
Cause of distribution  not needed
species abundance models have assumptions about these
Geometric – niche pre-emption, regular arrivals
Log – niche pre-emption, irregular arrival intervals
Log-Normal – successively apportioning available niche space of all resources in proportion to abundance
Broken Stick – simultaneously apportioning available niche space of one resource in proportion to abundance
Shape of curve
“Non-parametric”
Normality is not needed
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

7. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Abundance Model
Some indices perform better under a specific abundance model
Example: Simpson – probability that two individuals are of the same species
Geometric
Underestimate Simpson diversity value
Log
Log-Normal
Best for Simpson diversity analysis analysis
Broken Stick
May overestimate Simpson diversity value
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

8. Abundance Model Fitting
Main problem: Often have multiple models that fit the data
Occasionally because of low number of abundance classes
Log2 has only 11 classes (octaves) even possible
Most data have less than 11
E.g., less than 256 individuals in a species
Resulting in only 8 classes
Fewer classes, mean fewer opportunities for departures from fit
Small data sets  fit many models
Few spp in each abundance class  decreased discriminability
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

9. Abundance Model Fitting
Secondary problem: Log-Normal is a frequent consequence
Often because of the central limit tendency of large data sets
If a data set has many species  often log-normal distribution results
Does not necessarily mean that the community has assembled by a successive breaking of the available nice space
As is the assumption with the log-normal distribution
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

10. Appropriateness – Sample Effort
Some indices are tremendously sensitive to sample size
Low replication  skewed values
Idiosyncratic results
Not truly representative of the environment
Indices sensitive to inadequate sampling
S = very sensitive to sampling effort
Dominance indices (Simpson, Berger-Parker, McIntosh)
Information statistics indices (Shannon)
Evenness indices
Indices insensitive to sampling effort
Always: Log series a, 1/d (influenced by abd of most abd sp)
If more than 50% of spp represented: Q
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

11. Appropriateness - Sampling Effort
How to determine when you have completely sampled the environment?
Assuming prior information
Leveling off of S with adding more samples
If interest is largely richness
Leveling off of Pielou’s t point
If interest is proportional abundance
Leveling off point = adequate sample size
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

12. Appropriateness, Sample Size - When is Enough Enough?
Leveling off of S with adding more samples
If interest is largely richness
S is more sensitive to sample size than diversity indices
Need more samples
Leveling off of Pielou’s t point
If interest is proportional abundance
Any diversity index can be used 50 S
Pielou’s t 40
Pielou’s t
Diversity Index Value 30
H’ Shannon
Pielou’s t 20
1/D Simpson
Pielou’s t
1/d Berger-Parker 10 10 20 30 40
Sample Addition Sequence
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

13. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Sampling Effort
Need consistency in sampling effort
Need to use the same effort throughout experiment
Helps to ensure comparability of indices
All would then be equal(ly biased)
When sample sizes are unequal?
Rarefy the larger sample to the smaller
More next week on rarefaction (WE 1)
Better to have many small samples than few large samples
Increases replication
Decreases thoroughness of each replicate
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

14. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Bases for Choice
Appropriateness of each index for your data
Discriminant ability of the index
Statistical Comparability
Widespread utility of the index
Your Question
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

15. Discriminant Ability of the Index
Differences are usually very subtle
Need analytical rigor to differentiate
Assuming differences exist, how best to see them?
No two sites are identical in terms of S, N, relative abundance
All sites will differ, how can we detect this?
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

16. Studies of Discriminability I
Taylor (1978)
Using 8 indices on moths, 9 sites, over 4 years
Rothamsted Insect Survey, England
Best: Log a (by far)
Next: H’, S, log-normal l, 1/D, log biomass
Useless: log-normal S* and s
Kempton (1979)
Using 4 indices on same data, 14 sites, 7 years
Best: S, H’
Useless: 1/D, 1/d
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

17. Studies of Discriminability II
Kempton & Taylor (1976)
Transformed indices > untransformed form
exp H’ > H’
1 / D > D
Kempton & Wedderburn (1978)
a and Q > any H’ and any D
Magurran (1981)
Best: Margalef (Dmg = (S-1) / ln N); U, S
Less well: HB > H’, exp H’
Worst: 1/d, D or 1/D, McIntosh D, H’ E, HB E
Morris & Lakhani (1979)
H’ > D or 1/D
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

18. Diversity Aspects Measured
Not just discriminability, but what is the index measuring?
Richness, Dominance, Evenness, Abundance…
What most affects the index?
Rare species or species richness?
Type 1 Measures
log a, log-normal l, Q, S, H’, HB, Dmg, McU
Abundance of the most common species or dominance?
Type 2 Measures
1/D or D, 1/d, McD, H’E, HBE
Within each type  significant correlation
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

19. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Bases for Choice
Appropriateness of each index for your data
Discriminant ability of the index
Statistical Comparability
Widespread utility of the index
Your Question
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

20. Statistical Comparability
Historically, statistical comparisons not made
Mostly descriptive comparisons in past
Statistical Options
H’, Var H’, t-test
Replication
Most sets of replicated estimates are normally distributed
Non-normal data can be transformed for normality
Jackknife data
Provides standard error and confidence limits as well
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

21. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Bases for Choice
Appropriateness of each index for your data
Discriminant ability of the index
Statistical Comparability
Widespread utility of the index
Your Question
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

22. Widespread Utility of the Index
Important for ensuring comparability
Between studies
Between sites
Between researchers
Most commonly used
S, H’, 1/D, log a
Less commonly used
Log-Normal l, Q
Even though highly valuable as discussed above
Dmg, McU, McE, HB, 1/d
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

23. Widespread Utility - Cautions
Be careful with the H’ Shannon
Heavily criticized, despite widespread use
“no direct biological interpretation” (Goodman 1975)
Be careful with the log a
Based only on S & N
Insensitive to changes when both stay constant
Uncommon situation
“there can be no universal best buy but there are rich opportunities for inappropriate usages” (Southwood 1978)
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

24. Indices Performance Summary
Index
Discriminant Ability
Sample Size Sensititivy
Richness, Evenness, Dominance
Calculation
Widely used?
Sensitivity to Abd models
Log a
Good
Low
Richness
Simple
Yes
N (?)
Log Normal l
Moderate
Complex N Q S
High
Margalef
Shannon
Intermediate
Brillouin
McIntosh U
Simpson
Dominance
Berger-Parker
Poor
Shannon E
Evenness
Brillouin E
McIntosh D
= Desirable Traits in an Index
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

25. Index Choice Guidelines
Clearly formulate question you are studying
Ensure equal sample sizes
Draw a Rank Abundance graph
Calculate Margalef (Richness) and Berger-Parker (Dominance) indices
Determine Log a and Q
Test fit to abundance models
Use ANOVA to test for treatment differences
Use Jackknife to improve estimate of indexes
Be consistent in choice of index across your studies
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

26. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Bases for Choice
Appropriateness of each index for your data
Discriminant ability of the index
Statistical Comparability
Widespread utility of the index
Your Question
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

27. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Your Question
What you want to know determines how you analyze your data
How important is each aspect of diversity?
Richness?
Evenness?
Dominance?
Abundance?
Per-species (relative) abundance?
Taxon diversity?
Trophic structuring?
Guild diversity?
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

28. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Your Question
What answers your question?
What are the most important aspects of diversity?
What data directly addresses your question?
How should it be presented?
How should it be emphasized?
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

29. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Your Question
Work through the gardens example
Abundance model?
Diversity aspects?
Form of the data?
Appropriate analyses?
Answer for a few subcomponent questions
Assignment:
Do above using your thesis (or the gardens data)
3-5 pages
Due 2nd April before class
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

30. Studies of Discriminability III – Gotelli & Colwell 2001
Purpose / Goal
To discuss the different ways of presenting richness
To discuss the ways in which we can approximate total species richness
To discuss the difficulties encountered when using richness
Proportional abundances
Species Density
Standardizing number of species across different sized areas
Species / Genus
Important in biogeography
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

31. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Gotelli & Colwell 2001
Differentiates between Individual-based and Sample-based assessment methods
Individual: life lists, Christmas bird counts, collector’s curves
Sample: replicated quadrats, mist nets, trap data
Hybrid: m-species lists (observing to a point)
Differentiates between accumulation and rarefaction curves (either individual or sample based)
Accumulation – total # of spp during process of data collection
Rarefaction – repeatedly subsampling without replacement from the data (progressive data reduction)
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

32. Sample Addition Sequence
Gotelli & Colwell 2001
Richness
Samples always below Individuals
Individuals are clumped
Random assortments of traps through time unclumped
Rarefaction smoothed curves
Replicated data removal process (like Pielou)
Built “right to left”
Accumulation stepped line
Observed fact
Built “left to right” 50
Individuals:
Rarefaction
Individuals: Accumulation 40 30
Samples: Rarefaction
Samples: Accumulation 20 10 10 20 30 40
Sample Addition Sequence
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

33. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Gotelli & Colwell 2001
Factors influencing richness estimates
Underlying species richness
Relative abundance distributions
Sampling effort
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

34. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Gotelli & Colwell 2001
Cautions when using richness
Scaling different sized areas to species density
Susceptible to non-linearity of increasing area and richness
Leads to an inability to extrapolate from smaller to larger areas
Species per comparable unit area
Problem with non-linear relationship between sampling efforts and richness
Species per genus
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

35. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Gotelli & Colwell 2001
Solution?
Use rarefaction on your data to standardize sampling effort
Bring larger sample down to size of smallest one
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

36. © 2003 Dr. James A. Danoff-Burg, jd363@columbia.edu
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,

37. Hypothetical Model Curves
100
Geometric Series 10
Log-Normal Series
Broken Stick Model
Log Series 1
Per Species Abundance
0.1
0.01
0.001 10 20 30 40
Species Addition Sequence
Lecture 5 – Choosing Between Diversity Indices
© 2003 Dr. James A. Danoff-Burg,