1. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Presenting statistical results to nonstatistical audiences Jane E. Miller, PhD

2. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Overview Academic and nonstatistical audiences – Defined – Interests and background Adapting description of methods Adapting presentation of results

3. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Why adapt material? A survey by Sorian and Baugh of government policy makers showed that they – Want to know how the findings relate to issues – Don’t want to wade through a formal research paper Complaints about many research reports – “too long, dense, or detailed” – “too theoretical, technical, or jargony”

4. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Example: Policy analysts and consultants Policy analysts must explain results of their models to experts in government or nonprofit agencies. Economic consultants have to communicate results of their models to corporations or community development agencies. Those experts are principally interested in – How to interpret and apply the findings. – Reassurance that you know the correct statistical methods.

5. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. How to adapt material Familiarize yourself with your audience’s interests and likely applications of your study findings. – Present your analyses to match issues of concern to them. – DON’T make them translate statistical results to fit their interests.

6. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. What statistics courses teach Statistics courses emphasize – understanding statistical assumptions – estimating models – interpreting statistical tests – assessing coefficients and model fit. Students expected to demonstrate mastery by – working with equations written in statistical notation – identifying the numbers for formal hypothesis testing

7. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. What academic papers look like Detailed review of the literature Comprehensive data and methods section Statistical tables Jargon and equations used as shorthand A real mismatch with many applied audiences

8. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Example: Study of family and county level factors associated with SCHIP disenrollment SCHIP = State Children’s Health Insurance Program – Health insurance for children in low- to moderate-income families who lack other health insurance Collaborative effort of – Rutgers University’s Center for State Health Policy – New Jersey Department of Human Services Project applied discrete time hazards models in a multilevel (hierarchical linear model [HLM]) framework

9. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Academic audiences for SCHIP study Northwestern/University of Chicago Joint Center for Poverty Research (JCPR) – Funding agency – Policy oriented Rutgers Institute for Health, Health Care Policy and Aging Research and Bloustein School of Planning and Public Policy – Both policy-oriented research units University of Pennsylvania – Academic but not policy oriented Health Services Research – Journal with research emphasis

10. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Applied audiences for SCHIP study New Jersey Department of Human Services – Raised policy question – Provided data – Client for deliverable US Department of Health and Human Services – Funding source

11. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Which would you rather have?

12. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Which would your client rather have? Table 1. Multilevel discrete-time hazards models of disenrollment from SCHIP, New Jersey, 1998–2000 Variable County Fixed Effects Model Random Effects Model Family Factors Only Random Effects Model Family + County Factors Log Rel. Hazs.e.Log Rel. Hazs.e.Log Rel. Hazs.e. Intercept–5.581(.159)–5.421(.142)–5.455(.159) Family-Level Characteristics Black Race0.047(.150)0.038(.149)0.198(.165) Hispanic Race0.121(.064)0.109(.063)0.124(.064) Plans C and D (ref = Plan B)0.826(.142)0.823(.142)0.825(.142) Interactions Black * Plans C/D0.449(.154)0.456(.154)0.451(.154) Plans C/D * Months0.078(.036)0.078(.036)0.077(.036) Plans C/D * Months 2 –0.0069(.0019)–0.0069(.0019)–0.0068(.0019) County-Level Characteristics % Black Physicians.007(.012) Cross-Level Interaction Black * % Black Physicians–0.039(.019) Random Effects Between-County Variance0.012(.007)0.005(0.006) Scaled Deviance Statistic30,824.530,948.430,895.4

13. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Chances of disenrollment by race, SCHIP plan, and county physician racial composition

14. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Adapting results for nonstatisticians Increase prominence of the substantive question. Reduce emphasis on technical details of data and methods. – Rephrase jargon and statistical concepts into colloquial language. – Avoid equations or Greek symbols. – Minimize use of formal citations. Translate results to show how they apply to real-world issues of interest to that audience.

15. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Writing style and organization Write a clear, well-organized narrative – What questions did you address? – What answers did you find? – How can the findings be applied? Use standard expository writing guidelines – Good introduction – Present numbers as evidence Explain what question each is intended to answer – Good summary of findings and what they mean See article in Chance and podcast on presenting numbers as evidence.

16. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. How to write about technical stuff Explaining why your methods are needed – Especially if using multivariate models Showing how key variables are measured Interpreting numeric values (coefficients) Reporting statistical significance Adapting tables and charts

17. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Acronyms and statistical vocabulary Even with a quantitatively sophisticated audience, don’t assume that people will know the statistical vocabulary used in other fields. – Define the term you use, then mention synonyms. If you use acronyms, spell them out at first usage. – “HEDIS”(Health Plan Employer Data and Information Set) – “HLM”(hierarchical linear model) Avoid acronyms if they are not familiar to the field or are used only once or twice.

18. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Why your methods are needed Explain what your model did that couldn’t have been answered with simpler techniques. Incorporate the specific concepts you study. Poor: “We use logistic regression and a discrete-time hazards specification to assess relative hazards of SCHIP disenrollment, with plan level as our key independent variable.” Better: “Because chances of disenrollment from the State Children’s Health Insurance Program (SCHIP) vary by the amount of time enrolled, our analyses correct for differences in duration of enrollment across families when estimating the patterns for different income levels.”

19. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Application of a method to your topic Replace technical terms with familiar names. Show how that method applies to your research question and data. Poor: “The data structure can be formulated as a two-level hierarchical linear model, with families (the level-1 unit of analysis) nested within counties (the level-2 unit of analysis).”

20. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Better presentation of methods: Tailored to the audience Better [for a nonstatistical but academic audience]: “The data have a hierarchical (or multilevel) structure, with families clustered within counties.” Better [for a lay audience]: “To disentangle the contributions of families’ and counties’ characteristics to the problem of program disenrollment, we used models that incorporated information at both levels.”

21. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Measurement of key variables To report an unfamiliar type of statistic, embed the definition in your explanation. Poor: “The sensitivity of the new screening test for diabetes is 0.90.” Better: “The new screening test had a sensitivity of 0.90, correctly identifying 90% of diabetics.”

22. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Adapting tables and charts Create small tables or charts – Divide up large complex tables into smaller parts – Focus each on one fact or pattern – Use simple, familiar formats Replace standard errors and test statistics with – p-values – Symbols such as asterisks or daggers – Formatting such as color, italics, or bold

23. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Birth weight and socioeconomic characteristics by race/ethnicity, US, 1988–1994 NHANES III Non-Hispanic white Non-Hispanic black Mexican AmericanAll Birth weight Mean (grams)3,426.83,181.33,357.33,379.2 % Low birth weight5.811.37.06.8 Socioeconomic characteristics % Teen mother9.422.918.412.5 % Mother <high school14.730.158.421.6 % Poor14.748.550.723.9 Unweighted N3,7332,9683,1129,813 Statistics are weighted to population level using weights provided with the NHANES III (US DHHS 1997). Differences across racial/ethnic origin groups were statistically significant for all variables shown (p < 0.01). OK, but…

24. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Low birthweight by race/ethnicity From second row of preceding table. p < 0.05

25. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Minority racial groups have lower SES From bottom three rows of table. All p < 0.05

26. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Interpreting OLS coefficients Emphasize direction and size of the association Name the specific variables involved Incorporate units of measurement Use colloquial language “OLS” = ordinary least squares regression

27. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Examples of interpreting βs Poor: “Age and weight were correlated.” Poor version number2: “Beta was 10.7.” Better: “For each additional year of mother’s age at the time of her child’s birth, birth weight increased by 10.7 grams.”

28. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Coefficients from logit models Replace log-odds (logit coeffs) with odds ratios. – Can be described in terms of simple multiples. – Don’t need to use the term “odds ratio” at all! Poor: “The log-hazard of disenrollment for one-child families was 0.316.” Better: “Families with only one child enrolled in the program were about 1.4 times as likely as larger families to disenroll.”

29. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Wording for statistical significance State the conclusion of the statistical test, not the raw numbers or calculations. Poor: “The log-relative hazard for SCHIP plans C and D was 0.826 with a standard error of 0.142. Because the beta was more than 2.56 times the standard error, we conclude that the effect is statistically significant at p < 0.01.” Better: “Families in SCHIP plans C and D were roughly 2.3 times as likely to disenroll as those in plan B. A difference that large is unlikely to occur by random chance alone.”

30. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Wording for LACK of statistical significance “The difference between the disenrollment rates for Plans C and D could easily have occurred by chance alone.”

31. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Summary Get to know your audience before you write. – What questions do they want answered? – How familiar are they with statistics? Avoid statistical language. – Report direction and size of associations in plain English. – Mention conclusions of inferential statistics, not the raw numbers or calculations. Use charts or simple tables to convey shape and size of numeric patterns visually.

32. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Suggested resources Chapter 20 in Miller, J. E. 2013. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Miller, J.E. 2006. “How to Communicate Statistical Findings: An Expository Writing Approach.” Chance. 19(4):43-49. Nelson, D. E., R. C. Brownson, P. L. Remington, and C. Parvanta, editors. 2002. Communicating Public Health Information Effectively: A Guide for Practitioners. Washington DC: American Public Health Association. Sorian, R., and T. Baugh. 2002. “Power of Information: Closing the Gap between Research and Policy.” Health Affairs 21 (2): 264–73.

33. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Suggested online resources Podcasts on – Reporting one number – Comparing two numbers or series of numbers – Creating effective tables and charts – Interpreting multivariate coefficients – Designing slides for a speech

34. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Suggested practice exercises Study guide to The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. – Questions #1 through 3 in the problem set for chapter 20 – Suggested course extensions for chapter 20 “Reviewing” exercises #1 through 5 “Writing” exercises #1, 2, 3, 6, 7 and 9 “Revising” exercises #2 and 4

35. The Chicago Guide to Writing about Multivariate Analysis, 2 nd edition. Contact information Jane E. Miller, PhD jmiller@ifh.rutgers.edu Online materials available at http://press.uchicago.edu/books/miller/multivariate/index.html