1. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Implementing “generalization, example, exception”: Behind-the-scenes work for summarizing a pattern Jane E. Miller, PhD

2. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Overview Steps – Display the data in a chart – Indentify the dimensions of the comparison – Choose a representative example – Characterize the pattern – Identify exceptions – Write the description “Generalization, example, exception” useful for – Three-way patterns – Interaction patterns calculated from multivariate coefficients

3. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Objectives of a GEE GEE = “generalization, example, exception” Aims to describe the forest, not the trees, in a complex pattern involving a lot of numbers – Which aspects can be summarized? – Which aspects deviate from that summary? In a three-way association, describe the pattern among – Two independent variables (IVs) and a dependent variable (DV) – One IV and a series of DVs

4. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Display the data in a chart Create a chart of the association Charts are the easiest way to see patterns – Which values are highest and by how much compared to other values? – Whether trends are up, down, or level

5. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Table and corresponding chart Mode of transmission Language spoken at home/ language used on questionnaire English Spanish /English ques. Spanish /Spanish ques. Likely modes of transmission Sexual intercourse with an infected person93.687.595.0 Shared needles for IV drug use92.490.665.0 Pregnant mother to baby89.575.080.0 Blood transfusion from infected person87.581.360.0 Knowledge about AIDS transmission, by language spoken at home and ability to speak English, New Jersey, 1998

6. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Design issues for your chart Draw it to scale, so you can assess – Directionality – Size of differences and trends across values Organize variables in the order you will describe them – Theoretical criteria – Empirical order – See podcast on organizing data in tables and charts

7. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Use a spreadsheet to create your chart A spreadsheet can – Test different ways of organizing the variables in your chart – Calculate magnitude of differences across values for use in your description Difference (subtraction) Ratios (e.g., relative risks) Percentage change

8. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Identify dimensions of the comparison One dimension for each variable or set of variables in your table or chart In a table, one dimension for each – Row – Column – Panel In a chart, one dimension for each – Axis – Legend item

9. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Dimensions in a three-way table Moving down the rows within one column Moving across the columns within one row Language spoken at home/ language used on questionnaire Mode of transmissionEnglish Spanish /English ques. Spanish /Spanish ques. Likely modes of transmission Sexual intercourse with an infected person93.687.595.0 Shared needles for IV drug use92.490.665.0 Pregnant mother to baby89.575.080.0 Blood transfusion from infected person87.581.360.0 Knowledge about AIDS transmission, by language spoken at home and ability to speak English, New Jersey, 1998

10. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Dimensions in a clustered bar chart Moving across clusters within one bar color Moving across bar colors within one cluster

11. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Dimensions in a multiple-line chart Moving left to right along one line Moving vertically across lines at one value of the x variable

12. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Choose a representative example Identify one representative example as the basis for each generalization about the shape of the pattern – Must characterize many if not most values in the pattern – Not an arbitrary choice! Find one value of the first independent variable to use to describe the general shape of the association between the second IV and the DV. E.g., – Hold one IV constant while you vary the other – Don’t vary values of both IVs at the same time

13. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. What does “representative” mean? A representative example of a general pattern will be characteristic of the association between the second IV and the DV for many if not all subgroups of that first IV. E.g., – A region that shows a time trend in prices that is similar to that observed in several other regions – A topic for which the rank order of language groups in test performance is the same

14. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Possible starting points for identifying representative examples Theoretically based – A value of particular interest for your research question – A summary measure that combines results for the various component variables Useful if comparing across a series of related outcomes Empirically based – Start at one end of an empirically organized axis, column, or row – Overall sample, for a comparison across groups – Modal (most common) group

15. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Checking for representativeness of example in a three-way table Describe the pattern across columns within one row Test whether the direction and magnitude of that pattern also applies to other rows Language spoken at home/ language used on questionnaire Mode of transmissionEnglish Spanish /English ques. Spanish /Spanish ques. Likely modes of transmission Sexual intercourse with an infected person93.687.595.0 Shared needles for IV drug use92.490.665.0 Pregnant mother to baby89.575.080.0 Blood transfusion from infected person87.581.360.0 Mean percentage of “likely” questions correct91.783.6 75.0 Knowledge about AIDS transmission, by language spoken at home and ability to speak English, New Jersey, 1998

16. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Checking for representativeness of example in a three-way table Describe the pattern down the rows within one column Test whether the direction and magnitude of that pattern also applies to other column Language spoken at home/ language used on questionnaire Mode of transmissionEnglish Spanish /English ques. Spanish /Spanish ques. Likely modes of transmission Sexual intercourse with an infected person93.687.595.0 Shared needles for IV drug use92.490.665.0 Pregnant mother to baby89.575.080.0 Blood transfusion from infected person87.581.360.0 Knowledge about AIDS transmission, by language spoken at home and ability to speak English, New Jersey, 1998

17. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Checking representativeness of an example in a multiple-line chart Describe the pattern cutting vertically across lines (regions) at one value of the x variable (year) Check whether that rank order of regions also applies to the other dates

18. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Checking representativeness of an example in a multiple-line chart Describe the pattern along one line moving left to right Test whether that pattern applies to the other lines

19. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Trial and error in identifying a representative example Might have to try several groups or values to find one that has a pattern that is similar to that in other groups If your initial choice isn’t similar to any of the others in terms of direction or magnitude, it might be an exception See whether other values have a pattern that can be generalized

20. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Characterize the pattern For your representative value of one independent variable, characterize these aspects of the association between the other independent variable and the dependent variable: – Direction – Magnitude – Statistical significance

21. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Characterizing direction in a cross-sectional comparison Which categories of the independent variable (IV) have the highest and lowest values of the dependent variable (DV)? For multiple category IVs, describe the rank order of categories according to values of the DV

22. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Characterizing direction for trends Describe whether each line slopes – Upward – Downward – Is approximately level – Follows a standard statistical distribution e.g., a normal curve – Asymptotically approaches some value

23. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Characterizing magnitude Do some behind-the-scenes math to measure the size of differences between values you are comparing. E.g., – Difference (subtraction or OLS coefficient) – Ratio (division, or odds ratio from logit model) – Percentage difference or change See chapters on types of quantitative comparisons to help you choose the kind of calculation that best suits your objectives

24. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Take notes about direction and magnitude On a hardcopy chart, make notes about: – Direction: Use “>,” “<,” “=,” or arrows – Size: Jot results of calculations to compare values To help you later write a correct, complete description, mention – Units – Type of calculation – Shape: Verbs, adjectives, adverbs, analogies, or metaphors that capture the shape of the pattern – Statistical significance: of specific contrasts, marked with Symbols Shading (highlighter) Don’t worry about full sentences at this point

25. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Identify exceptions If parts of your table or chart depart substantially from the general pattern you have identified, they are exceptions Exceptions can occur in terms of – Direction – Size – Statistical significance

26. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Exceptions in direction of a trend The median sales prices dipped in the early 1990s in the West, but continued upward or remained level in each of the other three regions – The West was the exception

27. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Exceptions in magnitude of a cross-sectional difference The difference between odds ratios across racial groups is much larger among the non-poor than in the other two income groups (compare brackets 1 and 2 to bracket 3) – Generalize based on the two income groups for which the racial gap in ER use is similar (poor and near poor) – Non-poor are the exception

28. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Exceptions in statistical significance Examples of this type of exception – All but one of the coefficients in a regression model are statistically significant at p < 0.05 – Only one of the coefficients in a regression model is statistically significant The generalization is lack of statistical significance – In stratified models of the same outcome for several groups, the coefficient on one or more specific independent variables are statistically significant only for one group

29. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Write the description Organize the material into paragraphs – One paragraph for each major pattern to be described Write topic sentences for each paragraph – Introduce the substantive concepts to be discussed in that paragraph Write evidentiary sentences to describe the – General pattern – Any exceptions In those descriptions, mention – Direction – Magnitude – Statistical significance

30. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Wording for a generalization If a pattern characterizes many of the numbers you are summarizing, introduce the generalization with – “In general,...” – “Typically,...” – “By and large...” – If the pattern characterizes all groups, say so! – “In every region studied,...” – “Throughout the entire period of observation...” – “Regardless of [characteristic],...”

31. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Wording to present the example To document a pattern – Introduce numeric evidence – Refer to the table or chart that reports all of the numbers E.g., – “For instance,... [numeric contrast, with the W’s specified].” – “For example, as shown in Figure X... [numeric fact, with its W’s attached].” Reminder: “the W’s” refer to when, where, who, and what

32. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Wording to introduce exception(s) To introduce exceptions to a general pattern, write – “An exception [to that pattern] is... ” – “On the other hand,...” – “However,...” E.g., “In seven out of 10 years studied, [general pattern and example]. However, in the other three years, [contrasting pattern = exception].”

33. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Wording for exceptions in direction Some exceptions are literally the opposite of the pattern described in the generalization. E.g., – A falling rather than rising trend – A deficit instead of excess for one group compared to another Having described the direction of the general pattern, introduce such exceptions with phrases such as: – “On the contrary,...” – “Conversely,...”

34. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Summary Use a systematic, step-by-step approach to identify which parts of a broad pattern are – Similar to one another – Different from one another In terms of direction, magnitude, and statistical significance Create a chart or table that you refer to in your prose description of that pattern Organize material into substantively cohesive paragraphs Use vocabulary and phrasing to differentiate general patterns from exceptions

35. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Suggested resources Chapter 2 and appendix A in – Miller, J.E., 2004. The Chicago Guide to Writing about Numbers. (“WA#”) OR – Miller, J.E., 2013. The Chicago Guide to Writing about Multivariate Analysis, 2nd edition. (“WAMA II”) Chapter 7 in WA# or chapter 6 in WAMA II on creating effective charts Chapter 5 in WA# or chapters 8 and 9 in WAMA II on types of quantitative comparisons Chapter 9 of WA#, section on three-way associations Chapter 16 of WAMA II, on interactions

36. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Suggested online resources Podcasts on – Reporting one number – Comparing two numbers or series of numbers – Summarizing a pattern with many numbers – Organizing data in tables and charts – Visualizing shapes of interaction patterns – Writing prose to present results of an interaction

37. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Suggested practice exercises Study guide to The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. – Question #9 in problem set for chapter 2 – Question #9 in problem set for chapter 6 – Suggested course extensions for chapter 2 – “Reviewing” exercise #7 – “Writing” and “revising” exercise #4 chapter 14 – “Applying statistics and writing” exercises #3 and 4 – “Revising” exercise #3

38. The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition. Contact information Jane E. Miller, PhD jmiller@ifh.rutgers.edu Online materials available at http://press.uchicago.edu/books/miller/multivariate/index.html