1. “In the space of 176 years the Lower Mississippi has shortened itself 242 miles. This is an average of a trifle over 1 mile and 1/3 per year. Therefore any calm person, who is not blind or idiotic can see that in the … Silurian Period, just a million years ago next November, the Lower Mississippi River was upward of one million three hundred thousand miles long, and stuck out over the Gulf of Mexico like a fishing rod. … There is something fascinating about science. One gets such wholesale returns of conjecture out of such a trifling investment of fact.” Mark Twain, 1874, Life on the Mississippi

2. The QuIRK Experience with Curriculum Reform Nathan D. Grawe Director of the Quantitative Inquiry, Reasoning, and Knowledge (QuIRK) Initiative Carleton College Serc.carleton.edu/quirk With support from the US Department of Education’s Fund for the Improvement of Post-Secondary Education, the National Science Foundation, and the WM Keck Foundation.

3. Agenda How does QuIRK understand quantitative reasoning (QR)? (20 minutes) How does QuIRK understand quantitative reasoning (QR)? (20 minutes) QuIRK’s model of collaboration with the College’s writing program (10 minutes) QuIRK’s model of collaboration with the College’s writing program (10 minutes) Key Elements of the QuIRK professional development program (10 minutes) Key Elements of the QuIRK professional development program (10 minutes) Discussion: Where are there complementarities between QR and other Quinnipiac initiatives? (10 minutes) Discussion: Where are there complementarities between QR and other Quinnipiac initiatives? (10 minutes) Models of QR Curricula (10 minutes) Models of QR Curricula (10 minutes) How do we know if curricular reforms are working? (45 minutes) How do we know if curricular reforms are working? (45 minutes) Open Q&A/Discussion (15 minutes) Open Q&A/Discussion (15 minutes)

4. What is QR?

5. The habit of mind to consider the power and limitations of quantitative evidence in the evaluation, construction, and communication of arguments in public, professional, and personal life.

6. What is QR? Four facets of QR: 1) QR requires a basic skill set Arithmetic, including percentages, graphs Arithmetic, including percentages, graphs Estimation Estimation Elementary probability and statistics Elementary probability and statistics Basic geometry of measurement (volume, area, perimeter) Basic geometry of measurement (volume, area, perimeter) Elementary growth patterns: linear (constant quantity per unit time) and exponential (constant percentage change per unit time). Elementary growth patterns: linear (constant quantity per unit time) and exponential (constant percentage change per unit time).

7. What is QR? Four facets of QR: 1) QR requires a basic skill set 2) QR demands application in context "The test of numeracy, as of any literacy, is whether a person naturally uses appropriate skills in many different contexts" -Lynn Steen (2001)

8. What is QR? Four facets of QR: 1) QR requires a basic skill set 2) QR demands application in context 3) QR involves argument

9. What is QR? Four facets of QR: 1) QR requires a basic skill set 2) QR demands application in context 3) QR involves argument “Deploying numbers skillfully is as important to communication as deploying verbs.” -Max Frankel, The New York Times Magazine

10. What is QR? Four facets of QR: 1) QR requires a basic skill set 2) QR demands application in context 3) QR involves argument 4) QR is a habit of mind “[QR] is not a discipline but a way of thinking….” -Lynn Steen Achieving Quantitative Literacy

11. What do the numbers show?

13. Compared to what?

14. How was the variable operationalized?

15. Controlling for what?

16. About 50% of the differential has to do with different career choices…. About 50% of the differential has to do with different career choices…. 25% percent involves greater time women spend on care-taking…. 25% percent involves greater time women spend on care-taking…. The other 25 percent is due to bias and prejudice….” The other 25 percent is due to bias and prejudice….”

17. What is QR? “…sophisticated reasoning with elementary mathematics more than elementary reasoning with sophisticated mathematics.” -Lynn Steen Achieving Quantitative Literacy

18. QuIRK’s Collaboration with the Writing Program Shared workshops Shared workshops Coordinated professional development support (summer grants) Coordinated professional development support (summer grants) Inter-related leadership structures Inter-related leadership structures Co-authored work Co-authored work

19. QuIRK’s Collaboration with the Writing Program Addresses key aspects of QR (rhetorical issues including the social construction of numbers) Addresses key aspects of QR (rhetorical issues including the social construction of numbers) Defuses “redmedial math” misunderstanding Defuses “redmedial math” misunderstanding Surmounts institutional culture Surmounts institutional culture

20. QuIRK’s Collaboration with the Writing Program Writing Program Writing Program EthIC EthIC ACE ACE Visuality Visuality

21. Key Elements of QuIRK Program AssessmentProfessionalDevelopment New Courses/ Assignments Summer Curricular Revision

22. Discussion 1. What programs have been most effective at engaging faculty at Quinnipiac? 2. What programs seem to have a potential overlap with QR? 3. Who are possible leaders outside math?

23. Models of QR Curricula Single Skills Course Requirement Single Skills Course Requirement

24. Models of QR Curricula Skills Qualification + Application Course Skills Qualification + Application Course

25. Models of QR Curricula Skills-Based (multi-?) Course Skills-Based (multi-?) Course

26. Models of QR Curricula Multi-Experience Application in Context Multi-Experience Application in Context

27. How Would We Know if Reform is Working? “If you would understand something, try to measure it.” -Lloyd Bond

28. How Would We Know if Reform is Working? Four facets of QR: 1) QR requires a basic skill set 2) QR demands application in context 3) QR involves communication 4) QR is a habit of mind

29. How Would We Know if Reform is Working? “[QR] is largely absent from our current systems of assessment and accountability.” -Lynn Steen (2001) -Lynn Steen (2001)

30. How Would We Know if Reform is Working? Four facets of QR: 1) QR requires a basic skill set 2) QR demands application in context 3) QR involves communication 4) QR is a habit of mind

31. How Would We Know if Reform is Working? Four facets of QR: 1) QR requires a basic skill set 2) QR demands application in context 3) QR involves communication 4) QR is a habit of mind

32. How Would We Know if Reform is Working? Application in context: “Standardized conditions are decontextualized by design" -Grant Wiggins -Grant Wiggins “'Get Real!': Assessing for Quantitative Literacy ” “'Get Real!': Assessing for Quantitative Literacy ”

33. How Would We Know if Reform is Working? Communication: While traditional assessment tools effectively measure comprehension, the ability to read others’ QR exposition does not guarantee the ability to engage in the creation of QR arguments.

34. How Would We Know if Reform is Working? Habit of mind: “As in book literacy, evidence of students’ ability to play the messy game of the [QR] discipline depends on seeing whether they can handle tasks without specific cues, prompts, or simplifying scaffolds from the teacher- coach or test designer.” -Grant Wiggins -Grant Wiggins “'Get Real!': Assessing for Quantitative Literacy” “'Get Real!': Assessing for Quantitative Literacy”

35. How Would We Know if Reform is Working? "[QR] requires creativity in assessment, since neither course grades nor test scores provide a reliable surrogate." -Lynn Steen -Lynn Steen Achieving Quantitative Literacy

36. How Would We Know if Reform is Working? Student Attitude Surveys & Self-Reports

37. How Would We Know if Reform is Working? JMU’s QR Test Web-based Web-based Multiple-choice items Multiple-choice items 24 operational items 24 operational items Scores correlate positively with grades in and exposure to relevant courses Scores correlate positively with grades in and exposure to relevant courses Contact: Donna Sundre Contact: Donna Sundre

38. How Would We Know if Reform is Working? “Life is not like a multiple choice test.” Example Performance Task: Analyze various possible causes of several recent airplane accidents so that your employer can evaluate recent decision to buy a plane for sales staff.

39. How Would We Know if Reform is Working? Available information: 1: Newspaper articles about the accident 1: Newspaper articles about the accident 2: Federal Accident Report on in-flight breakups in single engine planes 2: Federal Accident Report on in-flight breakups in single engine planes 3: Your company’s internal email 3: Your company’s internal email 4: Charts on SwiftAir's performance characteristics 4: Charts on SwiftAir's performance characteristics 5: Amateur Pilot article comparing SwiftAir 235 to similar planes 5: Amateur Pilot article comparing SwiftAir 235 to similar planes 6: Pictures and description of SwiftAir Models 180 and 235 6: Pictures and description of SwiftAir Models 180 and 235

40. How Would We Know if Reform is Working? 357 institutions in 46 states have tested more than 70,000 students with the CLA Contact: cla@cae.org

41. How Would We Know if Reform is Working? "The interdisciplinary and contextual nature of [QR] cries out for a cross cutting approach." -Lynn Steen -Lynn Steen Achieving Quantitative Literacy “We want to regularly assess student work with numbers and numerical ideas in the field….” -Grant Wiggins “'Get Real!': Assessing for Quantitative Literacy”

42. How Would We Know if Reform is Working? QuIRK’s idea: Measure QR in the natural context of papers written in courses across the curriculum—papers written for “authentic” purposes.

43. How Would We Know if Reform is Working? Sophomore Writing Portfolio 3-5 papers plus reflective essay written in 2 of the 4 college divisions -observation-analysis-interpretation -documented sources -thesis-driven argument

44. How Would We Know if Reform is Working? Writing requirement assessment: 30 faculty members, 3 days, 450 portfolios QR assessment: 6-8 faculty members, 3 days, random sample of portfolios, 1 paper each drawn from analysis, interpretation, or observation categories

45. How Would We Know if Reform is Working? "The interdisciplinary and contextual nature of [QR] cries out for a cross cutting approach." -Lynn Steen -Lynn Steen Achieving Quantitative Literacy “We want to regularly assess student work with numbers and numerical ideas in the field….” -Grant Wiggins “'Get Real!': Assessing for Quantitative Literacy”

46. How Would We Know if Reform is Working? QuIRK’s idea: Measure QR in the “natural” context of papers written in courses across the curriculum—papers written for “authentic” purposes.

47. How Would We Know if Reform is Working? The Carleton Sample: Sophomore Writing Portfolio 3-5 papers plus reflective essay written in 2 of the 4 college divisions -observation-analysis-interpretation -documented sources -thesis-driven argument

48. How Would We Know if Reform is Working? Writing requirement assessment: 30 faculty members, 3 days, 450 portfolios QR assessment: 6-8 faculty members, 3 days, random sample of portfolios, 1 paper drawn from each

49. How Would We Know if Reform is Working? 1. A little book-keeping

50. How Would We Know if Reform is Working? 2. Did the student take the paper in a QR-relevant direction?

51. How Would We Know if Reform is Working? Central Use: Use of numbers to address a central question, issue, or theme Peripheral Use: Use of numbers to provide useful detail, enrich descriptions, present background, or establish frames of reference

52. How Would We Know if Reform is Working? The importance of the periphery: “Even for works that are not inherently quantitative, one or two numeric facts can help convey the importance or context of your topic.” -Jane Miller -Jane Miller The Chicago Guide to Writing About Numbers

53. How Would We Know if Reform is Working? 3. If relevant, to what extent did the student actually use QR?

54. How Would We Know if Reform is Working? 4. If present, rate the holistic quality of the QR.

55. How Would We Know if Reform is Working? 5. Spotlight on several recurring problems:

56. How Would We Know if Reform is Working? 6. Does assignment explicitly call for QR?

57. Some Carleton Results “…numeracy is not something mastered in a single course….Thus quantitative material needs to permeate the curriculum, … so that students have opportunities to practice their skills and see how useful they can be in understanding a wide range of problems.” “…numeracy is not something mastered in a single course….Thus quantitative material needs to permeate the curriculum, … so that students have opportunities to practice their skills and see how useful they can be in understanding a wide range of problems.” -Derek Bok (2006) “…authentic and enduring learning…can rarely succeed one course at a time.” -Lee Shulman (1997) -Lee Shulman (1997)

58. Some Carleton Results “If [QR] remains the responsibility solely of mathematics departments—especially if it is caged into a single course such as ‘Math for Liberal Arts’—students will continue to see [QR] as something that happens only in the mathematics classroom.” -Lynn Steen -Lynn Steen Achieving Quantitative Literacy

59. Some Carleton Results Changing faculty values Changing faculty values QuIRK 18-month participation rate Overall: 65% Natural & Social Sciences:75% Arts, Lit, & Humanities:55%

60. Some Carleton Results Changing faculty practice Changing faculty practice

61. Some Carleton Results Changing faculty practice Changing faculty practice New curriculum requirement: Three Quantitative Reasoning Encounters

62. Some Carleton Results Changing student attitudes Changing student attitudes Percent of seniors reporting quantitative tools very important or essential to their lives? 54% in 2005 to 68% in 2007

63. Some Carleton Results Changing student practice Changing student practice Seniors reporting their use of quantitative tools is stronger or much stronger since college entry: 67% in 2005 to 80% in 2007 Students who have taken QR first-year seminar are MUCH more likely to write papers deemed of highest quality (50% vs. 4.1%).

64. Some Carleton Results Informing campus discussions Informing campus discussions Neither relevance, extent, nor quality ratings significantly differed by gender, ethnicity.