1. 0 U S News & World Report Undergraduate College Rankings Ruth Kallio, Associate Director for Institutional Research Office of Budget and Planning November 14, 2007 8:30 – 10:30 a.m., Michigan Room Office of Budget & Planning

2. 1 USN&WR Undergraduate College Rankings Background What do they rank? What do they rank? Undergraduate programs overall Undergraduate programs overall Some specialized undergraduate programs Some specialized undergraduate programs Graduate and professional programs overall Graduate and professional programs overall Specialized graduate & professional programmatic areas Specialized graduate & professional programmatic areas When did they begin ranking programs? When did they begin ranking programs? 1983 undergraduate 1983 undergraduate 1987 graduate and professional 1987 graduate and professional

3. 2 USN&WR Undergraduate College Rankings Background How many undergraduate colleges and universities are ranked? How many undergraduate colleges and universities are ranked? Institutions are grouped according to Carnegie classifications and for some categories, geographic region Institutions are grouped according to Carnegie classifications and for some categories, geographic region 262 national universities 262 national universities 266 liberal arts colleges 266 liberal arts colleges 574 master’s universities in 4 geographic regions 574 master’s universities in 4 geographic regions 320 baccalaureate colleges in 4 geographic regions 320 baccalaureate colleges in 4 geographic regions 1,422 institutions in total 1,422 institutions in total

4. 3 USN&WR Undergraduate College Rankings Background Who is U-M ranked against? Who is U-M ranked against? U-M is in the “national universities” category which includes: U-M is in the “national universities” category which includes: Doctoral/Research Universities -- Extensive Doctoral/Research Universities -- Extensive Doctoral Research Universities -- Intensive Doctoral Research Universities -- Intensive USN&WR publishes two sets of national universities rankings USN&WR publishes two sets of national universities rankings All national universities (N=262) All national universities (N=262) 59 of which we commonly use as “peers” 59 of which we commonly use as “peers” All public national universities (N=163) All public national universities (N=163) 33 of which we commonly use as “peers” 33 of which we commonly use as “peers”

5. 4 USN&WR Undergraduate College Rankings Background Where does U-M rank overall? Where does U-M rank overall? Prior to 1988: 7th or 8th Prior to 1988: 7th or 8th Since 1988: consistently between 21st and 25th Since 1988: consistently between 21st and 25th -- Currently: 25th (in a tie with UCLA) Where does U-M rank among public universities? Where does U-M rank among public universities? Consistently 2nd or 3rd Consistently 2nd or 3rd Currently: 3rd (in a tie with UCLA and behind California Berkeley and Virginia) Currently: 3rd (in a tie with UCLA and behind California Berkeley and Virginia)

6. 5 Ten-Year History of UM-Ann Arbor Rankings Year Overall Score Rank 1998-9987 25th (tied w/ UCLA) 1999-0073 2000-0178 25th (tied w/ UCLA & UNC-Chapel Hill) 2001-027725th 2002-0372 25th (tied w/ UCLA & Wake Forest) 2003-047525th 2004-0576 22nd (tied w/ Carnegie Mellon & Univ. of Virginia) 2005-0675 25th (tied w/ UCLA) 2006-0775 24th (tied w/ Univ. of Virginia) 2007-0873 25th (tied w/ UCLA)

7. 6 USN&WR Data Collection Procedures Surveys of colleges and universities Surveys of colleges and universities Data are supposed to conform to national standards for formatting and definitions developed by college guide publishers in conjunction with representatives from the higher education community (aka, “Common Data Set”) Data are supposed to conform to national standards for formatting and definitions developed by college guide publishers in conjunction with representatives from the higher education community (aka, “Common Data Set”) 92.4% of the institutions surveyed in 2007 responded 92.4% of the institutions surveyed in 2007 responded USN&WR will estimate data for any institution that cannot or will not provide them with some or all of the necessary data. USN&WR will estimate data for any institution that cannot or will not provide them with some or all of the necessary data. Survey of presidents, provosts, and deans of admissions (51% response rate in 2007) Survey of presidents, provosts, and deans of admissions (51% response rate in 2007)

8. 7 How does USN&WR arrive at the overall score and ranking for each national university? Three types of components Three types of components Reputation Reputation Measures of Educational Success Measures of Educational Success Resources Resources

9. 8 Breakdown of the Reputational Component Peer Assessment Ranking 25.0%12th Peer Assessment Ranking 25.0%12th Avg. academic reputation score (survey of presidents, provosts, deans of admission) Avg. academic reputation score (survey of presidents, provosts, deans of admission) Student Selectivity 15.0%23rd Student Selectivity 15.0%23rd Avg. SAT/ACT scores of enrollees7.5% Avg. SAT/ACT scores of enrollees7.5% % of enrollees in top 10% of high school class6.0%15th % of enrollees in top 10% of high school class6.0%15th % of applicants who are admitted1.5%50th % of applicants who are admitted1.5%50th Weight inU-M Overall ScoreRank

10. 9 Breakdown of the Educational Success Component Graduation & Retention 25.0%26th Graduation & Retention 25.0%26th % of entering class still enrolled one year later4.0%21st % of entering class still enrolled one year later4.0%21st % of entering class who graduated in 6 years or less16.0%29th % of entering class who graduated in 6 years or less16.0%29th Graduation Rate Performance (Difference between a school’s actual and predicted 6-year graduation rate for an entering class)5.0% 33rd Graduation Rate Performance (Difference between a school’s actual and predicted 6-year graduation rate for an entering class)5.0% 33rd Weight inU-M Overall ScoreRank

11. 10 Breakdown of the Resources Component Faculty Resources 20.0 % 69th Faculty Resources 20.0 % 69th Avg. faculty salary + benefits adjusted for regional differences in cost of living7.0% Avg. faculty salary + benefits adjusted for regional differences in cost of living7.0% % of undergrad class sections fewer than 20 students6.0% 69th % of undergrad class sections fewer than 20 students6.0% 69th % of undergrad class sections 50 or more students2.0%103rd % of undergrad class sections 50 or more students2.0%103rd Student/faculty ratio1.0% 73rd Student/faculty ratio1.0% 73rd % of faculty with highest degree in their fields3.0% % of faculty with highest degree in their fields3.0% % of faculty who are full-time1.0% 83rd % of faculty who are full-time1.0% 83rd Financial resources10.0% 29th Financial resources10.0% 29th Expenditures per student Expenditures per student Alumni Giving5.0% 83rd Alumni Giving5.0% 83rd % of undergraduate alumni who donated money to their school % of undergraduate alumni who donated money to their school Weight inU-M Overall ScoreRank

12. 11 U-M’s Strengths and Weaknesses in the Rankings U-M scores well on measures of reputation and educational success U-M scores well on measures of reputation and educational success U-M’s scale causes it to rank less well on per capita resource measures U-M’s scale causes it to rank less well on per capita resource measures

13. 12 USN&WR Supplementary Information Lists of leading institutions based on other information not included in the rankings calculations Lists of leading institutions based on other information not included in the rankings calculations Quantitative measures Quantitative measures Racial diversity Racial diversity Economic diversity (U-M ranked 6th) Economic diversity (U-M ranked 6th) International Students International Students

14. 13 USN&WR Supplementary Information Qualitative information Qualitative information Internships Internships Senior Capstone Senior Capstone First-Year Experience* First-Year Experience* Undergraduate Research/Creative Projects* Undergraduate Research/Creative Projects* Learning Communities* Learning Communities* Study Abroad Study Abroad Service Learning* Service Learning* Writing in the Disciplines Writing in the Disciplines *U-M listed as a leading institution

15. 14 What affects the rankings from one year to the next? A change in the USN&WR methodology A change in the USN&WR methodology A real change in an institution’s data A real change in an institution’s data A real change in the data for other institutions A real change in the data for other institutions A change in how many institutions USN&WR chooses to publish on a given ranking A change in how many institutions USN&WR chooses to publish on a given ranking

16. 15 Issues/Concerns regarding USN&WR rankings Affect of rankings on students’ decisions Affect of rankings on students’ decisions Subjectiveness of the reputational component Subjectiveness of the reputational component Subjectiveness of the assigned weight for each component Subjectiveness of the assigned weight for each component Component items may not measure what they are assumed to Component items may not measure what they are assumed to Consistency and definition problems found in data collection Consistency and definition problems found in data collection Ability of institutions to manipulate their data Ability of institutions to manipulate their data Rankings measures are biased in favor of private institutions Rankings measures are biased in favor of private institutions

17. 16 Issues/Concerns regarding USN&WR rankings Rankings falsely imply a precision in the numbers and a significant difference between one rank and the next Rankings falsely imply a precision in the numbers and a significant difference between one rank and the next Rankings do not capture some of the most important components of a student’s college experience -- especially learning outcomes Rankings do not capture some of the most important components of a student’s college experience -- especially learning outcomes

18. 17 Alternatives to USN&WR Rankings Undergraduate Undergraduate Media sponsored rankings Media sponsored rankings Washington Monthly, Business Week, Kiplinger’s, Wall Street Journal Washington Monthly, Business Week, Kiplinger’s, Wall Street Journal Other Other Gourman Report, Princeton Review Gourman Report, Princeton Review Graduate Graduate Media sponsored rankings Media sponsored rankings Business Week, Wall Street Journal, etc. Business Week, Wall Street Journal, etc. Other Other NRC, Gourman Report NRC, Gourman Report For more examples of other rankings see: For more examples of other rankings see: http://www.library.uiuc.edu/edx/rankings.htm http://www.library.uiuc.edu/edx/rankings.htm Many of these share the same problems and weaknesses of USN&WR rankings--if not more! Many of these share the same problems and weaknesses of USN&WR rankings--if not more!

19. 18 Movements Away From Rankings Current protest by liberal arts colleges on the value of USN&WR rankings Current protest by liberal arts colleges on the value of USN&WR rankings 40 institutions have pledged not to fill out the USN&WR reputational survey and not to use the rankings in promotional efforts 40 institutions have pledged not to fill out the USN&WR reputational survey and not to use the rankings in promotional efforts The national “Accountability” movement The national “Accountability” movement Voluntary System Accountability (VSA) Voluntary System Accountability (VSA) NASULGC, AASCU NASULGC, AASCU Annapolis Group, NAICU, The Education Conservancy Annapolis Group, NAICU, The Education Conservancy

20. 19 How OBP Uses USN&WR Rankings In general not used to define our peers In general not used to define our peers “Official” peers (19 institutions) “Official” peers (19 institutions) Big Ten peers Big Ten peers AAU peers (61 institutions) AAU peers (61 institutions) Publish but do not endorse the rankings on our website Publish but do not endorse the rankings on our website (http://sitemaker.umich.edu/obpinfo/home ) (http://sitemaker.umich.edu/obpinfo/home )

21. 20 How OBP Uses USN&WR Rankings The rankings provide a quick source of selected comparative information on our “peers” The rankings provide a quick source of selected comparative information on our “peers” Mainly use on an ad hoc basis Mainly use on an ad hoc basis To identify other peers that may not be in our traditional peer groups (e.g., Notre Dame for undergraduate tuition comparisons) To identify other peers that may not be in our traditional peer groups (e.g., Notre Dame for undergraduate tuition comparisons) To narrow down our official peers (e.g., institutions with the most top ranked graduate programs in engineering and the sciences for the Advanced Project) To narrow down our official peers (e.g., institutions with the most top ranked graduate programs in engineering and the sciences for the Advanced Project)