1. Does Contact with Advisors Predict Advising Learning Outcomes? A Multi-Institutional Study (CODE: 186) National NACADA Conference October 5, 2012 Cathleen L. Smith, Ph.D. Professor Emerita of Psychology Portland State University smithc@pdx.edu Janine M. Allen, Ph.D. Professor Emerita of Education Portland State University allenj@pdx.edu

2. Acknowledgements Grant from the NACADA Foundation Support of members of our research collaborative and their institutional research offices

3. Agenda Describe the advising curriculum and what we would expect students to learn from advising encounters Present a study that examines advising learning of students from nine institutions Discuss implications of findings

4. Advising Learning: A New Emphasis New emphasis: What students should learn in academic advising encounters NACADA Concept of Academic Advising: –Advising is “integral to fulfilling the teaching and learning mission of higher education” –And, as such, has its own curriculum, pedagogy, and student learning outcomes (NACADA, 2006)

5. Learning-Centered Advising: State of the Literature More conceptual than empirical Focused on: Identifying the advising curriculum (e.g., Hemwall & Trachte, 2005; Lowenstein, 2005) and learning outcomes advising should produce in students Distinguishing between learning-centered advising and more traditional approaches (i.e., prescriptive and developmental advising)

6. Learning-Centered Advising: State of the Literature Focused on: Speculating about the mechanisms by which learning takes place in advising encounters Differentiating learning outcomes from other aspects of advising (e.g., student responsibilities) Advocating for the adoption and use of a learning-centered advising paradigm

7. Learning-Centered Advising: State of the Literature A logical next step in the evolution of this new advising paradigm is to gather empirical data on the learning outcomes that are thought to arise from participation in advising encounters

8. Deriving our Advising Learning Outcomes In formulating our learning outcomes, we began with our conception of quality academic advising as a multi-dimensional process encompassing five domains –Integration –Referral –Information –Individuation –Shared responsibility

9. Deriving our Advising Learning Outcomes Integration of the student’s academic, career, and life goals with each other and with aspects of the curriculum and co-curriculum Referral to campus resources for academic and non-academic problems Provision of information about degree requirements and how the university works with regard to policies and procedures

10. Deriving our Advising Learning Outcomes Individuation, or consideration of students’ individual characteristics, interests, and skills Shared responsibility, or encouraging students to assume responsibility for their education by providing them with opportunities to develop and practice planning, problem-solving, and decision- making skills

11. Advising Curriculum Advising Content: Integration, Referral, Information Advising Pedagogy: Individuation, Shared Responsibility

12. Advising Content: Information Our past research (Allen & Smith, 2008; Smith & Allen, 2006) has shown the primary importance to students of the information domain Thus it was represented by two learning outcomes

13. Advising Content: Information Advising assists students in understanding the multitude of requirements they face in order to successfully complete their program of study

14. Information: 1 st Learning Outcome 1. Knows Requirements Community college students: “I know what requirements (e.g., prerequisites, general education, transfer requirements) I must fulfill at name of community college in order to meet my educational goals” University students: “I know what requirements (e.g., major, general education, other university requirements) I must fulfill in order to earn my degree”

15. Advising Content: Information Advising helps students navigate their complex institution by assisting them in understanding how things work with regard to its timelines, policies and procedures

16. Information: 2 nd Learning Outcome 2. Understands How Things Work “I understand how things work at name of institution (timelines, policies, and procedures with regard to registration, financial aid, grading, graduation, petition and appeals, etc.)”

17. Advising Content: Referral Advising is a conduit through which the student becomes aware of resources at the institution that assist with –Academic problems (e.g., writing, test anxiety, tutoring) –Non-academic problems (e.g., child care, financial, physical and mental health)

18. Referral: Learning Outcome 3. Knows Resources “When I have a problem, I know where at name of institution I can go to get help”

19. Advising Content: Integration Advising promotes connected learning: –One of the primary goals of liberal education (Cronon, 1998) –Central to developmental advising –Considered by students as especially influential (Light, 2001)

20. Integration: Learning Outcome 4. Understands Connections “I understand how my academic choices at name of institution connect to my career and life goals”

21. Advising Learning Outcomes = Retention Predictors Advising may be implicated in retention Having a plan to achieve one’s educational goals Having a significant relationship with faculty or staff on campus

22. Retention-Related Learning Outcomes 5. Has Educational Plan “I have a plan to achieve my educational goals ” 6. Has Significant Relationship “I have had at least one relationship with a faculty or staff member at name of institution that has had a significant and positive influence on me”

23. Affective Learning Outcomes Advising may change students’ values We wanted to measure not only what students know and can do, but also what they might appreciate or value, as a result of participation in advising We wanted outcomes that might reflect that students who received quality academic advising benefited from it and thought others might too

24. Affective Learning Outcomes 7. Values Advisor/Advisee Relationship “It is important to develop an advisor-advisee relationship with someone on campus” 8. Supports Mandatory Advising “There should be mandatory academic advising for students”

25. What are the Advising Learning Outcomes Measuring? All eight learning outcomes are measures of students’ meta-cognition Meta-cognition: What students know about their own knowledge and values

26. Multi-Institutional Study: Nine study institutions in Oregon Institution Carnegie Classification Community Colleges Community College 1Associate’s/Public/Urban Serving/Multi-campus Community College 2Associate’s/Public/Rural Serving/Large Private Universities Private University 1Master’s (larger programs) Private University 2Master’s (larger programs) Public Universities Public University 1Research University (very high research activity) Public University 2Research University (very high research activity) Public University 3Research University (high research activity) Public University 4Master’s (medium programs) Public University 5Master’s (small programs)

27. Method Online administration of the Inventory of Academic Advising Functions – Student Version Administered in 2010 or 2011 Students invited to participate: –Universities: All fully admitted students –Community colleges: All students enrolled in credit-bearing classes

28. Method To ensure that all students in the study had similar educational goals –We selected students at the two community colleges who indicated that their main reason for attending the college was to earn credit toward a bachelor’s (4- year) degree

29. Research Sample Institution Number of Participants Participation Rate Community Colleges Community College 1601333.7 Community College 2115921.1 Private Universities Private University 1 43743.1 Private University 2 159952.5 Public Universities Public University 1402622.1 Public University 2366421.1 Public University 3274815.9 Public University 4149532.7 Public University 5122538.3 Total22,36626.1

30. Respondent Demographic Characteristics 64.5% Female 33.1% New Students (enrolled at their institution for the first time during the academic year in which the survey was administered) 76.6% White Mean age 25.3 years (SD 8.5 years)

31. Research Questions Does advising learning vary as a function of: Frequency of contact with advisors in the formal advising system? Source of information students use to choose required classes?

32. Research Question 1 Does advising learning vary as a function of frequency of contact with advisors in the formal advising system? Are scores on the 8 advising learning outcomes higher for students who have contacted advisors than for those who have not? Among students who have contacted advisors, are scores higher for those who have more contacts than for those with fewer encounters?

33. Research Question 1: Formation of Groups We grouped students based upon their responses to two survey items

34. Research Question 1: Formation of Groups 1 st Survey Item: “Which of the following describes where at name of institution you get your PRIMARY academic advising, that is, the advising you consider most central to your academic progress?” –Institutional representatives common to all institutions (e. g., “faculty advisor in my program of study”) –Advising offices unique to each institution (e. g., “advising center”) –No advising option “I have not received academic advising from faculty or staff at name of institution”

35. Research Question 1: Formation of Groups 2 nd Survey Item: “On average, how often do you get advice from your primary source of advising, that is, the advising you consider most central to your academic progress?” –At least once per term –At least twice per year –At least once per year –“I’m not currently getting academic advising from faculty or staff at name of institution.”

36. Research Question 1: Formation of Groups We assigned students to one of three groups: 1. Not advised (n = 3443) (Had not received or were not currently getting advising) 2. Advised occasionally (n = 3538) (Advised at least once per year) 3. Advised frequently (n = 14,886) (Advised at least twice per year or at least once per term)

37. Measures of Advising Learning Outcomes 8 advising learning outcomes, each measured by a 6 point Likert-type scale –1 = Strongly Disagree –6 = Strongly Agree

38. Research Question 1 To examine the relationship between frequency of contact (independent variable) and advising learning outcomes (dependent variables) We used ANCOVA, controlling for –Institution –Institution size –Enrollment status (new vs. continuing) –GPA

39. I know what requirements I must fulfill in order to meet my educational goals / earn my degree 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 20551) = 195.78, MSE = 1.35, p <.001, η 2 =.02 Results of post hoc analysis: All three groups significantly different from each other. Advised Frequently Advised Occasionally Not Advised Knows Requirements 5.09 a (1.11)4.97 b (1.19)4.56 c (1.44)

40. I understand how things work at name of institution 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 20544) = 118.61, MSE = 1.59, p <.001, η2 =.01 Results of post hoc analysis: All three groups significantly different from each other. Advised Frequently Advised Occasionally Not Advised Understands How Things Work 4.59 a (1.24)4.43 b (1.31)4.23 c (1.40)

41. When I have a problem, I know where at name of institution I can go to get help 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 20527) = 522.17, MSE = 1.95, p <.001, η2 =.05 Results of post hoc analysis: All three groups significantly different from each other. Advised Frequently Advised Occasionally Not Advised Knows Resources 4.52 a (1.36)4.02 b (1.46)3.69 c (1.59)

42. I understand how my academic choices at name of institution connect to my career and life goals 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 20510) = 189.60, MSE = 1.24, p <.001, η2 =.02 Results of post hoc analysis: All three groups significantly different from each other. Advised Frequently Advised Occasionally Not Advised Understands Connections 5.04 a (1.07)4.83 b (1.19)4.64 c (1.29)

43. I have a plan to achieve my educational goals 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 21418) = 120.75, MSE =.81, p <.001, η2 =.01 Results of post hoc analysis: All three groups significantly different from each other. Advised Frequently Advised Occasionally Not Advised Has Educational Plan 5.48 a (0.84)5.36 b (0.94)5.21 c (1.11)

44. I have had at least one relationship with a faculty or staff member at name of institution that has had a significant and positive influence on me 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 21442) = 303.02, MSE = 2.37, p <.001, η2 =.03 Results of post hoc analysis: All three groups significantly different from each other. Advised Frequently Advised Occasionally Not Advised Has Significant Relationship 4.51 a (1.51)4.12 b (1.67)3.79 c (1.76)

45. It is important to develop an advisor-advisee relationship with someone on campus 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 20585) = 594.13, MSE = 1.21, p <.001, η2 =.06 Results of post hoc analysis: All three groups significantly different from each other. Advised Frequently Advised Occasionally Not Advised Values Advisor/Advisee Relationship 5.20 a (1.01)4.79 b (1.19)4.42 c (1.39)

46. There should be mandatory academic advising for students 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 20564) = 215.81, MSE = 2.32, p <.001, η2 =.02 Results of post hoc analysis: All three groups significantly different from each other. Advised Frequently Advised Occasionally Not Advised Support Mandatory Advising 4.51 c (1.47)4.04 b (1.60)3.89 a (1.72)

47. Frequency of Contact: Summary of Results Advised FrequentlyAdvised OccasionallyNot Advised Knows Requirements5.09 a (1.11)4.97 b (1.19)4.56 c (1.44) Understands How Things Work 4.59 a (1.24)4.43 b (1.31)4.23 c (1.40) Knows Resources4.52 a (1.36)4.02 b (1.46)3.69 c (1.59) Understands Connections 5.04 a (1.07)4.83 b (1.19)4.64 c (1.29) Has Educational Plan5.48 a (0.84)5.36 b (0.94)5.21 c (1.11) Has Significant Relationship 4.51 a (1.51)4.12 b (1.67)3.79 c (1.76) Values Advisor/Advisee Relationship 5.20 a (1.01)4.79 b (1.19)4.42 c (1.39) Support Mandatory Advising 4.51 a (1.47)4.04 b (1.60)3.89 c (1.72)

48. Research Question 2 Does advising learning vary as a function of the source of information students use to choose required classes? Are scores on the 8 advising learning outcomes higher for students who have relied on advisors for help in choosing required classes than for students who have self-advised using official advising materials or advice from informal sources (friends/other students or family members)?

49. Research Question 2: Formation of Groups We grouped students based upon their responses to one survey item “Please select the circle that best describes where at name of institution you get most of your information about classes to take to meet degree requirements.” –institutional representatives and advising offices –institutional tools students might use to self-advise (“catalog,” “advising website,” “advising guide”) –members of the student’s informal social network (“friend(s)/other student(s),” “family member(s)”)

50. Research Question 2: Formation of Groups We assigned students to one of three groups: 1.Advisor (n = 12,957) (students who selected institutional representatives or advising offices.) 2. Advising tools (n = 7210) (students who selected institutional tools) 3.Informal social network (n = 1245) (students who selected “friend(s)/other student(s)” or “family member(s)”)

51. Research Question 2 To examine the relationship between source of information (independent variable) and advising learning outcomes (dependent variables) We used ANCOVA, controlling for –Institution –Institution size –Enrollment status (new vs. continuing) –GPA

52. I know what requirements I must fulfill in order to meet my educational goals / earn my degree 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 20307) = 184.43, MSE = 1.34, p <.001, η 2 =.02 Results of post hoc analysis: All three groups significantly different from each other. AdvisorAdvising ToolsInformal/Social Network Knows Requirements 5.09 a (1.11)4.92 b (1.25)4.39 c (1.46)

53. I understand how things work at name of institution 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 20299) = 139.47, MSE = 1.57, p <.001, η 2 =.01 Results of post hoc analysis: All three groups significantly different from each other. AdvisorAdvising ToolsInformal/Social Network Understands How Things Work 4.61 a (1.24)4.42 b (1.31)3.97 c (1.40)

54. When I have a problem, I know where at name of institution I can go to get help 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 20283) = 402.36, MSE = 1.95, p <.001, η 2 =.04 Results of post hoc analysis: All three groups significantly different from each other. AdvisorAdvising ToolsInformal/Social Network Knows Resources 4.54 a (1.36)4.03 b (1.49)3.62 c (1.52)

55. I understand how my academic choices at name of institution connect to my career and life goals 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 20267) = 188.62, MSE = 1.23, p <.001, η 2 =.02 Results of post hoc analysis: All three groups significantly different from each other. AdvisorAdvising ToolsInformal/Social Network Understands Connections 5.60 a (1.07)4.81 b (1.19)4.51 c (1.34)

56. I have a plan to achieve my educational goals 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 21157) = 75.77, MSE =.81, p <.001, η 2 =.01 Results of post hoc analysis: All three groups significantly different from each other. AdvisorAdvising ToolsInformal/Social Network Has Educational Plan 5.48 a (0.85)5.35 b (0.95)5.18 c (1.10)

57. I have had at least one relationship with a faculty or staff member at name of institution that has had a significant and positive influence on me 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 21179) = 169.09, MSE = 2.39, p <.001, η 2 =.02 Results of post hoc analysis: All three groups significantly different from each other. AdvisorAdvising ToolsInformal/Social Network Has Significant Relationship 4.51 a (1.52)4.10 b (1.67)3.92 c (1.72)

58. It is important to develop an advisor-advisee relationship with someone on campus 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 20338) = 329.09, MSE = 1.23, p <.001, η 2 =.03 Results of post hoc analysis: All three groups significantly different from each other. AdvisorAdvising ToolsInformal/Social Network Values Advisor/Advisee Relationship 5.20 a (1.02)4.73 c (1.26)4.85 b (1.21)

59. There should be mandatory academic advising for students 1=strongly disagree 6=strongly agree Results of ANCOVA: F(2, 20318) = 174.31, MSE = 2.31, p <.001, η 2 =.02 Results of post hoc analysis: All three groups significantly different from each other. AdvisorAdvising ToolsInformal/Social Network Support Mandatory Advising 4.52 a (1.47)4.04 c (1.63)4.22 b (1.57)

60. Source of Information: Summary of Results AdvisorAdvising ToolsInformal/Social Network Knows Requirements5.09 a (1.11)4.92 b (1.25)4.39 c (1.46) Understands How Things Work 4.61 a (1.24)4.42 b (1.31)3.97 c (1.40) Knows Resources4.54 a (1.36)4.03 b (1.49)3.62 c (1.52) Understands Connections 5.06 a (1.07)4.81 b (1.19)4.51 c (1.34) Has Educational Plan5.48 a (0.85)5.35 b (0.95)5.18 c (1.10) Has Significant Relationship 4.51 a (1.52)4.10 b (1.67)3.92 c (1.72) Values Advisor/Advisee Relationship 5.20 a (1.02)4.73 c (1.26)4.85 b (1.21) Support Mandatory Advising 4.52 a (1.47)4.04 c (1.63)4.22 b (1.57)

61. Summary of Findings Scores on all eight learning outcomes were significantly higher for students who had been advised than for those who had not Among students who had seen advisors, scores were significantly higher for those who had more contacts than for those who had fewer encounters Students who got most of their information about required classes from advisors scored significantly higher on all eight learning outcomes than those who self-advised using advising materials or who relied on advice from family or friends

62. Implications of Findings The more advising, the more learning Institutions need to ensure that all students have frequent contact with advisors

63. Implications of Findings The advising relationship matters It can be supplemented, but not supplanted, by web sites, advising guides, etc. Institutions need to ensure that students see advisors and have access to quality advising tools

64. Limitations of the Study Students were not randomly assigned to frequency of contact or source of information groups All measures were self-reported

65. Discussion Questions Comments

66. References Allen, J. M., & Smith, C. L. (2008). Faculty and student perspectives on advising: Implications for student dissatisfaction. Journal of College Student Development, 49, 609-624. Cronon, W. (1998). Only connect: The goals of a liberal education. The American Scholar, 67(4), 73-80. Hemwall, M. K., & Trachte, K. C. (2005). Academic advising as learning: 10 organizing principles. NACADA Journal, 25(2), 74-83. Light, R. J. (2001). Making the most of college: Students speak their minds. Cambridge, MA: Harvard University Press. Lowenstein, M. (2005). If advising is teaching, what do advisors teach? NACADA Journal, 25(2), 65-73. National Academic Advising Association. (2006). NACADA concept of academic advising. Retrieved from www.nacada.ksu.edu/Clearinghouse/AdvisingIssues/Concept-Advising.htm Smith, C. L., & Allen, J. M. (2006). Essential functions of academic advising: What students want and get. NACADA Journal, 26(1), 56-66.