1. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Frederick S. Foster-Clark, Daniel F. ONeill, Laurie Hanich, and Carol Y. Phillips Millersville University of Pennsylvania Presented at AAC&Us Network for Academic Renewal conference, General Education and Assessment: Engaging Critical Questions, Fostering Critical Learning, Miami, FL, March 1, 2007

2. Dr. Fred Foster-ClarkGeneral Education Coordinator Dr. Linda McDowellFirst Year Experiences Coordinator Dr. Daniel ONeillCounseling and Human Development Dr. Laurie HanichEducational Foundations Dr. Thomas BurnsAssociate Provost for Academic Administration Dr. Carol PhillipsAssociate Provost Emerita Ms. Kris VitucciGraduate Assistant FYE Assessment Team Engagement and Retention: Assessment of a General Education Initiative for First-Year Students

3. Undergrad enrollment - 7200 –91% fulltime –14% minority –96% instate (Pennsylvania) Entering fall 2006 class of 1363 students (new freshman) – –Mean SATs = 1050; –Mean Percentile Rank = 69% 12th Ranked Public in US News & World Reports Masters Universities in the North Top Majors: –Business Administration (892) –Elementary Education (808) –Undecided (785) –Biology (503) –Psychology (449) –Industry & Technology (444) –Communications (438) Profile of Millersville University

4. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Started FYE in Fall 2001: Outgrowth of the Student Alcohol Abuse Task Force recommendations Develop potential models for a holistic first-year program to encourage: Social engagement Civic engagement Intellectual engagement Components to be included: Enhanced advisement Seminar course, the integrating element Living/learning community with related programming The Charge

5. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Components All freshman housing Special programming (WOW*) Peer mentors Team building experiences Community building Tutoring on-site *Whats on Wednesday Outcomes Students make friends, congregate in groups Students form study groups with classmates Students like the residence hall experience Students involved in campus life Collaboration with Student Affairs/Resident Life

6. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students UNIV 101 1-credit extended orientation Seminar with linked fundamentals course (i.e., ENGL 110 or COMM 100) Socratic format Problem-based learning Co-curricular/ extracurricular assignments Service-learning Faculty as advisor UNIV 179 3-credit, content-rich passion Seminar linked with fundamentals course General education credit Co-curricular/ extracurricular assignments Service-learning Faculty may or may not serve as advisor Seminar/Learning Community Models

7. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students –FY Perspectives course paired with first-year fundamental course (Composition or Speech) –Living-learning connection (Students live together in Freshman residence halls) –Service Learning (15 hours recommended) –Attentive advising by seminar instructors with support from Resident Life and Exploratory programs –Peer mentors (live in residence halls; one assigned to each seminar) Components of the First-Year Perspectives

8. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students First Year Perspectives (FYP) is a component of General Education specifically designed for first semester students and offered in a seminar format, typically linked to a foundations course (either ENGL 110 or COMM 100) as a part of a living/learning community. A major function of these seminars is to introduce a process of critical inquiry applied to important social, cultural, scientific, technological, and/or aesthetic problems. Each FYP course will introduce multiple perspectives related to the understanding and resolution of these problems. First Year Perspectives (UNIV 179)

9. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Common learning objectives: 1.Communicating orally and in writing 2.Obtaining and evaluating information (information literacy) 3.Engaging in critical inquiry 4.Appreciating the importance of civic engagement 5.Understanding the importance of a liberal arts education 6.Making a successful transition into university life, both academically and socially First Year Perspectives (UNIV 179)

10. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Dream of America Why We Hate Facing Fear Homes and Homelessness The Amish and the Media Witchcraft in 17th Century England and New England Our Bodies/Ourselves: Sexuality and Gender in the Global Village A Different View: How Can We Change the World? Scientific Revolutions: An Exploration of Method Culture, Science and Mathematics in the Pre-Columbian Americas Food or Free Speech? Human Rights and You The Monsters Under Our Beds Liberty and Justice for All: The Promise of American Education The Deindustrialization of America: Jobs Today, Gone Tomorrow First-Year Perspectives Topics (Fall 2006)

11. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students 110 of the 284 Exploratory (Undecided) students were randomly assigned to the pilot-test program Pilot-test students assigned to one of five learning community topics based on their stated preferences Students completed mid-semester and end-of-semester Web-based surveys Focus group of pilot-test students held at end of semester by assessment staff Instructors completed opened-ended surveys Students followed to check academic progress and persistence Fall 2005 Pilot-test Evaluation Methods

12. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Students in the freshman seminar had more serious conversations with different students, worked more with classmates outside of class, researched for a paper more, contributed more to class, and came to class more prepared than students who were not in a freshman seminar. Freshman Year Mid-Term Survey – Fall 2005 Results Students who were involved in a freshman seminar... Means (all differences p <.05) [Based on scale of 1 (Never) to 4 (Very Often)] Had more serious conversations with students who were very different in terms of their religious beliefs, political opinions, or personal values than students who were not involved in a freshman seminar. Seminar = 2.28 No Seminar= 2.02 Worked more with classmates outside of class to prepare class assignments than students who were not involved in a freshman seminar. Seminar = 2.59 No Seminar= 2.32 Worked more on a paper or project that required researching ideas or information from various sources than students who were not involved in a freshman seminar. Seminar = 2.71 No Seminar= 2.40 Asked more questions in class and contributed to class discussions than students who were not involved in a freshman seminar. Seminar = 2.87 No Seminar= 2.63 Came to class without completing readings or assignments less than students who were not involved in a freshman seminar. Seminar = 1.61 No Seminar= 1.83

13. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Pretest-Posttest Survey –Higher Education Values Inventory (HEVI) –Openness to Diversity & Challenge Pretest-Posttest Open-Ended Responses –Civic Responsibility –Liberal Arts Information Literacy Assessment (forthcoming) NSSE (End of spring term) End-of-Semester Focus Groups (students) Faculty Survey Persistence/Retention Fall 2006 Assessment Components

14. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Comparison of Pre-Test/Post-Test Means (Fall 2006) Note: Ns ranged from 212 to 215; standard deviations in parentheses. Ranges: HEVI Subscales: (0 = strongly disagree to 4 = strongly agree) Openness to Diversity/Challenge Scale: (1 = strongly disagree to 5 = strongly agree) ScalePre-TestPost-Testp value HEVI: Family Expectations 1.98 (.79) 1.98 (.86) ns HEVI: Scholastic Focus 2.03 (.74) 1.84 (.79) <.001 HEVI: Achievement Value 2.39 (.66) 2.37 (.65) ns HEVI: General Education Value 2.45 (.59) 2.41 (.71) ns HEVI: Achievement Obstacles 1.35 (.64) 1.49 (.78).002 Openness to Diversity/Challenge 3.80 (.56) 3.94 (.68).002

15. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Writing Prompts (Fall 2006) Civic Responsibility –What is civic responsibility? [pre- & post-] –How has your understanding of civic responsibility changed as a result of your experiences this semester? What in particular had an impact? [posttest only] Liberal Arts –What is a liberal arts education? [pre- & post-] –How has your understanding of a liberal arts education changed as a result of your experiences this semester? What in particular had an impact? [posttest only]

16. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Responses to Liberal Arts Writing Prompt (Pretest -- Fall 2006) Off Target (1)Middle category (2)On Target (3) Description The response is not clear with regard to conceptualizing or understanding liberal arts. The response indicates a more limited understanding of a liberal arts education. Responses focus on individual skills or dispositions that may be gained through a liberal arts education but fail to capture a larger conceptualization of the liberal arts. The response indicates a well developed conceptualization and understanding of liberal arts. Responses focus on the notion of a liberal arts education providing broad exposure to different disciplines and ways of knowing across math and science, social science, and humanities. Examples of responses Developing personal character Required for graduation Related to arts, music, performance Related to politics or political ideology I dont know Blank responses Other off target responses that fit the description above. Study of different cultures Exposure to different perspectives General education (or a basic education) Reading, writing, thinking Other middle responses that fit the description above. Exposure to different disciplines, courses, or subject areas Well rounded Other on target responses that fit the description above. Percentages (N) 32% (39) 35% (43) 33% (41)

17. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Responses to Civic Responsibility Writing Prompt (Pretest-Fall 2006) Off Target (1)Middle category (2)On Target (3) Description The response is not clear with regard to conceptualizing or understanding civic responsibility. The response indicates a developing conceptualization or understanding of civic responsibility yet is limited in scope. Responses focus on individual characteristics or interpersonal traits/skills, but dont extend to how collective members of society are affected or how the community is impacted. The response indicates a well developed conceptualization and understanding of civic responsibility. Responses focus on the notion of civic responsibility in a broad sense, by identifying how collective members of society are affected or how the community in which one resides is impacted. Examples of responses Doing well in school I dont know Other off target responses that fit the description above. Descriptive traits---respectful, kind, nice, etc. Personal/individual responsibilities Other middle responses that fit the description above. Being a good citizen Giving back to the community Helping others in need Adhering to laws or standards of the community Other on target responses that fit the description above. Percentages (N) 17% (20)33% (39)50% (58)

18. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Levels of Elaboration in Writing Prompts (Pretest -- Fall 2006) Level of Elaboration – Liberal Arts (Pretest) Shallow (1)Partial/Emerging (2)Deep (3) DescriptionResponse includes shallow ideas with limited and unelaborated details. Response is superficial. Response includes minimal ideas that are supported with vague or simplistic details. Response lacks insight and/or reflection. Response includes deep and complex ideas that are supported by rich details. Response includes evidence of reflection and/or insight. Percentages (N) 48% (59)41% (50)11% (14) Level of Elaboration – Civic Responsibility (Pretest) Shallow (1)Partial/Emerging (2)Deep (3) DescriptionResponse includes shallow ideas with limited and unelaborated details. Response is superficial. Response includes minimal ideas that are supported with vague or simplistic details. Response lacks insight and/or reflection. Response includes deep and complex ideas that are supported by rich details. Response includes evidence of reflection and/or insight. Percentages (N) 40% (47)42% (49)18% (21)

19. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Comparison of Pre-Test/Post-Test Means (Fall 2006) ScalePre-TestPost-Testp value Liberal Arts Content 2.03 (.80) 2.32 (.71) <.001 Liberal Arts Level of Elaboration 1.64 (.68) 1.83 (.63).016 Civic Responsibility Content 2.37 (.72) 2.45 (.74) ns Civic Responsibility Level of Elaboration 1.79 (.74) 1.86 (.74) ns Note: N = 108 for Liberal Arts coding; N = 106 for Liberal Arts coding; standard deviations in parentheses.

20. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Pretest and Posttest Responses – Conceptions of the Liberal Arts – Fall 2006

21. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Criteria for Information Literacy Competency The student… –Identifies a variety of types and formats of potential sources. –Summarizes the main ideas. –Synthesizes main ideas to construct new concepts. –Compares new knowledge with prior knowledge to determine the value added, contradictions, or other unique characteristics of information. –Applies new and prior information to the planning and creation of the essay. Being assessed by Library faculty using a sample of 41 student papers drawn from five different sections of UNIV 179.

22. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students The Seminar/Learning Community Really focused in depth on one area Service-learning really opened my eyes I got to pick the class Made lots of friends –living and studying together Professor knew everybody; small class great Peer mentors really helpful The Living Community Really liked living together Appreciated the fact that we were all freshmen going thru the same things Easier being with people in the same situation Great for working on group presentations and studying Peer mentors lived with us and helped us with classes, with adjustment and registration Liked the relational aspects the best Student Focus Group Findings 2006

23. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Faculty Reactions: Rewards Energizing--got me out of a teaching rut! I enjoyed a chance to spread my wings after a steady diet of required courses. Camaraderie with other faculty Got to know students wellboth academically and personally. Being both teacher and advisor made me better at both! I could convey to students that college involves a high level of intellectual rigor…students rose to my high expectations. I enjoyed seeing the students blossom. They came in quiet and reserved and by the end of the semester they were engaging in discussion and debate. Students were incredible…attendance was phenomenal, the students were engaged, they participated, and they asked questions routinely. It was like teaching seniors. The students knew each other very well, and they often worked on course material or discussed that material in the dorms.

24. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Faculty Reactions: Challenges Developing a brand new course involves a tremendous amount of work and preparation. Choosing the course topic well. The ideal topic has clear impact on students personal lives…and can be treated with academic integrity. Balancing course content and goals with attention to students general academic and personal progress Service Learning component is often the most difficult part of the course to administer. Identifying service opportunities, arranging transportation and other logistics etc. requires considerable advance planning and coordination with other campus offices. Students didnt get the seminar concept. At the beginning of the semester they were frustrated I wasnt lecturing. I need to better prepare them for what a seminar is.

25. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Persistence into Sophomore Year for Exploratory Students at Millersville – Fall 2001 to Fall 2005

26. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Starting small and building up Close collaboration between Academic and Student Affairs Faculty development Early and heavy use of assessment data –Learning how to do assessment better Getting innovations to fit existing curricular structures Conclusion: Lessons Learned

27. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Starting small and building up –Began with more traditional one-credit FYE as experimental course, evolved and developed itself, morphed into three-credit passion seminar. –New FYE programming started as small pilot-test with assessment opportunities, expanded in second and third years, with full adoption planned for Year 4. Conclusion: Lessons Learned

28. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Close collaboration between Academic and Student Affairs –Brings the living and learning together –Tutoring in dorms, study groups, and support programming benefits student learning and student socialization –Faculty more attuned to student learning and advisement needs –Maximizes the bang for the buck Conclusion: Lessons Learned

29. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Faculty development –Outside experts (e.g., Jodi Levine Laufgraben, Peggy Maki, John Gardner, Randy Swing, Steven Briggs, Ed Zlotkowski, Nancy King, Ed Napieralski, John OConnor, Doug Howard) brought to campus for consultation & training –Spring/summer workshop –Continued opportunities for discussion and faculty development during semester –End-of-term follow-up luncheon/workshop –Peer mentor training Conclusion: Lessons Learned

30. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Early and heavy use of assessment data –Use of assessment for program development –Use of assessment to justify expansion –Learning how to do assessment better Conclusion: Lessons Learned

31. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Getting innovations to fit existing curricular structures –Counting for credit in General Education –Faculty and department buy-in Conclusion: Lessons Learned

32. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Thanks for attending our presentation. We hope our work is helpful to you in your own program development and assessment efforts. Further information is available from our website: http://muweb.millersville.edu/~gened/ Contact information follows on the next slide.

33. Engagement and Retention: Assessment of a General Education Initiative for First-Year Students Frederick S. Foster-Clark, Department of Psychology and Coordinator of General Education Email: Frederick.Foster-Clark@millersville.eduFrederick.Foster-Clark@millersville.edu Phone: 717-872-3933 Daniel F. ONeill, Department of Counseling and Human Development Email: Daniell.ONeill@millersville.eduDaniell.ONeill@millersville.edu Phone: 717-872-3122 Laurie B. Hanich, Department of Educational Foundations Email: Laurie.Hanich@millersville.eduLaurie.Hanich@millersville.edu Phone: 717-871-2231 Linda L. McDowell, Department of Educational Foundations and Coordinator of First Year Experiences Email: Linda.McDowell@millersville.eduLinda.McDowell@millersville.edu Phone: 717-871-2388 Thomas D. Burns, Associate Provost for Academic Administration Email: Thomas.Burns@millersville.eduThomas.Burns@millersville.edu Phone: 717-872-3703 Carol Y. Phillips, Associate Provost Emerita Email: Carol.Phillips@millersville.eduCarol.Phillips@millersville.edu Phone: 717-872-3703 Go to http://muweb.millersville.edu/~gened/ for information about our Gen Ed program and FYE efforts.http://muweb.millersville.edu/~gened/ Contact Information