Presentation on theme: "Maximizing Your Assessment by Combining Student Survey Results: EBI, NSSE, and CIRP Jillian Kinzie, Ph.D. Associate Director, Center for Postsecondary."— Presentation transcript:
1 Maximizing Your Assessment by Combining Student Survey Results: EBI, NSSE, and CIRP Jillian Kinzie, Ph.D.Associate Director, Center for Postsecondary Research and National Survey of Student Engagement (NSSE) InstituteJohn Pryor, M.A.Director, Cooperative Institutional Research Program, Higher Education Research InstituteDarlena Jones, Ph.D.Director of Research and Development, Educational Benchmarking
2 Session Objectives Introduction Overview of CIRP, NSSE, and EBI Data TriangulationUsing Studies to Measure…RetentionIntegration to CampusFirst-Year Program ImprovementLearning OutcomesUsing Multiple Data SourcesOther Data ConsiderationsSmall Group Discussion
4 CIRP: Cooperative Institutional Research Program What is CIRP?
5 IntroductionCIRP surveys have been used for decades to understand and illustrate the impact of college.Can use the surveys as cross-sectional surveys, but the key advantage of CIRP surveys is that they are longitudinal.
6 Astin’s I – E – O Model Environments YFCY/CSS Inputs Outcomes (e.g., place of residenceduring college, interactionswith peers and faculty, curricularand co-curricular experiences)InputsCIRP Freshman Survey(e.g., academic performancein high school, financial concernsprior to college entry,expectations for college,degree aspirations,self-concept in high school)OutcomesYFCY/CSS(e.g., satisfaction with college, retention, gains in college, post-college plans)
7 Higher Education Research Institute at UCLA Cooperative InstitutionalResearch ProgramFundedResearchFreshmanSurveyYFCYCSSFord FoundationTempleton FoundationNational Institutesof HealthFaculty Survey
8 CIRP Freshman Survey (TFS) Largest and longest-running national study of American college students…2008 is 43nd administrationInitiated in 1966 at the American Council on Education; Housed at HERI (UCLA) since 1973
9 CIRP Freshman Survey (TFS) Three types of input items:Pretests on possible outcomes measuresSelf-predictions about possible future outcomesPersonal characteristics that might effect the propensity to attain certain outcomes
10 1,708 colleges and universities 20071966206,865 respondents251CollegesandUniversities374, respondents536CollegesandUniversitiesTotal over 42 years:13,408,942 students1,708 colleges and universities
11 Your First College Year (YFCY) Launched in 2000208,157 students at 378 schoolsDesigned to assess academic and personal development over the first year of collegeDeveloped in collaboration with the Policy Center on the First Year of College
12 Your First College Year Survey (YFCY) Mission: Longitudinal assessment of student development and institutional/programmatic impact during the first year of collegeStudy Launch: Pilot Studies in Spring 2000 & 2001; National Assessment launched in Fall 2002Data Collected: Since its inception, surveyed nearly 150,000 first-year students at nearly 250 institutions nationwideAdministration: Disseminated at end of the first year in an on-campus administration format; paper and/or web versions; overall response rates 35-82%.
13 YFCY…Comprehensive in content to assess academic & social experiences as well as cognitive & affective developmentDesigned as a follow-up instrument:40% of items are direct post-tests to questions on the CIRP Freshman SurveySurvey ContentAcademic achievement & engagementLearning strategies and pedagogical practicesResidential & employment experiencesInteractions with family, peers, faculty & staffPatterns of behaviorStudent values and life goalsSatisfaction, self-concept, & feelings of personal successPlans for the next academic yearSpace for 20 questions of local relevanceJP changes
14 Standard Deliverables YFCY Deliverables and Data…Standard DeliverablesInstitutional ProfileLongitudinal ProfileElectronic Data FileReports on SpreadsheetAdditional Data ServicesInstitutional Data in PowerPoint Executive SummaryPeer Group ReportsData MergesAll institutions participating in the survey receive a standard package of deliverables that contains:The YFCY Institutional Profile is a report describing the results of YFCY for all respondents at your institution broken out by gender. Also included are comparative results for participating institutions of a similar type.The YFCY Means Profile reports your institution’s mean responses for items on the YFCY instrument. In addition, this report indicates statistically significant differences between the mean responses of students at your institution and the means of respondents in comparison groups.The YFCY Follow-Up Report is a comparison of students’ YFCY responses to their CIRP Freshman Survey responses (if available). This report also includes comparative longitudinal data from participating institutions of a similar type.A Factor Analysis of the YFCY National Aggregate Data, which clusters YFCY variables under broader topic headings, is intended to help institutions streamline their data for presentations of YFCY results to various campus constituents and to aid in institutional analyses of YFCY data, including more advanced multivariate analyses.In addition to the reports included in the YFCY package of deliverables, a number of special supplemental services are available:Electronic Data File; Reports on Spreadsheet; Peer Group Reports; Data Merges***Data file no longer automatic
15 Using YFCY Data to Enhance Campus Assessment Efforts PurposesSelf-study reports & strategic planningRetention studiesExamining group differences among studentsProgrammatic assessmentMeasuring student development & institutional impactFeedback to campus constituentsCreation of a student information systemMethodologiesDescriptive analyses with campus dataComparative analysesInstitutional subgroupsInstitutional vs. national dataAspirant group or consortiumMeasures of associationFactor analysesTrends analysesMultivariate analyses
16 NSSE: National Survey of Student Engagement Who is NSSE?
17 NSSE Stats…Launched in 2000, supported by institutional participation feesParent survey CSEQ (1979)National design team created surveyAdministered by Indiana University Center for Postsecondary ResearchMore than 1,300 different baccalaureate colleges/universities from 50 states, Puerto Rico, and Canada (CCSSE is for 2-year institutions)Consortia can add up to 20 questions (e.g., women’s colleges, HBCUs, state systems)Spring administration to first-year & senior studentsThird party, standardized administration; average 40% response rate, random sample, and select comparison groups.
18 NSSE PurposesProvide reliable indicators of “good educational practices”Support institutional improvement and accountability effortsFoster comparative & consortium activityRefocus conversations about quality in undergraduate educationDirect measure of frequency of participation in educational practices associated with learning and development.
19 Survey InstrumentCollege student survey that assesses the extent to which students engage in educational practices associated with high levels of learning and development
20 Two Components of Student Engagement What students do – time and energy devoted to educationally purposeful activitiesWhat institutions do – using effective educational practices to induce students to do the right things
21 NSSE ReportsComparative data (National, Carnegie Class, Select Comparison Groups, and/or Consortium)Respondent CharacteristicsFrequency DistributionsMean ComparisonsBenchmark ComparisonsMulti-year Benchmark ComparisonsPocket Guide Report (for prospective students)Executive Snapshot ReportData File [so you can link to other data!!]
22 BCSSE – Beginning College Survey of Student Engagement Based on requests for related pre-college measuresMeasures selected high school experiences and students interest in and expectations for participating in educationally purposeful activities during collegeDesigned to be combined with spring NSSE data for “pre- and post”- views of the first-year experience; Use to examine gap between expectations and engagement and study effect of students’ background on NSSE scoresCan also be used as stand alone assessmentPilots in 2005, 2006; Officially launched in 2007The Center for Survey Research at Indiana University helps NSSE administer the survey utilizing best practices in the field. See: for more information.NSSE design parameters:Relatively short surveyItems directly related to college outcomesAdministered to first-year and senior students at public and private 4-yr institutionsAdministered directly by a credible third-party survey organization
23 NSSE Possibilities Merging NSSE data with school records Descriptive displays of engagement patterns by any number of student characteristicsPrediction models for retention, degree attainment, grades, other outcomesTracking student engagement year to yearComparisons against aspirational, regional, and mission-related institutionsProgram assessmentAccreditation reportingConsortium and system data sharingScholarly research
24 EBI: Educational Benchmarking, Inc. Who is EBI?
25 EBI Stats…Founded in 1994 by Joseph Pica, Ed.D and Glenn Detrick (retired in 2002)Over 1500 Colleges and Universities (U.S.A. and 15 other countries like Australia, Mexico, Spain, and Egypt) have participated in EBI’s studiesSurveyed over 12 million peopleNearly 100 Assessments in 10 areas of higher educationOver 500 custom assessments for schools like MIT, Ohio State, UCLA, University of Georgia, and University of FloridaProduced over 17,000 customized reportsCreation of WESS
26 EBI’s Partners (alphabetical order) AACN (American Association of Collegiate Nursing)ACUHO-I (The Association of College and University Housing Officers International)ACUI (Association of College Unions International)AFA (Association of Fraternity Advisors)Ball State University –MHLI (Military Housing and Lodging Institute)OTC (Outside the Classroom)Policy Center on the First Year of College
27 Relevant Assessments… EBI First-Year Initiative (FYI)Mission: To assess the effectiveness of first-year courses / seminars.Survey Development: Created in conjunction with the Policy Center for the First Year of College. Initial funding of project by the Pew Charitable Trusts. Currently funded by institution participation.Study Stats: Pilot Study in Spring 2001; National Assessment launched in Fall colleges and universities have submitted over 185,000 responses.Why Participate: Provides institution information on course effectiveness. Provides instructors’ their class results for individual improvement.f
28 Relevant Assessments… ACUHO-I/EBI Resident AssessmentMission: To assist college and universities in assessing resident satisfaction of residence hall functions and the effectiveness of residence hall life.Survey Development: Created in conjunction with ACUHO-I. Housing professionals collaborated with EBI to design survey questions.Study Stats: National Assessment began Spring Since then, 567 colleges and universities worldwide have submitted over 2.6 million responses.Why Participate: Provides evidence of student satisfaction of their on-campus living experience and student learning outcomes linked to CAS Standards.
29 Relevant Assessments… MAP-WorksMission: An early warning indicator of retention and student successSurvey Development: Created in conjunction with Ball State University. Sherry Woosley from BSU collaborated with EBI to design survey questions.Study Stats: Study piloted in 2006 – full national study in Over 40 institutions are utilizing MAP-Works in 2008Why Participate: Provides reporting directly to each first-year student regarding their behaviors benchmarked against their first-year cohort. Provides information directly to advisors, hall directors, and FYE instructors regarding their students so early intervention can occur.
30 Relevant Assessments… FoE/EBI Foundation of Excellence Student and Faculty/Staff AssessmentsMission: Assist campus professionals in the self-study of their first-year program in support of the Foundations of ExcellenceSurvey Development: Created in conjunction with the Policy Center on the First Year of College. Randy Swing, John Gardner, and Betsy Barefoot collaborated with EBI to design survey questions.Study Stats: Studies piloted Summer National Assessment launched Fall Since 2005, 67 schools have participated in the 4 year study and 39 schools have participated in the 2 year study.
31 Assessment Features… General Features… Professional Survey Development: EBI’s survey development team working in partnership with professional organizationsData Collection: Paper surveys (where available) or onlineWESS: State-of-the art data collection, reporting, and distribution system. Programmed and maintained by EBI software engineers.View Reports Online: Provides visualizations of results and ability to download responses and/or calculated data for higher-level analysisShare Reports Online: Ability to allow access to online reporting to othersWritten Analysis: Analysis notebooks with full detailed descriptive analysis (factor and question means, frequency distributions, etc.) Analysis also available online in PDF format.
32 Assessment Features… Comparisons… Customization… Inter-Institutional Comparisons: Provide comparative information for high level decision-making and resource allocationIntra-Institutional Comparisons: The ability to code units (areas/hall/floors, chapters, or course sections) for internal comparison to support internal improvementLongitudinal Comparisons: Provide trend data to assess the impact of initiativesCustomization…Statistical Analysis: Provides statistical testing and information for resource allocationInstitutional Specific Questions: Institutions may add up to 10 questions (5 can be open-ended if using our online surveying system, WESS)
34 What Is Data Triangulation? Application and combination of several data points or sources to overcome weaknesses of single-measure studiesWhy Use Data Triangulation?Increase confidence in findings through convergence of different perspectivesThe point at which perspectives converge is seen to represent realityBuilds a rich data resource
35 CIRP Freshman, BCSSE, MAP-Works Astin’s I-E-O ModelEnvironmentsCSS, YFCY, EBI, NSSE(e.g., place of residence,interactions with peers and faculty, engagement, programmatic features,curricular & co-curricular experiences)InputsCIRP Freshman, BCSSE, MAP-Works(e.g., academic performancein high school, financial concernsprior to college entry,expectations for college,degree aspirations,self-concept in high school)OutcomesYFCY, CSS, EBI, NSSE(e.g., post-college aspirations, satisfaction with college, academic and social adjustment, degree completion rates)One of the most exciting possibilities for research utilizing the CIRP surveys is linking the Freshman Survey, YFCY, and CSS for a three time-point analyses of the student experience. Since YFCY is entering it’s fourth year now, the 2003 CSS represents the first opportunity to conduct this type of assessment on both the institutional and national data.When you consider the possibilities it is really quite exciting. Let’s take the example we discussed today--the impact of first-year seminars. A one-year follow-up is informative, but in some instances too short a timeline to measure the outcomes of such a curricular intervention. Perhaps this institutional effort during the first year has far reaching outcomes and implications for students’ trajectory through college.Using all of the CIRP surveys in this manner allows you to broaden the scope of your assessment to include experiences beyond the first year.
36 Similarities and Differences Comparing these instruments, we’ll be answering the questions...What are the unique features of these instruments?What are the similarities between these instruments?What are the differences between these instruments?How do these instruments complement each other?Next, how these instruments address…RetentionIntegrationLearning OutcomesFirst Year Program Improvement
38 NSSE and Retention… Links between NSSE & Retention Goals: NSSE is a stimulus for reflection on what the institution does well & areas for improvementFocuses attention on ultimate goal of persistence – improved educational experiences (not just retention)Identifies what distinguishes the institution, whereStudent experience matches espoused missionStudent experience falls shortNSSE results can be connected to institutional data to build persistence models
39 NSSE and Retention…NSSE items and Tinto’s Model of Student PersistenceStudents engagement in effective educational practices as indication of level of academic & social integration (1a-v; 6a-f; 9a-g etc.)Institutional environment items indicate the level of commitment to student success (10a-g)Student intentions to engage in educational activities before graduating (7a-h)
40 NSSE and Retention… Retention Indicators Retention Studies Monitor satisfactionExamine intent to engage in enriching educational experiencesIdentify problematic student behaviors (e.g., % preparing for class; % preparing 2 or more drafts; frequency of contact with faculty outside of class)Retention StudiesStudy relationship between engagement and first-year retentionCompare persisters vs. non-persistersStudy program impact (e.g., first-year learning communities; course-based service experience)
42 CIRP and Retention…CIRP Freshman Survey asks students to predict the likelihood that they will transfer, stop-out, or leaveYFCY asks if they have transferred from a different institution; CIRP TFS pre-test data will be merged with the YFCY responses even if the pre-test was administered at a different institutionYFCY asks students to share their plans for their sophomore year, i.e., intent to re-enrollData can be merged with information from the registrar to compare predicted vs. actual retention rates
43 CIRP and Retention…One of the biggest problems with studying retention is that it is inextricably linked with who you admit (pre-college characteristics and performance)Longitudinal assessment (e.g. CIRP Freshman Survey and YFCY) allows institutions to control potentially-biasing background characteristics when assessing first-to-second year retention rates“Many of the so-called highly productive institutions turn out to be underproductive when the caliber of their entering students is taken into account.” Alexander Astin, 2003
44 CIRP and Retention…Consideration of “inputs” and “environments” in retention analyses identifies populations of at-risk students and programs/experiences that facilitate or inhibit persistenceEx. What entering characteristics positively predict first-to-second year enrollment? Emotional health is a positive predictor, but recent cohorts of students are exhibiting more stress & greater levels of depressionEx. How do service-learning, first-year seminars, and learning communities affect the decision to re-enroll? Service learning is a positive predictor on its own; First-year seminars and learning communities are effective when taken in combination
45 Enrollment Management/Retention EBI and Retention…Enrollment Management/RetentionAcademic AffairsStudent AffairsWho is responsible for student success on your campus?Enrollment Management/ Retention?Student Affairs?Academic Affairs?What information do you know about this first-year student?Student ID: YD HS GPA: 3.93 SAT Verbal: 29 Location: In state Gender: Female Race: African American Age: 18 Major: UndecidedDo you really know them?
46 Paradigm Shift EBI and Retention… What would happen if… I’m struggling in my math classI’m thinking about transferringParadigm ShiftEnrollment Management / RetentionStudent AffairsAcademic AffairsWhat would happen if…ALL faculty/staff were responsible for student success?YOU knew student was struggling?Could you do something about it before it was too late?Residence Hall StaffFirst-Year Seminar InstructorAcademic AdvisorI’m really homesickFinancial AidMinority Student AffairsAcademic Department HeadsI don’t think I can afford collegeMy roommate and I argue all the time
47 MAP-Works Process Expectations Behaviors Student Profile Institution ProfileCampus ResourcesSocial NormingExpectationsCampusResourcesStudent SummaryScan Students
49 NSSE and Engagement…What is the level of student engagement on your campus?NSSE data demonstrates what students do and what students believe the institution emphasizesResults can be used to shape new student behaviorCriterion reference benchmarking to compare against predetermined value (e.g., 20% FY students worked with peers on assignments outside of class – should this be higher??)
50 CIRP and Student Involvement Astin’s Involvement TheoryThe amount of student learning and development associated with any educational program is directly proportional to the quantity and quality of student involvement in that programThe effectiveness of any educational policy or practice is directly related to the capacity of that policy or practice to increase student involvement
51 CIRP and Academic Involvement TFSYFCYStudied > 6 hours/week49%68%Look up scientific research articles and resources23%29%Revise your papers to improve your writing54%53%Seek feedback on your academic work42%Ask questions in class59%34%Been bored in class38%37%Asked professor (teacher) for advice after class28%15%“A” average21%
52 CIRP and Co-Curricular Involvement TFSYFCYPerformed volunteer work89%63%Attended religious services79%53%Socialized with Friends >6 hrs/week70%Student Clubs or Groups > 3 hrs/week40%31%Online Social Network Sites > 3 hours/week43%56%
53 EBI and Interaction… First-Year Initiative Assessment Course Improved Connections with FacultyCourse Improved Connections with PeersSense of Belonging and AcceptanceAFA/EBI Fraternity/Sorority Assessment“Fraternity/Sorority Enhanced Interpersonal Relationship Skills”Top predictor of Overall Program Effectiveness
54 EBI and Interaction…FoE/EBI Foundations of Excellence Student AssessmentTop two predictors of Overall Evaluation of Program:Overall Evaluation: Transition SupportAll Students: Campus EnvironmentACUHO-I/EBI Resident Assessment“Interaction with Others in the Hall” is a top predictor of Overall Evaluation of Program for 90% of schools
55 EBI and Interaction… MAP-Works – all about Interaction… Levels of Personal One-on-One InteractionStudent to Faculty/Staff (like FYE Instructor / Advisor / Hall Director)Faculty/Staff interactions with other Faculty/StaffUpper Administration interaction with Faculty/StaffUpper Administration interaction with StudentAggregate Study of Student InteractionThe factor, “Social Integration” is top predictor of a successful social transition to college
57 NSSE and FYP Improvement… Identify strengths and areas for growthEx. FY students frequently engage in active learning (+); but are involved in enriching educational experiences at low levels (-)Assess effectiveness of FY interventionsEx. Low student faculty interaction scores at small college prompts institution to establish mentoring program, and involve students in undergraduate research; 2 years later NSSE results rise on this benchmark
58 NSSE and FYP Improvement… Sewanee: University of the South: Concern about FY students’ low active and collaborative learning results led Sewanee faculty to rejuvenate first-year seminar course, increase active pedagogies & collaborate with student affairs to create rich co-curricular elements.Worcester Polytechnic Institute: NSSE results showed FY students less engaged than seniors. Created new FY interdisciplinary, inquiry-based seminars; better integration of disciplines; engaging introductory courses. Associate Dean appointed to Office for the First Year. Assessment plan for the FY in development with NSSE indicators as key component.UW Green Bay: Used NSSE items in Freshman Seminar Pilot Study to test impact of revised seminar.Sewanee’s NSSE results are plenty to cheer about. At the same time, it says volumes about the character of the institution that the former College of Arts and Sciences dean, Tom Kazee, used the NSSE report to not only celebrate the institution’s achievements, but also to draw attention to one area in particular where Sewanee’s performance was sub-par – Active and Collaborative Learning. Kazee stated, "Our students are telling us that our classes may often have an active learning focus -- but that in this area, at least, other schools are doing more…it is something we'd like to address" (Sewanee Magazine, Winter, 2002). The result was the creation of a new, innovative First Year Program (FYP) that addresses various elements of effective educational practice but emphasizes “active learning.” We discuss the FYP later in the active and collaborative learning section.
59 CIRP and FYP Improvement… Several common first-year programs are included on the surveyFirst-year seminarsService learningStudent-centered pedagogiesSpecial Interest HousingTutoringRemedial educationDiversity/multicultural experiencesStudent-faculty research partnershipsMeasures satisfaction with numerous campus services and facilities
60 CIRP and FYP Improvement… Using YFCY Data to Improve ProgramsIdentify patterns of participation in specific programsMeasure levels of satisfaction with services/programsConduct sub-group analyses of involvement in programs to examine differential involvementEx. First-year seminars and involvement with facultyIdentify the impact of programs and services on key outcomes of the first year individually and in combination
61 EBI and FYP Improvement… First-Year Initiative Assessment - Mission of study is the improvement of the first-year course/seminar. Pertinent factors include “Course Improved…”Study StrategiesAcademic and Cognitive SkillsCritical ThinkingConnections with FacultyConnections with PeersOut-of-Class EngagementKnowledge of Campus PoliciesKnowledge of Academic ServicesManaging Time and PrioritiesKnowledge of Wellness
62 EBI and FYP Improvement… ACUHO-I/EBI Resident AssessmentAFA/EBI Fraternity/Sorority AssessmentACUI/EBI College Union Assessment and Student Activities AssessmentAll studies contain class standing as a categorical question. Institutions can filter results to first-year students to understand the impact of their programsAll assessments’ mission is program improvement by understanding students’ perceptions of their program
63 EBI and FYP Improvement… Foundations of Excellence Student and Faculty/Staff AssessmentsBoth studies measure the institutions’ delivery of the first year of college from the first-year students’ perceptions and the faculty/staff’s perceptionsAssessments are integrated into a full self-study of the institution’s first year of college overseen by the Policy Center.
64 EBI and FYP Improvement… MAP-Works!Study measures first-year students’ expectations of their first-year and their perceptions of their environment during that initial transitionOnline reports are sent to each student and their advisor, hall director, and/or first-year course instructor.Students’ reports focus on the top areas of transition (i.e. study time, self-management, making friends, residence hall transition, etc.)Faculty/staff’s reports focus on areas of risk self-reported by student (i.e. no support system, few expected study hours, expected graduation, etc.)Institution’s reports focus on areas of low performance by large segments of population to support improvement of first-year programs.
66 Self-Reported Outcomes… Can be trusted so long as…Questions are clear, with singular meanings and interpretations.The information requested is known to respondents.Respondents believe the questions merit a thoughtful response.Response options are complete and appropriate.Questions avoid risk to the respondent, and do not encourage the respondent to respond in socially desirable ways.
67 NSSE and Learning Outcomes… Provides student learning process and outcomes indicatorsTo what extent are FY students developing their writing skills?% FY writing papers > 5 pgs.% FY students who prepare 2 or more drafts% FY reporting substantial gains in writing effectively% FY worked on paper requiring integrating ideas, sources
68 NSSE & Educational and Personal Gains (% “very much” or “quite a bit”) NSSE and Learning Outcomes…NSSE & Educational and Personal Gains (% “very much” or “quite a bit”)Self-Reported Educational and Personal Gains from CollegeFirst-YearStudentsSeniorsThinking critically and analytically81%87%Acquiring a broad general education82%86%Working effectively with others66%78%Writing clearly and effectively72%77%Learning effectively on your own70%Using computing and information technology65%76%Acquiring job or work-related knowledge and skills57%Speaking clearly and effectively60%Understanding yourselfAnalyzing quantitative problems55%Solving complex real-world problems49%58%Understanding people of other racial/ethnic backgrounds50%52%Voting in local, state, or national elections24%23%
69 CIRP and Learning Outcomes… Using TFS & YFCY: Academic & Cognitive IndicatorsGrade point averageGoes down by ½ point during first yearLongitudinal change in self-ratings of skillsWriting ability (+8%)Mathematical ability (-3%)Academic ability (-5%)Self-perceived changesGeneral knowledge (21% report much stronger)Critical thinking skills (19% report much stronger)Analytical/problem-solving skills (17% report much stronger
70 CIRP and Learning Outcomes… Using TFS YFCY: Other IndicatorsIdentity development+4% in personal rating of self-understandingInterpersonal skills+13% report their ability to work as part of a teamCivic engagement+8% in goal to keep up to date with political affairsAwareness and acceptance of diversity and multiculturalism+3 in goal to improve understanding of other countries/cultures
71 EBI and Learning Outcomes… Academic Assessments are linked to professional learning outcomes standards…Undergraduate Business Exit Assessment (AACSB Standards)Nursing Exit Assessment (AACN Standards)Teacher Exit Assessment (NCATE Standards)Engineering Exit Assessment (ABET Standards)Student Affairs studies are linked to professional learning outcomes standards, for instance…ACUHO-I/EBI Resident Assessment (ACUHO-I, CAS, and NASPA/ACPA Learning Reconsidered)AFA/EBI Fraternity/Sorority Assessment (AFA, CAS, NASPA/ACPA Learning and Leadership Reconsidered)ACUI/EBI College Union Assessment (ACUI and CAS Standards)ACUI/EBI Student Activities Assessment (Learning Reconsidered, Leadership Reconsidered, CAS Standards)
73 Linking Data… In-house surveys National surveys (a growing list) CIRPTFS/ YFCY / CSSNSSECSEQ / CSXQEBI Benchmarking surveysNoel Levitz Student Satisfaction InventoryETS Major Field TestsACT Collegiate Assessment of Academic ProficiencyInstitutional data: GPA, financial aid, transcripts, retention, certification tests, etc.JP change
74 NSSE, EBI, and Engagement… Washington State University study: What is the relationship between residence hall environment and student engagement? Is there a difference by gender? (Robert Tattershall, Director, Housing and Conference Services, Washington State University)Used EBI data to characterize those residence hall floors with high and low interactionIdentified correlations between NSSE’s engagement measures and the residence hall interaction levelsFound gender differences
75 NSSE, CIRP, and Retention… Disaggregated NSSE results from seniors by those who started at the institution as first-year students and those who entered as transfer studentsBetter understand the transfer student experience.Combines with CIRP to develop a fuller portrait of the transfer student experience.
76 CIRP, NSSE, and FYP Improvement at Elon… Elon UniversityHigh institutional ethos of assessmentBuilding a student databaseValue-added assessment methodology guides studiesBoth CIRP/YFCY and NSSE…Lead to additional researchAre utilized for longitudinal study & benchmarkingProvide valuable measures of cognitive skillsAssess student interaction with facultyHelp to understand overall educational experience
77 CIRP, NSSE, and FYP Improvement at Elon… Key uses of YFCYTime allocationBenchmark for issues of campus communityCognitive and skill changesMeasuring use and satisfaction with campus programs and servicesFacilities evaluation (classrooms, labs, library)Social activities and behaviorsPrimarily used by Student Life &, more recently, General Education Office
78 CIRP, NSSE, and FYP Improvement at Elon… Key uses of NSSEActivities inside the classroom (e.g., speaking up, presentations, working with students)Activities outside the classroom (e.g., plays, museums, events)Evaluation of various functions (e.g., administrators, academic advising)Academic rigorPrimarily used by Student Life & Academic Affairs
79 Survey Administration Cycles… Some Examples…Administer all surveys to all students and merge them together with campus data to create an extensive student information systemNSSE every 3 years, alternate CIRP, EBI institutional surveysAdminister NSSE and EBI two years in a row to establish baseline data and assess needs for first-year learning communitiesAdminister NSSE to a random sample of first-year students and YFCY to those remainingAdminister CIRP TFS and YFCY and/or BCSSE and NSSE for program participation patterns and outcomes; supplement more detailed programmatic information from EBI
81 Contact Us… Cooperative Institutional Research Program (CIRP) Higher Education Research Institute at UCLANational Survey of Student Engagement (NSSE)Center for Postsecondary Research, Indiana UniversityEducational Benchmarking (EBI)EBI Service Center, Springfield, MO