Presentation on theme: "“Strategies for Harnessing Information Technology to"— Presentation transcript:
1 “Strategies for Harnessing Information Technology to September 5, 20008th Improving Student Learning SymposiumImproving Student Learning Strategically“Strategies for HarnessingInformation Technology toFacilitate Institutional Assessment”Gloria M. Rogers, Ph.D.Institutional Research, Planning, and AssessmentRose-Hulman Institute of TechnologyTerre Haute, Indiana USAG. Rogers, Rose-Hulman Institute of Technology
2 OverviewUse of models to guide institutional strategies for improving student learningAssessing student learningBest practices for student assessmentBrief history of RHIT processAssessment model/taxonomyA case study - demonstrationBenefits to teaching/learningAssessment method truismsBarriers to faculty involvementAdvice from the field
3 Use of Principles of Best Practice for Assessment of Student Learning in guiding development of assessment “system”Value of using models to guide practiceRecognition of local constraintsOUTCOMESINPUTS
4 Rose-Hulman Institute of Technology September 5, 2000Rose-Hulman Institute of TechnologyTerre Haute, Indiana, USA1600+ undergraduate studentsB.S. degrees in engineering, science, and mathematicsMedian SAT scores 1350 (700M,650V)80%+ engineering studentsG. Rogers, Rose-Hulman Institute of Technology
5 BRIEF HistoryPresidential Commission of faculty, staff and students appointed in Spring of 1996 to develop a plan for the assessment of student outcomesProvide for continuous quality improvementMeet outcomes-based accreditation standardsRegional (NCA)Program (ABET)
6 Rose-Hulman Institute of Technology Gloria Rogers -Rose-Hulman Institute of TechnologyAccreditation RequirementsInstitutional MissionConstituentsEducational Goals &ObjectivesMeasurable Performance CriteriaEducational Practices/StrategiesFeedback for Continuous ImprovementProgram OutcomesAccreditationAssessment:Collection, Analysisof EvidenceEvaluation:Interpretation of EvidenceAssessment for Continuous Improvement
7 X X X Taxonomy of Approaches to Assessment Placement “Gatekeeping” BEHAVIORATIUDES&VLSKILKNOWLEDGCompetency-BasedInstructionAssessment-Based CurriculumIndividual Perf. TestsPlacementAdvanced Placement TestsVocational Preference TestsOther Diagnostic TestsXX“Gatekeeping”Admissions TestsRising Junior ExamsComprehensive ExamsCertification ExamsIndividualLevelofAssessment(Who?)XProgram EnhancementIndividual assessmentresults may be aggregated to serve program evaluation needsCampus and ProgramEvaluationProgram ReviewsRetention StudiesAlumni Studies“Value-added” StudiesGroupObject ofAssessment(What?)Learning/Teaching(Formative)Accountability(Summative)Purpose of Assessment (Why?)(Terenzini, JHE Nov/Dec 1989)
8 Rose-Hulman’s Mission To provide students with the world’s best undergraduate education in engineering, science, and mathematics in an environment of individual attention and support.
9 InputRecruit highly qualified students, faculty, and staffProvide an excellent learning environmentQualityEncourage the realization and recognition of the full potential of all campus community membersClimateInstill in our graduates skills appropriate to their professions and life-long learningOutcomesResourcesProvide resource management & development that supports the academic mission
10 Instill in our graduates skills appropriate to their professions and life-long learning OutcomesEthics and professional responsibilityUnderstanding of contemporary issuesRole of professionals in the global society and ability to understand diverse cultural and humanistic traditionsTeamworkCommunication skillsSkills and knowledge necessary for mathematical, scientific, and engineering practiceInterpret graphical, numerical, and textual dataDesign and conduct experimentsDesign a product or process to satisfy a client's needs subject to constraints
11 Why portfolios? Authentic assessment Capture a wide variety of student workInvolve students in their own assessmentProfessional development for faculty
12 Why “electronic” portfolios? Student-owned laptop computer program since 1995Classrooms, residence halls, common areas, library, fraternity houses all wiredAccessEfficientCost effectiveAsynchronous assessment
15 Assessment of student material Faculty work in teamsEach team assesses one learning objectiveScore holisticallyEmerging rubricsDoes the reflective statement indicate an understanding of the criterion?Does the reflective statement demonstrate or argue for the relevance of the submitted material to the criterion?Does the submitted material meet the requirements of the criterion at a level appropriate to a graduating senior at R-HIT?
17 Example of Results Understand criterion? Submission relevant to criterion?Meet standards for R-HIT graduate?
18 Example of Results Does submission meet the standards for a graduate of R-HIT? Appropriate for audienceOrganizationContent factually correctTest audience responseGrammatically correct
19 Linking results to Practice Development of Curriculum MapLinking curriculum content/pedagogy to knowledge, practice and demonstration of learning outcomesShow Me!
20 Curriculum Map Results Fall 1999-2000 (181 courses/labs) Communication Skills
21 Curriculum Map Results Fall 1999-2000 (181 courses/labs) Ethics
22 Institute acts on the recom-mendations of the Eval. Comm. Closing the loopJANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDECWINTERSPRINGSUMMERFALLEval Committee receives and evaluates all data; makes report and refers recom-mendations to appropriate areas.Institute acts on the recom-mendations of the Eval. Comm.Reports of actions taken by the Institute and the targeted areas are returned to the Eval Comm. for iterative evaluation.Institute assessment cmte. prepares reports for submission to Dept. Heads of the collected data (e.g. surveys, e-portfolio ratings).
23 Primary focus It is not about electronic portfolios. It is about: supporting teaching and learningfaculty and student developmentthe transformation of the teaching/learning environment
24 Benefits to teachingFaculty are asked to reflect on learning outcomes in relation to practiceConsider the value of stated outcomesRight ones?Right performance criteria?Individual faculty role in creating the context for learningDevelop a common language and understanding of program/institutional outcomesExplicit accountabilityPromotes interdisciplinary discussions/collaborations
25 Benefits to learningStudents review their own progress as it relates to expected learning.Portfolios provide a way for students to make learning visible and becomes the basis for conversations and other interactions among students and faculty.Learning is viewed as an integrated activity not isolated courses.Students learn to value the contributions of out-of-class experiences.Student reflections are metacognitive as they appraise their own ways of knowing.Promotes a sense of personal ownership over one’s accomplishments.
26 Assessment method truisms There will always be more than one way to measure any outcomeNo single method is good for measuring a wide variety of different student abilitiesConsistently inverse relationship between the quality of measurement methods and their expediencyImportance of pilot testing to see if method is good for your program (students & faculty)
27 Barriers to implementation Facultycurrent workloadlack of incentive to participate in the process (rewards)“what’s in it for me” (cost/benefits)Institutional/program leadershipLack of vision for the program/institutional assessment process (no existing, efficient models)Cost/benefit unknownDifficulty of restructuring the reward system to facilitate faculty participation
28 Portfolio deficiencies Process deficienciesLack of understanding of the dynamics of organizational changeAbsence of “tools” to facilitate collaborative workPortfolio deficienciesIll-defined purposeLack of efficient ways to manage the portfolio processSystematic review of portfolio contents is ill-defined or non-existentStudent and faculty roles not clearPortfolio process not integrated into the teaching/learning environmentResource deficienciesExpertise in portfolio developmentDevelopment of “authentic” portfolio
29 Advice from the field E=MC2 You cannot do it all - prioritize All assessment questions are not equalOne size does not fit allIt’s okay to ask directionsTake advantage of local resourcesDon’t wait until you have a “perfect” planDecouple from faculty evaluation