Overview  Portfolio Basics Portfolio purposes Learning objectives to be addressed Roles of student and faculty in portfolio development, implementation and use Institutional strategies for data use  A case study  Promise and Pitfalls  Lessons learned

What is purpose of the portfolio?  What outcomes are desired from portfolio use? Clear purpose Design of portfolio driven by its use and the knowledge and skills to be assessed  Growth model  Showcase  Hybrid

What are the MEASURABLE learning objectives?  Facilitates meaningful evaluation  Provides a common language  Use tools to facilitate the process  Involve your key constituents  Communicate with students

Program Learning Outcome: “Students will demonstrate effective oral communication skills”

Performance Criteria: Acceptable Standard of Performance Effective oral communication Personal appearance is appropriate Speaks clearly and with sufficient volume Achieves rapport with audience Uses engaging vocalization Responds effectively to questions and comments Uses audience-appropriate vocabulary, content, and style

Public Speaking Evaluation Sheet Student: __________________________________ Date: ________ Title of Presentation: _____________________________________ Evaluation Scale: Yes, a lot (  ) (  ) No, not at all CriteriaScoreTotals Presentation Style: 1. Personal appearance is appropriate _____ 2. Speaks clearly and with effective volume _____ 3. Achieves rapport with audience _____ 4. Uses engaging vocalization _____ 5. Responds effectively to questions and comments _____ 6. Uses audience-appropriate vocabulary, content, and style _____ Presentation style total:______ Content: 7. Uses the grammar of standard English _____ 8. Presentation includes introduction, body, and conclusion _____ 9. Organizes content logically and sequentially _____ 10. Presents ideas and arguments clearly and logically _____ 11. Uses appropriate audiovisual materials _____ 12. Cites sources appropriately _____

Student and Faculty roles in portfolio design and use  Be clear about how faculty are involved in the design and use of portfolio regardless of portfolio platform Meet their needs for information Sensitivity to workload Non-intrusive as possible  Involve students in meaningful ways Be clear about what’s in it for students Process should reinforce and be aligned with the educational process

Assessment  Development of scoring rubrics Linked to performance criteria Known to students Scales consistent with purpose of assessment (i.e., student/program)  Involve students in meaningful ways Be clear about what’s in it for students Process should reinforce and be aligned with the educational process

What strategies for data use? Know what you are going to do with the data  Who is going to review/rate portfolio contents?  Who is going to be responsible for analysis of data?  What mechanisms do you have in place to evaluate the data?  How do the results relate to your educational delivery strategies – curricular and co-curricular?

Close the loop ● Importance of feedback that provides meaningful information ● Target those who can do something about it ● Focus on decision-making Improvement Accountability Celebration ● Inform those who have been involved about how results have been used ● Communicate to stakeholders

AAHE Principles of Best Practice for Assessment of Student Learning  Use the “best practices” to guide development of assessment process  Recognition of local constraints INPUTS OUTCOMES

Rose-Hulman Institute Of Technology  Terra Haute, Indiana  undergraduate students  B.S. degrees in engineering, science, and mathematics  85%+ engineering students

Rose-Hulman’s Mission To provide students with the world’s best undergraduate education in engineering, science, and mathematics in an environment of individual attention and support.

Recruit highly qualified students, faculty, and staff Input Quality Climate Outcomes Resources Provide an excellent learning environment Encourage the realization and recognition of the full potential of all campus community members Instill in our graduates skills appropriate to their professions and life-long learning Provide resource management & development that supports the academic mission

Instill in our graduates skills appropriate to their professions and life-long learning  Recognition of ethical and professional responsibility  Understanding of contemporary issues  Recognition of the role of professionals in the global society  Understanding of diverse cultural and humanistic traditions  Ability to work effectively in teams  Ability to communicate effectively  Apply the skills and knowledge necessary for mathematical, scientific, and engineering practices  Ability to interpret graphical, numerical, and textual data  An ability to design and conduct experiments  An ability to design a product or process to satisfy a client’s needs subject to constraints Outcomes

Assessment of student material  Faculty work in teams  Each team assesses one learning objective  Score holistically  Emerging rubrics Does the reflective statement demonstrate or argue for the relevance of the submitted material to the criterion? Is the submitted material at a level expected of a student who will graduate from Rose-Hulman?

Example of Results 1. Reflection relevant to criterion? 2. Expected for R-HIT graduate?

Example of Results 1. Reflection relevant to criterion? 2. Expected for R-HIT graduate?

Example of Results Is the submitted material at a level expected of a student who will graduate from Ross-Hulman? 1.Appropriate for audience 2.Organization 3.Content factually correct 4.Test audience response 5.Grammatically correct

Linking results to Practice  Development of Curriculum Map  Linking curriculum content/pedagogy to knowledge, practice and demonstrating learning outcomes Show Me !

Curriculum Map results Fall (181 courses/labs) Communication Skills

Curriculum Map results Fall (181 courses/labs) Ethics

Closing the loop DEC JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV Eval committee receives and evaluate all data; makes report and refers recom- mendation to appropriate areas. Institute assessment center prepares reports for submission to Dept Heads of the collected data (e.g. surveys, portfolio ratings) Institute acts on the recommendations of the Eval Comm. Reports of actions taken by the institute and the targeted areas are returned to the Eval Comm. for iterative evaluation.

Primary focus  It is not about electronic portfolios.  It is about: teaching and learning faculty and student development support for the transformation of the teaching/learning environment Implement Develop Revise

Benefits to teaching  Faculty are asked to reflect on learning outcomes in relation to practice Consider the value of stated outcomes Right ones? Right performance criteria? Individual faculty ownership in creating the context for learning  Develop a common language and understanding of program/institutional outcomes  Promotes interdisciplinary discussions/ collaborations  Explicit accountability for learning outcomes (e.g., state mandates/accreditation)

Benefits to learning  Students review their own progress as it related to expected learning.  Portfolios provide a way for students to make learning visible and becomes the basis for conversations and other interactions among students and faculty.  Learning is viewed as an integrated activity not isolated courses.  Students learn to value the contributions of out-of-class experiences.  Student reflections are metacognitive as they appraise their own ways of knowing.  Promotes a sense of personal ownership over one’s accomplishments.

Students  Rewards  Relevance  Commitment

Lessons Learned  Assessment language – developing a discourse community  Allocate resources  Living, iterative process  Focus is on improvement and celebration  Start early  Prepare for the need for “in-service” for faculty and students  Decouple from faculty evaluation

DEMO Site AAHE/HLC