1. Overview  Portfolio Basics Portfolio purposes Learning objectives to be addressed Roles of student and faculty in portfolio development, implementation and use Institutional strategies for data use  A case study  Promise and Pitfalls  Lessons learned

2. What is purpose of the portfolio?  What outcomes are desired from portfolio use? Clear purpose Design of portfolio driven by its use and the knowledge and skills to be assessed  Growth model  Showcase  Hybrid

3. What are the MEASURABLE learning objectives?  Facilitates meaningful evaluation  Provides a common language  Use tools to facilitate the process  Involve your key constituents  Communicate with students

4. Program Learning Outcome: “Students will demonstrate effective oral communication skills”

5. Performance Criteria: Acceptable Standard of Performance Effective oral communication Personal appearance is appropriate Speaks clearly and with sufficient volume Achieves rapport with audience Uses engaging vocalization Responds effectively to questions and comments Uses audience-appropriate vocabulary, content, and style

6. Public Speaking Evaluation Sheet Student: __________________________________ Date: ________ Title of Presentation: _____________________________________ Evaluation Scale: Yes, a lot (  ) 4 3 2 1 0 (  ) No, not at all CriteriaScoreTotals Presentation Style: 1. Personal appearance is appropriate _____ 2. Speaks clearly and with effective volume _____ 3. Achieves rapport with audience _____ 4. Uses engaging vocalization _____ 5. Responds effectively to questions and comments _____ 6. Uses audience-appropriate vocabulary, content, and style _____ Presentation style total:______ Content: 7. Uses the grammar of standard English _____ 8. Presentation includes introduction, body, and conclusion _____ 9. Organizes content logically and sequentially _____ 10. Presents ideas and arguments clearly and logically _____ 11. Uses appropriate audiovisual materials _____ 12. Cites sources appropriately _____

7. Student and Faculty roles in portfolio design and use  Be clear about how faculty are involved in the design and use of portfolio regardless of portfolio platform Meet their needs for information Sensitivity to workload Non-intrusive as possible  Involve students in meaningful ways Be clear about what’s in it for students Process should reinforce and be aligned with the educational process

8. Assessment  Development of scoring rubrics Linked to performance criteria Known to students Scales consistent with purpose of assessment (i.e., student/program)  Involve students in meaningful ways Be clear about what’s in it for students Process should reinforce and be aligned with the educational process

9. What strategies for data use? Know what you are going to do with the data  Who is going to review/rate portfolio contents?  Who is going to be responsible for analysis of data?  What mechanisms do you have in place to evaluate the data?  How do the results relate to your educational delivery strategies – curricular and co-curricular?

10. Close the loop ● Importance of feedback that provides meaningful information ● Target those who can do something about it ● Focus on decision-making Improvement Accountability Celebration ● Inform those who have been involved about how results have been used ● Communicate to stakeholders

11. AAHE Principles of Best Practice for Assessment of Student Learning  Use the “best practices” to guide development of assessment process  Recognition of local constraints INPUTS OUTCOMES

12. Rose-Hulman Institute Of Technology  Terra Haute, Indiana  1650+ undergraduate students  B.S. degrees in engineering, science, and mathematics  85%+ engineering students

13. Rose-Hulman’s Mission To provide students with the world’s best undergraduate education in engineering, science, and mathematics in an environment of individual attention and support.

14. Recruit highly qualified students, faculty, and staff Input Quality Climate Outcomes Resources Provide an excellent learning environment Encourage the realization and recognition of the full potential of all campus community members Instill in our graduates skills appropriate to their professions and life-long learning Provide resource management & development that supports the academic mission

15. Instill in our graduates skills appropriate to their professions and life-long learning  Recognition of ethical and professional responsibility  Understanding of contemporary issues  Recognition of the role of professionals in the global society  Understanding of diverse cultural and humanistic traditions  Ability to work effectively in teams  Ability to communicate effectively  Apply the skills and knowledge necessary for mathematical, scientific, and engineering practices  Ability to interpret graphical, numerical, and textual data  An ability to design and conduct experiments  An ability to design a product or process to satisfy a client’s needs subject to constraints Outcomes

16. Assessment of student material  Faculty work in teams  Each team assesses one learning objective  Score holistically  Emerging rubrics Does the reflective statement demonstrate or argue for the relevance of the submitted material to the criterion? Is the submitted material at a level expected of a student who will graduate from Rose-Hulman?

17. Example of Results 1. Reflection relevant to criterion? 2. Expected for R-HIT graduate?

18. Example of Results 1. Reflection relevant to criterion? 2. Expected for R-HIT graduate?

19. Example of Results Is the submitted material at a level expected of a student who will graduate from Ross-Hulman? 1.Appropriate for audience 2.Organization 3.Content factually correct 4.Test audience response 5.Grammatically correct

20. Linking results to Practice  Development of Curriculum Map  Linking curriculum content/pedagogy to knowledge, practice and demonstrating learning outcomes Show Me !

21. Curriculum Map results Fall 1999-2000 (181 courses/labs) Communication Skills

22. Curriculum Map results Fall 1999-2000 (181 courses/labs) Ethics

23. Closing the loop DEC JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV Eval committee receives and evaluate all data; makes report and refers recom- mendation to appropriate areas. Institute assessment center prepares reports for submission to Dept Heads of the collected data (e.g. surveys, portfolio ratings) Institute acts on the recommendations of the Eval Comm. Reports of actions taken by the institute and the targeted areas are returned to the Eval Comm. for iterative evaluation.

24. Primary focus  It is not about electronic portfolios.  It is about: teaching and learning faculty and student development support for the transformation of the teaching/learning environment Implement Develop Revise

25. Benefits to teaching  Faculty are asked to reflect on learning outcomes in relation to practice Consider the value of stated outcomes Right ones? Right performance criteria? Individual faculty ownership in creating the context for learning  Develop a common language and understanding of program/institutional outcomes  Promotes interdisciplinary discussions/ collaborations  Explicit accountability for learning outcomes (e.g., state mandates/accreditation)

26. Benefits to learning  Students review their own progress as it related to expected learning.  Portfolios provide a way for students to make learning visible and becomes the basis for conversations and other interactions among students and faculty.  Learning is viewed as an integrated activity not isolated courses.  Students learn to value the contributions of out-of-class experiences.  Student reflections are metacognitive as they appraise their own ways of knowing.  Promotes a sense of personal ownership over one’s accomplishments.

27. Students  Rewards  Relevance  Commitment

28. Lessons Learned  Assessment language – developing a discourse community  Allocate resources  Living, iterative process  Focus is on improvement and celebration  Start early  Prepare for the need for “in-service” for faculty and students  Decouple from faculty evaluation

29. DEMO Site http://www/rose-hulman.edu/ira/reps http://www.rose-hulman.edu/irpa AAHE/HLC gloria.rogers@rose-hulman.edu