Presentation is loading. Please wait.

Presentation is loading. Please wait.

Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 1 USING PERFORMANCE REVIEWS IN CAPSTONE DESIGN COURSES.

Similar presentations


Presentation on theme: "Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 1 USING PERFORMANCE REVIEWS IN CAPSTONE DESIGN COURSES."— Presentation transcript:

1 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 1 USING PERFORMANCE REVIEWS IN CAPSTONE DESIGN COURSES FOR DEVELOPMENT AND ASSESSMENT OF PROFESSIONAL SKILLS (Paper 1041, Session 3525) Greg Kremer Associate Professor and Chair, Mechanical Engineering (kremer@ohio.edu) David Burnette MSME, 2007 Ohio University, Athens, OH

2 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 2 OR ---------------------------------- Expanding ME program outcomes from "How-2-Do" engineering to "How-2-Be" a good engineer

3 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 3 HOLD PARAMOUNT…

4 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 4 HOLD PARAMOUNT… 1 ST DO NO HARM

5 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 5 ENGINEERS MAKE A WORLD OF DIFFERENCE WILL IT BE FOR GOOD OR BAD? SOCIAL CONTEXT

6 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 6 CASTL 2005-2006 Integrative Learning Project Overall Goal My true passion is to craft a capstone design experience that is truly an “education of the whole person” in order to encourage engineering students to appreciate the importance of integrity and wholeness in life. I want our graduates to see the profession of engineering as an opportunity to serve the world and support social justice, and I want them as individuals and as groups to “care as well as know”. My integrative learning project deals with integrating the non-technical but critically important "professional skills" into student conceptions of what it means to be a good engineer.

7 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 7 CASTL 2005-2006 Integrative Learning Project Initial research tasks 1. Develop a taxonomy of professional skills outcomes for engineers that are understandable to undergraduate engineering students and that are linked to the ABET professional skills outcomes. 2. Develop an authentic way to get undergraduate engineering students to value and develop professional skills. 3. Begin initial assessments of the effectiveness of purposeful identity development activities (integrated throughout the Mechanical Engineering program and organized around a team-based capstone design project) on student achievement of an integral engineering identity.

8 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 8 OU ME Program Educational Objectives Informed & aware of contemporary issues and the impact of engineering on society Skills to perform in the work environment (Professional Skills) Technical Skills INDUSTRIAL ADVISORY BOARD

9 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 9 Skills to perform in the work environment (Professional Skills) Safety and health Technical communication: Oral and written Teamwork Project management: planning, scheduling, budgeting Self evaluation, leading to improvement Awareness of international standards, quality standards, and systems of units Appreciation of engineering integration with business (market awareness, customer satisfaction, quality, continuous improvement, profit, mission / vision / core values,...) Ethical and effective decision making, involving environmental health and safety, economics, time, quality, performance / operability, and reliability / life. ABET PROFESSIONAL SKILLS

10 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 10 Student behavior (responding to what the instructor asks) Professional behavior (responding to open-ended problems, dealing with uncertainty, making “good” decisions based on informed engineering judgment) OU ME CAPSTONE DESIGN

11 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 11 Professional Skills – Prior Work Lewis and Bonollo (2002) Identified 5 professional skills that professional design clinic supervisors valued highly in students: 1. Negotiation with clients 2. Problem solving 3. Acceptance of responsibility for outcomes (professional behavior), 4. Interpersonal skills 5. Project management Shuman et al. (2005) Claim that “portfolios, along with performance appraisals and behavioral observations, offer the most comprehensive information for measuring many outcomes and are conducive to evaluating professional skills.”

12 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 12 Professional skills must be presented and developed in an authentic way ---------------- Add-on requirement vs Integral to engineering identity DECISION: ACTIVITY WITHIN CAPSTONE PROJECT VS ACTIVITY CONSTRUCTED FOR ASSESSMENT

13 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 13 Build and test a prototype Year-long Capstone Design Project Large (6-8) and diverse teams Select a concept and refine the design Develop feasible concepts Need statement based on contemporary issues Develop a production plan Develop target specs based on customer needs Identify a customer & market Create and maintain a project schedule OU ME CAPSTONE DESIGN ~50 STUDENTS/YEAR ~ENERGY EFFICIENT CAMPUS TRANSPORTATION, NISH

14 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 14 Build and test a prototype Year-long Capstone Design Project Large (6-8) and diverse teams Select a concept and refine the design Develop feasible concepts Need statement based on contemporary issues Develop a production plan Develop target specs based on customer needs Identify a customer & market Create and maintain a project schedule OU ME CAPSTONE DESIGN Professional Skills

15 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 15 Learning Needs

16 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 16 Build and test a prototype Year-long Capstone Design Project Large (6-8) and diverse teams Select a concept and refine the design Develop feasible concepts Need statement based on contemporary issues Develop a production plan Develop target specs based on customer needs Identify a customer & market Create and maintain a project schedule Performance Reviews with self and peer rating of professional skills OU ME CAPSTONE DESIGN Create list of Professional Skills Important to their team with explanations and examples and use it for peer ratings Participate in face-to-face performance reviews with “supervisor” to review accomplishments and set project and personal development goals

17 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 17 Attitude and character Doesn’t want to do any work & doesn’t care about others; always looks at the negative side of things. Motivated by external factors ($, boss) and personal gain. Only acts when prodded. Selfish motives, seeks personal gain. Sometimes is optimistic about doing work but would usually prefer to avoid it. Usually takes on responsibilities only when asked to. Expects others to make no mistakes. Usually is optimistic about doing work and takes on tasks willingly, but sometimes is stressed and wants to avoid extra work. Is not unrealistic in expectations of others. Often motivated internally and for the common good. Creates a foundation of honor. Thrilled to face a challenge & do work; always in a good mood and looks at the positive side of things. Tackles tasks wholeheartedly and works diligently to see the task completed fully. Allows others room to "not know" [Feito], and has a helpful rather than critical attitude. Acts selflessly for the benefit of others. I always bring positive energy to the group. An example is when we don't do as well as we expected on one of our presentations or reports, I am the one that will make a joke to lighten up the tension in the room. I usually make fun of Phil or someone. Levels of Development # (correlated with Bloom's affective levels ^), and what the characteristics look like in practice for the various levels No Evidence (Unrated) Ignorance (Receiving) 0 Awareness (Responding) 1 Importance (Valuing / Organizing) 2 Embodiment (Internalizing) 3 DECISION: ASSESSMENT RUBRIC VS TAXONOMY DECISION: STUDENT DEVELOPED VS FACULTY DEVELOPED DECISION: DEVELOPMENT LEVEL VS EXPECTED BEHAVIOR

18 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 18 PROFESSIONAL SKILLS RATING SHEET Developed by students in 2006-2007

19 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 19 PROFESSIONAL SKILLS RATING SHEET Developed by students in 2006-2007

20 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 20 PROFESSIONAL SKILLS RATING SHEET Developed by students in 2006-2007

21 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 21 PROFESSIONAL SKILLS RATING SHEET Developed by students in 2006-2007

22 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 22 PROFESSIONAL SKILLS RATING SHEET Developed by students in 2006-2007

23 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 23 Professional Skills Performance Review: PURPOSE IS DEVELOPMENT AND ASSESSMENT Adequate example / experience that demonstrates use of a range of professional skills Honest / valid rating and explanation of the example situation relative to the team / professional expectation Ability to identify influences or temptations that prevent professional-level performance Ability to design and implement a plan to improve individual performance in specific areas (to meet or exceed the expectations of the profession)

24 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 24 Assessment Results from 2005-2006 Class (Developmental Taxonomy, Development Project Defined in Winter and Assessed in Spring) Adequate example / experience that demonstrates use of the professional skill Low/Marginal: 9/48 Acceptable: 26/48 Above Expectations: 13/48 > 81% met outcome * Plan to bring individual performance up to an acceptable professional level > 75% met outcome** * Students gave good examples but tended to over-rate their performance - but truth emerges in face-to-face dialogue as part of the performance review **Senioritis prevented some from documenting the results from their development plans

25 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 25 Adequate example / experience that demonstrates use of the professional skill 100%* met outcome Plan to bring individual performance up to an acceptable professional level 100%* met outcome Started the process earlier and was more clear in describing why we were doing performance reviews and how the would impact grades Performance level approach was more effective than development level Being part of Carnegie Scholar project made it seem important to students * Treated as mastery outcome with recycling Assessment Results from 2006-2007 Class (Expected behavior Taxonomy, Development Project Defined in Fall, 1 st assessment in Winter, Follow-up Assessment in Spring)

26 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 26 Observations and Lessons Learned Context Matters: >95% self report awareness of social impact of engineering, also high for the importance of professional skills When asked to discuss and describe professional skills that are important for being a good engineer, students come up with impressive lists and examples  “They know professionalism when they see it”.  Student involvement in creating the list is essential Although students can identify areas for personal improvement and come up with improvement plans, most need reminders and assignments or else the plan will get “lost in the shuffle.”  Most students are not self-motivated for “self evaluation, leading to improvement”

27 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 27 Observations and Lessons Learned Although it takes a lot of time, face-to-face performance review meetings are very effective and allow a more authentic and professional discussion of goals, accomplishments, behaviors, etc.  Increases student engagement and accountability There are some advantages to using team consensus rather than class consensus or consensus of the profession for establishing the professional skills list  Universal consensus is difficult (especially with language and terms that are understandable by all), and most behavioral expectations and norms are set and evaluated locally  Most behavioral expectations can be integrated into the “team contract” to model “effective teamwork” practices

28 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 28 Thanks! Greg Kremer (kremer@ohio.edu)

29 Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 29 ABET and Professional Skills Outcomes A sense of professional and ethical responsibility An appreciation for the impact of engineering solutions in a global and societal context A knowledge of contemporary issues An awareness of the integral nature of learning and engineering practice (lifelong learning) An ability to function effectively on multi-disciplinary teams (including an understanding of the positive role of diversity and specialization) An ability to communicate effectively


Download ppt "Performance Reviews for Prof Skills, Greg Kremer, Ohio University, Mechanical Engineering, June 2008, 1 USING PERFORMANCE REVIEWS IN CAPSTONE DESIGN COURSES."

Similar presentations


Ads by Google