Presentation is loading. Please wait.

Presentation is loading. Please wait.

Culture of Assessment CREATING A CULTURE OF ASSESSMENT Dean Kenneth B. Roberts, Ph.D. Heidi M. Anderson, Ph.D. Eleanora Bird, M.S. Jeff Cain, M.S. Office.

Similar presentations


Presentation on theme: "Culture of Assessment CREATING A CULTURE OF ASSESSMENT Dean Kenneth B. Roberts, Ph.D. Heidi M. Anderson, Ph.D. Eleanora Bird, M.S. Jeff Cain, M.S. Office."— Presentation transcript:

1 Culture of Assessment CREATING A CULTURE OF ASSESSMENT Dean Kenneth B. Roberts, Ph.D. Heidi M. Anderson, Ph.D. Eleanora Bird, M.S. Jeff Cain, M.S. Office of Education Innovation University of Kentucky College of Pharmacy Closing the Loop

2 Culture of Assessment 2 Outline  Literature “Creating a Culture of Assessment”  UKCOP Case Study  Created an office to handle the College assessment efforts  Faculty development and involvement  Student involvement  Documenting current assessment efforts  Future Directions at UKCOP

3 Culture of Assessment Creating a Culture of Assessment 3

4 Culture of Assessment 4 1.Leadership must be ‘on board’  Mission, resources, use results for improvement 2.Involve all stakeholders  Faculty  Students  Alumni

5 Culture of Assessment 5 3.Make assessment relevant  Course reviews  Surveys, rubrics 4.Focus on teaching and learning rather than assessment  Focus conversations not on assessment, but on helping students learn

6 Culture of Assessment 6 5.Provide opportunities to learn about assessment 6.Minimize the burden of assessment on faculty and students  Provide centralized leadership, coordination and support for assessment  Provide resources  Keep paperwork to a minimal

7 Culture of Assessment 7 7.Start small, Set up clear expectations and be flexible 8.Celebrate and reward assessment efforts  Give faculty and staff written recognition  Encourage leaders to commend publicly  Provide special funding  Honor faculty and staff who obtain external grants that will strengthen assessment

8 Culture of Assessment 8 9.Use assessment results appropriately  Keep faculty, students and staff informed  Don’t penalize faculty whose assessments results are less than positive 10.Don’t expect everyone to get on board

9 Culture of Assessment (UKCOP) 9

10 Culture of Assessment 10 Vision  Office of Education Innovation (OEI) was developed to assist faculty  Teaching  Learning  ASSESSMENT  Technology  Scholarship of Teaching Today

11 Culture of Assessment 11 Mission  The mission of the “Office of Education Innovation” (OEI) is to provide assistance to faculty in developing and evaluating the professional pharmacy and graduate programs within the University of Kentucky, College of Pharmacy (UKCOP).

12 Culture of Assessment 12 Community  Dean’s Leadership Team  Office of Academic Affairs  Office of Education Innovation (4 FTEs)  Assistant Dean  Assessment Coordinator  Information Technology Officer  Administrative Assistant  Faculty  Students  Other Participants TEAM

13 Culture of Assessment 13 www.mc.uky.edu/Pharmacy/oei

14 Culture of Assessment 14 Who the office works with?  The “Office of Education Innovation ” (OEI) works closely with:  UKCOP Course Coordinators/Directors  UKCOP Individual Faculty  UKCOP Departments  UKCOP Students  External Constituents to College

15 Culture of Assessment 15 OEI Activities and Services  Curriculum  Assessment  Instructional Technology  Faculty Development (Enrichment)  Scholarship of Teaching Today

16 Culture of Assessment UKCOP Faculty Development: How, What and Why of Assessment? 16

17 Culture of Assessment 17 Why do assessment?  Accountability  ACPE Accreditation  University Review  Curriculum committee requirements  Enhance student’s educational development  Decision-making

18 Culture of Assessment 18 Accreditation  July 2000 accreditation standards require assessment activities to be aligned with mission and goals:  achievement indicators,  formative and summative formats,  outcomes assessment, and  systematic assessment plan.

19 Culture of Assessment 19 Assessment Process Driven by Mission Used for Improvement Ongoing Use Multiple Measures

20 Culture of Assessment 20 What is the Assessment Process?  “Systematic collection, review and use of information about educational programs undertaken for the purpose of improving student learning and development”  Palumbo and Banta, 1999, p. 3

21 Culture of Assessment 21 Levels of Assessment  Institutional  Academic  CURRICULUM  COURSES  Teaching- learning Methods  Assessment Methods  Individual  STUDENTS:  Learning  Performance  Attitudes  Behaviors  Skills  Satisfaction

22 Culture of Assessment 22 Levels of Assessment  Institutional  UNIV REVIEW  ACCREDITATION  Individual  Faculty  SELF  Teaching Portfolio  Peer  Observation  Scholarly Productivity

23 Culture of Assessment 23 What is Assessed?  Student  Student Learning Outcomes  Performance  Faculty  Contribution to teaching  Scholarly productivity  National visibility  Program  Graduation rates  Licensure pass/fail  Curriculum  Courses  Teaching-Learning Methods

24 Culture of Assessment 24 Difference: Student Assessment vs. Program Assessment  Student Assessment is a teaching process designed to improve learning  Program Assessment is an institutional process designed to improve the program

25 Culture of Assessment 25 Student Assessment  Carefully define your student learning outcomes  If you don’t know what you’re looking for, you will have a difficult time measuring it.  Outcome statements should describe student learning or behavior  Outcome: “the result”

26 Culture of Assessment 26 Student Assessment  Start simple. Choose a limited number of outcomes to measure at first.  Begin modestly so that the data collected can help to define and refresh accomplishments

27 Culture of Assessment Faculty Involvement 27

28 Culture of Assessment 28 Faculty Involvement  Faculty learning about assessment:  Website developed to keep them abreast of activities  Faculty participation on committees:  Assessment committee  Curriculum committee

29 Culture of Assessment 29 Faculty Involvement  Self-study reports  Curricular purposes  Curricular Outcomes  Course Review (online submission)  Course Evaluation (online)  Survey development  PS Graduate Program  Requested peer evaluation of teaching

30 Culture of Assessment 30 UKCOP General Outcomes  Foundation Knowledge  Critical Thinking  Information Management  Communications  Professionalism  Social Interaction, Citizenship, Leadership  Life-long Learning

31 Culture of Assessment 31 UKCOP Professional Outcomes  Patient Care  Medication Distribution  Human Resources Management and Operations  Research  Teaching

32 Culture of Assessment 32 Faculty Involvement  Focus on Teaching/Learning  Generate the appropriate assessment questions  Determine appropriate ways to measure outcomes  Decide on when to assess the students  Prepare and utilize the reports

33 Culture of Assessment 33 Faculty Involvement  Developing Questions to Ask:  What do we want students to know, and do?  How will we know they are learning what we hope they will?  What is the effect of our curricula, courses, teaching, styles and educational tools?  Did our changes bring about the effect we desired?

34 Culture of Assessment Student Involvement 34

35 Culture of Assessment 35 Student Involvement  Incoming orientation:  Explain the College’s assessment, rationale relating to accreditation standards, role of OEI  Review IRB protocol and informed consent  Assessments throughout their 4 years  Anonymous (code for tracking)  Reports in aggregate fashion

36 Culture of Assessment 36 Student Participation  Curriculum Committee  Student Liaison Committee (SLC)  meets 3x per semester  Assessment Committee  Course  Performance Assessments  Other  Accreditation Visit, Program Review

37 Culture of Assessment UKCOP Case Study: Documenting Current Effort 37

38 Culture of Assessment 38 Documenting Current Efforts Purpose  Determine what assessments were already taking place within the College  Develop an assessment inventory list and appropriate questions about student learning

39 Culture of Assessment 39  Self-assessment of outcome abilities  Attitudes towards peer/self assessment  Performance records from courses  CAPP results (integrated abilities)  Student portfolios  Yearly interviews  Post-graduation surveys Goal: Develop a Complete Student Profile

40 Culture of Assessment 40 Three Phases  Phase 1: Needs Analysis  Phase 2: Recommendations and Plan Development  Phase 3: Align Assessment Plan, Outcome Indicators

41 Culture of Assessment 41 Research  Literature review was conducted to determine types of assessment data that might be useful to gather for the program’s assessment functions  Student records  Program evaluations  Student performance and competencies Needs Analysis

42 Culture of Assessment 42 Data Audit  Class of 2002-2005: Student and educational outcomes records  Data obtained from various offices and files within the college:  Demographic  Entry-exam scores  Learning styles  GPAs Needs Analysis

43 Culture of Assessment 43 Data Audit  Data portfolios for several student cohorts were examined  Faculty  Other sources:  Institutional Research Office  Boards of Pharmacy  State/local pharmacy associations Needs Analysis

44 Culture of Assessment 44 Data Audit Report

45 Culture of Assessment 45  Success  External IR officer requested to use the college audit form as a model for the campus  Data are being collected from all audiences in a wide variety of venues  Some data are being disseminated in meaningful way Data Audit Needs Analysis

46 Culture of Assessment 46 Lessons Learned from Data Audit  Amount & breadth of information was considerable  Gaps/Duplication  Some areas require more or different data

47 Culture of Assessment 47 Lessons Learned from Data Audit  Length and frequency of surveys  Faculty engagement  Incorporate findings into College assessment plan  Reassess

48 Culture of Assessment 48 Phase 2 Recommendations about research questions to assessment committee  I-E-O data management model used to classify student and program data (Astin, 1993) I=inputs E=environments O=outcomes

49 Culture of Assessment 49 Phase 2 A. Content Analysis B. Educational Impact Which educational experiences/environments maximize student learning during the first year of pharmacy school?

50 Culture of Assessment 50 Phase 2 Ho: Structured Clinical Instructive Models (SCIMs), prior to students participating in EPPE, statistically improve student critical thinking skills. Ho: Videotaped counseling sessions statistically improve student communication skills during the first year of pharmacy school.

51 Culture of Assessment 51 PHASE 3  Align and map curricular outcomes  Disseminate reports and use for improvement

52 Culture of Assessment Future Assessment Directions 52

53 Culture of Assessment 53 Next Steps  OEI reviewing literature  Formulating appropriate research questions  Prioritization and approval of research questions by assessment committee  Identifying appropriate measures  Conducting the analysis  Reporting and using the results for improvement

54 Culture of Assessment 54 Assessing Faculty Teaching  Classroom teaching  Portfolio  Student advising  Scholarship of Teaching  Publications  Presentations  Supervision of clinical experiences  Supervision of graduate students  Consultation about teaching Next Step:

55 Culture of Assessment Other Activities of the Office of Education Innovation (OEI) 55

56 Culture of Assessment 56 Instructional Technology  Introduce and foster the use of innovative technologies that enhance teaching, learning and assessment  Consultation and chair side assistance  Remove obstacles

57 Culture of Assessment 57 Assessment Software  CourseEval  TrueOutcomes

58 Culture of Assessment 58 Technology  Smart classrooms  Blackboard  Audience Response System  PDAs  FLASH animations  Assessment hardware/software

59 Culture of Assessment 59 Faculty Enrichment  Education Research Focus Group (2002)  Seminars Seminars  Technology (primary focus)  Teaching (Forthcoming)  Learning (Forthcoming)  Assessment (Forthcoming)  Website: primer materials

60 Culture of Assessment 60 Scholarship of Teaching  Publications Publications  Presentations  AACP Annual Meeting poster, presentation  Other Presentations  Syllabus  International Education Conference  Assessment Institute

61 Culture of Assessment 61 Summary  Assessment evaluates the effectiveness of the program in terms of measurable outcomes  The plan consists of:  Student learning outcomes  Measures of student achievement of the outcomes (and objectives)  Methods of collecting data  Process for using and reporting the results  Assessment of the process

62 Culture of Assessment 62 Summary  Fostering a culture of assessment:  Takes time and commitment  Is challenging  Involves various stakeholders at some levels  Requires resources  Continuous

63 Culture of Assessment 63 http://www.mc.uky.edu/Pharmacy/oei/faculty_assessment.asp

64 Click to edit Master title style  Click to edit Master text styles  Second level  Third level  Fourth level  Fifth level 5/9/2015164 Culture of Assessment Questions ? 64


Download ppt "Culture of Assessment CREATING A CULTURE OF ASSESSMENT Dean Kenneth B. Roberts, Ph.D. Heidi M. Anderson, Ph.D. Eleanora Bird, M.S. Jeff Cain, M.S. Office."

Similar presentations


Ads by Google