1. 6/12/2016 1 Introduction to Assessment of Student Learning Kathleen M. Morley, Ph.D. University Director of Assessment

2. 6/12/2016 2 Overview I. Assessing Learning II. Goals & Objectives III. Measures IV. Rubrics

3. 6/12/2016 3 I. Assessing Learning

4. 6/12/2016 4 Why Assess Student Learning? “Natural human, scholarly act” – Barbara Walvoord, 2009 Identify student strengths and weaknesses National movement that is not going away

5. 6/12/2016 5 What is Assessment? “Effective assessment is best understood as a strategy for understanding, confirming, and improving student learning” – Higher Learning Commission

6. 6/12/2016 6 Long Island University Mission Campus Mission Academic Program Goals Program Effectiveness (e.g., admissions, curriculum, satisfaction) Student Learning (e.g., student knowledge, skills, and attitudes) ASSESSMENT FRAMEWORK School/College Mission

7. 6/12/2016 7 Student Learning Assessment Cycle Clear Learning Goals Clear Learning Objectives (i.e., how goal is demonstrated) Identify Measures Collect Data Analyze Findings Action for Improvement

8. 6/12/2016 8 Assessment of Student Learning How do we know students are achieving program learning outcomes? Establish program learning goals Apply curricular mapping Implement a program-level assessment plan

9. 6/12/2016 9 II. Goals & Objectives

10. 6/12/2016 10 Definitions Goal: “General statement about knowledge, skills, attitudes, and values expected in graduates” * Objective: “A clear concise statement that describes how students can demonstrate their mastery of a program goal” * * Allen, Mary J., Assessing Academic Programs in Higher Education. Anker Publishing Company, 2004.

11. 6/12/2016 11 Examples Goal: Students will utilize quantitative methods in social scientific research. Objective: Students will independently design and carry out correlational research that yields valid results.

12. 6/12/2016 12 Bloom’s Taxonomy Bloom’s Taxonomy: classification for levels of learning moves from the least to most complex has list of verbs associate with each level: www.odu.edu/educ/roverbau/Bloom/blooms_taxono my.htm

13. 6/12/2016 13 Bloom’s Taxonomy www.odu.edu/educ/roverbau/Bloom/blooms_taxonomy.htm Complexity

14. 6/12/2016 14StudentVoice.Com Writing Objectives: The 3 Ms How does the objective support the department goal or mission? Meaningful What is needed to foster the achievement of the objective? Is the objective realistic? Manageable How will you know if the objective has been achieved? What will be the assessment method? Measurable

15. 6/12/2016 15 Characteristics of a Good Objective Explains how students can demonstrate the mastery of a learning goal Uses an active verb Specifies definite observable behaviors Is clearly linked to a goal

16. 6/12/2016 16 III. Measures

17. 6/12/2016 17 Why Grades Are Insufficient Do not tell you about student strengths and weaknesses in learning content Do not tell you about student strengths and weaknesses in learning content Cannot use average grade to tell you where to make changes in the curriculum Cannot use average grade to tell you where to make changes in the curriculum

18. 6/12/2016 18 Definitions Direct Measure: A measure that directly demonstrates that students achieved a learning objective A measure that directly demonstrates that students achieved a learning objective Indirect Measure: A measure that offers students’ opinion about meeting the learning objective

19. 6/12/2016 19 Examples Direct Measure: Set of test items Set of test items Written assignment scored with a rubric Written assignment scored with a rubric Direct observation scored with a rubric Direct observation scored with a rubric Indirect Measure: Student reflections/self-assessments Student reflections/self-assessments Student opinion surveys Student opinion surveys Focus groups Focus groups

20. 6/12/2016 20 Choosing Measures Begin with what you are already assessing Begin with what you are already assessing Use what will tell you about student strengths and weaknesses Use what will tell you about student strengths and weaknesses Look for culminating experiences Look for culminating experiences

21. 6/12/2016 21 Example Assessment AY09-10 Phase I – due November 15, 2009Phase II – due May 31, 2010 Program Learning Goal: Students will … Student Learning Objective s Students will … Direct and Indirect Measures Findings (Learning Outcomes) InterpretationActions Planned Goal 1 (continued ): Read proficientl y Learning Objective 2: Analyze college- level readings proficientl y Measure 1: Scores for the interpretive questions on the Comprehensio n portion of the Nelson- Denny Reading Test Measure 1: On average, the 210 ENG XX students tested correctly answered XX% of the interpretive comprehension questions they attempted within the time limit, or XX% of all the interpretive comprehension questions. There was a considerable spread of results from low to high scores. Measure 1: In contrast with their fine performance comprehending the literal meaning of texts (see above measure), ENG XX students on average tend to be quite challenged in interpreting meanings that are not literally spelled out. Measures 1 & 2: The (program) will provide (faculty) with pedagogical resources on inference building, such as a handout for students on reading skills (e.g., connotation, denotation, etc.). Senior (faculty) will be leading a series of seminars in 2010-11 to educate faculty both within (the Department) and across campus on reading pedagogy. The Department also plans to expand its assessment of students ’ reading proficiency by administering the Nelson- Denny Reading Test to students at other stages in the (program). Measure 2: A rubric assessing students ’ written summaries of readings Measure 2: On a 5-point scale, the Exit Exam essays of 481 ENG XX students most frequently received ratings of 3 (XX%) or 4 (XX%), with an “ average ” rating of X.XX. Measure 2: The rubric defines a rating of 4 as signifying a “ good, fairly concise synthesis of most of the key points, ” and a rating of 3 as signifying a “ fair, less focused synthesis that may misidentify key points. ” ENG XX students ’ grasp of key points in readings could be better developed.

22. 6/12/2016 22 IV. Rubrics

23. 6/12/2016 23 Introduction to Rubrics by Stevens and Levi, 2005

24. 6/12/2016 24 Definition Rubric: “At its most basic, a rubric is a scoring tool that lays out the specific expectations for an assignment.” - Stevens and Levi, 2005

25. 6/12/2016 Descriptive Scoring Guides Student Learning *Adapted from Suskie, L. (2008). Assessing Student Learning: A Common Sense Guide, 2nd Edition. Jossey-Bass, San Francisco, CA, p. 144. Well Done (5) Clearly, concisely written. Logical, intuitive progression of ideas and supporting information. Clear and direct cues to all information. Motivating questions and advance organizers convey main idea. Information is accurate. Presents overall topic. Draws in audience with compelling questions or relating to audience’s interests or goals. Satisfactory (4-3) Logical progression of ideas and supporting information. Most cues to information are clear and direct. Includes persuasive information Clear, coherent, and related to topic. Needs Improvement (2-1) Vague in conveying viewpoint and purpose. Some logical progression of ideas and supporting information, but cues are confusing or flawed. Includes persuasive information with few facts. Some structure but does not create a sense of what follows. May be overly detailed or incomplete. Somewhat appealing. Incomplete (0) Lacks a clear point of view and logical sequence of information. Cues to information are not evident. Information is incomplete, out of date, or incorrect. Does not orient the audience to what will follow. Organization Persuasiveness Introduction Etc. Exhibit 9.6. Descriptive Rubric for a Slide Presentation on Findings from Research Sources*

26. 6/12/2016 26 Do You Need A Rubric? Carpal tunnel from writing the same comments? Weeks behind in your grading? Worry last papers may be graded differently than first? - Stevens and Levi, 2005

27. 6/12/2016 27 Benefits of a Rubric Timely feedback Increased likelihood of feedback being read Encourages critical thinking Helps to refine teaching skills - Stevens and Levi, 2005

28. 6/12/2016 28 Rubric Components 1. Assignment description 2. Scale 3. Dimensions 4. Level of achievement - Stevens and Levi, 2005

29. 6/12/2016 29 Rubric Components 3 Level Rubric Task Description: DimensionsExemplary 3 Competent 2 Developing 1

30. 6/12/2016 30 Rubric Components Scoring Guide Rubric Task Description: DimensionsDescription of highest level of performance CommentsPoints

31. 6/12/2016 31 Rubric Components Scoring Guide Rubric Task Description: DimensionsDescriptionCommentsPoints ▫ 2 ▫ 1

32. 6/12/2016 32 Rubric Components Scoring Guide Rubric with Boxes and Circles Task Description: DimensionsDescriptionCommentsPoints ▫ text, text, text text, text, text 2 ▫ text, text, text 1

33. 6/12/2016 33 Look for Models ( http://www2.acs.ncsu.edu/UPA/assmt/resour ce.htm#gened_rubrics) http://www2.acs.ncsu.edu/UPA/assmt/resour ce.htm#gened_rubrics

34. 6/12/2016 34 Steps to Creating a Rubric 1. Reflecting on the assignment 2. Listing learning objectives 3. Grouping and labeling 4. Applying to rubric grid - Stevens and Levi, 2005

35. Resources on Rubrics* How to create a rubric: http://intranet.cps.k12.il.us/Assessments/Ideas_and_Rubrics/Create_Rubric/ Create_rubric.html http://intranet.cps.k12.il.us/Assessments/Ideas_and_Rubrics/Create_Rubric/ Create_rubric.html Rubric for a given task: http://webquest.sdsu.edu/rubrics/rubrics.htmlhttp://webquest.sdsu.edu/rubrics/rubrics.html Designing scoring rubrics: http://pareonline.net/getvn.asp?v=7&n=25http://pareonline.net/getvn.asp?v=7&n=25 Scoring rubrics: http://pareonline.net/getvn.asp?v=7&n=3http://pareonline.net/getvn.asp?v=7&n=3 Recommendations for scoring rubrics: http://pareonline.net/getvn.asp?v=8&n=14 Scoring rubric development: validity and reliability: http://pareonline.net/getvn.asp?v=7&n=10 Rubrics for web lessons: http://www.teachervision.fen.com/teaching-methods-and- management/rubrics/4521.html Creating rubrics: http://pareonline.net/getvn.asp?v=9&n=2http://pareonline.net/getvn.asp?v=9&n=2 Performance criteria: http://pareonline.net/getvn.asp?v=9&n=2http://pareonline.net/getvn.asp?v=9&n=2 Student Learning *Suskie, L. (2008). Assessing Student Learning: A Common Sense Guide, 2nd Edition. Jossey-Bass, San Francisco, CA, p. 154.

36. 6/12/2016 36 Resources Kathleen M. Morley University Director of Assessment Kathleen.morley@liu.edu x4258 Assessment Website: http://liu.edu/Academic- Affairs/Outcomes-Assessment.aspx