1. Showing Growth through Integrated Performance Assessments Karen Lichtman, Ph.D. klichtman@niu.edu Assistant Professor of Spanish Linguistics Director of Teacher Licensure Department of Foreign Languages & Literatures Northern Illinois University New Ideas in Foreign Language: Teaching StrategiesNovember 6, 2015 1

2. FL 513 Foreign Language Assessment Southern Oregon University Guanajuato, Mexico 2

3. Level of comfort with IPA (Integrated Performance Assessment)? 1.I have developed and used IPAs 2.I know what one is, but I haven’t really used them in my class 3.I think I’ve heard of them 4.Are you talking about beer? 3

4. Discuss with your table How do you currently assess students? What goes into their grades? 4

5. 1a. Backward Design: Assessment is at the center of planning 5

6. 6 Brown & Abeywickrama Plan the unit around a summative performance assessment

7. Integrated Performance Assessment (IPA) What does “performance assessment” mean? Knowledge: Do you know it? Performance: Can you use it? Proficiency: Can you use it without any preparation, with strangers? KnowledgePerformanceProficiency 7

8. Teachers will use both knowledge and performance assessments. Performance assessment: application of what students have learned in a unit to a real-life or “authentic” communication situation. – Realistic reflection of real-world situation – Require judgment and innovation – Task requires meaningful target language use – Simulate work, civic or personal life with an audience beyond the teacher – Students use a variety of skills and knowledge to complete a complex task – Integration of language skills – Includes opportunities for feedback (Shrum & Glisan, 2010, p. 410) 8

9. So what is an Integrated Performance Assessment? Integrating Communication with the other goal areas of the standards Integrating the 3 modes of communication – Each task in the assessment builds on the previous task Assessment Framework by Foreign Language National Assessment of Educational Progress (FLNAEP) 9

10. 1b. 3 modes of communication Interpretive Listen to or read an authentic text; assess comprehension Interpersonal Spontaneous communication with someone Presentational Sharing research/ideas in presentations, posters, brochures, websites, etc. THEME 10

11. 1b. 3 modes of communication Interpretive Listen to or read an authentic text; assess comprehension Presentational Sharing research/ideas in presentations, posters, brochures, websites, etc. Feedback Clear expectations Discuss strategies Feedback Clear expectations (seeing/using rubrics, samples) Feedback Clear expectations Interpersonal Spontaneous communication with someone THEME 11

12. 1c. ACTFL proficiency guidelines Used to select appropriate materials & describe target performance on an IPA http://www.actfl.org/pu blications/guidelines- and-manuals/actfl- proficiency-guidelines- 2012 http://www.actfl.org/pu blications/guidelines- and-manuals/actfl- proficiency-guidelines- 2012 Page 2 of handout 12

13. Sounds great, but… isn’t it time-consuming? Yes However, teachers often feel that each assessment should be new and different Showing growth is actually easier if you reuse the SAME assessment… ideas from the research process 13

14. Discuss How much does student growth currently count in your evaluation? What kind of assessments is your program using to show student growth? 14

15. 2. The research process Teachers do research all the time! 15

16. Quantitative research Research questions – Is this teaching method better than that teaching method? – How proficient are these students? – (How much) do students improve after ___? Common research designs: – Comparing two or more groups to each other – Comparing a group to itself (pretest/posttest) – Both of these designs can tell you whether there are statistically significant differences between the groups. 16

17. Research example 1: Ashley Soriano’s masters thesis “Using tablet-based instruction to improve communicative responses in the world language classroom” Research questions: 1.Will tablet-based instruction increase the use of vocabulary in communicative responses? 2.Can self-assessment and reflection aid in student awareness and improvement of communicative skills? 3.Will fluidity increase as students are exposed to more input and produce more output in the target language through the use of apps and other websites? 17

18. Recordings every other Monday Prompt: What did you do on the weekend? Spanish II and III Meeting standard = 3 out of 4 points; Exceeding standard = 3.5 out of 4 points Research example 1: Ashley Soriano’s masters thesis 18

19. Students significantly improved both fluidity and ability to elaborate by the end of the semester (3 rd hour = p <.05, 5th hour = p <.05, 8 th hour = p <.05) Research example 1: Ashley Soriano’s masters thesis 19

20. Digression: what is statistical significance? Pretest Bob: 85 Jenny: 89 Phil: 90 Jane: 64 Elizabeth: 75 Jonny: 62, Steve: 88, Missy: 70, Kara: 84, Paul: 81, Mark: 84 Posttest Bob: 80 Jenny: 93 Phil: 99 Jane: 96 Elizabeth: 91 Jonny: 100, Steve: 94, Missy: 93, Kara: 99, Paul: 91, Mark: 93 20

21. Digression: what is statistical significance? You use different statistical tests depending on your data. Each test spits out a “p-value” The p-value is the probability that your result is not due to random chance. p =.05 95% chance I’m right 5% chance my result is due to random chance 21

22. Digression: what is statistical significance? Pretest Bob: 85 Jenny: 89 Phil: 90 Jane: 64 Elizabeth: 75 Jonny: 62, Steve: 88, Missy: 70, Kara: 84, Paul: 81, Mark: 84 Posttest Bob: 80 Jenny: 93 Phil: 99 Jane: 96 Elizabeth: 91 Jonny: 100, Steve: 94, Missy: 93, Kara: 99, Paul: 91, Mark: 93 p=.46 Goal: p <.05 p=.00 * p=.07 22

23. Research example 2: Julie Weitzel’s masters thesis “The Effect of Communication Strategy Training on Students’ Willingness to Communicate in the Interpersonal Mode” Research questions: 1.Does communication strategy training improve students’ willingness to communicate in the interpersonal mode? 2.Which communication strategies do students find the most useful? 23

24. Research example 2: Julie Weitzel’s masters thesis Same questionnaire in August and December Likert scale questions 1234 StronglyStrongly DisagreeAgree Open-ended questions 24

25. Research example 2: Julie Weitzel’s masters thesis Students significantly improved their willingness to communicate By far the most helpful strategy was circumlocution 25

26. 3. Creating IPAs and rubrics Creating an IPA: http://www.carla.umn.edu/assessment/vac/in dex.html > Create an assessment unit step-by-step http://www.carla.umn.edu/assessment/vac/in dex.html Today we will all come up with an IPA idea! 26

27. Creating your IPA idea 1 minute: Think of a unit that you currently teach, but you are not 100% happy with the current final exam or final project. Write down the topic. Example: airport unit 1 minute: Describe the unit to a partner 27

28. Creating your IPA idea 2 minutes: write one “Students Will Be Able To (SWBAT)” statement for the content of your unit and one “SWBAT” statement for the language of your unit Example: middle school market unit on CARLA 28

29. Creating your IPA idea 2 minutes: with your partner, brainstorm an interpretive task for the unit. Students will read/listen to _____________ and answer questions/ complete an information chart/ other activity 29

30. Creating your IPA idea 2 minutes: with your partner, brainstorm an interpersonal speaking task where students compare their opinions about their information from the interpretive task 2 minutes: with your partner, brainstorm a presentational task for your theme. If you can’t think of one, have your students create a “webpage” (real or on paper!). 30

31. Creating a rubric (Sandrock) Do include on a rubric A description of a quality performance – Clear expectations – Realistic expectations for language use at proficiency level – Observable indicators – Criteria appropriate for the task – Criteria appropriate for the mode of communication – List of elements that you want students to think about in doing the task Don’t include on a rubric Non-negotiables (followed directions, completed task) Things that are not about language (has nice drawings) 31

32. Creating a rubric Sample “master rubric” from Jefferson public schools 32

33. What is realistic? 33

34. Creating a rubric Splitting the master rubric for level 1 vs. level 3 34

35. 3a. Scoring rubrics Handout p. 3 (Lackey & Weitzel, 2015) https://sites.google.com/site/assess mentmatterssbc/standards-based-learning https://sites.google.com/site/assess mentmatterssbc/standards-based-learning 1.Create your rubric 2.Put points in each box (1, 2, 3, 4) 3.Figure out the minimum and maximum possible points 4.Put every possible point value across the top 5.Assign 50% to the minimum value and 100% to the maximum value 35

36. 3a. Scoring rubrics 6. Time for some math! 50 divided by the number of jumps between possible point values = how many percentage points you go up between possible point values 50 / 9 = 5.55 50 + 5.55 = 55.6 55.6 + 5.55 = 61.1, etc. 36

37. 4. Using rubrics multiple times to show growth Remember… it’s good to reuse the SAME RUBRIC! By using a rubric tied to the ACTFL proficiency levels, you can show growth over time Grading criteria can become more stringent without having to change the rubric itself 37

38. Last page of handout: different expectations for different levels 38

39. In conclusion… If you can invest the time to create 3 rubrics that cover the 3 modes of communication for all your proficiency levels, you can keep using them for YEARS! 39

40. Questions? klichtman@niu.eduKaren Lichtman klichtman@niu.edu 40