1. Introduction to Formative Tools Rural Schools Conference Kim Rathke, Julie Quinn, and Jared Wright Utah State Office of Education

2. Summer, 2013 2

3. What is the most emphasized indicator of success in your school? Formative Tools Interim AssessmentSummative Assessment Benchmark Assessment Teacher-led Classroom Assessment Teacher Evaluation Accountability Student Reflection and ownership of learning Teacher Reflection Analysis of student work

4. What informs data conversations in your school? Formative Tools Interim AssessmentSummative Assessment Benchmark Assessment Teacher-led Classroom Assessment Teacher Evaluation Accountability Student Reflection and ownership of learning Teacher Reflection Analysis of student work

5. What is the purpose of the assessment? With Assessment, Purpose Is Everything We assess for two reasons: 1) to gather evidence to inform instructional decisions and 2) to encourage students to try to learn Stiggins, R. (2008). Assessment Manifesto: A Call for the Development of Balanced Assessment Systems. Portland, OR: ETS Assessment Training Institute.

6. FIRST Components (Teacher Tools) ToolAccess and SupportFeaturesTimeline “Old” UTIPS12 regional servers E.g., Myutips.org, Ucutips.org Available through July 31, 2014 UTIPS CoreCentral server currently located at UEN New servers come online at USOE August 1, 2014 E.g., District.utips.org Help Desk: help@utips.orghelp@utips.org Documentation: http://utipscore.wikispaces.com/ http://utipscore.wikispaces.com/ Teacher-created items and tests Share tests to groups Current State Standards ELA, Math, Science, CTE, Health, all content areas State item pool was realigned to current state standards – no new items have been added Available through at least June 30, 2015 6

7. FIRST Components (Teacher Tools) ToolAccess and SupportFeaturesTimeline Learning Point Navigator (Formerly SAGE Formative) http://sageportal.org Help Desk: sagehelpdesk@air.org or 855-570-7239sagehelpdesk@air.org Documentation: User Guides - http://sageportal.org/resources/?section=1- general-family http://sageportal.org/resources/?section=1- general-family Tutorials - http://sageportal.org/resources/?section=1- general-family http://sageportal.org/resources/?section=1- general-family Students log in with SSID and password – can log in from home and create their own assignments, separate from teacher-created assignments Math, Science, ELA only Multiple Item Types (MC, CR, technology-enhanced (haptic)) Teachers create assessments for their own classrooms to support instruction Shared assessments supported beginning fall 2014 Instructional support materials School and district level libraries, sharing between teachers coming fall 2014 Students loaded from UTREx May, 2014 Teacher-created items, can share to school or district library Fall, 2014 Ad-hoc groups can be created to share assignments between COMPOSE (Formerly Utah Write) Access and support TBA… Online writing with AI scoring for grades 3-12 Available to all content teachers, grades 3-12 Launches September, 2014

8. Interim and Benchmark Assessments ToolPurposeFeaturesTimeline SAGE Interim OPTIONAL Provide “growth” Predictive of SAGE Summative Computer Adaptive Full mirror of the SAGE Summative assessment (blueprint, length) Immediate Results flow to Learning Point Navigator Fall Interim Winter Interim Students may take one interim during the fall and/or one interim during the winter DIBELS REQUIRED Reading on grade level determination “Amplify” available for online data collection (LEA option) Ongoing Shmoop OPTIONAL ACT Prep Tool http://www.shmoop.com/utah/ Sample questions and lesson plans Ongoing Explore, Plan OPTIONAL Predictive of ACTAligned to ACT, provides indicators of strengths/weaknesses Ongoing

9. What is “Formative” Assessment? (FAST) State Collaborative on Assessment and Student Standards (SCASS) of the Council of Chief State School Officers (CCSSO): Formative assessment is a process used by teachers and students during instruction that provides feedback to adjust ongoing teaching and learning to improve students’ achievement of intended instructional outcomes. The formative assessment process involves gathering evidence and providing feedback to adjust ongoing teaching moves and learning tactics.

10. What is “Formative” Assessment? Similarly, an assessment is an important part of the formative-assessment process, but it is only that—a part of the formative-assessment process. The entire process involves decisions about when to test and what to test, selection or construction of suitable assessment procedures, judgments about whether assessment-elicited evidence should lead to adjustments, and choices about the nature of any adjustments. Assessments are a key component of the formative-assessment process, but they are not the entire process. Popham, W. J. (2011, February 22). Formative Assessment-A Process, Not a Test. Retrieved June 23, 2014, from http://www.edweek.org/ew/articles/2011/02/23/21popham.h30.html?tkn=PSCCGmSb/B5QkuTaRS6t7BoT2I7Q/Q9 NdgMl http://www.edweek.org/ew/articles/2011/02/23/21popham.h30.html?tkn=PSCCGmSb/B5QkuTaRS6t7BoT2I7Q/Q9 NdgMl

11. Elements Of an Assessment Event 1.Is the assessment high-quality? 2.Do teachers have access to the data? 3.Do teachers use the data? 4.Do students have access to the data? 5.Do students use the data? Summer, 2013 11

12. Four Major Assessment Types Daily Events – Things that happen everyday in the normal activity of a classroom Periodic Activities – Structured or planned classroom activities Formal Assessments – Documented evidence about something is gathered Typically student understanding is what the evidence points to External Assessments – Evidence of student understanding is gathered for purposes outside of the classroom

13. Data-Informed Instruction Good Assessment  Should generate good data Good data  Should generate good discussion and conclusions Good conclusions  Should generate good actions

14. Using Learning Point Navigator LPN

15. Key Terminology Item: an assignment activity that requires a student response (a question). Activity: an individual component of a resource—a lesson, a practice question, or a quiz question Assignment: a collection of student activity resources (a quiz or test) Stimulus: a component in a resource that provides context for an activity— includes reading passages, images, graphs, tables, video or audio clips. Stem: The question or prompt in an activity that students must respond to. Content Editor: The tool in the Activity Builder that is used to align activities to standards, and to create and edit the content for activities and stimuli.

16. Learning Point Navigator Formerly “SAGE Formative” Three main tools—grades K-12, Math, Science and ELA ONLY Formative test bank for ELA, math, and science—to inform instruction Includes items types that look and feel like the SAGE Summative item types Activity builder – educator-created items, must align to math, science or ELA standards (multiple choice – single and multiple select; constructed response) Lesson plans—to enhance instruction and professional development

17. Learning Point Navigator The landing page for educators provides access to the five major sections of the site: Browse Resources Assignments Activity Builder Classes & Students Reports

18. Browse Resources Resources are aligned to ELA, Math, and Science standards and benchmarks. Resources are also categorized by ELA, Math, and Science content areas, grade level, library, material type, and learning modality. These categories are organized as filter boxes on the search panel. Student activities can be added to assignments. Lesson plans and professional development aids are not designed for students and cannot be added to assignments.

19. Browse Resources Browse Resources Search Panel (With Filters Selected)

20. Browse Resources: Resource Search  Sorting search results  Previewing resources  Marking resources as favorites  Rating and reviewing resources  Viewing related resources  Removing resources  Copying resources to the Activity Builder  Adding resources to the Resource Cart The Resources Search Results list displays the resource(s) that meet the search parameters you selected. This results section provides you with several options:

21. Browse Resources: Previewing You can preview and navigate through resources to see how they will look to students. You can also use the student tools, mark resources as favorites, add resources to the Resource Cart, and print resources. On the Resources Search Results page, click the arrow button next to the name of the resource. The Resource Preview will load. Resource Preview Page Previewing a Resource

22. Browse Resources: Marking Favorites You can mark resources as favorites, allowing you to easily find them later. Resources can be “favorited” from the Resources Search Results page or from the Resource Preview page.  To view a list of only your favorite resources in the search results section, click the [Favorites] button next to “View by.” If you have not marked any resources as favorites, no resources will be displayed.  To view all the resources in the search results, click the [All] button next to “View by.”

23. Adding Resources The Resource Cart allows educators to temporarily store resources. Resources that have been added to the Resource Cart can be combined into a new resource or saved as an assignment. Added resources will remain in the resource cart until they are removed or they are saved as a resource or assignment.

24. Saving Resources as Assignments Once resources have been saved in the Resource Cart as an assignment, the Add to Assignment page will load. On the Add to Assignment page, you can choose to add the resources to a new or existing assignment for your students.

25. Saving Resources as Assignments 1.In the Assignment Name field, enter the assignment’s name. 2.Click inside the Start Date field. A pair of calendars will pop up. Click the date when the assignment should start. Repeat the process for the Due Date field.

26. Saving Resources as Assignments 3.In the Directions field, enter the directions for the assignment. 4.You can adjust the proficiency level cut scores. Click and drag the sliders on the Proficiency scale.

27. Saving Resources as Assignments 5.To preview the assignment, click the [Preview Assignment] button. To finish and assign to students, click the [Save] button. To exit this page without creating the assignment, click the [Cancel] button.

28. Adding Resources to Existing Assignments The process for adding a resource to an existing assignment is similar to the process for adding a resource to a new assignment. On the Add to Assignment page, click the “Select Assignment” drop-down list, and select the assignment that you want to add the resource to.

29. Creating Your Own Activities Using the Activity Builder

30. Activity Builder Enables users to develop their own activities using the Content Editor and assign them to students

31. Creating an Activity To create an activity, click the [+ Create New Activity] button above the Activities table.

32. Creating an Activity The details page of the Content Editor will open. You are creating one question so name the activity as a question. In the Name field, enter a name for the activity.

33. Creating an Activity To align the activity with a standard, click the [Add Alignment] button. The “Grades” drop-down menu will appear. Continue to select options from the drop-down menus until you can click the [Add] button.

34. Creating an Activity Click the [Save] button and then click the [Open Content Editor] button. The content authoring page of the Content Editor will open.

35. Adding a Stem Step 1: In the Activity Canvas, click [Item Body].

36. Adding a Stem Step 2: In the Elements Pane, click the [Stem] element. The Content Editor text box will open.

37. Adding a Stem Step 3: In the editing area of the Content Editor text box, enter the text for the stem. Step 4: When you are finished, click [Save]. The Stem box will appear in the Activity Canvas.

38. Adding Response Choices Step 5: In the Activity Canvas, click the blue [Insert] bar below the Stem box. Step 6: In the Elements Pane, click the [Response Choices] element.

39. Adding Response Options  Step 7: In the Activity Canvas, click [Response Choices]. en  Step 8: In the Elements Pane, click [Response Option] element. The Content Editor text box will open.  Step 9: In the editing area, enter the response option.  Step 10: Click the [Save] button. The Response Option will be added to the Activity Canvas. To add additional Response Options, you will click on every other blue bar.

40. Adding Response Feedback  Step 11: To add feedback for the response option, click the blue [Insert] bar below the Response Option box.  Step 12: In the Elements Pane, click the [Response Feedback] element. The Content Editor text box will open.  Step 13: In the editing area, enter the response feedback.

41. Adding Response Feedback Step 14: When you are finished, click the [Save] button. The Response Feedback box will be added to the Activity Canvas. To add multiple response options to the activity, repeat steps for each response option.

42. Adding a Multiple Choice Rubric  To access the rubric, click the [File] button above the Elements Pane and select “Multiple Choice Rubric.” The rubric window will open.  Click the correct response option(s). The correct answer(s) will be marked in the rubric.  When you are finished configuring the rubric, click the [Save] button.  When you have finished creating the activity, click the [Save] button located in the bottom right corner of the Content Editor page.

43. Reviewing a Multiple Choice Rubric  To view the rubric, click the [File] button above the Elements Pane and select “Multiple Choice Rubric.” The rubric window will open.  The correct answer will be selected.  When you have finished reviewing the rubric, click the [Save] button located in the bottom right corner of the Content Editor tool.

44. Adding a Text Response Area  After adding a stem to the Activity Canvas, click the blue [Insert] bar directly below the stem box.  Click the [Text Response Area] element. The Text Response Area box will be added to the Activity Canvas.

45. Adding a Human Rubric  To access the rubric, click the [File] button above the Elements Pane and select “Human Rubric.”  The Content Editor text box will open. Enter the hand-scoring rubric and then click [Save].

46. Reviewing a Human Rubric  To view the rubric, click the [File] button above the Elements Pane and select “Human Rubric.” The Content Editor text box will open.  When you have finished reviewing the rubric, click the [Save] button located in the bottom right corner of the Content Editor tool.

47. Hand Scoring  Allows users to manually enter scores for certain activities in student assignments.  If an assignment requires hand scoring, the Actions toolbar on the Assignments page will include a [Hand Score] button.

48. Viewing an Activity  You can preview an activity by clicking the View button to the left of the activity on the Activities table.

49. Publishing an Activity as a Resource Navigate to the Activities table on the Activity Builder page. Click the [<<] button in the Actions column. The Actions toolbar will expand.

50. Publishing an Activity as a Resource 1.Click the [Publish] button in the Actions toolbar. The “Create New Resource” page will open. 2.In the Name field, enter a name for the resource. 3.In the Description field, enter a description for the resource.

51. Create New Resource Page: Managing Standards The Aligned Standards section displays the standards that are aligned to the activities in the resource. To remove a standard, uncheck the checkbox next to that standard. Note: A resource must be aligned to at least one standard. If there is only one standard listed in this section, you cannot uncheck it.

52. Select Library to Publish to Window When you are finished, click the [Save and Publish] button in the bottom right corner of the page. If you are a School Educator, the resource will automatically be added to My Library. If you are an Administrative Educator, the “Select a Library to Publish to” window will open. Click the name of the library you want to publish the resource to and then click [Publish]. The resource will be added to the selected library. Note: This step only applies to Administrative Educators.

53. Where Do I Find My Activities? After you have published your activity, you will find each individual question under Browse Resources in your own library. To locate them quickly, make sure you check only the box titled “My Library.” To assign your activities, follow the same steps used to assign resources from the formative test bank.

54. Creating Common Assignments LEAs/Administrators Only Teachers Fall 2014

55. Common Assignments  Provides ability for school or LEA level users to develop common assignments that can be assigned by teachers in their school or district.  Stored in an entity library that any School Educator belonging to that entity can access.

56. Creating a Common Assignment Administrative Educators can create common assignments, but they cannot assign them directly to students. 1.In the Name field, enter a name for the common assignment.

57. Creating a Common Assignment 1.Click inside the Assignment Window Open field. A pair of calendars will pop up. 2.Click the date when the assignment should become available. 3.Click inside the Assignment Window Close field. A pair of calendars will pop up. 4.Click the date when the assignment’s availability should end.

58. Creating a Common Assignment 5.In the Directions field, enter directions for the assignment.

59. Creating a Common Assignment 6.To adjust the proficiency level cut scores for the assignment, click and drag the sliders on the Proficiency Level scale. These levels determine which student scores will be considered “not proficient,” “needs improvement,” and “proficient.”

60. Creating a Common Assignment 7.Optional: To add a calculator to the assignment, select an option from the “Select a Calculator” drop-down menu. 8.Optional: To upload a file to the assignment, click the [Select File] button and then select a file from your computer. It’s important to note that only pdf files should be uploaded because students can edit others.

61. Creating a Common Assignment 9.To manage the resources included in the common assignment, click [Manage Resources]. The Assigned Resources window will open. 10.When you are finished creating the resource, click the [Save] button in the bottom right corner of the Create New Assignment page. Remind Admin Educators that their assignments are saved in the LEA Library if they need to go back to review or edit.

62. Reporting with Learning Point Navigator

63. Common Assignment Reports Assignment Summary and Item Analysis Reports available Display data only for the selected assignment at the selected library level Available at the state, LEA, and school level for Administrative Educators. These reports are similar to assignment reports that school educators can view however they are only available to administrative educators and can only be accessed from the assignments page.

64. Common Assignment Summary Reports Provide overall assignment information for the selected common assignment at the selected level. Available at the state, LEA, and school level.

65. Common Assignment Item Analysis Reports Available for common assignments at various library levels. Provide information on how students responded to each quiz item included in the selected common assignment.

66. Educator-Level Reporting Three types of reports Assignment Summary Reports Benchmark Proficiency Reports—not available for common assignments until Fall 2014 Item Analysis Reports

67. Educator-Level Reporting Who?... Who do you want to see data for? All rosters, a single roster, only students given the assignment, or an individual student? What?... What data do you want to view? Assignment results, benchmark results, or item-specific information? When?... At what point in time do you want to view data? Do you want to view performance during a specific window in time? Do you want to view performance on a single assignment? How do you want to handle items not answered or attempted by students?”

68. Educator-Level Reporting When you click [View Reports], the main Reports Widget will display. This widget allows you to select the parameters for each available report dimension: Who, What, and When. This widget is also available from within each report.

69. Educator-Level Reporting Sample Assignment Summary Report Screen for a Selected Assignment

70. Educator-Level Reporting Benchmark Proficiency Reports provide information on students’ proficiency for each benchmark associated with the selected assignment(s). These reports display students’ benchmark proficiency based on their response to activities. Activities can be aligned to one or more benchmarks. ONLY available at educator level until Fall 2014

71. Educator-Level Reporting Benchmark Proficiency Reports provide information on students’ proficiency for each benchmark associated with the selected assignment(s). These reports display students’ benchmark proficiency based on their response to activities. Activities can be aligned to one or more benchmarks. ONLY available at educator level until Fall 2014

72. New Features Expected in Fall 2014

73. New Features Simplified item authoring tool—tabs added to reduce mouse clicks Peer-to-Peer sharing—Educators can share assignments.csv Export of individual reports Assignment print feature—Educators will be able to print an entire assignment Answer key: Option to see answer key for all machine-scored items UTIPS upload: Ability to upload ELA, math, and science items from teacher-created UTIPS files into educator libraries

74. New Features Accessibility features: Masking Text-to-Speech capability in Chrome browser and iOs iPads will be able to download a browser for TTS Creating a Chrome plug-in for PCs Ability to expand and contract passages