2. Presented By: Sarah McManus, Section Chief, Testing Policy & Operations/Accountability Services CCSSO Education Leaders Conference Using Data to Improve Instruction Implementing a Formative Assessment Model with Districts and Schools: What North Carolina has learned

3. What is formative assessment? Read the following vignette and discuss with a partner Is this formative assessment? Why or why not?

4. The teacher provides students with an open-ended question related to a concept they are studying and asks the students to identify the information or details necessary for a response to demonstrate full understanding of the concept. A list of these details is recorded on the board. The teacher then provides students with examples of several student responses that were given by students in previous years. The students are asked to analyze the responses and to determine if the responses show full understanding, partial understanding, or no understanding of the concept. Students must justify their answers. As this thinking is shared, the list of details or supports necessary for a response to the question is further refined until a set of criteria emerges.

5. Historical Background The CCSSO Formative Assessment Initiative began in March 2006 under the direction of Don Long, former CCSSO SCASS Director

6. Concerns “Inside the Black Box” Too much time spent on measuring learning rather than promoting and helping learning to occur Most summative assessments look like “mini state tests” Forced to “teach to the test” Less motivated students

7. Where do we want to go? As a Nation….. Where are we now? How do we close the gap?

8. Focus on improving teaching and learning Formative assessment must be part of a comprehensive, balanced, and coherent standards-based system for teaching and learning in the 21st century.

9. There is an emphasis on: “The test” State test results Practice items Alignment Remediation What is getting the most attention and what is getting left behind? There is little emphasis on: Student learning Developing self-directed learners Increasing student motivation Delivering quality professional development for teachers content delivery classroom assessment

10. How do we close the gap? Create a National Initiative Gather a group to oversee the initiative –CCSSO Formative Assessment Advisory Group –Formative Assessment for Students and Teachers SCASS Develop a Common language Develop a Common vision Share the vision

11. CCSSO Definition of Formative Assessment Formative assessment is a process used by teachers and students during instruction that provides feedback to adjust ongoing teaching and learning to improve students’ achievement of intended instructional outcomes. Developed and approved by the CCSSO Formative Assessment Advisory Group and Formative Assessment for Teachers and Students (FAST) SCASS, October 2006

12. What does this mean for teacher practice? Teachers should elicit evidence about learning during instruction using a variety of methods that result in adjustments to teaching and learning. These methods can include, but are not limited to, tasks and activities. The tasks and activities alone are not formative assessments but are used during formative assessment.

13. Attributes of Effective Formative Assessment Learning Progressions Learning Goals and Criteria for Success Descriptive Feedback Self- and Peer-Assessment Collaboration

14. FAST SCASS Areas of Focus Professional Development –Definition and Attributes –Vignettes Technology –Identify tools that can be used during the process not as a formative assessment –Create a website to house all information Research –Identify supporting literature Policy –Working on a White paper for policy makers

15. North Carolina’s Response to the Initiative Gathered stakeholders (assessment, curriculum, EC, LEP, policy) including State Superintendent and SBE representative to create a vision Involved in the Delaware Enhanced Assessment Grant (focus on mathematics and science) Joined FAST SCASS Created a brochure to share the vision Presented at conferences to share the vision

16. Creating and Sharing a Vision

17. Classroom Assessment (Formative and Summative) Interim/Benchmark Assessments (Summative) Statewide Assessments (Summative) Aligned to State Standards

18. Focus on the Importance of Classroom Assessment Guides students’ judgment of what is important to learn Affects their motivation and self perceptions of competence Structures their approaches to and timing of personal study… Consolidates learning Affects the development of enduring learning strategies and skills “It appears to be the most potent force influencing education.” Crooks (1988)

19. BALANCED CLASSROOM ASSESSMENT SYSTEM FORMATIVE ASSESSMENT SUMMATIVE ASSESSMENT A process used by teachers and students during instruction that provides feedback to adjust ongoing teaching and learning to help students improve their achievement of intended instructional outcomes. A tool used after instruction to measure student achievement which provides evidence of student competence or program effectiveness.

20. FORMATIVE SUMMATIVE Occurs During Instruction Not Graded Process Descriptive Feedback Continuous Occurs at the end Graded Product Evaluative Feedback Periodic COMPARISON OF ASSESSMENTS

21. Possible Assessment Methods Formative Assessment includes Questions Classroom Discussions Learning Activities Feedback Conferences Interviews Student Self-Assessment Summative Assessment Selected Response –Multiple Choice –True/False –Matching –Fill-in Extended Written Response Performance Assessment

22. Both formative and summative assessment are valuable and important Without both the classroom assessment system is not balanced

23. Research Research shows that if students are formatively assessed, learning will improve. When learning is improved, students are able to demonstrate that learning in a variety of ways including scoring well on standardized assessments. Black and Wiliam (1998)

24. ProductsSkillsReasoningKnowledge ©Copyright Educational Testing Service/Assessment Training Institute

25. Teachers use a variety of methods to assess what each student has learned. Teachers use multiple indicators, including formative and summative assessment to evaluate student progress and growth. Teachers provide opportunities, methods, feedback, and tools for students to assess themselves and each other. Teachers use 21 st Century assessment systems to inform instruction and demonstrate evidence of students’ 21 st Century knowledge, skills, performance, and dispositions. Standard IV: Teachers Facilitate Learning For Their Students

26. Participation in the Delaware EAG provides an opportunity to discuss formative assessment and to share the vision

27. Delaware Enhanced Assessment Grant (EAG) 10 states 2 -3 Low-performing HS 2 years Professional Development in FA as part of Comprehensive Balanced Assessment System

28. Goals of the EAG… Increased commitment to a high-quality formative assessment as a process –Increase the use of descriptive feedback, reduce evaluative feedback Focus on formative assessment as a part of a comprehensive balanced assessment system Increase the quality of classroom assessment (formative and summative) –Increase student involvement

29. EAG Project Pilot semester/Implementation Year –Small budget was provided to support travel, copies –Identified 3 high schools to participate in Pilot; extended to a middle school in Implementation Year –2 Teachers per school in Pilot; all HS Mathematics teachers and all Middle Mathematics and Science teachers in Implementation Year –NC Team consisted of 2 mathematics curriculum specialists, 1 school improvement specialist, 2 testing specialists, and an LEP specialist in pilot semester; added another mathematics specialist and a science specialist during Implementation Year

30. Professional Development Plan –All FA schools were introduced to how formative assessment could be used to improve learning through a 1 day meeting –Schools required to form a Learning Team/PLC –Ongoing support from the state through web meetings, site visits, and additional face-to-face meetings –Student and teacher surveys and interviews –Professional Development Materials shared

31. Challenges during EAG Pilot One high school dropped out One teacher struggled with finding ways to use formative assessment Difficulty finding time for the learning team at the local level Materials were not used

32. Meeting the Challenges and Preparing for Implementation Principals set aside time for staff to meet (staff meetings, common planning time) School that dropped out rejoined Teachers that went through the process during the pilot were asked to share ideas and support others

33. EAG Project Themes

34. Theme 1: Dialogue was used as a tool to elicit evidence of learning Beginning: General questions; Univocal; Teacher- centered Emerged at the end: Probing; Dialogic, Student-centered

35. Theme 2: Teachers Reported Increased student involvement “I think that in my situation my students became a lot more comfortable discussing in class because it became more their class.” Teacher more accessible. Students became partners in the learning. Emerging social- constructivist environment

36. Theme 3: Using Descriptive Feedback was difficult for teachers but found useful by students “During traditional, when you looked at a check you don’t know what you did wrong. With this one, it is easier so that you know what you did, When we miss it [the teacher] puts information on there to help us understand what we did wrong so that we could do better on the next test.”

37. Theme 4: Implementation Challenges for teachers “When you get up to 9, 10 years of teaching, you have your way and you don’t want to do it any other way.” More work for the teacher and the student Power sharing was an issue Teachers have to be on top of their game

38. Theme 5: Teachers and students reported that student-developed rubrics helped student learning “They owned their own learning and they got to see what was important.” Better understanding of the criteria More transparency Improvement in performance

39. Student perspective “We got to make up our own rubric so we made it so we knew what we were looking for unlike traditional grading so we knew exactly what to do so we could get a better grade.”

40. Theme 6: Peer-assessment and self- assessment was beneficial It’s like, if we are doing something new. Some people might not get it and [the teacher] doesn’t have time to come to everyone at the same time. So, the people who do get it come to the people who don’t understand it. And, the person who is explaining it gets a better understanding by explaining it to the other person.

41. EAG Influence on other state work Comprehensive Support to Districts –Subcommitee on Formative Assessment Adopted definitions of formative and interim assessment Blue Ribbon Commission on Testing and Accountability –Committee looked at ways to redesign the testing and accountability system Able to discuss power of formative assessment

42. State Board of Education’s Vision Framework for Change: The Next Generation of Assessments and Accountability was the response to the Blue Ribbon Commission –Calls for a move to a system of formative, benchmark, and summative assessment which will equip teachers and administrators with data and feedback needed to align instruction to student’s needs

43. The New Assessment System Must: Align with graduation project Include performance-based, authentic, real-world tasks Provide diagnostic information to teachers on individual students

44. Professional Development Must: Provide teachers and administrators with skills and understanding of data to inform instructional practice and make formative assessment a daily practice in the classroom

45. Response to the Framework Online Professional Development Modules are being created All work in the agency around formative assessment is being aligned

46. 1 st essential element for effective formative assessment Open discourse about content between the students and the teacher –Students must take risks –Trust is essential –Teachers must probe for understanding –Students must be able to articulate learning targets and criteria for success –Dialogic questioning must be prominent

47. 2nd essential element for effective formative assessment Teacher must consciously elicit evidence of learning for the purpose of recognizing learning gaps –Focus on what learning is occurring not just providing activities –Plan ahead for what misconceptions may occur

48. 3rd essential element for effective formative assessment Teachers must take action by providing descriptive feedback or advice that provides hints and props to scaffold student learning –Teachers apply pedagogical content knowledge to provide the best opportunities to close gaps in learning –Students must have time to improve before being graded –Students must be involved in this process

49. Modules under development A Comprehensive Balanced Assessment System: What is FA and how is it used for learning in NC? The Process of Deconstructing NCSCS-Teacher and Student Friendly Language I know what they don’t know—now what?: Data Driven Decisions Descriptive Feedback and Grading Assessment Methods-Designing and Selecting Assessments to Do What You Want Assessment Literacy 101

50. Modules (continued) Writing Lesson Plans to Incorporate Formative Assessment Student Ownership: Peer Assessment, Self- Assessment and Goal Setting Transforming the Classroom Assessment Environment: Helping Teachers, Students and Parents Understand Formative Assessment Effective and Ineffective Questioning in the Classroom Collecting and Documenting Evidence of Learning Teaching Scenarios: Is This Formative Assessment? Administrator Roles: What Should I See in the Classroom and How do I Support FA?

51. Collaboration with Appalachia Regional Comprehensive Center (ARCC), the Assessment and Accountability Comprehensive Center (ACC) at the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) to develop and implement an online professional development community

52. Including all students Developing a Portfolio for LEP Students (“I Can” statements, self-assessment and monitoring) Linking to Responsiveness to Instruction for students (EC initiative) Reviewing ways to include formative assessment in agency work to support capacity building with districts

53. Possible Next steps for YOU… Promote the use of formative assessment as a process –Reference the CCSSO definition and attributes Offer professional development in the area of formative assessment for teachers –Use the vignettes as examples Resources from CCSSO are posted on the website http://www.ccsso.org/projects/SCASS/Projects/ Formative%5FAssessment%5Ffor%5FStudent s%5Fand%5FTeachers/

54. Challenges you might have Getting buy-in This is not a quick fix, so why do it? Teachers are doing this, so why do it? We will never get teachers to do this well? Do we believe that all teachers can effectively teach/assess students? Process vs. Product Focus on learning vs. Focus on measurement Finding the time Changing priorities and using time wisely vs. Add-on for teachers

55. Helpful resources Black, P., & Dylan, W. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 139-148. Butler, R. (1987). Task-involving and ego-involving properties of evaluation: effects of internally focused feedback on enhancement of academic self-concept. Journal of Educational Psychology, 79,474-482. Butler, R. (1988). Enhancing and undermining intrinsic motivation; the effects of task-involving and ego-involving evaluation on interest and performance. British Journal of Educational Psychology, 5, 1-14 Crooks,T. J. (1988). The impact of classroom evaluation practices on students. Review of Educational Research 58(4) 439-481. Sadler, D. R. (1989) Formative assessment and the design of instructional systems. Instructional Science 18.119-144. Stiggins, R., Arter, J. A., Chappuis, J. & Chappius, S. (2006). Classroom Assessment for Student Learning Doing it Right – Using it Well.

56. Thank you!!!! Q & A

57. Contact Information Dr. Sarah McManus Section Chief, Testing Policy and Operations NC Department of Public Instruction Accountability Services Division smcmanus@dpi.state.nc.us