© 2005 CCSSO ALL RIGHTS RESERVED. Tools for Aligning Instruction, Standards, & Assessments SEC Project Planning & Leader Training February 2005 SEC State Collaborative Tampa, Florida Surveys of Enacted Curriculum

© 2005 CCSSO ALL RIGHTS RESERVED. New Member Orientation What? … are the Survey of Enacted Curriculum tools? How?… are the collected, analyzed, reported? Why?… are they useful to educators, leaders, researchers? Now what?… do we do to implement and apply to our programs and to schools?

© 2005 CCSSO ALL RIGHTS RESERVED. Rationale Assessment Standards Curriculum

© 2005 CCSSO ALL RIGHTS RESERVED. What SEC offers Core Components of SEC tools/services 1) Surveys with teachers – instruction for year 2) Alignment analysis – content code standards and assessments 3) Data reports – pre-designed charts 4) Workshops – a) train for orientation b) use of data

© 2005 CCSSO ALL RIGHTS RESERVED. SEC Collaborative 15 States, 3 Districts CCSSO – Project Management Rolf Blank, Carlise Greenfield WCER – Surveys, reporting, analysis John Smithson, Alissa Minor, Eric Osthoff Training & Technical Assistance Learning Point Associates, TERC Regional Alliance, Consultants Diana Nunnaley Carolyn Karatzas Jennifer Unger Bryan Chumbley Lani Seikaly

© 2005 CCSSO ALL RIGHTS RESERVED. SEC Collaborative Functions/ Assumptions (04-05) Training and assistance to state leaders State SEC budget to meet state objectives – supported by SEC Collaborative Each state makes decisions – on services & project plan Multi-state benefits of collaborative

© 2005 CCSSO ALL RIGHTS RESERVED. Multi-State Collaborative Build Knowledge of leaders Share Strategies Produce joint products, tools

© 2005 CCSSO ALL RIGHTS RESERVED. Key Question -- SEC Tools Research into Practice How can Educators obtain reliable, valid data to determine Alignment of instruction with required standards and assessments?

© 2005 CCSSO ALL RIGHTS RESERVED. School & Class Description Instructional Activities General Problem Solving Activities Pairs & Small Group Work Use of Hands-on Materials Use of Calculators/Computers & other Ed. Tech. Assessment Use Instructional Influences Instructional Readiness Teacher Opinions Professional Development Types Content Focus Active Learning Collegial Participation Coherence Time Span Teacher Characteristics Instructional Content Survey Sections

© 2005 CCSSO ALL RIGHTS RESERVED. Steps in SEC Development R&D studies curriculum-- Early ‘90s Models:TIMSS, NAEP, Analyze assessment OTL interest of States, Porter/ Smithson research, CCSSO Science Assessment project State collaborative –Develop surveys-’98-’01 (NSF) 11-State field study, Reports, Alignment, USI Use of data experiment – DEC (NSF/ROLE:‘01-04) Evaluation: MSP RETA PD Study (‘02 – 05, NSF) English/ Language Arts survey (‘03 – 04) Current projects: 15 States, 4 Districts, 5 MSPs

© 2005 CCSSO ALL RIGHTS RESERVED. Questions Addressed by SEC Data How can in-depth data on content of instruction be collected and reported–not topic checklists, to analyze teaching content in relation to standards, assessments, achievement? How can methods of teaching practices be compared across classrooms, schools, districts, and states? How can enacted curriculum data be reported in a manner to encourage use by teachers to improve instruction?

© 2005 CCSSO ALL RIGHTS RESERVED. Key Education Questions (cont’d) How can we measure the effects of standards- based initiatives on instructional practices and curriculum in classrooms? How can we analyze effectiveness of professional development on changes in teachers’ instructional practices? (i.e. determine the quality of professional development)

© 2005 CCSSO ALL RIGHTS RESERVED. Applications Alignment analysis --instruction, standards, assessments Instructional improvement in schools Needs assessment/ Evaluation Indicators – monitoring change over time

© 2005 CCSSO ALL RIGHTS RESERVED. What if… [for Intro of SEC to Educators] You could use data on instructional quality and content to guide professional development? You could have consistency across grade levels? You could know how well aligned your state standards were to the state assessment? You could compare how you teach content compared to how others across the nation teach? You could use anonymous teacher data to start a powerful school discussion about what the teacher needs are?

© 2005 CCSSO ALL RIGHTS RESERVED.

Content Matrix

© 2005 CCSSO ALL RIGHTS RESERVED. Content Maps

© 2005 CCSSO ALL RIGHTS RESERVED. Surveys of Enacted Curriculum A neutral content grid with cognitive demand The intended curriculum: State content standards— What students should learn The enacted curriculum: What teachers teach The learned curriculum: Student outcomes based on school learning The assessed curriculum: State (and other) assessments— tested learning

© 2005 CCSSO ALL RIGHTS RESERVED. Curriculum Analysis: Enacted Intended Curriculum Analysis: Enacted Intended What teachers teachWhat standards require

© 2005 CCSSO ALL RIGHTS RESERVED. Full Alignment Analysis Intended— standards Enacted— taught Assessed— state tests

© 2005 CCSSO ALL RIGHTS RESERVED. SEC Content Topics Math, Science, ELA The SEC provides a neutral, research- based language to describe content of English language arts, mathematics, and science. NCLB anyone?

© 2005 CCSSO ALL RIGHTS RESERVED.

The Enacted Curriculum

© 2005 CCSSO ALL RIGHTS RESERVED.

Descriptive results can be reported using a variety of graphic displays

© 2005 CCSSO ALL RIGHTS RESERVED. SEC Collaborative Members Current States participating: Alabama Idaho Indiana Iowa Kansas Maine Michigan Mississippi Montana New Hampshire North Carolina Ohio Oklahoma Oregon Wisconsin Districts Miami-Dade Milwaukee Winston-Salem

© 2005 CCSSO ALL RIGHTS RESERVED. SEC Collaborative Services  Surveys, Data Reports  Alignment Analysis – content coding  Planning, Training-- workshops  Leader materials, e.g. Orientation Education Leaders Survey Administrators Explaining SEC  Using Data on Enacted Curriculum (Model)

© 2005 CCSSO ALL RIGHTS RESERVED.

Data on Enacted Curriculum PROJECT DESIGN 5 Urban Districts—40 Middle Schools Standards Assessment Curriculum Science/Math Initiatives Professional Development Improved Student Achievement Teacher & School Data Reports Assist Schools/ Professional Development Teacher Data Measure Change Assist Schools/ Professional Development

© 2005 CCSSO ALL RIGHTS RESERVED. SEC Collaborating Organizations Council of Chief State School Officers Wisconsin Center for Education Research Learning Point Associates/NCREL TERC Regional Alliance DEC Project American Institutes for Research — MSP RETA Study – PD Evaluation

© 2005 CCSSO ALL RIGHTS RESERVED. Longitudinal study design Spring 2003 Spring Baseline Data: Teacher Survey Instruction Practices, Content, Alignment Preparation; PD Visits to 5 sites Professional Dev. [5 MSP Sites] A) MSP Activities (treatment) B) Other PD in sites (control) PD Log w/ Teachers, Interviews-- Measure PD Quality Type - Content Duration Collective Active learning Coherence Content focus After Participation: Teacher Survey Compare practices, Content, Alignment Measure change Empirical Study of Effects of PD on Instruction Longitudinal study design Spring 2003 Spring 2005 Baseline Data: Teacher Survey Instruction Practices, Content, Alignment Preparation; PD Visits to 5 sites Professional Dev. [5 MSP Sites] A) MSP Activities (treatment) B) Other PD in sites (control) PD Log w/ Teachers, Interviews-- Measure PD Quality Type - Content Duration Collective Active learning Coherence Content focus After Participation: Teacher Survey Compare practices, Content, Alignment Measure change

© 2005 CCSSO ALL RIGHTS RESERVED.

The teacher survey tool… Collects data on what content is taught and how it is taught. Produces reports indicating the degree of alignment between the taught (enacted) curriculum and state assessments or standards. Collects data on teacher beliefs, readiness to teach the content and readiness to instruct special groups of students. Provides a rich source of information to support teachers analysis of student learning challenges.

© 2005 CCSSO ALL RIGHTS RESERVED. The SEC can help your schools… Align curriculum, instruction, and assessment including vertical alignment Improve instruction within and across grade levels, Determine professional development needs Focus teacher planning and development on instructional practice and its effects on student performance Focus coaching and mentoring activity around content and practice

© 2005 CCSSO ALL RIGHTS RESERVED. Help teachers monitor changes in practice as a result of implementing new curricula. Help teachers and leaders determine areas of need when planning for professional development. Help teachers see areas for deepening content expertise and knowledge of how students learn that content. Help teachers identify gaps/redundancies in curriculum –weak materials to support instruction The SEC can …

© 2005 CCSSO ALL RIGHTS RESERVED. SEC Survey Administration Procedures for completing on-line surveys Procedures for completing paper surveys

© 2005 CCSSO ALL RIGHTS RESERVED. Online Survey Administration Approximately minutes to compete May be completed in multiple sittings Data is saved as each section is completed

© 2005 CCSSO ALL RIGHTS RESERVED. SEC Online Registration

© 2005 CCSSO ALL RIGHTS RESERVED. SEC Online Registration Registrar Area

© 2005 CCSSO ALL RIGHTS RESERVED. Your data is saved each time you click on a Submit button.

© 2005 CCSSO ALL RIGHTS RESERVED. You may log-off at any time by simply closing your browser.

© 2005 CCSSO ALL RIGHTS RESERVED. Your progress will be noted on survey menu when you return.

© 2005 CCSSO ALL RIGHTS RESERVED. SEC Contacts Rolf Blank, CCSSO, SEC Collaborative – – Andrew Porter, Vanderbilt University WCER, SEC Online Survey and Reports