1. San Diego County Office of Education
Smarter Balanced Digital Library and Interim Assessments:  Resources to Support Implementation
San Diego County Office of Education
August 10, 2015
Mary Tribbey Senior Assessment Fellow

2. Topics Interim Assessments Digital Library
Considerations for Implementation

3. Assessment Cycles by Purpose
Figure 8.4. Assessment Cycles by Purpose
Take a moment to browse this graphic illustrating the assessment cycles by purpose. Notice the range of assessments operating in different cycles: short, medium, and long, providing different levels of details about student learning over time to be used for various decision-making purposes.
All assessment cycles should provide a continuous picture of student learning, yielding data about the extent to which students have met or are on the way to meeting learning goals.
Remind participants that the three components of the Smarter Balanced Assessment System provide tools and resources for all three assessment cycles.
Interim assessments occupy a middle position between short-cycle formative assessment and long-cycle summative assessment. They may be used “for” instruction to inform next steps, and/or may also be used “of” learning for evaluative purposes.
Formative Assessment
Interim/Benchmark
Large-scale Summative
Adapted from Herman and Heritage (2007)
California Department of Education (2014)
English Language Arts/English Language Development Framework for California Public Schools: Kindergarten Through Grade Twelve
© 2015 Sacramento County Office of Education | K–12 Curriculum & Instruction | SBAC Formative Assessments |

4. Summative assessments
benchmarked to college and career readiness
(Grades 3-8 and 11)
Teachers and schools have information and tools to improve teaching and learning
Standards set
expectations on path to college and career readiness
All students graduate college and career ready
Digital Library: Formative assessment tools and practices
for teachers to improve instruction
Interim assessments Flexible, open, used for actionable feedback

5. Interim Assessments Support Teaching and Learning
Elicit evidence of student learning to inform teaching and learning.
Engage in professional learning.
Deepen teacher content knowledge.
Evaluate grade level instructional progress and plan adjustments to increase student learning.
Observe student use of the testing platform and accessibility supports with actual test items.
© 2015 Sacramento County Office of Education | K–12 Curriculum & Instruction | SBAC Interim Assessments

6. Interim Assessments
Designed as resources to help teachers determine how students are progressing throughout the school year.
Interim assessments are provided free for local educational agency (LEA) use.
Available for Grades 3 through 8 and high school; ELA and mathematics
Uses are determined by the LEA.
Reminder − practice and training tests remain available and there is no limit on their use.

7. Interim Assessment Types
Interim comprehensive assessments (ICAs)
Assess the same standards and claims as the summative assessments
Can administer up to three times per year
Interim assessment blocks (IABs)
Assess smaller sets of targets or specific areas
Allows flexibility by selecting blocks aligned with curriculum and instruction

8. Administration of ICAs and IABs
Administered online (no paper/pencil administration)
Same teacher registration process and test delivery interface as the summative assessment
Fixed length test format in
Both of the tests are administered on line. Teachers will register one time and use the same login for the summative assessment and the Interim Assessments. Test administrators and students will experience the same interface as they will during the summative assessment. When the tests are administered is up to the LEA, but during this initial rollout phase, it’s important to remember that the item pool from which the IABs and ICAs draw is limited. That means if a district uses both the IABs and the ICAs, it is likely that students will see the same items in both tests and their results might be influenced by having seen the same item multiple times. Districts need to think this through when making agreements with teachers about the purpose and use of the tests locally.

9. Interim Comprehensive Assessments (ICAs)

10. Interim Assessment Blocks (IABs)

11. Access to the Interim Assessments
Access to the Interim assessments is restricted to test administrators (TAs), as defined in the California Code of Regulations.
Interim assessments are not secure − TAs may view tests and use scoring materials for professional development purposes.
Interim assessments are not public − Only viewable or shareable with anyone who is identified as a user in the Test Operations Management System (TOMS).

12. TOMS
Access to TOMS is provided to site coordinators and test administrators (TAs) by the LEA CAASPP coordinator.
Access to TOMS requires a current-year CAASPP Test Security Agreement or Affidavit that is signed and on file with the LEA

13. Student Demographics and Settings
All students, grades K through 12, are eligible to take interim assessments
Demographic information (e.g., grade, SSID, program participation) in TOMS needs to be reviewed and updated, if needed, for all students.
Online test settings need to be configured and reviewed for all students in TOMS to ensure students receive the appropriate accessibility supports.
See the Administration of the Online Interim Tests Resource Guide for additional information.

14. Accessibility Supports
Who does the work of identifying and entering test settings?
Do principals know that schools should be paying attention to these test settings?
Teachers can verify whether students are benefitting from the appropriate supports.

15. Preview the Tests
TAs can access and view the interim assessments for professional development and/or training purposes.

16. Preview the Tests f

17. Preview the Tests f

18. Steps to Preview the Interim Assessments
Hover over TA Resources tab
Select Interim Assessments
Select Interim Assessments Viewing System
Select appropriate grade level
Choose from available tests
Choose form
Answer questions to move forward

19. Questions?

20. Interim Assessment Scoring
Most interim assessment items are scored by the Smarter Balanced test delivery engine.
Hand scoring of constructed-response items and performance tasks is a local responsibility.
Results for individual students are generated only after the constructed-response item scores and performance task scores are input into the Interim Assessment Hand Scoring System (IAHSS). (Processing time of up to 48 hours.)
Scoring is supported by training guides, rubrics, exemplars (i.e., anchor and practice papers).

21. Training Guides and Exemplars
Training Guides and Exemplars for all constructed response items and performance tasks are accessible in TOMS.
LEAs can access these materials for professional development and training purposes.

22. Training Guides and Exemplars
Also, archived information and training webinars

24. Preview the Interim Scoring Materials
Select TOMS button
(Log in to TOMS)
Select the Help button
Select the Training tab
Select appropriate grade level
Choose from available items

25. Interim Assessment Hand Scoring System

26. Interim Assessment Hand Scoring Home Page
Once you log in to the interim assessment hand scoring system, the home page or the page you land on is called the Response List and it looks like this. Your Response List can be many pages long depending on how many tests have been administered and how many student responses are in the system that require local scoring.

27. IABs That Do Not Require Local Hand Scoring
ELA, all grades
Edit/Revise
Listen/Interpret
Research
Mathematics, all grades
All blocks, except Performance Task blocks

28. Questions?

29. Interim Assessment Reporting System
LEA CAASPP coordinators provide users access to the Interim Assessment Reporting System to view student data.
Users then have access to student Personally Identifiable Information (PII) and results for all students at a site or LEA.

30. Interim Assessment Reports of Results
Reports for lists of students and individual student reports. No aggregate reports.
ICA: Overall scale score (i.e., 2000 to 3000); Achievement Level (Level 1, 2, 3, or 4); and Claim level (Below, At/Near, or Above Standard)
IAB: Level of Below, At/Near, or Above Standard for each Block
Exportable data file

31. Please Remember!
Scores and achievement levels from the interim assessments cannot and should not be compared with data from STAR Program or CAHSEE.
Different standards are assessed.
Different assessment methods and score scales are used.
Different levels of cognitive rigor are assessed.

32. Interpreting Interim Assessment Results
The extent to which interim assessments were administered under standardized conditions will affect the interpretations.
Interim assessment results should only be used in combination with other evidence of student learning; they should not be the sole basis for making any high-stakes decisions.

33. Interim Assessment Reporting System
Requires same logon credentials used for the Digital Library:
User Name: LEA Password: ****** (6 characters)
Data Warehouse
Receive and view student reports
LEAs need Digital Library logins to see the reports.

34. Reporting Home Page List of Districts Search
Once you log in to the Interim Assessment Reporting System, this is the landing or home page.
All the LEAs is California are listed on this page. You can use the search box, here to the upper right, to more quickly find your LEA instead of scrolling through the list of districts, which is displayed on the left.
However, one can only access data for their own LEA. Notice everything is grayed out and the words “Interim Data Only” or “No Data Available” are visible here. It is only when you go to your own LEA, and select the name of your LEA that you will navigate to a screen where you can see your results.

35. ICA − List of Students
Note the blacked out boxes are the names of students…

36. ICA − Student Report

37. IAB − List of Students

38. IAB − Student Report
On this screen, we have a sample of the Individual Student Report for the ELA/literacy block Edit/Revise. In this particular example, the student took this block once. If they took it a second time, you would see the most recent results listed first and then these particular results would move down to the Previous Results part of the report.
Remember that the results for the blocks are reported in terms of the three classifications, above standard, at or near standard, or below standard. In this example, for Edit/Revise, this grade four student is above standard for this block. If the student were administered other blocks, they would similarly show up on this report. Each blocks shows up in its own little square of this report.

39. Interim Assessment Blueprints
Interim Comprehensive Assessments (ICAs)
Because the ICAs are built to the same blueprints as the summative assessments, use the summative blueprints found at:
Interim Assessment Blocks (IABs)
IAB blueprints found under Additional Resources at: .
Connecting the interims to the standards…

40. ELA IAB Fixed Blueprint

41. Cognitive Rigor Matrix
This matrix from the Smarter Balanced Content Specifications for ELA draws from both Bloom’s (revised) Taxonomy of Educational Objectives and Webb’s Depth-of-Knowledge Levels below.
The Cognitive Rigor matrix combines two common taxonomies that categorize levels of cognition and shows how the Smarter Balanced depth of knowledge categories relate to these taxonomies. As shown here, the concept of remembering information only relates to the first depth of knowledge level. In contrast, evaluation does not relate to either of the first two levels of depth of knowledge. This table, which can be found in the Smarter Balanced Content Specifications for ELA, is a useful aid for guiding the development of items at different depth of knowledge levels.
Now let’s examine the item specifications.

42. ELA Claim 1
Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts.
Targets 1–7 correspond with literary texts
Targets 8–14 correspond with informational texts
The assessment targets incorporate the content clusters from the Common Core State Standards
From the content specifications:
For Claim 1, there are fourteen assessment targets.
{+}
Assessment targets 1-7 focus on reading literary texts.
Assessment targets 8-14 focus on reading informational texts.
Assessment targets are linked to the content clusters within the Common Core State Standards. Now let’s look at a specific target for Claim 1.

43. ELA IAB Fixed Blueprint
Here are the seven assessment targets that the content specifications referred to.
Let’s focus on Reasoning and Evidence.

44. Grade 5 ELA Claim 1 Target 4 4. REASONING & EVIDENCE:
Use supporting evidence to justify interpretations (theme, events, conflicts/challenges, setting, character development/ interactions, point of view)
Standards: RL-2, RL-3, RL-6
(DOK 3)
RL-2 Determine a theme of a story, drama, or poem from details in the text, including how characters in a story or drama respond to challenges or how the speaker in a poem reflects upon a topic; summarize the text.
RL-3 Compare and contrast two or more characters, settings, or events in a story or drama, drawing on specific details in the text (e.g., how characters interact).
RL-6 Describe how a narrator’s or speaker’s point of view influences how events are described.
This grade 5 target focuses on reading literary text.
{+}
Notice that several standards are associated with this target. There is not a one-to-one correspondence between a given assessment target and a standard. Instead, assessment targets focus on skills and knowledge that cross a cluster of standards.
Also note that the depth of knowledge level that must be applied to achieve this assessment target is also specified.

45. Questions? Ask: What are one or two things that jumped out at you?
What one or two wonderings might you have?
(Two minutes)
Popcorn around the rooms and share before moving on…

46. Reflection Think about implementation:
Who needs to learn about these tests?
What are the implications for administrator and teacher training?
What resources might your “champions” need for this work?
What existing collaboration structures might you use?
What obstacles might you face?

47. Assessment Cycles by Purpose
Figure 8.4. Assessment Cycles by Purpose
Formative Assessment
Interim/Benchmark
Large-scale Summative
Adapted from Herman and Heritage (2007)
California Department of Education (2014)
English Language Arts/English Language Development Framework for California Public Schools: Kindergarten Through Grade Twelve
© 2015 Sacramento County Office of Education | K–12 Curriculum & Instruction | SBAC Formative Assessments |

48. Summative assessments
benchmarked to college and career readiness
(Grades 3-8 and 11)
Teachers and schools have information and tools to improve teaching and learning
Standards set
expectations on path to college and career readiness
All students graduate college and career ready
Digital Library: Formative assessment tools and practices
for teachers to improve instruction
Interim assessments Flexible, open, used for actionable feedback

49. Digital Library On-line collection of resources:
?? COE
date
On-line collection of resources:
Aligned with the intent of the Common Core State Standards
Contributed by educators for educators
Vetted by a national network of educators against a set of quality criteria
Supports implementation of the formative assessment process
Incorporates collaboration features
Supports diverse groups of learners
Senior Assessment Fellow, Mary Tribbey

50. What the Digital Library Is Not…
Rocklin USD
September 21, 2014
a bank of assessment items
a learning management system
a library for the general public
a site to freely post resources
First, let’s make clear what the Digital Library is not:
It is not a bank of items; you cannot use the Digital Library to create a test, although you might find a test in the library that will meet your needs.
It is not a learning management system. This is a library of resources that help teachers learn to use or improve the use of assessment as a tool to improve teaching and learning.
It is not a library available to the general public. The Digital Library has been licensed by the state of California to support its K-12 teachers in the use of formative assessment and requires that each user have his or her own login and password.
It is not a site to freely post resources. Before they are posted, each of the resources must meet strict criteria, and be vetted by the Statewide Network of Educators in order to be accepted.
Senior Assessment Fellow, Mary Tribbey

51. Criteria for Resources
Rocklin USD
Criteria for Resources
September 21, 2014
Aligns with Common Core State Standards
Demonstrates high-quality instruction
Addresses learner differences
Is engaging/user-friendly
Incorporates formative assessment practices
Each of the resources accepted into the Digital Library have met these, and other, criteria.
They must align with the Common Core state standards.
They must demonstrate high quality instruction, address learner differences, and be engaging for teachers and students.
And, because this is the whole point of the Digital Library, each reasourc MUST incorporate formative assessment practices.
So what do we mean by formative assessment and formative assessment practices?
Senior Assessment Fellow, Mary Tribbey

52. Definition of the Formative Assessment Process
?? COE
date
Definition of the Formative Assessment Process
Formative assessment is a deliberate process used by teachers and students during instruction that provides actionable feedback that is used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.
This is the definition that underpins the resources found in the Digital Library.
(Note: Read the definition and stress the words in bold.)
As we’ve stated, each of the resources offered in the Digital Library are intended to support the use of assessment to guide teaching and learning. This process is also called “assessment FOR learning,” as distinguished from “assessment OF learning.”
Senior Assessment Fellow, Mary Tribbey

53. Four Attributes of the Formative Assessment Process
?? COE
date
Four Attributes of the Formative Assessment Process
Clarify
intended
learning
Elicit
evidence
Act on
Interpret
All of resources found in the Digital Library must have or support at least one of the attributes of the formative assessment process, shown here.
Clarify Intended Learning: This attribute covers learning goals and success criteria. An example of a resource with this attribute would be one that helped teachers or students to create student friendly language for the skills the students are expected to learn and demonstrate.
Eliciting Evidence: This attribute covers the process by which the teacher designs what the students will be doing in order to surface the students’ knowledge and skills. Crafting a question for an exit ticket that will inform the teacher about individual student learning would be an example of this attribute.
Interpreting Evidence: This attribute covers the process by which the teacher reflects on each student’s learning and progress toward the intended learning and identifies any gaps
Acting on Evidence: This attribute covers the next steps the teacher will take in light of what the students have and have not learned. At this point, the learning goal, or intended learning, might be differentiated for students and the process begins again.
Senior Assessment Fellow, Mary Tribbey

54. The Process of Formative Assessment
Feedback lope is pivotal to the process
Also pivotal is knowing the crucial steps of the learning progression (points in this graphic where you cannot proceed to the next step without giving feedback, to close the gap, and give differentiated instruction.

55. How It Works: Filtering, Searching, Favorites and Collaboration
Basically a gigantic search engine.

56. Assessment Literacy Modules
Assessment Literacy Module: Understanding the Formative Assessment Process
Experts in the field
Examples of each attribute
Resources
Want you to see how to make effective use of the interims as part of a larger system; need to know what that larger systems; want classroom practice to change – want to makes sure teachers are aware of these other pieces and that the interims and summative validate and verify what they are doing in the classroom with kids.
Show #8, which has the researchers quotes
If time,
Show #10, and stop after each attribute to let the audience talk

57. Digital Library Resources
Cognitively Based Assessment of Learning (CBAL)
Using a Learning Progression to Formatively Assess the Concept of Slope 
The Using a Learning Progression to Formatively Assess the Concept of Slope webinar is intended for teachers of first-year algebra. This webinar provides an example of a CBAL formative digital assessment task in the Linear Functions and Nonlinearity module that incorporates a learning progression to develop students’ deep understanding of the concept of slope. Viewers hear how the learning progression, its application in the task, and ancillary resources support the teacher’s use of the task in the classroom.

58. Digital Library Connection
Using results from either the summative or interim assessments, use the Digital Library to identify resources such as:
Using Fluency Stations as Formative Assessment (RF 3.4 and RF 4.4)
fluency-stations-formative-assessment

59. Other Examples
Rocklin USD
September 21, 2014
Informational Text Learning and Formative Assessment Strategies
Accommodating ELLs: Thinking Strips to Document Reading Comprehension
Aligning Formative Assessment with SBAC ELA Claim 1 Assessment Targets
The Components of Effective Feedback
Senior Assessment Fellow, Mary Tribbey

60. Questions?

61. Reflection Think about implementation:
Who needs access to the Digital Library?
What are the implications for administrator and teacher training?
What existing collaboration structures might you use?
How might this align with other professional learning?
What obstacles might you face?

62. For Further Information
California Department of Education
CAASPP Office
Smarter Balanced Web Page
CalTAC
(for Digital Library: )

64. IAB Blueprints – Mathematics

65. California Content Standards

66. California Content Standards