1. SAT and Accountability Evidence and Information Needed and Provided for Using Nationally Recognized High School Assessments for ESSA Kevin Sweeney, College Board National Conference on Student Assessment Austen, TX June 2017

2. SAT and Accountability

3. Overview SAT structure and scores
Points of Conflict Between Accountability and Admissions Uses

4. SAT Structure and Scores
SAT and Accountability
SAT Structure and Scores

5. SAT Structure and Scores
SAT and Accountability
SAT Structure and Scores
Section Scores are placed on a vertical scale.
Test, Cross-Test Scores and Total Score are also on a vertical scale.

6. SAT and Accountability
SAT Structure and Scores
Overall, in terms of content and technical characteristics, the SAT is well aligned to meet state accountability needs
College and Career Ready Benchmarks
Vertical scale supporting various growth models
Content alignment studies
However, there are some areas that where there are potential points of conflict

7. SAT and Accountability
Points of Conflict
Accommodations
Benchmarks versus Performance Levels
Native Language Assessments
Tension between an off-the-shelf test and customized needs

8. SAT and Accountability
Accommodations
Tension between fairness and invalidity
Historically, CB has focused on invalidity
Because of input from states, we have recently expanded our allowable accommodations
Ideally, accommodations level an “uneven playing field”
Typically, empirical evidence to support accommodations policy is scarce
What to use as basis for policy?
How to handle conflicting accommodations policies between states?

9. SAT and Accountability
Benchmarks versus Performance Levels
CCR benchmarks derived from empirical prediction of student academic performance
Performance levels typically derived from judgmental standard setting process, grounded in achievement level descriptors
No a priori reason for existing benchmarks to equal Performance Levels coming out of standard setting
If a state wants a certain cut score to be the CCR benchmark it must be explicitly incorporated into the achievement level descriptors, and into the standard setting process
Hope is not a strategy

10. SAT and Accountability
Benchmarks versus Performance Levels
Including benchmarks may create results discrepant with standards in earlier grades.
Alternately, if a state does not incorporate the CCR benchmarks, they must be ready to explain why their Proficient Level (for example), is higher or lower than the national benchmark.

11. SAT and Accountability
Native Language Assessments
According to ESSA States are obliged to:
Identify languages that are present to a significant extent
Identify languages for which assessments are needed
Identify at least the most populous language other than English that is most spoken in the state
Articulate the process by which the languages are identified
Articulate the reasons for why assessments in other languages are not developed “despite making every effort”
States are NOT obliged to:
Develop a specific number of assessments in other languages

12. SAT and Accountability
Native Language Assessments
While we do offer glossaries and various other accommodations, the SAT is currently offered in only English.

13. SAT and Accountability
Tension between off the shelf test and Customized test needs
The SAT is developed for use in national administrations using data from across the country
This provides very good information for college admissions purposes
However, this presents a tension for states that have an interest/desire/need to assure that their students are represented in the development process (e.g., included in pre-test item statistics)

14. SAT and Accountability
Tension between off the shelf test and Customized test needs
Similarly, because the essay is optional, a score that combines the essay with the Reading and/or Writing sections is not possible at the national level
However, some states have a need for an ELA score that includes a student produced written component
Again, this produces a tension between the needs of customization and the constraints of off-the-shelf

15. SAT and Accountability
Conclusion
The SAT solves many problems for state accountability testing
In some cases it is an imperfect solution
The points of conflict arising form this imperfection need to be worked on jointly by College Board and their state partners

16. SAT and Accountability
Thank You!
Kevin Sweeney
Questions???