1. 2015 DAC Meetings
2015 February DAC Meetings
KDE:OAA:DSR: 3/10/2015

2. Agenda College/Career Readiness A Quick Look to the Future ACT Compass
2015 DAC Meetings
Agenda
College/Career Readiness
ACT
Compass
KYOTE
Career Readiness
A Quick Look to the Future
Proposed Regulation Changes
Fall 2015 Reporting
Hot Topics
Test Security/Data Quality
Accountability
SDRR
Text-Reader Accommodation
K-PREP
Alternate K-PREP
KDE:OAA:DSR: 3/10/2015

3. 2015 DAC Meetings
Hot Topics
KDE:OAA:DSR: 3/10/2015

4. Hot Topics
2015 DAC Meetings
Continued emphasis on “getting the rosters right” to support data quality.
The Infinite Campus EOC Early Warning Report highlights students with missing EOC test scores. Senior students with missing EOC scores that are scheduled to graduate (i.e., G-Code), but do not have a non-participation exemption must be tested.
KDE:OAA:DSR: 3/10/2015

5. 2015 DAC Meetings
Hot Topics
These seniors without EOC scores or a non-participation will be added to SDRR rosters for cleanup (i.e., non-participation, accountability). Use the additional non-participation reasons for EOC (i.e., took course out of state/private, EOC Graduation Issue).
At the end of data review, students remaining on any test roster, without an approved exemption or a score from the vendor, will have a zero score applied in reporting.
KDE:OAA:DSR: 3/10/2015

6. Hot Topics
2015 DAC Meetings
Revised Administration Code and Inclusion of Special Populations regulations are in effect. These 2014 versions govern test administration beginning this year ( ).
Science testing for K-PREP grades 4 and 7 is the Norm-Referenced Test (NRT) section only (previously labeled Part A) . NRT scores will be reported, but not included in accountability. At high school, EOC Biology continues and it is still included in accountability.
KDE:OAA:DSR: 3/10/2015

7. 2015 DAC Meetings
Hot Topics
New Alternate K-PREP science attainment tasks at all levels will be administered during the second window. These are based on new science standards. Scores will be reported, but not included in accountability.
Text-reader is provided online for read aloud/reader for K-PREP (Pearson) in grades 3-8.
·       
KDE:OAA:DSR: 3/10/2015

8. 2015 DAC Meetings
Hot Topics
ACT has increased the capacity for EOC Computer-Based Tests to allow for more concurrent test-takers. Return of paper/pencil testing scores will remain (5 days from receipt of answer documents at ACT).
·       
KDE:OAA:DSR: 3/10/2015

9. Hot Topics
2015 DAC Meetings
Test windows are required to KDE, submitted through SDRR. Necessary to keep district calendars up to date in Infinite Campus.
Shipping Windows to Pearson
Testing Windows to KDE
5 consecutive days in the last 14 instructional days
May vary by level (E, M, H) or be a single window for all
Used to verify accountability dates (1st day of 5 day window) and site visits
Due in SDRR (March)
Option #1:   4/15 – 5/1
IC pull 2/13. Materials by 3/30.
Option #2: 5/4 – 5/15
IC pull 2/27. Materials by 4/14.
Option #3:  5/18 – 6/1
IC pull 3/13. Materials by 4/28.
Entered in PearsonAccess.
KDE:OAA:DSR: 3/10/2015

10. Test Security and Data Quality
2015 DAC Meetings
Test Security and Data Quality
Key assessment themes require the focus of schools, districts and the state
KDE:OAA:DSR: 3/10/2015

11. Key Assessment Themes
2015 DAC Meetings
TEST SECURITY—checks and processes to monitor appropriate administration of state-required tests
DATA QUALITY—collaborative process between OAA and school/district staff to ensure accurate reporting of test results
KDE:OAA:DSR: 3/10/2015

12. Test Security
2015 DAC Meetings
Continues to be a critical issue in Kentucky and nationally.
Loss of certification for Kentucky educators emphasizes the seriousness of the issue.
OAA implements and monitors the recommendations from Kentucky’s security audit.
Increased analysis of test results for irregularities and triggers for follow up from KDE’s allegations coordinator.
KDE:OAA:DSR: 3/10/2015

13. Test Security: Local Requirements
2015 DAC Meetings
Test Security: Local Requirements
Training
Administration Code and Inclusion of Special Populations (2014 versions)
Signatures
Student Honor Code
Nondisclosure Agreement
Appropriate Practices Form and Administration Code
DAC/BAC testing certification
Documentation of Testing
Seating charts (all required tests)
Testing room location codes (K-PREP Pearson)
DAC and BAC Testing Certification forms will be sent in a DAC Monday .
KDE:OAA:DSR: 3/10/2015

14. Test Security: Training
2015 DAC Meetings
Test Security: Training
KDE:OAA: DSR: 8/7/2012
Requires annual training on regulations
Administration Code
Inclusion Regulation
Administration Code
703 KAR 5:080
Inclusion of Special Populations 703 KAR 5:070
All persons involved with testing in any way
Anyone assisting with testing accommodations
Requires signature to confirm individual has read, understands and will abide by document
Documentation of training required (who, what, when, where and how)
KDE:OAA:DSR: 3/10/2015

15. Test Security: Signatures
2015 DAC Meetings
Test Security: Signatures
Appropriate Practices Form
Student Honor Code
Spring 2015
The Student Honor Code is new for BACs need to make test administrators aware of this new code. Specific directions are provided in the scripts of the TAMs.
KDE:OAA:DSR: 3/10/2015

16. Test Security: Signatures
2015 DAC Meetings
Test Security: Signatures
Certification Form DACs and BACs
from KDE/OAA
Available on KDE website
Completed at end of year to reflect all required testing in the school year
DACs collect from BACs
KDE collects from DACs
DAC and BAC Testing Certification forms will be sent in a DAC Monday . Copies are included in the meeting materials.
KDE:OAA:DSR: 3/10/2015

17. Test Security: Documentation of Testing
2015 DAC Meetings
Test Security: Documentation of Testing
Seating Charts are required for
all state-required test administrations.
optional tests that are included in accountability if the test is administered by the school.
Seating Chart— Large group
KDE:OAA:DSR: 3/10/2015

18. Test Security: Documentation of Testing
2015 DAC Meetings
DAC will collect and maintain in case of potential investigations.
Templates are provided by KDE and may be modified at the local level.
Seating Chart— Chart—Small group & individual students
KDE:OAA:DSR: 3/10/2015

19. Test Security: Documentation of Testing
2015 DAC Meetings
Test Security: Documentation of Testing
Collection of testing room location codes on each Student Response Booklet allows security data analysis at the room level.
KDE:OAA:DSR: 3/10/2015

20. Test Security: State Monitoring
2015 DAC Meetings
Test window visits
Review of seating charts
Collection of DAC certification forms
Review/audit of randomly collected AAA Folders for alternate assessment students
Analysis of outlier test results by Office of Education Accountability
Analysis by vendors (i.e., erasure and others)
Pearson has hired Caveon to analyze performance data analysis
KDE:OAA:DSR: 3/10/2015

21. Test Security: State Monitoring
2015 DAC Meetings
Test Security: State Monitoring
Analysis by vendor can produce multiple types of information/reports at room, school or district levels.
Erasure analysis (Right (R) to Wrong (W), RTR, WTR)
Large score changes for a student
Large mean score increases
Probability of matched response analysis
Previous year’s distribution around scores
Pattern analysis
Comparison of individual student scores by item type
Analysis can trigger/identify schools and districts for follow up with Allegations Coordinator.
Examples of possible reports and analysis are listed.
KDE:OAA:DSR: 3/10/2015

22. Test Security Reporting of Allegations
2015 DAC Meetings
Test Security Reporting of Allegations
Get the facts straight.
Record the date and time of the incident.
List all involved (i.e., staff, volunteers, students).
Identify content area and test parts impacted.
Identify specific content question(s), if applicable.
Record what you learn.
Report the incident.
KDE:OAA:DSR: 3/10/2015

23. Test Security Reporting of Allegations
2015 DAC Meetings
Test Security Reporting of Allegations
Support Documents
Testing Allegation Reporting Form
Notarized Statements written and signed by each individual involved (teacher, proctor, BAC, principal and/or others). One statement signed by multiple individuals is not acceptable.
Testing Schedule
Seating Chart and Room Number
KDE:OAA:DSR: 3/10/2015

24. Test Security Reporting of Allegations
2015 DAC Meetings
Test Security Reporting of Allegations
Support Documents
Verification of Administration Code Training
Verify/Proctor Signature Sheet
Nondisclosure Agreement
Student Name & SSID
For an allegation involving special education:
verification of student test accommodation(s).
verification of Inclusion of Special Populations training.
a signed copy of the Code of Ethics, record of testing room and test administrator for allegations regarding alternate assessment.
KDE:OAA:DSR: 3/10/2015

25. Key Assessment Themes
2015 DAC Meetings
TEST SECURITY—checks and processes to monitor appropriate administration of state-required tests
DATA QUALITY—collaborative process between OAA and school/district staff to ensure accurate reporting of test results
KDE:OAA:DSR: 3/10/2015

26. Data Quality
2015 DAC Meetings
The authoritative source of student data in Kentucky is the state student data system, Infinite Campus (IC).
Data are downloaded from IC for vendors and to populate internal applications (i.e., SDRR).
Accountability change requests made to OAA are cross-checked with Infinite Campus.
KDE:OAA:DSR: 3/10/2015

27. Data Quality Keys to Improved Data Matching
2015 DAC Meetings
Keys to Improved Data Matching
Correct entry of student information in IC is critical (name, date of birth and SSID).
Consistently use the student’s name as it appears in IC on all testing records.
State-testing programs are provided SSIDs.
KDE:OAA:DSR: 3/10/2015

28. Data Quality
2015 DAC Meetings
KDE/OAA Student Data Review and Rosters (SDRR) Application, organized by test, has two critical data accuracy functions:
Rosters
Required by regulation
Testing information for some tests
Accountability record for all grades 3-12
(Grade 9 is needed for ACT Plan and grade 12 is needed for College/Career Readiness reporting.)
Data Review (cleanup before reporting)
KDE:OAA:DSR: 3/10/2015

29. Data Quality REMOVE from Roster INCLUDE on Roster For End of Course:
2015 DAC Meetings
Data Quality
Accurate
Account-ability
Report-ing
Get the Rosters Right
INCLUDE on Roster
Tested students with scorable documents returned to vendor
Not tested students:
1) With non-participation exemption (Extraordinary or Medical)
2) Who should have tested, but did not test
REMOVE from Roster
Not tested students that have withdrawn from the school
For End of Course:
1) Not tested students that are still completing coursework
2) Added students with missing scores that are NOT seniors
KDE:OAA:DSR: 3/10/2015

30. Data Quality
2015 DAC Meetings
Accurate
Account-ability
Report-ing
Get the Rosters Right
Students left on roster, after data review, without an approved exemption or test score, will be included in reporting as a zero (0) score.
KDE:OAA:DSR: 3/10/2015

31. Data Quality Importance of Rosters Clean Rosters in Spring
2015 DAC Meetings
Data Quality
Importance of Rosters
Clean Rosters in Spring
Longer roster window means more time for work.
Flexible routine changes include student add or delete.
Demographic changes made directly in Infinite Campus.
Better availability of school staff to assist.
Easier Data Review and Cleanup in August
Data review changes require documentation and KDE approval.
Data review goal is small number of changes—”ones or twos”.
Accurate Data for Public Release
KDE:OAA:DSR: 3/10/2015

32. 2015 DAC Meetings
Accountability Rules
How to determine whether students are in a school’s accountability report
KDE:OAA:DSR: 3/10/2015

33. Accountability Rules Tested Accountable
2015 DAC Meetings
Accountability Rules
Tested
Every student enrolled on the first day of the 5-day spring summative test window must be tested for participation rate purposes.
Accountable
Accountability determined by full academic year (any 100 instructional days of enrollment). The first day of the 5-day spring summative test window is included in the 100 day count.
KDE:OAA:DSR: 3/10/2015

34. 1st day of the 5-day Summative Test Window
Accountability Rules
2015 DAC Meetings
The first day of K-PREP five-day spring summative window (in the last 14 instructional days of the district calendar) is the point in time where the 100 days of enrollment (full academic year) is determined.
April-June
K-PREP (Pearson)
1st day of the 5-day Summative Test Window
Across
Year
K-PREP EOC
Grade 12 CCR
March
ACT
Nov-Dec
April-May
Alternate
K-PREP
September
(Prior year) EXPLORE
PLAN
The 100-days count is from the first day of the five day consecutive test window(s).
KDE:OAA:DSR: 3/10/2015

35. Accountability Terms for Schools
2015 DAC Meetings
Accountability Terms for Schools
“A1” means a school under administrative control of a principal and eligible to establish a school-based decision-making council.
“A1” school is not an alternative program operated by, or as a part of, another school.
“Alternative” program means all other education programs not classified as A1.
KDE:OAA:DSR: 3/10/2015

36. START NO NO NO YES YES YES
2015 DAC Meetings
Flowchart for Determining Accountability For Students in A1 Schools
Kentucky Performance Rating for Educational Progress (K-PREP)
Every student enrolled on the first day of the five-day spring summative test window is tested for participation rate purposes. Accountability is determined by full academic year (any 100 days of enrollment) including the first day of the five-day window.
START NO NO
Has this
student not been
in any one school but
has been enrolled in
a district for 100
instructional days
this school
year? NO
Has this
student been
enrolled in
another school for 100
instructional
days this school
year?
Has this
student been
enrolled in
this school for 100
instructional days
this school
year?
Student scores
will count
in state’s
accountability
only.
YES
YES
YES
Student scores
will count
in this school’s
accountability.
Student scores
will count
in other school’s
accountability.
Student scores
will count
in district’s
accountability.
KDE:OAA:DSR: 3/10/2015

37. Kentucky Performance Rating for Educational Progress (K-PREP)
2015 DAC Meetings
Flowchart for Determining Accountability For Students in Alternative Programs
Kentucky Performance Rating for Educational Progress (K-PREP)
START:
Was the student who is in an alternative program enrolled in an A1 school in a district for 100 days this school year ? (Note: The 100 day count includes the first day of the five-day spring summative test window.)
YES NO
Was student placed by a
governmental
agency or the
court system?
Accountable to A1 school
Accountable to A1 school
where the student would
have attended if not
enrolled in the alternative
program(s) or to district
(if student would have
attended multiple A1
schools).
Was the
student enrolled
100 days
in the A1(or multiple A1s in a district)
and alternative
program(s) combined or
in the alternative
program(s) only? NO
YES NO
YES
Accountable to the state
Note: A student at Kentucky School for the Blind, Kentucky School for the Deaf or Gatton Academy tracks back to the A1 school that he or she would have attended if not in the alternative program.
KDE:OAA:DSR: 3/10/2015

38. ALTERNATIVE Program Scenarios
2015 DAC Meetings
Applying the Accountability Rules
ALTERNATIVE Program Scenarios
KDE:OAA:DSR: 3/10/2015