1. ABE Performance Measures, Accountability and Assessment
Brad Hasskamp| Adult Secondary Credential and Education Policy Specialist
Jodi Versaw| Adult Basic Education Program Quality Specialist

2. The 6 Accountability Measures
Brad

3. 6 Measures for WIOA Success
Measurable skill gain
Employment at second quarter after exit*
Employment at fourth quarter after exit*
Median earnings at second quarter after exit*
Credential earned (and Entering Postsecondary or Employment)*
Employer engagement**
Aggregated performance outcomes analyzed statewide, by consortium and by 11 groups and by age, gender, race.
*These measures will apply for ABE programs starting in 2020.
**This measure is not yet full defined nor being measured for ABE programs.

4. Getting MSG (Measurable Skill Gain)
Heather

5. Measurable Skill Gain Indicator
The Federal Definition:
The percentage of participants who, during a program year, are in an education or training program that leads to a recognized postsecondary credential or employment and who are achieving measurable skill gains, defined as academic, technical, occupational, or other forms of progress, towards such a credential or employment.

6. 4 Types of Measurable Skill Gain for ABE
Secondary Diploma/ Equivalent
Secondary or Post-Secondary Transcript
Educational Functioning Level Gain
Pre-Post Test
Completion of Carnegie Units
Program Exit + Entry into Postsecondary Education
Progress toward Milestones
Passing Technical / Occupational Knowledge Based Exam
Used in Title II

7. Counting Measurable Skill Gains (MSG)
Evaluation Learning Community Training
December 7-8, 2011
Counting Measurable Skill Gains (MSG)
Multiple Ways to Count MSG
(Only one type of gain will be counted for each participant)
1. Earn higher scores on approved pre- and post-tests to advance to a higher level
2. Earn a secondary credential (high school or equivalency diploma) by June 30
3. Enter postsecondary education (only after participant exits ABE) by June 30
4. Participants in “adult high school” can complete Adult Basic Education (ABE) Level 5 by earning enough Carnegie Units or credits to move to 11th- or 12th-grade status*
(*Not yet applicable for ABE in Minnesota)

8. National Reporting System Targets for Measurable Skill Gain
NRS Educational Functioning Level and Core Indicator Goals
MN FY 2017
( )
FY 2019
( )
FY 2020
( )
% Completing Each Level/Goal
% Earning Measurable Skill Gain
Target
MN Actual
U.S. Actual
ALL ABE LEVELS
N/A* 43 44
ABE 1: ABE Beginning Literacy
78% 59 48 61
ABE 2: ABE Beginning Basic Education
67% 46 45
ABE 3: ABE Intermediate Low
50% 39 42
ABE 4: ABE Intermediate High
34% 36 40 41
ABE 5: Low Adult Secondary
26% 52
ABE 6: High Adult Secondary 28
36*
42*
ALL ESL LEVELS
ESL 1: ESL Beginning Literacy
48%
ESL 2: ESL Beginning Low
58% 54 50 56
ESL 3: ESL Beginning High
51% 47 51
ESL 4: ESL Intermediate Low
39% 38
ESL 5: ESL Intermediate High
35% 34 37
ESL 6: ESL Advanced
16% 23 24 25 26

9. Important Notes about the Accountability Measures
The U.S. Department of Education has set overall ABE-level and ESL-level targets.
There are now targets for ABE Level 6.
The U.S. Department of Education is setting higher targets for states. Minnesota is expected to be at the median or higher in all ABE and ESL levels.

10. What questions do you have about MSG, the targets and your ABE program measuring it?
Measurable Skill Gain
Secondary Diploma/ Equivalent
Secondary or Post-Secondary Transcript
Educational Functioning Level Gain
Pre-Post Test
Completion of Carnegie Units
Program Exit + Entry into Postsecondary Education
Progress toward Milestones
Passing Technical / Occupational Knowledge Based Exam

11. Program Improvement

12. Program Improvement
Take Note! Results from program year will be used for a Program Improvement process starting Fall 2019.

13. Program Improvement Q&A
Q: What is Program Improvement? A: A technical assistance process for ABE programs that fall below set benchmarks on key accountability measures. Q: Why Program Improvement? A: To work towards consistent implementation of accountability practices state- wide; to ensure compliance with federal accountability standards.
Program Improvement Q&A

14. Program Improvement Q&A
Q: What accountability measures will Program Improvement be based on for PY data?
Measurable Skill Gains (Level Gains and Diplomas) for ABE students
Measurable Skill Gains (Level Gains and Diplomas) for ESL students
Post-Testing Rates for students with 40+ contact hours
Program Improvement Q&A

15. Program Improvement flags
ABE consortia will be flagged for possible participation in Program Improvement if: Measurable Skill Gains (Level Gains and Diplomas) for ABE students falls more than 30% below NRS target Measurable Skill Gains (Level Gains and Diplomas) for ESL students falls more than 30% below NRS target Post-testing rate for students who have 40+ hours falls below 60%

16. Program Improvement Q&A
Q: How will we know if our consortium is flagged for Program Improvement?
Pay attention to “Level Gains and Post-Test Rates” report in SiD.
MNABE Report Card
Report card for 17/18 will be produced this fall – NO Program Improvement associated with this report card!
Report card for 18/19 will be produced late summer/early fall 2019
Program Improvement Q&A

17. Report card preview – Measurable Skill Gains

18. Report card preview – Post-Testing Rates

19. Program Improvement cohort participation
From all ABE consortia flagged for possible participation in Program Improvement, a group of ABE consortia will be selected for participation in the Program Improvement Cohort.
Factors that will be taken into consideration include:
Total number of flagged consortia
Participant numbers
Unique circumstances

20. Program Improvement Cohort
The Program Improvement Cohort group will participate in (details TBD):
Full-day workshop in Fall 2019
Site visit by state ABE staff
Improvement planning
Technical assistance from state ABE staff

21. Program Improvement Next Steps
Watch for Report Card coming out this fall!

22. Assessments: The how and when of the new tests
Brad

23. Collecting your questions
What questions do you have about ABE assessments, especially the new TABE and CASAS GOALS? (Jot down your questions at your table.)

24. What’s Happening? Date Approved ABE Level Tests
Approved ESL Level Tests
July June 2018
TABE 9-10
CASAS Life and Work
BEST Plus
July 2018 – June 2019
TABE 11-12
CASAS GOALS
July June 2020
Programs can use old tests but may be able to transition to new tests
July 2020 – June 2021
Programs can use old tests but may need to transition to new tests

25. Awkward Test Transitions
New tests are coming and your program needs to switch! : All ABE programs need to buy and start implementing TABE 11/12 and/or CASAS GOALS for ABE level students As of July 2019 CASAS Life and Work will be valid for ESL students but not ABE-level students. Publishers submitting ESL tests for approval this fall to U.S. Department of Education.

26. ABE Spring Application Workshop
TABE Approved!
The U.S. Department of Education’s Office of Career, Technical and Adult Education (OCTAE) has approved TABE Forms 11 and 12 for the NRS and for local ABE programs in Minnesota. Computer and paper versions 3 Subject Tests: Reading, Math, Language 5 Level Tests in each subject: L, E, M, D, A Each test has only 1 option (no survey) TABE 9 and 10 will be dropped from the approved list starting July 1, 2019
Brad

27. TABE Test Times
Subject
Revised Maximum Allowable Test Time (levels E, M, D, A)
Math
75 minutes (2 parts in levels M, D, A timed separately)
Reading
120 minutes (2 parts timed separately)
Language
60 minutes

28. ABE Spring Application Workshop
From TABE 9-10 to 11-12
TABE 9 & 10 are not considered the same as 11 & 12 for data purposes, meaning that if a student has a pretest in 9 or 10, the student: Cannot be post-tested with an 11 or 12. Does not need to wait 40 contact hours before taking a 11 and 12 as a new pre-test to determine a student’s level.
Brad

29. Scoring TABE 11-12
Not all correct answers are +1 point. Some two- part answers have different scores and some items are not worth any points.
Hand scoring will be more difficult than with TABE , but you may be able to scan official test answer sheets for electronic scoring.
It would not be surprising to see students drop levels as the take their first TABE since the levels are different and more challenging with content standards alignment.
Some students may need TABE Level L tests now. The TABE Level L test is intended to be a consumable book.

30. Buying TABE 11/12 Paper Test Books $156/25 test books
$86.50/25 locators
Paper Answer Sheets
$24/25 (test)
$22.50/25 (locator)
Computer Testing
$ /test
Administration
$32.50/11 or 12 Test Directions
$25/Test Administration Manual
$30.50/Scoring Guide

31. CASAS New GOALS Reading Test Approved!
ABE Spring Application Workshop
CASAS New GOALS Reading Test Approved!
The U.S. Department of Education approved the new CASAS GOALS Reading test for ABE levels (not ESL) in the NRS.
Math test still in development.
Same test being submitted for ESL Levels this fall.
Brad

32. Buying CASAS GOALS Reading
Paper Test Books
$100/25 test books or locators
$190/5 test books each form
Paper Answer Sheets
$63/100 (test)
Computer Testing
$ /test
Administration
$40/Test Administration Manual

33. CASAS and TABE Look
Each group will get a copy of new and old CASAS and TABE. Discuss:
What is different between the tests?
What is similar?
What issues with the new tests might your students struggle with?
What might you need to change locally as you implement the new tests?

34. New Assessment Policy See the new Assessment Policy at
that includes:
TABE 11 and 12
CASAS Goals is added to the approved list for ABE 1-6 in Reading
New Educational Functioning Level Descriptors for ABE levels 1-6
Please note that the revised Educational Functioning Level Descriptors for ESL levels are not included since there are no approved tests for the new ESL descriptors yet!

35. SW ABE as Central (Computer) Test Purchaser
The Assessment Training ABE Support Network Provider (SouthWest ABE) is coordinating central purchasing of computer-based tests (TABE 11 & 12 and CASAS GOALS) with local ABE programs This allows programs to get the best bulk purchasing rate. Local programs will need to pay for the tests Purchases made potentially annually or semi-annually, upon demand (Computer-based test voucher purchasing information will be distributed this fall. Look for an from Cherie later this fall.)

36. Assessment Professional Development
Professional development on tests is offered at:
Summer Institute
Regionals
Support Services Conference (November 8-9, 2018)
Sessions Offered:
Test Certification (required for all staff working with testing) will provide an overview of the allowable tests
Critical Issues in Assessment will dive into the new tests, computer-based testing and other topics

37. Answering your Questions
What additional questions did you have about ABE assessments, especially the new TABE and CASAS GOALS?

38. Table Discussion: Test Transition Planning with Local ABE Programs
Start thinking now about when you will transition testing:
When will you purchase tests? Which ones will you purchase?
Which tests will you purchase for ABE students (TABE 11/12 or CASAS GOALS)?
Will you purchase paper and/or computer tests?
How will you train yourself and your staff on the new tests?
Which staff need test certification and which staff just need new test info?
How will your program implement the new tests?
With current students that already have a pretest? With new students?
How will you adjust your program for the longer testing times?