TOPSpro Special Topics Data Detective III: Federal Reporting

Data Detective III Agenda Description: Federal Tables Federal Table 4 Federal Table 4B Federal Table 5 Description: Corresponding Federal Reports Federal Tables Exceptions NRS Level Completion Level Completion Calculations & Special Considerations Persister Report NRS Performance

Federal Tables Federal Tables display accountability information according to standards mandated by the National Reporting System (NRS). TOPSpro uses the Federal Tables to report statewide aggregation totals to the Federal Government.

Federal Tables Guidelines Learner must have at least one Entry Record Learner must be in one of five authorized Instructional Programs – 1) ESL, 2) ABE, 3) ESL Cit, 4) HS Diploma, 5) GED/Spanish GED Learner must have a valid Instructional Level, either from a pretest or from the Entry Record Must have >12 hours of instruction Must be > 16 years of age Cannot be concurrently enrolled in a K-12 program

List of Federal Tables Table Title 1 Participants by Entering Educational Functioning Level, Ethnicity and Gender 2 Participants by Age, Ethnicity and Gender 3 Participants by Program Type and Age 4a Educational Gains and Attendance by Educational Functioning Level 4b/c 5/5a Core Follow-up Outcome Achievement 6 Participant Status and Program Enrollment 7 Adult Education Personnel by Function and Job Status 8 Outcomes for Adults in Family Literacy Programs (Optional) 9 Outcomes for Adults in Workplace Literacy Programs (Optional) 10 Outcomes for Adults in Correctional Education Programs (Optional) 11 Secondary Outcome Measures (Optional) 12 Work-based Project Learners by Age, Ethnicity and Gender (Optional)

Report Series Use the Report Series tab to select which of the Federal Tables you want to display.

Federal Table 4 Federal Table 4 displays Educational Gains by Educational Functioning Level. Specifically, it reports learners who entered program at one Instructional Level and finished the year at a higher level.

Federal Table 4 Column A: Educational Functioning Levels (EFL), are defined by the National Reporting System (NRS) and correlate to CASAS scale score ranges Column B: Total Number Enrolled includes all students who meet Fed Table Guidelines. Enrollees are placed into an EFL from a pretest score or Entry Record field 18, Instructional Level

Federal Table 4 Does the Instructional Level field #18 on the Entry Record still matter?

Federal Table 4 TOPSpro still uses Entry Field #18 as a backup to determine level placement. As always, ensuring you pre- and post-test your students is a much better approach.

Federal Table 4 TOPSpro also uses scores from CASAS appraisals to determine level placement. CASAS appraisals are NEVER used in pre/post pairs. For level placement on the Fed Tables, TOPS will always look for (in order): Valid pretests CASAS Appraisals Entry Field #18.

Federal Table 4 Column D: based on valid paired tests and any gain that results in student achieving the highest possible score in that EFL. Refer to CASAS Relationship to NRS Educational Functioning Level charts for scale score equivalents. Column C: Total Attendance Hours from tests, Update Records, or attendance system import

Federal Table 4 Wait a minute… did I hear you say attendance hours are generated from Updates and tests??

Federal Table 4 Yes. The Federal Tables will look at the aggregated hours across all Test Records as well as the aggregated hours across Update Records and use whichever total is larger. It will not, of course, combine hours across Updates and tests together.

Federal Table 4 Column E: shows those who completed a level and advanced to the next level (also based on paired test scores.) Column F: students who left program before completion (as indicated on Update Record field 7, Status-Left program)

Federal Table 4 Column G: Those students who did not complete the level or leave prior to completion Column H: The percent of students who completed the level in which they began the program year. Comparison of the number who completed the level to total enrollees in that level

Federal Table 4 Federal Table 4 tracks the outcomes of all learners who qualify for federal reporting. Total Enrolled B = Number Completed D + Number Separated F + Number Remaining G. In this example – 3 + 1 + 23 = 27

Federal Table 4 Column H: calculated from columns D and B. In this example, ABE Beginning Literacy shows 3 students completing the level, out of 27 total enrollees (3/27 = 11.11%). The percentage completing a level includes students who did not test, resulting in a lower completion rate. A student cannot demonstrate level completion or advancement without paired tests.

Federal Table 4B Federal Table 4B displays the same level advancement information as Table 4, but it only includes learners who completed a valid pre- and post-test.

Federal Table 4B On Table 4B, column B is limited to students who have valid paired tests. All columns except Total Number Enrolled (B) are determined in the same manner as Table 4.

Federal Table 4B Column H: calculated from columns D and B. In this example, ABE Beginning Literacy shows 3 students completing the level, out of 13 total pre-and post tested enrollees (3/13 = 23.08%). If Column B totals are the same on Table 4 and Table 4B, there is no difference in the Percentage Completing Level (H).

Federal Table 4C Federal Table 4C displays the same level advancement information as Table 4, but it only includes learners who are enrolled in Distance Learning. This table is new for 2007-08.

Federal Table 5 Federal Table 5 displays core performance achievements.

Federal Table 5 Column B: Source is Entry Record, field 12 Column C: Source is Update Record, fields 7 (Status = Left Program), and corresponding achievement from field 9 (Learner Results) Column D example: 2/ 5 = 40%

Federal Table Exceptions Federal Tables Exceptions displays student records that are ineligible for inclusion, sorted by Drop Reason. Many of the same Drop Reasons are on the Payment Points reports.

NRS Level Completion NRS Level Completion lists learners by Educational Functioning Level and indicates which learners completed a level.

NRS Level Completion vs. Payment Points How does level completion on the Federal Tables compare to Payment Points?

NRS Level Completion vs. Payment Points NRS Level Completion does not use the same logic as Payment Points. Note that students who progress within a level can make significant learning gains without level completion or advancement.

Educational Functional Levels

Educational Functional Levels

Level Completion Examples “Completed a Level” Examples for Low Intermediate ESL: Pretest = 205 Post-test = 208. Completed a level? NO. Pretest = 205 Post-test = 212. Completed a level? YES. Pretest = 209 Post-test = 210. Completed a level? YES.

NRS Level Completion So this only explains level completion with test scores. What about Field #8 on the Update Record – that also indicates that a student completed a level… correct?

NRS Level Completion The self reported values on the Update Record Progress field are not used for determining level completion. Instead, only test scores or HS/GED accomplishments are used.

What was that about HS/GED accomplishments? NRS Level Completion What was that about HS/GED accomplishments?

NRS Level Completion Earning GED and passing High School Diploma is considered as Completing a Level for the ASE High category. Prior to 2006-07, this applied to ABE Intermediate Low, ABE Int High, as well as ASE Low and High… but now it applies only to ASE High.

Federal Table 4 So, back to the Update Record issue. It really has nothing to do with these Fed Tables, then, does it?

Federal Table 4 Well, not exactly. Remember columns E & F? We’ll start with column F. Number Separated Before Completed draws its total entirely from Update Field #7, the number of students with Status = “Left Program.”

Federal Table 4 Notes on Column E : From a test score perspective, Advanced One or More Levels is not much harder than completing a level—usually it just requires a post-test score that is one point higher than what is required to complete a level. The Column E total, however, is often much lower than Column D because to advance a level, the learner must achieve the necessary post-test score AND mark “Retained in Program” on Update Field #8.

Report Series Additional Options Under Report Series there are Additional Options where you can run four different detail reports.

Federal Tables Summary Audit The Federal Tables Summary Audit displays an overview of your agency’s Federal Table information, similar to the “Run Statistics” that appear on the California Payment Points Summary.

Federal Tables Detail Audit The Federal Tables Detail Audit provides a page by page account of the Federal Table logic applied to each individual student in the database.

Federal Tables Monitor The Federal Tables Monitor is structured like the California Payment Points Monitor, and provides additional detail by class and by student.

Persister Report The Persister Report compares the percentage completing a level in Federal Table 4 with the percentage completing a level in Federal Table 4B.

Persister Report Fed Table 4B Fed Table 4 D (76.92%) = G (23.08)/ H (30.00)

Persister Report The Persister Report compares the Table 4 completion rate to the Table 4B completion rate. As the difference between G and H gets smaller, the percentage of persistence rises. Post-testing correlates with student persistence.

NRS Performance Report Table 4, Column H. Table 4b, Column H. The NRS Performance Report allows you to compare your own agency’s results to your statewide goals.

NRS Performance Report What if my agency is performing below standards as compared to the statewide goals?

NRS Performance Report One possible explanation may be found by comparing your percentages from FT 4 (middle column) with your percentages from FT 4B (far right column.)

NRS Performance Report If your agency’s Federal Table 4 percentages are significantly lower than the percentages from Federal Table 4B, then the issue is with getting your students pre- and post-tested. If your agency’s Federal Table 4 percentages are equal to or close to Federal Table 4B, then there’s a possibility the issue is your students’ performance on the pre- and post-tests. The Persister report is another resource to compare this information.

Thanks for participating in Web-based training!!