1. Lifting the Lid on Completion Rates
Alice Bednarz
AVETRA Conference, 2012

2. Problems 35 100 Completion rate = 35%
Say stuff about different levels of completion rates, and what a cohort is, here. These students could be students enrolled in a course, students enrolled in a subject, or apprentices and trainees with a contract of training, or particular sub-groups of students, such as full-time students under 25. A group of entities who started at the same time are called a cohort. The type of completion rate determines which cohort we need to track.
Need to wait for the course duration to elapse
Can take many years to occur
Reduces the usefulness of the data
Completions not reported immediately
Instead, consider a “tracking window”, then estimate the true completion rate
Completion rate = 35%
100

3. Completion rates overview
Several different levels:
Qualifications
Subjects
Apprentices and trainees
Subgroups (e.g. students under 25)
Track a certain group of entities
Called a ‘cohort’

4. Completion rate – 3 components
Who or what are we tracking?
How long are we tracking them for?
How is the rate calculated?

5. VET qualifications
Qualification completion rate (course completion rate)
Qualification 2
Qualification 1
Completed
Cancelled
3 years
Subject completion rate (load pass rate)
Subject 1
Subject 2
Passed
Failed/withdrew
hours studied by those who passed total hours committed to by all students
1 year

6. Apprentices & trainees
Cancelled
Contract completion rate
Contract 1
Contract 2
Completed
4 years
1 quarter or
Individual completion rate
Completed
Cancelled
A contract completion rate refers to the proportion of contracts of training started in a given time period that have since been completed.
An individual completion rate is an estimate of the proportion of apprentices and trainees who started in a given year and who eventually completed their training in the same occupation in which they began, but not necessarily with the same employer.
4 years

7. Estimating course completion rates
Commencing
Commencing ?
Continuing
Continuing
Quit
Quit
Completed
Completed 1 1
Commencing
Continuing
Completed

8. Transition probabilities
2007
2008
2009
2/5 =0.4
commencing
quit
commencing
continuing
completed
2/5 =0.4
commencing
continuing
commencing
quit
commencing
continuing
1/5 =0.2
commencing
completed
continuing
continuing
continuing

9. Markov Chain model 0.3 0.4 0.2 0.1 0.6 1 Commencing Continuing Quit
Completed 1

10. Probability of eventually completing
commence
continue
quit
complete or p1 p3
commence
continue
quit
complete q3 q1
Talk about assumptions and limitations here

11. Assumptions and Limitations
2nd yr onwards, student behaviour remains constant.
Limitations:
Model cannot be validated.
Too difficult to match students across > 3 years.
0.2
0.3
0.6
0.1
0.4
Commencing
Continuing
Quit
Completed 1

12. Latest estimates – VET students
Cohort
Commencing in
Estimated qualification completion rate
(%)
Subject load pass rate
Students enrolled in AQF qualifications
2008 28 80
Full-time students aged 25 yrs and under enrolled in AQF qualifications 37 78

13. Completion rates by level
Full-time students, aged 25 years and under, commencing in 2008
AQF Qualifications
Estimated qualification completion rate (%) 2008
Subject load pass rate (%) 2008
Diploma and above 35 79
Certificate IV 34 75
Certificate III 46 80
Certificate II 32 71
Certificate I 31 69
TOTAL 37 78

14. Is completing a qualification beneficial?
Yes, but size and type of benefits vary (Karmel and Fieger, NCVER, forthcoming)
Compared the payoff associated with completing, versus not completing
Outcome
Average payoff to completion (%)
Proportion with a positive pay-off (%)
Completion has highest payoff for...
Employed after training 22 98
Not employed before training (108%)
Further study
123
99.9
Cert. I & II (275%)
If you complete, you’re 123% more likely to go on to further study than if you don’t complete.
If you complete, you’re 22% more likely to be employed after training than if you don’t complete.
If you complete, your wage will be approximately 0.3% higher than if you didn’t complete.
98% of students will have some degree of gain, but the highest payoff associated with completing vs not completing will be for those who weren’t employed before training.
Everyone is more likely to go on to further study if they complete versus if they don’t complete. This makes sense. The group who are most likely to go on to further study if they complete are Certificate I and II students.

15. Where to now?
Calculate completion rates for different subgroups of students, not just broken down by level and by state
e.g. for ‘unemployed before’ vs ‘employed before’
Completion particularly important for:
not employed before training (employment after)
Cert I/II (further study)
Policy: target those who have the highest payoff associated with completing.

16. Additional slides

17. Do students intend to complete?
NCVER Student Intentions Survey
Surveyed ~ 11,000 students
Randomly selected sample, stratified by type of training provider
93% of students enrolled to start a VET course in February 2011 intend to complete it.
7% of students enrolled to start a VET course in February 2011 intend to complete subjects only.
Four months after training commences, of those students who intended to complete the course:
68% are still in training
19% report to have completed the course
13% left without completing the course.

18. Benefits of completing – more details
For each student, look at an outcome, such as the probability of being employed, conditional on first the student completing the course and second on not completing it.
The ratio of the outcome if they complete vs the outcome if they don’t complete gives us a pay-off function.
Fit a model based on graduates’ data only
Fit a second model based on non-completers’ data only
Pr(employed after training | graduate)
Pr(employed after training | non-completer)
= 1.22
Therefore pay-off associated with completing = 22%
(Same student, substituted into 2 different models)

19. Completion rates – apprentices & trainees
Commencing in
Trades (%)
Non-trades (%)
Contract completion rate
2006 45 53
Individual completion rate 56 54
Projected contract completion rate
Dec. quarter 2010 46 60
Completion rate

20. Benefits – apprentices and trainees
Average income in last week of apprenticeship
Average income upon completion
% increase
Trade apprentices
$36,600
$52,500
43%
Non-trade apprentices
$40,700
$45,900
13%