Presentation on theme: "Hubble Bubble Some Qualifications Bring Trouble"— Presentation transcript:
1 Hubble Bubble Some Qualifications Bring Trouble June 2012Debbie GrossmanEmma WilliamsPlanning and Statistics
2 Aim:To understand the way qualification data is collected, processed and used both within the University and by outside organisations.
3 Qualification Levels GCSE grades G - D GCSE grades A* - C A Levels, BTECs, International Baccalaureate etc.HNCHND, Foundation Degree etc.Honours degreeMasters DegreePhD
4 Qualifications recorded Currently have 231 codes in BANNER142 Level 3126 sent to HESA
5 2010-11 UG New Entrants (Excluding Foundation Degrees) TaskHighest QualificationFull timePart timeLevel 4 and aboveA LevelsBTECsAccess Course (accredited)ActivityThink about what you know about the student population and in groups estimate the percentage of new entrants that would be in each of these categories. Note: The qualifications listed are not a complete list therefore the total will not equal 100%
6 2010-11 New Entrants (Excluding Foundation Degrees) ValuesHighest QualificationFull timePart timeLevel 4 and above22.05%67.74%A Levels38.39%5.97%BTECs8.74%1.83%Access Course (accredited)3.94%0.88%Activity answersOpportunity to discuss the answers and how close the teams answers were to the actual values.Did they come close to peoples perceptions?
7 Applicant Qualifications submitted as part of the application process L3 and/or aboveGCSEs & equivalents fromProfessional qualifications/experienceFoundation degrees studied at GreenwichUCASElectronic Applications SystemABLNon UCAS ApplicantsEntered manually by AdmissionsNote for Debbie: BOTH go into the Electronic Applications System (EAS) which is an internal system and are then ‘SOATEST-ed’.Some will go in via ABL, but only when the qual comes through UCAS I think i.e. this year’s results. Qualifications held before ABL in August go in manually to SOATEST.Non UCAS applications come in through OLAF (On-Line Application Form) and go into EAS, to then be entered manually by admissions into SOATEST, as stated.
8 Student OLR Qualifications checked at Stage 2 registration. Review of qualifications held in BANNERUpdate or commentQualifications checked at Stage 2 registration.Original CertificatesSOATEST (BANNER) updated if required
9 ELQStatus determined by qualifications held prior to starting the new programmeGenerated within BANNERRules updated by PAS
10 AAB+ and Equivalents Unlimited recruitment for 2012-13 Not based on a tariff scoreDetermined by HEFCESpecific list of qualifications with certain gradesBest 3 A Levels, BTECs, Best 5 Scottish Highers, 35 or more International BaccalaureateABB for
11 Tariff Tariffable Qualifications UCAS determine tariff scores Generally Level 3Not all level 3 have tariff scoresUCAS determine tariff scoresCurrently based solely on GLH
12 Tariff Challenge 1Place the qualifications in order (lowest to highest) according to their tariff score.
13 Tariff Challenge 1 Position Qualification & Grade Points 1 International Baccalaureate - 314132BTEC Diploma QCF - D*D*2803CACHE Diploma in Childcare & Education - C2404GCE A Level with additional AS – AB1705AAT NVQ level 3 in Accounting - Pass1606GCE A Level – A*1407Cambridge Pre–U – D31308Irish leaving Certificate – A1909Diploma in Fashion Retail – Pass8010Sports Leaders UK - Pass30
14 HESA Tariff Most qualifications that have UCAS tariff No qualifications that don’t have UCAS tariffProvided to institutionsProvided to league table providersDependent on league tables request
15 Internal Tariff Score HESA tariff methodology In session calculation Available for student and applicantApplicantPredictedActualStudent
16 Tariff Challenge 2Which of the qualifications listed have tariff points?UCASHESA
17 Tariff Challenge 2 Qualification UCAS HESA Access course British Horse Society Horse knowledge & careBTEC: Subsidiary Diploma QCFFoundation DegreeHong Kong Diploma of Secondary EducationiMedia DiplomaInternational Baccalaureate Diploma or CertificateCertificate in Mathematics for EngineersMusic Theory ExaminationsProfessional Qualifications at Level 3Vocational (VCE) A-levelsCity & Guilds Land Based Services Level 3 Qualification: Extended DiplomaKey Skills level 2
18 Tariff Challenge 2 Answers QualificationUCASHESAAccess courseNBritish Horse Society Horse knowledge & careYBTEC: Subsidiary Diploma QCFFoundation DegreeHong Kong Diploma of Secondary EducationiMedia DiplomaInternational Baccalaureate Diploma or CertificateCertificate in Mathematics for EngineersMusic Theory ExaminationsProfessional Qualifications at Level 3Vocational (VCE) A-levelsCity & Guilds Land Based Services Level 3 Qualification: Extended DiplomaKey Skills level 2
19 Four Main League Tables TimesComplete GuideGuardianSunday TimesThere are other league tables as well:Thomson Reuters World University Rankings published by Times Higher EducationalShanghai Jiao Tong University Academic Rankings of World Universities
20 Measures in League Tables TimesCompleteGuardianSunday TimesStudent satisfaction√Research qualityStudent – staff ratioSpend per student on academic services and/or facilitiesTariff scores on entryGood honours (1st & 2:1)Completion ratesValue addedDropout compared with benchmarkGraduate prospectsPeer assessmentsThey all use slightly different versions of these measures and the weighting they use and how they are compiled into an overall measure varies. The newspapers vary on how much information they give on this – and this is where some scope for newspapers to add their own interpretative overlay over the HESA dataGoing to unpack these in detail shortlyIn brief:All include student satisfaction – from the NSSResearch quality (Times & Complete)Student staff ration (not Sunday Times)Other spending measures (Complete & Guardian)All look at tariff on entryGood honours & completion rates (Times & Complete)Guardian & Sunday Times have more sophisticated measures – Guardian value added & Sunday Times dropout compared with expected dropoutAll look at graduate prospects – from DLHESunday Times also uses peer assessmentNOTETimes: student satisfaction & research quality 1.5, others 1Complete Guide: Z-transformation of each measure, then 1.5 weight for Student Satisfaction & Research Assessment, 1.0 for restBuilt up from subject level tables: teaching 10%, assessment & feedback 10%, overall satisfaction 5%, value added 15%, student-staff ratio 15%, expenditure 15%, entry score 15%, career prospects 15%Sunday Times: teaching excellence 250 points, student satisfaction +50 to -55 points, peer assessment 100 points, research quality 200 points, tariff 250 points, unemployment 200 points, good degrees 100 points, dropout rate +57 to -74 points
21 Tariff in League Tables Average tariffFirst degree, new entrantsLevel 3 Tariffable qualificationsIncludes partner college studentsExcludes those on year 0Tariff on entry is an important measure in all the league tables.Average UCAS tariff (as counted by HESA)Of first degree new entrantsWith A-levels or equivalentIncludes partner colleges but excludes year 0Thought to be seen by applicants and their parents as a measure of quality of student populationThe data comes from the HESA student return – some of you will be aware that we have been making major efforts to improve the quality of the information we provide to HESA about qualifications on entry – and this is one of the reasons.
22 Tariff on entry 2012 Guides Times Complete Guardian Sunday Times First year, first degreeAll modesFull-time & sandwichAged under 21No age limitExcluding those on foundation yearsExcluding those with 0 tariff pointsBased on those with A-levels or equivalent, SQA Higher or Advanced Higher,ONC and OND including BTEC, Baccalaureate-----Ceiling set at 500 points & %-age of available points calculated – scores then halved to give score out of 250Data from 2009/10 HESA student return i.e entrants2009 entrantsEitherFull-time and sandwich onlyOrThose under 21 only.Excludes those with no tariff points at allAndBased on A-levels and close equivalentSunday Times do a transformation that disregards very high scores.
23 Tariff on entry 2012 Guide (2009/10 data) Greenwich scoreApprox equivalent toGreenwich rankTimes215CDD-CCD109 of 114Complete111 of 116Guardian209115 of 119Sunday Times104 (=208)118 of 122Based on 2009 entrants we were somewhere like CDD-CCDTarget of 50th ranked university is BBB (assuming they don’t improve as well)Two things we can do:Up the tariff we require before we admit studentsImprove the way we record students’ qualifications on entry so that we really get every tariffable qualification on recordMatters more to us than, say, Russell group universities, because most of their entrants would have A-levels only so not so complicatedCDD=200CCD=220BBB=300ABB=320
24 Times and Complete University Guide: Tariff on Entry Changes from 2013 Up to 2012In 2013First year, first degree, all modesAged under 21Times: aged under 21Complete: dropped age cut-offExcluding those with 0 tariff pointsExcluding those on foundation yearsBased on those with A-levels or equivalent, SQA Higher or Advanced Higher, ONC and OND including BTEC, BaccalaureateBased on wider group than previously: A-levels, Scottish & Welsh Baccalaureate, IB, various qualifications at level 3 and, crucially, ‘Level 3 qualifications of which some or all are subject to UCAS tariff’Data from 2009/10 HESA student return i.e entrantsData from 2010/11 HESA student return, i.e entrantsBecause of problems with the coding of qualifications on entry at some universities (not here), the Times and Complete Guide are changing the way they calculate tariff on entry.Crucially they are proposing to include a group of students who have a mixture of tariffable and non-tariffable qualifications.This is unhelpful to us: if a student has one A-level and then other qualifications that got them the entry but are not tariffable, they’ll look as if they have a very low tariff.
25 Expected values for 2013 Guides compared with 2012 TimesComplete2012 Greenwich score215A Level Grade equivalentCDD-CCDExpected 2013 score including mixed qualifications254245CCC-BCCCCC+Expected 2013 score excluding mixed qualifications257251Almost BCCWe have asked for these students to be excluded from our data –which makes a difference of 6 tariff points in the Complete Guide and 3 tariff points in the Times.Either way, the 2013 Guides (based on 2010 new entrants) should show a substantial improvement. Now in the CCC to BCC range.Remember target was BBB.CDD=200CCD=220CCC=240BCC=260