Presentation is loading. Please wait.

Presentation is loading. Please wait.

DLHE Good Practice 17 September 2009 London. Objectives Introduce good practice within DLHE record Demonstrate tools designed to aid good practice Provide.

Similar presentations


Presentation on theme: "DLHE Good Practice 17 September 2009 London. Objectives Introduce good practice within DLHE record Demonstrate tools designed to aid good practice Provide."— Presentation transcript:

1 DLHE Good Practice 17 September 2009 London

2 Objectives Introduce good practice within DLHE record Demonstrate tools designed to aid good practice Provide institutional perspectives on completion of survey and data collection Highlight data quality issues and checking

3 Target response rates

4 80% for FT UK domiciled students (including Research Council students) 70% for PT UK domiciled students 50% all other EU

5 Difficulties in meeting targets Increasing student mobility reduces the reliability of contact information Public scepticism about what personal data is held against them at a governmental level Survey saturation High targets set

6 Current response rates 78.5% for FT UK domiciled (-0.4% when compared with 2006/07) 70.7% for PT UK domiciled (-0.4% when compared with 2006/07) 54.7% for all other EU (+0.7% when compared with 2006/07)

7 Preparing for DLHE 2008/09

8

9 April target list system Uses a student file from that year Reduced validation Generates an accurate population file to survey against System currently underused

10 Population Important to check the POPDLHE file generated from the Student Record submission… …make sure you are not missing students… …or that no one is going to be surveyed that shouldn’t be The Match Report will show you how successful you were at this

11 Know your graduates Up-to-date contact information Some graduates will like to complete forms online, others will not, so offer as many methods of completing the survey as possible Call at the right times Identify your part-time and ITT graduates

12 Sections D and E Ensure part-time and/or ITT students complete the sections relevant to them Failure to do so will result in validation warnings Be aware of students who changed to a PT mode of study at the end of their course but studied the majority as a full- time student

13 Telephone interview Ask the respondent the core questions first if graduate is in a hurry… …without them answered you do not have a valid response Clarify contradictions or information that would appear incorrect

14 Location information Location of employment information is very important to statutory users of data Allows analysis of student’s geographical movement from entry to HE to the labour market Try to code to full postcode, however at least outward postcode level or town postcode should be reported

15 Audit The data should be kept only as long as is necessary The completed questionnaires should be kept (as either a hardcopy or electronic version) for each individual leaver for one year after the data is returned to HESA If the purpose is research then data may be kept for a longer period of time There is no need to anonymise the data

16 Aardvark system changes From now on, in addition to the Access Codes, users will also need a PIN code to create new accounts and/or add permissions to existing accounts. The PIN codes will be distributed by letter a few days in advance of the Access Codes Both the Access Codes email and the PIN letters are sent to the nominated record contact at institutions The system will now send an email to the transaction owner when a transaction has completed processing

17 DLHE online service

18 Why use it? Help to improve response rates by offering an alternative to paper or phone based questionnaires Students can complete it at their own accord and at their own pace Intuitive and free to register and use Built in validation

19 System Usage

20 Online DLHE Questionnaire for both January and April census Graduates complete form at www.dlhe.ac.uk or www.dlhe.ac.uk?INST=nnnn to bypass institution selection Institutions download data from submit.hesa.ac.uk

21 DLHE Online Statistics 1

22 DLHE Online Statistics 2

23 DLHE Online Statistics 3

24 Uses of online system Existing institutions have used the system in different ways… …either as an additional tool to the mailings and telephone follow-ups… …or as part of a full integration policy Many institutions also use the system as a data inputting tool for staff

25 Registering for system Ahead of each DLHE survey you must pre-register for use of the system by the date we publish… …and as such requires incorporation into planning

26 The Aardvark system Register to use DLHE online on Aardvark Control panel allows you to specify your requirements for system e.g. inclusion of institution's student number or including a PIN Download the data of completed surveys from Aardvark

27 CASCOT

28 CASCOT (IER) CASCOT is designed to assign a code to a piece of text. e.g. a SOC code to a job title from the DLHE survey When CASCOT assigns a code to a piece of text it also calculates a score from 1 to 100 which represents the degree of certainty that the given code is correct Available to download from: https://submit.hesa.ac.uk/aardvark/CASCOT.ASP

29 Benefits of using CASCOT Ensures consistency within institution and also across sector Reduces difficulty of interpreting text Improves efficiency and reduces burden of data inputting

30 Potential problems The quality of coding depends on the quality of the input text: "advertising & marketing" -These are the area of work. Are they an advertising executive, an advert designer or an advertising salesman? "answering phone enquiries“ -This is a job description, not a title.

31 Working with batches CASCOT can be used to code on a record-by-record basis or can work in batches Input files allow user to code as many records as they require Setting up an output file will then store the results in a spreadsheet for you

32 Interpreting results Always interpret the information accurately… …however you do not have to use the recommendation made by CASCOT (regardless of the score it is given) Use your judgement to assess whether it is the most accurate code and reflects the true nature of the job

33 For example Student employed as an architectural model maker for which CASCOT gives the following code… ‘Modeller (Architectural) 5315’ …however that is classified as a non- graduate job… …whereas ‘Modeller, Artistic 3411’ is a graduate job

34 DLHE good practice The School of Pharmacy 17 th September 2009 Mark Gittoes Head of Quantitative Analysis for Policy, HEFCE m.gittoes@hefce.ac.uk

35 Areas of use Institutional/subject area reporting Research Understanding of provision Overview of HE Other bodies

36 Institutional/subject area reporting HEI Performance indicators Teaching quality information and UniStats Regional profiles

37 HEI Performance indicators (1) Proportion of graduates who are working or studying (or both) by institution Currently published for full-time first degree graduates Extending to other cohorts Interest in the development of other indicators

38 HEI Performance indicators (2)

39 Unistats (1) “Employment prospects” broken down by institution and subject area Aimed at potential students Reports: – Employment status – Top 10 profession types – Job category (graduate, non-graduate)

40 Unistats (2)

41 Research Student ethnicity Job types

42 Student ethnicity (1) Relationship between a student’s ethnicity and experience in HE: – Entrant profiling – Progression through HE – Attainment in HE – Satisfaction – Graduate destinations

43 Student ethnicity (2)

44 Research: Job types (1) Examination of Standard Occupational Classification (SOC) Study of relationship with student perception of criteria needed for job? Makes use of responses to DLHE question: – “Would you have been able to get the job you were doing on 14 April 2008/12 January 2009 without the qualification you recently obtained (the actual qualification, not the subject of study)?”

45 Research: Job types (2)

46 Understanding of provision Strategically important and vulnerable subjects Foundation degrees Erasmus and placement courses Further Education Initial Teaching Training

47 SIVS: Early careers of graduates “To address the lack of information on the supply and demand of STEM skills, HEFCE should … publish an annual report describing: undergraduate subject trends; recent graduate jobs and salaries; and the subjects where employers and government departments believe that there are, or are likely shortly to be, shortages of graduates with key skills.”

48 SIVS: Early careers of graduates

49 Destination No.% Studying (not employed)1,85047% Studying and employed82521% Employed1,01526% Total employed1,84047% Unemployed1053% Other1103% Total DLHE respondents3,900100% Note: Full-time foundation degree qualifiers registered at English HEIs, 2005-06 Foundation degrees: destinations

50 Erasmus and placement courses: Salary six months after qualifying

51 Other bodies Office of National Statistics – UK migration mapping Department for Business, Innovation and Skills – Parliamentary questions – Research Research Councils UK Higher Education Policy Institute – Male and female participation in HE Other UK HE funding bodies

52 Quotes “It could be that males are more likely to be rejected for the jobs that they apply for, …, and females are experiencing discrimination in the salaries that they are paid.” Male and female participation and progression in HE, HEPI June 2009 “Graduate employment data makes grim reading” The Times Higher, July 2009 “More graduates finding work” The Times Higher, August 2008

53 The future? Move from DIUS to DBIS Increased focus of employability Development of additional indicators HE framework and capturing institutional strength

54 DLHE Good Practice Seminar: 17 th September 2009 Nathalie Rees Senior Careers Information Officer

55 Background & current role Survey experience Review of First Implementation & good practice document Survey co-ordinator/manager at the University of Chester Departmental contact for data

56 Who makes DLHE work @ Chester?

57 But also: MIS colleague Temporary telephone team Career Link Tutors Departmental Administrators

58 Procedures & training Preparation: procedures checklist Training for new staff members: coding and SITS Regular updates at team meetings; keeping the team informed of progress

59 Mailing & processing questionnaires Preparation for first mailing & online survey First mailing Processing incoming questionnaires Coding

60 Telephone follow-up Recruitment of phoners Training Documentation/resources Quality checking

61 Liaison with departments Contact details Obtaining information Recruitment of students for telephone follow-up Raising awareness of DLHE survey

62 Inputting & data quality control Tools used Coding checking, inputting & validation Spelling & postcodes Excel spreadsheets – MIS partnership

63 Challenges Up-to-date contact details Staffing/administration of the survey Survey overload Meeting targets for specific cohorts etc…

64 Room for improvement Updating procedures/documentation Increasing online replies Using DLHE data Survey awareness raising with final years & academic departments

65 Internal use of DLHE data Academic departments Careers Advisers Publications & website

66 Contact Details Email address: n.rees@chester.ac.ukn.rees@chester.ac.uk Telephone: 01244 513066

67 Any questions

68 DLHE Good Practice University College Falmouth

69 Contents of Presentation Profile of University College Falmouth DLHE outcomes at Falmouth over the last 10 years Outline of the DLHE process at Falmouth The role of Alumni Lessons to be learnt Tricks of the trade Wish list

70 Profile of University College Falmouth Small HEI – just under 3000 students Specialist – Art, Design, Media and now Performance Practice based – most courses are studio based with students spending 80% of their studies doing hands- on practical work & 20% of their time on written work. No exams – continual assessment High levels of dyslexia and mental health issues

71 DLHE at Falmouth over the last decade Admissions staff at Falmouth have responsibility for HESES, HESA, DLHE, Enrolment, Assessment Processing, Student Transcripts, Open Days etc. Average response rate is just above 50%, Falmouth’s employability rate is normally between 91%-94%. The worst employability rate came the year we employed an ex-BBC researcher to work on the DLHE for 6 months. Response rate went up to 97%. We came bottom of the employment tables with 43% of graduates obtaining jobs. Luton University came top with a mere 42% response rate.

72 Outline of the DLHE process at Falmouth Inclusion of a letter in their graduation pack Letter & form to their home address Letter & form again to their home address Using ‘text tools’ to send text reminders to mobile ‘phones A postcard ‘where are you’ to their parents asking them to forward on any envelopes bearing the College logo. Emailed reminders Phoning graduates during the day Phoning graduates between 6 & 7:30 pm Utilising tutors private Alumni Use other contacts and other methods

73 The Role of Alumni Tutors tend to operate a private Alumni, whilst often effective for DLHE purposes, transgress the boundaries of the Data Protection Act. Falmouth have established an official Alumni in the last 18 months which it is hoped will improve communication with graduates by maintaining up-to- date contact details, networking, course cameraderie and another conduit for the DLHE.

74 Lessons we have learnt Visual thinkers such as art and design students do not engage well with the DLHE official questionnaire 31 questions are 27 questions too many for our graduates Students are suffering from questionnaire fatigue Postcards rather than letters have a better response rate Mothers, house-mates, student ambassadors are invaluable in nudging graduates and/or providing new contact details Graduates with dyslexia or mental health issues do not engage well with the DLHE Cheap tricks like ipods or book tokens don’t work Don’t try too hard to contact 100% of your graduate cohort Graduates tell lies or bend the truth.

75 Tricks of the trade (1) Use student ambassadors, they know who knows who and they talk the same language (doh..) Utilise technology such as social networking sites e.g. face book, friendsreunited, twitter. Parents and mature graduates prefer printed post e.g. reminder postcards and stamped addressed envelopes. Graduates prefer electronic communication – invest in ‘text tools’ to perform mass SMS texting, download spreadsheets of private email addresses to upload into Outlook and b.c.c DLHE messages to graduates Ask course teams to feed back on references for graduates. Set up an Alumni; use case studies in Alumni newsletters.

76 Tricks of the trade (2) In ‘phone and text chase-ups, there are 4 key questions to ask Have you got a job, where, who with, doing what. You can use the web to find out post codes (Q10), what the organisation does (Q9), size of organisation (Q11), the type of school (Q26, Q27) etc. The kind of work the graduate is in, can often provide missing answers – e.g. you don’t need a degree in Graphic Design to work as a shelf stacker in Asda (Q12, Q13). Don’t offer gifts, extend the use of workshops, studios, library to Alumni instead. Don’t use careers to give DLHE talks, invite successful Alumni to give DLHE talks to 3 rd years.

77 Our wish list Severely cut the number of DLHE questions Create a visual mind map version of the DLHE Produce a summarised parent’s version of the special issue ‘statistics focus’ viewable via http://www.direct.gov.uk http://www.direct.gov.uk A DLHE user group on JISCMAIL

78

79 79 University of London Good Practice Oliver Gardham & Julie Cheung-Inhin

80 80 Introduction Who we are Where we are How many? The Careers Group

81 81 Who We Are A ‘family’ of universities and institutes affiliated with or bound to The University of London Some colleges constitute one HESA return, others are combined

82 82 Who We Actually Are Royal Holloway, UoLKings College LondonUCL GoldsmithsCourtauld IoAQueen Mary, UoL SOASSchool of PharmacyLSHTM Cancer ResearchHeythropInstitute of Education BirkbeckRoyal Veterinary Coll.St George’s HMS Others….UoL Institutes (below) Warburg InstituteUoL in ParisInst. Advanced Legal Studies Inst. Classical StudiesInst. Commonwealth StudiesInst. English Studies Inst. Germanic & Romance StudInst. Historical ResearchInst. Musical Research Inst. PhilosophyInst. Study of the AmericasMillport Marine Biological Station

83 83 Where We Are Royal Holloway

84 84 University Marine Biological Station, Millport

85 85 Millport

86 86 University of London Institute in Paris

87 87 Senate House and Stewart House

88 88 How Many? Between 45,000 and 50,000 total graduates Around 30,000 of which are U of L Most colleges request internationals, due to diversity of London graduates It increases every year

89 89 The Careers Group Central Careers Administration Products and services designed to help individuals achieve and maintain job satisfaction throughout their working lives Customers include academic institutions, students, graduates, graduate recruiters, and other careers services providers Products and services include careers advice, graduate fairs, online services, courses, professional training Graduate Research

90 90 Graduate Research Full-time DLHE team of 4 all year round, one of which is the Call Centre Manager On-site call centre Department with a collective 20 years’ experience, including manager with 10 years of experience

91 91 DLHE Problems Spotting trends and problems in time –Bad numbers –Unemployment Quality control Language barriers Promoting/engaging with the data

92 92 Bad Numbers Near-instant reporting on: Bad numbers Unemployment rates Response rates

93 93 Quality Control Invest in a good capture system Call afternoons and evenings, week and weekends if necessary) Experienced callers Employee retention is valuable International codes and time zones Language diversity

94 94 Colleges with Language Schools UCL (Dutch, French, German, Greek, Hebrew, Italian, Spanish, the BA Modern Languages course allows over 250 combinations of modern languages) SOAS (over 50 languages, including Arabic, Amharic, Bengali, Chinese, Hausa, Hebrew, Hindi, Japanese, Korean, Nepali, Swahili, Thai, Tibetan, Urdu, Vietnamese and Zulu) King’s (Arabic, Bengali, Catalan, Dari, Farsi, French, German, Greek, Gujarati, Hebrew, Hindi, Italian, Japanese, Mandarin, Punjabi, Polish, Portuguese, Russian, Sanskrit, Spanish, Swedish, Turkish, Urdu) Royal Holloway (French, German, Hispanic Studies, Italian) Birkbeck (French, German, Japanese, Spanish, Portuguese) Goldsmiths (French, German, Spanish, Portuguese) Queen Mary (French, German, Hispanic, Russian) Institute of Germanic and Romance Studies (French, German, Hispanic, Italian) University of London Institute in Paris (French)

95 95 Engaging with the Data Familiarise yourself with both survey and registry data. Over 100 fields of information are available for each graduate allowing simple or complex cross-referencing Prepare a menu of available reports for academics or other interest groups who may not be aware of DLHE’s potential, for example… –Graduate level work and/or post-graduate study –Destination summaries –Work sectors –Geographical reporting –Top ten employers/jobs Allow for various breakdowns/versions for each report, for example –PT/FT, UG/PG, UK/EU, Department, Course, Gender, Ethnicity, Class

96 96 SIC/SOC & Sectors SIC CODESIC DescriptionThe Careers Group Sector 6010Radio broadcastingMotion Picture Production/Sound Recording/Broadcasting 6020Television programming and broadcasting activitiesMotion Picture Production/Sound Recording/Broadcasting 6100Telecommunications 6110Wired telecommunications activitiesTelecommunications 6120Wireless telecommunications activitiesTelecommunications 6130Satellite telecommunications activitiesTelecommunications 6190Other telecommunications activitiesTelecommunications 6200 Computer programming, consultancy and related activitiesComputer Programming, Consultancy & Related 6201Computer programming activitiesComputer Programming, Consultancy & Related 6202Computer consultancy activitiesComputer Programming, Consultancy & Related 6203Computer facilities management activitiesComputer Programming, Consultancy & Related

97 97 A Few of Our Sectors The Careers Group Sector Architectural/Engineering Activities Arts, Entertainment & Recreation Building & Landscape Services Computer Programming, Consultancy & Related Construction Motion Picture Production/Sound Recording/Broadcasting Telecommunications Transportation & Storage

98 98 “Helping Graduates in Work” Website Mayor of London website http://www.london.gov.uk/graduates/Defau lt.aspxhttp://www.london.gov.uk/graduates/Defau lt.aspx “What can I do with my XXXX degree?” Jobs, salaries, employers “What does a XXXX do?”

99 99 Destinations Website Online access to wide range of reports for all DLHE years with different levels of access for Head of Careers Services, Careers Advisers, academics Destinations Breakdown, Top Work Locations (using Google Maps), Average Salary, Employment Index, Graduate-Level Employers, and Graduate-Level Jobs Institutional breakdowns at UoL, college, department and course level Different levels of log-in

100 10 0 Summary Monitor and report collection data asap Retain staff where possible Be prepared for international calling Make the data interesting and useable for academics/CAs/principles Give people reporting options

101 10 1 Thank you

102 10 2 Q & A?

103 DLHE Data Quality

104 Exception checks Designed to pick up peculiarities within the data Can be errors or warnings Feedback to user given at point of ‘COMMIT’ transaction Significant exceptions will be raised by Data Quality Assurance at HESA

105 Common issues Students working and studying in full-time modes Incorrect salaries being reported – salary information should be reported as FTE Don’t return ‘0’s for salary if information not collected - use Xs instead Part-time and/or ITT students not completing the relevant sections Incomplete postcodes – unknown regions

106 Teaching Quality Information Teaching Quality Information (TQI) is displayed in three tables: -Table 5: Destinations information -Table 6: Job categories information -Table 7: Common job types information Ensure you check each table as the information is widely used…

107 UNISTATS In 2008 – 565,250 unique visits In 2009 so far – 409,157 unique visits (comparable time in 2008 there had been 299,982 unique visits) International readership Significant advertising campaign placing links to UNISTATS on popular websties

108 Table 6: Job categories information

109

110 Check documentation…

111 Future developments Catherine Benfield

112 Review of Early DLHE Implementation 2011/12 (Apr 2012/Jan 2013) Review will take place from Autumn 2009 and be completed by February 2011 Usual HESA review procedures including review group with institutional representation

113 The review will include… Revisions to the questionnaire and telephone script Review of supporting documentation Development of online systems and support Technical update to use XML Other aspects of the survey

114 Review items – questionnaire and telephone script As well as the usual DLHE review items which include… –Reassess core and non-core questions –Ensure routing works (issues with sections D and E) –Any design improvements –Explore any problems that have been reported

115 Review items – questionnaire and telephone script (cont.) …there are some specific items for review/consideration this time round: –Remove NHS question (Q8) –Update method categories to reflect the current options –Capture of free text information –Q1 and Q2 options –Use of JACS for Q21 (subject area of study or training) –Section E (part-time leavers) –How the survey works for self-employed leavers

116 Review items – supporting documentation Review letter and email texts Review the methodology document Possible SOC 2010 implementation

117 SOC10 implementation SOC 2010 –Will be finalised by ONS soon –There are significant changes planned, particularly to Major Group 1 (Managers) –Some of the disaggregation currently done at 5-digit level will be available at 4-digit (e.g. separate codes for barristers and solicitors) –No decision yet on the need for a 5 th digit –“Graduate jobs” will need to be re-specified

118 Review items – technical update to XML eXtensible Markup Language The HESA Student Record has been translated to XML Data standards - Information Standards Board Hierarchical data structures

119 Review items – other issues Coverage –BIS is commissioning research to investigate feasibility of including international students in DLHE. This will feed into DLHE development –There is ongoing discussion about the exclusion from DLHE of those who receive their awards from dormant status –Multiple qualifications for a student - HIN

120 Review items – development of online systems and support Investigate the development of an input-only version of the online questionnaire Remove the HTML provision in order to focus on improving the DLHE Survey System Assess the use of the April target List System and whether or not it can be removed Look at how the DLHE Survey System is hosted

121 The future for the Longitudinal Survey? Results of second survey published earlier this month The funders of the survey (Funding Councils, DELNI, TDA and RCUK) will take a decision on the future of the survey in the next few weeks Quite separately, RCUK intend to undertake further follow-up with the 2004/05 cohort of PGR graduates

122 Operational Training DLHE Introductory seminar 9 February 2010 in Cheltenham www.hesa.ac.uk/seminars

123 Institutional Liaison Helpdesk Telephone 01242 211144 Email Liaison@hesa.ac.uk


Download ppt "DLHE Good Practice 17 September 2009 London. Objectives Introduce good practice within DLHE record Demonstrate tools designed to aid good practice Provide."

Similar presentations


Ads by Google