The Workplace Relevance Scale in the UES David Collings (ECU) and Bruce Guthrie (GCA)

Workplace Relevance Scale In this session: Supplementing the UES Why workplace relevance? WRS Development Source, versions, items

Supplementing the UES Dennis Trewen (AO) past Australian Statistician “The university is key stakeholder. The success of the survey depends on their co-operation. This is more likely if they are significant users of the UES itself and also possibly use it as a vehicle to ask for information of particular interest to them. The Department should work on this possibly in collaboration with Universities Australia” Jan 2014

Supplementing the UES SRC/GCA offering additional items and/or populations 2012 UES – sector consultations with ACER, Melbourne CSHE etc Internal focus institution specific, localised concerns Rationalising multiple surveys; External focus benchmarking opportunities UES is inherently benchmarking oriented (funding device?) E.g. Online Learning; Community Engagement; WIL CEQ optional items as a model

ECU and workplace relevance Below average employment outcomes (GUG! despite BGS) Curriculum 2012 project – embed employability into curriculum new ECU Strategic Priority – graduate outcomes Teaching and Learning Functional Plan 80% u/g courses with WIL and/or workplace elements All u/g courses with employability activities 80% of u/g courses with integrated careers and employability activities 80% of u/g courses using e-portfolios Careers unit broadened scope: closer to faculties Mentoring; Leadership; Volunteering

ECU and workplace relevance (cont) Engagement (community includes industry/employers) Course Consultative committees WIL; products/services; guest lecturers etc. Internal BGS phone survey Graduates (+12 mths and +24mths) Rated a range of strategies re employability, e.g. in-class employer visits; jobsearch skills; career fairs. The two clearly most highly rated were: “Courses and unis which include industry practicums or placements”: 63% of respondents rated 9/10 or 10/10 “Courses and unis which have work-relevant content”: 61% of respondents rated 9/10 or 10/10 Where to find a good measure of student perception rather than structural inputs or GDS outcomes ….with BENCHMARKING POTENTIAL

ECU and workplace relevance (cont) AUSSE? – Work Integrated Learning Scale ECU used this in AUQA 2012 Some time-series Some problems – only 5 items, but 3 different response frames/scales ! AUSSE supplanted by UES! Griffith OLT study? 2013 approx 12 unis; Calvin Smith Over 100 items

BRUCE GUTHRIE and ALAN RICHARDSON to the rescue! (via our esteemed ATN colleagues)

Workplace Relevance Why workplace relevance? Generic Skills (early 80s) Graduate Attributes (last 10 years) Higher Education Standards Panel (HESP) advises on the Threshold Standards of the Higher Education Standards for TEQSA

Workplace Relevance Draft Standards for Learning Outcomes (Coursework) The learning outcomes for each course of study are informed by: a) the mastery of specific disciplinary and/or interdisciplinary knowledge and skills that characterise the field of study b) the generic skills and attributes required of graduates c) the application of generic skills and attributes in the context of the field of study including the communication skills required, and d) the requirements of employment related to the field of study

Workplace Relevance ESS Pilot report gives a useful overview Conceptual skills and attributes frameworks roughly split into What graduates leave uni with What their courses seek to impart What employers need

Workplace Relevance Scale Workplace Relevance Scale developed from open-ended CEQ responses arising from the development of the CEQuery package. Alan Richardson & Boris Kabanoff RMIT, UTS, QUT

WRS Original The course developed an understanding of workplace skills. The course provided the opportunity for putting theory into practice. The course developed current professional skills. The course was a good combination of theory and practice. I got practical experience dealing with actual work situations. * What I learnt benefited my future work. The course helped prepare me for the workforce. Subjects had no direct relation to the world of work (reverse scored) * *dropped in subsequent version

WRS Original Updated The course developed an understanding of workplace skills. The course provided the opportunity for putting theory into practice. The skills developed in the course were relevant and useful.* The course developed current professional skills. The course was a good combination of theory and practice. What I learnt benefited my work. The course helped prepare me for the workforce. *added in this version

WRS for the UES WRS developed for graduates in the workplace Past tense wording Liaison with Alan Richardson, colleagues and GCA Updated, shortened and fine-tuned for use in the UES

WRS for the UES Five items, present-perfect tense: Q9. The course has provided the opportunity for putting theory into practice. Q12. The skills developed in the course are relevant and useful. Q16. The course has developed current professional skills. Q25. What I have learnt will benefit my future work. Q29. The course has helped prepare me for the workforce.

WRS for the UES ECU Swinburne UTAS UTS ACU Notre Dame Currently: Open ended question for comments Any more?!

WRS for the UES What value? A scale is a better measure than single item WRS has some decent validation behind it (at least for grads) More likely to generate useful comments re employability Benchmarking Simple comparisons: where do we stand? WRS comparisons less likely to be complicated by regional labour market factors than GDS outcomes, though discipline mix still a factor. Who is doing well, and WHY/HOW? Who does the work (data aggregation, liaison, reporting?) CEQ optional scales? – how valuable have they been? (see national AGS reports for selection, but what of usage?) Assess psychometric properties as UES add-on? (over to Alan?) UES as appropriate vehicle for WRS vs GOS/ESS?