PHEE April 2001 Meeting the Assessment Challenge: finding practical approaches Su White and Hugh Davis IAM Learning Technologies: ECS, The University of Southampton

PHEE April 2001 e 3 an Electrical and Electronic Engineering Assessment Network a three year HEFCE project from the Fund for the Development of Teaching and Learning (FDTL)

PHEE April 2001 e 3 an - the participants South Coast HEIs Southampton Portsmouth Bournemouth Southampton Institute consultants across the sector plus steering group (IEE, CAA Centre)

PHEE April 2001 the importance of assessment - 1 QAA feedback says we are all bad at feedback QAA wish to see how assessment links to learning outcomes subject benchmarks National Qualifications Framework

PHEE April 2001 the importance of assessment - 2 assessment shapes students’ learning the importance of dialogue and feedback early diagnosis of problems lecturer student

PHEE April 2001 e3an – the context … much existing good practice expertise built up over years solutions for particular contexts technology and people approaches identify, explore, acknowledge, disseminate work collaboratively across sector

PHEE April 2001 e 3 an - the plan question banks as a focus peer reviewed test banks of questions for electrical and electronic engineering questions produced and reviewed by paid consultants supporting materials on methods and practices, workshops and consultancies

PHEE April 2001 e 3 an - the plan... questions banks may be: multiple choice objective tests text based traditional tests self assessment, feedback, course assignments, exam questions mainly for formative and diagnostic assessment – but also as source of ideas for summative assessment

PHEE April 2001 so what is innovative about this? it has never been done like this before the questions will form a yardstick against which achievement may be gauged the project aims to build a network of “champions” across the sector the project focuses on assessment outcomes rather than technology

PHEE April 2001 themes and teams theme leader plus expert consultants categorising syllabus cash available for travel plus small fee local teams in phase one - trialling the process, four themes national teams in phase two, eight themes

PHEE April 2001 theme process Working together the theme teams: identify question targets discuss levels and approach agree division of tasks write and review questions trial and evaluate questions

PHEE April 2001 phase one themes Analogue Electronics Digital and Micro Processors Circuit Theory Signal Processing

PHEE April 2001 phase two target themes Power Electronics Electromagnetism Computer Programming Control Engineering Maths Telecomms Physics/ Semiconductors Datacomms NB all open for discussion

PHEE April 2001 database and interoperability XML database IMS QTI interoperability standards ++ objective tests for common test engines collaboration with other projects print as the lowest common denominator some examples...

PHEE April 2001 project questions interoperability standards - reality of vapourware? how should authors produce questions for maximum interoperability? are some question types more interoperable than others? how do we choose a system for interoperability?

PHEE April 2001 more project questions creating test banks in a cost-effective manner choosing themes (usefulness, greatest gains, ++) classifying questions - level of skills, difficulty, cognitive skills, time required subdividing question banks into sub themes which themes/items suit CAA, which don't how to establish an effective peer review process

PHEE April 2001 looking to the future greater involvement of existing expertise barriers and drivers for use and dissemination distribution models continuation growth and maintenance of the test bank

PHEE April 2001 e 3 an - the benefits identifying and spreading good practice evaluated trial use of automated assessment on campus potential to trial new software (e.g. web based QM perception) potential time savings students feedback improved (in time for next QA / academic review

PHEE April 2001 e 3 an - more details Su White tel +44 (0) fax +44 (0) please contact us if you would like to be involved :-)