University Library Making the connections: assessment, training, progression and performance Dr A. Harrison: Institute of Psychological Sciences A.Newton: Leeds University Library Lilac 2007

Why this project? BSc Psychology (Hons) A progressive and integrated degree programme that encourages students to become more autonomous thinkers and practitioners of Psychology Culminating in an independent, empirical project reporting on a substantial piece of research

Do we need Information Literate students? Quality Assurance Agency Generic Skills Retrieve and organise information effectively:- books, journals, computer and internet sources Handle primary source material critically Become more independent and pragmatic as learners

The story so far Lilac 2005 Second year students are more information literate than first year students First year students have a strong preference for using the internet to source information Second year students use the Library Catalogue as their primary literature searching tool Both groups use the Library Catalogue ineffectively

Current study Aims: Assess the reliability of the data produced by the information literacy questionnaire Provide detailed information about the progression of information literacy skills acquisition by students Assess the relevance of information literacy skills to academic performance

A reliable tool? Comparing results No significant differences between the performance of the two first year groups ( / ): t = 0.909, df = 382 p=0.364 No significant differences between the performance of the two second year groups ( / ): t =1.58, df = 383, p>0.115 Year of entry First year Students Mean % correct +/- (SED) (9.89)47.96 (11.67) Second Year Students Mean % correct +/- (SED) (12.80)59.63 (13.22)

Tracking progression Second year students do progress in Information Literacy

Progression Areas of significant progression Search strategies Locating and accessing information Organising and applying information Areas of less progression Recognising an information need Identifying ways to fill an information need Comparing and evaluating information

IL progression Second year students are: Better at keyword searches Better users of Boolean logic

IL progression Second year students are far more competent at referencing Neither group have sufficient understanding of copyright

IL progression Both groups fail to use search facilities effectively More second year students understand the usefulness of a bibliography in a printed book

Less progression Good performance by both groups: Very good at acknowledging gaps in their information Appropriate strategies for dealing with information overload Room for improvement: Lack skepticism First year students use of online encyclopedias

Less progression Good performance in both groups: Understand purpose of journal article abstracts Over ½ correctly identified differences between websites and peer reviewed journal articles Understand peer review process

Less progression Poor performance by both groups: First year students source most information from internet search engines Second year students are more discerning Second year students look to the internet for recent academic information Review articles: both groups unaware of appropriate use

IL in the curriculum

Academic performance First year Students In general, Information Literacy ability of students entering Higher Education is strongly related to their academic performance in year one semester one at University Mean Academic Performance: (r =0.349, n =130, p<0.001) HIGHLY SIGNIFICANT Multiple choice examination performance: (r = 0.326, n=130, P<0.001) HIGHLY SIGNIFICANT Practical report writing performance: (r = 0.208, n = 130, p<0.05) SIGNIFICANT Essay writing performance: (r = 0.152, n = 130, P=0.084) NOT SIGNIFICANT

Academic performance Second year students After some initial Information Literacy training in Higher Education the strong relationship between IL ability and academic performance is enhanced Mean Academic Performance: (r =0.355, n =118, p<0.001) HIGHLY SIGNIFICANT Multiple choice examination performance: (r = 0.290, n=118, P=0.001) HIGHLY SIGNIFICANT Practical report writing performance: (r = 0.269, n = 118, p=0.03) SIGNIFICANT Essay writing performance: (r = 0.181, n = 118, P<0.05) SIGNIFICANT

Summary Reliability Data produced by the questionnaire is reliable across different cohorts of students and the LILAC 2005 data suggesting a progression of IL ability from year 1 to year 2 has been replicated Progression The Questionnaire now provides detailed information about student abilities including areas of substantial or no progression Academic performance Information Literacy is strongly related to academic performance

Newton-Harrison 7 pillars of evidence based IL training Pre-training Recognise info needof your students Identify ways to fill theneeds of your students Construct a training programme Delivery of training Compare & evaluatestudent abilities and progression Analyse studentperformance data Synthesise data & createevidence-based training Post-training Pre-training

Future plans 3 rd year student data to be obtained Re-examine IL in the Psychology curriculum Further investigation into academic performance: A-level subjects Dissertation results (3 rd year students) Final degree results

References Harrison, A. and A. Newton (2005). How information literate are our incoming undergraduates? Lilac 2005, Imperial College.[Online] [accessed 20th November 2006] Available from world wide web F941D2D096CB/0/newton.pdf F941D2D096CB/0/newton.pdf Mittermeyer, Diane, and Quiron, Diane. Information Literacy: Study of incoming First-year undergraduates in Quebec. [online] 2002 [accessed 20th November 2006] Available from world wide web Ofcom (2006). Media Literacy Audit: Report on media literacy amongst children [Online] [accessed 14 th March 2007] Available from world wide web dren/ dren/ SCONUL Information Skills Task Force (2004). Seven pillars of information literacy [[Online] [accessed 14 th March 2007] Available from world wide web