Presentation on theme: "Getting it right: on-line formative assessment of numeracy skills"— Presentation transcript:
1Getting it right: on-line formative assessment of numeracy skills Dr Amanda Cain, Ms Charmian Dawson and Professor Elizabeth ShephardStructural and Molecular BiologyUniversity College London
2Aims of the projectTo assess the underlying numerical abilities of students within our second year courseTo improve these skillsTo support development of ability and confidence in solving numerical problemsIdentified a weakness in project students who struggled to make up solutionsIn year 2 analysis found concepts of percentage solutions and dilutions were often misunderstoodAdded numerical questions to all pre-practical quizzes, tests and exam paper
3Numeracy question from examination paper 75.2% could work out simple molarity (How much NaCl must you weigh out to prepare 300mL of a 0.1M solution of NaCl? The MW of NaCl is ), only 52.3% could work out how much salt needed to prepare 1L of a 6% (w/v) solution?
4Relationship between A’ level grades and numeracy score Aftr first year of pre-prac quizes, 2006, analysed the scores in numerical exercises to look for trends. Found no link to mathematical or overall academic ability – see above, First plot compared the a’level maths grade to scores in numeracy tests. Second takes their overall a’level point score and compares to numeracy grade. See little correlation with eiother previous maths score or overall ability.Looked at results by degree programme (not shown here)– saw a trend to the more molecular subjects scoring better (Biochem, Biotech, Mol Biology) in comparison to Biol, Genetics, Biomed Sci. Group that performed worst were the Biochemical engineers
5Pre-Practical Quizzes Completion required prior to attendance in labShort series of questions on moodle - theoretical and numericalPass mark of at least 60%First trialled in second year Molecular BiologySolution was to introduce a series of pre-practical quizes that had to be completed prior to arrival at the labs.Delivered by VLE – in our case moodle. Used multiple choice to ensure they have read and understood the theoretical basis of each session and calculations to give them practice of the maths behind each session.Monitor completion of quiz as we take attendance – link barcode to quiz score so as they scan in we can see immediately if they have completed each quiz – if not they have to go away!Quizes do not count towards assessment of module but are required to be complete and calculations of the same format are included in both in-course tests and examsSecond year Mol Biol – large course. Taken by Biochem, Biotech. Mol Biol, Genetics, H Genetics, Neuro and some Biol and Biomeds. This year have 165 registered.
6Support for numerical questions Numeracy helplinesPractice quizzesAnswer filesVideosRegular pracice at calculations is important but studens also need to understand the concepts first. Therefore produced a series of numerical helplines as a summer project. Had a team of undergrads in – tell us what they find difficult and come up with ways of explaining it to fellow students. Broke down each step of the calculations.Also set up a practice quiz they could access at any time. Faced with same basic questions but the numers change each time so they can re-inforce their earning.Finally after each lab session we put up a sheet explaining how each of the calculations should be attempted.
7Numeracy Helplines Designed by students Simple explanations Equations to memoriseWorked examples
8Numeracy helplinesDesigned by a student for students. Range of help lines to cover different subjects. Have theme running through – little hearts for forulae they should know by heart etc. Not necessarily the formatting/design we would have chosen! Also gives worked examples they can follow – take them through step by step (student who designed explained that sometimes academics miss bits out as they should be obvious.
15Advantages of on-line support resources Used at anytime and repeatedlyRegular practice of numerical conceptsNo fear of embarrassmentImmediate results/feedbackHelps us to identify areas of weakness which require additional support
16Comparison of equivalent questions on molarity and percentage solutions Can see from above results that regular practice at these sorts of calculations improves overall performance. Loked at the percentage of students getting full marks on both a molarity calculation and one involving % solutions. See a good increase in performance between a year when pre-practical quizes were not used and this last academic year where students were practicing regularly.
17Analysis of numerical results Analysis of this years scores.Students had three possible attempts at each numeracy quiz. Looked at how their mark improved over the weeks. Graph shows averages of best attempt for each student, by week and the number of attempts required to gain that score. Average scores remain similar each time but the number of attempts drops over the first three weeks. (bit of a blip in week 4 – lower average and slight rise in no. of attempts – getting near test time?, deadlines looming?) Week 4 questions more wordy – relate to Southern buffers – several references to stock solutions 10X etc (only one of these in week 3). May be week 4 was harder – required more thought. By end of period also wise up to the fact that they don’t have to get 100% every week.
18Student feedbackTalk about free text comments as well.
19SummaryContinued practice at numerical problems has had a positive effectThe support materials are appropriate and well utilisedThanks to UCL for funding the creation of our support materials