We have carried out studies at different levels and in different educational contexts Levels 4 and 7 Pilot study Formative assessment Summative assessment BACKGROUND
We asked students and tutors what they thought about the Flexilevel approach… Focus groups Surveys Overall Formative- sounds good Summative- not so keen ATTITUDE
We have quite a lot of data from studies in real educational contexts, so why this study? Practical considerations associated with: Setting up an assessment Setting up the lab Timing- limited number of opportunities for running assessments and we need data Opportunity to see what the results look like without examinees actually experiencing the flexilevel test WHY SIMULATION?
We need more data – lots of points of interest. Requirements for calibration How many items are needed to obtain the same results as standard CBT The influence of different contexts WHY SIMULATION?
Examinee responses from two different computer-based tests (CBTs) were collated for analysis. Level 4 Databases Level 6 Databases Calculate the difficulty of each item PREPARING THE SIMULATION
The items were ranked according to their difficulty The flexilevel algorithm was applied to the selection of the items Stop the simulation after half (or almost half) the number of items are presented APPLYING THE FLEXILEVEL ALGORITHM
In the case of Databases (Level 4), there was a strong, positive correlation between the CBT and Flexilevel scores, which was statistically significant (rs(9) = 0.949, p < 0.01). In the case of Databases (Level 6), there was also a strong, positive correlation between the CBT and Flexilevel scores, which was statistically significant (rs(63) = 0.930, p < 0.01). RESULTS
Further simulation studies Further studies in real educational contexts- points of interest include: Formative assessment in mobile contexts Applying the flexilevel approach to learning activities more generally than assessment NEXT WORK