Presentation is loading. Please wait.

Presentation is loading. Please wait.

WJEC Examination Review (Examples of forthcoming online feedback available from the autumn term)

Similar presentations


Presentation on theme: "WJEC Examination Review (Examples of forthcoming online feedback available from the autumn term)"— Presentation transcript:

1 WJEC Examination Review (Examples of forthcoming online feedback available from the autumn term)

2 General overview Item level data 1 1 Exam paper 2 2 Marking scheme 3 3 Examiner’s report 4 4 At WJEC we are always looking for ways to improve provision to our customers. We provide the following information relating to examination sessions: At WJEC we are always looking for ways to improve provision to our customers. We provide the following information relating to examination sessions: However, we want to ensure that these materials are used in the most effective way, to recognise weaknesses and improve standards within the classroom.

3 Exam paper Examiner’s report Marking scheme General overview Recognising weaknesses Using the ‘Facility Factor’ from the Item Level Data, it is possible to focus on those questions which caused the most problems for candidates. The online examination review provision is intended to lead teachers through the process of studying the data in detail, referring to the examination paper, marking scheme and examiner’s report to evaluate candidate responses and consider the way in which these elements are taught. At the same time teachers are encouraged to compare the information presented with their own centre specific data. Using the ‘Facility Factor’ from the Item Level Data, it is possible to focus on those questions which caused the most problems for candidates. The online examination review provision is intended to lead teachers through the process of studying the data in detail, referring to the examination paper, marking scheme and examiner’s report to evaluate candidate responses and consider the way in which these elements are taught. At the same time teachers are encouraged to compare the information presented with their own centre specific data.

4 Exam paper Examiner’s report Marking scheme General overview Recognising weaknesses What do you think caused the problems for the candidates? The question Q2 The online exam review first identifies questions which appear to have caused some candidates problems. It then asks users to consider what candidates might have found difficult when answering the question. The online exam review first identifies questions which appear to have caused some candidates problems. It then asks users to consider what candidates might have found difficult when answering the question.

5 Exam paper Examiner’s report Marking scheme General overview Recognising weaknesses What do you think caused the problems for the candidates? A chance to mark an example answer using the mark scheme Exemplar answers The question Q2 Then users are asked to consider some examples of answers given by the candidates, and mark them using the mark scheme.

6 Exam paper Examiner’s report Marking scheme General overview Recognising weaknesses What do you think caused the problems for the candidates? A chance to mark an example answer using the mark scheme Compare this to the examiners’ comments Exemplar answers The question Q2 Then there is an opportunity for users to compare their thoughts with examiner comments.

7 Exam paper Examiner’s report Marking scheme General overview Recognising weaknesses What do you think caused the problems for the candidates? A chance to mark an example answer using the mark scheme Compare this with the examiner’s comments Do we have to adapt the way in which we teach this aspect of the course? Have we learnt something new from this? Exemplar answers The question Q2

8 An example of online feedback from the January 2013 exams

9 A feedback module will look like this when opened. The following slides will take you through this example module, providing instructions on how it should be used and explaining the tools available. A feedback module will look like this when opened. The following slides will take you through this example module, providing instructions on how it should be used and explaining the tools available.

10 A calculator is available to use from any page by clicking the spanner icon, and then the maths symbols icon. A timer is available to use from any page by clicking the spanner icon, and then the clock icon. This might be of use if the module is being used in the classroom.

11 Click the X to close the calculator and the clock. Start Reset Stop The clock functions are:-

12 Click the question mark icon while using the unit for more information on the tools available.

13 Click this icon to maximise the window. Press Esc to exit the full screen mode.

14 Click this icon to get a list of pages available. This is a quick way of navigating to the desired page. Just click the page you need.

15 Click one of these icons to move backwards or forwards through the pages.

16 NOTES: 1.The item level marks used in the item level analysis and reported for each candidate are those submitted by the examiner at the time of marking. They will not include any changes made as a result of a remark, neither will they include changes made as a result of errors discovered during the internal script check that takes place prior to the issue of results. 2.There is a note of caution to be sounded. If the number of candidates is relatively small, the performance of one individual may have a disproportionate effect on the performance of the group as a whole. Therefore, care must be taken when making judgement about performance where the number of candidates is small.

17 - Pointer tool - Pen tool - Highlighter tool - Rubber tool (selective delete) - Delete tool - Palette tool (pen and highlighter colour) - Pointer tool - Pen tool - Highlighter tool - Rubber tool (selective delete) - Delete tool - Palette tool (pen and highlighter colour)

18 The pen tool allows the user to make notes on any part of the screen. Choose the colour palette at any time to change the pen colour. This can be useful if there is need to distinguish between different types of comment, or if being used by students in the classroom. The pen tool allows the user to make notes on any part of the screen. Choose the colour palette at any time to change the pen colour. This can be useful if there is need to distinguish between different types of comment, or if being used by students in the classroom.

19 The highlighter tool allows the user to highlight any part of the screen. Choose the colour palette at any time to change the highlighter colour. The highlighter tool allows the user to highlight any part of the screen. Choose the colour palette at any time to change the highlighter colour.

20 Click this icon to delete parts of notes or highlights made. Click this icon to delete all notes and highlights. You will be asked to confirm that you want to clear all. Click this icon to delete all notes and highlights. You will be asked to confirm that you want to clear all.

21 This part of the screen shows the item level data. An item is a questions or part question within a script. The table shows the data for all WJEC candidates attempting this paper. It can be compared with a particular centre’s data in order to evaluate the achievements of the centre’s candidates. (Item level data for individual candidates and centres is available through the WJEC secure website. See the presentation on this facility for more information). WJEC secure website This part of the screen shows the item level data. An item is a questions or part question within a script. The table shows the data for all WJEC candidates attempting this paper. It can be compared with a particular centre’s data in order to evaluate the achievements of the centre’s candidates. (Item level data for individual candidates and centres is available through the WJEC secure website. See the presentation on this facility for more information). WJEC secure website

22 1. Question Title Usually the question number 1. Question Title Usually the question number

23 2. N The number of candidates attempting that question 2. N The number of candidates attempting that question

24 3. Mean (average) mark attained (Mean) The mean score is calculated by adding up the individual candidate scores and dividing by the total number of candidates. If all candidates perform well on a particular item, the mean score will be close to the maximum mark. Conversely, if candidates as a whole perform poorly on the item there will be a large difference between the mean score and the maximum mark. A simple comparison of the mean marks will identify those items that contribute significantly to the overall performance of the candidates. However, because the maximum mark may not be the same for each item, a comparison of the means provides only a partial indication of candidate performance. Equal means does not necessarily imply equal performance. For questions with different maximum marks, the facility factor should be used to compare performance. 3. Mean (average) mark attained (Mean) The mean score is calculated by adding up the individual candidate scores and dividing by the total number of candidates. If all candidates perform well on a particular item, the mean score will be close to the maximum mark. Conversely, if candidates as a whole perform poorly on the item there will be a large difference between the mean score and the maximum mark. A simple comparison of the mean marks will identify those items that contribute significantly to the overall performance of the candidates. However, because the maximum mark may not be the same for each item, a comparison of the means provides only a partial indication of candidate performance. Equal means does not necessarily imply equal performance. For questions with different maximum marks, the facility factor should be used to compare performance.

25 4. Standard Deviation of the marks attained (S.D.) The standard deviation measures the spread of the data about the mean score. The larger the standard deviation is, the more dispersed (or less consistent) the candidate performances are for that item. An increase in the standard deviation points to increased diversity amongst candidates, or to a more discriminating paper, as the marks are more dispersed about the centre. By contrast a decrease in the standard deviation would suggest more homogeneity amongst the candidates, or a less discriminating paper, as candidate marks are more clustered about the centre. 4. Standard Deviation of the marks attained (S.D.) The standard deviation measures the spread of the data about the mean score. The larger the standard deviation is, the more dispersed (or less consistent) the candidate performances are for that item. An increase in the standard deviation points to increased diversity amongst candidates, or to a more discriminating paper, as the marks are more dispersed about the centre. By contrast a decrease in the standard deviation would suggest more homogeneity amongst the candidates, or a less discriminating paper, as candidate marks are more clustered about the centre.

26 5. The Max Mark This is the maximum mark for a particular question. 5. The Max Mark This is the maximum mark for a particular question.

27 6. The facility factor (F.F.) The facility factor for an item expresses the mean mark as a percentage of the maximum mark (Max. Mark) and is a measure of the accessibility of the item. If the mean mark obtained by candidates is close to the maximum mark, the facility factor will be close to 100 per cent and the item would be considered to be very accessible. If on the other hand the mean mark is low when compared with the maximum score, the facility factor will be small and the item considered less accessible to candidates. 6. The facility factor (F.F.) The facility factor for an item expresses the mean mark as a percentage of the maximum mark (Max. Mark) and is a measure of the accessibility of the item. If the mean mark obtained by candidates is close to the maximum mark, the facility factor will be close to 100 per cent and the item would be considered to be very accessible. If on the other hand the mean mark is low when compared with the maximum score, the facility factor will be small and the item considered less accessible to candidates.

28 7. Percentage of candidates who attempted the item (Attempt %) For each item the table shows the number (N) and percentage of candidates who attempted the question. When comparing items on this measure it is important to consider the order in which the items appear on the paper. If the total time available for a paper is limited, there is the possibility of some candidates running out of time. This may result in those items towards the end of the paper having a deflated figure on this measure. If the time allocated to the paper is not considered to be a significant factor, a low percentage may indicate issues of accessibility. Where candidates have a choice of question the statistics evidence candidate preferences, but will also be influenced by the teaching policy within centres. 7. Percentage of candidates who attempted the item (Attempt %) For each item the table shows the number (N) and percentage of candidates who attempted the question. When comparing items on this measure it is important to consider the order in which the items appear on the paper. If the total time available for a paper is limited, there is the possibility of some candidates running out of time. This may result in those items towards the end of the paper having a deflated figure on this measure. If the time allocated to the paper is not considered to be a significant factor, a low percentage may indicate issues of accessibility. Where candidates have a choice of question the statistics evidence candidate preferences, but will also be influenced by the teaching policy within centres.

29 Click this icon to see the ‘Facility Factor’ information in graphical form. The pointer can be moved to wherever needed on the graph.

30 To use the information in the unit as printed sheets or for other purposes, click this icon to open a new window containing the documents in a zip file. Select the ‘Extract all files’ to save the information displayed to folder of your choice

31 The ‘Compressed (zipped) Folders Extraction Wizard’ will open – click Next>

32 You will be asked to choose where you would like the folder to be placed (use the Browse button to select your preferred location), then click Next>.

33 You will now see the ‘Extraction Complete’ message. If the ‘Show extracted files’ box is ticked (as shown) when Finish is clicked, the folder containing all the files will open automatically.

34 Clicking the question mark icon will display questions designed to encourage the user to think about the data, and compare it with his/her own centre based data which may be found on the WJEC secure website. (A presentation on this facility is available here.)WJEC secure website Clicking the question mark icon will display questions designed to encourage the user to think about the data, and compare it with his/her own centre based data which may be found on the WJEC secure website. (A presentation on this facility is available here.)WJEC secure website

35 Scroll through questions by clicking the arrows.

36 Clicking the ‘Show suggestions’ dialogue will highlight the questions the feedback module focuses on.

37 A similar facility is also available in graph view.

38 Clicking on any one of the highlighted sections (in this case Q3, Q6 and Q8) will take the user to a new screen where the actual question can be looked at in detail. For example, if Q3 is chosen… Clicking on any one of the highlighted sections (in this case Q3, Q6 and Q8) will take the user to a new screen where the actual question can be looked at in detail. For example, if Q3 is chosen…

39

40 There are some familiar tools here, and some extra ones to help with use of the module. Generic tool set. Navigation within the module. Download the materials. Pen tool for making notes, highlighting etc. Scroll bar to move through larger questions.

41 The question may be annotated, using the pen or highlighter tool. You are able to scroll up or down the question if it is too long to fit on the screen. The question may be annotated, using the pen or highlighter tool. You are able to scroll up or down the question if it is too long to fit on the screen.

42 Click here to view the item level data again. Click here to get a ‘floating’ mark scheme. Close by clicking the ‘x’ at the top right corner. There are two different candidate answers available to view for this question. Click one to view the candidate response. There are two different candidate answers available to view for this question. Click one to view the candidate response.

43 Candidate 1 answer:

44 Click here to view the item level data for this question. Click here to get a ‘floating’ mark scheme. Close by clicking the ‘x’ at the top right corner. Click here to go back to the question. Click here to see the marks awarded. Click here to see the examiner’s comments.

45 Examiner comments are shown in speech bubbles. Responses for some subjects will have audio comments by the examiner.

46 Examples Click any of the buttons to open examples of other feedback modules. (You must have online access to view these exemplars). NB: These modules relate to examination sessions previous to summer GCE Electronics ET1 GCSE English Literature GCSE Hospitality and Catering GCSE Hospitality and Catering GCSE Media Studies GCE Electronics ET4

47 Advantages Teachers teaching more than one subject have access to the relevant subject materials at any time. These are materials for heads of department to use during departmental training. The materials provide information for NQTs or others teaching subjects for the first time. Leads practitioners through the process of data evaluation, recognition of problem areas and encourages the thought process of evaluating the way some aspects are taught. In the hands of a good teacher, these materials could be used with students in the classroom as assessment for learning tools. Units can be built into a bank of information tackling issues on a similar aspect in a number of subjects. Over time, this bank of materials will grow and provide a comprehensive store of support files for teachers, eventually linking to resources and examples of good practice. Low carbon footprint, as teachers can access the materials at anytime from anywhere. Materials are available totally free of charge. A dedicated website is being developed to host the materials that will be updated regularly as more units become available. The units can provide a basis for discussion, and the sharing of ideas during regional meetings. All staff will have access to the materials. Although the units are available as interactive web based files, all pages can be printed and used as paper-based activities.

48 We hope you have found this presentation useful. Online review of the summer 2013 exams will be available in the autumn. If you have any questions, please us on


Download ppt "WJEC Examination Review (Examples of forthcoming online feedback available from the autumn term)"

Similar presentations


Ads by Google