1. Development of Assessments Laura Mason Consultant

2. Aims  To familiarise you with factors to be considered when staff are developing their own assessments.

3. What is a Unit specification?  An important document which defines the boundaries within which a learner is assessed, and the standards to which assessment or achievement must be made.  Not primarily concerned with material for learning and teaching  Unit specifications are made up of three sections:

4. Unit specification General information Section  Summarises what the Unit is about  Includes key points, e.g. the credit value, recommended entry, a summary of the Unit and the Outcomes, administrative information etc.

5. Unit specification Statement of Standards Sectio n Mandatory part of the Unit specification which clearly states:  The Outcomes that have to be achieved  The Knowledge and Skills values for each Outcome (i.e. the content)  The standard of achievement which must be reached  What evidence must be produced to successfully complete the Unit

6. Unit specification Support notes Section  Doesn’t have any mandatory force  Provided for useful guidance  Expands on the information given in the statement of standards:  Outcome Knowledge and Skills values with examples of content  Assessment Guidelines with examples  Gives clear advice on major aspects of delivery

7. Recap on The Principles of assessment  All of SQA’s assessment principles must be applied whenever instruments/methods of assessment are selected and used. These principles are:  Valid  Reliable  Equitable  Fair  Practicable

8. Principles of assessment Valid  Appropriate assessment methods/ instruments have been applied.  Fit for purpose  Two key aspects of validity used by SQA — content and construct.

9. Principles of assessment Content validity  Detail and level of Knowledge and Skills  The range of subject content to be covered is stated clearly under each Outcome in the Unit Specification  The Instrument of Assessment will be produced to test these K & S Values under the context of the Outcome  Refined by the Unit Evidence Requirements

10. Principles of assessment Construct validity  Does an assessment actually measure what the Unit Specification states it is intended to measure  Relates to either Knowledge or Skills  The method of assessment chosen must be suitable and match e.g. –A practical exercise for skills –A set of restricted response questions for knowledge  Refined by the Unit Evidence Requirements  Guidance in Assessment approaches

11. Activity 1  Analyse the “Extract from IT in Business Spreadsheets Unit” and ‘Extract from Office Management Unit’  Consider whether the proposed assessment tasks are valid.  Choose ‘green card’ if valid and ‘red card’ if not.  Under each card on the flipchart explain the reasons for your choice.  (Note: ER = Evidence Requirements)

12. Activity 1 – Excel Unit The Excel Assessment would be considered NOT valid  The Unit Specification states “create”, Assessment Instrument says write down  The Evidence Requirements states “Link 3 workbooks”, The task does not cover this  While ER 2 may be met, task does not ensure this  No further assessor or marking guidance

13. Activity 1 – Office Management The assessment would be considered VALID  Evidence Requirement states “given scenario” Assessment Instrument says “Case Study provided”  Task instructions state number of examples for levels and characteristics  Candidate instructions on what to included

14. Principles of assessment Reliable Clearly-defined standards of performance were used during assessment.  comparability of the evidence that arises from particular assessments.  an assessment must produce results that are consistent over a range of conditions and circumstances  Suggested solutions developed  Marking Guidelines / Assessment Schemes

15. Activity 2  The Using IT in Business Spreadsheets assessment has been amended and is now valid, please refer to this version.  Analyse the two unit assessments and whether you consider they are Reliable.

16. Activity 2 - Excel  The Using IT in Business Spreadsheets assessment would be considered NOT Reliable  Although task covers the Evidence Requirements –Different assessment conditions applied (timed and not timed) –No Marking Scheme / Guidelines – led to different interpretations –No checklist to ensure coverage of Unit Specification Evidence Requirements

17. Activity 2 – Office Management The Office Management assessment would be considered Reliable  Clear Marking Guidelines with suggested solutions  Checklist referring to Evidence Requirements

18. Principles of assessment Equitable  Assessment is accessible to all candidates who have the potential to be successful in it.

19. Principles of assessment Fair  Assessments have been objectively devised/selected and are free from barriers to achievement.  Assessment practice must ensure all candidates have equal opportunities to succeed.

20. Activity 3  Analyse the two unit assessments and whether you consider they are equitable and fair

21. Activity 3  IT Unit Possibly not –use of wages accounts and language may not be familiar to all candidate groups  OM Unit Probably is –Case study should be in a context suitable for all groups and individuals –Instructions clearly written – scenario should also be  For both, assessor can give reasonable assistance (interpretations)

22. Principles of assessment Practicable  Assessments have been carried out efficiently and effectively.  Assessments are manageable in the context of the available resources, time and within the existing constraints –E.g. Equipment and software for an IT Unit

23. Activity 4  Analyse the two unit assessments and whether you consider they are Practicable

24. Activity 4  The Office Management assessment would be considered practicable –it is a case study (check on reference resources)  The IT in Business Spreadsheets Unit could be practicable –Check on sufficient access to equipment & software  Both would need sufficient assessor / candidate time

25. Developing assessments  Where possible Combine / integrate Units / Outcomes  Use existing Exemplar ASP / Exemplars as guide  Get another member of staff to try it out  After first delivery of an assessment, review areas where candidates show difficulty / poor responses

26. Developing assessments  When writing assessment task instructions keep language simple in line with candidates ability  Avoid jargon – but use business / subject language where appropriate  Can put scenarios in context of country  Assignments can be related to candidates major subject of study e.g. in IT Units can do reports on marketing

27. Developing assessments  Prepare combined checklists –against assessment task –against Outcomes / Knowledge and Skills Values –against Evidence Requirements  If major changes to an existing ASP submit for Prior Verification. Small changes to parameters or context, no need.

28. Questions