Quality Assurance processes

What was the focus? Assessment Component Specifications Assessment Design Criteria Task Specifications Reliable Assessment Decisions Focus Group: Teachers of Stage 2 Tourism Interest in understanding connections – dates back to Grad Dip GIS – final unit – research. When asked what criteria were we going to be assessed against and what were the indicators for HD/D/C/P … response was ‘I don’t know, I was going to wait until they all came in and work it out from there’

Context Chief Assessors’ Report 2015 'best practice task design also features careful selection of assessment design criteria' (p.3) ‘…the task set must reflect the definition of a folio task‘(p. 3). ‘…well-designed tasks supported student achievement and were associated with more-reliable assessment decisions' ( p. 5). Connecting assessment design to the reliability of assessment decisions in this action plan was in response to feedback from the moderation team in 2014 and 2015 identifying areas of concern in relation to teachers' assessment practices. Chief Assessors’ Report 2015 ‘…best practice task design also features careful selection of assessment design criteria' (p.3) ‘…the task set must reflect the definition of a folio task‘(p. 3). ‘…well-designed tasks supported student achievement and were associated with more-reliable assessment decisions' ( p. 5). . Areas of concern with teachers’ assessment practice included; · Assessment tasks not fully reflecting subject specifications, · Incorporation of a significant number of non-specified assessment design criteria diluting emphasis of assessment decisions specific to the assessment component, and · Task specifications that require students to complete complex tasks within very ‘tight’ specifications.

Concerns – Assessment Practice Assessment tasks not fully reflecting subject specifications, Incorporation of a significant number of non-specified assessment design criteria Dilutes emphasis of assessment decisions specific to the assessment component, and Task specifications that require students to complete complex tasks within very ‘tight’ specifications. . Areas of concern with teachers’ assessment practice included; · Assessment tasks not fully reflecting subject specifications, · Incorporation of a significant number of non-specified assessment design criteria diluting emphasis of assessment decisions specific to the assessment component, and · Task specifications that require students to complete complex tasks within very ‘tight’ specifications.

Timeline 2014 2015 2016 Stage 2 Moderation process Moderators identify task design as a factor influencing reliability of teachers’ assessment decisions. 2015 Clarifying Forum – focus on task design for Practical Activities Moderation team reiterate concerns 2016 Clarifying Forum – focus on task design for Folio Activities Moderation team identify best practice examples of task design.

Concepts Validity -‘the extent to which the outcome of the assessment measures what it has been designed to measure’. (Institute of Educational Assessors, 2015) Reliability -‘the degree in which an assessment tool produces stable and consistent results’ (Institute of Educational Assessors, 2015). Cognitive Demand- ‘the knowledge and development of intellectual skills’ required by a task (Institute of Educational Assessors, 2015). Consistent application of assessment standards stems from tasks that are fit-for-purpose and thus valid. Validity refers to ‘the extent to which the outcome of the assessment measures what it has been designed to measure’. (Institute of Educational Assessors, 2015). Stiggins and Chappuis (2006, p. 12) maintain that assessments need to be developed that allow accurate results to be collected. Teachers can therefore apply assessment standards consistently if they understand the way in which task design influences their ability to interpret assessment data accurately. To do so means that their assessment instruments have been developed to ensure that the task measures what it was intended to measure. Where teachers do not maintain the relationship between task and intended outcome, they therefore compromise their ability to apply an assessment standard accurately. Consistent application of assessment standards influences the reliability of the assessment instrument. Reliability refers to ‘the degree in which an assessment tool produces stable and consistent results’ (Institute of Educational Assessors, 2015). The teacher’s ability to apply an assessment standard is therefore compromised if the assessment tool does not produce results that are stable and consistent What is an A in one classroom should be an A in another classroom irrespective of the specifications of the task as long as the tool is design in such a way as to produce stable and consistent results. Cognitive demand influences the accuracy of teachers’ application of assessment standards. Cognitive demand is the ‘knowledge and development of intellectual skills’ required by a task (Institute of Educational Assessors, 2015). It determines the extent to which an assessment task provides opportunities for students to achieve at all grade bands. Tourism moderators expressed concern that teachers had omitted specific features in their task design that target ‘higher order’ skills that require students to demonstrate skills such as evaluate issues or propose solutions. Where this was consistent across an assessment component, students were denied the ability to meet the specifications of the subject outline.

A closer look … Assessment Type 2: Practical Activity (25%) In this assessment type, students develop and demonstrate practical tourism skills. A practical activity is an assessment that in most cases is undertaken beyond the classroom and involves the collection of information from primary sources. This could be undertaken as a class or group. A practical activity may be a field trip where students gather information from primary sources through observation and recording during a visit to a site or activities of tourism interest or relevance. Primary sources could include surveys, interviews, observations, counts, photographs, and field sketches. Students use primary sources of information to explain and make connections between their findings and their knowledge of tourism concepts and/or models, and secondary sources of information about tourism. Students present each practical activity in a form that is a maximum of 1000 words for a written text or a maximum of 6 minutes for an oral presentation, or the equivalent in multimodal form.

. Knowledge and Understanding The specific features are as follows: KU1 Understanding of tourism knowledge. KU2 Knowledge and understanding of tourism concepts and models in different contexts — familiar and less familiar. Analysis and Evaluation AE1 Interpretation and critical analysis of different perspectives on tourism trends, developments, and/or contemporary issues. AE2 Evaluation of the validity, bias, and accuracy of sources of information. AE3 Analysis and evaluation of tourism concepts and models. AE4 Development of informed opinions, conclusions, and recommendations Investigation and Application IA1 Investigation and critical analysis of connections between primary and secondary sources of information. IA2 Application of practical tourism skills in different contexts. IA3 Investigation of tourism trends, developments, and/or contemporary issues. IA4 Application of tourism knowledge, and tourism concepts and models in different contexts — local, national, and global. Communication C1 Communication of informed ideas about tourism, using appropriate forms to suit the purpose and audience. C2 Appropriate use of tourism terminology. C3 Incorporation of a variety of visual and graphical evidence (e.g. maps, photographs, data tables, statistical data, charts, drawings, graphs). C4 Appropriate acknowledgment of sources of information. . Kitchen sink approach to assessment task Cognitive load is too high Teachers make assessment decisions based on one or two sentences using a full A-E spectrum

IA2 Application of practical tourism skills in different contexts. Assessment Component A practical activity is an assessment that in most cases is undertaken beyond the classroom and involves the collection of information from primary sources. This could be undertaken as a class or group. A practical activity may be a field trip where students gather information from primary sources through observation and recording during a visit to a site or activities of tourism interest or relevance. IA2 Application of practical tourism skills in different contexts. Primary sources could include surveys, interviews, observations, counts, photographs, and field sketches. Students use primary sources of information to explain and make connections between their findings and their knowledge of tourism concepts and/or models, and secondary sources of information about tourism. KU2 Knowledge and understanding of tourism concepts and models in different contexts — familiar and less familiar. IA1 Investigation and critical analysis of connections between primary and secondary sources of information. IA4 Application of tourism knowledge, and tourism concepts and models in different contexts — local, national, and global. Students present each practical activity in a form that is a maximum of 1000 words for a written text or a maximum of 6 minutes for an oral presentation, or the equivalent in multimodal form. C1 Communication of informed ideas about tourism, using appropriate forms to suit the purpose and audience.

What did this achieve? Professional discussion Building confidence in Clarifying Forum participants 2017 - Reflective discussion regarding application of assessment concepts, Developing professional network Emphasis of benefit of assessment based, hands-on PD

Where to from here … Clarifying forums moving from live to online formats in 2018 Importance of selection of assessment task samples for SACE mini- sites Role for Teachers’ Associations in supporting assessment literacy