1. Moving NAPLAN Online ACE 2015 National Conference 24 September, 2015 Stanley Rabinowitz, Ph.D General Manager, Assessment and Reporting

2. A changing landscape Continual refinements and improvements to NAPLAN (2014 – 2016) Highlight achievement and growth Federal, State and Territory education ministers have agreed that NAPLAN will move online from 2017, over a two-to-three year period. Significant planning, development, research and trialling to ensure we are all ready to move NAPLAN online. State/territory/jurisdiction education authorities will: –make decisions about the logistics and timing to move NAPLAN online for their sector. –will lead implementation. 2

3. Expanding the debate… What is a “successful” school? 3 Growth Status III I IV II

4. Advantages of moving online NAPLAN online will provide better assessment, more precise results and faster turnaround of information. NAPLAN online will mean that the tests better meet the needs of all students, including those with a disability. NAPLAN online will use adaptive ‘tailored testing design’ which gives students questions better suited to their achievement level, resulting in better assessment and more precise results. 4

5. Why computer adaptive (TTD) testing? 5

6. NAPLAN online research program Essential studies to move online Informed by “best practice in Australia and internationally Therefore every study focuses on 4 key questions: –What does the research show? –What are the “lessons learned” in other programs? –How do these “lessons” apply to the Australian/NAPLAN context? –How do “real” students react when they sit for online items (Cognitive Interviews) 6

7. NAPLAN online research and development plan StudyPurpose of study Trialling study – 2015 To evaluate:  item and testlet performance  testlet construction and targeting of items To finalise parameters for TTD in all domains (numeracy, reading, spelling, G&P) Device effect study - 2015 What is the magnitude of measurement invariance when NAPLAN is delivered across devices? Are there variances that are specific to domains? Readability and layout study - 2015 Investigate students’ engagement with Reading prompts and material in NAPLAN Reading tests; investigate the presentation of material on the screen, access to the material during the tests, options for scrolling, and manipulating size and position of the Reading prompts 7

8. NAPLAN online research and development plan cont. StudyPurpose of study Impact of accessibility adjustments on performance - 2016 Need accurate/valid measures of proficiency of all students, including those using adjustments Autoscoring (Automated scoring of writing essays and CRs) - 2015/2016 Conduct research needed to complete customisation of AES Does customisation of algorithms provide valid results for NAPLAN writing? Setting of algorithms for training of marking Scaling study (includes linking study) - 2016 Establish the online scale in advance of the 2017 NAPLAN tests in order to provide faster turnaround of NAPLAN results using the representative sample of Australian schools and students. Proficiency levels and standards - 2016 Establish new and enhanced proficiency standard(s) and performance bands for NAPLAN tests aligned with the new Assessment Framework and Australian Curriculum 8

9. NAPLAN online Item Types 9 Interactive Item TypeDetails Choice Select one or more options by clicking checkboxes or buttons Inline choiceSelect an option from a drop-down list Interactive gap matchDrag options to blank spaces within text. Text entryTyping into an answer box – short response Extended textTyping into an answer box – long response. Hot text Select predefined words or phrases from a text passage by clicking. Interactive match Classify sources into categories by dragging them into the appropriate cell of a table. Match Classify sources into categories by clicking checkboxes in a grid Interactive match (draw lines) Match sources to destinations by dragging out lines from a list of sources to a list of destinations. Interactive orderDrag sources into correct order

10. NAPLAN online Item Types 10 Interactive Item TypeDetails SequenceOrder sources by numbering them from 1 to n. Slider Drag a slider marker along a slider bar to select a numerical value. Hotspot Select predefined regions on an image by clicking them. Select pointSelect a point on an image by clicking. Position objectDrag a source to position it correctly on an image Interactive graphic order Order hotspots on an image by dragging numbers from 1 to n to the hotspots. Interactive associate Associate pairs of sources by dragging them into table columns Interactive graphic associate Associate pairs of predefined regions on an image by dragging out lines between them Interactive graphic gap match Drag sources onto predefined hotspots on an image DrawingUse system drawing tools to construct an image

11. NAPLAN online Project Plan 11