1. Georgia Alternate Assessment Teacher Training Workshop Fall 2009

2. 2 A New Year Begins  This slide presentation is designed to assist educators as they work to improve and fine-tune student portfolio submissions for the 2009-2010 GAA.  A companion presentation is available online for new teachers to guide them through the initial process of compiling portfolios.  These presentations serve as introductory components for training. Reading and understanding the GAA Examiner’s Manual, 2009-2010, is necessary to implement the portfolio process. 2009-2010

3. 3 A New Year Begins  A Look Back at the 2008–2009 Administration  It’s Raining Elements—Don’t Forget the Umbrella  Alignment  Being True to the Standard  Aligning to the Specific Aspects of the Element

4. 4 A New Year Begins  It’s All in the Details  Filling out the Entry Sheet  Choosing the Appropriate Type of Evidence  Annotating Evidence  Documentation of Level of Independence/Prompting  Who Said What to Who?  Documentation of Interaction for Generalization  Documentation of Interaction to Demonstrate Independence  Interactive Documentation Activity

5. 5 A New Year Begins  All Eyes On It  How to Structure a Peer Review of the Portfolio  Checklist for Teachers and Portfolio Reviewers  And the Results Are In  Training Procedures  Scoring Dimensions–What Do We Look for During Scoring  It’s Your Turn  A Sneak Peek at the 2010-2011 Administration

6. 6 That Was So Last Year! A Look Back at 2008-2009  Congratulations on a very successful administration of the 2008-2009 GAA!  Each administration has shown marked improvements in the quality of the portfolio entries.  Instructional activities were well thought out and purposeful for the students.  In general, the portfolios were complete and well executed.  Entries were well organized. 2008-2009

7. 7 That Was So Last Year! GAA Portfolios Submitted 2006-20072007-20082008-2009 Number of Portfolios submitted 10,64710,82210,993 Number of Portfolio Entries 59,02059,94460,778 The number of portfolios submitted for the GAA has gone up with each subsequent administration. The number of portfolio entries submitted is a factor of how many portfolios are submitted by grade multiplied by how many entries are required for that grade.  Grades K–2:4 entries  Grades 3–8 and 11:6 entries

8. 8 That Was So Last Year!  Hooray! The overall percentage of nonscorable entries went down.  There continue to be decreases in both number and percent of nonscorables seen in most categories.  Analysis of nonscorables provides invaluable information for the focus of future professional development (especially in those areas that increased or remained the same).  These decreases can be attributed to both the experience of the “Year 3” administration and the focus on training and peer review of portfolios.  Continued emphasis on Peer Review is critical.

9. 9 Frequency of Nonscorables by Number and Percent 2008-2009 NS CodeNumberPercentComparison to 07-08 ME2520.41% ▼ ES1930.32% ▲ NA31525.19% ▲ IE18723.08% ▼ IT1430.24% ▼ OG370.06% ▼ IS3 ≈ 0.0% ▼ Total5652100% 5652 of 60,7789.30% ▼

10. 10 That Was So Last Year!  The portfolios provided valuable information as to areas of focus for ongoing professional development.  Alignment  Annotating Evidence  Documentation of Level of Independence  Documentation of Interactions  Training and support will continue to be provided in the form of:  GAA Fall Workshops,  Elluminate sessions focusing on GAA administration as well as access to the curriculum,  the GAA Resource Board, and  portfolio review by peers and building administrators.

11. 11 Instruction should be adjusted during the school year based on student performance on the GAA and other formative assessments. Don’t wait until reports arrive in June to reevaluate the most appropriate mode of instruction and/or assessment for the individual student. How does the GAA connect to daily practice?

12. 12 It’s Raining Elements: Don’t Forget the Umbrella! Instruction and assessment should promote individual student growth through alignment to the academic content via alternate achievement standards.  Alternate achievement standards are decreased in depth, breadth, and complexity, but still demonstrate a clear connection to the academic content standards. Standards

13. 13 It’s Raining Elements: Don’t Forget the Umbrella!  Consider alignment first and foremost when designing instructional tasks.  The instructional task must be true to the standard.  The task must address the distinct characteristics of the element.  The task must be appropriately challenging for the individual student.

14. 14 What do we mean by Alignment?  Alignment is the match between the written, taught, and tested curriculum.¹  In order for an instructional task to be considered aligned, it must demonstrate a clear connection to the Academic Content Standard and element being tested. 1. Diane Browder, 2006 Curriculum Standard Instruction Assessed Task

15. 15 It’s Raining Elements: Don’t Forget the Umbrella!  Be True to the Standard  The curriculum standards are the goals for instruction, learning and assessment.  Achievement of the concepts and skills inherent in the elements leads to the achievement of the overall standard.  Although tasks for assessment must align to the distinct aspects of the element, they must do so under the umbrella of the standard.

16. 16 It’s Raining Elements: Don’t Forget the Umbrella!  Address the distinct characteristics of the element.  What are the specific components that make-up the element ? focus on the language/terminology as written  What are some prerequisite skills to give the student access to the element?  Look to the GA Frameworks* for guidance to understanding the enduring concepts and essential components targeted by the standards and elements. *https://www.georgiastandards.org/Frameworks.

17. 17 It’s Raining Elements: Don’t Forget the Umbrella! Example: Standard: ELA11LSV1– The student participates in student-to- teacher, student-to student, and group verbal interactions. Element: c– responds to question with appropriate information  The essence of this standard is reciprocal interaction between the student and another person.  The essence of the element is the response to questions.  The skill assessed must demonstrate the student’s ability to respond to questions via reciprocal interaction between the student and teacher/ another student/ group.  All 4 pieces of evidence must align to the standard and element. Consider the following examples:

18. 18 The student is responding to questions via his voice output device. Does this task align to the standard and element? YES ELA11 LSV1

19. 19 Teacher annotation recognizes the requirement that the essence of the element is the reciprocal interaction. Does this task align to the standard and element? NO. The task was completed independently with no reciprocal interaction. Although the student is responding to test questions. the annotation clearly states that there was no interaction. NOT ALIGNED–DO NOT USE

20. 20 It’s Raining Elements: Don’t Forget the Umbrella!  As teaching academic curriculum through the academic content standards and elements becomes more a part of daily instruction, lesson plans are being designed that provide access to the curriculum while still embedding the student’s IEP goals.  But Alignment MUST come first!

21. 21 It’s Raining Elements: Don’t Forget the Umbrella!  Creating instructional units and activities that can be used for multiple students is encouraged.  However, the activities must be tailored to the needs and the abilities of the individual student.  As such, be certain that the task is still the best choice for the individual student and that the evidence submitted clearly aligns to the academic content standard and element.

22. 22 It’s All in the Details– Completing the Entry Sheet  The Entry Sheet serves as the Table of Contents which organizes the entry.  The Entry Sheet must be filled out completely and accurately in order for the entry to be scorable.  An electronic version of the Entry Sheet with drop- down boxes will be available online.  Instructions for completing the electronic Entry Sheet will be provided online along with the Entry Sheet.

23. 23 2009-2010 GAA Entry Sheet pg. 1

24. 24 pg. 2

25. 25 It’s All in the Details Completing the Entry Sheet  It is of utmost importance that the Entry Sheet be filled out completely with all required information.  Dates recorded for the tasks on the Entry Sheet must match those found on the evidence.  Task descriptions written on the Entry Sheet must be the same as those submitted as evidence.  A Characteristic of Science must be recorded on the Science Entry Sheet and be clearly documented in the evidence.  Should any of the necessary fields not be completed correctly, the entry could be nonscorable.  The Entry Sheet is NOT the place to include annotations about student performance, prompting, settings, or interactions.

26. 26 It’s All in the Details Choosing the Appropriate Type of Evidence  The type of evidence submitted should be the best means through which to demonstrate the student’s knowledge and skills.  Primary Evidence must SHOW the student’s responses during and at the completion of the instructional activity.  It is therefore vital that the type of evidence used is the appropriate choice to clearly demonstrate the student’s response.  It is important that the criteria for the type of evidence has been met and that all necessary information has been documented.  It is not recommended that worksheets or captioned photos be used to document “verbal” responses–this makes them more like an observation than a primary type of evidence and puts excessive burden on the teacher in their documentation.

27. 27 It’s All in the Details Annotating Evidence  Complete and thorough documentation of evidence is critical!  Incomplete or ineffective documentation can result in lower scores or in the entry being nonscorable.  The student’s response must be clearly and specifically evaluated or graded.  If the correctness of the student response cannot be verified, the entry will receive the Nonscorable Code of IE (Insufficient Evidence).  Information regarding the nature of the task, the setting in which it was completed, any interactions that occurred during the task, and the type and frequency of prompting must be included.

28. 28 It’s All in the Details Annotating Evidence Each piece of evidence must be clearly labeled with the following information: the student's name (Who) and date (When) description of task (What) the setting in which the task was completed (Where) specific evaluation of student response (How Well) interactions that occurred during the task (With Whom) Independence–type and frequency of prompting (Prompts)

29. 29 It’s All in the Details– Annotating Evidence Who: the student’s name must be on each piece of evidence always refer to the student by name, not “the student” or “students” What: specific description of the task what was the student asked to do? do not be vague or overly general in the description Where: setting in which the task was completed must be purposeful for the particular task

30. 30 It’s All in the Details– Annotating Evidence When: date each piece of evidence record the date on which the task was completed, even if the task took multiple days to complete for Data Sheets, a minimum of 3 distinct dates are required for scoring How Well: evaluate the student’s performance document the questions or actions asked of the student and his/her actual responses grade, score, evaluate, or provide an answer key so that the student’s performance can be clearly determined

31. 31 It’s All in the Details– Annotating Evidence With Whom: describe the interaction with whom did the student interact during the task and what was the nature of the interaction reciprocal communication should be specifically documented Prompts: Level of Independence prompting should be documented only if it guides the student to the correct answer (e.g., directions and encouragement are not considered prompting) annotate both the type (e.g., physical, gestural, model, verbal) and frequency (e.g., continuous, frequent, limited, independent) of prompting provided

32. 32 It’s All in the Details– Annotating Evidence  Level of Independence  Increased independence, whether during academic or functional tasks, is a primary goal for our students and an effective way through which to demonstrate Achievement/Progress.  Tasks should be designed to demonstrate the highest level of independent response of which the student is capable.  If the student can demonstrate a differentiated response via eye gaze, gesture, vocalization, or assistive technology, the student’s performance should be a reflection of that independent response before full physical prompting is employed.

33. 33 It’s All in the Details– Annotating Evidence  Prompting– The amount of support the student requires and is given to accurately complete a task  For the purpose of the assessment, prompting refers only to that which leads the student to the correct answer.  Do not include task instructions, encouragement, or behavioral interventions when documenting prompting.  Document the Type of Prompt provided (verbal, gestural, model, physical, etc.)  Document the frequency of prompting (continuous, frequent, limited, none/independent, etc.)  Documentation of Prompting should include a description of reciprocal interaction when appropriate.

34. 34 Who Said What to Who? Interaction:  The reciprocal exchange/communication between the student being assessed and others which occurs during the instructional activity. This can include: peers (both with and without disabilities) instructional personnel (including the special education teacher, para-pro, general education teachers, OT, PT, Speech Therapist, or anyone else who provides regular support and instruction to the student) school staff (principal, nurse, cafeteria worker, etc.) community members (job supervisor, neighbor, bus driver, wait staff, cashier, etc.)

35. 35 Who Said What to Who? Interaction  In order to demonstrate Generalization, annotation of interaction must describe the nature of the interaction. the interaction must occur during the instructional task who said what to whom in reference to the task do not provide a list of all the people with whom the student has interacted without describing how that interaction occurred during the instructional task differentiate between prompting and interaction whenever possible

36. 36 Who Said What to Who?  The interaction must occur during the instructional task. Scenario 1: M4G1, element c, Examine and classify quadrilaterals. Billy is in the school hallway looking for and recording the quadrilaterals he observes in the school environment (door: rectangle; library book: square) when Scott, a non-disabled general education peer, passes by and says hello. Billy returns the greeting and they speak for a few moments about the Brave’s win against the Cubs before Billy resumes his task. Does this reciprocal communication qualify as interaction that occurred during the instructional task? NO.

37. 37 Who Said What to Who?  The interaction must occur during the instructional task. Scenario 2: M4G1, element c, Examine and classify quadrilaterals. Billy is in the school hallway looking for and recording the quadrilaterals he observes in the school environment (door: rectangle; library book: square). Scott, a non-disabled general education peer, is working with Billy on this task and asks him questions (Is that book a quadrilateral? What shape is the book?) as they walk through the school. Does this reciprocal communication qualify as interaction that occurred during the instructional task? YES.

38. 38 Who Said What to Who?  The interaction must occur during the instructional task. Scenario 3: ELA4LSV1, element c, Responds to questions with appropriate information. For his second ELA entry, Billy will be assessed on his ability to respond appropriately to questions during a conversation. He is in the school hallway looking for and recording the quadrilaterals he observes in the school environment when Scott, a non-disabled general education peer, passes by and says hello. He asks Billy if he watched the Braves game last night, and Billy responds that he did. Scott then asks if Billy caught the final score, and Billy responds, “yes, it was 4-3, Braves.” Billy then goes on to complete his geometry task. Does this reciprocal communication qualify as interaction that occurred during the instructional task? YES.

39. 39 Who Said What to Who?  What is the difference between interaction and prompting?  Although there is certainly interaction between student and teacher via prompting, beginning in grade 3, L/S/V standards require that the interaction documents “student-to-teacher, student-to-student, and group verbal interaction.”  “Verbal” interaction refers to whatever mode of reciprocal communication the student is capable of.  Interaction must be specific to the element assessed. If the student is to “respond to questions,” it should be during “student-to-teacher, student-to-student, and group verbal interaction.” I asked him... And he replied...

40. 40 Interactive Documentation Activity Read the Observation Form provided in your packet and consider the following questions. ELA8R1 b. Applies, analyzes, and evaluates common organizational structures (e.g., graphic organizers, logical order, cause and effect relationships, comparison and contrast) What is the task? Does it align to the standard and element? Has all required documentation been included? Has the student’s performance on the task been clearly and specifically evaluated? Is the documentation of interactions clearly described and specific to those that occurred during the instructional task?

41. 41 Interactive Documentation Activity Example: By focusing on one specific task that aligns to the standard and element, this Observation Form includes all information required for scoring and will allow the student to receive the most appropriate score.

42. 42 All Eyes On It Portfolio Review  The importance of peer review during and after completion of a portfolio cannot be overstated!  Missing forms/pieces of evidence  Incomplete Entry Sheet  Lacking documentation  Missing dates  Inappropriate materials  Alignment issues  It is critical that the peer review process go beyond counting pieces of evidence and consider all of the portfolio requirements.

43. 43 All Eyes On It Portfolio Review Portfolio review by the teacher should be an ongoing process. Have I selected the best standard and element for this student? Does he seem engaged? Is he making progress? Are the tasks and materials appropriate? Is he able to demonstrate what he knows? Can he communicate with a reliable response?

44. 44 All Eyes On It Structuring a Peer Review  Who should conduct the portfolio review?  Trained GAA Administrator  Core Access Teacher  School Test Coordinator  SPED Coordinator  Building Administrator  When should the review be conducted?  1 st Review: mid-administration, after Collection Period 1 evidence has been compiled  Final Review: before submission, after all evidence has been compiled and organized in the portfolio binder

45. 45 All Eyes On It Structuring a Peer Review Steps to complete a peer review of the portfolio: 1.Carefully review the Entry Sheet Student name Teacher name Required standard Eligible standard Strand, standard, element match up Characteristic of Science Task Descriptions

46. 46 All Eyes On It Structuring a Peer Review 2. Consider all aspects of the evidence requirements All four pieces of evidence align to standard and element Primary and Secondary for each collection period 21 calendar days from Primary to Primary Type of evidence is the best choice to clearly demonstrate the student response Grade-appropriate materials

47. 47 All Eyes On It Structuring a Peer Review 3. Review documentation to ensure that all necessary annotation has been provided Collection period labels Name (Who) Dates (When) Task (What) Setting (Where) Student performance (How well) Interactions (With Whom and Describe) Independence (Prompts) 4.Date and sign Checklist for Teachers and Portfolio Reviewers 5.Validation Form must be signed by Building Administrator and Person Responsible for submitting the Portfolio.

48. 48 A Checklist for Teachers and Portfolio Reviewers will be provided in the portfolio binder to ensure that all procedures and requirements have been satisfied before the portfolio is submitted. The checklist can be part of the portfolio validation process as the reviewer signs and dates the form after each content area entry is checked. It is recommended that the portfolio be reviewed twice– once after the first collection period has been completed, and again before the portfolio is submitted. The GAA Evidence Checklist, specific to each type of evidence submitted, can be found in the GAA Examiner’s Manual.

49. 49 The GAA Validation Form will be provided in the portfolio binder to verify that all requirements and procedures have been followed and that the contents are the work of the student being assessed. This is an important step- the signatures validate the contents of the portfolio Be certain that signatures have been obtained from both the person submitting the portfolio and the building administrator. For transfer students, Validation Forms must be submitted by both the sending and the receiving school/system. Required signature Administrator checks one box

50. 50 And the Results Are In What Do We Look for When Scoring the GAA Portfolios? Evidence has been compiled, Entry Sheets have been completed, and the entries have been organized. Portfolios have undergone peer review and have been determined to be ready for submission. Binders have been packed in boxes and sent from the School to the System Test Coordinator and on to Questar. It’s time for scoring. 2 3 3

51. 51 And the Results Are In Training Procedures  Readers are trained to score portfolios using entries that have been scored during rangefinding sessions in Georgia with Georgia educators.  Rangefinding is a process wherein teachers score actual student entries to set the score point ranges in each dimension (e.g., determining what it takes to get a “3” in Achievement/Progress).  Entries with consensus scores are used to create training and qualifying sets for readers.  Representatives from the GaDOE are involved throughout rangefinding and are on-site and/or in constant contact throughout training and scoring.

52. 52 And the Results Are In Training Procedures  Readers undergo 4–5 days of extensive training and must pass a series of qualifying tests to demonstrate that they know how to apply the scoring rubric before they can begin scoring.  Readers are monitored throughout the scoring process to ensure they are scoring accurately and consistently.  Team leaders, who serve as nonscorable experts, have previous experience in scoring the GAA and go through additional extensive training before being charged with assigning scores and nonscorable codes.

53. 53 And the Results Are In  GAA Portfolios are scored for 4 discrete dimensions Fidelity to Standard Context Achievement/ Progress Generalization  Scoring is holistic – all pieces of evidence are considered and the totality of the information we have about the student’s achievement is used to make scoring decisions.

54. 54 And the Results Are In Fidelity to Standard assesses the degree to which the student’s work addresses the grade-level standard to which it is aligned. Does the instructional activity demonstrate a clear connection to the standard and element? Is the student work focused on academic content at a very introductory level considering the student’s grade level? Is the student work focused on academic content at or approaching the student’s grade level? Does the student work address all aspects of the element?

55. 55 And the Results Are In Context assesses the degree to which the student work exhibits the use of grade-appropriate materials in a purposeful and natural/real-world application. Are all the materials grade appropriate? Is the instructional activity a purposeful means through which the student can learn and demonstrate what they know and can do? Is the student working in a simulated (practice) situation? (Almost all classroom instruction is considered “simulated.”) Is the student working in a real-world (following a list to purchase groceries) or natural situation (working in the general education classroom on the same activity as general education peers)?

56. 56 And the Results Are In Achievement/Progress assesses the increase in the student’s proficiency of skill across the two collection periods. Are the skills assessed across the collection periods similar enough to reliably assess progress? Is there an increase in accuracy from one collection period to another? Is there an increase in independence from one collection period to another? Is there an increase in the complexity of the tasks from one collection period to another?

57. 57 And the Results Are In Generalization assesses the student’s opportunity to apply the learned skill in other settings and/or with various individuals in addition to the teacher or paraprofessional. In what meaningful settings is the student performing the activities? (The setting should be purposeful for the instructional task.) With whom and it what way is the student interacting during the standards-based instructional activity?

58. 58

59. 59 And the Results Are In Now, it’s your turn... Review the following student entry and assign scores for Fidelity to Standard, Context, and Achievement/Progress Discuss as a group.

60. 60

61. 61

62. 62

63. 63

64. 64

65. 65

66. 66 A Sneak Peek New in 2009-2010  The state’s new graduation rule went into effect for entering 9 th graders in 2008-2009.  Students with significant cognitive disabilities are eligible for a high school diploma two criteria are met: 1.Enrollment in and successful completion of a series of access courses. 2.A score of established or extending on all sections of the high school GAA. The grade 11 GAA serves an alternate for the GHSGT.

67. 67 A Sneak Peek New in 2009-2010 A GAA must be developed for these courses Several access courses correspond to End of Course Tests (EOCT):  Ninth Grade Literature and Composition  American Literature and Composition  Mathematics I  Mathematics II  Physical Science  Biology  U.S. History  Economics.

68. 68 A Sneak Peek New in 2009-2010 Development of new EOCT for GAA will occur in 2009-2010, with operational administration in the 2010-2011 school year.

69. 69 A Sneak Peek  GHSGT math will transition from QCC to GPS  GAA math for grade 11 will transition to GPS  EOCTs are mandated for all students who are enrolled in an EOCT course.  Retest opportunities will be available for the high school GAA.  New W2 standards/elements assessed for grades K-3.  The 2010-2011 Entry Sheets will be web-based in order to collect standards and elements. There will be no paper Entry Sheets in binders you receive– save some trees! You can still download from the website. 2010-2011 Have on Your Radar for 2010-2011

70. 70 Contact Information Questions About Test Administration  Call:GaDOE Assessment Administration Division Toll free (800) 634-4106  Call: Deborah Houston, Assessment Specialist (404) 657-0251  Email: dhouston@doe.k12.ga.us

71. 71 Contact Information For information about access to the GPS for students with significant cognitive disabilities  Contact: Toni Bowen, Ph.D. Educational Program Specialist  Call: 404-463-0616  E-Mail:TBowen@doe.k12.ga.us

72. 72 Contact Information Questions About Materials, Distribution, or Collection  Call:Questar’s GAA Customer Service Toll free (866) 997-0698  Email:Questar’s GAA Customer Service GA@QuestarAI.com