Presentation is loading. Please wait.

Presentation is loading. Please wait.

Overview of the 2012-2013 Alternate Proficiency Assessment Administrator/Test Coordinator Training 12012-2013 NJ APA Administrator Training.

Similar presentations


Presentation on theme: "Overview of the 2012-2013 Alternate Proficiency Assessment Administrator/Test Coordinator Training 12012-2013 NJ APA Administrator Training."— Presentation transcript:

1 Overview of the 2012-2013 Alternate Proficiency Assessment Administrator/Test Coordinator Training 12012-2013 NJ APA Administrator Training

2 Administrator and Teacher Training This session is only for APA administrators. Districts are responsible for teacher training. Teacher training modules are available on the Pearson website http://pem.ncspearson.com/nj/apa An administrator must ensure that all teachers participate in the training sessions. Teachers may be trained immediately, but every effort must be taken to ensure all teachers are trained prior to October 1, 2012. If a student enrolls after October 1 and a teacher needs training, train the teacher as quickly as possible. The modules will be available all year, though it is recommended that you download these modules and retain them in your district. 22012-2013 NJ APA Administrator Training

3 Agenda APA Participation pages 4 – 22 Student Transfers, Medical Leave, and Excessive Absenteeism pages 23 – 31 Administrator Responsibilities pages 32 – 34 Material Survey, Pre-ID data, record changes, etc., pages 36 – 45 APA Test Design pages 46 – 80 Common Errors and APA Revisions pages 81 – 84 Scoring Rubric Information pages 85– 131 Sample Entry from 2011-2012 pages 132 – 140 Professional and Ethical Responsibility When Compiling and Submitting APA page 141 – 146 Score Reports and Accountability pages 147 – 156 Administrator APA Review and Other Tasks pages 157 – 167 Request for Teacher /District Assistance pages 168 - 169 Contact Information pages 170 – 174 Winter Training Materials page 175 32012-2013 NJ APA Administrator Training

4 What is the Purpose of the APA? To measure performance of students with the most significant cognitive disabilities on the CCCS – the knowledge and skills that ALL New Jersey students are learning To ensure that ALL New Jersey students are included in the accountability system that is required by the No Child Left Behind Act 42012-2013 NJ APA Administrator Training

5 What is the APA? An alternate assessment that measures achievement of CCCS specifically designed for students with the most significant cognitive disabilities. A portfolio of student work that demonstrates a students performance of knowledge and skills that are LINKED to grade level knowledge and skills. This is a requirement of the USDOE. 52012-2013 NJ APA Administrator Training

6 Who are the Students who Participate in the APA? Students with the most significant cognitive disabilities Decisions about who participates in the APA are not based on: Students disability category Students educational placement On which assessment the student is most likely to score highest 62012-2013 NJ APA Administrator Training

7 Participation Decisions Students with the most significant cognitive disabilities will participate in: The APA in one/some content area(s) and the general assessment with accommodations in the other content area(s) or The APA in all content areas (LAL, Mathematics, and Science). Note a student may not take the general education assessment and the APA assessment in the same content area. 72012-2013 NJ APA Administrator Training

8 Determining Who Participates In The APA Decisions are made: By the IEP team Separately for each content area assessed by the general statewide assessment Using information about the student and the assessments Test Specifications/Sample Test forms Present Levels of Academic Achievement and Functional Performance/Goals and Objectives Student Work Samples 82012-2013 NJ APA Administrator Training

9 Participation Guidelines Information on APA participation guidelines is provided in your training packet. This checklist must be completed in order to determine which students are eligible for the APA. An administrator must sign the form and include it in the APA binder. It is usually the sending school administrator, but if both sending and receiving school agree on the answers to the questions, then a receiving school administrator may sign the form. A teacher may not sign the form. Access to the Directory of Test Specifications and Sample Test for the general assessment is necessary in order to complete the checklist correctly. 92012-2013 NJ APA Administrator Training

10 Participation Criteria Checklist 1. Is the nature of the students cognitive disability severe? The students general level of cognitive functioning is significantly below the grade level expectations for his or her same aged peers and the student requires substantial modifications to the grade level curriculum. The student requires more intensive direct instruction than his or her peers to acquire knowledge and skills in the grade level curriculum. The student requires more extensive systems of support in order to participate in the grade level curriculum. 102012-2013 NJ APA Administrator Training

11 Participation Criteria Checklist 2. Is the students cognitive disability so severe that the student is not receiving instruction in any of the knowledge and skills measured by the general statewide assessment? While all students are to be instructed in the CCCS for their grade level, students with more significant cognitive disabilities typically receive instruction in the knowledge and skills at a less complex level than that measured by the general statewide assessment. Upon reviewing the sample test forms, if the student is NOT receiving ANY instruction at the same skill level, then the answer to this question is YES. 112012-2013 NJ APA Administrator Training

12 Participation Criteria Checklist 3. Is the students cognitive disability so severe that the student cannot complete any of the types of questions on the general statewide assessment in the content area, even with accommodations and modifications? Upon reviewing the sample test forms and questions for each content area of the general assessment, along with the state approved testing accommodations and modifications, if it is determined that the student is NOT able to complete any types of questions even with accommodations and supports, then the answer to this question is YES. 122012-2013 NJ APA Administrator Training

13 Participation Criteria Checklist 4. Is the students Individualized Education Program (IEP) aligned with grade level CCCS through modified expectations? The student is receiving grade level instruction at a significantly modified and less complex level than their grade level peers. The student may need more extensive systems of support, as identified in the IEP, in order to access the curriculum. 132012-2013 NJ APA Administrator Training

14 The answer to all four questions must be YES in order to assess the APA for a content area. A signed copy of the checklist must be placed in the inside front pocket of the APA binder. Usually the Sending district administrator signs the form. However, if the Sending and Receiving district agree on the APA status, the Receiving district administrator may sign the form. 14 Participation Criteria Checklist (continued) 2012-2013 NJ APA Administrator Training

15 What Must Be Written in Each Students IEP? If the student is taking the general assessment, list the assessment and the accommodations/modifications (from approved list) that will be provided. If the student is not taking the general assessment in one or more content areas, include a rationale for this decision. If the student is not taking the general assessment for LAL, Math, and/or Science, list that he/she will be participating in the APA. IEP model forms and information can be found on the DOE website http://nj.gov/education/specialed/iep_form_ann.pdf 152012-2013 NJ APA Administrator Training

16 Assessment and CCCS Materials Additional copies of all APA materials, including the teacher training materials, may be obtained on the Pearson website: http://pem.ncspearson.com/nj/apahttp://pem.ncspearson.com/nj/apa Copies of the Core Curriculum Content Standards, test specifications, and sample tests for the general assessments can be obtained on the DOE website: www.state.nj.us/education Copies can be ordered for a fee by calling the DOE publications office at 609-984-0905. 162012-2013 NJ APA Administrator Training

17 What Grades and Subjects Are Assessed? Students in grades 3 – 8 and 11 must be assessed in language arts literacy and mathematics. Students in grades 4 and 8 must also be assessed in science. Students in high school must be assessed in science the year they receive instruction in biology (grade 9, 10, 11, or 12). High school students may have an APA submitted for science in a different year than the year the APA is submitted for language arts and math. 172012-2013 NJ APA Administrator Training

18 Grade 11 and 12 Students If a senior is new to the state this year and has not participated in either the APA or the HSPA, the IEP team must determine which assessment is appropriate and the student must participate in this assessment. Students who were juniors last year and should have participated in the APA, but did not, must participate in the APA this year whether they are still in 11 th grade or are now in 12 th grade. Students in high school do not need to retake the APA if the test result was Partially Proficient. Score report results on the high school APA do not affect the ability to graduate. 182012-2013 NJ APA Administrator Training

19 Retained Students If a student has been retained in a grade due to poor academic performance, the student must be reassessed in that grade. Students who remain in the same assigned grade twice due to reasons not related to academic performance must be assessed only once in that assigned grade, except for the circumstances described on the next slide. Example of reason not related to academic performance: student is kept in the middle school an extra year because it was determined that the student was not yet ready for the high school environment (socially, structure of new large building, etc.) 192012-2013 NJ APA Administrator Training

20 Retained Students If a student received a score report with a Void Code rather than a Proficiency Level in a content area, and the student is retained in that grade, the student must participate in the APA again in the same grade. Also, if a student in 11 th grade received a Void Code in language arts literacy and/or mathematics, the student must be assessed again in these content areas, either in 11 th grade if they remain in that grade, or as a 12 th grader. 202012-2013 NJ APA Administrator Training

21 Assigned Grade Level The grade level of the student must be marked on the Demographic Scan Sheet that is included in the portfolio. This is the assigned grade level used during scoring of the APA. If a record change for the students grade level is submitted in April 2013, the APA assigned grade level becomes the grade level provided during record changes. The assigned grade level will be matched against the grade level of the CPI Links selected for assessment during the scoring process. If the grade level of the CPI Links does not match the assigned grade level provided during data collection, the entry will receive a zero score. Please note that students in high school (grade 9, 10, 11, or 12) are tested on the grade 12 CPI Links. 212012-2013 NJ APA Administrator Training

22 LEP LAL Exemption If a student has entered the United States after July 1, 2012, AND is currently enrolled in a language assistance program, this student is EXEMPT from the LAL assessment. This exemption is not allowable for high school students. The APA scan sheet delivered to districts in February 2013 must indicate the LEP LAL exemption. 222012-2013 NJ APA Administrator Training

23 Student Transferred to NJ District from Out of State after October 31, 2012 Students who require an APA, and enter your school after October 31, 2012, transferring from out of state, have missed the cut-off date for the development of the APA assessment. Submit a binder, a completed scan sheet, and a letter from an administrator that indicates from which state the student transferred, the date of the transfer, and noting this as the justification for missing APA evidence. The score report will indicate a Void 4, no portfolio evidence, when the student is a late registrant. Please note that instruction related to the CCCS must still occur throughout the year, regardless of the enrollment date. 232012-2013 NJ APA Administrator Training

24 Student Transferred from One NJ District to Another NJ District The district that has the student during any part of a collection period must compile the portfolio. Once the student transfers within NJ, the district must send the portfolio to the new district. The new district must complete the portfolio and submit it for scoring. If a student is enrolled in a school or district after July 1, 2012, the administrator may bubble on the scan sheet Time In School less than one year. This will remove the students results from the Accountability report. Note this is different than last year and is based on the accountability waiver calculation. 242012-2013 NJ APA Administrator Training

25 Student Transferred from One NJ District to Another NJ District If the student moved to another district you must send the portfolio to the district where the student is newly enrolled. As long as a student is in your district, even for a limited time, you must begin instruction of the CPI Links, and begin collecting APA evidence. Be sure to transfer the APA materials to the new district in a timely manner. You, the former district, do not need to submit a binder or letter to the APA program since the new district will send in the portfolio.. 252012-2013 NJ APA Administrator Training

26 Students Leaving the State Please note that if a student transfers out of New Jersey prior to the end of the second collection period (February 15, 2013), the portfolio does not need to be submitted for scoring. It may be sent to the out-of-state school for use when reviewing the students educational program, or given to the parent 262012-2013 NJ APA Administrator Training

27 Medical Emergency Medical emergency is the occurrence of a severe medical or psychiatric condition or episode which requires medical attention or supervision during which time the student is not able to participate in state assessments. APA students who are not receiving instruction due to a medical emergency may be eligible for a Void Code due to Medical Emergency when there is not enough APA evidence. APA uses the term extensive sick leave/hospitalization interchangeably with the term medical emergency. Note that no other definition may be used when determining a void due to medical emergency. See the requirements for this void on the next slide. 272012-2013 NJ APA Administrator Training

28 When a student is out of school for an extended amount of time and not receiving instruction due to extensive sick leave or hospitalization, the portfolio may be eligible to receive a Void 1 (medical emergency code). Eligibility is based only on the information below. If the student is receiving instruction for 10 days or less during a collection period, and The student has an extended hospitalization or leave due to illness and is not receiving instruction, and An official record documenting the student absences is available, then you must: include a letter on school letterhead from an administrator documenting the medical emergency. include a formal district/school record of the students attendance and absences. If all of these criteria are met then the portfolio will be voided due to extended illness during one or both collection period(s). 28 Medical Emergency 2012-2013 NJ APA Administrator Training

29 Record of Absences The record of absences must be a formal attendance record produced from the districts record system. It must indicate all days school is open and which days school is closed for holidays, etc. This record must be marked with the students attendance – present or absent. You may not submit an informal record of attendance such as a memo indicating the student was absent during the year. 292012-2013 NJ APA Administrator Training

30 Medical Emergency Paperwork If the paperwork submitted in the APA binder does not include all necessary components as defined on the previous slides, a Void for Medical Emergency/Illness will not be granted. An administrator may contact the Office of Assessments with any questions on this topic. 302012-2013 NJ APA Administrator Training

31 Absenteeism – Non Medical Emergency If a student is receiving instruction for 10 days or less in a collection period (excessive absenteeism), and the reason for absence is unrelated to a medical emergency/extensive illness, then the school administrator should contact the Office of Assessments. If it is determined that a Void for No Evidence is appropriate, two things must be submitted by an administrator in the APA binder: a letter on letterhead indicating the reason for the excessive absenteeism (parent took student out of country, etc.) a formal record of the students attendance/absences produced from the districts record system. 312012-2013 NJ APA Administrator Training

32 Test Coordinators Main Responsibilities Ensure APA participation guidelines are used correctly by IEP teams. Ensure IEP teams have all of the necessary information about the general assessments and the APA participation guidelines clarifying questions to ensure the decision is made based on the requirements. Ensure all APA teachers complete the teacher training modules and review all training materials including the Procedures Manual and Content Guide Five Item Samples document. Have the teachers sign in to your district training and keep a record of attendance. Ensure all teachers view all modules. Ensure all teachers utilize the critical additional training materials listed above as the materials as a whole deliver the information necessary to successfully implement the APA. 322012-2013 NJ APA Administrator Training

33 Test Coordinators Main Responsibilities Periodically review teachers APA process, use of test specifications and selected CPI Links, instructional lessons, etc. This review must occur at the beginning of each collection period, and at least once prior to the end of each collection period. Reviewing the materials before the cut off date allows time if necessary for production of additional/different APA evidence. Ensure that only the 2012-2013 CPI Link documents are used. The APA continues to use the older standards until the other assessments finalize the alignment to the Common Core Standards and the APA specifications are revised. 332012-2013 NJ APA Administrator Training

34 Ensure registration of APA students, review and update district contact info, and submission of Student Pre-Identification file prior to the deadline. Complete Demographic Scan sheet providing accurate data Complete final portfolio review prior to shipping Ensure timely shipment of portfolios Review APA record change rosters and correct/revise any inaccurate data 34 Test Coordinators Main Responsibilities 2012-2013 NJ APA Administrator Training

35 Vendor Transition The APA program is transitioning to a new vendor, Questar Assessment. This transition will begin in September. Details and deadlines related to student registration, Pre-ID submission, the new Online Entry Cover Sheet, etc., will be forthcoming. We appreciate your patience and will send details on these topics regularly as they become available. 352012-2013 NJ APA Administrator Training

36 Materials Survey – Registration of APA Students and District Information Register the number of APA students in your district no later than (date to be determined). Failure to register your students will result in a failure of delivery of APA materials to your district. Survey should be completed online (see memo). Do not register out-of-district students that are assessed in New Jersey. The receiving district will register these children. Register students that are sent by your district to an out-of- state placement as if they are in their home school. If no APA students this year, then indicate 0 (zero) for the applicable grade level. Confirm/revise district address, emails, and contact information to ensure accurate deliver of information. 362012-2013 NJ APA Administrator Training

37 Online Registration – Material Survey The online material survey is not yet available. A memo will be sent to districts when the website is ready. https://nj-servicepoint.questarai.com/Login.aspx is the Questar website address for use in the future. Log on information including User IDs and Passwords will be provided to your Chief School Administrator (CSA) If your CSA does not receive this information by October 30, call Questar customer service at 1- 866-690-4733 or send an email to NJAPACustomerSupport@questarai.com NJAPACustomerSupport@questarai.com 372012-2013 NJ APA Administrator Training

38 Students Sent to an Out-of-State School The Sending District is responsible for communication to the out-of-state placement. This includes providing training materials to the school and ensuring that the CCCS is instructed and assessed. This includes ensuring that the APA is in process, evidence is being collected, and that the APA will be completed and submitted to you prior to the return portfolio shipment deadline. Portfolios must be sent from the out-of-state school to the sending district prior to February 15, 2013. The sending district includes these portfolios in the districts shipment. Registration of these students, completion of scan sheets, record changes, etc., is the responsibility of the sending district. Include these students in the sending school APA enrollment counts. 382012-2013 NJ APA Administrator Training

39 Electronic Pre-ID File Submission Pre-ID File This is the first opportunity to submit accurate student information/data. Administrators will need to provide such info as student birth date, SE classification code, grade level, etc. Student demographic information must be submitted electronically on the Questar website. Please be patient while the website is developed. Pre-ID Instructions will be sent to your Chief School Administrator. Pre-ID information will be printed on the demographic scan sheet in lieu of gridding the information. 392012-2013 NJ APA Administrator Training

40 Electronic Pre-ID File Last date to submit pre-ID files is: Date to be determined. It is not advisable to wait until the last day since you may find that the file is not accepted due to data errors. Within the submission window you may submit a revised file when necessary. Pre-ID information will be printed on scan sheets for those participating districts and sent with the Return Materials shipment in February 2013. Blank scan sheets will also be provided. Errors/omissions on the Pre-ID label must be corrected by gridding the info on the scan sheet. 402012-2013 NJ APA Administrator Training

41 Scan Sheet Student demographic scan sheets, along with directions for the completion, are shipped to districts in February. This is the second opportunity for districts to submit accurate student information/data. A county-district-school code (CDS) must be provided for the school the student is attending. This is either a Home (neighborhood) school or a Receiving school. If the student is in a receiving school, a CDS code for the sending school must also be provided. The sending school must provide this code to the receiving school for inclusion on the scan sheet. The home or receiving school code is pre-gridded on the scan sheet. APA will begin to use the most up-to-date CDS codes for the private and specialized, separate schools beginning this year. Please ensure you know your current code which is also being used for NJASK and HSPA. We will no longer be using CDS codes that start with 90. 412012-2013 NJ APA Administrator Training

42 Scan Sheet If a Pre-ID file was submitted, check the preprinted information on the scan sheet for accuracy. You may correct information submitted in the Pre-ID file by gridding the correct information on the scan sheet. The gridded information will override the preprinted information. The only exception to this is changes to the student name, which may only be corrected during the record change process. We must maintain the same exact student name on the file in order to process the information until record changes. Only at that point can we make changes to the spelling of the students name. 422012-2013 NJ APA Administrator Training

43 Taking General Assessment Field Detailed instructions are provided in the use of this field in the Pre-ID directions and the Scan Sheet directions. Only code this field when the student is NOT taking the APA, but is taking the general assessment (NJASK, HSPA, NJBCT) in a content area. Remember, a student may only take the APA OR the general assessment, not both, for a content area. Marking the student as Taking the General Assessment will Void all results for the APA. 432012-2013 NJ APA Administrator Training

44 Record Change Process A roster of APA student demographic information is shipped to all districts in April 2013. (Will verify date with new vendor and confirm in Record Change Memo.) For students who remain in their Home (neighborhood) school, the Home district must review and submit roster changes. For students who have both a receiving and sending school, both districts must review the record change rosters for accuracy. Beginning this administration, the Sending School, not the Receiving School will submit the record changes to the vendor. 442012-2013 NJ APA Administrator Training

45 Record Change Process The Sending School/Home School is responsible for correcting all student information that was provided on the scan sheet. The Receiving School must also review the roster, as errors (e.g., CDS code, demographic information) may impact the student score reports and accountability reports. Corrections and revisions to the data must be submitted to the students Sending School at least several days prior to the record change deadline. The Sending School must submit these changes to Questar during the record change timeframe. This is your final opportunity to correct/revise the data. This includes the field Taking the General Assessment. The final record change roster data will be used for the accountability report. No changes will be made after this time. 452012-2013 NJ APA Administrator Training

46 APA What will this portfolio look like? What are the required components? 462012-2013 NJ APA Administrator Training

47 What is the Format of the APA? Portfolio: Paper based, submit in a 3-ring binder Binders are mailed to districts in October, based on the APA registration counts the district administrator submitted. Content Areas: Mathematics, Language Arts Literacy, and Science Science is included only in grade 4, 8, and high school. 472012-2013 NJ APA Administrator Training

48 What is the Format of the APA? Entries relate to content standards, grade-level CPIs, and CPI Links. Each content area will have four entries, each one reflecting the assessment of one CPI Link. Evidence documents educational instruction and student performance of skills. The instructional activity used at the beginning and at the end of the CPI Link instruction is submitted as the assessment evidence. 482012-2013 NJ APA Administrator Training

49 Test Design NOTE: NO entry should contain more than four pieces of evidence. This graphic represents a student who is being assessed in all three subjects with the APA. Not all portfolios will include all three subjects. 49 APA Portfolio Table of Contents Language Arts 4 Entries Mathematics 4 Entries Science 4 Entries 2012-2013 NJ APA Administrator Training

50 Test Design Example – Language Arts (Same Structure for Each Content Area) NOTE: NO entry should contain more than four pieces of evidence. An entry containing more than four pieces of evidence will result in zero scores for all dimensions. 50 APA Portfolio Table of Contents Language Arts Entry 1 Entry Cover Sheet reflecting 1 st Standard, Strand, CPI, and CPI Link 2 pieces of evidence Language Arts Entry 2 Entry Cover Sheet reflecting 2 nd Standard, Strand, CPI, and CPI Link 2 pieces of evidence Language Arts Entry 3 Entry Cover Sheet reflecting 3 rd Standard, Strand, CPI, and CPI Link 2 pieces of evidence Language Arts Entry 4 Entry Cover Sheet reflecting 4 th Standard, Strand, CPI, and CPI Link 2 pieces of evidence 2012-2013 NJ APA Administrator Training

51 When Do I Collect Evidence? ActivityCollection Period Initial ActivitySeptember 4 - November 16, 2012 Final ActivityDecember 10, 2012 - February 15, 2013 Dates on the evidence must include month, day, and year. Be sure to plan early in the year when you will instruct and assess each CPI Link and content area. All students will need to be assessed on an initial activity within the correct time frame above, even if the majority of instruction will not occur until the winter. 512012-2013 NJ APA Administrator Training

52 Four Entries per Content Area Based on Standard and Strand 1 Based on Standard and Strand 2 Based on Standard and Strand 3 Based on Standard and Strand 4 Select CPI & CPI Link Select CPI & CPI Link Select CPI & CPI Link Select CPI & CPI Link Provide instruction & document activities and results One Piece of Evidence from September 4 - November 16, 2012 One Piece of Evidence from December 10, 2012- February 15, 2013 Provide instruction & document activities and results Provide instruction & document activities and results Provide instruction & document activities and results One Piece of Evidence from September 4 - November 16, 2012 One Piece of Evidence from December 10, 2012- February 15, 2013 One Piece of Evidence from September 4 - November 16, 2012 One Piece of Evidence from December 10, 2012- February 15, 2013 One Piece of Evidence from September 4 - November 16, 2012 One Piece of Evidence from December 10, 2012- February 15, 2013 Entry One Entry TwoEntry ThreeEntry Four 522012-2013 NJ APA Administrator Training

53 Page Numbering A table of contents that references page numbers in the portfolio must be included. Page numbering for evidence that is longer than one page should be numbered using a combination of page number and letters (i.e., 2, 2a, 2b, etc.) For example, a student was given a reading comprehension test (matches the CPI Link) with 8 test items. The test was two pages long, but it is one activity/piece of evidence. The pages might be numbered page 3 and 3a. Entry Cover sheets get their own page numbers. 532012-2013 NJ APA Administrator Training

54 Entry Cover Sheet The vendor will build an electronic entry cover sheet that will allow teachers to use drop-down boxes to populate most of the fields on the form. This form will then be printed from the web. Teachers must still type/write in their activity descriptions on the form. This form will not be ready until later in the fall. 542012-2013 NJ APA Administrator Training

55 Entry Cover Sheet Teachers should use/complete the old version of the cover sheet as a place holder until the new form is available. The Procedures Manual and Pearsons website has the old style Entry Cover Sheet for 2012-2013. Once the new online form is available teachers should use the old form to assist in transferring the information to produce the new Entry Cover Sheet. 552012-2013 NJ APA Administrator Training

56 562012-2013 NJ APA Administrator Training

57 Contents of the Entry Each entry requires documentation (evidence) of student performance of the skill stated in the CPI Link. Two pieces of evidence is the requirement for an entry. Includes two different activities. One piece of evidence from the initial activity, and one piece of evidence from a different, final activity. Evidence may come from classroom work, community-based settings, or other places where student is working on grade-level CPI Links. 572012-2013 NJ APA Administrator Training

58 Contents of the Entry (continued) The first piece of evidence documents the initial, or very early, performance of the CPI Link. This evidence must capture what the student does/does not know related to the CPI Link. This evidence must reflect the entire CPI Link in one activity, and be collected between September 4 and November 16, 2012. In order for the CPI Link to be eligible for the students APA entry, the accuracy score on the activity can not be above 39%. Additional information on scoring the evidence is included later in the training. 582012-2013 NJ APA Administrator Training

59 The second piece of evidence documents final performance of the CPI Link. This evidence should capture what the student has learned related to the CPI Link. This evidence should assess the same CPI Link using a different instructional activity, and be collected between December 10, 2012, and February 15, 2013. More information on what makes a different activity is found in the Teacher Training Modules. Contents of the Entry (continued) 592012-2013 NJ APA Administrator Training

60 Evidence Requirements Evidence must document the entire CPI Link noted on the Entry Cover Sheet. Evidence should relate only to the CPI Link, not assess other skills as well. Providing evidence that documents multiple skills (skills not part of the selected CPI Link), or less than the entire CPI Link, will result in a low scoring entry. Have at least 5 items/questions/task elements included in each activity 602012-2013 NJ APA Administrator Training

61 How are CPI Links Related to the CCCS? Grade-level CPIs are the building block. Content Centrality establishes what concept must be assessed. Intent/Essence of the CPI Performance Centrality directs the level of skill statement. Complexity/difficulty APA CPI Link can be less complex than the grade-level CPI and/or less difficult Three types of links–Matched, Near, and Far–reflect the degree of complexity/difficulty when compared to the grade-level CPI. 612012-2013 NJ APA Administrator Training

62 CPI 3.1.5G13 Recognize figurative language in text (e.g., simile, metaphor, personification, alliteration) Essence of the CPI: Identify figurative language Matched LinkNear LinkFar Link Find examples of three different types of figurative language in a text* Match at least two different types of figurative language to its type* Identify examples of similes Identify examples of personification Identify examples of alliteration Identify examples of metaphors 62 CPI Link Grade Level CPI Grade Level CPI Essence Essence Near Links Near Links Far Links Far Links Matched Links Matched Links Links Links 2012-2013 NJ APA Administrator Training

63 Using the CPI Links for Instruction 63 CPI Links should not be used in isolation. Read the Standard and Strand for the overall concept. Read the CPI and Essence. Make sure instruction is within the Essence of the Standard, Strand, and CPI. Evidence of instruction on CPI Link will be scored not only on the discrete Link but also within the Essence of the standard, strand, and CPI For example, 3.1.3G3 is a reading standard with a Matched Link that states Interpret information from graphs, charts, and/or diagrams found in text. The evidence must assess the students interpretation of graphs/charts/diagrams within the context of reading text and not within the mathematical context of data analysis. 2012-2013 NJ APA Administrator Training

64 Assess ONLY one CPI Link Remember, when reviewing the CPI Links in the APA Procedures Manual, a CPI Link is only one bullet. Once the type of link to assess a student is determined (Matched, Near, or Far Link), then select only one bullet in that column. Assess the entire CPI Link providing the necessary systems of support Include the students response to the test item Must be completed in the students mode of communication Ensures that the student can review his own work This is a good way to test if the evidence documentation includes the necessary components, but it is not a requirement that you document the students actual review of the work. 64 Evidence Must 2012-2013 NJ APA Administrator Training

65 Acceptable Format for the Evidence Evidence must show the test items and student responses. Acceptable evidence includes –Samples of graded student work –Series of captioned photographs Must evidence the student actually performing the skill –Snapshot of completed student work that documents student performance of the skills contained within the CPI Link Must be a photo of a student work product that is too large or bulky to include in the portfolio (i.e., a poster, a written piece using objects or textured cues, etc.) 65 2012-2013 NJ APA Administrator Training

66 66 Acceptable Evidence What else is needed to make evidence acceptable? All evidence must include: Students Name Date of Activity Completion using Month, Day, and Year (must be within the collection windows to be valid) The accuracy of each student response Use + (plus) for correct and – (minus) for incorrect responses The independence level of each response I=Independent performance, V=Verbal prompt, G=Gestural prompt, M=Model prompt, P=Physical prompt (presented from least to most intrusive) More information on use of prompting appears later in the presentation 2012-2013 NJ APA Administrator Training

67 What else is needed to make evidence acceptable? All evidence must include: A minimum of 5 items that assess the CPI link A piece of evidence must include at least 5 test items that assess the skills contained within the CPI Link. For example Ordering decimals would require 5 sets of numbers with which to ordernot just that 5 decimals were ordered. More information is found in Modules IV, V, and VI of the online training, in the 2012-2013 Procedures Manual, and in the CPI Content Guide and Five Items Resource. 67 Acceptable Evidence 2012-2013 NJ APA Administrator Training

68 For CPI Links, an activity must include at least 5 test items. For example A student will answer 5 comprehension questions about a completed job application. A student will use multiplication to check the answers to 10 division problems. NOTE via an example: If a student is given 7 pictures, and is asked to circle the plants, they must also be asked to cross out the pictures that are not plants. They may not have items that do not require a response and then score it as if the item was correct. Also, there must be enough plant pictures to test the skill, and a balance of right to wrong options. A Minimum of 5 Test Items 682012-2013 NJ APA Administrator Training

69 Rule 5 : Example Five Items When the Link has an and in it, 5 items across the skills are required rather than 5 items for each part of the Link. For instance, 4.2.7E1 Near Link Calculate the area of a triangle, rectangle and square 692012-2013 NJ APA Administrator Training

70 Acceptable Evidence Student work samples serve as the best evidence to document all of the necessary components of the assessment activity. This type of evidence includes the test items/questions, response options (if applicable), and student responses. Teachers may need to scribe the students responses when necessary. The use of videotape, audiotape, data charts, and most photographs are not acceptable evidence. 702012-2013 NJ APA Administrator Training

71 Student Work Samples Used as Evidence May Be Computer screen printouts of students test item and response after hitting a switch. Summary printouts where each response is not evidenced is unacceptable. An essay written using picture symbols and scored using a rubric. A worksheet completed by the student. Drawings to extend patterns. Completed job application using name, address, and other stamps. A graph made from ordered pairs. 71 2012-2013 NJ APA Administrator Training

72 Student Work Samples May Be A paragraph written using sentence strips A printout of a sentence written using a picture writer (i.e., Writing with Symbols, PicWriter, etc.) A snapshot of work too large to fit into the portfolio (a poster, a model, etc.) 722012-2013 NJ APA Administrator Training

73 Acceptable Photographic Evidence A series of captioned photographs that document the student performing the skills included in the CPI Link will be accepted only if: Each item and the students response is clearly present in each photograph Must include a description for each photo Must include running record of the items performed Must include accuracy and independence information for each item and include total percent scores for each dimension Must meet the requirements for the Universal Scoring Rules 732012-2013 NJ APA Administrator Training

74 Acceptable Photographic Evidence (continued) A photograph of a large or bulky completed project will be accepted only if the picture captures the evidence requirements for the activity. A description of the activity must be included along with the documentation of the scoring, but the actual work must be clearly shown in the picture. The Universal Scoring Rules requirements must be met. 742012-2013 NJ APA Administrator Training

75 Scribing Work Samples Acceptable Scribing TechniqueUnacceptable Scribing Technique Uses the students communication mode to document the students answer Does not use the students communication mode Allows the student to review her/his work Does not allow the student to review her/his work Is used to record the students response on a work sample from eye gaze, a voice-output device (VOD), Augmentative Communication Device, reach and grasp, etc. Does not show the student performance of the skill (observational data) 752012-2013 NJ APA Administrator Training

76 Unacceptable Evidence Data charts Single photographs of student working on an activity Photographs of student work that could have been included Any work that is not completed by the student Note, the student responses must be marked at the time of the activity, not at a later point in time, in order to be acceptable evidence. 762012-2013 NJ APA Administrator Training

77 Has fewer than 5 items/questions/task elements Has no name, date, or incomplete date Is scored using anything other than percentages e.g., ratios (4/5), percent ranges (90-100%), letter grades (A, B+) Has percent scores that can not be replicated by scoring staff 77 Unacceptable Evidence 2012-2013 NJ APA Administrator Training

78 Universal Scoring Rules must be met on two pieces of evidence for the entry to be scored based on the rubric. Otherwise, the entry will receive a score of zero for all dimensions. If any entry does not have at least two pieces of evidence OR has more than four pieces of evidence per entry, it will receive a score of zero. Two pieces of evidence are all that is required for APA. 78 Evidence 2012-2013 NJ APA Administrator Training

79 79 Universal Scoring Rules 1.The evidence must include the students name. 2.The evidence must include the complete date (month/day/year). Verify that the dates fall within the appropriate collection period: September 4, 2012 - November 16, 2012 for the first piece of evidence December 10, 2012 - February 15, 2013 for the second piece of evidence 3.The evidence must be presented in the appropriate format. The evidence must include a writing rubric when specified in a Writing CPI Link. 4.The evidence must reflect the students mode of communication. 5.The evidence must include at least 5 items. 6.The evidence must show student responses for at least 5 items.. 7.The evidence must assess the entire link while connecting to the essence of the CPI, standard, and strand. 8.Both pieces of evidence must assess the same CPI Link and skills. 9. The evidence must not include more than the skills contained within the CPI Link. Note: The USRs pertain to both student work samples, as well as the writing rubric when required. If the entry does not adhere to all rules it will receive zero scores. 2012-2013 NJ APA Administrator Training

80 Activities – Errors to Avoid The first piece of evidence should not include a more difficult application/context of the skill than the final activity/evidence. Examples of this type of error (a more difficult initial activity) are below: The initial activity has longer test items with more complex steps (e.g., open-ended essay questions vs. multiple-choice) than the final activity. The initial activity has multiple-choice test items where no correct answers are offered as options. Fewer additional supports (steps, formulas, pictures, application, calculators, etc.) are given in the initial activity than the final activity. Regular grade-level text is used in the initial activity, but summarized text with picture cues and key details highlighted are used in the final activity. No explanation of what to do is given for the initial activity, but directions are given for the final activity. Note: directions should ALWAYS be given during an activity. No prompts on the first piece of evidence with no answers written, or all answers are clearly off topic (answers are not at all related to concept). The initial activity is not in the students mode of communication, but the final activity is. 802012-2013 NJ APA Administrator Training

81 Common Mistakes found in 2011 - 2012 Assessments Assessing the skills contained in the CPI Link without connection to the critical essence, intent or big idea of the content and concepts of CPI, Strand, and/or Standard Assessing the skills contained within the CPI Link without correct understanding of the Content or concepts of CPI, Strand, and/or Standard Assessing more than one CPI Link in an entry Assessing different parts of a CPI Link in each piece of evidence Assessing only part of the CPI Link 812012-2013 NJ APA Administrator Training

82 Common Mistakes found in 2011 - 2012 Assessments Failure to mark all test items with accuracy and/or independence/prompt information Failure to give supports on the first activity Failure to give prompts, instead marking student response as Dont Know or No Response For assessment, an item may not be marked as dont know or no response. Teachers must utilize a prompt(s) in order to assist the student in generating a response. 822012-2013 NJ APA Administrator Training

83 Changes for 2012-2013 Districts must use the 2012-2013 updated CPI Links when assessing a standard, strand, and CPI for the APA. A few CPI Links were revised to make them more clear. The asterisk following a link is a reminder that there is a sample test item available in the Five Items document. Important CPI Link clarifications are included in the new Content Resource Guide and Five Items document. Be sure to read all of the relevant grade-level materials thoroughly. 832012-2013 NJ APA Administrator Training

84 Content Guide Revision Some curriculum issues have been identified and must be avoided in the future. Some evidence contained errors in the instruction of the skills. Example: Our Solar System only contains one star, the Sun. Therefore, stars may not be used as a correct answer when asking what is in our Solar System. Example: Teachers are submitting math Symmetry work sheets that dont correctly assess the CPI Link that was selected. These topics were addressed in last years document. These errors will result in zero scores for 2012- 2013. 842012-2013 NJ APA Administrator Training

85 Developing Standards-Based Entries 852012-2013 NJ APA Administrator Training

86 86 Step 1: Select a CPI and one related CPI Link to be assessed. Step 2: Plan instruction and assessment relating to the CPI and CPI Link. Step 3: Design activities that will be used to assess the CPI Link. Step 4: Assess the student to get an initial piece of evidence for APA purposes. Step 5: Implement instruction. Ensure instruction reflects the essence of the CPI, strand, and standard. Ensure the instruction is age- and grade-level appropriate. Retain a working folder of instructional activities and classroom-based assessments implemented between the activities that generate the initial and final pieces of APA assessment evidence. Step 6: Determine when evidence can be collected to document the final instructional assessment of the CPI Link for APA purposes. Step 7: Based on the students accuracy score and level of prompt information on the final activity, determine if additional instruction and collection of evidence needs to occur for the entry. Step 8: Review APA evidence to ensure that all information related to test design requirements are included. Developing an Entry 2012-2013 NJ APA Administrator Training

87 How is the APA Scored? Each portfolio entry is scored based on specific scoring criteria described in the APA scoring rubric found in the Procedures Manual. The criteria are grouped into three categories called dimensions. 872012-2013 NJ APA Administrator Training

88 APA Scoring Rubric Scoring Dimensions Complexity Link Match, near, or far link selection Performance Percentage of accuracy when performing skill Independence Percentage of time skill was performed independently 882012-2013 NJ APA Administrator Training

89 Scoring Dimensions 89 Dimension01234 Complexity Evidence provided is unscorable; all dimensions will receive a score of zero CPI Link was assessed, but there are major flaws in the evidence CPI Link is a Far Link to the grade-level indicator CPI Link is a Near Link to the grade-level indicator CPI Link is a Matched Link to the grade- level indicator PerformanceEvidence is not clear or all items are not marked as correct/incorrect Accuracy of work is 0-39% based on the final activity OR Second activity includes more intrusive prompt Accuracy of work is 40-59% based on the final activity Accuracy of work is 60-80% based on the final activity Accuracy of work is 81-100% based on the final activity IndependenceEvidence is not clear or all items are not marked for Independence/ prompt level Student completed items/tasks independently 0-39% of the time based on the final activity Student completed items/tasks independently 40-59% of the time based on the final activity Student completed items/tasks independently 60-80% of the time based on the final activity Student completed items/tasks independently 81-100% of the time based on the final activity 2012-2013 NJ APA Administrator Training

90 Scoring Rubric and APA Forms The APA Scoring Rubric found on the previous slide is also found in the 2012- 2013 APA Procedures Manual. Dont forget to review the Procedures Manual, which includes the APA Scoring Rubric, scoring clarifications, and other topics. 90 2012-2013 NJ APA Administrator Training

91 Complexity Dimension Evaluates the CPI Link assessed and how closely the complexity and difficulty (Matched, Near, Far) links to the Core Curriculum Content Standards (CCCS) and grade-level cumulative progress indicators (CPI). 912012-2013 NJ APA Administrator Training

92 92 01234 Evidence provided is unscorable; all dimensions will receive a score of zero CPI Link was assessed, but there are major flaws in the evidence CPI Link is a Far Link to the grade-level indicator CPI Link is a Near Link to the grade-level indicator CPI Link is a Matched Link to the grade- level indicator Complexity Dimension 2012-2013 NJ APA Administrator Training

93 Complexity Score of 2, 3, or 4 A score of 2, 3, or 4 is obtained if the CPI Link is assessed correctly and a different activity is used for the initial and final piece of evidence. A score of 2 is obtained when a Far Link is assessed correctly. A score of 3 is obtained when a Near Link is assessed correctly. A score of 4 is obtained when a Matched Link is assessed correctly. 93 2012-2013 NJ APA Administrator Training

94 Two activities are required when submitting evidence for an entry. Both of these activities must assess the same CPI Link. The two activities must be different activities in order to address the Universal Scoring Rules. An activity is considered to be different than another when the context of the instruction of the CPI Link is different. An activity is also considered to be different from another when the application of the skill in the CPI Link is different. 94 What Makes an Activity Different? 2012-2013 NJ APA Administrator Training

95 While the skills assessed and the supports needed to perform the skills remain the same, an activity is considered different: With a different context in which the framework or events within which the student is demonstrating the skill changes With a different application in which the types of questions used, or the way in which the student demonstrates the skill changes 95 What Makes an Activity Different? 2012-2013 NJ APA Administrator Training

96 Examples of two activities with different context: The student is working on following vertex-edge graphs 1.after whole class introduction and demonstration of vertex-edge graphs, the student completes a worksheet 2.after having a lesson on flight plans, the student uses a vertex edge graph to follow flight paths The student is working on adding two matrices 1.by completing a worksheet 2.by completing a different worksheet with different problems than used in the initial activity The student is working on writing a cover letter 1.after researching a job of a particular interest, write a cover letter that would be used when applying for the job 2.after listening to speakers about different careers, write a cover letter for a job of interest The student is working on identifying theme in a given work of fiction 1.by reading The Outsiders and answering multiple-choice questions 2.by reading Flowers for Algernon and answering multiple-choice questions 96 What Makes an Activity Different? 2012-2013 NJ APA Administrator Training

97 Examples of two activities with different applications: The student is working on classifying types of planets 1.by using a graphic organizer 2.by matching The student is working on addition and subtraction 1.by completing a worksheet with problems on it 2.by using a Smart Board to complete math problems The student is working on writing an essay 1.by using a computer or word processor 2.by using paper and pencil The student is working on identifying theme in a given work of fiction 1.by reading a passage of The Outsiders and then answering multiple- choice questions 2.by reading a passage of The Outsiders and then answering short answer questions 97 What Makes an Activity Different? 2012-2013 NJ APA Administrator Training

98 Teachers Must Think about the Two Activities in Conjunction As you plan for instruction, you should not think about the initial and final activity in isolation. Things to think about: What type of activities might be easier and harder for the student? Do not use the easier activity as the final activity. What type of prompt, if any, might the student need on the final activity? Do not use a more intrusive prompt level on the final activity than was used on the initial activity. How can I keep the student engaged in the various activities? How will the initial activity be different than the final activity and still engage the student? 98 2012-2013 NJ APA Administrator Training

99 99 Complexity Score of 1: Major Flaws An entry that demonstrates work in a CPI Link but has a major flaw will score a 1 in Complexity. A major flaw includes 1. Assessing the same activity for both pieces of evidence 2. Assessing only part of the CPI Link –This may be found in the work sample, and both pieces of evidence must be assessing the same part of the link. –This may also be found in the writing rubric. 2012-2013 NJ APA Administrator Training

100 100 Complexity Major Flaw Example (Same Activity) 9/10/12 2/11/13 The directions given to the student were Match herbivores, carnivores, and omnivores to the type of food they eat. 2012-2013 NJ APA Administrator Training

101 101 Complexity Major Flaw Example (Activity assesses only part of a link) Complexity score would be 1 for only assessing part of the link. 3.1.8G1 Matched Link: After reading newspapers, magazines, or electronic texts, identify the type of propaganda used and explain its purpose 10/1/12 1/10/13 2012-2013 NJ APA Administrator Training

102 Complexity Score of Zero When the Universal Scoring Rules are not adhered to, the entry will receive a score of zero for Complexity, Performance, and Independence. Please review Teacher Training Module VII: Universal Scoring Rules for more information. 102 2012-2013 NJ APA Administrator Training

103 This dimension measures the students accuracy performing the skills represented in the CPI Links identified within the portfolio. Performance Dimension 1032012-2013 NJ APA Administrator Training

104 Performance Dimension 104 Each item must be marked as correct (+) or incorrect (–). The score for accuracy should be expressed as a percentage. Any item for which the student received a physical prompt should be marked as incorrect (–P). The second piece of evidence should not include a more intrusive prompt than was used on the first piece of evidence. The students accuracy score on the first piece of evidence should not exceed 39%. The first piece of evidence should not be more difficult than the second piece of evidence. When assessing a 3.2 Writing Link that requires a writing rubric, there should be feedback on the students work explaining how/why the work was scored in relation to the writing rubric. 2012-2013 NJ APA Administrator Training

105 Performance Dimension 01234 Evidence is not clear or all items are not marked as correct/incorrect Accuracy of work is 0-39% based on the last activity OR Second activity includes more intrusive prompt Accuracy of work is 40-59% based on the last activity Accuracy of work is 60-80% based on the last activity Accuracy of work is 81-100% based on the last activity 1052012-2013 NJ APA Administrator Training

106 Each item must be marked as correct (+) or incorrect (–). The score for accuracy should be expressed as a percentage. Scoring the activity for accuracy requires a consistent understanding of when to mark an answer right or wrong. Certainly, if the student performed the skill independently, the answer is either correct or incorrect. But what about when the student receives a prompt? How do you score the item correct or incorrect? 106 Scoring Student Evidence for Accuracy 2012-2013 NJ APA Administrator Training

107 Understanding the difference between the following provisions will help ensure that the scoring information on the evidence is accurate: providing task directions providing supports providing indirect prompts (verbal, model, and gestural) providing physical prompts providing the answer (directly prompting the student using a verbal, model, or gestural prompt to correctly answer the question) 107 Scoring Student Evidence for Accuracy 2012-2013 NJ APA Administrator Training

108 If the student performs the item independently, then Score as correct (+) any item the student performs correctly. Score as incorrect (–) any item the student performs incorrectly. Any system other than +/– must include a key. If you used a different scoring system, such as equals incorrect and equals correct, you must explain that clearly on the evidence. 108 Scoring Student Evidence for Accuracy 2012-2013 NJ APA Administrator Training

109 If the student received an indirect verbal, gestural, or model prompt when performing the skill, then Score as correct (+) any item the student answered correctly. Score as incorrect (–) any item the student answered incorrectly. Remember to mark next to each item the prompt level used: V, G, or M (e.g., +V means that an indirect verbal prompt was given and that the student answered the item correctly.) 109 Scoring Student Evidence for Accuracy 2012-2013 NJ APA Administrator Training

110 Any item for which the student received a physical prompt should be marked as incorrect (–P). A physical prompt is a direct prompt whereby the teacher physically guides the student to the correct answer (e.g., hand over hand). However, if the student receives a physical prompt when performing the skill, then Score the item as incorrect (–P). 110 Scoring Student Evidence for Accuracy 2012-2013 NJ APA Administrator Training

111 Total accuracy score must be in the form of a percent. The number of items correct divided by the total number of items multiplied by 100 equals the total accuracy score. Accuracy # items correct divided by # of total items times 100 equals % Correct ____/_____ x 100 = ______% 111 Scoring Student Evidence for Accuracy 2012-2013 NJ APA Administrator Training

112 The second piece of evidence should not include a more intrusive prompt than was used on the first piece of evidence. Prompting can affect the reliability of the performance score if not implemented correctly. Therefore, it is important to adhere to the prompt level hierarchy described in your 2012-2013 Procedures Manual. If the second piece of evidence includes a more intrusive prompt level than the first piece of evidence, a score of 1 will be assigned for the Performance dimension. Performance 112 2012-2013 NJ APA Administrator Training

113 Scoring Performance - Non-Example 10-11-12 2-1-13 1132012-2013 NJ APA Administrator Training

114 The initial piece of evidence should not be more difficult than the final piece of evidence. If a support (e.g., a calculator or formula) is provided in the final piece of evidence, it must also be provided in the initial piece of evidence. Otherwise, the initial piece of evidence would be more difficult. Supports provided in the initial piece of evidence do not have to be present in the final piece; they can be faded if no longer needed by the student. The final piece of evidence can be of the same difficulty level as the initial piece of evidence, or it can be more difficult. The student should be given enough task directions, supports, and prompting (if needed) on the initial piece of evidence to elicit on-topic responses, even if incorrect. Performance 1142012-2013 NJ APA Administrator Training

115 Busy, Overly Complicated 10-25-12 Sandy Problems: the coordinate grid isnt labeled, the answer choices indicate the grid was intended to be marked by 2s, the colored lines make the graph jumbled, and use of colored pencils makes the task overly complicated. 4.3.12B2: Near Link:Given a graph of a linear function, identify the x- and y- intercept 1152012-2013 NJ APA Administrator Training

116 116 Make It Clear 4.3.12B2: Near Link: Given a graph of a linear function, identify the x- and y- intercept This activity clearly and simply assesses the link. Sandy Date: 1-17-13 100% Independent 2012-2013 NJ APA Administrator Training

117 When assessing a 3.2 Writing Link that requires a writing rubric, there should be feedback on the students work explaining how the work was scored in relation to the writing rubric. If the student did not earn all of the points for a particular dimension on the writing rubric, there should be feedback on the students work explaining why/how the work was insufficient or incorrect. Even when a student earns all of the points allotted on a writing rubric, there should still be feedback affirming the students work or explaining why it was particularly acceptable in accordance with the writing rubric.Performance 1172012-2013 NJ APA Administrator Training

118 Writing Feedback Non-example: Evidence does not include any feedback that corresponds to writing rubric. 118Performance Larissa 10/28/12 3.2.8B2 Near: Write a short story that includes a setting, a plot, and characters rubric Amanda T. 2012-2013 NJ APA Administrator Training

119 Independence evaluates the extent to which the student completed items/tasks independently. 119 Independence Dimension 2012-2013 NJ APA Administrator Training

120 Not all direction is considered a prompt. Some direction is simply a support, which does not need to be indicated, tracked, or part of the score. Examples of supports: Redirecting the student to stay on task is not a prompt; it is a support. Reading directions with the student is a support. Restating the directions for the student is a support. Scribing for the student is a support. Providing accessible materials (large print, textured paper, manipulatives, etc.) is a support. Supports are not included when calculating the independence score for the students work. 120 Supports vs. Prompts 2012-2013 NJ APA Administrator Training

121 Examples of Supports for Mathematics Use of manipulatives or pictures Use of a calculator Task-analyzed skills Number line Stamps to write numbers Word problems read to student Enlarged text Chunked problems 1212012-2013 NJ APA Administrator Training

122 Examples of Supports for Reading Student reads by Following along with the text Following along with picture symbols Following along with objects Following along with tactile cues Adapted text Shortened text Chunked information Large font Highlight to emphasize important information 1222012-2013 NJ APA Administrator Training

123 Examples of Supports for Writing Stamps Graphic organizers Write using pictures or objects Write by completing cloze sentences Scribing (work must use the students communication mode) Speech-to-text writing programs Large writing utensils or pencil grips Raised lined paper 1232012-2013 NJ APA Administrator Training

124 Examples of General Supports Reading initial task directions to the student Redirecting the student to stay on task you need to finish math now If a student requires supports, he/she must be given supports for both the first and second piece of evidence. 1242012-2013 NJ APA Administrator Training

125 125 Independence Dimension 01234 Evidence is not clear or all items are not marked for Independence/ prompt level Student completed items/tasks independently 0-39% of the time, based on the final activity Student completed items/tasks independently 40-59% of the time, based on the final activity Student completed items/tasks independently 60-80% of the time, based on the final activity Student completed items/tasks independently 81-100% of the time, based on the final activity 2012-2013 NJ APA Administrator Training

126 A prompt level must be documented next to each item performed by the student. I = independent V= verbal G= gestural M= model P= physical If some other system is used there must be a key. Remember, if your prompt hierarchy is not the same as the one above, you must explain your prompt hierarchy clearly, giving the names of the prompts, hierarchical order, and letter key for the prompts. Even if the student performs with 100% independence, each item must be marked with an I. Independence scores must be summarized as a percent. Reviewers must be able to recreate the score. If the score is not replicable, the dimension can not be scored and will receive zero for Independence. 126 Teacher Scoring for Independence 2012-2013 NJ APA Administrator Training

127 An indirect verbal prompt (V) can provide the student with a clue to try and spark the students recollection of the activity or lesson so that he/she can respond to the question (e.g., Who was the story about? or Remember, the main character has red hair and pigtails. Point to the main character.) An indirect gestural prompt (G) can Provide the student with a clue as to the general location of an answer (e.g., when looking up a word in the dictionary, the teacher may tap the page on which the word can be found but not exactly where the word is on the page) 127 Prompting During Assessment- Allowable Prompts 2012-2013 NJ APA Administrator Training

128 An indirect model prompt (M) can provide the student with a clue through demonstration of the skill (e.g., demonstrate how to carry in an addition problem then give the student a different problem) Provide the student with a clue through acting out a scenario (e.g., when presenting a choice of three pictures and asking the student which picture represents an unbalanced force, the teacher may make a sweeping or moving motion to represent an unbalanced force) 128 Prompting During Assessment - Allowable Prompts 2012-2013 NJ APA Administrator Training

129 A direct verbal prompt provides the student with the specific answer to a question or item (e.g., Remember, the main character was Pippi. Point to the picture of the main character.) A direct gestural prompt points out the specific answer to the student (e.g., when presenting a choice of three pictures and asking the student which picture represents an unbalanced force, the teacher points to or taps the correct picture) A direct model prompt models the exact problem and answer that the student must perform (e.g., when sorting producers and consumers, the teacher says, Remember, corn is a producer, picks up the picture of corn, and places it in the producer column of a chart then asks, Which one is a producer?) 129 Direct Prompting During Assessment - NOT ALLOWED 2012-2013 NJ APA Administrator Training

130 A physical prompt (P) requires physical contact between the student and teacher may be a touch at the elbow prompting the student to move his/her hand to a particular answer, or it may be more invasive, involving the teacher physically moving the students hand to the correct answer If the student must be given a physical prompt to correctly answer the test question, the answer must be marked wrong. This applies to full and partial physical prompts. 130 Prompting During Assessment - Allowable Prompts 2012-2013 NJ APA Administrator Training

131 Scoring Evidence for Independence The total Independence score must be in the form of a percent. Take the number of items completed independently divided by the total number of items multiplied by 100 to calculate the total Independence score. When scoring for Independence, the information does not include accuracy. 131 Independence # items performed independently divided by # of total items times 100 equals % Correct ____/_____ x 100 = ______% 2012-2013 NJ APA Administrator Training

132 LANGUAGE ARTS LITERACY Sample Entry From 2011-2012 Albert 132 2012-2013 NJ APA Administrator Training

133 Language Arts Literacy Entry - Example 133 1 2012-2013 NJ APA Administrator Training

134 134 9-15-11 2012-2013 NJ APA Administrator Training

135 2a 135 2012-2013 NJ APA Administrator Training

136 2b 136 2012-2013 NJ APA Administrator Training

137 137 12-15-11 2012-2013 NJ APA Administrator Training

138 3a 138 2012-2013 NJ APA Administrator Training

139 3b 139 2012-2013 NJ APA Administrator Training

140 Language Arts Literacy Evidence Index – Sample Entry for Albert 2011-2012 4th Grade LAL Entry Entry 1 Standard 3.1 Strand F CPI 2 Matched Link : After reading a passage, identify the meaning of unknown words using context cues Rubric DimensionScoreEvidence Complexity4 Pgs. 2-2b and 3-3b show the student working on the complete CPI Link from the Matched column Pgs. 3-3b (evidence 2) are a different activity than pgs. 2-2b (evidence 1) Different context: different passages and words assessed Different application: multiple choice vs. fill in the blank Performance3 Pgs. 2-2b student scored 20% (which is below 39%) on initial activity Pgs. 3-3b student scored 80% Each item is clearly marked Pgs. 3-3b do not include a more intrusive prompt than pgs. 2-2b Independence3 Pgs. 3-3b student scored 80% Independence Each item is clearly marked 140 2012-2013 NJ APA Administrator Training

141 Professional and Ethical Responsibility 141 The Alternate Proficiency Assessment is intended to assess the academic progress of students with significant cognitive disabilities toward the standards established by the state of New Jersey. The format of the APA requires teachers administering this assessment to develop assessment activities appropriate for each participating student and to provide appropriate documentation of how the activities were used to assess students on what they gained from instruction. Instruction is the key. Assessments should never be used as the instruction. Instruction should take place first, and then assessments should be designed to reflect what was taught. The first piece of APA evidence is based on the initial introduction of the skill, which usually occurs prior to implementing extensive instruction. 2012-2013 NJ APA Administrator Training

142 142 Professional and Ethical Responsibility Teachers are to follow the steps for the development of an entry described in the teacher training materials. Teachers are to save not only copies of the assessments that are being included in the students portfolio but also copies of student work that, while not included in the portfolio, provide further proof that the student received instruction and was given opportunities to practice and show what he/she was taught prior to the final assessment activity. These additional student work activities may be reviewed during a site visit or required to be presented for a security breach visit. 2012-2013 NJ APA Administrator Training

143 143 Professional and Ethical Responsibility Each assessment presented to the student should be unique, meaning that the student should not have previously seen the assessment. The following practices are not allowed: Teaching students to memorize answers for an assessment. Giving students the same assessment (activity) over and over until they complete the assessment satisfactorily. Allowing students to see the assessment and its test questions ahead of time. Allowing someone else to complete work for a student. Remember, there are guidelines for scribing that must be followed if a student needs assistance documenting their answer. Giving an assessment when no instruction has occurred. Giving the students a writing task response and having them copy it and submitting it as their own work. Leaving the date blank on a completed activity and adding a date later. Changing the date on a completed activity. Any other practice that results in misleading information regarding what the student was taught and has learned. 2012-2013 NJ APA Administrator Training

144 144 Professional and Ethical Responsibility Some students may not be able to complete an entire assessment activity all at once. It is acceptable to administer assessment items one or two at a time, as appropriate for the student, but when doing so, the student should only be presented with the item or items that he/she will be expected to complete at that time. It is acceptable to mask the remaining items on the page until they are completed by the student. If a student becomes sick during an assessment or cannot complete an assessment at the time it is given, that assessment should be used as practice and a new assessment should be given at a better time. Remember that the APA has two rather long testing windows that should help address most special testing conditions that may arise for students participating in the APA. 2012-2013 NJ APA Administrator Training

145 Professional and Ethical Responsibility It is the professional and ethical responsibility of all contributors to a students portfolio to ensure that any and all documentation reflect authentic, accurate, and truthful information. Reminder, it is not appropriate to manipulate the date the evidence was created, the prompt/independence levels, the students work, or the scoring percentages. Include only evidence that reflects the students actual performance, even if the student did not make as much progress as you expected. Follow all APA guidelines when administering the assessment. 145 2012-2013 NJ APA Administrator Training

146 Any student portfolio that is found to contain inauthentic documentation may result in professional consequences for school and district staff. All contributors to a portfolio must sign and submit the assessment security form verifying that the APA was conducted properly and in accordance with all directions and training materials. District administrators will provide this form. This is a new policy, and is in line with the general assessment procedures. It is important that administrators review the teacher training modules to ensure you know what is acceptable, and not acceptable, regarding APA assessment. Remember that this is an individual assessment, not a group assessment. 146 Professional and Ethical Responsibility 2012-2013 NJ APA Administrator Training

147 Test Security Agreement Give the teachers the agreement this fall so that they can see what they will sign and submit when they turn in the portfolio. Every teacher that produces APA evidence must submit a signed Test Security Agreement to be placed in the binder. If teachers worked on only one subject, they may write that on the agreement. 1472011-2012 NJ APA Administrators Training

148 Pearson Scoring Center staff scored the portfolios during April and May. Score reports were shipped in June. Most portfolios have been returned. Portfolios designated as security breaches (Void 5) are in process. Information regarding the breach consequences will be forthcoming. Please contact me to arrange a meeting to discuss the districts security breach portfolios. Status of 2011 - 2012 Portfolios and Score Reports 1482012-2013 NJ APA Administrator Training

149 Contact Information For information on interpreting the 2011-2012 APA score reports, please review the Score Interpretation Manual on Pearsons website http://pem.ncspearson.com/nj/apa http://pem.ncspearson.com/nj/apa For missing score reports please contact: NJ Customer Service Team – 888-705-9416 For score report interpretation please contact: Jo-Lin Liang – 609-633-6884 1492012-2013 NJ APA Administrator Training

150 What is Proficient? New cut scores were set in June 2009 which determine how the total score for each subject area is classified. Total scores may be classified as advanced proficient, proficient, or partially proficient. The cut scores were determined based on the grade-level linked skills and content being assessed; therefore, the same number of total points may be classified as advanced proficient in one subject area for a grade level, and classified as proficient for a different grade level, due to differences in the CPI Links and developmental demands of students in different grades. This mirrors the general assessments where each grade level and subject area cut scores are determined independently. 1502012-2013 NJ APA Administrator Training

151 How is a Total Score Calculated? The scoring formula was determined by reflecting on the purpose of the APA, which is to: assess the district and school grade-level instructional program provided to the student, assess a students attainment of knowledge and skills of the CCCS, and encourage high but appropriate instructional standards for all students, while providing necessary supports to access the curriculum and foster the highest level of independence possible for the student. The scoring formula uses the three dimension scores that each entry receives: Complexity Independence Performance 1512012-2013 NJ APA Administrator Training

152 How is a Total Score Calculated? The total score for each content area is calculated by adding the scores for all three dimensions for each of the four entries. Each entry has a minimum possible score of zero and a maximum possible score of 16 points. The total score for each subject area has a minimum score of 0 and a maximum score of 64 points. 1522012-2013 NJ APA Administrator Training

153 How is a Total Score Calculated? Each scorable entry will receive a score of 0 – 4 for each dimension. Unscorable entries receive a zero for each of the three dimensions. Each entry is scored by a minimum of two people (readers). First person is reader one (R1) and second person is reader two (R2). The performance dimension score for any entry is the sum of the two reader scores (R1+R2). The addition of the two reader scores gives more weight to this dimension in the overall total score than the other two dimensions. The complexity dimension score for any entry is the average of R1 & R2. The independence dimension score for any entry is the average of R1 & R2. 1532012-2013 NJ APA Administrator Training

154 Accountability APA test results will be combined with the results from the general assessment for accountability purposes for state and federal reports. For accountability purposes, the APA is both a student assessment, and a school/district program assessment. APA test results are included in the Final Accountability report. 1542012-2013 NJ APA Administrator Training

155 Federal Results – 1% Cap The NCLB 1% cap limits the number of APA proficient and advanced proficient results that can be used in the accountability calculation. It does not limit the number of students who may participate in the APA. Districts legitimately testing more than 1% of the total grade population with the APA should apply for a waiver from the Office of Special Education. The waiver, if approved, allows proficient and advanced proficient results greater than 1% to be used in the accountability calculations. This form is usually mailed by the department to districts in the winter. 1552012-2013 NJ APA Administrator Training

156 Accountability Reports Student data is collected on the APA scan sheet. This data is submitted to districts for verification. All requested revisions to student data must be made during the record change process. No changes may be submitted after this time frame. Accountability reports are produced based on the student data on file as of the close of the record change process. Reports for sending districts include students assessed out of district, out of state, and in district. 1562012-2013 NJ APA Administrator Training

157 Information – Title I Office Up-to-date information on the calculation of Accountability Reports can be obtained on the Department of Educations website www.state.nj.us/education www.state.nj.us/education Review the information provided from the Office of Title I. The Accountability Workbook provides specific details. If you have questions after reviewing the online information, please call the Office of Title I at 609-943-4283. 1572012-2013 NJ APA Administrator Training

158 Main Responsibilities Complete scan sheet materials in February 2013. Work directly with administrators from the childs sending or receiving school to ensure accurate information. Errors on the scan sheet may affect your Accountability Report calculations. Several fields on the scan sheet must be completed in February since they are time sensitive and can not be completed during the Pre-ID process. A few fields on the scan sheet may need to be completed by the students teacher. Collect portfolios from teachers by end of day on February 15, 2013. Administrators may choose to collect them earlier in order to complete portfolio review process when high number of portfolios are produced. Review final student data and submit record change rosters. This is the last opportunity to make data changes. 1582012-2013 NJ APA Administrator Training

159 Final Administrator Review Test Design Review Did the evidence address the required components? Prescribed standards and grade-level CPIs Correct amount of evidence Meets the Universal Scoring Rules Conducting this review periodically will allow you to find and correct errors during the collection period. Security Breach Review Is white out being used to change information? Were prompt levels changed to independent performance? Were dates on the evidence changed? Were wrong answers changed to correct answers? Were wrong answers marked correct? Are the percentage scores accurate? Are there other indications of inauthentic information? 1592012-2013 NJ APA Administrator Training

160 Final Administrator Review Review portfolios for accuracy and authenticity by February 22, 2013. Review the portfolios in light of the students assigned grade level. Review the portfolios in light of the current CPI Links. Review the portfolios in light of the test design requirements. Review the portfolio to ensure all evidence is authentic. Reminder that portfolios should be reviewed multiple times within each collection period. Doing so helps you identify errors early, giving enough time to produce new evidence without errors prior to the end of the collection period. 1602012-2013 NJ APA Administrator Training

161 Shipment of Portfolios The shipment process will be communicated to districts at a later time. Portfolios must be shipped between February 26 and March 11, 2013. Any portfolio shipment dated after March 11 will not be accepted for scoring. Portfolios must be submitted on time in order to be processed. Shipping materials and instructions will arrive in the Chief School Administrators office in February 2013. 1612012-2013 NJ APA Administrator Training

162 Administrator/Coordinator Activities General Responsibilities When information is sent to you, pass it on to district and school administrators and teachers. Review the time lines and request APA information from your chief school administrator when it is scheduled to be delivered. Review participation guidelines carefully. Ensure that all students who need to be assessed are identified (both in-district and out-of-district). 1622012-2013 NJ APA Administrator Training

163 General Responsibilities (continued) Review school educational programs and ensure alignment with dimensions of the scoring rubric. Ensure that the APA teachers participate in the online teacher training sessions. Review the APA Procedures Manual. Pass along questions and concerns to NJ DOE or contractor. 163 Administrator/Coordinator Activities 2012-2013 NJ APA Administrator Training

164 Administrator/Coordinator Checklist Before Assessment Identify a School Coordinator for each school. Register number of students participating in APA and update mailing addresses using Material Survey no later than Date to Be Determined. Create a teacher training schedule for your school, identify teachers required to participate, and implement training. Ensure all materials are downloaded prior to the selected training day(s). Have a sign in sheet for all teachers to verify attendance and completion of the training. Utilize all APA training materials, including the Content Resource Guide and Five Item documents, in addition to the Teacher Training Modules and Procedures Manual. Distribute materials to schools/teachers. 1642012-2013 NJ APA Administrator Training

165 During Assessment Distribute materials to schools/teachers. Binders and manuals will be shipped to district/receiving school. Copies of the APA training materials may be downloaded as of September 7, 2012, from the Pearsons website http://pem.ncspearson.com/nj/apa. Click on Documentation, then Procedures Manual. http://pem.ncspearson.com/nj/apa Content Resource Guide and Five Item Sample documents may not be available until September 19, 2012. 165 Administrator/Coordinator Checklist 2012-2013 NJ APA Administrator Training

166 During Assessment (continued) Gather information on students sending/receiving status and collect student demographic info. Work with the receiving/sending school to confirm student data. Check periodically that teachers are working on the APA. 166 Administrator/Coordinator Checklist 2012-2013 NJ APA Administrator Training

167 After Assessment Notify teachers of timeframe for returning the completed APA portfolios. Collect and review all APA portfolios for accuracy and authenticity of information and inclusion of all required components. Be sure to review the portfolios for adherence to the universal scoring rules. Review the scan sheet information to ensure accuracy of all data. 167 Administrator/Coordinator Checklist 2012-2013 NJ APA Administrator Training

168 After Assessment (continued) Package the APA materials securely and label them according to instructions. Contact the designated carrier for return of the portfolios to arrange pickup of the materials. Ensure all portfolios are submitted by March 11, 2013 deadline. Portfolios not received by shipping carrier by March 11, 2013 will not be accepted for scoring. 168 Administrator/Coordinator Checklist 2012-2013 NJ APA Administrator Training

169 APA Teacher Involvement Teachers are needed in March 2013 for a week to score portfolios. Send an email to Elizabeth Celentano if someone is interested in assisting in the scoring process. Email address is Elizabeth.Celentano@doe.state.nj.usElizabeth.Celentano@doe.state.nj.us Summer committees will be formed to assist in the test specification revision for alignment to the general assessments and Common Core Standards. Teachers and curriculum specialists will be needed to complete this work. More information will be forthcoming in Spring 2013. 1692012-2013 NJ APA Administrator Training

170 Dynamic Learning Maps New Jersey is one of 13 states collaborating with the University of Kansas on the future version of an alternate assessment for students with the most significant cognitive disabilities. Opportunities arise for districts to participate in the process via document reviews, meetings, and implementing pilot components. If you would like to become involved please email Elizabeth Celentano. This revised assessment is expected to be implemented in some form (pilot?, operational?) in 2014-2015. As information becomes available it will be disseminated. 1702012-2013 NJ APA Administrator Training

171 Where To Get More Information? Educational Links, Information, and Resources on Assistive Technology and Universal Design for Learning, an excellent reference document for teachers and administrators can be downloaded from the DOE website http://www.state.nj.us/education/specialed/tech/ 1712012-2013 NJ APA Administrator Training

172 Where To Get More Information? Other educational materials and resources can be accessed through the New Jersey Learning Resource Center Network website: http://www.nj.gov/education/lrc http://www.nj.gov/education/lrc 1722012-2013 NJ APA Administrator Training

173 Where To Get More Information? Elizabeth Celentano (test design, training, test administration, etc.) Office of Assessments New Jersey Department of Education 609-984-0107 elizabeth.celentano@doe.state.nj.us Jo-Lin Liang (student data, Pre-ID, record changes, score interpretation, etc.) Office of Assessments New Jersey Department of Education 609-633-6884 Jo-Lin.Liang@doe.state.nj.us Office of Special Education (general special education questions) New Jersey Department of Education 609-292-0147 http://www.nj.gov/education/specialed/ Department of Education website: www.state.nj.us/education 1732012-2013 NJ APA Administrator Training

174 Where to Get More Information? Topics: Training materials, score reports 2011-2012, portfolio binder return Pearson - Bruce Murphybruce.murphy@pearson.combruce.murphy@pearson.com Customer Servicenjapacustsvc@support.pearson.comnjapacustsvc@support.pearson.com Phone: 888-705-9416 Pearson website: http://pem.ncspearson.com/nj/apa http://pem.ncspearson.com/nj/apa Topics: Explanation and Error Sheet issues, Evidence questions (through November 2012) ILSSA – Lou Ann Landlland@uky.edulland@uky.edu Alson Colealson.cole@uky.edu 1742012-2013 NJ APA Administrator Training

175 Where to Get More Information? Topics: Student registration, Pre-ID, scan sheets, portfolio shipments, record changes, online entry cover sheets, etc. Questar Assessment, Inc. NJ APA Customer Support: NJAPACustomerSupport@questarai.com Teri Broderick Tbroderick@questarai.com Phone: 866-690-4733 NJ APA Website address: https://nj-servicepoint.questarai.com/Login.aspx 1752012-2013 NJ APA Administrator Training

176 Winter 2013 Training Materials In January 2013 a new Power Point presentation will be uploaded for your review. You will be notified when it is posted and on which website. It will include reminders about key concepts provided today as well as information about necessary tasks related to returning the portfolios. Please make sure you read these materials and contact the Office of Assessments if you have questions. 1762012-2013 NJ APA Administrator Training

177 Thank You Thank you for taking time to attend this session. If you have not had all of your questions answered, please see one of the presenters for more information. 1772012-2013 NJ APA Administrator Training


Download ppt "Overview of the 2012-2013 Alternate Proficiency Assessment Administrator/Test Coordinator Training 12012-2013 NJ APA Administrator Training."

Similar presentations


Ads by Google