Overview of the Alternate Proficiency Assessment

Overview of the 2012-2013 Alternate Proficiency Assessment
Administrator/Test Coordinator Training NJ APA Administrator Training 1

Administrator and Teacher Training
This session is only for APA administrators. Districts are responsible for teacher training. Teacher training modules are available on the Pearson website An administrator must ensure that all teachers participate in the training sessions. Teachers may be trained immediately, but every effort must be taken to ensure all teachers are trained prior to October 1, 2012. If a student enrolls after October 1 and a teacher needs training, train the teacher as quickly as possible. The modules will be available all year, though it is recommended that you download these modules and retain them in your district. NJ APA Administrator Training

Agenda APA Participation pages 4 – 22
Student Transfers, Medical Leave, and Excessive Absenteeism pages 23 – 31 Administrator Responsibilities pages 32 – 34 Material Survey, Pre-ID data, record changes, etc., pages 36 – 45 APA Test Design pages 46 – 80 Common Errors and APA Revisions pages 81 – 84 Scoring Rubric Information pages 85– 131 Sample Entry from pages 132 – 140 Professional and Ethical Responsibility When Compiling and Submitting APA page 141 – 146 Score Reports and Accountability pages 147 – 156 Administrator APA Review and Other Tasks pages 157 – 167 Request for Teacher /District Assistance pages Contact Information pages 170 – 174 Winter Training Materials page 175 NJ APA Administrator Training 3

What is the Purpose of the APA?
To measure performance of students with the most significant cognitive disabilities on the CCCS the knowledge and skills that ALL New Jersey students are learning To ensure that ALL New Jersey students are included in the accountability system that is required by the No Child Left Behind Act NJ APA Administrator Training 4

What is the APA? An alternate assessment that measures achievement of CCCS specifically designed for students with the most significant cognitive disabilities. A portfolio of student work that demonstrates a student’s performance of knowledge and skills that are LINKED to grade level knowledge and skills. This is a requirement of the USDOE. NJ APA Administrator Training 5

Who are the Students who Participate in the APA?
Students with the most significant cognitive disabilities Decisions about who participates in the APA are not based on: Student’s disability category Student’s educational placement On which assessment the student is most likely to score highest NJ APA Administrator Training 6

Participation Decisions
Students with the most significant cognitive disabilities will participate in: The APA in one/some content area(s) and the general assessment with accommodations in the other content area(s) or The APA in all content areas (LAL, Mathematics, and Science). Note a student may not take the general education assessment and the APA assessment in the same content area. NJ APA Administrator Training 7

Determining Who Participates In The APA
Decisions are made: By the IEP team Separately for each content area assessed by the general statewide assessment Using information about the student and the assessments Test Specifications/Sample Test forms Present Levels of Academic Achievement and Functional Performance/Goals and Objectives Student Work Samples NJ APA Administrator Training 8

Participation Guidelines
Information on APA participation guidelines is provided in your training packet. This checklist must be completed in order to determine which students are eligible for the APA. An administrator must sign the form and include it in the APA binder. It is usually the sending school administrator, but if both sending and receiving school agree on the answers to the questions, then a receiving school administrator may sign the form. A teacher may not sign the form. Access to the Directory of Test Specifications and Sample Test for the general assessment is necessary in order to complete the checklist correctly. NJ APA Administrator Training 9

Participation Criteria Checklist
Is the nature of the student’s cognitive disability severe? The student’s general level of cognitive functioning is significantly below the grade level expectations for his or her same aged peers and the student requires substantial modifications to the grade level curriculum. The student requires more intensive direct instruction than his or her peers to acquire knowledge and skills in the grade level curriculum. The student requires more extensive systems of support in order to participate in the grade level curriculum. NJ APA Administrator Training

Is the student’s cognitive disability so severe that the student is not receiving instruction in any of the knowledge and skills measured by the general statewide assessment? While all students are to be instructed in the CCCS for their grade level, students with more significant cognitive disabilities typically receive instruction in the knowledge and skills at a less complex level than that measured by the general statewide assessment. Upon reviewing the sample test forms, if the student is NOT receiving ANY instruction at the same skill level, then the answer to this question is YES. NJ APA Administrator Training

Is the student’s cognitive disability so severe that the student cannot complete any of the types of questions on the general statewide assessment in the content area, even with accommodations and modifications? Upon reviewing the sample test forms and questions for each content area of the general assessment, along with the state approved testing accommodations and modifications, if it is determined that the student is NOT able to complete any types of questions even with accommodations and supports, then the answer to this question is YES. NJ APA Administrator Training

Is the student’s Individualized Education Program (IEP) aligned with grade level CCCS through modified expectations? The student is receiving grade level instruction at a significantly modified and less complex level than their grade level peers. The student may need more extensive systems of support, as identified in the IEP, in order to access the curriculum. NJ APA Administrator Training

Participation Criteria Checklist (continued)
The answer to all four questions must be YES in order to assess the APA for a content area. A signed copy of the checklist must be placed in the inside front pocket of the APA binder. Usually the Sending district administrator signs the form. However, if the Sending and Receiving district agree on the APA status, the Receiving district administrator may sign the form. NJ APA Administrator Training 14

What Must Be Written in Each Student’s IEP?
If the student is taking the general assessment, list the assessment and the accommodations/modifications (from approved list) that will be provided. If the student is not taking the general assessment in one or more content areas, include a rationale for this decision. If the student is not taking the general assessment for LAL, Math, and/or Science, list that he/she will be participating in the APA. IEP model forms and information can be found on the DOE website NJ APA Administrator Training

Assessment and CCCS Materials
Additional copies of all APA materials, including the teacher training materials, may be obtained on the Pearson website: Copies of the Core Curriculum Content Standards, test specifications, and sample tests for the general assessments can be obtained on the DOE website: Copies can be ordered for a fee by calling the DOE publications office at NJ APA Administrator Training 16

What Grades and Subjects Are Assessed?
Students in grades 3 – 8 and 11 must be assessed in language arts literacy and mathematics. Students in grades 4 and 8 must also be assessed in science. Students in high school must be assessed in science the year they receive instruction in biology (grade 9, 10, 11, or 12). High school students may have an APA submitted for science in a different year than the year the APA is submitted for language arts and math. NJ APA Administrator Training 17

Grade 11 and 12 Students If a senior is new to the state this year and has not participated in either the APA or the HSPA, the IEP team must determine which assessment is appropriate and the student must participate in this assessment. Students who were juniors last year and should have participated in the APA, but did not, must participate in the APA this year whether they are still in 11th grade or are now in 12th grade. Students in high school do not need to retake the APA if the test result was Partially Proficient. Score report results on the high school APA do not affect the ability to graduate. NJ APA Administrator Training 18

Retained Students If a student has been retained in a grade due to poor academic performance, the student must be reassessed in that grade. Students who remain in the same assigned grade twice due to reasons not related to academic performance must be assessed only once in that assigned grade, except for the circumstances described on the next slide. Example of reason not related to academic performance: student is kept in the middle school an extra year because it was determined that the student was not yet ready for the high school environment (socially, structure of new large building, etc.) NJ APA Administrator Training

Retained Students If a student received a score report with a Void Code rather than a Proficiency Level in a content area, and the student is retained in that grade, the student must participate in the APA again in the same grade. Also, if a student in 11th grade received a Void Code in language arts literacy and/or mathematics, the student must be assessed again in these content areas, either in 11th grade if they remain in that grade, or as a 12th grader. NJ APA Administrator Training

Assigned Grade Level The grade level of the student must be marked on the Demographic Scan Sheet that is included in the portfolio. This is the assigned grade level used during scoring of the APA. If a record change for the student’s grade level is submitted in April 2013, the APA assigned grade level becomes the grade level provided during record changes. The assigned grade level will be matched against the grade level of the CPI Links selected for assessment during the scoring process. If the grade level of the CPI Links does not match the assigned grade level provided during data collection, the entry will receive a zero score. Please note that students in high school (grade 9, 10, 11, or 12) are tested on the grade 12 CPI Links. NJ APA Administrator Training

LEP LAL Exemption If a student has entered the United States after July 1, 2012, AND is currently enrolled in a language assistance program, this student is EXEMPT from the LAL assessment. This exemption is not allowable for high school students. The APA scan sheet delivered to districts in February 2013 must indicate the LEP LAL exemption. NJ APA Administrator Training 22

Student Transferred to NJ District from Out of State after October 31, 2012
Students who require an APA, and enter your school after October 31, 2012, transferring from out of state, have missed the cut-off date for the development of the APA assessment. Submit a binder, a completed scan sheet, and a letter from an administrator that indicates from which state the student transferred, the date of the transfer, and noting this as the justification for missing APA evidence. The score report will indicate a Void 4, no portfolio evidence, when the student is a late registrant. Please note that instruction related to the CCCS must still occur throughout the year, regardless of the enrollment date. NJ APA Administrator Training

Student Transferred from One NJ District to Another NJ District
The district that has the student during any part of a collection period must compile the portfolio. Once the student transfers within NJ, the district must send the portfolio to the new district. The new district must complete the portfolio and submit it for scoring. If a student is enrolled in a school or district after July 1, 2012, the administrator may bubble on the scan sheet Time In School less than one year. This will remove the student’s results from the Accountability report. Note this is different than last year and is based on the accountability waiver calculation. NJ APA Administrator Training

Student Transferred from One NJ District to Another NJ District
If the student moved to another district you must send the portfolio to the district where the student is newly enrolled. As long as a student is in your district, even for a limited time, you must begin instruction of the CPI Links, and begin collecting APA evidence. Be sure to transfer the APA materials to the “new” district in a timely manner. You, the “former” district, do not need to submit a binder or letter to the APA program since the “new” district will send in the portfolio. . NJ APA Administrator Training

Students Leaving the State
Please note that if a student transfers out of New Jersey prior to the end of the second collection period (February 15, 2013), the portfolio does not need to be submitted for scoring. It may be sent to the out-of-state school for use when reviewing the student’s educational program, or given to the parent NJ APA Administrator Training

Medical Emergency Medical emergency is the occurrence of a severe medical or psychiatric condition or episode which requires medical attention or supervision during which time the student is not able to participate in state assessments. APA students who are not receiving instruction due to a medical emergency may be eligible for a Void Code due to Medical Emergency when there is not enough APA evidence. APA uses the term extensive sick leave/hospitalization interchangeably with the term medical emergency. Note that no other definition may be used when determining a void due to medical emergency. See the requirements for this void on the next slide. NJ APA Administrator Training

Medical Emergency When a student is out of school for an extended amount of time and not receiving instruction due to extensive sick leave or hospitalization, the portfolio may be eligible to receive a Void 1 (medical emergency code). Eligibility is based only on the information below. If the student is receiving instruction for 10 days or less during a collection period, and The student has an extended hospitalization or leave due to illness and is not receiving instruction, and An official record documenting the student absences is available, then you must: include a letter on school letterhead from an administrator documenting the medical emergency. include a formal district/school record of the student’s attendance and absences. If all of these criteria are met then the portfolio will be voided due to extended illness during one or both collection period(s). NJ APA Administrator Training

Record of Absences The record of absences must be a formal attendance record produced from the district’s record system. It must indicate all days school is open and which days school is closed for holidays, etc. This record must be marked with the student’s attendance – present or absent. You may not submit an informal record of attendance such as a memo indicating the student was absent during the year. NJ APA Administrator Training

Medical Emergency Paperwork
If the paperwork submitted in the APA binder does not include all necessary components as defined on the previous slides, a Void for Medical Emergency/Illness will not be granted. An administrator may contact the Office of Assessments with any questions on this topic. NJ APA Administrator Training

Absenteeism – Non Medical Emergency
If a student is receiving instruction for 10 days or less in a collection period (excessive absenteeism), and the reason for absence is unrelated to a medical emergency/extensive illness, then the school administrator should contact the Office of Assessments. If it is determined that a Void for No Evidence is appropriate, two things must be submitted by an administrator in the APA binder: a letter on letterhead indicating the reason for the excessive absenteeism (parent took student out of country, etc.) a formal record of the student’s attendance/absences produced from the district’s record system. NJ APA Administrator Training

Test Coordinator’s Main Responsibilities
Ensure APA participation guidelines are used correctly by IEP teams. Ensure IEP teams have all of the necessary information about the general assessments and the APA participation guidelines clarifying questions to ensure the decision is made based on the requirements. Ensure all APA teachers complete the teacher training modules and review all training materials including the Procedures Manual and Content Guide Five Item Samples document. Have the teachers sign in to your district training and keep a record of attendance. Ensure all teachers view all modules. Ensure all teachers utilize the critical additional training materials listed above as the materials as a whole deliver the information necessary to successfully implement the APA. NJ APA Administrator Training 32

Periodically review teacher’s APA process, use of test specifications and selected CPI Links, instructional lessons, etc. This review must occur at the beginning of each collection period, and at least once prior to the end of each collection period. Reviewing the materials before the cut off date allows time if necessary for production of additional/different APA evidence. Ensure that only the CPI Link documents are used. The APA continues to use the older standards until the other assessments finalize the alignment to the Common Core Standards and the APA specifications are revised. NJ APA Administrator Training

Ensure registration of APA students, review and update district contact info, and submission of Student Pre-Identification file prior to the deadline. Complete Demographic Scan sheet providing accurate data Complete final portfolio review prior to shipping Ensure timely shipment of portfolios Review APA record change rosters and correct/revise any inaccurate data NJ APA Administrator Training

Vendor Transition The APA program is transitioning to a new vendor, Questar Assessment. This transition will begin in September. Details and deadlines related to student registration, Pre-ID submission, the new Online Entry Cover Sheet, etc., will be forthcoming. We appreciate your patience and will send details on these topics regularly as they become available. NJ APA Administrator Training

Materials Survey – Registration of APA Students and District Information
Register the number of APA students in your district no later than (date to be determined). Failure to register your students will result in a failure of delivery of APA materials to your district. Survey should be completed online (see memo). Do not register out-of-district students that are assessed in New Jersey. The receiving district will register these children. Register students that are sent by your district to an out-of- state placement as if they are in their home school. If no APA students this year, then indicate 0 (zero) for the applicable grade level. Confirm/revise district address, s, and contact information to ensure accurate deliver of information. NJ APA Administrator Training

Online Registration – Material Survey
The online material survey is not yet available. A memo will be sent to districts when the website is ready. is the Questar website address for use in the future. Log on information including User IDs and Passwords will be provided to your Chief School Administrator (CSA) If your CSA does not receive this information by October 30, call Questar customer service at or send an to NJ APA Administrator Training 37

Students Sent to an Out-of-State School
The Sending District is responsible for communication to the out-of-state placement. This includes providing training materials to the school and ensuring that the CCCS is instructed and assessed. This includes ensuring that the APA is in process, evidence is being collected, and that the APA will be completed and submitted to you prior to the return portfolio shipment deadline. Portfolios must be sent from the out-of-state school to the sending district prior to February 15, The sending district includes these portfolios in the district’s shipment. Registration of these students, completion of scan sheets, record changes, etc., is the responsibility of the sending district. Include these students in the sending school APA enrollment counts. NJ APA Administrator Training 38

Electronic Pre-ID File Submission
This is the first opportunity to submit accurate student information/data. Administrators will need to provide such info as student birth date, SE classification code, grade level, etc. Student demographic information must be submitted electronically on the Questar website. Please be patient while the website is developed. Pre-ID Instructions will be sent to your Chief School Administrator. Pre-ID information will be printed on the demographic scan sheet in lieu of gridding the information. NJ APA Administrator Training

Electronic Pre-ID File
Last date to submit pre-ID files is: Date to be determined. It is not advisable to wait until the last day since you may find that the file is not accepted due to data errors. Within the submission window you may submit a revised file when necessary. Pre-ID information will be printed on scan sheets for those participating districts and sent with the Return Materials shipment in February Blank scan sheets will also be provided. Errors/omissions on the Pre-ID label must be corrected by gridding the info on the scan sheet. NJ APA Administrator Training

Scan Sheet Student demographic scan sheets, along with directions for the completion, are shipped to districts in February. This is the second opportunity for districts to submit accurate student information/data. A county-district-school code (CDS) must be provided for the school the student is attending. This is either a Home (neighborhood) school or a Receiving school. If the student is in a receiving school, a CDS code for the sending school must also be provided. The sending school must provide this code to the receiving school for inclusion on the scan sheet. The home or receiving school code is pre-gridded on the scan sheet. APA will begin to use the most up-to-date CDS codes for the private and specialized, separate schools beginning this year. Please ensure you know your current code which is also being used for NJASK and HSPA. We will no longer be using CDS codes that start with “90”. NJ APA Administrator Training

Scan Sheet If a Pre-ID file was submitted, check the preprinted information on the scan sheet for accuracy. You may correct information submitted in the Pre-ID file by gridding the correct information on the scan sheet. The gridded information will override the preprinted information. The only exception to this is changes to the student name, which may only be corrected during the record change process. We must maintain the same exact student name on the file in order to process the information until record changes. Only at that point can we make changes to the spelling of the student’s name. NJ APA Administrator Training

Taking General Assessment Field
Detailed instructions are provided in the use of this field in the Pre-ID directions and the Scan Sheet directions. Only code this field when the student is NOT taking the APA, but is taking the general assessment (NJASK, HSPA, NJBCT) in a content area. Remember, a student may only take the APA OR the general assessment, not both, for a content area. Marking the student as Taking the General Assessment will Void all results for the APA. NJ APA Administrator Training

Record Change Process A roster of APA student demographic information is shipped to all districts in April (Will verify date with new vendor and confirm in Record Change Memo.) For student’s who remain in their Home (neighborhood) school, the “Home” district must review and submit roster changes. For student’s who have both a receiving and sending school, both districts must review the record change rosters for accuracy. Beginning this administration, the Sending School, not the Receiving School will submit the record changes to the vendor. NJ APA Administrator Training

Record Change Process The Sending School/Home School is responsible for correcting all student information that was provided on the scan sheet. The Receiving School must also review the roster, as errors (e.g. , CDS code, demographic information) may impact the student score reports and accountability reports. Corrections and revisions to the data must be submitted to the student’s Sending School at least several days prior to the record change deadline. The Sending School must submit these changes to Questar during the record change timeframe. This is your final opportunity to correct/revise the data. This includes the field Taking the General Assessment. The final record change roster data will be used for the accountability report. No changes will be made after this time. NJ APA Administrator Training

APA What will this portfolio look like?
What are the required components? NJ APA Administrator Training 46

What is the Format of the APA?
Portfolio: Paper based, submit in a 3-ring binder Binders are mailed to districts in October, based on the APA registration counts the district administrator submitted. Content Areas: Mathematics, Language Arts Literacy, and Science Science is included only in grade 4, 8, and high school. NJ APA Administrator Training

What is the Format of the APA?
Entries relate to content standards, grade-level CPIs, and CPI Links. Each content area will have four entries, each one reflecting the assessment of one CPI Link. Evidence documents educational instruction and student performance of skills. The instructional activity used at the beginning and at the end of the CPI Link instruction is submitted as the assessment evidence. NJ APA Administrator Training

Test Design NOTE: NO entry should contain more than four pieces of evidence. This graphic represents a student who is being assessed in all three subjects with the APA. Not all portfolios will include all three subjects. NJ APA Administrator Training

Test Design Example – Language Arts (Same Structure for Each Content Area)
NOTE: NO entry should contain more than four pieces of evidence. An entry containing more than four pieces of evidence will result in zero scores for all dimensions. NJ APA Administrator Training

When Do I Collect Evidence?
Activity Collection Period Initial Activity September 4 - November 16, 2012 Final Activity December 10, February 15, 2013 Dates on the evidence must include month, day, and year. Be sure to plan early in the year when you will instruct and assess each CPI Link and content area. All students will need to be assessed on an initial activity within the correct time frame above, even if the majority of instruction will not occur until the winter. NJ APA Administrator Training

Four Entries per Content Area
Entry One Entry Two Entry Three Entry Four Based on Standard and Strand 1 Based on Standard and Strand 2 Based on Standard and Strand 3 Based on Standard and Strand 4 Select CPI & CPI Link Select CPI & CPI Link Select CPI & CPI Link Select CPI & CPI Link Provide instruction & document activities and results Provide instruction & document activities and results Provide instruction & document activities and results Provide instruction & document activities and results One Piece of Evidence from September 4 - November 16, 2012 One Piece of Evidence from September 4 - November 16, 2012 One Piece of Evidence from September 4 - November 16, 2012 One Piece of Evidence from September 4 - November 16, 2012 One Piece of Evidence from December 10, 2012- February 15, 2013 One Piece of Evidence from December 10, 2012- February 15, 2013 One Piece of Evidence from December 10, 2012- February 15, 2013 One Piece of Evidence from December 10, 2012- February 15, 2013 NJ APA Administrator Training

Page Numbering A table of contents that references page numbers in the portfolio must be included. Page numbering for evidence that is longer than one page should be numbered using a combination of page number and letters (i.e., 2, 2a, 2b, etc.) For example, a student was given a reading comprehension test (matches the CPI Link) with 8 test items. The test was two pages long, but it is one activity/piece of evidence. The pages might be numbered page 3 and 3a. Entry Cover sheets get their own page numbers. NJ APA Administrator Training

Entry Cover Sheet The vendor will build an electronic entry cover sheet that will allow teachers to use drop-down boxes to populate most of the fields on the form. This form will then be printed from the web. Teachers must still type/write in their activity descriptions on the form. This form will not be ready until later in the fall. NJ APA Administrator Training

Entry Cover Sheet Teachers should use/complete the old version of the cover sheet as a place holder until the new form is available. The Procedures Manual and Pearson’s website has the old style Entry Cover Sheet for Once the new online form is available teachers should use the old form to assist in transferring the information to produce the new Entry Cover Sheet. NJ APA Administrator Training

2012-2013 NJ APA Administrator Training

Contents of the Entry Each entry requires documentation (evidence) of student performance of the skill stated in the CPI Link. Two pieces of evidence is the requirement for an entry. Includes two different activities. One piece of evidence from the initial activity, and one piece of evidence from a different, “final” activity. Evidence may come from classroom work, community-based settings, or other places where student is working on grade-level CPI Links. NJ APA Administrator Training

Contents of the Entry (continued)
The first piece of evidence documents the initial, or very early, performance of the CPI Link. This evidence must capture what the student does/does not know related to the CPI Link. This evidence must reflect the entire CPI Link in one activity, and be collected between September 4 and November 16, 2012. In order for the CPI Link to be eligible for the student’s APA entry, the accuracy score on the activity can not be above 39%. Additional information on scoring the evidence is included later in the training. NJ APA Administrator Training

Contents of the Entry (continued)
The second piece of evidence documents final performance of the CPI Link. This evidence should capture what the student has learned related to the CPI Link. This evidence should assess the same CPI Link using a different instructional activity, and be collected between December 10, 2012, and February 15, 2013. More information on what makes a different activity is found in the Teacher Training Modules. NJ APA Administrator Training

Evidence Requirements
Evidence must document the entire CPI Link noted on the Entry Cover Sheet. Evidence should relate only to the CPI Link, not assess other skills as well. Providing evidence that documents multiple skills (skills not part of the selected CPI Link), or less than the entire CPI Link, will result in a low scoring entry. Have at least 5 items/questions/task elements included in each activity NJ APA Administrator Training 60

How are CPI Links Related to the CCCS?
Grade-level CPIs are the building block. Content Centrality establishes what concept must be assessed. Intent/Essence of the CPI Performance Centrality directs the level of skill statement. Complexity/difficulty APA CPI Link can be less complex than the grade-level CPI and/or less difficult Three types of links–Matched, Near, and Far–reflect the degree of complexity/difficulty when compared to the grade-level CPI. NJ APA Administrator Training 61 61

CPI Link Grade Level CPI Far Links Near Links Matched Links Essence
CPI 3.1.5G13 Recognize figurative language in text (e.g., simile, metaphor, personification, alliteration) Essence of the CPI: Identify figurative language Matched Link Near Link Far Link Find examples of three different types of figurative language in a text* Match at least two different types of figurative language to its type* Identify examples of similes Identify examples of personification Identify examples of alliteration Identify examples of metaphors Essence Links NJ APA Administrator Training

Using the CPI Links for Instruction
CPI Links should not be used in isolation. Read the Standard and Strand for the overall concept. Read the CPI and Essence. Make sure instruction is within the Essence of the Standard, Strand, and CPI. Evidence of instruction on CPI Link will be scored not only on the discrete Link but also within the Essence of the standard, strand, and CPI For example, 3.1.3G3 is a reading standard with a Matched Link that states “Interpret information from graphs, charts, and/or diagrams found in text.” The evidence must assess the student’s interpretation of graphs/charts/diagrams within the context of reading text and not within the mathematical context of data analysis. NJ APA Administrator Training

Evidence Must Assess ONLY one CPI Link
Remember, when reviewing the CPI Links in the APA Procedures Manual, a CPI Link is only one bullet. Once the type of link to assess a student is determined (Matched, Near, or Far Link), then select only one bullet in that column. Assess the entire CPI Link providing the necessary systems of support Include the student’s response to the test item Must be completed in the student’s mode of communication Ensures that the student can review his own work This is a good way to “test” if the evidence documentation includes the necessary components, but it is not a requirement that you document the student’s actual review of the work. NJ APA Administrator Training

Acceptable Format for the Evidence
Evidence must show the test items and student responses. Acceptable evidence includes Samples of graded student work Series of captioned photographs Must evidence the student actually performing the skill Snapshot of completed student work that documents student performance of the skills contained within the CPI Link Must be a photo of a student work product that is too large or bulky to include in the portfolio (i.e., a poster, a written piece using objects or textured cues, etc.) NJ APA Administrator Training

Acceptable Evidence What else is needed to make evidence acceptable?
All evidence must include: Student’s Name Date of Activity Completion using Month, Day, and Year (must be within the collection windows to be valid) The accuracy of each student response Use + (plus) for correct and – (minus) for incorrect responses The independence level of each response I=Independent performance, V=Verbal prompt, G=Gestural prompt, M=Model prompt, P=Physical prompt (presented from least to most intrusive) More information on use of prompting appears later in the presentation NJ APA Administrator Training

Acceptable Evidence What else is needed to make evidence acceptable?
All evidence must include: A minimum of 5 items that assess the CPI link A piece of evidence must include at least 5 test items that assess the skills contained within the CPI Link. For example Ordering decimals would require 5 sets of numbers with which to order—not just that 5 decimals were ordered. More information is found in Modules IV, V, and VI of the online training, in the Procedures Manual, and in the CPI Content Guide and Five Items Resource. NJ APA Administrator Training

A Minimum of 5 Test Items For CPI Links, an activity must include at least 5 test items. For example A student will answer 5 comprehension questions about a completed job application. A student will use multiplication to check the answers to 10 division problems. NOTE via an example: If a student is given 7 pictures, and is asked to circle the plants, they must also be asked to cross out the pictures that are not plants. They may not have items that do not require a response and then score it as if the item was correct. Also, there must be enough plant pictures to test the skill, and a balance of right to wrong options. NJ APA Administrator Training 68

Rule 5 : Example Five Items
When the Link has an “and” in it, 5 items across the skills are required rather than 5 items for each part of the Link. For instance, 4.2.7E1 Near Link “Calculate the area of a triangle, rectangle and square” NJ APA Administrator Training

Acceptable Evidence Student work samples serve as the best evidence to document all of the necessary components of the assessment activity. This type of evidence includes the test items/questions, response options (if applicable), and student responses. Teachers may need to scribe the student’s responses when necessary. The use of videotape, audiotape, data charts, and most photographs are not acceptable evidence. NJ APA Administrator Training

Student Work Samples Used as Evidence May Be
Computer screen printouts of student’s test item and response after hitting a switch. Summary printouts where each response is not evidenced is unacceptable. An essay written using picture symbols and scored using a rubric. A worksheet completed by the student. Drawings to extend patterns. Completed job application using name, address, and other stamps. A graph made from ordered pairs. NJ APA Administrator Training

Student Work Samples May Be
A paragraph written using sentence strips A printout of a sentence written using a picture writer (i.e., Writing with Symbols, PicWriter, etc.) A snapshot of work too large to fit into the portfolio (a poster, a model, etc.) NJ APA Administrator Training 72

Acceptable Photographic Evidence
A series of captioned photographs that document the student performing the skills included in the CPI Link will be accepted only if: Each item and the student’s response is clearly present in each photograph Must include a description for each photo Must include running record of the items performed Must include accuracy and independence information for each item and include total percent scores for each dimension Must meet the requirements for the Universal Scoring Rules NJ APA Administrator Training

Acceptable Photographic Evidence (continued)
A photograph of a large or bulky completed project will be accepted only if the picture captures the evidence requirements for the activity. A description of the activity must be included along with the documentation of the scoring, but the actual work must be clearly shown in the picture. The Universal Scoring Rules requirements must be met. NJ APA Administrator Training

Scribing Work Samples Acceptable Scribing Technique
Unacceptable Scribing Technique Uses the student’s communication mode to document the student’s answer Does not use the student’s communication mode Allows the student to review her/his work Does not allow the student to review her/his work Is used to record the student’s response on a work sample from eye gaze, a voice-output device (VOD), Augmentative Communication Device, reach and grasp, etc. Does not show the student performance of the skill (observational data) NJ APA Administrator Training

Unacceptable Evidence
Data charts Single photographs of student working on an activity Photographs of student work that could have been included Any work that is not completed by the student Note, the student responses must be marked at the time of the activity, not at a later point in time, in order to be acceptable evidence. NJ APA Administrator Training 76

Unacceptable Evidence
Has fewer than 5 items/questions/task elements Has no name, date, or incomplete date Is scored using anything other than percentages e.g., ratios (4/5), percent ranges (90-100%), letter grades (A, B+) Has percent scores that can not be replicated by scoring staff NJ APA Administrator Training

Evidence Universal Scoring Rules must be met on two pieces of evidence for the entry to be scored based on the rubric. Otherwise, the entry will receive a score of zero for all dimensions. If any entry does not have at least two pieces of evidence OR has more than four pieces of evidence per entry, it will receive a score of zero. Two pieces of evidence are all that is required for APA. NJ APA Administrator Training 78

Universal Scoring Rules
The evidence must include the student’s name. The evidence must include the complete date (month/day/year). Verify that the dates fall within the appropriate collection period: September 4, November 16, for the first piece of evidence December 10, February 15, for the second piece of evidence The evidence must be presented in the appropriate format. The evidence must include a writing rubric when specified in a Writing CPI Link. The evidence must reflect the student’s mode of communication. The evidence must include at least 5 items. The evidence must show student responses for at least 5 items.. The evidence must assess the entire link while connecting to the essence of the CPI, standard, and strand. Both pieces of evidence must assess the same CPI Link and skills. The evidence must not include more than the skills contained within the CPI Link. Note: The USRs pertain to both student work samples, as well as the writing rubric when required. If the entry does not adhere to all rules it will receive zero scores. . NJ APA Administrator Training

Activities – Errors to Avoid
The first piece of evidence should not include a more difficult application/context of the skill than the final activity/evidence. Examples of this type of error (a more difficult initial activity) are below: The initial activity has longer test items with more complex steps (e.g., open-ended essay questions vs. multiple-choice) than the final activity. The initial activity has multiple-choice test items where no correct answers are offered as options. Fewer additional supports (steps, formulas, pictures, application, calculators, etc.) are given in the initial activity than the final activity. Regular grade-level text is used in the initial activity, but summarized text with picture cues and key details highlighted are used in the final activity. No explanation of what to do is given for the initial activity, but directions are given for the final activity. Note: directions should ALWAYS be given during an activity. No prompts on the first piece of evidence with no answers written, or all answers are clearly off topic (answers are not at all related to concept). The initial activity is not in the student’s mode of communication, but the final activity is. NJ APA Administrator Training

Common Mistakes found in 2011 - 2012 Assessments
Assessing the skills contained in the CPI Link without connection to the critical essence, intent or big idea of the content and concepts of CPI, Strand, and/or Standard Assessing the skills contained within the CPI Link without correct understanding of the Content or concepts of CPI, Strand, and/or Standard Assessing more than one CPI Link in an entry Assessing different parts of a CPI Link in each piece of evidence Assessing only part of the CPI Link NJ APA Administrator Training

Common Mistakes found in 2011 - 2012 Assessments
Failure to mark all test items with accuracy and/or independence/prompt information Failure to give supports on the first activity Failure to give prompts, instead marking student response as “Don’t Know” or “No Response” For assessment, an item may not be marked as “don’t know” or “no response.” Teachers must utilize a prompt(s) in order to assist the student in generating a response. NJ APA Administrator Training

Changes for Districts must use the updated CPI Links when assessing a standard, strand, and CPI for the APA. A few CPI Links were revised to make them more clear. The asterisk following a link is a reminder that there is a sample test item available in the Five Items document. Important CPI Link clarifications are included in the new Content Resource Guide and Five Items document. Be sure to read all of the relevant grade-level materials thoroughly. NJ APA Administrator Training

Content Guide Revision
Some curriculum issues have been identified and must be avoided in the future. Some evidence contained errors in the instruction of the skills. Example: Our Solar System only contains one star, the Sun. Therefore, stars may not be used as a correct answer when asking what is in our Solar System. Example: Teachers are submitting math Symmetry work sheets that don’t correctly assess the CPI Link that was selected. These topics were addressed in last year’s document. These errors will result in zero scores for See Teacher Training Module 4. NJ APA Administrator Training

Developing Standards-Based Entries
NJ APA Administrator Training 85

Developing an Entry Step 1: Select a CPI and one related CPI Link to be assessed. Step 2: Plan instruction and assessment relating to the CPI and CPI Link. Step 3: Design activities that will be used to assess the CPI Link. Step 4: Assess the student to get an initial piece of evidence for APA purposes. Step 5: Implement instruction. Ensure instruction reflects the essence of the CPI, strand, and standard. Ensure the instruction is age- and grade-level appropriate. Retain a working folder of instructional activities and classroom-based assessments implemented between the activities that generate the initial and final pieces of APA assessment evidence. Step 6: Determine when evidence can be collected to document the final instructional assessment of the CPI Link for APA purposes. Step 7: Based on the student’s accuracy score and level of prompt information on the “final” activity, determine if additional instruction and collection of evidence needs to occur for the entry. Step 8: Review APA evidence to ensure that all information related to test design requirements are included. NJ APA Administrator Training

How is the APA Scored? Each portfolio entry is scored based on specific scoring criteria described in the APA scoring rubric found in the Procedures Manual. The criteria are grouped into three categories called dimensions. NJ APA Administrator Training

APA Scoring Rubric Scoring Dimensions Complexity Link Performance
Match, near, or far link selection Performance Percentage of accuracy when performing skill Independence Percentage of time skill was performed independently NJ APA Administrator Training 88

Scoring Dimensions Dimension 1 2 3 4 Complexity
1 2 3 4 Complexity Evidence provided is unscorable; all dimensions will receive a score of zero CPI Link was assessed, but there are major flaws in the evidence CPI Link is a Far Link to the grade-level indicator CPI Link is a Near Link to the grade-level indicator CPI Link is a Matched Link to the grade-level indicator Performance Evidence is not clear or all items are not marked as correct/incorrect Accuracy of work is 0-39% based on the final activity OR Second activity includes more intrusive prompt Accuracy of work is 40-59% based on the final activity 60-80% based on the final activity 81-100% based on the final activity Independence Evidence is not clear or all items are not marked for Independence/ prompt level Student completed items/tasks independently 0-39% of the time based on the final activity Student completed items/tasks independently 40-59% of the time based on the final activity Student completed items/tasks independently 60-80% of the time based on the final activity Student completed items/tasks independently % of the time based on the final activity NJ APA Administrator Training

Scoring Rubric and APA Forms
The APA Scoring Rubric found on the previous slide is also found in the APA Procedures Manual. Don’t forget to review the Procedures Manual, which includes the APA Scoring Rubric, scoring clarifications, and other topics. NJ APA Administrator Training

Complexity Dimension Evaluates the CPI Link assessed and how closely the complexity and difficulty (Matched, Near, Far) links to the Core Curriculum Content Standards (CCCS) and grade-level cumulative progress indicators (CPI). NJ APA Administrator Training 91

Complexity Dimension 1 2 3 4 Evidence provided is unscorable; all dimensions will receive a score of zero CPI Link was assessed, but there are major flaws in the evidence CPI Link is a Far Link to the grade-level indicator CPI Link is a Near Link to the grade-level indicator CPI Link is a Matched Link to the grade-level indicator NJ APA Administrator Training

Complexity Score of 2, 3, or 4 A score of 2, 3, or 4 is obtained if the CPI Link is assessed correctly and a different activity is used for the initial and final piece of evidence. A score of 2 is obtained when a Far Link is assessed correctly. A score of 3 is obtained when a Near Link is assessed correctly. A score of 4 is obtained when a Matched Link is assessed correctly. NJ APA Administrator Training

What Makes an Activity Different?
Two activities are required when submitting evidence for an entry. Both of these activities must assess the same CPI Link. The two activities must be different activities in order to address the Universal Scoring Rules. An activity is considered to be different than another when the context of the instruction of the CPI Link is different. An activity is also considered to be different from another when the application of the skill in the CPI Link is different. NJ APA Administrator Training

What Makes an Activity Different?
While the skills assessed and the supports needed to perform the skills remain the same, an activity is considered different: With a different context in which the framework or events within which the student is demonstrating the skill changes With a different application in which the types of questions used, or the way in which the student demonstrates the skill changes NJ APA Administrator Training

Examples of two activities with different context:
What Makes an Activity Different? Examples of two activities with different context: The student is working on following vertex-edge graphs after whole class introduction and demonstration of vertex-edge graphs, the student completes a worksheet after having a lesson on flight plans, the student uses a vertex edge graph to follow flight paths The student is working on adding two matrices by completing a worksheet by completing a different worksheet with different problems than used in the initial activity The student is working on writing a cover letter after researching a job of a particular interest, write a cover letter that would be used when applying for the job after listening to speakers about different careers, write a cover letter for a job of interest The student is working on identifying theme in a given work of fiction by reading The Outsiders and answering multiple-choice questions by reading Flowers for Algernon and answering multiple-choice questions NJ APA Administrator Training

Examples of two activities with different applications:
What Makes an Activity Different? Examples of two activities with different applications: The student is working on classifying types of planets by using a graphic organizer by matching The student is working on addition and subtraction by completing a worksheet with problems on it by using a Smart Board to complete math problems The student is working on writing an essay by using a computer or word processor by using paper and pencil The student is working on identifying theme in a given work of fiction by reading a passage of The Outsiders and then answering multiple-choice questions by reading a passage of The Outsiders and then answering short answer questions NJ APA Administrator Training

Teachers Must Think about the Two Activities in Conjunction
As you plan for instruction, you should not think about the initial and final activity in isolation. Things to think about: What type of activities might be easier and harder for the student? Do not use the easier activity as the final activity. What type of prompt, if any, might the student need on the final activity? Do not use a more intrusive prompt level on the final activity than was used on the initial activity. How can I keep the student engaged in the various activities? How will the initial activity be different than the final activity and still engage the student? NJ APA Administrator Training

Complexity Score of 1: Major Flaws
An entry that demonstrates work in a CPI Link but has a major flaw will score a 1 in Complexity. A major flaw includes 1. Assessing the same activity for both pieces of evidence 2. Assessing only part of the CPI Link This may be found in the work sample, and both pieces of evidence must be assessing the same part of the link. This may also be found in the writing rubric. NJ APA Administrator Training

Complexity Major Flaw Example (Same Activity)
9/10/12 2/11/13 These activities use the same graphic organizer and the same cut and paste/matching application of the skill. Therefore, they are not considered different activities, which is a major flaw. One change that would make these activities different would be to use two different graphic organizers, which is considered a different application of the skill. Another way to make the activities different would be to use a different instructional context within each of the activities. For instance, for the initial activity, the student may have watched a film about what different animals eat and then completed notes from the film, using the graphic organizer to demonstrate performance of the skill. For the final activity, the student may have reviewed internet research on what different animals eat and then completed the graphic organizer to demonstrate performance of the skill. So, an entry containing this major flaw would score a 1 for Complexity. The scores for Performance and Independence would be determined based on their respective criteria using the APA Scoring Rubric. The directions given to the student were “Match herbivores, carnivores, and omnivores to the type of food they eat.” NJ APA Administrator Training

Complexity Major Flaw Example (Activity assesses only part of a link)
3.1.8G1 Matched Link: “After reading newspapers, magazines, or electronic texts, identify the type of propaganda used and explain its purpose” 1/10/13 10/1/12 The evidence in this entry does not demonstrate the student working on the full CPI Link, only part of the link. The evidence only assesses the student explaining the purpose of the propaganda type; it does not assess identifying the type of propaganda. The student is only completing part of the CPI Link. Notice that the same part of the Link (explain only) is assessed in both pieces of evidence; therefore, this entry will score a 1 in Complexity. The scores for Performance and Independence would be determined based on their respective criteria using the APA Scoring Rubric. Complexity score would be 1 for only assessing part of the link. NJ APA Administrator Training

Complexity Score of Zero
When the Universal Scoring Rules are not adhered to, the entry will receive a score of zero for Complexity, Performance, and Independence. Please review Teacher Training Module VII: Universal Scoring Rules for more information. NJ APA Administrator Training

Performance Dimension
This dimension measures the student’s accuracy performing the skills represented in the CPI Links identified within the portfolio. NJ APA Administrator Training

Each item must be marked as correct (+) or incorrect (–). The score for accuracy should be expressed as a percentage. Any item for which the student received a physical prompt should be marked as incorrect (–P). The second piece of evidence should not include a more intrusive prompt than was used on the first piece of evidence. The student’s accuracy score on the first piece of evidence should not exceed 39%. The first piece of evidence should not be more difficult than the second piece of evidence. When assessing a 3.2 Writing Link that requires a writing rubric, there should be feedback on the student’s work explaining how/why the work was scored in relation to the writing rubric. NJ APA Administrator Training

1 2 3 4 Evidence is not clear or all items are not marked as correct/incorrect Accuracy of work is 0-39% based on the last activity OR Second activity includes more intrusive prompt 40-59% based on the last activity 60-80% based on the last activity Accuracy of work is 81-100% based on the last activity NJ APA Administrator Training

Scoring Student Evidence for Accuracy
Each item must be marked as correct (+) or incorrect (–). The score for accuracy should be expressed as a percentage. Scoring the activity for accuracy requires a consistent understanding of when to mark an answer right or wrong. Certainly, if the student performed the skill independently, the answer is either correct or incorrect. But what about when the student receives a prompt? How do you score the item correct or incorrect? NJ APA Administrator Training

Understanding the difference between the following provisions will help ensure that the scoring information on the evidence is accurate: providing task directions providing supports providing indirect prompts (verbal, model, and gestural) providing physical prompts providing the answer (directly prompting the student using a verbal, model, or gestural prompt to correctly answer the question) NJ APA Administrator Training

If the student performs the item independently, then Score as correct (+) any item the student performs correctly. Score as incorrect (–) any item the student performs incorrectly. Any system other than +/– must include a key. If you used a different scoring system, such as ✗ equals incorrect and ✓ equals correct, you must explain that clearly on the evidence. NJ APA Administrator Training

If the student received an indirect verbal, gestural, or model prompt when performing the skill, then Score as correct (+) any item the student answered correctly. Score as incorrect (–) any item the student answered incorrectly. Remember to mark next to each item the prompt level used: V, G, or M (e.g., +V means that an indirect verbal prompt was given and that the student answered the item correctly.) NJ APA Administrator Training

Any item for which the student received a physical prompt should be marked as incorrect (–P). A physical prompt is a direct prompt whereby the teacher physically guides the student to the correct answer (e.g., hand over hand). However, if the student receives a physical prompt when performing the skill, then Score the item as incorrect (–P). NJ APA Administrator Training

# items correct divided by # of total items times 100 equals % Correct
Scoring Student Evidence for Accuracy Total accuracy score must be in the form of a percent. The number of items correct divided by the total number of items multiplied by 100 equals the total accuracy score. Accuracy # items correct divided by # of total items times 100 equals % Correct ____/_____ x 100 = ______% NJ APA Administrator Training

Performance • The second piece of evidence should not include a more intrusive prompt than was used on the first piece of evidence. Prompting can affect the reliability of the performance score if not implemented correctly. Therefore, it is important to adhere to the prompt level hierarchy described in your Procedures Manual. If the second piece of evidence includes a more intrusive prompt level than the first piece of evidence, a score of 1 will be assigned for the Performance dimension. NJ APA Administrator Training

Scoring Performance - Non-Example
2-1-13 100% Accurate 80% Independent NJ APA Administrator Training

Performance The initial piece of evidence should not be more difficult than the final piece of evidence. If a support (e.g., a calculator or formula) is provided in the final piece of evidence, it must also be provided in the initial piece of evidence. Otherwise, the initial piece of evidence would be more difficult. Supports provided in the initial piece of evidence do not have to be present in the final piece; they can be faded if no longer needed by the student. The final piece of evidence can be of the same difficulty level as the initial piece of evidence, or it can be more difficult. The student should be given enough task directions, supports, and prompting (if needed) on the initial piece of evidence to elicit on-topic responses, even if incorrect. NJ APA Administrator Training

Busy, Overly Complicated
Sandy 4.3.12B2: Near Link: “Given a graph of a linear function, identify the x- and y- intercept” Problems: the coordinate grid isn’t labeled, the answer choices indicate the grid was intended to be marked by 2’s, the colored lines make the graph jumbled, and use of colored pencils makes the task overly complicated. NJ APA Administrator Training

Make It Clear This activity clearly and simply assesses the link.
Sandy Date: 100% Independent 4.3.12B2: Near Link: “Given a graph of a linear function, identify the x- and y- intercept” This activity clearly and simply assesses the link. NJ APA Administrator Training

Performance • When assessing a 3.2 Writing Link that requires a writing rubric, there should be feedback on the student’s work explaining how the work was scored in relation to the writing rubric. If the student did not earn all of the points for a particular dimension on the writing rubric, there should be feedback on the student’s work explaining why/how the work was insufficient or incorrect. Even when a student earns all of the points allotted on a writing rubric, there should still be feedback affirming the student’s work or explaining why it was particularly acceptable in accordance with the writing rubric. NJ APA Administrator Training

Performance Writing Feedback
Non-example: Evidence does not include any feedback that corresponds to writing rubric. 10/28/12 3.2.8B2 Near: Write a short story that includes a setting, a plot, and characters rubric 10/28/12 Amanda T. Larissa NJ APA Administrator Training

Independence Dimension
Independence evaluates the extent to which the student completed items/tasks independently. NJ APA Administrator Training

Supports vs. Prompts Not all direction is considered a prompt. Some direction is simply a support, which does not need to be indicated, tracked, or part of the score. Examples of supports: Redirecting the student to stay on task is not a prompt; it is a support. Reading directions with the student is a support. Restating the directions for the student is a support. Scribing for the student is a support. Providing accessible materials (large print, textured paper, manipulatives, etc.) is a support. Supports are not included when calculating the independence score for the student’s work. NJ APA Administrator Training

Examples of Supports for Mathematics
Use of manipulatives or pictures Use of a calculator Task-analyzed skills Number line Stamps to write numbers Word problems read to student Enlarged text Chunked problems These are only some of the possible supports available for students. Certainly, there are many other options for providing academic support. NJ APA Administrator Training

Examples of Supports for Reading
Student reads by Following along with the text Following along with picture symbols Following along with objects Following along with tactile cues Adapted text Shortened text Chunked information Large font Highlight to emphasize important information Remember, listening is not reading. NJ APA Administrator Training

Examples of Supports for Writing
Stamps Graphic organizers Write using pictures or objects Write by completing cloze sentences Scribing (work must use the student’s communication mode) Speech-to-text writing programs Large writing utensils or pencil grips Raised lined paper Again, these are only some of the examples, not an inclusive list of the only possible supports available for our students. NJ APA Administrator Training

Examples of General Supports
Reading initial task directions to the student Redirecting the student to stay on task “you need to finish math now” If a student requires supports, he/she must be given supports for both the first and second piece of evidence. Any supports a student uses on a daily basis must be provided to the student throughout instruction, including the initial and final collection of evidence NJ APA Administrator Training

Independence Dimension
1 2 3 4 Evidence is not clear or all items are not marked for Independence/ prompt level Student completed items/tasks independently 0-39% of the time, based on the final activity Student completed items/tasks independently 40-59% of the time, based on the final activity Student completed items/tasks independently 60-80% of the time, based on the final activity Student completed items/tasks independently % of the time, based on the final activity NJ APA Administrator Training

Teacher Scoring for Independence
A prompt level must be documented next to each item performed by the student. I = independent V= verbal G= gestural M= model P= physical If some other system is used there must be a key. Remember, if your prompt hierarchy is not the same as the one above, you must explain your prompt hierarchy clearly, giving the names of the prompts, hierarchical order, and letter key for the prompts. Even if the student performs with 100% independence, each item must be marked with an I. Independence scores must be summarized as a percent. Reviewers must be able to recreate the score. If the score is not replicable, the dimension can not be scored and will receive zero for Independence. NJ APA Administrator Training

Prompting During Assessment- Allowable Prompts
An indirect verbal prompt (V) can provide the student with a clue to try and spark the student’s recollection of the activity or lesson so that he/she can respond to the question (e.g., “Who was the story about?” or “Remember, the main character has red hair and pigtails. Point to the main character.”) An indirect gestural prompt (G) can Provide the student with a clue as to the general location of an answer (e.g., when looking up a word in the dictionary, the teacher may tap the page on which the word can be found but not exactly where the word is on the page) NJ APA Administrator Training

Prompting During Assessment - Allowable Prompts
An indirect model prompt (M) can provide the student with a clue through demonstration of the skill (e.g., demonstrate how to carry in an addition problem then give the student a different problem) Provide the student with a clue through acting out a scenario (e.g., when presenting a choice of three pictures and asking the student which picture represents an unbalanced force, the teacher may make a sweeping or moving motion to represent an “unbalanced force”) NJ APA Administrator Training

Direct Prompting During Assessment - NOT ALLOWED
A direct verbal prompt provides the student with the specific answer to a question or item (e.g., “Remember, the main character was Pippi. Point to the picture of the main character.”) A direct gestural prompt points out the specific answer to the student (e.g., when presenting a choice of three pictures and asking the student which picture represents an unbalanced force, the teacher points to or taps the correct picture) A direct model prompt models the exact problem and answer that the student must perform (e.g., when sorting producers and consumers, the teacher says, “Remember, corn is a producer,” picks up the picture of corn, and places it in the producer column of a chart then asks, “Which one is a producer?”) NJ APA Administrator Training

Prompting During Assessment - Allowable Prompts
A physical prompt (P) requires physical contact between the student and teacher may be a touch at the elbow prompting the student to move his/her hand to a particular answer, or it may be more invasive, involving the teacher physically moving the student’s hand to the correct answer If the student must be given a physical prompt to correctly answer the test question, the answer must be marked wrong. This applies to full and partial physical prompts. NJ APA Administrator Training

Scoring Evidence for Independence
The total Independence score must be in the form of a percent. Take the number of items completed independently divided by the total number of items multiplied by 100 to calculate the total Independence score. When scoring for Independence, the information does not include accuracy. Independence # items performed independently divided by # of total items times 100 equals % Correct ____/_____ x 100 = ______% NJ APA Administrator Training

Language Arts Literacy
Sample Entry From Albert NJ APA Administrator Training

Language Arts Literacy Entry - Example
Notice that this Entry Cover Sheet indicates that this is LAL entry 1. The required grade-level standard, strand, and CPI to be assessed are documented; 3.1 represents reading; 4 represents the student’s assigned grade, F represents the strand, and 2 represents the specific CPI. The optional box was also completed. Next, the specific CPI Link to be assessed is documented. This must be recorded on the line labeled CPI Link. And the level of the link as indicated in the CPI Links document is circled. The bottom portion of the Entry Cover Sheet is used to describe the activities from which the initial and final pieces of evidence are collected. Take a minute to review the Entry Cover Sheet. NJ APA Administrator Training 1

Notice that the first piece of evidence assesses the entire CPI Link: After reading a passage, identify the meaning of unknown words using context clues. The student reads the text and then circles the correct definition for the underlined/unknown word. Notice the text is a three-sentence passage or paragraph and includes enough information to provide context for the unknown word—such as in item 1, the word “unproductive” is described earlier in the passage through examples of how Joe is unproductive (falling asleep a few times). The text is appropriate to the student’s grade-level and the CPI being assessed. Information about text can be found in Module VII of the training and in the Content Guide and Five Items Resource documents. NJ APA Administrator Training

2a Here is the second page of Albert’s first piece of evidence. Often, evidence is more than one page. Here, the teacher has numbered the pages 2, 2a, and 2b, as you’ll see on the next slide. NJ APA Administrator Training

2b Notice that there are 5 items across the 3 pages of this piece of evidence. Notice also that it is in the student’s mode of communication, each item is marked as correct or incorrect, and each item is marked as independent or prompted with the specific prompt documented. The accuracy and independence scores are recorded on the student’s work (page 2), and the initial accuracy score is below 39%. And of course the student’s name and the complete date are documented on the evidence as well. Review the Universal Scoring Rules in relation to this piece of evidence. NJ APA Administrator Training

In this activity, the student has a word bank to choose the correct definition for the underline word. Again, the context clues within the passage provide enough information that the student can use the passage to figure out the meaning of the underlined word. NJ APA Administrator Training

3a And again, this is a piece of evidence that has multiple pages.
NJ APA Administrator Training

Here is the last page of the evidence
Here is the last page of the evidence. So, there were five items, each item was marked for accuracy and independence, and all of the other Universal Scoring Rules have been met. 3b NJ APA Administrator Training

Language Arts Literacy Evidence Index – Sample Entry for Albert 2011-2012
4th Grade LAL Entry Entry 1 Standard Strand F CPI 2 Matched Link : “After reading a passage, identify the meaning of unknown words using context cues” Rubric Dimension Score Evidence Complexity 4 Pgs. 2-2b and 3-3b show the student working on the complete CPI Link from the “Matched” column Pgs. 3-3b (evidence 2) are a different activity than pgs. 2-2b (evidence 1) Different context: different passages and words assessed Different application: multiple choice vs. fill in the blank Performance 3 Pgs. 2-2b student scored 20% (which is below 39%) on initial activity Pgs. 3-3b student scored 80% Each item is clearly marked Pgs. 3-3b do not include a more intrusive prompt than pgs. 2-2b Independence Pgs. 3-3b student scored 80% Independence This index is just to help you understand why this entry scored as it did. You do not need to include it in your portfolio. It is for training purposes only. Review this index for each of the scoring dimensions: Complexity, Performance, and Independence. NJ APA Administrator Training

Professional and Ethical Responsibility
The Alternate Proficiency Assessment is intended to assess the academic progress of students with significant cognitive disabilities toward the standards established by the state of New Jersey. The format of the APA requires teachers administering this assessment to develop assessment activities appropriate for each participating student and to provide appropriate documentation of how the activities were used to assess students on what they gained from instruction. Instruction is the key. Assessments should never be used as the instruction. Instruction should take place first, and then assessments should be designed to reflect what was taught. The first piece of APA evidence is based on the initial introduction of the skill, which usually occurs prior to implementing extensive instruction. NJ APA Administrator Training

Teachers are to follow the steps for the development of an entry described in the teacher training materials. Teachers are to save not only copies of the assessments that are being included in the student’s portfolio but also copies of student work that, while not included in the portfolio, provide further proof that the student received instruction and was given opportunities to practice and show what he/she was taught prior to the final assessment activity. These additional student work activities may be reviewed during a site visit or required to be presented for a security breach visit. NJ APA Administrator Training

Each assessment presented to the student should be unique, meaning that the student should not have previously seen the assessment. The following practices are not allowed: Teaching students to memorize answers for an assessment. Giving students the same assessment (activity) over and over until they complete the assessment satisfactorily. Allowing students to see the assessment and its test questions ahead of time. Allowing someone else to complete work for a student. Remember, there are guidelines for scribing that must be followed if a student needs assistance documenting their answer. Giving an assessment when no instruction has occurred. Giving the students a writing task response and having them copy it and submitting it as their own work. Leaving the date blank on a completed activity and adding a date later. Changing the date on a completed activity. Any other practice that results in misleading information regarding what the student was taught and has learned. NJ APA Administrator Training

Some students may not be able to complete an entire assessment activity all at once. It is acceptable to administer assessment items one or two at a time, as appropriate for the student, but when doing so, the student should only be presented with the item or items that he/she will be expected to complete at that time. It is acceptable to mask the remaining items on the page until they are completed by the student. If a student becomes sick during an assessment or cannot complete an assessment at the time it is given, that assessment should be used as practice and a new assessment should be given at a better time. Remember that the APA has two rather long testing windows that should help address most special testing conditions that may arise for students participating in the APA. NJ APA Administrator Training

It is the professional and ethical responsibility of all contributors to a student’s portfolio to ensure that any and all documentation reflect authentic, accurate, and truthful information. Reminder, it is not appropriate to manipulate the date the evidence was created, the prompt/independence levels, the student’s work, or the scoring percentages. Include only evidence that reflects the student’s actual performance, even if the student did not make as much progress as you expected. Follow all APA guidelines when administering the assessment. NJ APA Administrator Training

Any student portfolio that is found to contain inauthentic documentation may result in professional consequences for school and district staff. All contributors to a portfolio must sign and submit the assessment security form verifying that the APA was conducted properly and in accordance with all directions and training materials. District administrators will provide this form. This is a new policy, and is in line with the general assessment procedures. It is important that administrators review the teacher training modules to ensure you know what is acceptable, and not acceptable, regarding APA assessment. Remember that this is an individual assessment, not a group assessment. NJ APA Administrator Training

Test Security Agreement
Give the teachers the agreement this fall so that they can see what they will sign and submit when they turn in the portfolio. Every teacher that produces APA evidence must submit a signed Test Security Agreement to be placed in the binder. If teachers worked on only one subject, they may write that on the agreement. NJ APA Administrators Training

Status of 2011 - 2012 Portfolios and Score Reports
Pearson Scoring Center staff scored the portfolios during April and May. Score reports were shipped in June. Most portfolios have been returned. Portfolios designated as security breaches (Void 5) are in process. Information regarding the breach consequences will be forthcoming. Please contact me to arrange a meeting to discuss the district’s security breach portfolios. NJ APA Administrator Training

Contact Information For information on interpreting the APA score reports, please review the Score Interpretation Manual on Pearson’s website For missing score reports please contact: NJ Customer Service Team – For score report interpretation please contact: Jo-Lin Liang – NJ APA Administrator Training 149

What is Proficient? New cut scores were set in June 2009 which determine how the total score for each subject area is classified. Total scores may be classified as advanced proficient, proficient, or partially proficient. The cut scores were determined based on the grade-level linked skills and content being assessed; therefore, the same number of total points may be classified as advanced proficient in one subject area for a grade level, and classified as proficient for a different grade level, due to differences in the CPI Links and developmental demands of students in different grades. This mirrors the general assessments where each grade level and subject area cut scores are determined independently. A group of 86 New Jersey educators, including general and special education teachers, child study team members, and other administrators, participated in a four day standard setting process. These educators reviewed portfolios, the CPI Links, educator produced proficiency level descriptors, and other related documents. After instruction on the process of determining classification of portfolios based on content, scoring dimensions, and total scores, the committee drafted and then finalized the proposed range of total scores for each content and grade level. Senior staff and the state board of education then reviewed, revised, and approved the results. NJ APA Administrator Training

How is a Total Score Calculated?
The scoring formula was determined by reflecting on the purpose of the APA, which is to: assess the district and school grade-level instructional program provided to the student, assess a student’s attainment of knowledge and skills of the CCCS, and encourage high but appropriate instructional standards for all students, while providing necessary supports to access the curriculum and foster the highest level of independence possible for the student. The scoring formula uses the three dimension scores that each entry receives: Complexity Independence Performance NJ APA Administrator Training

The total score for each content area is calculated by adding the scores for all three dimensions for each of the four entries. Each entry has a minimum possible score of zero and a maximum possible score of 16 points. The total score for each subject area has a minimum score of 0 and a maximum score of 64 points. NJ APA Administrator Training

Each scorable entry will receive a score of 0 – 4 for each dimension. Unscorable entries receive a zero for each of the three dimensions. Each entry is scored by a minimum of two people (readers). First person is reader one (R1) and second person is reader two (R2). The performance dimension score for any entry is the sum of the two reader scores (R1+R2). The addition of the two reader scores gives more weight to this dimension in the overall total score than the other two dimensions. The complexity dimension score for any entry is the average of R1 & R2. The independence dimension score for any entry is the average of R1 & R2. NJ APA Administrator Training

Accountability APA test results will be combined with the results from the general assessment for accountability purposes for state and federal reports. For accountability purposes, the APA is both a student assessment, and a school/district program assessment. APA test results are included in the Final Accountability report. NJ APA Administrator Training

Federal Results – 1% Cap The NCLB 1% cap limits the number of APA proficient and advanced proficient results that can be used in the accountability calculation. It does not limit the number of students who may participate in the APA. Districts legitimately testing more than 1% of the total grade population with the APA should apply for a waiver from the Office of Special Education. The waiver, if approved, allows proficient and advanced proficient results greater than 1% to be used in the accountability calculations. This form is usually mailed by the department to districts in the winter. NJ APA Administrator Training

Accountability Reports
Student data is collected on the APA scan sheet. This data is submitted to districts for verification. All requested revisions to student data must be made during the record change process. No changes may be submitted after this time frame. Accountability reports are produced based on the student data on file as of the close of the record change process. Reports for sending districts include students assessed out of district, out of state, and in district. NJ APA Administrator Training

Information – Title I Office
Up-to-date information on the calculation of Accountability Reports can be obtained on the Department of Education’s website Review the information provided from the Office of Title I. The Accountability Workbook provides specific details. If you have questions after reviewing the online information, please call the Office of Title I at NJ APA Administrator Training

Main Responsibilities
Complete scan sheet materials in February 2013. Work directly with administrators from the child’s sending or receiving school to ensure accurate information. Errors on the scan sheet may affect your Accountability Report calculations. Several fields on the scan sheet must be completed in February since they are time sensitive and can not be completed during the Pre-ID process. A few fields on the scan sheet may need to be completed by the student’s teacher. Collect portfolios from teachers by end of day on February 15, 2013. Administrators may choose to collect them earlier in order to complete portfolio review process when high number of portfolios are produced. Review final student data and submit record change rosters. This is the last opportunity to make data changes. NJ APA Administrator Training

Final Administrator Review
Test Design Review Did the evidence address the required components? Prescribed standards and grade-level CPIs Correct amount of evidence Meets the Universal Scoring Rules Conducting this review periodically will allow you to find and correct errors during the collection period. Security Breach Review Is white out being used to change information? Were prompt levels changed to independent performance? Were dates on the evidence changed? Were wrong answers changed to correct answers? Were wrong answers marked correct? Are the percentage scores accurate? Are there other indications of inauthentic information? NJ APA Administrator Training

Final Administrator Review
Review portfolios for accuracy and authenticity by February 22, 2013. Review the portfolios in light of the student’s assigned grade level. Review the portfolios in light of the current CPI Links. Review the portfolios in light of the test design requirements. Review the portfolio to ensure all evidence is authentic. Reminder that portfolios should be reviewed multiple times within each collection period. Doing so helps you identify errors early, giving enough time to produce new evidence without errors prior to the end of the collection period. NJ APA Administrator Training 160

Shipment of Portfolios
The shipment process will be communicated to districts at a later time. Portfolios must be shipped between February 26 and March 11, Any portfolio shipment dated after March 11 will not be accepted for scoring. Portfolios must be submitted on time in order to be processed. Shipping materials and instructions will arrive in the Chief School Administrator’s office in February NJ APA Administrator Training 161

Administrator/Coordinator Activities
General Responsibilities When information is sent to you, pass it on to district and school administrators and teachers. Review the time lines and request APA information from your chief school administrator when it is scheduled to be delivered. Review participation guidelines carefully. Ensure that all students who need to be assessed are identified (both in-district and out-of-district). NJ APA Administrator Training

General Responsibilities (continued)
Administrator/Coordinator Activities General Responsibilities (continued) Review school educational programs and ensure alignment with dimensions of the scoring rubric. Ensure that the APA teachers participate in the online teacher training sessions. Review the APA Procedures Manual. Pass along questions and concerns to NJ DOE or contractor. NJ APA Administrator Training

Administrator/Coordinator Checklist
Before Assessment Identify a School Coordinator for each school. Register number of students participating in APA and update mailing addresses using Material Survey no later than Date to Be Determined. Create a teacher training schedule for your school, identify teachers required to participate, and implement training. Ensure all materials are downloaded prior to the selected training day(s). Have a sign in sheet for all teachers to verify attendance and completion of the training. Utilize all APA training materials, including the Content Resource Guide and Five Item documents, in addition to the Teacher Training Modules and Procedures Manual. Distribute materials to schools/teachers. Note NJ APA Administrator Training

During Assessment Distribute materials to schools/teachers. Binders and manuals will be shipped to district/receiving school. Copies of the APA training materials may be downloaded as of September 7, 2012, from the Pearson’s website Click on Documentation, then Procedures Manual. Content Resource Guide and Five Item Sample documents may not be available until September 19, 2012. NJ APA Administrator Training

During Assessment (continued)
Administrator/Coordinator Checklist During Assessment (continued) Gather information on students’ sending/receiving status and collect student demographic info. Work with the receiving/sending school to confirm student data. Check periodically that teachers are working on the APA. NJ APA Administrator Training

After Assessment Notify teachers of timeframe for returning the completed APA portfolios. Collect and review all APA portfolios for accuracy and authenticity of information and inclusion of all required components. Be sure to review the portfolios for adherence to the universal scoring rules. Review the scan sheet information to ensure accuracy of all data. NJ APA Administrator Training

After Assessment (continued)
Administrator/Coordinator Checklist After Assessment (continued) Package the APA materials securely and label them according to instructions. Contact the designated carrier for return of the portfolios to arrange pickup of the materials. Ensure all portfolios are submitted by March 11, deadline. Portfolios not received by shipping carrier by March 11, 2013 will not be accepted for scoring. NJ APA Administrator Training

APA Teacher Involvement
Teachers are needed in March 2013 for a week to score portfolios. Send an to Elizabeth Celentano if someone is interested in assisting in the scoring process. address is Summer committees will be formed to assist in the test specification revision for alignment to the general assessments and Common Core Standards. Teachers and curriculum specialists will be needed to complete this work. More information will be forthcoming in Spring 2013. NJ APA Administrator Training

Dynamic Learning Maps New Jersey is one of 13 states collaborating with the University of Kansas on the future version of an alternate assessment for students with the most significant cognitive disabilities. Opportunities arise for districts to participate in the process via document reviews, meetings, and implementing pilot components. If you would like to become involved please Elizabeth Celentano. This revised assessment is expected to be implemented in some form (pilot?, operational?) in As information becomes available it will be disseminated. NJ APA Administrator Training

Where To Get More Information?
Educational Links, Information, and Resources on Assistive Technology and Universal Design for Learning, an excellent reference document for teachers and administrators can be downloaded from the DOE website NJ APA Administrator Training

Other educational materials and resources can be accessed through the New Jersey Learning Resource Center Network website: NJ APA Administrator Training

Elizabeth Celentano (test design, training, test administration, etc.) Office of Assessments New Jersey Department of Education Jo-Lin Liang (student data, Pre-ID, record changes, score interpretation, etc.) Office of Special Education (general special education questions) Department of Education website: NJ APA Administrator Training

Where to Get More Information?
Topics: Training materials, score reports , portfolio binder return Pearson - Bruce Murphy Customer Service Phone: Pearson website: Topics: Explanation and Error Sheet issues, Evidence questions (through November 2012) ILSSA – Lou Ann Land Alson Cole NJ APA Administrator Training

Where to Get More Information?
Topics: Student registration, Pre-ID, scan sheets, portfolio shipments, record changes, online entry cover sheets, etc. Questar Assessment, Inc. NJ APA Customer Support: Teri Broderick Phone: NJ APA Website address: NJ APA Administrator Training

Winter 2013 Training Materials
In January 2013 a new Power Point presentation will be uploaded for your review. You will be notified when it is posted and on which website. It will include reminders about key concepts provided today as well as information about necessary tasks related to returning the portfolios. Please make sure you read these materials and contact the Office of Assessments if you have questions. NJ APA Administrator Training

Thank You Thank you for taking time to attend this session. If you have not had all of your questions answered, please see one of the presenters for more information. NJ APA Administrator Training 177

Overview of the Alternate Proficiency Assessment

Similar presentations

Presentation on theme: "Overview of the Alternate Proficiency Assessment"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Overview of the Alternate Proficiency Assessment

Similar presentations

Presentation on theme: "Overview of the Alternate Proficiency Assessment"— Presentation transcript:

Similar presentations

About project

Feedback