BY Karen Liu, Ph. D. Indiana State University August 18,

 NCATE SPA Review Process  Definitions  Ground Rules  Response to Condition Report  NCATE Course Grade Guidelines  NCATE Course Grade Format  Evidence for Meeting Standards  EESE Assessments 1-7 2

 Institutions submit program review documents to NCATE.  NCATE assigns each institution’s program report to 2-3 reviewers. (One institution may submit several program reports.)  Each reviewer will provide an independent review and submit the report to NCATE.  The lead reviewer will provide a team report.  The audit committee will provide the audit report.  A staff member from NCATE will review the audit report and provide a final review.  The result of the final program review report will be sent to each institution. 3

 Revised Report  Submitted when the previous decision required further development or national recognition with probation  Response to Conditions Report  Submitted when the previous decision was national recognition with conditions 4

 Program must submit  Cover Sheet  Section VI: For Revised Reports or Response to Conditions Reports Only  All documentation to support what they’ve said in Section VI  Attachments in appropriate Section of program report (usually in Section IV: Evidence for Meeting Standards) 5

 Subsequent review teams cannot add any new criteria or expectations in subsequent recognition reports  Must reply solely on comments and information in original recognition report 6

 Part G: Decision  Date: The program is recognized through (ISU EESE)  The program has up to two opportunities to address conditions within an 18 month period  ISU EESE (9/15/11, 3/15/12, 9/15/12 or 3/15/13)  Addressing conditions (See the list on National Recognition Report)  Review comments  Part C: Evaluation of Program Report Evidence  Part D: Evaluation of the Use of Assessment Results  Part E: Areas for Consideration 7

1. Make clear connections between ISU's assessments and standards rather than simply repeating the ACEI standards. 2. Provide evidence that candidates are able to use or apply the content embedded in Standards 2.5 to Provide explicit evidence for Assessment 2 that courses align with standards. 4. Provide more in-depth analysis of data. See comments in Part C. 8

Part C: Evaluation of Program Report Evidence  C.1: Candidates’ Knowledge of Content  The report indicates that Assessments 1 and 2 provide evidence of candidate content knowledge. However, more detailed alignment between assessments and standards is needed.  Assessment 2 requires course descriptions to gain a comprehensive perspective of the alignment between individual courses and the identified standards.  If grades in courses are used as an assessment, the program must follow NCATE guidelines for the construction of the assessment: /GuidelinesAndProcedures/DocumentingCourseGra des/tabid/456/Default.aspx 9

C.2. Candidates' ability to understand and apply pedagogical and professional content knowledge, skills, and dispositions  The report indicates that Assessments 3 and 4 meet standards in this area. However, more detailed alignment between assessments and standards is required, rather than simply repeating ACEI standards.  Activities or requirements within assessments must be set off in such a way that each task or requirement is specifically designed to measure competency on a single standard or standard element.  When the assessment and/or rubric is designed to evaluate performance across multiple standards, the resultant data cannot be used as evidence of successful candidate performance on individual standards. 10

C.3. Candidate effects on P-12 student learning  Assessment 5 provides guidelines and rubrics for the TWS to ensure that candidates have an impact on student learning. However, more detailed alignment between assessments and standards is required. 11

This program has provided examples of the use of data and how they have informed program changes, although many of the changes are still in the proposed phase. A few are listed below: (1) proposed addition of phonics handbook or course to ensure candidates are adequately prepared to teach reading (2) adding benchmarks to the Degree Audit Reporting System (DARS) to manage when candidates take Praxis II (3) broaden the field placement options for students to provide a diverse body of pedagogical knowledge to candidates 12

 Be sure to report data on program graduates;  Maintain consistency with the "N" data that is reported;  Map standard alignment to assessment rather than simply repeating the standard;  Provide more explanation in of the alignment, analysis, and interpretation of data and/or assessment;  Provide comprehensive evaluation and interpretation of how standards are met; and  Continue to review and refine the assessment system as needed. 13

 Description of PRAXIS II  Indiana licensure requirement.  The content categories of the test.  Alignment with Standards  Describe how PRAXIS assessment specifically aligns with ACEI standards.  Analysis of Data Findings  What have we done to improve candidates’ test scores?  Evidence for Meeting Standards (Data)  NCATE asks the pass rate of 80%. (165 and higher).  EESE uses “quartile” to report its data. 14

1. Courses must be required for all candidates in the program; elective courses may not be used as evidence. 2. Faculty may choose which courses will be used in this assessment. 3. The documentation of course grades-based evidence must include curriculum requirements, including the course numbers of required courses. (Documentation must be consistent with course listings provided in the Program of Study submitted in Section I of the program report.) 4. The grade evidence must be accompanied by the institution’s grade policy or definitions of grades. 5. Grade data must be disaggregated by program level (e.g. baccalaureate), grade level (e.g. middle grade and secondary), licensure category (e.g. history or social studies), and program site. 6. Syllabi cannot be submitted. 15

Format for Submission of Grades as a Course- Based Content Assessment  Part 1: Description of the Assessment  Give a brief description of the courses and a rationale for the selection of this particular set of courses.  Provide a rationale for how these courses align with specific SPA standards as well as an analysis of grade data included in the submission. (Limit to two pages).  Part 2: Alignment with SPA Standards  Include a matrix that shows alignment of courses with specific SPA standard (Reqiored). 16 Course Name & No.SPA Standard(s) Addressed by Course Brief Description of How the Course Meets Cited Standards (if course title is unclear) ELED 324: Emergent Literacy2. 1, 3.1, 3.4

 Part 3: Grade Policy and Minimum Expectation  Must submit grading policies that are used by the institution or program and the minimum expectation for candidate grades (e.g., all candidates must achieve a C or better in all selected coursework)  Part 4: Sample Data Tables  Data tables must provide, at minimum, the grade distributions and mean course grades for candidates in the selected courses. (See NCATE Course Grade documentation handout) 17

 Assessments are organized into 3 areas to be aligned with ACEI standards.  Content knowledge (Assessment 1 & 2)  Pedagogical and professional knowledge, skills and dispositions (Assessment 3 & 4)  Focus on student learning (Assessment 5)  For each assessment, the compiler should prepare one document which includes the following items: a) A description of the assessment; b) A description of how this assessment specifically aligns with the ACEI standard it is cited; c) A brief analysis of the data findings; and d) An interpretation of how that data provides evidence for meeting standards o The assessment tool/instrument and/or the directions given to candidates. o The scoring guide for the assessment. o The assessment data. 18

 Focus on assessing candidates’ pedagogical and professional knowledge, skills and dispositions (Assessments 3 & 4).  Combine all four separate documents into one document and reorganize the content information presented in the document.  Map standard alignment to assessment rather than simply repeating the standard.  Provide detailed explanation to address how this assessment specifically aligns with the ACEI standard it is cited. 19

 Focus on assessing candidates’ pedagogical and professional knowledge, skills and dispositions (Assessments 3 & 4).  Combine all three separate documents into one document and reorganize the content information presented in the documents.  Give a better description of the assessment.  Provide detailed explanation to address how this assessment specifically aligns with the ACEI standard it is cited.  Offer comprehensive evaluation and interpretation of how standards are met. 20

 Focus on assessing student learning (Assessment 5).  Combine all three separate documents into one document and reorganize the content information presented in the documents.  Provide clear description of the Work Sample assignment (not just copy the information from the course syllabus).  Provide detailed explanation that addresses the alignment of data and/or assessment.  Give comprehensive analysis and interpretation of how standards are met. 21

 Combine 2 separate documents into one.  Provide better description of this assessment (May include the objectives/goals).  Follow the NCATE reporting format. (The document should include: description, alignment with standards, analysis and interpretation).  Provide detailed explanation that addresses the alignment, analysis, and interpretation of data and/or assessment. 22

 Follow the assessment report format: description, alignment with standards, analysis, and interpretation of data.  Make clear connections between assessment #7 and ACEI standards rather than simply repeating the ACEI standards.  Provide comprehensive analysis and interpretation of how standards are met base on the data presented. 23

24