1. Preparing IRA/CAEP Program Reports to Earn National Recognition Dr. Diane Kern, IRA SPA Coordinator CAEP Conference Fall, 2014 1

2. Agenda Introductions Introductions Overview of program report and IRA 2010 Standards for Reading Professionals Overview of program report and IRA 2010 Standards for Reading Professionals Requirements of the assessment system Requirements of the assessment system IRA specific guidelines IRA specific guidelines Literacy coaching tips and coaching continuum Literacy coaching tips and coaching continuum Sample report sections Sample report sections Resources, Contacts and Q & A Resources, Contacts and Q & A 2

3. Overview of program report Institutions must provide evidence of meeting IRA standards based on 6-8 assessments (recommendation: 8 assessments). Institutions must provide evidence of meeting IRA standards based on 6-8 assessments (recommendation: 8 assessments). Assessments and rubrics must align with the IRA 2010 Standards for all three roles of the reading specialist/literacy coach: Assessments and rubrics must align with the IRA 2010 Standards for all three roles of the reading specialist/literacy coach: Interventionist/reading specialist Interventionist/reading specialist Literacy coach Literacy coach Reading program supervisor/leader Reading program supervisor/leader Assessment system must build a case that all standards are met through utilizing the elements and potential sources of evidence = “preponderance of evidence”. Assessment system must build a case that all standards are met through utilizing the elements and potential sources of evidence = “preponderance of evidence”. Program level assessments must be administered to all candidates and should span across the program. Program level assessments must be administered to all candidates and should span across the program. 3

4. Format : What Option? OptionDescription/Standards Option ANCATE Report Format: 6-8 Assessments, Five Designated Assessments: Content (2) including State Licensure Data (if required); Professional and Pedagogical Knowledge (2); P-12 Student Learning (1) Additional Assessments (1-3) / Option BChoose Your Own Assessments: Up to 8 Assessments including 1) State Licensure Data (if required); 2) Impact on Student Learning; 3) Additional Content, Pedagogical Knowledge Assessments 4

5. IRA 2010 Standards for Reading Professionals 1. Foundational knowledge 2. Curriculum and instruction 3. Assessment and evaluation 4. Diversity 5. Literate environment 6. Professional learning and leadership 5

6. IRA specific guidelines Program must incorporate an equivalent of 21-27 graduate semester hours in reading, literacy, language arts, and related courses. Program must incorporate an equivalent of 21-27 graduate semester hours in reading, literacy, language arts, and related courses. Coursework must include a supervised practicum experience – typically the equivalent of 6 semester hours. Coursework must include a supervised practicum experience – typically the equivalent of 6 semester hours. Supervision may include observations, conferences, audiotape or videotape review of lessons, etc. Supervision may include observations, conferences, audiotape or videotape review of lessons, etc. Academic and practicum experiences must develop candidates for all three roles: interventionist, literacy coach, and literacy program leader Academic and practicum experiences must develop candidates for all three roles: interventionist, literacy coach, and literacy program leader 6

7. Literacy Coaching Tips Coaching initiatives should include experiences in the program (such as the reading clinic) AND also opportunities to support teachers and/or paraprofessionals in authentic school settings. Coaching initiatives should include experiences in the program (such as the reading clinic) AND also opportunities to support teachers and/or paraprofessionals in authentic school settings. Institutions should implement a developmentally appropriate continuum of coaching experiences across the program. Institutions should implement a developmentally appropriate continuum of coaching experiences across the program. 7

8. Coaching Activity: 3 Levels of Intensity (Bean, R.M., 2004) 8

9. Sections of the Program Report Cover Sheet Cover Sheet Section I: Context Section I: Context Section II: List of assessments (chart) Section II: List of assessments (chart) Section III: Relationship of assessments to standards (matrix) Section III: Relationship of assessments to standards (matrix) Section IV: Evidence for meeting standards (attached files using CAEP required format) Section IV: Evidence for meeting standards (attached files using CAEP required format) Section V: Use of assessment results to improve program (narrative) Section V: Use of assessment results to improve program (narrative) Section VI: For revised reports or response to conditions reports only Section VI: For revised reports or response to conditions reports only 9

10. Section I: Context Narrative or attachments, including the following: Any state or institutional policies that influence your program? Any state or institutional policies that influence your program? Describe supervised clinical/practicum and other experiences Describe supervised clinical/practicum and other experiences Attach program of study Attach program of study Table with three years of data on candidates enrolled in the program; separate if multiple sites Table with three years of data on candidates enrolled in the program; separate if multiple sites Faculty information Faculty information 10

11. Section II: List of Assessments 11

12. Section III: Matrix 12

13. Section IV: Sample Assessments #1 & #2 Content Knowledge Licensure exams or professional examinations of content knowledge (must be #1 if required by state, institutions must report results) Licensure exams or professional examinations of content knowledge (must be #1 if required by state, institutions must report results) Comprehensive Examinations Comprehensive Examinations Research Reports Research Reports Child Studies/Action Research Child Studies/Action Research Portfolio Tasks (that all candidates perform) Portfolio Tasks (that all candidates perform) Evidence Base for Professional Development Initiatives Evidence Base for Professional Development Initiatives 13

14. Sample Assessment #3 Ability to plan instruction Developing lesson plans and units of study Developing lesson plans and units of study Selecting, using, and evaluating assessment tools Selecting, using, and evaluating assessment tools Selecting, using, and evaluating curriculum, instructional techniques, and appropriate materials Selecting, using, and evaluating curriculum, instructional techniques, and appropriate materials Establishing and evaluating literate environment Establishing and evaluating literate environment Designing and implementing instruction Designing and implementing instruction Portfolio tasks Portfolio tasks Literacy coaching components utilizing assessments above Literacy coaching components utilizing assessments above 14

15. Sample Assessment #4 Assessment of internship, practicum, clinical experience Reflective practitioner observation protocol Reflective practitioner observation protocol Coaching cycle observation, reflection and evaluation by supervisor Coaching cycle observation, reflection and evaluation by supervisor Final school report, parent conference Final school report, parent conference Analysis of video segment with peer review, coaching and supervisor feedback Analysis of video segment with peer review, coaching and supervisor feedback Practicum portfolio Practicum portfolio 15

16. Sample Assessment #5 Impact on Student Learning Case studies Case studies Action research with a pretest-posttest design Action research with a pretest-posttest design Analyzing student work samples Analyzing student work samples Designing and implementing instruction and/or intervention Designing and implementing instruction and/or intervention Portfolio tasks Portfolio tasks Literacy coaching and leadership experiences utilizing assessments above Literacy coaching and leadership experiences utilizing assessments above 16

17. Sample Assessments #6, 7 & 8 Additional Assessments Eight assessments are strongly recommended, although only six are required Diversity, technology, creating literate environment, coaching and leadership experiences are typical areas to further develop Diagnostic portfolio for diverse learners at the elementary and secondary levels Diagnostic portfolio for diverse learners at the elementary and secondary levels Three levels of literacy coaching in the clinic and in school settings Three levels of literacy coaching in the clinic and in school settings Data reports for whole school reading program and Report Night Data reports for whole school reading program and Report Night Supporting teachers to select and evaluate materials and instructional methods to meet Supporting teachers to select and evaluate materials and instructional methods to meet Professional development series, book study leadership Professional development series, book study leadership 17

18. Section IV: Evidence for Meeting Standards Two-page narrative of each assessment includes: a) Brief description of assessment and its use in the program b) Description of how assessment aligns with IRA standards c) Brief analysis of data findings d) Interpretation of how data provide evidence for meeting specific standards by IRA standard number, title, and/or wording. 2) One attachment for each assessment, which includes the assessment tool itself (the detailed directions given to candidates), scoring guide/rubric, and data table disaggregated by TWO applications (and if multiple sites, by each site) 18

19. Section V: Assessments/data to inform program improvement Narrative with no more than 12,000 characters Narrative with no more than 12,000 characters Summarize principal findings from the assessment results, faculty interpretation of data, and how candidate performance informs plans to strengthen the program Summarize principal findings from the assessment results, faculty interpretation of data, and how candidate performance informs plans to strengthen the program Organized by three headings: Organized by three headings: Content knowledge Content knowledge Professional and pedagogical knowledge, skills and dispositions Professional and pedagogical knowledge, skills and dispositions Impact on P-12 student learning Impact on P-12 student learning 19

20. Section VI: For Revised programs only Response to conditions narrative—Organized by the conditions—what changes did you make? Only one application of data required Response to conditions narrative—Organized by the conditions—what changes did you make? Only one application of data required Revised reports narrative: Describe changes or additions have been made to address the IRA Standards Revised reports narrative: Describe changes or additions have been made to address the IRA Standards TIP—Even if IRA did not specifically ask for something, you can include any (all!) sections of the program report to make your case to earn national recognition. TIP—Even if IRA did not specifically ask for something, you can include any (all!) sections of the program report to make your case to earn national recognition. 20

21. Resources Fall IRA/CAEP Workshops and Individual 30 min. Consultation at Association of Literacy Educators and Researchers (ALER) annual meeting (Sat., Nov. 1, 2014— Delray Beach, FL) Fall IRA/CAEP Workshops and Individual 30 min. Consultation at Association of Literacy Educators and Researchers (ALER) annual meeting (Sat., Nov. 1, 2014— Delray Beach, FL) Spring/Summer IRA/CAEP Workshops and Individual30 min. Consultation at International Reading Association (IRA) annual meeting (Fri., July 17, 2015—St. Louis, MO) Spring/Summer IRA/CAEP Workshops and Individual30 min. Consultation at International Reading Association (IRA) annual meeting (Fri., July 17, 2015—St. Louis, MO) IRA Research Division, Accreditation website IRA Research Division, Accreditation website http://www.reading.org/Resources/ProfessionalDevelopment/ Accreditation.aspx http://www.reading.org/Resources/ProfessionalDevelopment/ Accreditation.aspx 21

22. Resources, cont. CAEP (NCATE) SPA Library CAEP (NCATE) SPA Library http://www.ncate.org/Accreditation/ProgramReview/ProgramRevi ewResources/SPAAssessmentLibrary/tabid/460/Default.aspx http://www.ncate.org/Accreditation/ProgramReview/ProgramRevi ewResources/SPAAssessmentLibrary/tabid/460/Default.aspx IRA CAEP Guidelines for Program Writers and Reviewers IRA CAEP Guidelines for Program Writers and Reviewers http://www.reading.org/Libraries/resources/ira-guidelines-for- program-writers-and-reviewers.pdf http://www.reading.org/Libraries/resources/ira-guidelines-for- program-writers-and-reviewers.pdf Consultation with member of IRA Audit Team when much support is needed; contracted service with IRA, either on-site or email/telephone Consultation with member of IRA Audit Team when much support is needed; contracted service with IRA, either on-site or email/telephone 22

23. Q & A Contacts IRA and CAEP want your program to align to IRA Standards, to prepare 21 st century reading specialists/literacy coaches, and to be Nationally Recognized! Dr. Diane Kern, IRA SPA Coordinator: dkern@reading.org dkern@reading.org Elizabeth Vilky, CAEP, Director of Program Review elizabeth.vilky@caepnet.org elizabeth.vilky@caepnet.org Dr. Stevie Chepko, CAEP, Senior Vice President, Accreditation stevie.chepko@caepnet.org stevie.chepko@caepnet.org CAEP telephone: 202-223-0077 23