Preparing to Succeed with the NCSS/CAEP SPA Review Process CAEP Conference - September 24, 2017 Dr. Brandie Benton NCSS/CAEP SPA Coordinator Henderson State University Associate Provost Contact: bentonb@hsu.edu or 870-230-5203 1

Session Agenda Introductions CAEP Requirements for Program Review The 2004 NCSS Standards The 2017 NCSS Standards Overview of the Program Report Progress Report Sections Required Assessments Report Options Resources, Contacts, Q & A

CAEP Requirements All EPPs (Educator Preparation Providers) seeking CAEP Accreditation must complete a form of program review. States define the program review options available to EPPs as part of the new CAEP State Partnership Agreement. http://caepnet.org/~/media/Fil es/caep/program- review/program-review-by- state- agreementv21july.pdf?la=en (09/19/17) CAEP Review with Feedback and State Review do NOT lead to National Recognition.

CAEP Vocabulary EPP = Educator Preparation Provider (previously called “the unit”) Program = specific discipline area Candidate = pre-service teachers (college students) Student = P-12 student SPA = Specialized Professional Association

Review Comparison EPP Program Standards CAEP NCSS Reviewed by Site Visitors Program Reviewers Process Electronic (AIMS)/ On-site Electronic (AIMS) Decision Accreditation Recognition Final Decision Maker Accreditation Council NCSS Program Reviewers/ Audit Team

Questions the Program Review Will Answer Does the program have a comprehensive assessment system in place that demonstrates candidate mastery of the NCSS Standards? Have candidates mastered the necessary knowledge for the subjects they will teach and the performance requirements? Do candidates meet state licensure requirements? Do candidates understand teaching and learning and can they plan their teaching? Can candidates apply their knowledge of content and pedagogy in classrooms and schools? Are candidates effective in promoting student learning?

NCSS National Standards for Social Studies Teachers (2017) The new (2017) NCSS Standards have been “conditionally” approved by the CAEP SPA Standards Review Committee. Final approval will be granted in October pending the submission of sample assessments and program/reviewer training materials. A full day of training will be offered at the NCSS Conference in November in San Francisco, California. The new standards become operational on January 1, 2018. This date also signifies the start of a two year sunset period for the 2004 standards (S18, F18, S19, F19), which will expire Spring 2020. All programs seeking national recognition during the Spring 2019 cycle and beyond will be required to submit using the standards found in the 2017 NCSS National Standards for the Preparation of Social Studies Teachers. Because at least two (2) applications of each assessment are required for national recognition, the Spring 2019 cycle is the earliest that programs would be able to submit using the revised 2017 standards.

2017 NCSS Standards: 5 Core Competencies Standard 1. Content Knowledge (3 elements) Candidates demonstrate knowledge of social studies disciplines. Candidates are knowledgeable of disciplinary concepts, facts, and tools; structures of inquiry; and forms of representation. Standard 2. Application of Content Through Planning (5 elements) Candidates plan learning sequences that leverage social studies knowledge and literacies, technology, and theory and research to support the civic competence of learners. Standard 3. Design and Implementation of Instruction and Assessment (5 elements) Candidates design and implement instruction and authentic assessments, informed by data literacy and learner self-assessment, that promote civic competence. Standard 4. Social Studies Learners and Learning (3 elements) Candidates use knowledge of learners to plan and implement relevant and responsive pedagogy, create collaborative and interdisciplinary learning environments, and prepare learners to be informed advocates for an inclusive and equitable society. Standard 5. Professional Responsibility and Informed Action (3 elements) Candidates reflect and expand upon their social studies knowledge, inquiry skills, and civic dispositions to advance social justice and promote human rights through informed action in schools and/or communities.

The Transition: How to Prepare Utilize existing resources: Familiarize yourself with the 2017 Standards https://www.socialstudies.org/standards/teacherstandards Prepare candidates to integrate the C3 Framework https://www.socialstudies.org/c3 Prepare candidates to plan instruction and assessment using the Inquiry Design Model http://www.c3teachers.org/inquiry-design-model/ Understand the SPA process, as defined by CAEP, as well as assessment, rubric, and data expectations http://caepnet.org/accreditation/caep-accreditation/spa-program-review-policies-and-procedur Take advantage of future training opportunities (in-person and via Webinar)

The 10 Thematic Standards NCSS National Standards for Social Studies Teachers (2004) The 10 Thematic Standards Why themes? What difference does it make for your institutions?

Summary of Content Expectations of NCSS by Standard 1.1 Culture and Cultural Diversity. This standard expects candidates to understand the basic scholarly concepts of anthropology (The program could make a case for relevance of Cultural Geography or Sociology.) 1.2 Time, Continuity, and Change. This standard expects candidates to be familiar with the history of the United States, western civilization, and non- western society. 1.3 People, Places, and Environments. This standard expects candidates to understand the basic scholarly concepts of geography. 1.4 Individual Development and Identity. This standard expects candidates to understand the basic scholarly concepts of psychology. In general, educational psychology courses alone do not meet the expectations of this standard but with explanation, a Human Growth & Development course might be sufficient . 1.5 Individuals, Groups, and Institutions. This standard expects candidates to understand the basic scholarly concepts of sociology. (Also see 1.1) 11

Standards Expectations (Cont.) 1.6 Power, Authority, and Governance. This standard expects candidates to understand the basic scholarly concepts of political science. 1.7 Production, Distribution, and Consumption. This standard expects candidates to understand the basic concepts of micro- and macro-economics. 1.8 Science, Technology, and Society. This standard expects candidates to understand the manner by which science and technology have enhanced or threatened the development of human society. Educational technology application courses do not meet the expectations of this standard. 1.9 Global Connections. This standard expects candidates to understand that our planet has been exposed to an ever-increasing human interdependence in a world made smaller by improvements in communication, transportation, and trade. 1.10 Civic Ideals and Practices. This standard expects candidates to understand the basic concepts of citizenship in a democratic society. 12

Specialized Standards *only if license includes titles NCSS Standards 2.1-2.5 These standards expect that candidates receiving licensure in a specific social science discipline will understand the basic scholarly concepts of that discipline. 2.1 History 2.2 Geography 2.3 Civics and Government 2.4 Economics 2.5 Psychology Must still meet 1.1-1.10

Specialized Standards 3.1. Course(s) on Teaching Social Studies NCSS expects the social studies methods course or courses will deal specifically with the nature of the social studies and with concepts, strategies, and techniques for teaching social studies at the appropriate licensure level. NCSS does not recognize generic methods courses where social studies is only a minor emphasis. 3.2. Qualified Social Studies Faculty This standard expects that faculty in the social studies education components of the program are recognized as exemplary teachers and as scholars in history or the social sciences and/or social studies education. The credentials of faculty members, including instructor(s) of the methods course(s) and supervisors of clinical experiences, especially student teaching/internship, are to be described in the faculty chart and Section I: Field and Clinical Experiences. Try to spell out the differences with different types of programs, and need or not need for a separate report. 14

Program Review Process The report Web-based form (Shells in CAEP AIMS) Limited pages and characters by section Will be able to access drafts Do we need to do more than one report? How do we report data? Individual (separate) Program Reports - Baccalaureate, Master’s, Post- Baccalaureate, or Alternate Route? Linked Reports - Common assessments used across different programs Program offered at multiple sites could be disaggregated by site or submitted as linked reports 15

How much data do we need? State licensure test data Must meet 80% pass rate requirement Three years (derived from most recent Title II report OR EPP’s records to provide a most current set of data) Exam sub-scores must be aligned to the NCSS themes Assessment data Data for two years (or two administrations) on each assessment required Program meets or exceeds SPA benchmarked licensure test data requirement, if applicable. CAEP Data Requirements http://caepnet.org/accreditation/caep- accreditation/program-review-options/data-requirements 16

More About Data… Rubrics must have operational terms and discernable levels of performance that clearly address standards. Data must be disaggregated by application of assessment (i.e., Fall 2017, Spring 2018). Aggregate data must be present with the number of candidates, means, and ranges of scores. Numbers (n) reported within assessments should match that reported in the candidate/completer table in Section I of the report. Data does not have to be reported by candidate, even when program numbers are small. Data should be used for affirmation or program improvement. Analysis should include goals and future plans. Analysis of data should address individual standards. Where is performance weak/strong?

Sections of the Report Cover Sheet Contextual Information Assessments and Related Data Standards Assessment Chart Evidence for Meeting Standards Use of Assessment Results to Improve Candidate and Program Performance (narrative) Revised Reports/Response to Conditions – changes, additions, etc.

Section I: Contextual Information State or institutional policies that may influence the application of NCSS standards Describe required field and clinical experiences, including qualifications and training for University Supervisors and Cooperating Teachers Complete Candidate Information table with number of candidates and completers over past three years (academic or calendar) Complete Faculty Information chart Required Attachment: A program of study (or advising sheet) that outlines required courses, including titles and brief catalog descriptions FE Describe all experiences in program: duration, criteria for selection of sites and cooperating teachers, and preparation of teachers and supervisors for their roles. This is crucial for NCSS 3.2: NCSS expects supervisors to be Social Studies specialists. Faculty Indicate their role in the program. This includes faculty who teach the social studies/content methods course(s) and who supervise student teaching/internship experiences. It is not necessary to include faculty who teach the content courses or faculty who teach the generic teacher education courses. Program of study Be sure there is coverage of all 10 themes, applicable content standards, and pedagogy courses (including Social Studies Methods course or courses). This should correlate with Assessment 2, if the program is using Course Grades. Must match

Section II: List of Assessments Table must be completed in CAEP AIMS Identify assessment by title used in the program; refer to Section IV for further information on appropriate assessment to include. Identify the type of assessment (e.g., performance assessment, unit of study, impact on student learning project, comprehensive exam, reflection, state licensure test, portfolio). Indicate the point in the program when the assessment is administered (e.g., admission to the program, admission to student teaching/internship, in required courses [specify course title and numbers], or completion of the program). 20

The Required Assessments Name of Assessment Type or Form of Assessment When the Assessment Is Administered 1 [Licensure assessment, or other content-based assessment] Praxis II or state required content exam Prior to Student Teaching 2 [Assessment of content knowledge in social studies] Course Grades Each Semester 3 [Assessment of candidate ability to plan instruction] Lesson Plans During Methods Course 4 [Assessment of student teaching] University INTASC rubric with Social Studies Supplement During Student Teaching Semester 5 [Assessment of candidate effect on student learning] Teacher Work Sample During Methods Course Field Placement 6 Additional assessment that addresses standards (required) ] Candidate Portfolio Prior to Completion of Student Teaching 7 Additional assessment that addresses standards (optional) ] Comprehensive Content Exam Prior to graduation 8 Senior Research Project In Historiography Course

Section III: Relationship of Assessments to Standards For each NCSS Standard, identify the assessment(s) that address that standard One assessment may apply to multiple standards and some standards will be addressed by more than one assessment; It is not expected that each assessment will address all standards All programs must address Standards 1.1 - 1.10, 3.1, and 3.2 2.1 - 2.5 are only completed for applicable major field programs

Section IV: Evidence for Meeting Standards For each of the 6-8 assessments, provide: 2-page narrative including: A brief description of the assessment and its use in the program The alignment with NCSS standards it addresses; Why/how does this assessment fulfill the theme(s) identified? A brief summary of the data findings An interpretation of how findings provide evidence of meeting standards. Required Assessment Documentation: Attach the Assessment Tool, Scoring Guide, and Data Table

The Required Assessments: CK #1 State or Professional Licensure Exam Data 80% must pass as a prerequisite for program recognition. Submit sub-scores as well as composite scores. Submit table for EACH content exam. Aligned Praxis sub-scores for NCSS Themes Behavioral Sciences -- 1.1, 1.4, 1.5 US History -- 1.2 World History -- 1.2 Geography -- 1.3 Civics/Government -- 1.6 , 1.10 Economics -- 1.7 Three years of data (or administrations), including the most recent academic year. Data must be presented on all program completers. Data must be disaggregated by Program Type, Program Level, and Year/Semester/Application. Three years of data is required for all programs. 24

Other State Licensure Exams Pearson Evaluation Systems Exams – Texas, Oklahoma, New York Massachusetts States determine the content area(s) to be tested. Usually at least one version covers multiple standards.

The Required Assessments: CK #2 Content Knowledge in the Social Studies (internal) Course Grades (preferred) Must follow guidelines for course grades http://caepnet.org/accreditation/caep-accreditation/program-review- options/grade-policy Courses must be required for ALL candidates; no electives Must match program of study attached in Section I Transcript Analysis Must follow guidelines for transcript analysis http://caepnet.org/accreditation/caep-accreditation/program-review- options/transcript-analysis Other evidence: comprehensive exams, research capstone projects/reports, content portfolios

The Required Assessments: #3 and #4 3. Candidate Ability to Plan Instruction 4. Performance Assessment Assessment that demonstrates candidates can effectively plan classroom-based instruction in social studies Possible evidence: Lesson plans Units of study Inquiry Design Models EdTPA rubrics 1-5 and 6- 10 reported by rubric elements Assessment that demonstrates candidate knowledge, skills, and dispositions are applied effectively in clinical practice (i.e., The social studies classroom) Possible evidence: Observation rubric designed to measure instruction in the NCSS thematic standards 3 - NCSS considers the edTPA Planning for History/Social Studies Understandings rubrics #1-5 and Instruction in History/Social Studies rubrics #6-10 as providing evidence for Planning/Assessment 3. We expect data to be reported for each group by elements of the rubric. 4- Must be aligned to the NCSS standards with emphasis on ability to teach social studies content. “Generic” assessments and final grades in student teaching/internship are not sufficient. The edTPA instruction rubrics #6-10 alone do not provide adequate evidence for Assessment 4 because they do not report data on candidate performance in the NCSS standards. We continue to expect that candidate performance be assessed with a rubric that aligns with the NCSS standards 1-10 for Assessment 4. 27

The Required Assessments: Impact #5 Assessment that demonstrates candidate effects on P-12 student learning in the social studies classroom. Objectives and instruction aligned to social studies standards/frameworks Pre- and post-test analysis Candidate reflection on data Data used to inform instruction Possible evidence: teacher work samples portfolio tasks units of study Inquiry Design Models with aligned assessments case studies EdTPA rubrics 11-15 disaggregated and reported by element

Other Assessments: #6, #7, #8 #6 is required – An additional assessment that demonstrates candidates are proficient in content knowledge, professional and pedagogical knowledge, skills, and dispositions and/or student learning Address any standards that might not have been covered with 1 - 5. Assessments #7 and #8 are optional but can also be used to address standards that are not adequately addressed by the first 6 assessments. Submit new assessments for feedback. 29

NCSS (2004) Alignment with EdTPA The EdTPA rubrics do not align with the 2004 NCSS Thematic Standards because they were written to measure content knowledge, versus content pedagogy. However, NCSS accepts the following EdTPA assessment evidence: NCSS accepts the edTPA Planning for History/Social Studies Understandings rubrics #1-5 and Instruction in History/Social Studies rubrics #6-10 as providing evidence for Planning/Assessment 3. NCSS accepts EdTPA Assessment in History/Social Studies rubrics #11-15 as providing evidence for Candidate Impact/Effect on Student Learning/Assessment 5. NCSS does not accept Instruction in History/Social Studies rubrics #6-10 as providing evidence for Planning/Assessment 4 because they do not report data on candidate performance in the NCSS standards. NCSS continues to expect that candidate performance be assessed with a rubric that aligns with the NCSS standards 1-10 for Assessment 4. Data must be reported for each group by elements of the rubric. An alignment matrix is currently being developed for the 2017 Standards.

Section V: Use of Assessment Results to Improve Program Evidence must be presented in this section that assessment results have been analyzed and have been or will be used to improve candidate performance and strengthen the program Describe findings based on aggregated data, and discuss how findings have been or will be applied to program improvement. Organize discussion around Content knowledge Professional knowledge, skills, and dispositions And impact on student learning

Review Calendar Fall Cycle – program reports due September 15 Spring Cycle - program reports due March 15 Effective for Fall 2016 CAEP/NCATE site visits, there is three-year out submission requirement.

Program Review Decisions Reviewers will make one of the following recommendations based on your program report: The program is nationally recognized. The program is nationally recognized with conditions. (18 months) Further development required or nationally recognized with probation or not nationally recognized.

Options for Program Reports: Option A This option is addressed in current processes. It requires that the program select 6 to 8 key assessments required of all candidates. CAEP has prescribed the type of assessments for #1 – 5: State licensure examinations of content knowledge An additional assessment of content knowledge Assessments of candidate ability to plan Performance assessments Impact on student learning Additional assessment

Options for Program Reports: Option B Programs have the option to select their own assessments and the manner in which to make the case for their validity and sufficiency of evidence within the following constraints: A program cannot use more than 8 key assessments. Assessments should be required of all candidates. The program must include the state licensure test in the program area for assessment #1. This requirement is waived if there is no state licensure test in the program area. One assessment must demonstrate candidate effect on student learning. In their entirety, the assessments and data should demonstrate that candidates have mastered the SPA standards.

Accreditation Information Management System (AIMS)

Using AIMS to Manage Your Programs AIMS facilitates the Program Review Process. EPPs, Reviewers, and CAEP staff access and utilize AIMS to complete each Program Review Cycle. New programs must be added to AIMS to have the program reviewed (existing programs are already in the system). Programs no longer offered by the EPP should be archived. Someone at your EPP is charged with managing the AIMS system and will add and archive programs, as well as request shells for your program review.

Using AIMS to Manage Your Programs EPPs use AIMS to: Add new programs and archive discontinued programs Request program review shells Complete and submit program reports Access recognition reports (current and previous) Update EPP contact information Update faculty information

Program Review Shells All Program Reports must be submitted through AIMS. To submit a report, the institution must first add the program to their Manage Programs list. Programs added to the list receive a program identification number that lasts for the life of the program.

Requesting & Linking Shells Click on Program Options in the Navigation Pane. At the bottom of the page, click Request Shells. Select the semester/year for submission from the drop down menu at the top of the screen. Click Add at the bottom of the screen. Selection the Option, Review Type, SPA, Standard Set for the appropriate program in the AIMS system. Click OK. A green check will appear if your request was successfully submitted. To link shells, click Link at the bottom of the screen. Check the box next to the programs you wish to link. Click Submit.

Linked Programs Shells can be “linked” if Key Assessments are the same. Data must be disaggregated by: Program level (Bac, Post Bac, Masters) Program type (initial licensure, advanced) Year/Semester/Application Complete the first program report shell in the list in AIMS and the additional linked shells will import the information automatically from the first report; all reports must be manually submitted.

97th NCSS Annual Conference November 17-19, 2017 (Pre-conference: November 15-16) Registration and Event Details: http://www.socialstudies.org/conference New Standards - SPA Report Training - Reviewer Training

Questions?