1. Preparing to Succeed with the NCSS/CAEP SPA Review Process
CAEP Conference - September 24, 2017
Dr. Brandie Benton
NCSS/CAEP SPA Coordinator
Henderson State University
Associate Provost
Contact: or 1

2. Session Agenda Introductions CAEP Requirements for Program Review
The 2004 NCSS Standards
The 2017 NCSS Standards
Overview of the Program Report Progress
Report Sections
Required Assessments
Report Options
Resources, Contacts, Q & A

3. CAEP Requirements
All EPPs (Educator Preparation Providers) seeking CAEP Accreditation must complete a form of program review.
States define the program review options available to EPPs as part of the new CAEP State Partnership Agreement. es/caep/program- review/program-review-by- state- agreementv21july.pdf?la=en (09/19/17)
CAEP Review with Feedback and State Review do NOT lead to National Recognition.

4. CAEP Vocabulary
EPP = Educator Preparation Provider (previously called “the unit”)
Program = specific discipline area
Candidate = pre-service teachers (college students)
Student = P-12 student
SPA = Specialized Professional Association

5. Review Comparison EPP Program Standards CAEP NCSS Reviewed by
Site Visitors
Program Reviewers
Process
Electronic (AIMS)/ On-site
Electronic (AIMS)
Decision
Accreditation
Recognition
Final Decision Maker
Accreditation Council
NCSS Program Reviewers/ Audit Team

6. Questions the Program Review Will Answer
Does the program have a comprehensive assessment system in place that demonstrates candidate mastery of the NCSS Standards?
Have candidates mastered the necessary knowledge for the subjects they will teach and the performance requirements?
Do candidates meet state licensure requirements?
Do candidates understand teaching and learning and can they plan their teaching?
Can candidates apply their knowledge of content and pedagogy in classrooms and schools?
Are candidates effective in promoting student learning?

7. NCSS National Standards for Social Studies Teachers (2017)
The new (2017) NCSS Standards have been “conditionally” approved by the CAEP SPA Standards Review Committee.
Final approval will be granted in October pending the submission of sample assessments and program/reviewer training materials.
A full day of training will be offered at the NCSS Conference in November in San Francisco, California.
The new standards become operational on January 1, 2018.
This date also signifies the start of a two year sunset period for the 2004 standards (S18, F18, S19, F19), which will expire Spring 2020.
All programs seeking national recognition during the Spring 2019 cycle and beyond will be required to submit using the standards found in the 2017 NCSS National Standards for the Preparation of Social Studies Teachers.
Because at least two (2) applications of each assessment are required for national recognition, the Spring 2019 cycle is the earliest that programs would be able to submit using the revised 2017 standards.

8. 2017 NCSS Standards: 5 Core Competencies
Standard 1. Content Knowledge (3 elements) Candidates demonstrate knowledge of social studies disciplines. Candidates are knowledgeable of disciplinary concepts, facts, and tools; structures of inquiry; and forms of representation.
Standard 2. Application of Content Through Planning (5 elements) Candidates plan learning sequences that leverage social studies knowledge and literacies, technology, and theory and research to support the civic competence of learners.
Standard 3. Design and Implementation of Instruction and Assessment (5 elements) Candidates design and implement instruction and authentic assessments, informed by data literacy and learner self-assessment, that promote civic competence.
Standard 4. Social Studies Learners and Learning (3 elements) Candidates use knowledge of learners to plan and implement relevant and responsive pedagogy, create collaborative and interdisciplinary learning environments, and prepare learners to be informed advocates for an inclusive and equitable society.
Standard 5. Professional Responsibility and Informed Action (3 elements) Candidates reflect and expand upon their social studies knowledge, inquiry skills, and civic dispositions to advance social justice and promote human rights through informed action in schools and/or communities.

9. The Transition: How to Prepare
Utilize existing resources:
Familiarize yourself with the 2017 Standards
Prepare candidates to integrate the C3 Framework
Prepare candidates to plan instruction and assessment using the Inquiry Design Model
Understand the SPA process, as defined by CAEP, as well as assessment, rubric, and data expectations
Take advantage of future training opportunities (in-person and via Webinar)

10. The 10 Thematic Standards
NCSS National Standards for Social Studies Teachers (2004)
The 10 Thematic Standards
Why themes?
What difference does it make for your institutions?

11. Summary of Content Expectations of NCSS by Standard
1.1 Culture and Cultural Diversity. This standard expects candidates to understand the basic scholarly concepts of anthropology (The program could make a case for relevance of Cultural Geography or Sociology.)
1.2 Time, Continuity, and Change. This standard expects candidates to be familiar with the history of the United States, western civilization, and non- western society.
1.3 People, Places, and Environments. This standard expects candidates to understand the basic scholarly concepts of geography.
1.4 Individual Development and Identity. This standard expects candidates to understand the basic scholarly concepts of psychology. In general, educational psychology courses alone do not meet the expectations of this standard but with explanation, a Human Growth & Development course might be sufficient .
1.5 Individuals, Groups, and Institutions. This standard expects candidates to understand the basic scholarly concepts of sociology. (Also see 1.1) 11

12. Standards Expectations (Cont.)
1.6 Power, Authority, and Governance. This standard expects candidates to understand the basic scholarly concepts of political science.
1.7 Production, Distribution, and Consumption. This standard expects candidates to understand the basic concepts of micro- and macro-economics.
1.8 Science, Technology, and Society. This standard expects candidates to understand the manner by which science and technology have enhanced or threatened the development of human society. Educational technology application courses do not meet the expectations of this standard.
1.9 Global Connections. This standard expects candidates to understand that our planet has been exposed to an ever-increasing human interdependence in a world made smaller by improvements in communication, transportation, and trade.
1.10 Civic Ideals and Practices. This standard expects candidates to understand the basic concepts of citizenship in a democratic society. 12

13. Specialized Standards *only if license includes titles
NCSS Standards These standards expect that candidates receiving licensure in a specific social science discipline will understand the basic scholarly concepts of that discipline.
2.1 History
2.2 Geography
2.3 Civics and Government
2.4 Economics
2.5 Psychology
Must still meet

14. Specialized Standards
3.1. Course(s) on Teaching Social Studies NCSS expects the social studies methods course or courses will deal specifically with the nature of the social studies and with concepts, strategies, and techniques for teaching social studies at the appropriate licensure level. NCSS does not recognize generic methods courses where social studies is only a minor emphasis.
3.2. Qualified Social Studies Faculty This standard expects that faculty in the social studies education components of the program are recognized as exemplary teachers and as scholars in history or the social sciences and/or social studies education. The credentials of faculty members, including instructor(s) of the methods course(s) and supervisors of clinical experiences, especially student teaching/internship, are to be described in the faculty chart and Section I: Field and Clinical Experiences.
Try to spell out the differences with different types of programs, and need or not need for a separate report. 14

15. Program Review Process
The report
Web-based form (Shells in CAEP AIMS)
Limited pages and characters by section
Will be able to access drafts
Do we need to do more than one report? How do we report data?
Individual (separate) Program Reports - Baccalaureate, Master’s, Post- Baccalaureate, or Alternate Route?
Linked Reports - Common assessments used across different programs
Program offered at multiple sites could be disaggregated by site or submitted as linked reports 15

16. How much data do we need? State licensure test data
Must meet 80% pass rate requirement
Three years (derived from most recent Title II report OR EPP’s records to provide a most current set of data)
Exam sub-scores must be aligned to the NCSS themes
Assessment data
Data for two years (or two administrations) on each assessment required
Program meets or exceeds SPA benchmarked licensure test data requirement, if applicable.
CAEP Data Requirements accreditation/program-review-options/data-requirements 16

17. More About Data…
Rubrics must have operational terms and discernable levels of performance that clearly address standards.
Data must be disaggregated by application of assessment (i.e., Fall 2017, Spring 2018).
Aggregate data must be present with the number of candidates, means, and ranges of scores.
Numbers (n) reported within assessments should match that reported in the candidate/completer table in Section I of the report.
Data does not have to be reported by candidate, even when program numbers are small.
Data should be used for affirmation or program improvement.
Analysis should include goals and future plans.
Analysis of data should address individual standards. Where is performance weak/strong?

18. Sections of the Report Cover Sheet Contextual Information
Assessments and Related Data
Standards Assessment Chart
Evidence for Meeting Standards
Use of Assessment Results to Improve Candidate and Program Performance (narrative)
Revised Reports/Response to Conditions – changes, additions, etc.

19. Section I: Contextual Information
State or institutional policies that may influence the application of NCSS standards
Describe required field and clinical experiences, including qualifications and training for University Supervisors and Cooperating Teachers
Complete Candidate Information table with number of candidates and completers over past three years (academic or calendar)
Complete Faculty Information chart
Required Attachment: A program of study (or advising sheet) that outlines required courses, including titles and brief catalog descriptions FE
Describe all experiences in program: duration, criteria for selection of sites and cooperating teachers, and preparation of teachers and supervisors for their roles.
This is crucial for NCSS 3.2: NCSS expects supervisors to be Social Studies specialists.
Faculty
Indicate their role in the program. This includes faculty who teach the social studies/content methods course(s) and who supervise student teaching/internship experiences. It is not necessary to include faculty who teach the content courses or faculty who teach the generic teacher education courses.
Program of study
Be sure there is coverage of all 10 themes, applicable content standards, and pedagogy courses (including Social Studies Methods course or courses). This should correlate with Assessment 2, if the program is using Course Grades. Must match

20. Section II: List of Assessments
Table must be completed in CAEP AIMS
Identify assessment by title used in the program; refer to Section IV for further information on appropriate assessment to include.
Identify the type of assessment (e.g., performance assessment, unit of study, impact on student learning project, comprehensive exam, reflection, state licensure test, portfolio).
Indicate the point in the program when the assessment is administered (e.g., admission to the program, admission to student teaching/internship, in required courses [specify course title and numbers], or completion of the program). 20

21. The Required Assessments
Name of Assessment
Type or
Form of Assessment
When the Assessment
Is Administered 1
[Licensure assessment, or other content-based assessment]
Praxis II or state required content exam
Prior to Student Teaching 2
[Assessment of content knowledge in social studies]
Course Grades
Each Semester 3
[Assessment of candidate ability to plan instruction]
Lesson Plans
During Methods Course 4
[Assessment of student teaching]
University INTASC rubric with Social Studies Supplement
During Student Teaching Semester 5
[Assessment of candidate effect on student learning]
Teacher Work Sample
During Methods Course Field Placement 6
Additional assessment that addresses standards (required) ]
Candidate Portfolio
Prior to Completion of Student Teaching 7
Additional assessment that addresses standards (optional) ]
Comprehensive Content Exam
Prior to graduation 8
Senior Research Project
In Historiography Course

22. Section III: Relationship of Assessments to Standards
For each NCSS Standard, identify the assessment(s) that address that standard
One assessment may apply to multiple standards and some standards will be addressed by more than one assessment; It is not expected that each assessment will address all standards
All programs must address Standards , 3.1, and 3.2
are only completed for applicable major field programs

23. Section IV: Evidence for Meeting Standards
For each of the 6-8 assessments, provide:
2-page narrative including:
A brief description of the assessment and its use in the program
The alignment with NCSS standards it addresses;
Why/how does this assessment fulfill the theme(s) identified?
A brief summary of the data findings
An interpretation of how findings provide evidence of meeting standards.
Required Assessment Documentation:
Attach the Assessment Tool, Scoring Guide, and Data Table

24. The Required Assessments: CK #1
State or Professional Licensure Exam Data
80% must pass as a prerequisite for program recognition.
Submit sub-scores as well as composite scores.
Submit table for EACH content exam.
Aligned Praxis sub-scores for NCSS Themes
Behavioral Sciences , 1.4, 1.5
US History
World History
Geography
Civics/Government , 1.10
Economics
Three years of data (or administrations), including the most recent academic year.
Data must be presented on all program completers.
Data must be disaggregated by Program Type, Program Level, and Year/Semester/Application.
Three years of data is required for all programs. 24

25. Other State Licensure Exams
Pearson Evaluation Systems Exams – Texas, Oklahoma, New York
Massachusetts
States determine the content area(s) to be tested.
Usually at least one version covers multiple standards.

26. The Required Assessments: CK #2
Content Knowledge in the Social Studies (internal)
Course Grades (preferred)
Must follow guidelines for course grades options/grade-policy
Courses must be required for ALL candidates; no electives
Must match program of study attached in Section I
Transcript Analysis
Must follow guidelines for transcript analysis options/transcript-analysis
Other evidence: comprehensive exams, research capstone projects/reports, content portfolios

27. The Required Assessments: #3 and #4
3. Candidate Ability to Plan Instruction
4. Performance Assessment
Assessment that demonstrates candidates can effectively plan classroom-based instruction in social studies
Possible evidence:
Lesson plans
Units of study
Inquiry Design Models
EdTPA rubrics 1-5 and reported by rubric elements
Assessment that demonstrates candidate knowledge, skills, and dispositions are applied effectively in clinical practice (i.e., The social studies classroom)
Possible evidence:
Observation rubric designed to measure instruction in the NCSS thematic standards
3 - NCSS considers the edTPA Planning for History/Social Studies Understandings rubrics #1-5 and
Instruction in History/Social Studies rubrics #6-10 as providing evidence for
Planning/Assessment 3. We expect data to be reported for each group by elements of the
rubric.
4- Must be aligned to the NCSS standards with emphasis on ability to teach social studies content. “Generic” assessments and final grades in student teaching/internship are not sufficient.
The edTPA instruction rubrics #6-10 alone do not provide adequate evidence for Assessment 4
because they do not report data on candidate performance in the NCSS standards. We
continue to expect that candidate performance be assessed with a rubric that aligns with the
NCSS standards 1-10 for Assessment 4. 27

28. The Required Assessments: Impact #5
Assessment that demonstrates candidate effects on P-12 student learning in the social studies classroom.
Objectives and instruction aligned to social studies standards/frameworks
Pre- and post-test analysis
Candidate reflection on data
Data used to inform instruction
Possible evidence:
teacher work samples
portfolio tasks
units of study
Inquiry Design Models with aligned assessments
case studies
EdTPA rubrics disaggregated and reported by element

29. Other Assessments: #6, #7, #8
#6 is required – An additional assessment that demonstrates candidates are proficient in content knowledge, professional and pedagogical knowledge, skills, and dispositions and/or student learning
Address any standards that might not have been covered with
Assessments #7 and #8 are optional but can also be used to address standards that are not adequately addressed by the first 6 assessments.
Submit new assessments for feedback. 29

30. NCSS (2004) Alignment with EdTPA
The EdTPA rubrics do not align with the 2004 NCSS Thematic Standards because they were written to measure content knowledge, versus content pedagogy.
However, NCSS accepts the following EdTPA assessment evidence:
NCSS accepts the edTPA Planning for History/Social Studies Understandings rubrics #1-5 and Instruction in History/Social Studies rubrics #6-10 as providing evidence for Planning/Assessment 3.
NCSS accepts EdTPA Assessment in History/Social Studies rubrics #11-15 as providing evidence for Candidate Impact/Effect on Student Learning/Assessment 5.
NCSS does not accept Instruction in History/Social Studies rubrics #6-10 as providing evidence for Planning/Assessment 4 because they do not report data on candidate performance in the NCSS standards. NCSS continues to expect that candidate performance be assessed with a rubric that aligns with the NCSS standards 1-10 for Assessment 4.
Data must be reported for each group by elements of the rubric.
An alignment matrix is currently being developed for the 2017 Standards.

31. Section V: Use of Assessment Results to Improve Program
Evidence must be presented in this section that assessment results have been analyzed and have been or will be used to improve candidate performance and strengthen the program
Describe findings based on aggregated data, and discuss how findings have been or will be applied to program improvement.
Organize discussion around
Content knowledge
Professional knowledge, skills, and dispositions
And impact on student learning

32. Review Calendar Fall Cycle – program reports due September 15
Spring Cycle - program reports due March 15
Effective for Fall CAEP/NCATE site visits, there is three-year out submission requirement.

33. Program Review Decisions
Reviewers will make one of the following recommendations based on your program report:
The program is nationally recognized.
The program is nationally recognized with conditions. (18 months)
Further development required or nationally recognized with probation or not nationally recognized.

34. Options for Program Reports: Option A
This option is addressed in current processes. It requires that the program select 6 to 8 key assessments required of all candidates. CAEP has prescribed the type of assessments for #1 – 5:
State licensure examinations of content knowledge
An additional assessment of content knowledge
Assessments of candidate ability to plan
Performance assessments
Impact on student learning
Additional assessment

35. Options for Program Reports: Option B
Programs have the option to select their own assessments and the manner in which to make the case for their validity and sufficiency of evidence within the following constraints:
A program cannot use more than 8 key assessments.
Assessments should be required of all candidates.
The program must include the state licensure test in the program area for assessment #1. This requirement is waived if there is no state licensure test in the program area.
One assessment must demonstrate candidate effect on student learning.
In their entirety, the assessments and data should demonstrate that candidates have mastered the SPA standards.

36. Accreditation Information Management System (AIMS)

37. Using AIMS to Manage Your Programs
AIMS facilitates the Program Review Process.
EPPs, Reviewers, and CAEP staff access and utilize AIMS to complete each Program Review Cycle.
New programs must be added to AIMS to have the program reviewed (existing programs are already in the system).
Programs no longer offered by the EPP should be archived.
Someone at your EPP is charged with managing the AIMS system and will add and archive programs, as well as request shells for your program review.

38. Using AIMS to Manage Your Programs
EPPs use AIMS to:
Add new programs and archive discontinued programs
Request program review shells
Complete and submit program reports
Access recognition reports (current and previous)
Update EPP contact information
Update faculty information

39. Program Review Shells
All Program Reports must be submitted through AIMS.
To submit a report, the institution must first add the program to their Manage Programs list.
Programs added to the list receive a program identification number that lasts for the life of the program.

40. Requesting & Linking Shells
Click on Program Options in the Navigation Pane.
At the bottom of the page, click Request Shells.
Select the semester/year for submission from the drop down menu at the top of the screen.
Click Add at the bottom of the screen.
Selection the Option, Review Type, SPA, Standard Set for the appropriate program in the AIMS system.
Click OK. A green check will appear if your request was successfully submitted.
To link shells, click Link at the bottom of the screen. Check the box next to the programs you wish to link. Click Submit.

41. Linked Programs
Shells can be “linked” if Key Assessments are the same.
Data must be disaggregated by:
Program level (Bac, Post Bac, Masters)
Program type (initial licensure, advanced)
Year/Semester/Application
Complete the first program report shell in the list in AIMS and the additional linked shells will import the information automatically from the first report; all reports must be manually submitted.

42. 97th NCSS Annual Conference
November 17-19, 2017
(Pre-conference: November 15-16)
Registration and Event Details:
New Standards - SPA Report Training - Reviewer Training

43. Questions?