Preparing to Succeed with the Electronic NCSS/NCATE SPA Review Process Accreditation, Accountability, and Quality An Institutional Orientation and Professional Development Conference September 17, 2006 D. Mark Meyers Rowan University

The Ten Themes  Why themes?  What difference does it make for your institutions?  Change in the Review Process and its implication  Why themes?  What difference does it make for your institutions?  Change in the Review Process and its implication

New Program Review Process The report  Web-based form  Limited pages by section  Will be able to access drafts Do we need to do more than one report?  Common Assessments in Different Programs  Baccalaureate, Master’s, Post-Baccalaureate, or Alternate Route?  Program offered at Multiple Sites

Questions the Program Review Will Answer Does the program have in place assessments that demonstrate mastery of the NCSS themes.  Have candidates mastered the necessary knowledge for the subjects they will teach or the jobs they will perform?  Do candidates meet state licensure requirements?  Do candidates understand teaching and learning and can they plan their teaching?  Can candidates apply their knowledge in classrooms and schools?  Are candidates effective in promoting student learning?

How much data do we need? State licensure test data  Must have 80% pass rate  For 3 years with latest derived from most recent annual report, (or own records to provide a more current set of data) Assessment data  For Fall Spring 07 the program has at least 1 semester’s data on at least 5 assessments  After Spring 07 Three years of data on each set of assessments  Note: It is better to have limited data on good assessments than tons of data on ineffective ones. However, there must be more than the ‘intention’ to assess

Sections of the Report I. Contextual Information II. Assessments and Related Data III. Standards Assessment Chart IV. Evidence for Meeting Standards V. Use of Assessment Results to Improve Candidate and Program Performance

I.Contextual Information Describe any state or institutional policies that may influence the application of NCSS standards  Explanations and clarifications, not a page for excuses Description of required field and clinical experiences  Describe all experiences in program, duration, criteria for selection of sites and teachers, and training of teachers and supervisors  This is crucial for NCSS 3.2 NCSS expects supervisors to be SS specialists. Description of criteria for admission, retention, and exit from the program.  For the program, not the entire department, school, college unless they are the same

Contextual Information Continued Description of the relationship of the program to the unit’s conceptual framework  A narrative describing how the CF is reflected in the program  It should be clear that your program is a part of the unit, not a separate entity Indication of unique program assessments and its relationship to the unit assessment system  This should clarify how the key assessments used in the program are derived from the unit assessment system Attach the following  A program of study that outlines required courses Titles and brief catalog descriptions, be sure there is coverage of all 10 themes and applicable content standards Remember to avoid the trap of ‘uncommon course titles”  Chart with number of candidates and completers over past three years (Title II Data)  Chart on program faculty expertise and experience

Summary of Content Expectations of NCSS by Standard 1.1 Culture and Cultural Diversity. This standard expects candidates to understand the basic scholarly concepts of anthropology. 1.2 Time, Continuity, and Change. This standard expects candidates to be familiar with the history of the United States, western civilization, and non- western society. 1.3 People, Places, and Environments. This standard expects candidates to understand the basic scholarly concepts of geography. 1.4 Individual Development and Identity. This standard expects candidates to understand the basic scholarly concepts of psychology. 1.5 Individuals, Groups, and Institutions. This standard expects candidates to understand the basic scholarly concepts of sociology.

Standards Expectations Continued 1.6 Power, Authority, and Governance. This standard expects candidates to understand the basic scholarly concepts of political science. 1.7 Production, Distribution, and Consumption. This standard expects candidates to understand the basic concepts of micro- and macro-economics. 1.8 Science, Technology, and Society. This standard expects candidates to understand the manner by which science and technology have enhanced or threatened the development of human society. In general, educational technology application courses alone do not meet the expectations of this standard. 1.9 Global Connections. This standard expects candidates to understand that our planet has been exposed to an ever-increasing human interdependence in a world made smaller by improvements in communication, transportation, and trade Civic Ideals and Practices. This standard expects candidates to understand the basic concepts of citizenship in a democratic society.

Specialized Standards 2.1 through 2.5. These standards expect the candidates receiving licensure in a specific social science discipline to understand the basic scholarly concepts of that discipline. 3.1 Course(s) on Teaching Social Studies. This standard expects that the social studies methods courses(s) will deal specifically with the nature of the social studies and with ideas, strategies, and techniques for teaching social studies at the appropriate licensure level. The NCSS frowns upon a generic methods course where social studies instruction is only a minor aspect of the emphases. 3.2 Qualified Social Studies Faculty. This standard expects that faculty in the social studies education components of the program are recognized as exemplary teachers and as scholars in history or the social sciences and/or social studies education. The credentials of faculty members are to be described in Attachment B.

II. Assessments and Related Data In Chart form For each assessment, indicate name, type, and administration point on chart

The Required Assessments 1. State or Professional Exam Data 80% must pass as a prerequisite for program recognition. Subscores as well as totals 2. Assessment of Content Knowledge in the Social Studies Comp exams, research reports, portfolios, course grades, test grades/unit assessments (Attachment) Be careful about course grades! 3. Assessment that demonstrates candidates can effectively plan classroom-based instruction Practicum evaluations, lesson/unit planning 4. Assessment that demonstrates candidates knowledge, skills, and dispositions are applied effectively in clinical practice Must be connected to the NCSS standards, not merely ‘generic’ 5. Assessment that demonstrates candidate effects on P-12 student learning Teacher work samples, portfolio tasks, case studies, Poster Sessions, etc 6. Assessment that demonstrates candidates are proficient in content knowledge, professional and pedagogical knowledge, skills, and dispositions and/or student learning Fill in any standards that might not have been covered with

Be careful of Labels On my campus, there have been some confusion with everyone using the term “Assessments” What we did, was to move towards the use of agreed upon terms. Assessments SPA Assessment for Program Reviews Benchmarks -- Items at Transition Points in the Unit Assessment System Signature Assignments -- Items inside of courses that are used no matter who teaches the course These are given only as an example not to imply this as the only means of approaching these tasks.

III. Standards Assessment Chart For each NCSS Standard, identify the assessments that address each standard One assessment may apply to multiple standards and some standards will be addressed by more than one assessment All programs must do and are only done by applicable major field programs In the next section you will describe the assessments in greater detail

IV. Evidence for Meeting Standards For each of the 6-8 assessments, provide: 1. 2-page narrative including: a) A brief description of the assessment and its use in the program b) The alignment with NCSS standards it addresses; Why does this assessment fulfill the theme(s) identified c) A brief summary of the data findings d) An interpretation of how findings provide evidence of meeting standards. 2. Make the information easy to follow and user-friendly 3. Assessment Instrument, Scoring Guide, and Data Tables

IV continued For state licensing tests or professional exams submit the following The names of all licensure tests or professional examinations required by the state for content and pedagogical or professional knowledge.  For example, Praxis II Social Studies: Content Knowledge. Description of the correlation between licensure test data and applicable NCSS standards. Praxis subscores are provided for History (1.2), Geography (1.3), Civics/Gov’t (1.6 & 1.10), Econ. (1.7) & Behavioral Sciences (1.1,1.4, & 1.5). Please note: quartile scores are not appropriate evidence Aggregated pass rates for each year over the past 3 years, including the most recent academic year. Data must be presented on all candidates

V. Use of Assessment Results Evidence must be presented in this section that assessment results have been analyzed and have been or will be used to improve candidate performance and strengthen the program Describe findings based on aggregated data, and discuss how findings have been or will be applied to program improvement. Organize discussion around  Content knowledge;  Professional knowledge, skills, and dispositions, and  Impact on student learning.

Review calendar Fall cycle – program reports due September 15 Spring cycle – program reports due February 1 It is recommended you submit 1 year in advance. Example: XYZ College plans to submit program reports one semester before its Spring 2007 visit; thus, reports are due by 2/1/06.

Program review decisions Reviewers will make one of the following decisions based on your program report: The program is nationally recognized. The program is nationally recognized with conditions. (18 months) The program is nationally recognized with areas in need of improvement The program is not nationally recognized; a new program report may be submitted.

How will my program be reviewed? Each program report will be reviewed by a 3-person team including a lead reviewer The lead reviewer will compile the final team report and send to NCATE NCSS Audit Committee will review reports for consistency and make final decision if review team was undecided Guidelines for Program reviewers are available for you to review.

Some Questions What if we do not have data for all submitted assessments? What if our current program assessments are not a close match to the 6-8 SPA assessments? Remember, for each assessment present title of assessment, group assessed, and number in group.  Specify the cohort  Provide data on each scored category  Provide legends for abbreviations

D. Mark Meyers Rowan University

NCATE Paper on Impact on Student Learning Available on the website Suggests the candidate:  Undertakes a diagnosis (a pre-test) of P-12 student learning in some area he or she will teach;  Plans an appropriate sequence of instruction to advance P-12 student learning, and teaches in ways that engage P- 12 students who bring differing background knowledge and learning needs;  Conducts some concluding assessment (or post-test); documents the student learning that occurred, or did not;  Reflects on changes in teaching that might have improved the results.

Sample Chart Courses/Segments

Sample Chart for Clinical practice

4,22

4.32

Behavioral Sciences , 1.4, 1.5 US History World History Geography Civics/Government , 1.10 Economics NCSS considers these sub-tests to be aligned with the enumerated standards. Praxis Subscores for NCSS Themes