1. Michel Dias NSTA SPA Coordinator Chair: Audit Team NSTA – CAEP
2017 Fall CAEP Conference 
NSTA/CAEP Preservice Standards: Preparing Your Program Report
Michel Dias NSTA SPA Coordinator
Chair: Audit Team NSTA – CAEP
Professor of Science Education Department of Middle and Secondary Education Kennesaw State University
This presentation is for the CAEP Con and needed to be revised accordingly

2. My goals for this session
I will provide…
support for science teacher educators who lead candidate and program assessment.
better understanding of NSTA SPA Standards via analysis of sample program data.
advice for developing an assessment system and for reporting program data in your SPA report. 

3. Objective 1
…recall the types of assessment data that may be used for NSTA SPA National Recognition.
SPAs do not provide accreditation, but National Recognition. Language adjusted

4. What is Inherent in these Standards?
The preservice standards:
Focus on student learning,
Describe what proficient science candidates should know and be able to do at this stage of their development, and
Founded on a research knowledge base.

5. Do Programs Provide Evidence of:
mastery of related knowledge base at the beginner level?
ability to apply knowledge base in the secondary (6-12) classroom?
developing candidates for work with science-specific technology and diverse students while developing cultural competence?
professionalism in their area of licensure and expertise?

6. NSTA Preservice Standards
1: Content Knowledge 2: Content Pedagogy 3: Learning Environments 4: Safety 5: Impact on Student Learning 6: Professional Knowledge and Skills

7. NSTA 2012 Standards and Elements Std. Learning Work Sample
Content Stand. Tests
Content Forms / Grades
PCK Unit Plan
PCK Obs. Form
Std. Learning Work Sample
PCK Professional
1a. Concepts, principles, theories, laws, & relationships
1b. Supporting content
1c. State/Nat’l Standards
2a. Variety of inq. approaches
2b. Active inquiry lessons
2c. naïve /preconceptions
3a. Variety teaching strategies
3b. Inquiry, labs, field, NOS
3c. Assessment
3d. Safety
4a. Chemicals
4b. Procedures
4c. Living organisms
5a. Coll., org., anal. on evidence
5b. ways of knowing
5c. Dev. appropriate inquiries
6a. Content Prof. Dev.
6b. Education Prof. Dev.

8. Objective 2
…analyze program assessments in relation to the six standards and 18 elements of the NSTA standards for science teacher preparation.

9. Preponderance of Evidence
See Reviewer Rubrics
Preponderance of Evidence
As defined by the CAEP Guidelines “means an overall confirmation that candidates meet standards in the strength, weight, or quality of evidence.” 
Reviewers will be using “professional judgments” to determine whether standards are met based on preponderance of evidence. 
SASB= the old NCATE Board. Refer to the CAEP Guidelines document for the SPAs: 
Second bullet removed as this is old policy, not one on the 2017 Guidelines.
Language for third bullet revised.

10. Decisions for Meeting Standards
Preponderance of evidence at the standard level.
Programs are required to provide evidence for all elements.
Recognition will not require that every aspect or element of each standard be met.
NSTA has rubrics for helping interpret the “preponderance of evidence.”
Second bullet added for clarification

11. For each assessment prepare one document that includes:
(1) A two-page narrative that includes the following:
a. A brief description of the assessment and its use in the program (one sentence may be sufficient);
b. A description of how this assessment specifically aligns with the elements and standards it is cited for in Section III. Cite SPA standards by number, title, and/or standard wording.
c. A brief analysis of the data findings;
d. An interpretation of how that data provides evidence for meeting standards, indicating the specific SPA standards by number, title, and/or standard wording;
(2) Assessment Documentation (five pages)
e. The assessment tool itself or a rich description of the assessment (often the directions given to candidates);
f. The scoring guide for the assessment; and
g. Charts that provide candidate data derived from the assessment.
Check order of parts in current Report template. Get PDF file from CAEP site – directions for Option A

12. All Assessments will have:
Rubrics with operational terms and discernible levels of performance that clearly address applicable standard or elements with minimum levels of performance.
Aggregated data presented with mean and range scores for each element.
Data used for affirmation and/or improving the program.
Analysis including goals and future plans.
Analysis of data addressing elements of the standard.

13. Objective 3
…analyze program assessments in relation to the six standards and 18 elements of NSTA:
1: Content Knowledge
2: Content Pedagogy 
3: Learning Environments
4: Safety
5: Impact on Student Learning
6: Professional Knowledge and Skills

14. NSTA Standard 1: Content Knowledge Effective teachers of science understand and articulate the knowledge and practices of contemporary science. They interrelate and interpret important concepts, ideas, and applications in their fields of licensure.
Preservice teachers will:
1a) Understand the major concepts, principles, theories, laws, and interrelationships of their fields of licensure and supporting fields as recommended by the National Science Teachers Association.
1b) Understand the central concepts of the supporting disciplines and the supporting role of science-specific technology.
1c) Show an understanding of state and national curriculum standards and their impact on the content knowledge necessary for teaching P-12 students.
Assessments – (1) State exams, (2) GPA, CAF, and (3) Unit Plan.

15. Assessment 1: Content Knowledge
Content Exam (Praxis II or State-specific)
E.g., Life science take test #5235
Praxis II is considered aligned with NSTA content standards
Reporting Data
List programs individually by the licenses they provide
Do not present one report for multiple licensure areas
Disaggregate by Program Type, Program Level, and Year and present as individual reports
Table for each content exam
Include subscores for analysis
The points under "Reporting data" had to be revised to align with CAEP policies. Information earlier shared by NSTA did not match CAEP policies.

16. Content Assessment
Data reflect, number, range, and percentage of candidates who have passed
Two applications of data for initial review
One application of data from revised assessments for resubmissions
Most recent year:
Total scores
Sub-scores
All program completes, even if less than 10
Information on this slide was incomplete. Had to be revised.
Programs need to provide 2 applications of data for assessments and NOT 3 Years of data.
Additionally, 80% pass rate requirement removed. NSTA dis not seek a waiver from CAEP to continue with the requirement. CAEP does not have the 80% pass rate requirement for National Recognition.

17. Preponderance of Evidence: Std. 1
Programs shall, at a minimum, meet element 1a of this standard and at least one other element at the Acceptable level in order to meet the standard. If only 1a is met, then the preponderance of evidence will indicate that this standard is Met with Conditions.

18. Assessment 2: Content Knowledge
Usually GPA
Courses must be required for all candidates (choose): Refer to CAEP policy for reporting course grades: 
Curriculum requirements: course #s
Bac.: Submit a Program of Study
PB or Masters: Advising or Content Checklist for admission (aligned to CAF)
Remediation description
Institution’s grade policy and definitions
Disaggregated data: program level & type and year

19. Content Analysis Form Greater or equal to 80% alignment
Graduate Programs:
Advising sheets
Programs of study
Transcript analysis
No syllabi
Course Title?
Provide description if content not obvious
Can use courses multiple times
Science education courses count

20. Content Analysis Form A. Core Competencies (numbers 1-13)
B: Required course number & name or advising requirements
Fundamental structures of atoms and molecules
Chem Gen. Chem I
Chem Gen. Chem II
Chem Gen. Chem II - Lab
Physical and chemical properties and classification of elements including periodicity
Chem Gen. Chem I - Lab
Principles of ElectroChem
Chem Analytical Chem
Transition elements and coordination compounds

21. You Review! Standard 1, Assessments 1 & 2.

22. Preservice teachers will:
NSTA Standard 2: Content Pedagogy Effective teachers of science understand how students learn and develop scientific knowledge. Preservice teachers use scientific inquiry to develop this knowledge for all students.
 Preservice teachers will:
2a) Plan multiple lessons using a variety of inquiry approaches that demonstrate their knowledge and understanding of how all students learn science.
2b) Include active inquiry lessons where students collect and interpret data in order to develop and communicate concepts and understand scientific processes, relationships and natural patterns from empirical experiences. Applications of science-specific technology are included in the lessons when appropriate.
2c) Design instruction and assessment strategies that confront and address naïve conceptions/preconceptions.
Assessment 3 – Unit Plan

23. Preponderance of Evidence: Std. 2
Programs shall meet two of the three elements of this standard at an Acceptable level in order to meet the Standard.
If one of the three elements is met at an Acceptable level and the preponderance of evidence is convincing, then this standard is Met with Conditions.

24. NSTA Standard 3: Learning Environment Effective teachers of science are able to plan for engaging all students in science learning by setting appropriate goals that are consistent with knowledge of how students learn science and are aligned with state and national standards. The plans reflect the nature and social context of science, inquiry, and appropriate safety considerations. Candidates design and select learning activities, instructional settings, and resources--including science-specific technology, to achieve those goals; and they plan fair and equitable assessment strategies to evaluate if the learning goals are met. 
Preservice teachers will:
3a) Use a variety of strategies that demonstrate the candidates’ knowledge and understanding of how to select the appropriate teaching and learning activities – including laboratory or field settings and applicable instruments and/or technology- to allow access so that all students learn. These strategies are inclusive and motivating for all students. 

25. NSTA Standard 3: Learning Environment
Preservice teachers will:
3b) Develop lesson plans that include active inquiry lessons where students collect and interpret data using applicable science-specific technology in order to develop concepts, understand scientific processes, relationships and natural patterns from empirical experiences. These plans provide for equitable achievement of science literacy for all students.
3c) Plan fair and equitable assessment strategies to analyze student learning and to evaluate if the learning goals are met. Assessment strategies are designed to continuously evaluate preconceptions and ideas that students hold and the understandings that students have formulated.
3d) Plan a learning environment and learning experiences for all students that demonstrate chemical safety, safety procedures, and the ethical treatment of living organisms within their licensure area.
Assessment 3 - Unit Plan

26. Preponderance of Evidence: Std. 3
Programs shall meet three of four elements of this standard at an Acceptable level in order to meet the standard.
If two of the four elements are met at an Acceptable level and the preponderance of evidence is convincing, then this standard is Met with Conditions.

27. You Review! Standards 2 & 3, Assessment 3.

28. Preservice teachers will:
NSTA Standard 4: Safety Effective teachers of science can, in a P-12 classroom setting, demonstrate and maintain chemical safety, safety procedures, and the ethical treatment of living organisms needed in the P-12 science classroom appropriate to their area of licensure.
  Preservice teachers will:
4a) Design activities in a P-12 classroom that demonstrate the safe and proper techniques for the preparation, storage, dispensing, supervision, and disposal of all materials used within their subject area science instruction.
4b) Design and demonstrate activities in a P-12 classroom that demonstrate an ability to implement emergency procedures and the maintenance of safety equipment, policies and procedures that comply with established state and/or national guidelines. Candidates ensure safe science activities appropriate for the abilities of all students.
4c) Design and demonstrate activities in a P-12 classroom that demonstrate ethical decision-making with respect to the treatment of all living organisms in and out of the classroom. They emphasize safe, humane, and ethical treatment of animals and comply with the legal restrictions on the collection, keeping, and use of living organisms.
Assessment 4 - Student Teaching Observation Form.

29. Assessment 4: Safety Observation Instrument
At a minimum, NSTA Standards/elements 4a-c
Best done as a content-specific addendum

30. Preponderance of Evidence: Std. 4
Programs shall meet two of the three elements of this standard at an Acceptable level in order to meet the standard.
If one of the three elements is met at an Acceptable level and the preponderance of evidence is convincing, then this standard is Met with Conditions.

31. You Review! Standard 4, Assessment 4.

32. Preservice teachers will:
NSTA Standard 5: Impact on Student Learning Effective teachers of science provide evidence to show that P-12 students’ understanding of major science concepts, principles, theories, and laws have changed as a result of instruction by the candidate and that student knowledge is at a level of understanding beyond memorization. Candidates provide evidence for the diversity of students they teach.
Preservice teachers will:
5a) Collect, organize, analyze, and reflect on diagnostic, formative and summative evidence of a change in mental functioning demonstrating that scientific knowledge is gained and/or corrected.
5b) Provide data to show that P-12 students are able to distinguish science from nonscience, understand the evolution and practice of science as a human endeavor, and critically analyze assertions made in the name of science.
5c) Engage students in developmentally appropriate inquiries that require them to develop concepts and relationships from their observations, data, and inferences in a scientific manner.
Assessment 5 – Evidence of P-12 Student Learning

33. Preponderance of Evidence: Std. 5
Programs shall meet two of the three elements of this standard at tan Acceptable level in order to meet the standard.
If one of the three elements is met at an Acceptable level and the preponderance of evidence is convincing, then this standard is Met with Conditions.
AS OF 3/30/16, edTPA may be used for meeting a preponderance of evidence for Standard 5.

34. You Review! Standard 5, Assessment 5.

35. Preservice teachers will:
NSTA Standard 6: Professional Knowledge and Skills Effective teachers of science strive continuously to improve their knowledge and understanding of the ever changing knowledge base of both content, and science pedagogy, including approaches for addressing inequities and inclusion for all students in science. They identify with and conduct themselves as part of the science education community.
Preservice teachers will:
6a) Engage in professional development opportunities in their content field such as talks, symposiums, research opportunities, or projects within their community.
6b) Engage in professional development opportunities such as conferences, research opportunities, or projects within their community.
Assessment 6 – Varies

36. Preponderance of Evidence: Std. 6
Programs shall meet one of the elements, at a minimum, of this standard, at an Acceptable level in order to meet the standard. Otherwise the standard is Not Met.
The only possible Met with Conditions decision is if insufficient candidate data are provided.

37. You Review! Standard 6, Assessment 6.

38. Objective 5
…recall four types of program report decisions.

39. Decisions for Programs
Nationally recognized
Nationally recognized with conditions (conditions must be specified, and must relate to the standards; 24 months to address conditions)
Further Development Required/Recognized with Probation (program has 24 months to meet SPA standards)
Not nationally recognized
Please use the time period as updated in the 2017 Guidelines. Programs now have 24 months to meet SPA Standards through resubmissions.

40. Data Rule Data Requirements:
For first submissions, minimum data required will be data resulting from two applications of the assessments
For resubmissions (Response to Conditions or Revised Reports), one application of new assessments

41. AIMS Organization: Programs & Shells
All Program Reports must be submitted through AIMS.
To submit a report, the institution must first add the program to their Manage Programs list.
Programs added to this list receive a program identification number.
This number lasts for the life of the program.
Program
(Ex: Secondary Science)
Program ID Number
(Ex: 123)

42. Shell Recognition Decision (Shell ID #) Program (Program ID#)
Once the program is added to AIMS, the institution is able to request a shell.
Shells are linked to the program.
Each shell comes a recognition decision, which is also linked to the program.
Program
(Program ID#)
Shell
(Shell ID #)
Recognition Decision

43. Nationally Recognized
Secondary Science
#123
Biology
#555
Nationally Recognized
Physics
#333
Recognized with Conditions
General Science
#111
Further Development Required

44. Linked Programs Shells can be linked if Key Assessments are the same.
Disaggregate data by:
Program level (Bac., Post Bac., and Maters)
Program type (bio., chem., etc.)
Year

45. CAEP website: (Standards, Program Standards, NSTA)
NSTA website: (Professional Development, NSTA Standards for Science Teacher Preparation)
CAEP website: (Standards, Program Standards, NSTA)
NCATE links may not be used. I replaced it with CAEP

46. Kennesaw State University
Questions?
Michael Dias
Kennesaw State University
(office)
(cell)