Alabama Professional Education Personnel Evaluation Program Field Test of Superintendent System July 2000

Superintendent Evaluation System Purpose of Evaluation is for Improvement Alabama Department of EducationSuperintendent Field Test Evaluation Criteria Superintendent system was developed from a Framework Framework contains Competencies, Tasks, and Knowledge/Skills Currently being field tested

Superintendent Evaluation System 13 Knowledge/Skill Areas 1. Communication 2. Collaboration 3. Assessment/Evaluation 4. Organization 6. Laws/Policies 7. Problem Solving 8. Innovation 10. School System Management 11. Fiscal Management/Leadership 9. Technology Management 13. Leadership of Human Resources 12. Professional Responsibilities 5. Planning Alabama Department of EducationSuperintendent Field Test Evaluation

Superintendent Evaluation System Scoring 4-point rating scale with rationale statements (rubrics) Holistic scoring cross-cutting 13 knowledge/skill areas Evaluators Trained in conducting evaluations Must meet reliability standards Superintendents Evaluation content varies with years of experience in position Evaluation process is same for all Alabama Department of EducationSuperintendent Field Test Evaluation

Superintendent Evaluation System 1. Evaluation Summary Report (shows scores and score rationales by knowledge/skill area) 2. Professional Development Plan (becomes a data source in next cycle) Instrumentation/Data Sources 1. Portfolio developed by superintendent 2. Surveys of various school system administrators and principals 3. Interviews (written or oral) 4. Board Questionnaire Summary Report Sources Optional Instrumentation 1. Self-Assessment Form (completed by superintendent, but not considered in evaluation scoring) Alabama Department of EducationSuperintendent Field Test Evaluation

Superintendent Evaluation System Superintendent Data: evidence of competence produced, assembled and submitted by the superintendent  Portfolio  Written interview  Interpretation of survey responses  Line of Evidence: all data/information supporting the score awarded in a knowledge/skill area, whether produced by the superintendent or the evaluator  Evaluator Data: evidence of superintendent competence developed by the evaluator  Oral Interview  Clarifying interview  Board Questionnaire  Alabama Department of EducationSuperintendent Field Test Evaluation

Steps in Evaluation Process 1. Orientation (LEA responsibility) 2. Self-assessment (superintendent) 3. Prepare portfolio (superintendent) 4. Distribute and analyze surveys (superintendent) 5. Board Questionnaire (board, evaluator) Superintendent Evaluation System Alabama Department of EducationSuperintendent Field Test Evaluation

Steps in Evaluation Process (cont.) 7. Conduct interviews (evaluator/superintendent) 8. Prepare Evaluation Summary Report (evaluator) 9. Summary conference and PDP (evaluator/ superintendent) Superintendent Evaluation System 6. Review portfolio (evaluator) Alabama Department of EducationSuperintendent Field Test Evaluation

Self-Assessment  Generates data for all areas of knowledge and skills  Optionally used by the superintendent and shared with evaluator only if superintendent agrees  Used by superintendent to: 1)Identify areas that need improvement 2)Compare the superintendent’s perceptions of performance with the results of the evaluation 3)Determine areas for professional growth activities to be included in a professional development plan. Superintendent Evaluation System Alabama Department of EducationSuperintendent Field Test Evaluation

 Two types of interviews:  A Structured Interview with 6 questions commonly asked of superintendents (oral or written response format)  A Clarifying Interview with probing questions focused on clarifying or bringing understanding to the portfolio and/or written interview (questions differ from superintendent to superintendent)  Conducted after the evaluator analyzes superintendent’s portfolio  Modified scripting is used to record responses Interviews Superintendent Evaluation System  Responses can be banked for one year Alabama Department of EducationSuperintendent Field Test Evaluation

 Oral Option: A discussion between the superintendent and the evaluator (or his/her designee) about particular areas of the superintendent’s knowledge, skills, and performance. Structured Interviews Superintendent Evaluation System  Generates data for seven areas of knowledge/skills:  Communication (oral)  Collaboration processes and skills  Assessment/measurement/evaluation  Organizing for results  Planning  Problem solving  Technology management Alabama Department of EducationSuperintendent Field Test Evaluation

 Each of the structured interview question sets is intended to cover one knowledge and skill area (although responses may cut across more than one area). Communication skills are considered as part of the oral presentation.  The structured interview itself is not scored, but responses are added to other evidence provided through other sources. Structured Interviews (cont.) Superintendent Evaluation System  Written Option: Superintendent’s written responses to interview questions (will have clarifying discussion). Alabama Department of EducationSuperintendent Field Test Evaluation

 Conducted after the structured interview is completed with a focus on the written interview and, optionally, the superintendent’s portfolio  No set number of questions, but key questions specific to the written interview and portfolio (optional) are developed by the evaluator to bring clarity and understanding of their contents  Responses are added to other evidence provided through other sources The Clarifying Interview Superintendent Evaluation System Alabama Department of EducationSuperintendent Field Test Evaluation

 Generates data for all areas of knowledge and skills  Questionnaire returned to state pool evaluator of superintendent  Used in superintendent system as a replacement for Supervisor’s Review Form  Questions relate to areas with which board members should be knowledgeable, although a “don’t know” response is allowable Alabama Department of EducationAdministrator Evaluation The Board Questionnaire Administrator Evaluation Systems

 Superintendent is responsible for providing some evidence of his/her knowledge and skills  The totality of the evidential material (lines of evidence for all knowledge/skill areas) will constitute superintendent’s evaluation portfolio The Superintendent Portfolio Superintendent Evaluation System  Generates data for 12 areas of knowledge and skills:  Communication  Collaboration  Assessment/Evaluation  Organizing for Results  Planning  Laws and Policies  Problem Solving  Innovation  Technology Management  School System Management  Fiscal Leadership  Human Resources Leadership Alabama Department of EducationSuperintendent Field Test Evaluation

 There is no one specific form of packaging required for the portfolio, but there are organizational requirements  Each entry (piece of evidence) is accompanied by a Portfolio Entry Form Superintendent Evaluation System  Total portfolio requires approximately 28 pieces of evidence  One piece of evidence might serve 2-3 purposes (e.g., Program Improvement Plan)  With multiple purposes, a portfolio may have entries (pieces of evidence) Alabama Department of Education The Superintendent Portfolio Superintendent Field Test Evaluation

 Explicit guidelines for developing a line of evidence in each area are provided in the evaluation manual "Extra" entries will not be considered by the evaluator   Evaluated on the quality of materials rather than the quantity of materials in the portfolio Superintendent Evaluation System Alabama Department of Education The Superintendent Portfolio Superintendent Field Test Evaluation

 Respondents are to remain anonymous and represent a cross-section within the group  Generates data for 12 areas of knowledge and skills:  Communication  Collaboration  Assessment/Evaluation  Organizing for Results  Laws and Policies  Problem Solving  Innovation  School System Management  Fiscal Leadership  Technology Management  Professional Responsibilities  Human Resources Leadership Surveys Superintendent Evaluation System  Two groups of respondents:  Central Office Administrators  Principals Alabama Department of EducationSuperintendent Field Test Evaluation

 Analyses and reporting are responsibilities of superintendent:  Information/report will be used with other evidence from other sources  Procedures and methodologies used by the superintendent will in itself be used as part of evidence for assessing the knowledge/ skill area of assessment/evaluation  Analyses and reporting:  Number of respondents by subgroup surveyed  Any problems with data  Conclusions by knowledge/skill area  Patterns and interpretation Surveys (cont.) Superintendent Evaluation System Alabama Department of Education  Numbers for administration:  20 or all, if fewer in school system  Both system administrators and principals must be in respondent group Superintendent Field Test Evaluation

 Scores are not generated by instrument  There will be a total of 13 scores, one for each knowledge/skill area  At end of process, evaluator(s) will: 1.Organize evidence by knowledge/skill area 2.Read through all evidence for a knowledge/skill area 3.Award a score (1, 2, 3, 4) based on the patterns in the evidence in relation to the score level rationales for that particular knowledge/skill area 4.Place score with comments/justification on the Evaluation Summary Report 5.Use the scores and evidence for determining what should be addressed in the Professional Development Plan Scoring Superintendent Evaluation System Alabama Department of EducationSuperintendent Field Test Evaluation

 Used to present the scores developed from the lines of evidence for each of the 13 knowledge and skill areas  To complete the Evaluation Summary Report, the evaluator: 1.Completes demographic information requested 2.Records the scores developed from the lines of evidence for each knowledge and skill area 3.Writes any comments, explanations, or justifications in the space provided that support the score for the specific area 4.Identifies three areas for focus in planning professional development for the coming year(s) Evaluation Summary Report Superintendent Evaluation System Alabama Department of EducationSuperintendent Field Test Evaluation

 Activities focus on areas of improvement as identified in the Evaluation Summary Report  Activities focus on the improvement of student achievement and school improvement  Used to generate portfolio entry for Human Resources Leadership after first year of evaluation Professional Development Plan Professional Development Plan Superintendent Evaluation System Alabama Department of EducationSuperintendent Field Test Evaluation

QUESTIONS ?

Administrator Evaluation Systems Alabama Department of EducationAdministrator Evaluation Instrument SuperintendentCentral OfficePrincipalAsst. Principal Self Assessment Portfolio Surveys Interviews Supr. Review Board Quest Observation Eval. Summary Prof. Devel. XXXX XXXX XXXX XXXX XXXX X XXX XXXX X

Administrator Evaluation Systems Alabama Department of EducationAdministrator Evaluation Instrument SuperintendentCentral OfficePrincipalAsst. Principal Self Assessment Portfolio Surveys Interviews Supr. Review Board Quest Observation Eval. Summary Prof. Devel. 12 know/skills 11 know/skills 12 know/skills All know/skills Cntrl Off/Prin Cntrl Off/Prin/ Faculty 7 know/skills 8 know/skills 7 know/skills Faculty/Parents Students/Comm Faculty/Parents Students All know/skills 6 know/skills 4 know/skills 6 know/skills 5 know/skills All know/skills 2-3 know/skills

Superintendent Portfolio Knowledge Skill AreaEvidence  Communication  3 pieces of written communication  Collaboration  3 artifacts (documents, tapes, etc.)  Assessment/Evaluation  1 personnel evaluation file  1 document/disk synthesizing and presenting student achievement results  1 document related to program evaluation  Organizing for Results  2 documents/artifacts representing marketing knowledge/skills, customer focus and/or change efforts 1

 Planning  Strategic Plan or evidence of planning for school system improvement  1 other document/artifact  Laws/Policies  2 artifacts/documents  Problem Solving  2 artifacts/documents  Innovation  2 artifacts/documents 2 Knowledge Skill AreaEvidence Superintendent Portfolio  Technology Management  2 artifacts/documents

 Leadership of Human Resources  2 artifacts/documents 3 Knowledge Skill AreaEvidence  Fiscal Management/Leadership  2 artifacts/documents Superintendent Portfolio  Program Operations/Management  2 artifacts/documents