1. Massachusetts Department of Education FFY2002 Annual Report: Response to OSEP Letter Attachment 2B – Overview of Educator Database Planning Project Note: This presentation was prepared for the MASSDE Educator Database Planning Project subcommittee in January, 2005. The presentation provides a comprehensive overview of the MASSDE’s work-to-date on developing a comprehensive system for collecting educator data, including data on special education licensure and vacancy needs.

2. 1 Archived Table of Contents Introduction Project Drivers, Goals & Approach Project Team & Subcommittee Current State Educator Data Flow Methods, Systems & Data Future State Guiding Principles Vision Users Requirements Implementation Roadmap Next Steps

3. 2 Archived Project Drivers and Goals No Child Left Behind (NCLB) requires “highly qualified” teachers in all classrooms by the end of 2005-2006. In order to meet this requirement, states must collect and report on educator data. Today, limited data and capabilities exist to help MASSDE and districts meet this requirement. Better data and additional analytical capabilities will help to improve recruitment, deployment, retention, certification, and professional development of MA educators. Ultimately, this work could set the foundation for linking educator quality with exceptional student performance. The goals of the Educator Database Planning project included the following: GoalTimingWork Product Understand MASSDE’s current educator data and reporting capabilities. Weeks 1-2Data Collection and Dissemination Inventory Collect and document user needs for an educator database. Weeks 3-4User and Stakeholder Analysis Future Vision Requirements Analysis Develop options (including cost, time estimates, resource needs, and implementation phases) for MASSDE to meet these needs. Weeks 5-8Implementation Activities and Roadmap

4. 3 Archived Approach - System Development Life Cycle (SDLC) The System Development Life Cycle (SDLC) is a standard set of iterative steps used to develop, maintain and replace information systems. Sample Activities Planning Initiate project Analyze current environment Identify users Develop vision Collect and Analyze Requirements Develop strategy & implementation plan (roadmap) Identify project costs/benefits Design Define detailed system specifications and technical requirements Create design document Conduct JAD sessions Develop and deploy communication and change management plans Conduct usability testing Data migration planning Develop Develop/select product Test Document traceability Install Data migration Document Train User support plans Implement & Enhance Launch User support Monitor system performance Retire legacy systems Gather user feedback and metrics on performance Collect and evaluate change requests Select, design and implement enhancements Maintain, Evaluate & Enhance & EnhanceImplementationDevelopDesign Planning & Analysis We focused here

5. 4 Archived Project Team and Subcommittee Board of Higher Education Data Collection Commissioner’s Office Educator Licensure Educator Preparation and Quality Steering Committee met during our 8-week project: Subcommittee Kick-off 11/17 WP #3: DOE Roadmap 1/6 Project Timeline Jan 6, 2004 Current data, requirements, & capabilities User Needs and AnalysisOptions, recommendations, & roadmap Nov. 15, 2004 WP #1: Draft DOE Capability Baseline 11/29 WP #2: Draft Needs Analysis 12/13 Subcommittee session 11/22 11:30am Subcommittee session 12/8 10:00am Subcommittee session 12/1 10:00am Subcommittee session 12/15 11:00am Subcommittee session 1/6 10:00am Finance IT Project Management Mass. Assoc. of School Superintendents Mass. Teachers Retirement Board Special Education 22 interviews and numerous working sessions with internal and external staff to discuss various educator data components. Planning & Analysis Initiate project

6. 5 Archived Current State What is happening today with educator data at MASSDE?

7. 6 Archived Current State – Educator Data Flow Diversity of data “owners” supplying data Multiple submission timeframes (ongoing vs. periodic) and units (individual vs. groups) Various submission mechanisms Numerous systems that store educator data Numerous mechanisms for reporting Increased compliance requirements Increased interest in educator data The complex flow of educator data is a function of several factors: Data Dissemination /Reporting Data Collection Data Owners/Suppliers Federal Government Schools and Districts Public Legislature MA DOE Policy Grants Licensure Finance Program Dsn Ed. Prep Paper E-mail IVR Online file submission Online Web Online Application Board of Ed Educator Potential Hires Educator Prep Programs Analysis / Data Extraction Decision- Makers & Users Delivery Method/Channel Collection Method/Channel Data Categories ? Related Agency Systems MTRS HEIRS MTRB BHE Ad hoc data collection Ad hoc data reporting MA DOE Educator Data Security Portal Districts Schools Educators Teachers Paraprof. Management Educator Prep Program NES Greater Lowell CTE NASDTEC Online Web Paper Form Online Drop Box Online System Mail/Disk E-mail MA DOE Licensure PQA Prof. Dev. Licensing Certification Demo- graphic Employ- ment Financial Test Results Education Standalone Structured and Unstructured Educator Data PQA EQA School Performance Program Applications DSSR Financial System Directory Administration VES SIMS MCAS Results ELAR Licensing MECC GEM Vo Ed. Register Community Profiles Data Planning & Analysis Analyze Current Environment

8. 7 Archived Current State – Methods, Systems and Data Identified systems with educator data and the categories of data that reside in each system. Multiple systems house “pieces” of data across categories There are opportunities to streamline/ optimize data. Data Categories and Elements Identified what and how data is collected and reported. Numerous collection and reporting tools. Each serve unique purpose. There are opportunities to streamline collection processes and improve reporting. Data Collection and Reporting Inventory To form our understanding of the current flow of educator data, we reviewed all MASSDE methods for data collection and dissemination and analyzed where educator data resides. Planning & Analysis Analyze Current Environment

9. 8 Archived Future State What should the future of educator data at MASSDE be?

10. 9 Archived Future State – Guiding Principles Opportunities exist to improve the quality of educator data as well as the mechanisms for data collection, storage and dissemination. Planning & Analysis Develop Vision

11. 10 Archived Future State – Educator Data Flow Vision The future vision reflects additional suppliers of data submitting data through a streamlined process into enhanced MASSDE educator systems. The reporting side also includes increased functionality and additional consumers. Planning & Analysis Develop Vision Recommendations focused here For Future Consideration

12. 11 Archived Future State – Users of Educator Data PartnersInternalExternalStakeholders External individual or organization with “special” data sharing relationship with MASSDE thereby requiring appropriate access to educator data MA DOE employee or Department External individual or organization Individuals or organizations with an interest in the Educator Data collected by MASSDE. Approved Educator Preparation Programs Board of Higher Education Schools/Districts Educators MA Teachers Retirement Board (MTRB) Private Schools & Collaboratives Professional Development Providers Testing companies Board of Education Career and Technical Education (CTE) Commissioner’s Office Educator Licensure (Teacher Certification) Educator Preparation & Quality Finance Office of Accountability and Targeted Assistance (ATA) Special Education Policy & Planning PQA Student Assessment Federal Government General Public Local City/Town Management MA State Legislature / Governor’s office Media Office of Educational Quality & Accountability (OEQA) Parents Policy Groups Researchers MA Teachers Associations Mass. Teachers Association (MTA) Mass. Teachers Federation (MTF) MA Educator Associations MA. Assoc. of School Superintendents (MASS) MA. Assoc. of School Committees (MASC) MA Secondary School Administrators Assoc. (MSSAA) MA Elementary School Principals Assoc. (MESPA) MA Assoc. of School Business Officials (MASBO) We identified and grouped users of educator data into four categories based on their relationship to DOE. Planning & Analysis Identify Users

13. 12 Archived Future State – Requirements This list of questions are representative of what would we like to be able to answer using educator data. Since we have limited resources, we cannot address all 30 at once. Requirements/Questions 1What is an educator's FTE status? 2Is an educator "Highly Qualified"? (NCLB/IDEA) 3Is this paraprofessional "Qualified"? (NCLB) 4Is an educator licensed to teach in a particular subject matter? 5What licenses do educators have at school X? 6What licensure path/requirements are educator's using (aggregated by license type)? 7How many educators are working on a waiver? (by school, by district, by discipline) 8How many individuals who are granted waivers eventually get licenses? 9How many educator FTE salaries are funded from what sources (school committee appropriations and collectively federal grants, state grants and special funds) 10How do educators impact student performance? 11Does student performance differ when educators are teaching outside their licensed field? 12What is the retention rate for new and current educators? 13Are districts successful in filling educator vacancies in high need subject areas? 14Where are current shortage or excess areas by role, subject, location? 15Where are the projected shortage or excess areas by role, subject, location? 16What are the projected educator retirements in the next five years? (by field, by district, by license) 17Where are new educators coming from? (another district? higher ed? another state, field outside of education) 18What factors support teacher retention? 19Why are educators staying or leaving the field? 20How long do educator's stay in the education field? 21How does educator retention relate to educator preparation? 22Who is in the pipeline as a potential educator at institutes of higher education in MA? (totals, by degree, graduation date, location, program) 23How many people complete educator preparation programs in Massachusetts each year? 24How many Massachusetts educator prep program completers end up working in Massachusetts as educators after completing their degree? 25How many Massachusetts educator prep program completers do not work in Massachusetts as educators after completing their degree? 26What are our most successful educator preparation programs? Do completers of one program consistently outperform others (as measured by parent & staff satisfaction and student achievement)? 27How are newly licensed educators meeting the educational requirements for their licenses? 28What are most effective training and professional development programs? 29What is the demographic makeup of our Educator workforce? (age, race/ethnicity) 30What are the factors that contribute to student achievement? Planning & Analysis Collect and Analyze Requirements

14. 13 Archived Future State –Requirements We analyzed requirements by gauging the perceived benefit of answering the question and the level of effort required. Level of Effort - Criteria DefinitionRating Detail Business Complexity for Provider What level of effort will be required for the provider of the data to give the data to DOE to meet the requirement? Estimated (H, M, L) based on: How often is data required from provider? How easy is it to provide data to DOE? Business Complexity for DOE What level of effort will be required for the DOE to define and acquire the data from a provider to meet the requirement? Estimated (H, M, L) based on: How many parties are providing data? Does DOE need to define/frame data provided? What level of support will DOE need to provide? Technical Complexity for DOE What level of effort will be required technically to develop system to store and manipulate data to meet requirement? Estimated (H, M, L) based on: Technical complexity of new data elements required Number of data domains required Number of data elements required Benefit - CriteriaDefinitionRating Detail Compliance Captures whether the requirement helps DOE meet a Federal or State mandate H=Meets a Federal/State requirement L=Does not address a Federal/State requirement Decision Support Reflects how much this requirement supports better decision-making for DOE and users. H=Improves decision making M=May improve decision making L=Does not improve decision making User Demand Reflects the demand and urgency of requirement. H=High level of demand by one or multiple users (urgent need) M=Requested by multiple users L=Requested by one user Improves Educator Quality Captures whether the requirements helps to improve training, recruiting, retention, and/or training in an effort to improve overall educator quality. H=Directly improves educator training, retention, recruiting, support M=May improve educator training, retention, recruiting, support L=Does not impact educator quality Improves Student Achievement Captures whether the requirement helps to improve student achievement. H= Directly supports improved achievement M=May support improved achievement L=Does not impale student achievement Planning & Analysis Collect and Analyze Requirements

15. 14 Archived Level of Effort - Criteria DefinitionRating Detail Business Complexity for Provider What level of effort will be required for the provider of the data to give the data to DOE to meet the requirement? Estimated (H, M, L) based on: How often is data required from provider? How easy is it to provide data to DOE? Business Complexity for DOE What level of effort will be required for the DOE to define and acquire the data from a provider to meet the requirement? Estimated (H, M, L) based on: How many parties are providing data? Does DOE need to define/frame data provided? What level of support will DOE need to provide? Technical Complexity for DOE What level of effort will be required technically to develop system to store and manipulate data to meet requirement? Estimated (H, M, L) based on: Technical complexity of new data elements required Number of data domains required Number of data elements required Future State –Requirements To analyze the level of effort, each of our questions was divided into required data elements. The level of effort criteria are all based at the data element level. What is an educator's FTE status? Required Data Elements Educator ID Educator Role Educator District/School Educator FTE Educator Date of Hire Educator Termination Date Planning & Analysis Collect and Analyze Requirements For each data element: Do we have it? Which “domain” does it belong to?

16. 15 Archived Future State - View of Educator Data Assignment: Information about the position an educator holds and/or held in the past Development Activities: Information about the training and professional. development activities required and participated in Educational History: Information about the education an person obtained to become an educator Identity: Information about who an individual educator is Potential Jobs: Info about jobs available for educators. May also include info regarding future openings. Licensure: Information about the licenses required for educators and the licenses obtained Salary: Information about the funding sources for educator salaries and average salary amounts Students: Information about the students in Massachusetts’ educational system As part of analyzing the data elements, we categorized each of our data elements into our eight “data domains”. Planning & Analysis Collect and Analyze Requirements

17. 16 Archived Refer to slide with questions and numbers Future State – Requirements Using the matrix, we analyzed the questions grouped by quadrant. Planning & Analysis Collect and Analyze Requirements

18. 17 Archived Refer to slide with questions and numbers Future State – Requirements After analyzing the results, we realized how the requirements were grouped would present challenges if used to prioritize system development. 3 Domains Impacted Assignment Educational History Identity 7 Domains Impacted Assignment Development Activities Educational History Identity 7 Domains Impacted Assignment Development Activities Educational History Identity Licensure Potential Jobs Salary 3 Domains Impacted Assignment Educational History Identity Licensure Potential Jobs Students Planning & Analysis Collect and Analyze Requirements

19. 18 Archived Future State – Requirements An alternative method of prioritizing the requirements is based on analyzing the required data elements, focusing on data elements that are not currently being captured or “insufficient” as currently captured. Planning & Analysis Collect and Analyze Requirements Captured, but insufficient to answer requirement Educator Completed Professional Dev. Activities Educator Degree Educator District/School Educator FTE Educator Prep Program Completer ID Educator Race/Ethnicity Educator Role Open Position ID Open Position/Role District Open Position/Role in District Required Licenses for Open Positions/Role Captured Applicable License Paths District Requesting Waiver Educator Date of Birth Educator Ed Prep Program Type Educator Licenses Educator Licenses Applied For Educator Licenses Applied For Date Educator MTEL Test Results Educator Prep Program Educator Prep Program Completed Educator Prep Program Completer Graduation Date Educator Prep Program Completers Educator Prep Program Participants Educator Prep School Educator Waivers Federal-State Grants and Special Fund Licensure Activities Completed to Date Number of Educators Teaching Out of Field in District Number of FTE Paid From Each Funding Source Number of Long Term Substitutes in District Program Area (for Funding Purposes) School Appropriation Type Student Test Results Not Captured Educator Classes Taught Educator Date of Hire Educator Evaluation Data Education Staff ID Educator Leave Status Educator Prep Program Participant Expected Degree Educator Prep Program Participant Expected Graduation Date Educator Previous Position Educator Projected Retirement Date Educator Reason for Leaving or Staying Educator Termination Date Educator Years of Experience/Service Licenses Required for Role Potential Retention Factors Projected Candidates from Outside of Education Prep Programs Projected Open Position (Role) Projected Open Position School/District Required Licenses for Projected Open Positions/Role Student Classes Enrolled In Use Data Domains to Analyze These

20. 19 Archived Future State - Requirements New assignment data is required to answer most of the requirements. Additionally, an identifier is required to answer the questions on an individual level. Planning & Analysis Collect and Analyze Requirements Identity – 26 Requirements Assignment Data – 23 Requirements Education History – 12 Requirements Prof. Development – 5 Requirements Potential Job – 4 Requirements Student – 4 Requirements License – 3 Requirements

21. 20 Archived Roadmap How do we begin to move toward the future vision?

22. 21 Archived Roadmap - Purpose MASSDE is beginning to map the preliminary steps to go from current state towards the future vision Current State Roadmap Future Vision Planning & Analysis Develop Strategy and Implementation Plan

23. 22 Archived Roadmap – Next Steps Throughout this process, MASSDE has used the System Development Life Cycle (SDLC) as a framework for understanding the iterative steps used to develop, maintain and replace information systems. The below graphic presents a possible framework for next steps. Sample Activities Planning (CONTINUE) Initiate project Analyze current environment Identify users Develop vision Collect and Analyze Requirements Develop strategy & implementation plan (roadmap) Identify project costs/benefits Design Define detailed system specifications and technical requirements Create design document Conduct JAD sessions Develop and deploy communication and change management plans Conduct usability testing Data migration planning Develop Develop/select product Test Document traceability Install Data migration Document Train User support plans Implement & Enhance Launch User support Monitor system performance Retire legacy systems Gather user feedback and metrics on performance Collect and evaluate change requests Select, design and implement enhancements Maintain, Evaluate & Enhance & EnhanceImplementationDevelopDesign Planning & Analysis Next Steps Maintain, Evaluate & Enhance & EnhanceImplementationDevelopDesign

24. 23 Archived Next Steps Business Process Communication/ Change Mgmt GovernanceTechnologyPolicy Formulation Survey schools districts to better understand their capabilities and issues. Develop detailed processes on how education staff IDs could be assigned. Develop Communication strategy and begin engaging Schools and Districts Estimate and obtain funding for project Assemble Steering Committee and Project Team Evaluate options for Data Collection Mechanism Evaluate option for Assignment Database Initiate policy discussion about the need for MASSDE to collect individual Assignment data MASSDE has identified the following as possible next steps.