Presentation on theme: "Statistical Modeling for Education Planning URBPL 5/6020 / April 19, 2007."— Presentation transcript:
Statistical Modeling for Education Planning firstname.lastname@example.org www.schools.utah.gov/finance URBPL 5/6020 / April 19, 2007
Who We Are Utah State Office of Education – Staff to the State Board of Education Financial and Business Services Division Finance and Statistics Section
What We Deal With Populations – Students – Staff – Schools Finance – Minimum School Program (MSP) Budget & NCLB Allocations – Financial Reporting & Auditing – Property Tax Operations – School Facilities – Student Transportation – Safety
How We Do It Acquire – Data Allocate – Money to local education agencies according to data Audit – For accuracy of data and appropriateness of expenditures
Analytics Cycle Population (Enrollment) Projections – How many people do we need to serve? Fiscal Impact Analysis – How much will service options cost? Formula Allocation – How do we get the right amount to the right place? Compliance Audit – How well did service providers follow the rules? Program Evaluation – How well did we serve the population?
Enrollment Projections: Institutional Context Common Data Committee – Legislative Fiscal Analyst – Governors Office of Planning and Budget – Utah State Office of Education Current Work – By county (then allocate to districts and adjust for charter schools) – In October – Single year (to next October) – Agreed upon figures for legislative session Future Plans – Multiyear project with GOPB using REMI for Baseline 2008
Enrollment Projections: Model Cohort Progression Participation Ratio – Kindergarten subset
Enrollment Projections: Data Historical Variables – Total Enrollment (Current and Prior Years) – Grade 12 Enrollment (Current and Prior Years) – Kindergarten Enrollment (Current Year) – Births (by Month, 4- and 5-Years Prior) Intermediate Variable – Projected Kindergarten Enrollment
Enrollment Projections: Formula FormulaElement E Y+1 = E Y Base Population + (B Y-4 * (K Y / B Y-5 )) - G Y )Cohort Progress. + (E Y - E Y-1 ) - (K Y - G Y-1 ) Implied Migration
Fiscal Impact Analysis: Example HB 222 (2002) – Make recommendation on … the ideal size of schools districts in this state … Optimization Problem Cost Function – Relates expenditures per student to enrollment (main cost driver) controlling for academic achievement (output measured in quality)
Fiscal Impact Analysis: Design Sample:Cross section of 40 Utah school districts Data:Superintendents Annual Report, 2000-01 Model:Y = m + (b 1 X + b 2 X 2 ) + b 3 Z + e Procedure:OLS regression Fit (adj R 2 ):.29 Predictors:EnrollEnroll 2 Lexile Coeff (b):-.138.0000016-665 Sig (p):.01.05.03
Fiscal Impact Analysis: Results Empirical Cost Function – exp =.0000016enr 2 -.138enr – 665lex + 6,468 Differentiated and Set Equal to Zero – 0 =.0000032enr -.138 Solution is Optimal Size – enr =.138/.0000032 = 43,125 students
Fiscal Impact Analysis: Politics (TTC, SL Tribune 2/23/04) Columbia University professors critique: – Id be happy to go with the [USOE] analysis rather than the fiscal analysts, which is opaque to the point of incomprehensibility Fiscal analysts defense: – Anybodys guess is as good as the next persons Opponents critique: – Foes have long accused the fiscal analysts office of working the numbers to achieve a favorable outcome Fiscal analysts concession: – At the outset, the intention is to have it come out in a positive way so theres not a cost
Fiscal Impact Analysis: Ethics Substantive claims must be warranted by evidence Production of evidence must be based on transparent procedures
Allocation Formulas: Minimum School Program (1) The purpose of this chapter [Utah Code 53A-17a] is to provide a minimum school program (MSP) for the state in accordance with the constitutional mandate. It recognizes that all children of the state are entitled to reasonably equal educational opportunities regardless of their place of residence in the state and of the economic situation of their respective school districts or other agencies. NOTE: Overriding concern with equity; adequacy is another issue of growing legal importance, but its operationalization is very unclear.
Allocation Formulas: Minimum School Program (2) It further recognizes that although the establishment of an educational system is primarily a state function, school districts should be required to participate on a partnership basis in the payment of a reasonable portion of the cost of a minimum program. NOTE: Utah sources of revenue (FY 2006): – State 55% from income tax – Local 36% from property tax – Federal 9% from who knows where
Allocation Formulas: Minimum School Program (3) Each locality should be empowered to provide educational facilities and opportunities beyond the minimum program and accordingly provide a method whereby that latitude of action is permitted and encouraged. NOTE: Local school boards can impose several additional property taxes for specified educational purposes.
Allocation Formulas: Budgeting for Basic Program (1) Majority of funding is based on Prior Year + Growth formula Prior year is Average Daily Membership (ADM) Growth is percentage difference between projected Fall Enrollment and current year Fall Enrollment Hold harmless in case of negative growth
Allocation Formulas: Budgeting for Basic Program (2) Result is number of Weighted Pupil Units (WPUs), a quantification of the basic service which each local education agency is obligated to provide Legislature sets monetary value of WPU every year Total WPUs times WPU $ value determines basic appropriation
Allocation Formulas: Budgeting for Basic Program (3) If LEA property tax revenue cover its obligation, then buck stops there; otherwise, state pays balance from income tax revenue In practice, all LEAs need some state assistance and appropriations often fall short, so funds are prorated according to WPUs Since FY 2001, K-12 funding has approximately kept pace with inflation
Allocation Formulas: Categorical Programs In addition to the basic program, the Legislature has established dozens of categorical programs to address particular concerns Cost drivers of categorical programs can be quite complex Special Education Add On is an especially striking example of what can happen when trying to reconcile competing interests through a funding program
Allocation Formulas: Categorical Program Example (1) Per WPU, which is the greater of the average of Special Education (Self Contained and Resource) ADM over the previous 5 years (which establishes the foundation [hold harmless] below which the current year WPU can never fall) or prior year Special Education ADM plus weighted growth in Special Education ADM.
Allocation Formulas: Categorical Program Example (2) Weighted growth is determined by multiplying Special Education ADM from two years prior by the percentage difference between Special Education ADM two years prior and Special Education ADM for the year prior to that, subject to two constraints:
Allocation Formulas: Categorical Program Example (3) Special Education ADM values used in calculating the difference cannot exceed the prevalence limit of 12.18% of total district ADM for their respective years. If this measure of growth in Special Education exceeds current year growth in Fall Enrollment, growth in Special Education is set equal to growth in Fall Enrollment (incidence limit).
Allocation Formulas: Categorical Program Example (4) Finally, growth is multiplied by a factor of 1.53. This weight is intended to account for the additional cost of educating a special education student. However, the weight is not based on an empirical analysis of the cost of special education relative to "regular" education.
An Australian Approach: Victorias Principles (1) Preeminence of Educational Considerations – Elimination of disparities reflecting historical and political decisions for which there is no current or future educational rationale Cost Effectiveness – Relativities among allocations should reflect knowledge of efficient ways of achieving school and classroom effectiveness
An Australian Approach: Victorias Principles (2) Fairness – Schools with the same mix of learning needs should receive the same total resources; this requires accurate and comprehensive information on those student characteristics which best predict academic achievement Transparency – Basis for allocations should be made public and readily understandable by all with an interest
An Australian Approach: Victorias Principles (3) Subsidiarity – Decisions on resource allocation should be made centrally only if they cannot be made locally Accountability – A school that has authority to make decisions on how resources will be allocated should be accountable for the use of the resources, including educational outcomes in relation to learning needs
An Australian Approach: Simple Budget Structure (1) Core Funding – Grade Level – School Size Student Disadvantage – Disabilities – Special Learning Needs – English as Second Language – Rurality and Isolation
An Australian Approach: Simple Budget Structure (2) Facilities (operation & maintenance) Administration Costs outside of control of schools – e.g., Transportation to and from school Priority Programs – Money for politicians of the day to play with
An Australian Approach: Special Learning Needs Sample (83 schools; 7,233 students) Hierarchical linear & Structural equation modeling Demographic index to predict achievement: – Poverty (qualified for education welfare payment) – Parental occupation (skill level) – Language spoken at home (other than English) – Family composition (two parent, one parent, none) – Aboriginality (= Alaska Native or American Indian) – Transience (recently changed schools, = Mobility)
An Australian Approach: Reference Hill, Peter W. (1996). Building equity and effectiveness into school based funding models: An Australian case study. 18p. http://nces.ed.gov/pubs97/97535i.pdf
Compliance Audit: Purpose Provide reasonable assurance that local education agencies are correctly applying State Board of Education rules in accounting for their students Statistical summaries from individual data files serve as written management assertions Auditors follow agreed upon procedures
Compliance Audit: Sampling Efficient auditing depends on selection of sample appropriate to purpose For example, if you want to adjust statistics based on audit, you need a probability sample The right sample size then depends on: – Variation in the population – Risk you are willing to take of being wrong
Sample Size: The Price of Precision pop = 80,000; mean = 154; sd = 25 90%95%99% 1%70310011718 5%294171 10%81118
Program Evaluation: Regression with Treatment as Dummy In the actual practice of applied social science, the most common mode of causal inference, the most common quasi- experimental design … (Cook & Campbell) Crucial to valid interpretation: – Specification of correct theory (of nonrandom selection process) as represented by equation – (Near) perfect measurement of variables
Program Evaluation: Recommendation Consider path analysis as extension of regression: – Explicit theory of how program works as a causal mechanism in form of path diagram Consider multiple indicators of each variable: – Use factor analysis to obtain composite measure representing only common variance In short, poor mans structural equation modeling
Program Evaluation: Bibliography Fitzpatrick, Sanders & Worthen (2004) Program Evaluation: Alternative Approaches and Practical Guidelines – LB2822.75.W67 Patton (1997) Utilization Focused Evaluation – H62.5.U5 P37 Mohr (1995) Impact Analysis for Program Evaluation – H97.M64 Cook & Campbell (1979) Quasi Experimentation: Design and Analysis for Field Settings – H62.C5857 Scriven (1991) Evaluation Thesaurus – AZ191.S37
Some Education Data Issues Is a Navajo living in a hogan homeless? Kanab is on the urban fringe of which city? Who decides the racial identity of a student? When is a person who leaves school without graduating not a dropout? Is being in a single parent family a reliable indicator of being at risk of low academic performance?