Presentation on theme: "MSP Program Evaluation"— Presentation transcript:
1MSP Program Evaluation Louisiana Math and Science Partnership ProjectMSP Regional Conference – ChicagoMarch 2008
2Louisiana MSP Project 2006-07 Content-rich professional development in math and science27 regional projectsCollaborations with 39 school systems, 4 archdioceses, 9 universities and 6 community resource centers600 teachers statewide15,237 students directly impactedBrief Background
3Types of ProjectsMiddle School – Math, Science, Integrated Math-ScienceAll designed differently to meet needs of districtsHigh School Algebra I/Physical ScienceState focus – project content coordinatedEnd of course test – Algebra IElementary Grades 3-4 Math Science (added )High stakes testing in grade 4Elementary projects added in
4Components of Evaluation Teacher Content AssessmentOn-line SurveyState Data SystemTwo components – Teacher content assessment and surveysimilar to components we have used in other professional development projectsrelatively easy to conduct and compileeach has been a little further refinedThe third component – difficult but critical in the evaluation procedure
5Teacher Content Assessment Pre-tests and post-tests for content knowledgeSubmitted to Assessment Research and Technology Section of the Louisiana Department of EducationHigh School and Elementary ProjectsSame Test – state-widePaired comparison T-testAll tests - Drafts are submitted to LDE and revisedMS tests are individual for each projectHS and Elementary Projects – University Faculty and district personnel developed statewide testsPaired comparison T-test was used to determine the statistical significance of the differences between the mean scores.A 17 point gain was found on the average score in 26 projectsTwenty-five programs showed statistically significant gains
6On-line Survey Pre-survey and post-survey on-line Multiple choice and open-ended responsesTeachers’ demographic background, educational level, teaching experiences, previous professional development opportunities, level of knowledge or ability in teaching math/science, personalRating of training experienceFeedback about programs (strengths and weaknesses)Comfort level in teaching math/science after trainingadministered to teacher participants before and after the summer institutes and at end of year
7Key QuestionCan teacher participation in the MSP program transfer effectively into the classroom and improve student learning?
8How to Address Key Question? StandardAssessment of teacher content knowledgeQualitative data from teacher surveysDifficultCollection and analysis of student data
9State Data SystemCreate on-line state system compatible with state test data basesRequire district agreement to provide access to dataIdentify students who are taught by teachers participating in the MSP programsConstruct comparison groups using students who are similar in demographic background but who were not taught by MSP teacher participantsAccess student data on high-stakes test for comparisonMSP created on-line data system that could access information in other state test data basesCondition of funding - district agreement to provide access to dataTeachers in project build their on-line data bases, selection of “their” studentsStudent performance can be viewed as a function of teachers' training experiences.Comparison groups – comparison of students of teachers in the project with students of teachers not in the project. All schools in which a MSP teacher participant taught math or science were selected for the study. Students who enrolled in these schools but were not instructed by a MSP teacher constituted the comparison group.
10Data Analyses Mean scaled scores Percent of students scoring “proficient” on the Louisiana standardized testsProgram level and state level analysesReport by grade and content areaStudent demographics and comparison groupsproficient - Basic or aboveAt the state level, regression analyses were used to determine the effects of the MSP program, students’ educational classification, SES, and ethnicity on the test scores.
11Student Demographics Math Science Number Percent Grade 5 1,330 9% 1,40510%64,04127%3,52624%72,60018%3,72625%84,19929%3,68492,54217%2,41816%GenderMale7,62750%7,711Female7,5837,587EthnicityNative American1281%670%Asian176170African American7,77951%7,10146%Hispanic3082%342White6,83445%7,636Free/Reduced Lunch9,68966%9,39663%Statewide Total15,237100%15,329
13Scores by Project - Grade 6 iLEAP Taught by Non-MSP ParticipantTaught by MSP ParticipantMean ScoreDifferenceProject NameTotalNumberEBR High School1626552275+10EBR Middle School Math1634278756288Iberville High School206250135282+32Iberville Middle School Math115254198292+38Jefferson MSMSP455256296+40Livingston Math Institute217298312315+17Math TIPS6525298281+29Meaningful Math through Modeling492293330+22Monroe City Impacts Math186294249291-3Northwest M&S Middle School856285978321+36Project Prism27164307+19SAVE Math246300333325+25Concordia140128299+34Parish Math Program1022875+70Statewide Total4,9092793,990305+26