1. Introduction to the MSP Management Information System Molly Hershey-Arista December 16, 2013

2. Overview of the MSP MIS  Web-based monitoring system –Initiated in 2004 by NSF and Westat –Collects common data from all MSP projects –Developed with considerable input from program officers and project PIs –Current database contains annual information dating back to the 2002-03 school year (year in which MSP program was initiated) 2

3. What is the purpose of the MSP MIS?  Collect annual data from all MSP projects  Assess the implementation and reach of the overall MSP program  Document the contributions of IHE faculty to MSP projects  Examine specific student outcomes  Monitor progress of individual MSP grants 3

4. MIS Surveys  Annual collection –Opens in the fall –Open for approximately four months –2012-13 collection opened Nov 4 and closes Feb 28  Comprised of four surveys –Survey for Partnership Projects –K-12 District Partner Survey –IHE Partner Survey –IHE Participant Survey 4

5. Survey for Partnership Projects  To be completed annually by the PI  Collects overall project-level data –Project partners –Project activities –Challenges  Allows PI to manage the data collection through an Administrative Screen –Register partners and participants to complete their own surveys –View completion status of all surveys –Perform a final submit of all surveys 5

6. IHE Participant Survey  To be completed annually by each: –STEM or Education faculty member –High-level administrator (e.g., dean, department chair) –Who participated in the MSP during the collection year  Collects participant-level data –Individual demographics –Estimate of the number of participation hours –Activities –Most significant contribution to the MSP –Any knowledge or experience gained through participation 6

7. IHE Partner Survey  To be completed annually by each IHE Partner  Collects IHE organizational-level data –Number and type of IHE individuals involved in MSP –Number and type of IHE individuals who were recipients of MSP services  Collects data on MSP-supported pre-service courses –Subject matter of the course –Undergraduate or graduate level –Course enrollment data 7

8. K-12 District Survey  To be completed annual by each K-12 District Partner  Collects district-level data –Number of participants in the development/delivery of MSP –Number of recipients of MSP professional development  Collects school-level data –Basic information (e.g., school name, grades served) about all schools participating in MSP –Additional data required from schools with more significant participation in MSP (criteria are specified in the survey) 8

9. School-level Information  Number of math and/or science teachers participating in MSP  Grade enrollment at targeted grades  High school math and science course data –Which math and science courses were offered –Number of students enrolled in each course –Number of students passing each course  Math and science assessment data –Number of students taking each assessment –Number of students scoring at or above proficient  Adequate Yearly Progress in math 9

10. Technical Assistance  Westat provides technical assistance throughout the collection cycle –Lessons Learned document is provided at the beginning of the collection –Assistance is provided by phone and email How to interpret survey questions How to report for special situations Troubleshooting project-specific issues Any questions you may have! 10

11. Data validation  Edit checks within the online system check for: –Completeness on each item –Columns and/or rows add to total Examples: Gender break-down and race break-down add to total; total above proficient and total below proficient add to total assessed –Consistency across survey items Example: 100 students are reported enrolled in 6 th grade, but 500 students took the 6 th grade math assessment  We might have questions for you  If errors are discovered in future years, we will work with you to correct previous years’ data 11

12. Reporting our findings  Annual Compendium Report to NSF –Presents aggregate data –Includes data from Comprehensive, Targeted and Institute projects –Approximately 150 data tables –2011-12 Compendium will provide 10 years of trend data 12

13. Other use of MSP MIS data  Thumb drive for NSF –Supplementary tables by cohort –Supplementary tables by project  Quick turnaround analyses for NSF on specific topics  Data are shared with national program evaluators  Excel files of your project-level data are available for your project to download through the MIS website 13

14. Lessons Learned  Start early! –Make sure your partners understand their responsibilities –Provide your partners with a copy of the survey now –Do not wait until the last few weeks to begin data entry  Who is in the best position to enter school-level data? –Data entry by each district –Data entry by MSP project staff  Build historical memory about the MSP MIS –Aim for consistency in how data are reported across years 14

15. Please call us! Molly Hershey-Arista 1-800-937-8182 x4908 Ben Korelitz 1-800-937-8182 x4459 mspmis@westat.com 15