1. Welcome to the San Francisco Mathematics and Science Partnerships Regional Meeting March 21-23, 2011

3. MSP Grant and Funding Cycle States have 15 months to award funds on a competitive basis to partnerships consisting of STEM faculty at an IHE and a “high-need” local education agency. Funds are released to the states through a formula grant (number of students at poverty level) each July. Congress appropriates funds for the program. Projects submit annual/final reports to U.S. Department of Education within 60 days at the end of each 12-month reporting cycle. U.S. Department of Education Program Cycle States fund winning project proposals. States submit a copy of each funded proposal to U.S. Department of Education 30 days after award date.

4. The Annual Progress Report (APR) SEC. 2202.(f): Each eligible partnership receiving a grant or sub-grant shall report to the Secretary… Partnerships/projects must send their APRs to their State Educational Agency (SEA), the SEA must review and approve the APR, and submit it to the U.S. Department of Education (USED) within 60 days from the end of the performance period.

5. Overview of the APR Submission Process (Norm) SEA creates an APR for the project to complete The project completes the APR and submits it to the SEA by the indicated date. The SEA reviews the APR and determines if the APR is acceptable OR if revisions are needed. SEA sends the APR to the USED. The USED reports MSP results to Congress.

6. OR…Partial Submission (extraordinary) SEA creates an APR for the project to complete The project completes the APR and performs a partial submission and informs the SEA of when the APR will be completed by the indicated date. (Only with SEA approval). The SEA reviews the partially submitted APR and determines if the APR is acceptable OR if revisions are needed & accepts or negotiates the final submission date with USED input. SEA sends the APR to the USED. The USED reports MSP results to Congress.

7. Annual Performance Report 1.Thank you for your efforts in submitting project reports. 2.All but 5 States have reported 100% 3.As of last week, 5 States had a total of 16 APRs that had not been submitted

8. Partial Submission Partial submission should be used… ONLY When student or teacher data will not be available on time. When a partial submission is selected, you must contact ED or Abt staff to agree upon a new Revised Due to ED date. The Revised Due to ED date must be entered on the Administration website at www.ed-msp.netwww.ed-msp.net All sections of the APR must be completed and reviewed, other than the sections for which data are not available. Once the data are received, the project will only be able to edit the sections corresponding to the missing information they identified. Contact your ED program officer or msphelp@abtassoc.commsphelp@abtassoc.com

11. Helpful Hints for Administrators Features of the administrative page of the www.ed-msp.net website: www.ed-msp.net – Sorting capabilities – Viewing/editing project information – Updating project user information – Copying annual reports – Archiving annual reports

12. APR ID Sort

13. **Login name displayed

16. Create New APR Form = Copy Annual Report

17. When creating a new APR, State Coordinators should: 1.Enter the performance period. 2.Enter appropriate due dates. The due to ED date should be within 60 days of each 12 month period. 3.MUST enter project abstracts at that time.

18. Archive = Archive Annual Report

19. Selected APR Items – III.C. Delivering Professional Development – IV.C.1. Total Number of Teachers receiving PD – V.B. Types of PD Activities – VII.J. Evaluation Report – VIII.A.1. Teachers receiving PD in Math and Science – VIII.A. Administering more than one test – VIII.B. GPRA indicators of Student Achievement

20. Section III.C. asks about delivering PD. Projects can indicate that they have not yet delivered PD by checking the N/A box. This will allow them to skip subsequent questions about PD that are not relevant to the project.

21. Section IV.C.1 asks for the total number of teachers receiving MSP professional development in Math or Science; projects should NOT double-count teachers who receive PD in multiple content areas here. Additionally, questions D.1, E.1, F.1, and G.1 should sum to the total reported in IV.C.1.

22. Section V.B asks about the type of professional development activities. Projects should pay close attention to the definition of summer institute (i.e., minimally 2 weeks or 60 hours) and follow-up (i.e., at least 3 days in the classroom during the academic year). For example, if a project selects summer institute only, but reports 32 hours in V.A, this would be conflicting data entry. Similarly, if the project selects summer institute plus follow-up, the hours reported in V.B.i for the summer institute must be at least 60 hours.

23. Section VII.J is where all evaluation reports should be attached. If it is a final evaluation report, check the box.

24. SectionVIII.A.1 asks for total number of teachers receiving PD in Math or Science. Here, projects SHOULD double-count teachers who received PD in Math AND Science. SectionVIII.A asks for total number of teachers receiving PD in Math or Science. Here, projects SHOULD double-count teachers who received PD in Math AND Science.

25. Grantees who administer more than one test to the same group of teachers within the same subject (Math or Science) should select and report results for only one test. Grantees who administer more than one type of teacher content knowledge test to different groups of teachers within the same subject (Math or Science) (e.g. an algebra test to algebra teachers and a geometry test to geometry teachers) should complete a separate MSPTCK spreadsheet for each test administered and then sum the results to respond to APR items VIII 2, 3, 5, 6. Grantees who administer more than one test may report data from all assessments. Use a separate MSP TCK spreadsheet for each assessment and aggregate results to respond to APR Items VIII 2, 3, 5, 6.

26. In Section VIII.B, projects should report the number of students with assessment data in Math and Science, the number of students who scored at basic or below, and number of students who scored at proficient or above for Math and Science. These two categories are mutually exclusive within Math and within Science and should sum to the total number of students with assessment data in Math or in Science.

27. Reminders Please send ED a copy of all State RFP’s for inclusion in your State File. Please send ED a copy of all State Evaluation Reports for inclusion in your State File. Please notify ED when there is a change in SEA personnel. Please update your project files when there are changes in personnel.

28. The number of professional development hours remained about the same in PP07 and PP08. Total Median Hours of Professional Development, By Model Type, Performance Periods 2007–2008 Professional Development Model Median Number of Hours PP07PP08 Summer institute only 80 Summer institute with follow-up activities 10096 Focus on school year activities 7480 Source: Annual Performance Report item V.A.1, V.B(i).1, V.B(ii).1

29. E x h i b i t 2 3 T y p e s o f E v a l u a t i o n D e s i g n s U s e d b y P r o j e c t s, P e r f o r m a n c e P e r i o d 2 0 0 7 E v a l u a t i o n D e s i g n P e r c e n t o f P r o j e c t s ( N = 5 7 4 ) E x p e r i m e n t a l d e s i g n 2 % Q u a s i - e x p e r i m e n t a l d e s i g n 4 2 M a t c h e d c o m p a r i s o n g r o u p s 2 5 N o n - m a t c h e d c o m p a r i s o n g r o u p s 1 7 O n e - g r o u p p r e - p o s t 2 0 M i x e d m e t h o d s 1 4 Q u a l i t a t i v e d a t a o n l y 1 1 O t h e r 1 1 S o u r c e : A n n u a l P e r f o r m a n c e R e p o r t i t e m V I I. C T h e t o t a l n u m b e r o f p r o j e c t s w a s 5 7 5 i n P P 0 7. T h e n o n - r e s p o n s e r a t e w a s < 1 p e r c e n t. The percent of projects conducting experimental and quasi-experimental designs has increased in the past 2 years. In fact, in PP06, 1 percent of projects reported using experimental designs and 26 percent reported using quasi-experimental designs. Types of Evaluation Designs Used by Projects, Performance Periods 2007–2008 Evaluation Design Percent of Projects PP07 (N=574) PP08 (N=625) Experimental (random assignment) 2%3% Quasi-experimental 4249 Matched comparison groups 25 27 Non-matched comparison groups 17 22 One-group2025 Qualitative / descriptive 1412 Mixed methods 118 Other 113 Source: Annual Performance Report item VII.B

30. Overall Percent of Teachers with Significant Gains In Content Knowledge, of Those Teachers with Pre-Post Content Assessments, Performance Periods 2006–2008 Content AreaPP06PP07PP08 Mathematics71%68%67% Science8073 Source: Annual Performance Report items VIII.A. 2, 3, 5, 6 The percent of teachers with significant gains in content knowledge decreased somewhat in the past 2 years.

31. Overall Percent of Students Taught by MSP Teachers Scoring at Proficient Level or Above, Performance Periods 2006–2008 Content Area Proficient Level or Above PP06PP07PP08 Mathematics47%45%58% Science294958 Source: Annual Performance Report items VIII.B. 2, 3, 4, 6, 7, 8 The percent of students taught by MSP teachers scoring at proficient or above increased substantially last year in both math and science.

32. Program Officer Pat O’Connell Johnson patricia.johnson@ed.gov 202-260-7813

33. Mathematics and Science Partnership (MSP) Programs U.S. Department of Education San Francisco Regional Meeting March 21-23, 2011