1. 2010-2012 TITLEIIA(3) IMPROVING TEACHER QUALITY COMPETITIVE GRANTS PROGRAM 1

2.  Evidence Partners Meet Federal Criteria  Student Data  Indication of student needs based on available test scores or other pertinent student data NEEDS DATA 2

3.  Participant Needs Survey Data  Standard Survey Form with place to add project-specific questions  Available for four core subject areas  Available in paper/pencil or electronic form  SAMPI compiles and returns data for use in proposal development (electronic survey only)  Submit summary of survey data and any additional data (student test scores, etc.) in narrative portion of proposal  Submit raw data with code numbers in database (data bases available for basic needs survey) NEEDS DATA 3

4.  Other Data  Provide data pertinent to partner needs if available  Previous Research/Evaluation Findings  Describe previous actions taken by IHE partner to address identified needs  Describe value of proposed intervention based on previous findings  Category 1: Describe results of previous TitleIIA(3) project on which proposal is based NEEDS DATA 4

5.  Teacher/School Commitment Forms  Informal “Understanding of Agreement” for teachers  Informal “Understanding of Agreement” showing partner school principal support  Participation in Technical Assistant Session— Part II  Prior to proposal submission to focus on needs data from survey and other NEEDS DATA 5

6. EVALUATION 6

7. 1)Improve programming through use of evaluation data and 2)Determine the impact of project and statewide effort on new and returning participants and identify strengths and limitations of the projects. EVALUATION GOALS: 7

8.  Use common cross-site pre/post surveys, pre/post lesson observations, interview sample of participants, collect teacher artifacts, gather evidence of impact on students  Use additional instruments/procedures specific to project as desired to determine impact or gather implementation data  Gather and report participation/program data  Support a person with adequate time dedicated to evaluation  External or internal, with appropriate expertise and without major program coordination duties PROJECT-LEVEL EVALUATION 8

9.  Coordinated by SAMPI—Western Michigan University  Advise on local project use of pre/post surveys, pre/post lesson observation, interviews, teacher artifact collection, student impact data collection procedures  Advise on evaluation as requested  Coordinate cross-site meetings (see below)  Conduct observation of project PD activities  Compile data from across projects  Prepare periodic statewide reports CROSS-SITE STATE-LEVEL EVALUATION 9

10. ELEMENTS OF THE EVALUATION 10

11.  Pre/post surveys (plus comparison with previous survey data for Category 1 projects)  Pre/post lesson observations (plus comparison with previous observation for Category 1 projects)  Sample of interviews of participants  Collect sample of teacher artifacts DETERMINE IMPACT ON TEACHERS/OTHER PARTICIPANTS 11

12.  Student test data as appropriate  Procedures to identify changes in student learning (pre/post tests, pre/post surveys, focus group interviews, assessment of student work)  Collect sample of teacher artifacts to show changes DETERMINE IMPACT ON STUDENTS OF PARTICIPATING TEACHERS 12

13.  1st six months—one face-to-face meeting in Lansing, one webinar  Remainder of project funding period—two face-to-face meetings in Lansing, two webinars PROJECT DIRECTOR AND PROJECT EVALUATOR PARTICIPATION IN REQUIRED CROSS-SITE ACTIVITIES 13

14.  Required Common Reporting  Participation levels and demographics, PD types, overall assessment of progress toward goals  Final evaluation report  End of project participant-specific data consistent with cross-site statewide data collection  Sharing findings at cross-site sessions REQUIRED COMMON REPORTING 14

15. END-OF-PROJECT REPORTING  Director Report  Use standard report format available in electronic version  SAMPI can provide Access database to projects to maintain participation records and to facilitate report preparation 15

16. PART #1 SAMPLE REPORTING TABLES CategoryNumberMajor Target Audience (Yes or No) 1. Number of different teachers served by the project 2. Number of different administrators served by the project 3. Number of different paraprofessionals served by the project 4. Number of different parents served by the project 5. Number of “others” served by the project 16

17. PART #2: DIRECTOR PERCEPTIONS OF PROJECT ACCOMPLISHMENTS  Rate progress toward outcomes  Provide evaluation/other evidence to support rating 17

18. PART #2: DIRECTOR PERCEPTIONS OF PROJECT ACCOMPLISHMENTS-SAMPLE FORM Section A: #1: Intended Outcome (Type in Outcome Statement): Rating of Progress Toward Outcome: Rate the degree of progress you believe was made in accomplishing this outcome on a 5-point scale, with 1 = no progress and 5 = fully accomplished. 12345 Evidence to Support Rating (Type in rational for your rating or, if pertinent, note that there is direct evidence in evaluation report section-see below): Discussion of Progress in Evaluation Section YesNoPage No. Complete the following tables, one for each of your intended outcomes as per your proposal. Outcomes are statements of intended impacts or results that will occur as a result of your professional development programming or other project interventions. 18

19. PART #3: NATURE OF PROFESSIONAL DEVELOPMENT/INTERVENTIONS  Hours, participants, schedule by type of intervention  Role of college faculty/content experts  Problems planning, implementing project  Problems recruiting teachers 19

20. PART #3 NATURE OF PROFESSIONAL DEVELOPMENT/INTERVENTIONS SAMPLE FORM PD Format No. Hrs. of this PD Provided Over Entire Project Participants for the PD (Key: T=Teachers, A=Administrators, PP=Para- Professionals, P=Parents, O=Others) When was this PD conducted? (Key: RSD=Regular School Day, AS=After School, EVE=Evening, SAT=Saturday, SUM=Summer) Workshops (usually one- or half-day sessions) Institutes (5 or more days usually in summer) College course work (for credit) E-Learning Courses (self paced web-based course) 20

21. NATURE OF PD/INTERVENTIONS, PART #3, CONTINUED - SAMPLE FORM NamePosition/ Institution Primary Role Hours of Involvement 3. Use the chart below to describe higher education faculty (both content and education faculty) or other external expert. PARTICIPATION IN YOUR PROJECT. 21

22. PART #4: REQUIRED COMPONENTS OF EVALUATION REPORT  Data collection  Progress toward project outcomes  Lesson observation data  Teacher and student artifacts  Impacts on participating students  Effectiveness of project partnership  Other 22

23. PART #4: REQUIRED COMPONENTS OF A PROJECT EVALUATION REPORT  Prepared by your internal evaluator  Based on their evaluation work  Evaluation reports will vary  Should be appropriately labeled as Core Evaluation Report Components 1 - 6. 23

24. PART #4: REQUIRED COMPONENTS OF A PROJECT EVALUATION REPORT, CONTINUED Core Evaluation Report Component 1: Data Collection  Describe the data collection activities that occurred over the course of the project. 24

25. PART #4: REQUIRED COMPONENTS OF A PROJECT EVALUATION REPORT, CONTINUED Core Evaluation Report Component 2: Progress Towards Project Outcomes  For each proposed outcome of the project, briefly summarize progress made toward its accomplishment based on evaluation findings. 25

26. PART #4: REQUIRED COMPONENTS OF A PROJECT EVALUATION REPORT, CONTINUED Core Evaluation Report Component 3: Lesson Observation Data  Detailed findings from lesson observations should be included in the evaluator report 26

27. PART #4: REQUIRED COMPONENTS OF A PROJECT EVALUATION REPORT, CONTINUED Core Evaluation Report Component 4: Lesson Observation Data  Detailed findings from lesson observations should be included in the evaluator report 27

28. Core Evaluation Report Component 5: Impacts on Students of Participating Teachers  For those projects that have gathered data related to impact on students, detailed findings should be presented in the evaluator’s report, if not part of the Progress Towards Project Outcomes (#2) above. PART #4: REQUIRED COMPONENTS OF A PROJECT EVALUATION REPORT, CONTINUED 28

29. PART #4: REQUIRED COMPONENTS OF A PROJECT EVALUATION REPORT, CONTINUED Core Evaluation Report Component 6: Effectiveness of Project Partnership  Briefly describe the effectiveness of the partnership in implementing project activities. Provide evidence for your findings. 29

30. OTHER REPORTING (OPTIONAL)  Report as appendix to core required report  Can include information Director wants to share about intervention, materials used, etc.  Can include additional evaluation or other pertinent data about project  Can include samples of teacher or student materials 30

31. QUESTIONS?  Contact information:  Kristin Everett – SAMPI, Western Michigan Unversity  kristin.everett@wmich.edu, (269) 387-3791 kristin.everett@wmich.edu 31