1. Noyce Program Participatory Evaluation FRANCES LAWRENZ JIM APPLETON MARJORIE BULLITT BEQUETTE ANN OOMS DEENA WASSENBERG University of Minnesota

3. What Existing Research Shows Attention to recruitment and retention has varied over time Different types of incentives to affect recruitment and retention have been tried Alternative certification is one type of incentive and has a variety of definitions Salary total amount and comparatively is a clear incentive for STEM teachers Retention in a school is highly dependent on the context of the school (supportive colleagues and administration, optimism and focus on student success)

4. Theoretical Framework: Noyce Logic Model –The Noyce Program Ideal Depicted by the main path as well as bold headings preceded by addition signs (e.g., “+Plan to teach”) –Decision points en route to becoming a STEM teacher Indicated by diverging routes from the main path describing alternative options and the Noyce Ideal in bold headings Dashed boxes denote retention/recruitment by school or program –Important STEM major decision factors Influenced by attributes of the candidate, pre-service program, and school/district (depicted as bulleted lists on the main path) Depicted as thought bubbles emerging from the decision point

5. Logic model

6. Evaluation Variable Data: Summary Program itself: many are gathering data on recruitment, demographics, and student performance; fewer on program retention Post-program monitoring: several to many are gathering data on teacher effectiveness, monitoring/fulfillment of scholarship requirements, school/district retention, teacher transition experiences/support, and inter-program/institution coordination; fewer on school/district recruitment and characteristics (including teaching assignment characteristics) Ways of gathering data: mostly formative and summative program effectiveness, and self-report (caution)

7. Use of Noyce Monitoring Data Whether the institution uses selected activities such as mentoring, early field experiences, etc. Information on the students in the program –e.g., the numbers of students graduating, the numbers of career changers Information on the applicants for the Noyce awards –e.g., numbers, GPA, support provided Information on the faculty involved in the program –e.g., numbers, discipline areas and roles. Information about both current and prior recipients of the Noyce funding. This includes –demographic information, GPA, prior background, intended teaching areas and levels, levels of funding, and completion dates. –prior recipients also includes if and where they are teaching and if so, what. List of the districts involved with the Noyce program.

8. Ideas for an Evaluation of Noyce Precise definitions of variables/separate district and program recruitment and retention Numbers and demographics at each stage in the process of developing a teacher Description of the experience at each stage Assessment of the state of the candidate (performance, attitudes, plans, beliefs) at each stage Examination of interactions Longitudinal studies of persistence, activity and impact

9. Issues Raised at Conference Thank You! Recruitment both program and district Retention in district (program?) Training-practice alignment Sustainability Definitional concerns e.g.,LEA, consistent Type of evidence Community building Long term considerations

10. Existing Resources: Some copies at Tables Evaluation instruments/processes presently used in Noyce CETP instruments LSC instruments CCSSO instruments Others??

11. Steps in Planning an Evaluation Brainstorm evaluation questions Focus through: audiences, existing research and data, importance, and effort Determine information necessary to answer the questions Establish indicators for the information including data sources and collection processes Gather, analyze and share the information

12. Group Work—15 Minutes: recruitment, retention, programs, and partnerships Brainstorm potential evaluation issues/questions about your topic Prioritize the list Speculate on information needs to answer the questions and possible indicators Consider possible data sources, processes, and instruments Prepare a 5 min summary for full group

13. Closure and Next Steps We will combine and post the suggestions on our web site We will structure continuing discussions with smaller groups of people on targeted issues via web or telephone (volunteers) We will host another virtual conference in early fall to discuss group progress and next steps

14. Contact Us! Frances Lawrenz lawrenz@umn.edu Marjorie Bullitt Bequette marjb@UMN.EDU R 3 URL: http://education.umn.edu/EdPsych/NOYCE/ Additional input to the site is welcomed