1. Overview of NSF Education R & D Programs with an Emphasis on the TUES Program Louis Everett Susan Finger leverett@nsf.gov sfinger@nsf.gov Sue Fitzgerald John Yu scfitzge@nsf.gov rpimmel@nsf.gov March 6 and 7, 2012

2.  Most of the information presented in this workshop represents the presenters’ opinions and not an official NSF position  Local facilitators will provide the link to the workshop slides at the completion of the webinar.  Participants may ask questions by “raising their virtual hand” during a question session. We will call on selected sites and/or enable their microphones so that the questions can be asked. You may also type questions into the chat box.  Responses will be collected from a few sites at the end of each Exercise. At the start of the Exercise, we will identify these sites in the Chat Box and then call on them one at a time to provide their responses. 2

3.  Learning must build on prior knowledge ◦ Some knowledge is correct ◦ Some knowledge is incorrect – misconceptions  Learning is ◦ Connecting new knowledge to prior knowledge ◦ Correcting misconceptions  Learning requires engagement ◦ Actively recalling prior knowledge ◦ Sharing new knowledge ◦ Forming a new understanding

4.  Effective learning activities ◦ Recall prior knowledge – actively, explicitly ◦ Connect new concepts to existing ones ◦ Challenge and alter misconceptions  Active & collaborative processes ◦ Think individually ◦ Share with partner ◦ Report to local and virtual groups ◦ Learn from program directors’ responses 4

5.  Coordinate the local activities  Watch the time ◦ Allow for think, share, and report phases ◦ Reconvene on time – 1 min warning slide ◦ With one minute warning, check Chat Box to see if you will be asked for a response  Ensure the individual think phase is devoted to thinking and not talking  Coordinate the asking of questions by local participants and reporting local responses to exercises 5

6.  GOAL: ◦ Participants should develop a better understanding of NSF undergraduate education R&D programs and the TUES program in particular  OUTCOMES: ◦ Can identify undergraduate R&D programs and find details on NSF website ◦ Can describe the kinds of projects that are appropriate for the TUES program ◦ Can describe the NSF review criteria ◦ Can describe important features of TUES proposals and effective approaches for addressing them 6

7.  What NSF programs support undergraduate educational R&D? ◦ Name the program, and briefly describe the type of projects supported and any special focus  Be brief ◦ Do not include scholarship programs or those focused on increasing diversity or general participation  Total time – 2 minutes ◦ Identify as many as you can in your local group  Watch time and reconvene after 2 minutes 7

8. 8

9. One Minute Look at Chat Box to see if you will be asked to respond

10.  Programs that provide funding that impacts undergraduate students are listed at: ◦ http://www.nsf.gov/funding/education.jsp?org=DRL&fund_type=1  A more comprehensive list of education R&D programs can be found here: ◦ http://www.nsf.gov/funding/pgm_list.jsp?org=EHR 10

11.  EHR/DUE (Division of Undergraduate Education ◦ Transforming Undergraduate Education in Science, Technology, Engineering and Mathematics (TUES) Transforming Undergraduate Education in Science, Technology, Engineering and Mathematics (TUES) ◦ Advanced Technological Education (ATE) Advanced Technological Education (ATE ◦ Federal Cyber Service: Scholarship for Service (SFS) Federal Cyber Service: Scholarship for Service (SFS)  EHR/DRL (Division of Research and Learning) ◦ Research and Evaluation on Education in Science and Engr (REESE) Research and Evaluation on Education in Science and Engr (REESE)  ENG/EEC (Engineering Education and Centers) ◦ Research in Engineering Education (REE) Research in Engineering Education (REE) ◦ Research Initiation Grants in Engineering Education (RIGEE) Research Initiation Grants in Engineering Education (RIGEE  CISE/CNS (Computer and Network Systems) ◦ Computing Education for the 21st Century (CE21) Computing Education for the 21st Century (CE21)  Cross-Cutting (Multiple Directorates) ◦ Cyberlearning and CAREER 11

12.  TUES – Focuses on all undergraduates ◦ Curriculum development (materials and instructional approaches) ◦ Faculty development ◦ Assessment and evaluation development and studies ◦ Applied education research  ATE – Focuses on technician education and two-year schools ◦ Curriculum development ◦ Faculty and teacher development ◦ Applied education research  SFS – Focuses on cybersecurity ◦ Capacity building track includes curriculum and faculty development 12

13.  Vision for TUES Program: Excellent STEM education for all undergraduate students

14.  What kinds of proposals are appropriate for the TUES Program? Define them in terms of a single goal statement.  Total Exercise Time6 min ◦ Think individually ~2 min ◦ Share with a partner ~2 min ◦ Report in local group ~2 min  Watch time and reconvene after 6 min  Use THINK time to think – no discussion, Selected local facilitators report to virtual group  With one minute warning, check Chat Box to see if you will be asked for a response 14

15. 15

16. One Minute Look at Chat Box to see if you will be asked to respond

17.  The TUES program seeks to ◦ improve the quality of STEM education for all undergraduate students, ◦ especially encourages projects that have the potential to transform undergraduate STEM education, ◦ for example, by bringing about widespread adoption of classroom practices that embody understanding of how students learn most effectively 17

18.  Create Learning Materials and Strategies: ◦ Guided by research on teaching and learning ◦ Incorporating and inspired by advances within the discipline  Implement New Instructional Strategies: ◦ Which contribute to understanding on how existing strategies  Can be widely adopted  Are transferred to diverse settings  Impact student learning in diverse settings  Develop Faculty Expertise: ◦ Enable faculty to acquire new knowledge and skills in order to revise their curricula and teaching practices ◦ Involve a diverse group of faculty

19.  Assess and Evaluate Student Achievement: ◦ Develop and disseminate valid and reliable tests of STEM knowledge ◦ Collect, synthesize, and interpret information about student understanding, reasoning, practical skills, interests, attitudes or other valued outcomes  Conduct Research on Undergraduate STEM Education to Explore How: ◦ Effective teaching strategies and curricula enhance learning and attitudes, ◦ Widespread practices have diffused through the community ◦ Faculty and programs implement changes in their curriculum

20.  Type 1 ◦ Total budget up to $200,000 for 2 to 3 years  $250,000 when 4-year and 2-year schools collaborate  Type 2 ◦ Total budget up to $600,000 for 2 to 4 years  Type 3 ◦ Budget negotiable -- not to exceed $5,000,000 over 5 years  Central Resource Projects ◦ Small focused workshop projects—budget negotiable—up to $100,000 for 1 to 2 years ◦ Large scale projects—budget negotiable—$300,000 to 3,000,000 for 3 to 5 years

21.  Uses a new instructional approach based on how students learn  Introduces content from current research  Integrates new instrumentation that demonstrably improves student learning  Provide the needed courses for a seamless 2-4 year transfer  Integrates current science and pedagogy into the teacher preparation curriculum  Novel methods for faculty professional development  Develops an instrument to assess students’ knowledge or attitude  Understand factors affecting how students learn particular content or skills

22.  A project that develops materials that use a new instructional approach based on the current understanding of how students learn, or introduces content from current research into an existing course.  A project that integrates new instrumentation or equipment into undergraduate laboratories or field work in a way that demonstrably improves student learning.  A collaborative project between faculty from two-year and four-year schools that develops a model to provide the needed courses for a seamless transfer in an efficient way.  A pilot project that explores the practical aspects of using remote laboratories or instruction among several institutions.  A pilot project that integrates current science and pedagogy into the teacher preparation curriculum.  A pilot study to explore Internet-based approaches for faculty professional development.  A project that develops an instrument to assess students’ knowledge in a particular area, their abilities with certain processes, or their attitude about some aspect of STEM.  A pilot study to begin understanding how various factors affect how students learn particular content or skills.

23.  … at several institutions …  … involving several diverse partnerships …  … part of a widespread beta–testing effort … faculty in several diverse institutions … disseminate proven …  … converts an effective … to improve accessibility and sustainability

24.  A project that develops material for a sequence of courses that vertically integrates a conceptual or pedagogical approach at several institutions.  A project involving several diverse partnerships between community colleges and four-year schools to develop robust models for providing community college courses needed for a true two-plus-two transfer program.  A project that uses faculty professional development as a part of a widespread beta–testing effort with faculty in several diverse institutions in order to disseminate proven, innovative instructional material or approaches.  A project that converts an effective, in-person faculty professional development approach to an Internet-based or blended approach in order to improve accessibility and sustainability.  A project involving several diverse institutions that uses an existing instrument to assess students’ knowledge in a particular area or their abilities with certain processes.  A study involving several diverse institutions to identify what factors and characteristics affect how faculty members and departments adopt innovative approaches.

25.  … involves a regional or national effort to disseminate proven materials or pedagogies  … develops a self-sustaining model for faculty professional development …  … involving a wide range of diverse institutions … systematically compares the efficacy and efficiency of …

26.  A project that involves a regional or national effort to disseminate proven materials or pedagogies.  A project that develops a self-sustaining model for faculty professional development that introduces new faculty to a field or provides retraining for experienced faculty.  A national or regional level project involving a wide range of diverse institutions that uses an existing assessment instrument to develop a database on students’ knowledge in a particular area or their abilities with certain processes.  A study involving a broad range of diverse institutions that explores how various factors affect how students learn particular content or skills.  A study involving a broad range of diverse institutions that systematically compares the efficacy and efficiency of several instructional methodologies such as hands-on, remote, and virtual laboratories.

27.  Projects that organize and implement meetings of Principal Investigators of projects funded by the TUES program and its predecessors.  Projects that conduct targeted research or evaluation studies in undergraduate STEM education addressed by TUES projects, the impact of TUES supported activities or a subset of awards. Proposals should state questions to be addressed, describe study design and methodology, and draw on relevant literature.  Projects that develop an approach for describing or characterizing the portfolio of TUES. Proposals should describe strategies and apply new techniques for organizing the characterization, for collecting the information, and for reporting and presenting the results.  Projects that provide leadership and implementation in seeing to it that development of the TUES community of practice is supported by current cyber tools for communication and collection of resources. Systems should integrate with NSDL.  Workshops that increase potential and current PIs’ understanding of various topics such as conducting project evaluations, broadening participation, utilizing cyberinfrastructure, and incorporating engaging pedagogies.

28.  Organize and implement meetings of …  Conduct targeted research or evaluation studies of …  Develop an approach for describing or characterizing the portfolio of …  Workshops that increase understanding of project evaluations, broadening participation, utilizing cyberinfrastructure, and engaging pedagogies of … … TUES projects! Other examples can be found in the solicitation

29.  For Type 1 -- states or territories beginning with A through M. ◦ May 28, 2012  For Type 1 -- states or territories beginning with N through W. ◦ May 29, 2012  For Type 2 and 3 and Central Resource Projects ◦ January 13, 2012 ◦ January 14, 2013  Central Resource Project proposals for small focused workshops ◦ Submitted at any time after consulting with a program officer

30.  What review criteria are used in reviewing TUES proposals?  Total time — 2 min ◦ Discuss in local group  Watch time and reconvene after 2 min 30

31. 31

32.  What review criteria do you think should be used in reviewing TUES proposals?  Every proposal submitted to NSF is reviewed on two criteria: ◦ Intellectual Merit ◦ Broader Impact 32

33.  What are some of the questions or factors that reviewers use in determining ◦ Intellectual merit? ◦ Broader impacts?  Total Exercise Time6 min ◦ Think individually ~2 min ◦ Share with a partner ~2 min ◦ Report in local group ~2 min  Watch time and reconvene after 6 min  Use THINK time to think – no discussion, Selected local facilitators report to virtual group  With one minute warning, check Chat Box to see if you will be asked for a response

34. 34

35. One Minute Look at Chat Box to see if you will be asked to respond

36. Questions “Hold-up your virtual hand” and you will be called upon after we unmute your mike. You may also type questions into your chat box. 36

37.  Will the project ◦ Include activities important in advancing knowledge? ◦ Involve qualified proposer(s)? ◦ Contain creative, original, and potentially transformative concepts? ◦ Have a well conceived and organized plan? ◦ Include sufficient access to resources?

38. Will the project  Produce one or more of the following: ◦ Exemplary materials, processes, or models that enhance student learning and can be adopted by other sites ◦ Important findings related to student learning?  Build on existing knowledge about STEM education?  Have explicit and appropriate expected measurable outcomes integrated into an evaluation plan?  Include an evaluation effort that is likely to produce useful information?  Institutionalize the approach at the investigator's college or university as appropriate for the Type

39.  Will the project ◦ Advance discovery - promote teaching & learning? ◦ Broaden participation of underrepresented groups? ◦ Enhance the infrastructure? ◦ Include broad dissemination? ◦ Benefit society?

40. Will the project  Involve a significant effort to facilitate adaptation at other sites?  Contribute to the understanding of STEM education?  Help build and diversify the STEM education community?  Have a broad impact on STEM education in an area of recognized need or opportunity?  Have the potential to contribute to a paradigm shift in undergraduate STEM education?

41. Questions “Hold-up your virtual hand” and you will be called upon after we unmute your mike. You may also type questions into your chat box. 41

42. BREAK 15 min

43. BREAK 1 min remaining

44.  How do you convince a reader that your project will ◦ Enhance student learning ◦ Build on existing knowledge about STEM education ◦ Contribute to the understanding of STEM education ◦ Have a broad impact on STEM education in an area of recognized need or opportunity  What should you include in the proposal’s rational background, and justification sections?  Total Exercise Time6 min ◦ Think individually ~2 min ◦ Share with a partner ~2 min ◦ Report in local group ~2 min  Watch time and reconvene after 6 min

45. 45

46. One Minute Look at Chat Box to see if you will be asked to respond

47.  Prior work ◦ By others -- Referenced to the literature ◦ By applicant –prior results or preliminary data ◦  Relevant theory ◦ Referenced to the literature (How People Learn)  Importance of the problem ◦ Incorporates new disciplinary knowledge ◦ Addresses an emerging area or known problem ◦ Meets an industry need  Potential impact of the work ◦ Number of students ◦ Transportable to a large number of institutions ◦ Serves as model for other areas

48.  How do you convince a reader that your project ◦ Can be adopted by other sites ◦ Contains a significant effort to facilitate adaptation at other sites ◦ Has the potential to contribute to a paradigm shift in undergraduate STEM education  What is your dissemination plan?  Total Exercise Time6 min ◦ Think individually ~2 min ◦ Share with a partner ~2 min ◦ Report in local group ~2 min  Watch time and reconvene after 6 min  Use THINK time to think – no discussion, Selected local facilitators report to virtual group

49. 49

50. One Minute Look at Chat Box to see if you will be asked to respond

51.  Build in transportability ◦ Develop an outward focus – how will approach work elsewhere ◦ Develop a product attitude instead of a course focus  Use active promotion approaches ◦ In addition to standard approaches (websites, conference papers, and journal articles) ◦ Add broader approaches  Workshops and webinars  Specialty websites and list servers – e. g., Connexions site  Commercialization of products  Target specific audiences ◦ Engage a few faculty members at other institutions  To periodically critique, advise, adapt and adopt materials

52. Questions “Hold-up your virtual hand” and you will be called upon after we unmute your mike. You may also type questions into your chat box. 52

53.  How do you convince a reader that your proposal has ◦ Explicit and appropriate expected measurable outcomes ◦ An evaluation plan that is likely to produce useful information  What are the characteristics of good goals and outcomes?  What should be addressed in an evaluation plan?  Total Exercise Time6 min ◦ Think individually ~2 min ◦ Share with a partner ~2 min ◦ Report in local group ~2 min  Watch time and reconvene after 6 min  With one minute warning, check Chat Box to see if you will be asked for a response

54. 54

55. One Minute Look at Chat Box to see if you will be asked to respond

56.  Goals: define your ambition or intention ◦ What is your overall ambition? ◦ What do you hope to achieve? ◦ Goals provide overarching statements of project intention  Characteristics: ◦ General and broad ◦ Abstract ◦ Intangible ◦ Not readily measurable Example: The goal of this project is to improve student learning in engineering mechanics 56

57.  Expected outcomes ◦ Identify the observable changes in behavior if goal is obtained ◦ One or more specific observable results for each goal  How will achieving your “intention” reflect changes in student or faculty behavior?  How will it change student learning or attitudes?  Characteristics ◦ Narrow and more specific ◦ Concrete ◦ Tangible ◦ Measurable Example:  Improve student learning gain by 15%  Improve faculty performance as measured through students evaluation and peer assessment

58.  Evaluator expertise and independence  Evaluation questions ◦ Derived from the expected outcomes  Methods ◦ Tools (instruments) and protocols (timing, etc) ◦ Data analysis and interpretation  Confounding factors – alternate explanations ◦ Approaches for minimizing their impact  Formative and Summative Evaluation

59.  How do you convince a reader that your project helps build and diversify the STEM education community?  Total Exercise Time6 min ◦ Think individually ~2 min ◦ Share with a partner ~2 min ◦ Report in local group ~2 min  Watch time and reconvene after 6 min  Use THINK time to think – no discussion, selected local facilitators report to virtual group  With one minute warning, check Chat Box to see if you will be asked for a response

60. 60

61. One Minute Look at Chat Box to see if you will be asked to respond

62. ◦ Describe the planned activities ◦ Explain the rational behind these activities ◦ Identify expected measureable outcomes ◦ Include evaluation strategies to monitor progress

63.  What ideas in this presentation surprised you?  Total time 4 min ◦ Think individually ~2 min ◦ Report in local group ~2 min  Watch time and reconvene after 4 min  Use THINK time to think – no discussion, selected local facilitators report to virtual group  With one minute warning, check Chat Box to see if you will be asked for a response

64. 64

65. One Minute Look at Chat Box to see if you will be asked to respond

66. Questions “Hold-up your virtual hand” and you will be called upon after we unmute your mike. You may also type questions into your chat box. 66

67.  To download a copy of the presentation go to: http://www.nsflsu.com  Please complete the assessment survey go to: http://www.nsflsu.com