Chemical, Bioengineering, Environmental, and Transport (CBET) Division Panel Program Director: _______________________ Program Assistant: ______________________ Number of Proposals: Number of Panels: Dates of Panels:

Outline Overview Administrative Points COI Confidentiality NSF Evaluation Criteria Panelist Responsibilities What Happens After the Panel? Introductions Thanks!

Safety Overview 4 Emergency exits (stairs) 2 exits Close to North and South elevators (close to restrooms) 2 other exits Restrooms located next to the elevators Visitor (ID) entrance (North Elevator, 1st floor) Exit towards North (9th Street and Stuart St) Towards Metro & Starbucks North elevator (2nd floor) Skywalk to Ballston Commons Mall Skywalk to Metro & Hilton

Administrative Points Sign-In Ensures reimbursement for each day served Check name and address for accuracy Please update contact information in FastLane Internet access: Wireless: take your computer to Room 357 in Stafford I for connection and laptop verification between 7 am and 9 pm EST, Monday - Friday For non-wireless LAN access: ensure your laptop meets the NSF External IT Screening measures outlined in the 07/24/06 Memo on the FastLane homepage; once verified, you can plug the LAN connection into directly your laptop.

Travel Questions Travel Travel should be arranged through the FedTravel Center (SATO) ( or ) Have you registered for this panel through FastLane? (Even if you have participated in previous NSF panels, you still have to register for this panel) EFT (electronic funds transfer) information must be provided Reimbursement will appear w/o notice to your specified financial institution and account ( U.S. Treasury - doesnt reference NSF) Reimbursement is considered taxable; NSF automatically sends a Form 1099 if $600 or greater is paid to a reviewer per calendar year Any problems to solve or to tell us?

Travel Questions (Continued) Reimbursement $480 for each meeting day and $280 for each travel day Local Participants $280 for each meeting day Did You Drive? Please complete the auto travel form.

Conflicts of Interest Sign and turn in Conflict-of-Interest form Typical relationships that could lead to a conflict: You must not participate in the discussion of any proposal for which you have a conflict. Please discuss any actual or perceived conflicts with your panel moderator. INSTITUTIONAL current or previous employment (12 months) or seeking employment award, honorarium, or travel payment (12 months) officer or governing board any financial interest PERSONAL co-author of paper or project collaborator (48 months) co-edited journal or proceedings (24 months) thesis advisor or student (life- long) family member or close friend

Confidentiality NSF receives proposals in confidence and is responsible for protecting the confidentiality of their contents and their review. Do not copy, quote, or otherwise use material from the proposals. Proposals contain sensitive information and are not in the public domain. Destroy all copies, including computer records, when you have completed your reviews. (You may leave your paper copies in the conference room.) Do not discuss proposal content, results, recommendations, or membership of this panel outside the meeting room, even at NSF. Except for copies to the Principal Investigator (excluding identifying information), reviews will not be disclosed to non- Governmental personnel. NSF considers reviews to be exempt from disclosure under the Freedom of Information Act, but it cannot guarantee that it will not be forced to release reviews under the FOIA or other laws.

NSF Evaluation Criteria Intellectual Merit Advancement and contribution of knowledge in its own field or across different disciplines? Creative and original concepts? Well-conceived and organized proposal? Qualification of the PIs? Sufficient access to resources?

NSF Evaluation Criteria Broader Impacts Advancement of discovery and understanding while promoting teaching, training, and learning? Benefits, as applicable, to society and industry? Educational Impact? Participation of underrepresented groups (e.g., gender, ethnicity, disability, geographic, etc.)? Enhancement of the infrastructure for research and education, such as facilities, instrumentation, networks, and partnerships? Dissemination of results?

NSF Evaluation Criteria Additional NSF considerations Integration of Research and Education One of the principal strategies supporting NSF's goals is to foster integration of research and education through the programs, projects, and activities it supports at academic and research institutions. These institutions provide abundant opportunities in which individuals may concurrently assume responsibilities as researchers, educators, and students and in which all can engage in joint efforts that infuse education with the excitement of discovery and enrich research through the diversity of learning perspectives. Integrating Diversity into NSF Programs, Projects, and Activities Broadening opportunities and enabling the participation of all citizens – women and men, underrepresented minorities, and persons with disabilities – is essential to the health and vitality of science and engineering. NSF is committed to this principle of diversity and deems it central to the programs, projects, and activities it considers and supports

Avoid Unintended Bias Implicit bias toward a group Lack of critical mass greater reliance on perceptions and generalizations Few women and minorities in sciences Accumulation of disadvantage Mitigate evaluation bias

Ways to Mitigate Evaluation Bias 1. Increase awareness of how implicit bias might affect evaluation 2. Decrease time pressure and distractions in evaluation process 3. Rate on explicit criteria rather than global judgments 4. Point to specific evidence supporting judgments Bauer & Baltes, 2002, Sex Roles, 47 (9/10), Please incorporate (3) & (4) in your discussions.

Panelist Responsibilities Ensure your reviews are entered in the Panel System correctly and are entered prior to the panel meeting Minimum of 3 reviews via FastLane for each proposal (OK to modify reviews, including change of overall rating) You may change your reviews during the meeting: However, modifications MUST be done BEFORE leaving the panel

Panelist Responsibilities For each proposal: Primary reviewer (lead) summarizes and then initiates comments on the proposal. The lead also reviews the proposal. One reviewer will be the scribe for a panel summary, to which all assigned reviewers provide input. The scribe can also be the lead. The panel summary generally reflects the panels discussion and the individual reviews and basis for recommendation. The summary should be written in 3rd-person. Other reviewers concur and/or add their comments. The floor is open for panel discussion. Once the final overall panel ranking is formalized, please sign on the ranking sheet.

Panelist Responsibilities Final Recommendation to the Program Director Place each proposal into rating categories outlined by the Program Officer, normally: For example, HR - Highly Recommend for funding (optional) R - Recommend, if funds are available DNR - Do Not Recommend for funding Rank the proposals within the categories as outlined by the Program Officer, if applicable

A Good Panel Summary Leads to Better Research! Objectives of the proposal Intellectual merit Strengths Weaknesses Broader impact Strengths Weaknesses Panel Summary Statement (optional) Panel recommendation and rationale Ensure that this statement agrees with the overall panel discussion and ranking sheet

A Good Panel Summary Leads to Better Research! The scribe should follow the Panel Summary Template while he/she writes the panel summary. Primary and secondary reviewers edit for substance and tone to develop a summary reflecting the consensus of the panel. Comments should be constructive, informative, non-inflammatory, and non-discriminatory. Finalize reviews and panel summaries before you leave.

Post-Panel Actions Reviews and panel summaries are important feedback to PIs. PI will receive: All reviews Panel summaries Program Directors analysis and recommendation (award/decline) Expected awards to be recommended to the Division Director for funding

Introductions

Lets Get Started! Thanks for participating in this panel!

Appendix I: Sample Panel Summary Template

Appendix II: Sample Panel Flow Chart