2006 ANNUAL MEETING Arlington, Virginia INDUSTRY / UNIVERSITY COOPERATIVE RESEARCH CENTERS PROGRAM Pre-Session: ROLE OF EVALUATOR 8:00 - 9:00 am Thursday, January 12, 2006 NATIONAL SCIENCE FOUNDATION

PANEL l ERIC SUNDSTROM - Facilitator. NSF IUCRC Evaluator, University of Tennessee, Knoxille l RONALD R. BECK - NSF IUCRC Evaluator, U.S. Army TARDEC (Retired) l DONALD R. PRICE - Executive Director, Center for Engineering Logistics & Distribution, University of Florida l JANE LAUX - Business Manager, Connection One: Communication Circuits & Systems Research Center, Arizona State University l JENNIFER SCHNEIDER. NSF IUCRC Evaluation Program, North Carolina State University

AGENDA Ronald R. Beck ROLE OF EVALUATOR: BASIC DUTIES Donald R. Price NSF PERSPECTIVE ON THE ROLE OF EVALUATOR Jane Laux CENTER OPERATIONS AND THE ROLE OF EVALUATOR Jennifer Schneider...IUCRC EVALUATION PROGRAM AND THE ROLE OF EVALUATOR Questions & dialogue

ROLE OF EVALUATOR: BASIC DUTIES RON BECK NSF IUCRC Evaluator

NSF Center Evaluator l Independent role for an NSF Industry / University Cooperative Research Center l Evaluate Center structure, process, results, adherence to Program policy & practices l Support the NSF I/UCRC Evaluation Project via data- collection, analysis, and reporting l Represent NSF I/UCRC Program at Center l Represent Center to NSF I/UCRC Program

NSF Evaluator Duties l Prepare the annual Evaluator Report for NSF l Represent NSF at the Center's annual Spring & Fall Industrial Advisory Board meetings l Administer NSF Industry Questionnaire & Faculty Questionnaire in Fall meeting; Analyze, archive, and report to the Center l Represent the Center at annual I/UCRC Director & Evaluator meetings in Arlington l Advise Center Director on I/UCRC practices l Do Exit Interviews with Industry sponsors

Evaluator Calendar Jan. - I/UCRC Director / Evaluator meeting, Arlington March- Evaluator Report for NSF renewal / proposal April- Spring Industrial Advisory Board meeting: LIFE Forms; I/UCRC research report to Center June- Annual I/UCRC Evaluator Meeting, Arlington Sept.- Fall planning meeting with Center Director(s), University site visits, exit interviews Oct.- Fall Industrial Advisory Board meeting: Industry / Faculty Questionnaires; LIFE Forms

Annual Evaluator Report l CENTER'S CURRENT STATUS: University sites, NSF Funding, Leadership, Members, Budget l OBJECTIVES & GOALS: Center's mission,... l ENVIRONMENTAL-INSTITUTIONAL CONTEXT l ORGANIZATION: Structure, Personnel l RESEARCH PROGRAM: Themes, projects l CENTER ACCOMPLISHMENTS: Knowledge products, Technology Transfer, Education

LIFE Forms: Level-of-Interest Feedback Evaluation l Member Representative rates organization's interest in each Center project, comments l Comments address scientific merit, design, future plans, offers of help, mentoring... l Board reviews LIFE comments, level-of- interest ratings, discusses projects l Member Representatives guide the research agenda, allocate member $ dues to projects

NSF Industry Questionnaire & Faculty Questionnaire l I/UCRC Evaluation Project at North Carolina State University l Nation-wide, longitudinal, evaluation of Center & Program processes, outcomes l Evaluator administers at Center's Fall meeting l Evaluation Project reports national results l Center results reviewed at Spring meeting

ROLE OF EVALUATOR: NSF PERSPECTIVE DON PRICE Executive Director, Center for Engineering Logistics & Distribution, University of Florida

GENESIS OF I/UCRC EVALUATORS Process began at the NSF with Alex Schwarzkopf in the early 1980’s Alex wanted a measurement of the performance of I/UCRC centers Denis Gray was recommended to Alex by an NSF manager who directed a program to study innovation Thus began a long relationship with N.C. State University and Dr. Denis Gray The I/UCRC Program has as its primary mission to build research partnerships between Universities and Industry. Evaluators collect data to measure this success

NSF PERSPECTIVE Why Evaluators?? To answer questions of program accountability Build data base of accomplishments and to provide historical perspective To give important feedback to NSF Program Directors Provide data to assist in NSF budget requests from Congress To develop a background of data to assist in the justification for the I/UCRC Program

NSF PERSPECTIVE: Expectations of Evaluators To arrange for all necessary data collection required by the NSF or requested by the N.C. State group Attend IAB meetings and manage the collection of feedback by use of LIFE forms Provide a summary of information collected from the LIFE forms to the IAB for review at the meeting Collect any additional data as requested by Center Director or N.C. State group Send out and collect all survey information requested

NSF PERSPECTIVE What is not Expected To be technically competent to assess research quality or performance within the center To take a managerial role in the center To be responsible for recruitment of center industry members To develop policy or operational procedures for the center To make IAB meeting arrangements and develop the agenda

NSF PERSPECTIVE The Value of Evaluators The information collected and made available in summary form is extremely valuable to the NSF The annual reports are used frequently and extensively by Program Directors and Division Management at the NSF Provides an amazing historical data base that gives a long term projection of trends The I/UCRC Evaluators Program has built a high level of confidence in the evaluators at the NSF

CENTER OPERATIONS AND THE ROLE OF EVALUATOR JANE LAUX Business Manager, Connection One: Communication Circuits & Systems Research Center, Arizona State University

Operations Management Who are they: Secretaries; Admin. Assistants; Exec. Assistants; Business Managers; Project Managers; Program Coordinators; Etc.

What do they do? Support Center Director Work closely with NSF Center Evaluator Coordinate Semi-Annual meetings Contact center industry members Daily running of center (office) Assist faculty with center funded project items

Interaction: Evaluator & Ops. Mgmt. A few items that the two work closely on are… Meeting agenda – review / input Jointly compile LIFE form input at meetings Current Listings: IAB Members; IAB members leaving the center; Center faculty members Evaluator travel; consultant fee processing Make sure NSF guidelines are followed What other center related items do the Evaluators and Operations Management interact closely on?

Operations Management goals for this session: Better understand the role of the Evaluator. How can Operations Management better assist NSF Center Evaluators?

IUCRC EVALUATION PROGRAM AND THE ROLE OF EVALUATOR JENNIFER SCHNEIDER NSF IUCRC Evaluation Program, North Carolina State University