Presentation on theme: "Research Position Evaluation System (RPES)"— Presentation transcript:
1Research Position Evaluation System (RPES) orHow you get what you deserveNew SY Orientation October 15-18, 2007
2Outline Purpose Process Panel Operations Results and Outcomes Resources
3PurposeExercise delegated authority from the Administrator to accurately determine the grade levels of ARS research scientist positions using peer panels.Not a “Promotion System”Not a “Tenure System”Based on “person-in-the-job” concept. Open-ended promotion potential based on personal research & leadership accomplishments.Only for Category 1 positions
4Process RL is notified that a SY will be reviewed. SY, with RL support, constructs the case write-upCase is sent to Area OfficeLouise Dalton checks the case for adherence to format.
5Process Mickey McGuire (AAD) reviews the case for content. Case is returned to SY for revision.Final case is sent to RPES staff through Area Office.Case is then sent to panel for evaluation.
6Timing GS 11 – Every 3 years GS 12 – Every 3 years ST – Every 5 years
7Approved Exceptions Early Review Delayed Review Reevaluation Poor Performer
8Anatomy of a Case Write-up Must be accurateRL/Supervisor assistsCareer long accomplishmentsRecency matters!Must be well documentedIn depth contact list is very importantPeer group is important
9ARS Peer Groups Animal Bioscience (ABS) Animal Health (AHL) Chemistry & Chemical Engineering (CCE)Entomology (ENT)Food Science & Human Nutrition (FHN)Plant Bioscience (PBS)Plant Health (PHL)Pre/Post Harvest Science & Engineering (PHS)Soil & Water Science (SWS)Systematics & Taxonomy (STX)
10Research Grade Evaluation Guide Four evaluation factorsResearch Assignment2 - Supervision Received3 - Guidelines & Originality4 - Qualifications and ContributionsFactor 4 is double-weighted... to implement the person-in-the-job concept
11Factor 1 – Research Assignment Assigned ResponsibilityMention formal leadership responsibility (Lead Scientist, Research Leader, Director)Research Objectives & Methodology(next 3-5 yr)Expected Results - impact on science/technologyKnowledge RequiredSupervisory Responsibilities
12Factor 2 – Supervision Received Assigned Responsibility - freedom to controlTechnical Guidance ReceivedReview of Results - manuscript review/approvalGeneral Supervision - frequency/nature/authority
13Factor 3 – Guidelines and Originality Available Literature - applies to research areaOriginality Required – to obtain expected results(End of the Position Description; Limited to 3 pages; all gender neutral)Demonstrated Originality – What is it about you that enabled the accomplishments. Use examples of accomplishments but don’t simply recite them.
14Factor 4 - Qualifications and Contributions: Entire Career Demonstrated AccomplishmentsWhat was accomplished?What was your role?Impact (So What? Who Cares?)Stature and Recognition (Awards, Invitations (even those you could not accept), Professional Society Activities)
15Factor IV - Qualifications and Contributions: Entire Career Advisory and Consultant ActivitiesOther (Education, Experience, Other Significant Information)PublicationsPeer reviewed journal articles and patentsNon Peer reviewed (proceedings, etc.)Abstracts not listed
16Demonstrated Accomplishments GS 11 and 12 – Allowed 5 accomplishment statementsGS 13 and above – Allowed 8Limited to a half page eachEach accomplishment must be documented with two exhibits, typically senior authored, peer reviewed manuscripts. Can be letters from stakeholders (through the AD)IDR list should contain at least one individual knowledgeable about each accomplishment
17Demonstrated Accomplishments What was accomplished?Created, Discovered, Solved a problem, etc.What role did you play?Conceived, Led, Wrote Manuscripts, Grant Proposals, etc.What is the impact (so what and who cares)?Who has adopted, How has it impacted science, What recognition was received (awards, grants funded, etc.)
18Types of Accomplishments ResearchSpecial Assignments or ProjectsTechnology TransferSystems Research and IntegrationLeadership (RL and Scientific)
19Accomplishment Statement Examples Accomplishment: Adjuvants for use with microbial and chemical pesticides must ensure protection from sunlight and rainfall and be useful in relatively small amounts. Wheat gluten was chemically modified to make an adjuvant that protects Bacillus thuringiensis from sunlight and washoff by rainfall when added to a spray tank at 1% solids. The adjuvant also helps a corn rootworm insecticidal formulation called SLAM adhere to corn plants and the adjuvant, when combined with feeding stimulants, can be used to stick other corn rootworm pesticides to plants thus enabling reduction of active ingredient.
20Accomplishment Statement Examples Role: Dr. X led the team and co-conceived, co-developed and co-patented the technology. He supervised writing the manuscripts and patent applications.
21Accomplishment Statement Examples Impact: This technology created another option for managers charged with controlling corn rootworm adults on corn. SLAM is the only product with reduced rates of active ingredient currently on the market. SLAM does not adhere well to plants and must be reapplied. By mixing the gluten adjuvant with SLAM, the product remains on the plants longer and can resist wash off. The Area-Wide Corn Rootworm Adult Management Project funded Dr. X’s work for ca. $100,000 from Microflo, the company that produces SLAM, licensed the use of gluten and reformulated 30,000 pounds for use in the summer of In addition, Trécé Corporation licensed the use of gluten to combine it with a potent feeding stimulant as an adjuvant that can be used with reduced rates of other pesticides registered for use against corn rootworm adults. The product was launched in Summer (Exhibit 3a, #24; Exhibit 3b, #25, and #71, #41)
22Accomplishment Statement Examples Accomplishment/Role: Dr. Y postulated that direct mechanical inoculation of the vascular tissues in seeds will bypass the need for vectors to transmit maize viruses. This elegant, unconventional, and simple approach resulted in a highly efficient method for mechanical transmission of (6 viruses listed) into maize.
23Accomplishment Statement Examples Impact: Among other benefits, this research provided a unique solution to study viruses without the confounding effect of vectors, eliminated or reduced the intensive labor requirements of insect rearing, expedited tests on infectivity of virus preparations, provided a means to study the mechanism of resistance to systemic virus movement and to study virus resistance independently from vector resistance, and facilitated studies that manipulate recombinant viral clones (Exhibit 8a, #85, Exhibit 8b #92, and #87)
24Panel OperationsComprised of 5 scientists GS 13 and above representing two peer groups; a personnel representative, and a panel chair. Panel members are anonymous.One panel member (within your peer group) is assigned to do an in depth review of your case (panel member may have additional cases to do the in depth review)Reads manuscriptsCalls in depth contacts and others (a minimum of 5 contacts, including supervisor, are required)Prepares draft panel report
25Panel Operations All members score each case prior to meeting Panel meets face to face to discuss proper grade according to the RGEGConsensus approachPanels will cover up to 12 cases in a day and a half; surprisingly most lower graded cases require the most amount of time so it is essential the case is well written.
28RPES Panel Decision Options Upgrade (UPG)Remain in Grade (RIG)Insufficient Factual Basis (IFB)Grade/Category Problem (GCP)Refer to Supergrade (REF)Split (SPL)
29Factor 1: Research Assignment Degree C:The incumbent is responsible for formulating and conducting a systematic research attack on a problem area of considerable scope and complexity…..Characteristically, research studies of this scope will result in a series of publishable contributions to knowledge….
30Factor 2: Supervision Received Degree CIn programmed or applied research, the researcher is typically assigned a broad problem area….The researcher is responsible, with little or no supervisory assistance, for formulating hypotheses, for developing and carrying out the plan of attack….The supervisor is kept informed, through occasional discussions, of general plans and progress of the work….
31Factor 3: Guidelines and Originality Degree CIn basic research, available guides and precedents, e.g. existing literature in the field, are limited in usefulness or largely lacking due to the novel character of the work being done.In applied research, this degree typically involves development and application of new techniques and original methods of attack to the solution of important problems.
32Factor 4: Qualifications and Scientific Contributions Degree CAt this degree, the researcher has demonstrated his or her ability as a mature, competent and productive worker. He/she will have typically authored one or more publications of considerable interest and value to the field.Contribution involves leadership of a productive research team or leadership in the conception and formulation of productive research ideas … with such soundness and value as to have marked him/her as a significant contributor to their professional field.
33FACTOR 4 - CONTRIBUTIONS, IMPACT, AND STATURE RESEARCH POSITION EVALUATION WORKSHEET Name:=====================================================================================================Initial Score Summary: F1 ____ + F2 ____ + F3 ____ + F4 ____ = ____ PointsFACTOR 4 - CONTRIBUTIONS, IMPACT, AND STATUREA. Demonstrated Accomplishments:In brief, complete sentences, summarize significance and impact of the 3 highest-rated accomplishments, and explain incumbent's role in each:B. Stature and Recognition:Incumbent’s stature is evidenced by the following 3-5 honors, awards, and/or invitations:Incumbent is [ ] locally [ ] regionally [ ] nationally [ ] internationally recognized for research in:Incumbent has a [ ] poor, [ ] fair, [ ] good, [ ] excellent record of participation in scientific meetings.C. Advisory and Consultant Activities:Incumbent's 3-5 most significant advisory and consultant activities include:The panel assigned Level ____ for this factor because:Concerns/deficiencies noted for criteria of this factor:[ ] Recency of accomplishment[ ] Diminished stature/recognition[ ] Diminished advisory/consultant activityARS Form 516 (9/06)
34FACTOR 1 - RESEARCH ASSIGNMENT A. Assigned Responsibility:Incumbent occupies a [ ] Lead Scientist [ ] Research Leader [ ] Center/Laboratory Director position, or [ ] has no formal leadership role.Incumbent's current assignment involves research to (identify assignment limits/boundaries):In team research, incumbent usually functions as [ ] leader or [ ] member, providing expertise in:Assignment is [ ] continuing or [ ] new (since _____) [NLT 4 years back; not applicable for first post-hire panel review].Incumbent has a substantial [ ] service [ ] technology transfer [ ] other special assignment (identify ________________________________________________________________________________________) requiring approximately____% of duty time.B. Objectives and Methodology: (for next 3-4 years)Objectives are to:Of particular significance are current studies on:Methods and approaches employed are best characterized as [ ] routine [ ] unusual [ ] novel [ ] sophisticated [ ] requiring considerable modification, because:C. Expected Results:Successful research should result in:The panel assigned Level ____ for this factor because:Concerns/deficiencies noted for criteria of this factor:[ ] Results are not reflected as accomplishments and/or advisory/consultant activities.[ ] There is little evidence that substantial progress is being made in this continuing assignment.ARS Form 516 (9/06) (p. 2)
35FACTOR 2 - SUPERVISORY CONTROLS A. Assigned Authority: (freedom to make decisions within scope of assignment)Within the [ ] specific [ ] identifiable [ ] broad [ ] unusually broad assignment, incumbent has the freedom to:Specific problems for study, such as work on _____________________________________________________________, are [ ] assigned by the supervisor [ ] selected by incumbent subject to approval by the supervisor [ ] the total responsibility of the incumbent.B. Technical Guidance Received: (extent of review by supervisor prior to conducting experiments)Technical guidance involves:C. Review of Results: (extent of review/verification of results/findings by the supervisor)[ ] Manuscripts and other reports are reviewed by the supervisor as required by ARS policy. OR[ ] Manuscripts and other reports are reviewed by the supervisor for:D. General Supervision:[ ] Any [ ] Major [ ] Only broad changes in research require the supervisor's approval.Incumbent has responsibility for:and freedom to:The panel assigned Level ____ for this factor because:Concerns/deficiencies noted for criteria of this factor:[ ] Extent of supervision is not consistent with the demonstrated capability of the scientist or progress being made.ARS Form 516 (9/06)
36FACTOR 3 - GUIDELINES AND ORIGINALITY A. Available Literature: (extent to which literature is available/relevant to the assignment)Literature on ____________________________ is available, but specific information on __________________________________________is lacking.Available techniques such as __________________________________________________________________________________________[ ] are useful [ ] require minimal adaptation [ ] require major adaptation [ ] are unusable.B. Originality Required: (aspects which make planning/conducting research and interpreting results especially difficult)Originality is required to:The assignment is difficult because:C. Demonstrated Originality:Incumbent's originality is evidenced by:The panel assigned Level ____ for this factor because:Concerns/deficiencies noted for criteria of this factor:[ ] The originality required in this position is not consistent with originality demonstrated by incumbent to date.ARS Form 516 (9/06)
37Quantity is Important......but it is the quantity of quality contributions--not mere numbers of publications--that matters.
38There are several degrees of activity: Program vs StudiesThere are several degrees of activity:Activity which maintains a career i.e. justifies current grade levelActivity which limits or diminishes a career ie. mundane, insignificant or undirected studiesActivity which advances a career because of its impact i.e. a program that addresses important problemsInactivity is the surest way to loseResearchers are not “tenured in grade”Must “keep running just to stay in place”
39Helpful Hints Keep a file Honors and AwardsInvitations (even those declined)Meetings attended and your participationManuscripts that you reviewedSpecial AssignmentsHave someone else critically read your write-upGet examples from successful colleagues
40Helpful Hints Be sure your 115s are up to date Your research program should develop short term results as well as long term solutions to major problems.Start with the end in mindHow will the research I am starting result in an accomplishment/impact statement? Who will care and why?If you are asked to serve on a panel, do so!
41Some StatisticsStudy of new scientists hired at the GS 11 and GS12 levels in 1999 and 2000A total of 135 scientists were hired108 are still with ARS (5 were terminated during probation, 1 was terminated after probation and 21 left “voluntarily”)77 scientists were promoted at first RPES, 33 received Remain in Grade scores.
42Effect of Publications on RPES Success Eval 2XEval 1X
43Resources Research Leader Area Office RPE Staff Manual 431.3 Merle ColePat HumphryManual 431.3Policies and Procedures 431.3