1. TeraGrid Allocations Discussion John Towns Director, Persistent Infrastructure National Center for Supercomputing Applications University of Illinois

2. Please help us with the following questions: How well does the current TeraGrid allocation process serve the needs of both the research community and the providers of TeraGrid resources and services? Does the SAB have any advice on how this process might evolve as both the nature of research problems and the nature of TeraGrid resources and services evolve?

3. Overview of Allocations Process (I) Who Can Receive Allocations? –any Ph.D.-level researcher (or NSF- eligible PI) from a U.S. academic or non-profit research institution –NSF Graduate Research Fellows and Honorable Mention awardees –any discipline, any institution, any funding agency support What Can Be Requested? –three request sizes Small, Medium, or Large –more than 20 resources Computational Data Storage Visualization Advanced User Support –no cost to PIs

4. Overview of Allocations Process (II) When Are Requests Accepted? –Small requests: Anytime 30,000 CPU-hours 5 TB disk 25 TB tape –Medium requests: Quarterly 500,000 CPU-hours 25 TB disk 100 TB tape –Large requests: Semi- annually Where Do You Apply? –POPS https://pops-submit.teragrid.org/

5. Overview of Allocations Process (III) How Do You Get Started? –for Small (start-up) requests, just need an abstract of the work and a CV for the PI allow 2 weeks to get everything set up. –for Medium or Large requests, a proposal is reviewed by independent committee at quarterly meetings.

6. Medium or Large Allocations PIs need to be aware of the lead time for getting an MRAC or LRAC award –Requires a written proposal –Reviewed by domain experts LRAC (Large) –Reviewed semi-annually –Awards begin April 1, Oct. 1 MRAC (Medium) –Reviewed quarterly –Awards begin Jan. 1, April 1, July 1, Oct. 1

7. Using Your Allocation New User Packet –provides your TeraGrid- wide and sSite-specific logins TeraGrid User Portal –online hub for monitoring and using your allocation –portal.teragrid.org guests welcome User Support –help@teragrid.org

8. Allocations Policies High-level description of the nuggets of the policy http://www.teragrid.org/userinfo/access/allocationspolicy.php –PI eligibility –proposal format, page length, etc –review process; review panel –review criteria –award management Who sets them and how are they evolved? –ad hoc team to develop recommendations and comments John Towns, Nancy Wilkin-Diehr, Ralph Roskies, Dave Hart, et.al. –community input to gain consensus PIs, xRAC Panel, NSF Program Officers, RP representatives, etc. –has evolved policies: community account/science gateways storage support services what’s next?

9. Specific Issues (I) How does the allocation panel review take into account the review of scientific merit and broader impacts of the proposed work already done by the NSF? How does “Double jeopardy” with separate financial and resource allocation proposals get handled? –policy if review of science has been done, it is NOT the purview of xRAC Panel panel only does this in cases of requests not supported financially from grants that result for proposals that have had such review –practice of enforcement remanded at start of every meeting; reigned in during panel reviews How does the allocation process take into account the science impact, if at all, of the proposed TG usage? –Actually, that is not really the specific purview of the review panel, though they do consider it if there is ambiguity amongst the panel on how to handle a request. This gets at the review of the science, which is typically NOT what the xRAC reviews. –Question for SAB - Should this factor be given more or less (or any) emphasis?

10. Specific Issues (II) How are the allocations reviewers selected? What process or criterion is used to ensure that they have adequate expertise? What term limits (if any) should be applied to their service? –typically look for recommendations of outgoing reviewers –also look for recommendations of NPS program officers –try to assess the work they do from on-line information –do not conduct a formal review process of potential reviewers –from policy document: “The committees consist of volunteers who are selected from the faculty and staff of U.S. universities, laboratories and other research institutions. All of the committee members have expertise in some area of computational science or engineering and serve a term of 2-5 years.” –typically try to keep most reviewer to 2-3 years. –some kept 4-5 years for various reasons (reviewer availability, quality of reviews, etc.)

11. Proposal Counts Typically 3-6 reviewers per proposal –larger number for really big LRAC requests Typically ~35 members of review panel High reviewer load: –some reviewers will have 15-20 proposals when MRAC and LRAC coincide 0 20 40 60 80 100 120 Sep-05Dec-05Mar-06Jun-06Sep-06Dec-06Mar-07Jun-07Sep-07Dec-07 # of Proposals MRAC LRAC

12. Specific Issues (III) Who should be serving as panel discussion chair at the allocations committee meetings, and what guidelines are in place for the conduct/role of this chairperson? –in practice, selected by Allocations Officers from panelists –guidance given to chair by Allocations Officers –Invite input from SAB!!

13. Specific Issues (IV) What role, if any, should NSF program directors funding projects in large-scale computation have in the allocations process? –in practice have only been observers in the process and provided responses on NSF-level policy issues

14. Specific Issues (V) Are the conflict-of-interest policies applied to allocation panelists more or less stringent than those used for NSF panels? –COI policy generally follow NSF COI rules –collaborators, advisors, etc. –institutional conflicts difference: proposers may be on panel that reviews their proposal conflicted reviewers are dismissed from room during discussions involving conflicts –Question for SAB - Should they be?

15. Specific Issues (VI) How are reviewers' areas of expertise matched to the allocation requests? –reviewers provide list of primary and secondary FOS –PIs indicate primary and secondary FOS of proposal –provides first-order matching –Allocations Officers review and address unassigned proposals –Question for SAB - Is this the most effective strategy?

16. Specific Issues (VII) How does the allocation process enable individual projects to have dedicated access to large fractions of a given system for significant periods of time? How do TG management practices and metrics likewise enable such partitioning of the systems? –PIs can specifically request –ask PIs to provide 1 one page additional description additional information regarding this needs; allow RPs to prepare –fundamentally, availability is controlled by the RP; nothing specifically requires this is seen as being in RP’s own best interest to provide requested by community and typically well justified. –Question for SAB - Are these sufficient?

17. Specific Issues (VIII) How will the allocation process take into account the jumps in available resources and system size when each Track 2 system and, eventually, the Track 1 system come online? Do the Track 2 and Track 1 systems present any particular challenges to the allocation process? –addressing this and related issues within the TG Extreme Scalability RAT –fundamentally, current process is sufficient –NSF has initiated “PRAC” solicitation and this will impact process for Track 1