Knowledge Environments for Science: Representative Projects Ian Foster Argonne National Laboratory University of Chicago Symposium on Knowledge Environments for Science, November 26, 2002

2 ARGONNE  CHICAGO Comments Informed By Participation in … l E-science/Grid application projects, e.g. –Earth System Grid: environmental science –GriPhyN, PPDG, EU DataGrid: physics –NEESgrid: earthquake engineering l Grid technology R&D projects –Globus Project and the Globus Toolkit –NSF Middleware Initiative l Grid infrastructure deployment projects –Alliance, TeraGrid, DOE Sci. Grid, NASA IPG –Intl. Virtual Data Grid Laboratory (iVDGL) l Global Grid Forum: community & standards

3 ARGONNE  CHICAGO Data Grids for High Energy Physics l Enable community to access & analyze petabytes of data l Coordinated intl projects –GriPhyN, PPDG, iVDGL, EU DataGrid, DataTAG l Challenging computer science research l Real deployments and applications l Defining analysis architecture for LHC

4 ARGONNE  CHICAGO NEESgrid Earthquake Engineering Collaboratory U.Nevada Reno

5 ARGONNE  CHICAGO Size distribution of galaxy clusters? Galaxy cluster size distribution Chimera Virtual Data System + GriPhyN Virtual Data Toolkit + iVDGL Data Grid (many CPUs) Communities Need Not be Large: E.g., Astronomical Data Analysis

6 ARGONNE  CHICAGO A “Knowledge Environment” is a System For … “Interpersonal collaboration” “Integrating data” “Accessing specialized devices” “Enabling large-scale computation” “Sharing information” “Accessing services” “Large communities” “Small teams”

7 ARGONNE  CHICAGO It’s All of the Above: Enabling “Post-Internet Science” l Pre-Internet science –Theorize &/or experiment, in small teams l Post-Internet science –Construct and mine very large databases –Develop computer simulations & analyses –Access specialized devices remotely –Exchange information within distributed multidisciplinary teams  Need to manage dynamic, distributed infrastructures, services, and applications

8 ARGONNE  CHICAGO Enabling Infrastructure for Knowledge Environments for Science (aka “The Grid”) “ Resource sharing & coordinated problem solving in dynamic, multi- institutional virtual organizations”

9 ARGONNE  CHICAGO Grid Infrastructure l What? –Broadly deployed services in support of fundamental collaborative activities –Services, software, and policies enabling on- demand access to critical resources l Open standards, software, infrastructure –Open Grid Services Architecture (GGF) –Globus Toolkit (Globus Project: ANL, USC/ISI) –NMI, iVDGL, TeraGrid l Grid infrastructure R&D&ops is itself a distributed & international community

10 ARGONNE  CHICAGO Lessons Learned (1) l Importance of standard infrastructure –Software: facilitate construction of systems, and construction of interoperable systems –Services: authentication, discovery, …, … –Needs investment in research, development, deployment, operations, training l Building & operating infrastructure is hard –Challenging technical & policy issues –Requisite skills not always available –Can challenge existing organizations

11 ARGONNE  CHICAGO Lessons Learned (2) l Importance of community engagement –“Maine and Texas must have something to communicate” –Big science traditions seem to help –Discipline champions certainly help –Effective projects often true collaborations between disciplines and computer scientistis l Importance of international cooperation –Science is international, so is expertise –Challenging, requires incentives & support

12 ARGONNE  CHICAGO Lessons Learned (3) l Collaborative science/Grids are a wonderful source of computer science problems –E.g., “virtual data grid” (GriPhyN): data, programs, derivations as community resources –E.g., security within virtual organizations l Work in this space can be of intense interest to industry –E.g., current rapid uptake of Grid technologies