1. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Comparative Investigation of Collaboratories: Cross-Cutting Themes June 20, 2003 University of Michigan Ann Arbor

2. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Reminder: Where We’ve Been n UM group – 15 years of experience with distributed collaboration n SOC project –~40 Collaboratories at a Glance (C@G) –10 in-depth studies Sept. 02: SPARC/UARC, CFAR, Bugscope, EMSL June 03: NEESgrid, InterMed, GriPhyN, iVDGL, AfCS, BIRN n “The Literature” n Your input

3. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN What We Learned Here n Review Cross-cutting Themes –Modify Refine Eliminate Add n Framework for generalizations –What leads to success, failure? n Source of design prescriptions –How to do the next one?

4. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Cross-cutting themes: From Prior Work n Collaboration readiness –Collaboration vs. competition in science –Bottom-up vs. top-down origins n Technology readiness –Experience with collaboration tools n Infrastructure readiness –Both technical and social n Common ground –Extent of shared knowledge; critical in interdisciplinary work n Coupling of work –The interdependencies among individuals

5. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Collaboration Readiness n Can a collaboratory be mandated by an external agency (e.g., funding source)? –NEESgrid – collaboratory capability as a condition of funding High risk – details in presentation & discussion n History of collaboration –High energy physics vs. earthquake engineering n Science driven –AfCS –BIRN

6. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Common Ground n NEESgrid –Differences in terminology between CS & EE communities n InterMed –Importance of establishing shared vocabulary –Boundary objects, pidgins n GryPhyN, iVDGL –Too much common ground  Boundary objects as key concept [G. Bowker] n BIRN –Attention to metadata, ontology

7. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Cross-cutting Themes from SOC Analyses n What is success? –Detailed discussion in June 2001 workshop n What are the incentives for participation? –Survey study in progress n What kinds of collaboratories are there? –Taxonomy – presented later n How do collaboratories evolve? –Some ideas based on our taxonomy – presented later

8. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN What is Success? n Use of the collaboratory tools n Software technology n Direct effects on the science n Science careers n Effects on learning, science education n Inspiration for other collaboratories n Learning about collaboratories in general n Effects on funding, public perception

9. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Measures of Success n GriPhyN, iVDGL –Persist beyond ITR funding –Spending less time on tools, more on science n BIRN –Cover story in Nature –Lots of publications n Multiple audiences –Beyond the scientists –Students, government, industry, general public Collaboratory  NSF STC

10. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Incentives n AfCS –Alliance with Nature n BIRN –Guidance re publications n LHC –Shift in time scale of experiments –Implications for careers

11. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Evolution***** n Ecology of collaborations –Movement from limited to full collaboration n Data – wisdom hierarchy [G. Furnas] –Movement up and down over time and space –Relates to social vs. technical processes n Where did the field come from, where is it going? –Historical context as critical n Multi-tasking of individuals (G. Mark) n Time scale issues –AfCS – bioinformatics earlier? –BIRN –savings across successive BIRNs

12. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN The relationships n Wisdom n Knowledge n Information n Data n The world Shared Instruments Distributed Research Centers Practice and Expertise Community Data Systems

13. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Cross-cutting Themes from SOC Analyses n Do collaborations have an ideal size? –Collaboratories allow for larger ones –How do they scale? n What are various organizational models for how to structure collaboratories? n How does the control and flow of resources affect collaboratory success? –The money flow; the relation to the sponsor(s) n How much flexibility should be designed in? –What kinds of early commitments? –How much flexibility will funders allow?

14. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Ideal size n ATLAS –Collaboration of 2000 But very organized –Beyond ATLAS? Manhattan Apollo n How many working groups can be supported? –Organizational science as source of clues –What does technology enable? –How to scale from literature on teams (G. Mark)

15. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Flexibility n Retrenchment, redefining of goals –G. Bowker – may be key to success n Funding models –AfCS – enough flexibility? (A. Prakash) n Adapting to new developments –InterMed – 1995 shift to focus on guidelines –AfCS – 2003 changing cells

16. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Cross-cutting Themes from SOC Analyses n How important are data issues in collaboratories? –Data seems to be a central component of all collaboratories n For what kind of work do you need real-time vs. asynchronous interactions? n How important is security? n What’s the mix of tailor-made vs. off-the-shelf tools?

17. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Data Issues n Metadata n Provenance n Persistence, archiving n Rationale for transformations –NEESgrid, GriPhyN, iVDGL, AfCS, BIRN n Details of size, usage – different software needs? n What level of processing? Different disciplines may vary [D. Sonnenwald] n Data sharing across jurisdictional boundaries – BIRN –IRB – data from humans –International

18. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Cross-cutting Themes from SOC Analyses n How crucial are platform issues? –What is the emerging role of middleware? n What is the role of emerging infrastructure such as the Grid? n How does one move from early prototypes to production versions of collaboratories? n Why isn’t there more reuse of collaboratory tools? n To what extent are the issues specific to science domain or are general?

19. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Moving to Production Versions n Tensions between CS and domain users –NEESgrid – “innovation vs. extrapolation” –GriPhyN & iVDGL n Moving beyond initial demo stages –Slow adoption InterMed –Sustaining the investment NEESgrid – NEES consortium infrastructure set up in advance GriPhyN, iVDGL – seeking a sustaining support process BIRN n Incentives –“build hardware” [J. Leigh] n Diffusion of Innovation literature

20. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Domain specificity n The unusual character of HEP –Long history – since Manhattan –Scale – LHC –Common knowledge, self-esteem, etc.

21. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN New Issues n Human subjects issues –IRBs across jurisdictional boundaries –Need for new approach? n Management –NEESgrid – management lags implementation –InterMed – need for tight management –GriPhyN & iVDGL – hiring project managers –AfCS – charismatic management –BIRN – governance manual; adding steering committee n Vision –Who’s vision –“Acephalous” projects (G. Bowker) –Leadership issues – charisma

22. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN New Issues n What kind of technology? –Specific applications vs. APIs –Generic collab vs collab in specialized tools (S. Poltrock) –Economics of the Grid (M. Cohen) n Standards as a unifying process –Politics of standards setting –BIRN in a box n “If you build it, they will come” –Highly flawed model NEESgrid InterMed GriPhyN, iVDGL –Tied to incentives –Expectation management

23. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN New Issues n Intellectual property –Who negotiates? –What are the arrangements? n Evaluation –Who does it? Within the project – formative Outside the project – summative –What is it? Cross sectional Longitudinal –Over what time period? Lag effects, long term indirect effects –Be sophisticated “science” talk vs. “informal” talk (G. Bowker)

24. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Biggest issues – my candidates n What is success? n Evolution – ecology n Transition to production versions, sustaining the vision n Data issues n How to manage collaboratories?

25. SCHOOL OF INFORMATION. UNIVERSITY OF MICHIGAN Some SOC Issues n Are we asking the right questions? n Are we doing the right kinds of analyses? –Measures –Control groups n Are our representations useful? –Resource diagrams n Mix of science and engineering