1. Marlon Pierce Indiana University February 14, 2012
Experiences Developing Tools for Scientific Communities in the Apache Software Foundation: Beyond Open Source
Marlon Pierce
Indiana University
February 14, 2012

2. IU Science Gateway Group Overview
IU Science Gateway Group members
Marlon Pierce: Group Lead
Suresh Marru: Principal Software Architect
Raminder Singh, Chathura Herath, Yu Ma, Lahiru Gunathilake: Senior team members
Five interns and research assistants
NSF SDCI funding of Open Gateway Computing Environments project
TACC (M. Dahan), SDSC (N. Wilkins-Diehr), SDSU (M. Thomas), NCSA (S. Pamidighantam), UIUC (S. Wang), Purdue (C. Song), UTHSCSA (E. Brookes)
We participate in two Apache incubators
Apache Rave:
Apache Airavata:

3. Key Problems for Science Gateway, CI Software
Reusability
Reuse or write own gateway software?
Sustainability
Small teams
Successful gateways become infrastructure
Hard to fund
Cyberinfrastructure Software Sustainability and Reusability: Report from an NSF-funded workshop
Governance
How are design decisions made?
Who decides if the software is suitable for release?
How do you handle contributions?
How do you add people to the development and project management teams?

4. Proposed Solution: Open Community Software
More than SourceForge, GitHub, Google Code, etc
Those provide excellent Web tools to help developers.
Here we are concerned with community building.
Diverse community of developers means reusability and sustainability
But diverse communities require governance

5. Some Open Model Examples in CI
NSF-funded CDIGS project
HUBzero Consortium, Sakai Foundation
Institutional level organizations
Eclipse Parallel Tools Platform
Jay Alameda, NCSA: NSF SI2 funding to develop HPC tools workbench
Enzo Project
Excellent talk at TG11 by Prof. Brian O’Shea on their open community efforts
Apache Software Foundation
OODT and TIKKA Data Management Projects at NASA JPL

6. Two Apache Software Foundation Case Studies
Apache Rave and Airavata Incubators

7. Apache Airavata Science Gateway software framework to More Information
Compose, manage, execute, and monitor computational workflows on Grids and Clouds
Web service abstractions to legacy command line-driven scientific applications
Modular software framework to use as individual components or as an integrated solution.
More Information
Airavata Web Site: airavata.org
Developer Mailing Lists:

8. Apache Airavata High Level Overview

9. Example Workflow: Nuclear Physics
Courtesy of collaboration with Prof. James Vary and team, Iowa State

10. Apache Rave
Open Community Software for Enterprise Social Networking, Shareable Web Components, and Science Gateways
Founding members:
Mitre Software
SURFnet
Hippo Software
Indiana University
More information
Project Website:
Mailing List: 1

11. Gadget Dashboard View
Gadget Store
View

12. Extending Rave for Science Gateways
Rave is designed to be extended.
Good design (interfaces, easily pluggable implementations) and code organization are required.
It helps to have a diverse, distributed developer community
How can you work on it if we can’t work on it?
Rave is also packaged so that you can extend it without touching the source tree.
GCE11 paper presented 3 case studies for Science Gateways

13. The Apache Way and Science Gateways

14. Why Apache for Gateway Software?
Apache Software Foundation is a neutral playing field
501(c)(3) non-profit organization.
Designed to encourage competitors to collaborate on foundational software.
Includes a legal cell for legal issues.
Foundation itself is sustainable
Incorporated in 1999
Multiple sponsors (Yahoo, Microsoft, Google, AMD, Facebook, IBM, …)
Proven governance models
Projects are run by Program Management Committees.
New projects must go through incubation.
Provides the social infrastructure for building communities.
Opportunities to collaborate with other Apache projects outside the usual CI world.

15. The Apache Way
Projects start as incubators with 1 champion and several mentors.
Making good choices is very important
Graduation ultimately is judged by the Apache community.
+1/-1 votes on the incubator list
Good, open engineering practices required
DEV mailing list design discussions, issue tracking
Jira contributions
Important decisions are voted on
Properly packaged code
Build out of the box
Releases are signed
Licenses, disclaimers, notices, change logs, etc.
Releases are voted
Developer diversity
Three or more unconnected developers
Price is giving up sole ownership, replace with meritocracy

16. Apache and Science Gateways
Apache rewards projects for cross-pollination.
Connecting with complementary Apache projects strengthens both sides.
New requirements, new development methods
Apache methods foster sustainability
Building communities of developers, not just users
Key merit criterion
Apache methods provide governance
Incubators learn best practices from mentors
Releases are peer-reviewed

17. Apache Contributions Aren’t Just Software
Apache committers and PMC members aren’t just code writers.
Successful communities also include
Important users
Project evangelists
Content providers: documentation, tutorials
Testers, requirements providers, and constructive complainers
Using Jira and mailing lists
Anything else that needs doing.

18. How To Get Involved Join the DEV mailing lists.
Grab the software and start complaining.
Post Jira tickets
Add your patches to Jira if you want to solve a problem.
Frequent patch submission is the best way to get voted in as a committer.