8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A3971 Can HEP benefit from Open Source ? Manuel Delfino* European Organization for Nuclear Research (CERN) *Permanent address: Physics Dept., Universitat Autonoma de Barcelona, Spain

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A3972 What is Open Source ? Direct descendent of Internet toolkit and Berkeley Unix “hackers” ? A technologically related social movement ? An emerging business model ? Make up your own mind by: 1.Reading “The Cathedral and the Bazaar” by Eric S. Raymond, ISBN Browsing

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A3973 Why a 20 year gap ? In my opinion, part of it is technological: Commoditization of computing hardware Huge increases in Internet speed …but also part of it is socio-political: Global familiarity brought by CNN, inexpensive travel, cultural exchanges Disappearance of many trade barriers Arrival of a world that WANTED to be interconnected and was READY to work together

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A3974 But there are more ingredients… Perhaps it has taken 20 years to realize that: Sharing your ideas and listening to other’s opinions is more valuable than carefully guarding your ideas Many people have similar problems The value is in the ideas, but to fully collaborate you have to share code Vision and architecture are more effective than management to create a following Humility is a great quality

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A3975 Bazaar-style development Some of Raymond’s 19 principles: 3. Plan to throw one away; you will, anyhow 5. When you loose interest in a program, your last duty … is to hand it off to a competent successor 6. Treating your users as co-developers is … route to rapid code improvement and … debugging 7. Release Early. Release Often. And listen to your customers 11. The next best thing to having good ideas is recognizing good ideas … 12. Often, the most striking and innovative solutions come from realizing that your concept of the problem was wrong

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A3976 Role of Project Leader “… the cutting edge of the open-source software will belong to people who start from individual vision and brilliance, then amplify it through the effective construction of voluntary communities…” Other characteristics of open-source contributors: “…self-promotion … mercilessly criticized” “…quality…must be left to speak for itself” “…attacking the author rather than the code is not done”

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A3977 So, can HEP benefit from Open Source ? (Actually, it already has… the real question is should HEP do some of its development in open-source)

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A3978 Almost 2/3 of CERN computer center is in Linux PCs

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A3979 Opportunities… Why does each major lab write its own management software for tape/disk ? (see E176, C223, C68, C308) Did you know that Cisco wrote a print server system whose goals are the same as E369, and place it in open-source ? Is our need for event-oriented statistical analysis similar enough to Migro’s ?

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A39710 …but no magic wand Warnings to the “bandwagon jumpers”: Going Open-source implies a commitment to deliver and maintain a high quality product. The product must be of sufficient interest to others and must be general enough. This can often result in the need to increase the resources in the original development team for quite some time before and after open-source release.

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A39711 Is HEP really ready for open source ? It is said that open source projects are tapping the top 5% in quality developers worldwide. Certainly the project leaders are impressive. Just as computing technologies build on the “commoditization and banalization” of previous ones, the open-source developers count on people being fluent in everything from architecture and design to multiple languages and details of CORBA… Does HEP have such people ? Are they allowed to contribute to community ?

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A39712 The importance of vision and architecture Linux is by far the biggest successful open- source project. However, it inherited its architecture from Unix. Emacs is another great success, but it is a relatively simple toolkit. It’s success seems to be due to its flexibility. Mozilla (eg. Open-Netscape) is not doing so well. Perhaps it lacks vision and architecture ? The closer we get to experiment’s software, the more we seem to depend on our own HEP architects. But how much does Torvalds know about chips ? Could we still benefit from others?

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A39713 How about HEP pseudo-open bazaar? Within an experiment ? It has been tried and it sort of worked Sadly, degenerates to personal battles Seems hard to convince experiment’s management fond of librarians and “official code” Between experiments ? Perhaps there is an opportunity ? How much can be moved from experiment- specific to “HEP Foundation”?

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A39714 What is the bottleneck for HEP ? My personal opinion is that the HEP community as whole has not really absorbed the “Internet way to do things” Over-emphasis of face-to-face meetings “Tribes” come together at meetings, show “their stuff”, then go back and continue to do “their own thing” -> “Institutionalized forking” Very little open e-discussion No productization: “Production on Prototypes” Management very zealous of people not working “directly for the experiment”

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A39715 So, can HEP benefit from Open Source? My conclusion: Yes, it probably can…  Thanks to the large effort made in the last 5 years to “align” software techniques used in HEP with the rest of the world, at least now we can probably explain ourselves to others (though maybe less so to our own collaborators !!!) But… Carefully evaluate on a case-by-case basis the cost and the benefits Need project leaders with vision and humility Continuous training aligned with the world Build trust using e-communication Face investment without clear direct benefit Convince management of virtues of open-source