1. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t SVN Pilot: CVS Replacement Manuel Guijarro Jonatan Hugo Hugosson Artur Wiecek David Horat Jonathan Brugge Michel Manent September 2008

2. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t 2 Outline Introduction Motivation Subversion Objectives Performance Tests Security Implementation Questions

3. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Version Control Systems Maintain current and historical versions of files and data (source code) There are many commercial and Open Source VC Systems: –(Centralised) CVS/SVN –(Distributed) GIT, Bazaar, Darcs, GNU arch, Mercurial, Monotone, etc –But subversion seems to be the most popular one (used by GCC, Phyton, PuTTY, Apache, GNOME, KDE, etc) Physics User Community: (IN2P3, ROOT, Totem..) 3

4. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t CERN Central CVS Service Hosts over 330 Software Projects –29 for Atlas –46 for CMS –8 for LHCb,….. Over 3000 developers registered Over 90 GBytes of source code Creates 250 Remedy tickets per year Over 100000 commits per month 4

5. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t CERN Central CVS Service

6. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Central CVS service features High Availability and Load Balancing Web interface to repositories Usage Statistics Repository Remote Replication + Mirroring Daily archive of Repositories and DR Developers Mailing list Pre/Post Commit Actions (such us e-mail notification, etc) Various access method (ssh/kerberos) Role split (CVS Admin/Librarian/Developer) 6

7. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Motivation for SVN Pilot Originally designed to host less than 100 projects Requests to provide a central SVN service: –From CMS –From ATLAS (case study in 2006) –And from many others CVS is over 20 years old while SVN is this millennium technology Requests for Read Access control

8. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t SVN vs. CVS 8 FeatureSVNCVS SpeedFasterSlower PermissionFullLimited File typesAllLimited Off line operationsYesNo Repository formatDatabaseFile system LocksNoYes Atomic commitsYesNo

9. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t New Features (SVN 1.5) Automatic update of working copy Merge tracking –Subversion keeps track of what changes have been merged where Sparse checkouts Interactive conflict resolution 9

10. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Pilot Objectives Provide current CVS service features Add new features (available with SVN) –Control Read access per path (module) –Authenticated Web access –Binary files handling Ease CVS to SVN migration Improved usage statistics (SVN Stats) Handling of first line support via the Help Desk Delegate administrative tasks to Software Librarians of each project Prevent uncontrolled setup of SVN servers Manpower: 1.2 FTE project

11. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Timetable Feb 2008 Preliminary study May 2008 CVS librarians feedback July 2008 SVN Pilot Dec 2008 SVN service in production CVS to SVN migration Dec 2009 CVS service close down 11

12. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t SVN Pilot study Access methods –https –ssh Shared storage –NFS 3/4 –AFS Securing service –Restricted Shell –Chrooted hooks (commit scripts) Infrastructure: –Librarian tools, Statistics, Web Interface,… 12

13. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Performance Tests SVN check out of a 110 Mb project Parameters –AFS/NFS3/NFS4 –HTTPS/SSH 13

14. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t AFS vs NFS3 (1 server) 14

15. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t AFS vs NFS4 (1 server) 15

16. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t AFS vs NFS4 (3 servers) 16

17. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Preliminary Conclusions AFS much faster than NFS SSH much faster than https SSH scales very well with high load … New tests ongoing (with mixture of read and write operations) 17

18. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Security Project Isolation Windows/Linux clients Worldwide access Shared file system independent Hooks executed on servers Librarians may put any script into the hooks Librarians might need file system level access to repository – being studied 18 Security risk!!

19. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Hooks (scripts) ClientServer 19 Svn commit Pre-commit hook is executed Post commit hook is executed SVN: Commit OK Email notification recieved

20. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Hook scripts chrooted: Server 20 svnserer hooks/post-commit hook Repositories System files Usr-hooks/post- commit hook Librarian hooks: jailed Repository (1)

21. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Architecture svn.cern.ch (rw) –Secured subversion server (only ssh) –Read and write access to repository svnweb.cern.ch (ro) –User documentation –Project request –SVN web interface –Usage statistics 21

22. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Pilot Implementation Summary SSH access for SVN clients –Restricted shell for all SVN clients –Hooks chrooted SVN web (ro) –Web interfaces: websvn, trac –SSO Authenticated access Administration delegated to librarian –Access rights –Hooks –Admin tools 22

23. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Conclusions Secure service This will replace CVS by end of 2009 The service is supported (pre-production) Pilot setup may differ from final setup –Access method, Web interface, shared file system, etc. –Changes will be transparent to the users 23

24. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Support http://cern.ch/svn –Try the pilot –Documentation Svn.support@cern.ch 24

25. CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Questions? Thanks For Listening…. M. Guijarro, A. Wiecek, David Horat, Jonathan Bugge, M. Manent, H. Hugosson 25