1. EGEE-II INFSO-RI-031688 Enabling Grids for E-sciencE www.eu-egee.org EGEE and gLite are registered trademarks SA3 partner collaboration tasks & process SA3 All Hands Meeting Dublin Andreas Unterkircher CERN Grid Deployment

2. Enabling Grids for E-sciencE Andreas Unterkircher Content 1.SA3 partner roles 2.SA3 collaboration process EGEE’07 2

3. Enabling Grids for E-sciencE Andreas Unterkircher Partner’s work areas YAIM certification Patch certification Contribution to the testbed Batch system support Test writing Testing Contribution to releases Porting (multi platform support) EGEE’07 3

4. Enabling Grids for E-sciencE Andreas Unterkircher YAIM certification Every partner can and (should) participate –updating services using a new version of YAIM This is very time critical (latency) –Must be done within a week because it blocks further steps Only documented testing is acceptable –Tests reports are the tangible results of this activity Information on YAIM (including instructions for certification): –https://twiki.cern.ch/twiki/bin/view/EGEE/YAIMhttps://twiki.cern.ch/twiki/bin/view/EGEE/YAIM EGEE’07 4

5. Enabling Grids for E-sciencE Andreas Unterkircher Patch certification Partner has/acquires expertise for certain service types and certifies patches concerning these services –Over time partners will broaden their “field of vision” Tracked in Savannah Response time: 1-3 weeks –depending on urgency of the patch Best being done on local machines using the certification testbed as an “environment” –Using virtual machine images might help Patch certification checklist available –https://twiki.cern.ch/twiki/bin/view/EGEE/CertificationChecklisthttps://twiki.cern.ch/twiki/bin/view/EGEE/CertificationChecklist EGEE’07 5

6. Enabling Grids for E-sciencE Andreas Unterkircher Batch system support Provide instructions on installation Provide rpms for jobmanagers and information system Test the batch system within gLite (accounting). Provide YAIM module for gLite specific configuration Information available –https://twiki.cern.ch/twiki/bin/view/EGEE/BatchSystemshttps://twiki.cern.ch/twiki/bin/view/EGEE/BatchSystems EGEE’07 6

7. Enabling Grids for E-sciencE Andreas Unterkircher Contribution to testbed Partner sets up a site (at least a CE and a WN) Site is being added to the certification testbed and regular tests are run We have to make sure we cover different deployment scenarios –Otherwise it is only a mirror of the central testbed ….. Test results available –https://lxb0714.cern.ch/easysam/perl/easysam.cgihttps://lxb0714.cern.ch/easysam/perl/easysam.cgi EGEE’07 7

8. Enabling Grids for E-sciencE Andreas Unterkircher Test writing Test writing guidelines –https://twiki.cern.ch/twiki/bin/view/LCG/LCGgliteTestWritingGuid elineshttps://twiki.cern.ch/twiki/bin/view/LCG/LCGgliteTestWritingGuid elines Tests are stored in CVS List of available tests –https://twiki.cern.ch/twiki/bin/view/LCG/SAMTestshttps://twiki.cern.ch/twiki/bin/view/LCG/SAMTests Here the focus is to get “deeper” tests –More complex use cases reflecting better use by VOs –More “stability” tests  Wrong input, operations on non existing entities  Checking return values and comparing them with actions Regression tests EGEE’07 8

9. Enabling Grids for E-sciencE Andreas Unterkircher Contribution to release Preparation for the release of a new service Define and test meta-rpm –Pre, pre deployment tests Write YAIM configuration –More and more moving to developers Write tests Verify documentation, complete documentation Test the service Release tracker: –https://twiki.cern.ch/twiki/bin/view/EGEE/Glite31NodeTrackerhttps://twiki.cern.ch/twiki/bin/view/EGEE/Glite31NodeTracker EGEE’07 9

10. Enabling Grids for E-sciencE Andreas Unterkircher Testing The test results must be visible. Partners must provide a test report (TWiki page etc.). Otherwise testing is useless. Test results entry page –https://twiki.cern.ch/twiki/bin/view/LCG/TestResultshttps://twiki.cern.ch/twiki/bin/view/LCG/TestResults EGEE’07 10

11. Enabling Grids for E-sciencE Andreas Unterkircher Collaboration process EGEE’07 11 Assigning of individual tasks in Savannah https://savannah.cern.ch/projects/sa3-coordination/ Certification of patches in Savannah Daily SA3 progress page gives overview of tasks being done per partner https://twiki.cern.ch/twiki/bin/view/EGEE/SAthreeProgress Interaction with a subset of SA3 on very specific issues. Weekly SA3 phone conference (minutes are available) https://twiki.cern.ch/twiki/bin/view/EGEE/EGEEMeetings Monthly SA3 all hands meeting https://twiki.cern.ch/twiki/bin/view/EGEE/EGEEMeetings 4 times a year

12. Enabling Grids for E-sciencE Andreas Unterkircher Task tracking in Savannah EGEE’07 12

13. Enabling Grids for E-sciencE Andreas Unterkircher SA3 progress page EGEE’07 13

14. Enabling Grids for E-sciencE Andreas Unterkircher Meeting page EGEE’07 14

15. Enabling Grids for E-sciencE Andreas Unterkircher Experiences 1 Introduction of the task tracking system (beginning of 2007) significantly improved the output produced by the partners. Several partners participate in YAIM certification. This has been very useful. Patch certification by partners started (slowly). –12 patches certified, 1 in certification by TCD –5 patches certified, 2 in certiification: GRNET –4 patches certified: INFN –3 patches certified, 1 in certification: PIC –2 patches certified, 2 in certification: UCY –1 patch certified, 1 in certification: CISC –1 patch certified: IMPERIAL –1 patch in certification: CESGA Progress in supporting different batch systems. –Excellent progress: SGE (CESGA), Torque (GRNET), Condor (PIC) –No progress: LSF (we need to find a partner) Some new tests made it into SAM. –LB (UKHB), BLAH (INFN), AMGA (UCY), SRM2.2 (CERN), LFC API (CERN) EGEE’07 15

16. Enabling Grids for E-sciencE Andreas Unterkircher Experiences 2 Since Aug. 2006 (when the first patch was certified by a partner) around 370 patches were treated, only 36 (~10%) were treated by partners. Delivering of tasks in time frequently is not achieved. –E.g.: RGMA security report by PSNC was 2 months late. Work done by partners sometimes is not visible. –No trace in task. –No webpage available. –E.g.: Imperial’s WMS test page contains only one entry which is 4 months old. Some people are only assigned to SA3 with a fraction of their work time. Steep learning curve for partners. EGEE’07 16

17. Enabling Grids for E-sciencE Andreas Unterkircher Improvements for next phase More detailed tracking of the tasks with weekly updates required. Timelines will be checked more rigorously. The outcome of any task must be visible (webpage, report, comments in task). Otherwise the task is considered as not delivered. Every partner selects one person that does the reporting and acts as the main contact. EGEE’07 17