LS1 Review BE-CO-SRC Section Contributions from: A.Radeva, J.C Bau, J.Betz, S.Deghaye, A.Dworak, F.Hoguin, S.Jensen, I.Koszar, J.Lauener, F.Locci, W.Sliwinski, M.Vanden Eynden 01/12/15 M.Vanden Eynden on behalf of BE-CO-SRC 1

BE-CO-SRC Portofolio  Communication & Middleware  Real-time software environment for Accelerator control and data acquisition  Accelerator Timing and sequencing  Signal acquisition and visualization systems  Generic software for the control of analog and digital I/Os based on standard BE-CO hardware modules  Communication libraries for interfacing industrial controllers and fieldbuses within the standard BE-CO infrastructure  Acronyms:  LS : Long Shutdown  EG : Equipment Groups  CGFES  CMW  CMX  DIP Gateways  FESA  GMT  OASIS  Passerelle  RBAC  SILECS 01/12/15 M.Vanden Eynden on behalf of BE-CO-SRC 2

LS1 - General comments What does a long shutdown mean for your services in general?  SOFTWARE SUPPORT  In general more user support as EG proceed with big changes and migrations during LS  SOFTWARE DEVELOPMENT  It should not be the time to implement changes  Software changes should be done before LS to allow EG to validate them them with beam  AVAILABILITY  Many services have to remain operational during the LS (CMW)  DEPLOYMENT  Deployment of new major functionality and services only possible during LS (CMW)  Some systems like GMT and OASIS have to proceed with changes during LS as these are the only possible slots for change in the HW infrastructure (local timing distribution and configuration, modification of OASIS infrastructure and configuration, etc.) 01/12/15 M.Vanden Eynden on behalf of BE-CO-SRC 3

LS1 - General comments How did you prepare for LS1?  MASTER PLAN  All Software activities were anchored on the ACCOR project for which a “stable control system baseline” was mandatory. The agreed dates were as follows:  Development environment ready by July 2013  Operational environment ready by December 2013  SOFTWARE DEPENDENCIES  Coordination and management of software dependencies was left to the various activity leaders  CMW + FESA + FGC teams for the RDA3 migration  Each project had dedicated meetings with EG representatives  HARDWARE INSTALLATIONS for OASIS, CGFES and GMT  All Hardware modifications were prepared and planned in the context of the ACCOR Hardware installation meetings run by C.Dehavay 01/12/15 M.Vanden Eynden on behalf of BE-CO-SRC 4

LS1 - General comments What worked well in LS1?  EARLY TESTS and MDs BEFORE LS1  All key CGFES solutions such as function generation were fully tested in operations (MDs in 2012)  IMPLICATION of OP during COMMISSIONING  Booster OP team involvement in the commissioning of the OASIS signals on the field during 2014 start-up  GOOD COORDINATION with INSTALLATION TEAM  About 40 coordination meetings, clear program for each Accelerator  ECR-like APPROACH  EDMS renovation specifications (huge effort) done by MCCs + EG + OP  ITERATIVE DEVELOPMENT APPROACH  Numerous short iterations with EG representatives  Demo, impact description, feedback from EG, presentation of next iteration 01/12/15 M.Vanden Eynden on behalf of BE-CO-SRC 5

LS1 - General comments What didn’t work well in LS1?  TOO LATE …  Some products were still in development during LS instead of being ready BEFORE the LS and tested under operational conditions  EG started to use products but all the necessary tools for their operational deployment were late or difficult to use  EVOLUTION Vs REVOLUTION  Some products went through a “revolution” during LS1, generating too much impact inside and outside CO  Better anticipation and “smoother” evolution should be our mission for LS2 01/12/15 M.Vanden Eynden on behalf of BE-CO-SRC 6

LS1 - General comments What didn’t work well in LS1?  LACK OF COORDINATED PLANNING of SOFTWARE ACTIVITIES  There was none, except the pressure put by the ACCOR project  There will be no ACCOR Project in LS2. We need a BE-CO wide coordination covering all Software & Hardware changes during LS2  EXPLANATION ≠ AGREEMENT  Explaining to the EG the impact of changes in separate team meetings is only a first step.  The implementation of these changes during the LS should have been evaluated, agreed and planned with them beforehand (resources, time table, etc.) as we did for the hardware renovations 01/12/15 M.Vanden Eynden on behalf of BE-CO-SRC 7

LS1 – Planning & Organization What tools and processes did you use?  JIRA for software issues  Wikis for support and documentation  EDMS for approval procedures  SCRUM for Software development process  Control System Test bed (CST) + Bamboo  The iterative software development approach and the regular meetings with EG representatives is the way to go  Finding the correct balance between software development activities and operational support is complex, especially if we want our products to be ready before the future LS  Improvements are currently discussed in order to better accommodate this situation 01/12/15 M.Vanden Eynden on behalf of BE-CO-SRC 8

LS1 – Planning & Organization What was your influence on setting the deadlines? 01/12/15 M.Vanden Eynden on behalf of BE-CO-SRC 9 All we had for SW is this …

LS1 – Technical What actions did you take towards quality and correctness of your changes before releasing them to the users?  Controls TEST BED is of prime importance for all activities  Bamboo and unit testing  We need more … for LS2  Organization of dedicated MDs for the control system is crucial before the LS (OASIS)  ACCOR “DRY RUNS” played a central role for all activities as it was the integration test of the full control system vertical stack. We need the same approach for LS2 01/12/15 M.Vanden Eynden on behalf of BE-CO-SRC 10

LS2 – Outlook How do you perceive LS2, and how do you think it will differ from LS1 in terms of impact on your services and user community?  LS1 represented the most massive software and hardware upgrade since the 90’s  Even if LS2 looks less ambitious for BE-CO, numerous improvements are needed for LS2:  Develop, test and validate Software releases and tools BEFORE LS so that EG can prepare themselves and start work from the beginning of LS and not by the end of it  Better coordinate software activities inside CO  Build solid agreements and roadmaps for software changes with EG beforehand (CO3 ECR-like approach) 01/12/15 M.Vanden Eynden on behalf of BE-CO-SRC 11

LS2 – Outlook Do you already have high-level work plans for LS2?  Yes, please check the BE-CO LS2 days presentation  Few examples:  Migration towards CentOS7  Post-ACCOR renovations until LS2 and EOL roadmaps for FESA, RDA, LynxOS, GM, SL-EQUIP, etc.  Use of White Rabbit technology for OASIS and Timing systems  Renovation of the Machine Timing distribution Network 01/12/15 M.Vanden Eynden on behalf of BE-CO-SRC 12

LS2 – Outlook CO needs to continuously introduce changes that are considered indispensable for the good functioning of the Accelerator complex How can CO motivate such changes to the sector?  By organizing in-depth discussions with the EG about the motivations (obsolescence, security, performance, etc.), risks and efforts involved on both parts.  CO3 plays this role for the “Post-ACCOR renovations until LS2”  Go for smooth evolutions rather than revolutions How can the controls upgrade plan become part of the work planning of the equipment groups?  This would be the natural follow-up if we correctly address the previous point  CO3 initiative starting for “Coordination of new developments”  BE-CO-SRC has and will continue to propose his practical help to EG in order to help them implement changes in their systems (peer-to- peer support, C++ hands-on, CERN wide training, and much more) 01/12/15 M.Vanden Eynden on behalf of BE-CO-SRC 13

Thank you to all BE-CO-SRC members for their valuable input! 01/12/15 M.Vanden Eynden on behalf of BE-CO-SRC 14