1. World Class Standards Recommended Improvements to The ETSI Standards Engineering Process Hans van der Veer Chair OCG-IOP & Steve Randall STF308 OCG#32, 12 June 2007, Sophia Antipolis

2. World Class Standards ETSI Standards Engineering Process: OCG#32 (12 June 2007) 2 Introduction Background Interoperability: identified as a topic of strategic importance for ETSI in 2005 STF308 study Review of the ETSI Standards Engineering Process and propose Recommendations for Improvements Started in June 2006 with OCG-IOP as Steering Group and TC MTS as the supporting TB Timeline Review with OCG: June 12, 2007 Review & Approval with Board: September 10-11, 2007 Implementation by ETSI Secretariat Objective of this Meeting Review recommendations of STF308 Get feedback on feasibility, gaps/omissions, etc Recommendations should be complete and implementable! Get agreement for the proposed Recommendations

3. World Class Standards ETSI Standards Engineering Process: OCG#32 (12 June 2007) 3 Sept. 2005 ETSI SWOT – Technical Interoperability Strengths STF, PTCC and Plugtest activities Global network of Collaborations Forums Hosting Weaknesses Ex-ante standardization No formal, imposed project control process No structured approach to interoperability testing One standard for everybody approach Threats: Diversity of technologies with different standards approaches Competing technologies Open standards & open source movements Other SDOs claiming integra- tor & project coordinator roles Opportunities Structured approach to interoperability testing Leadership position for (large) projects: e.g. inte- grator & architect role EU political momentum

4. World Class Standards ETSI Standards Engineering Process: OCG#32 (12 June 2007) 4 Overview of DTR/MTS-00101 Analysis of the Engineering Processes within other bodies: 3GPP IETF OMA IEEE ITU-T DVB Analysis of the Engineering Process within ETSI Recommendations for changes to ETSIs process which should help to assure the inherent interoperability of implementations

5. World Class Standards ETSI Standards Engineering Process: OCG#32 (12 June 2007) 5 Basis of the Analyses The analyses of engineering processes were based on the existence and use of the following aspects: A means of initiating new work items aligned with strategic objectives A means of distributing elements of new work to multiple development bodies A means of coordinating work distributed across multiple development bodies Document drafting methods and facilities Procedures for validating completed standards and sets of standards A means of controlling changes to approved standards

6. World Class Standards ETSI Standards Engineering Process: OCG#32 (12 June 2007) 6 Analysis of other Standards Bodies Protocol standards development in a number of bodies is still based on a 3 Stage method (as per I.130): User requirements Functional Architecture Detailed Protocol specification Most other bodies raise Project or Topic work items which then spawn lower-level work items In many organizations, responsibilities are distributed and monitored by a single central group Coordination of related standards is generally good and even quite formalized in a few organizations In most cases validation of standards is rather ad hoc and, in some instances, accidental!

7. World Class Standards ETSI Standards Engineering Process: OCG#32 (12 June 2007) 7 Analysis of ETSIs Standards Engineering Process Drafting procedures and support amongst the best but mainly in the area of cosmetic rather than technical quality The combined facilities offered by STFs, PTCC and MTS itself are unique and enviable The Work Programme management culture is document based rather than project based Distribution and coordination of projects across multiple TBs depends on individual TBs Agreements with other SDOs are oriented towards the sharing of documents rather than resources Validation facilities exist but validation is not mandatory Change control is ad hoc

8. World Class Standards ETSI Standards Engineering Process: OCG#32 (12 June 2007) 8 Recommendations (1) The analysis identified that ETSI's own process is very good in many aspects However, lessons can be learnt from the other SDOs There are a number of areas where changes in ETSI's process should result in improvements to the interoperability inherent in ETSI's standards

9. World Class Standards ETSI Standards Engineering Process: OCG#32 (12 June 2007) 9 Recommendations (2) Recommendation 1: ETSI's technical committees should use a hierarchical system for organizing and managing work items. Recommendation 2: If a hierarchical system for organizing work items is introduced to the ETSI standards engineering process (Recommendation 1), a coordinating editor should be appointed for each high-level work item with overall responsibility for the coordination of the content and schedule of the subordinate work items and deliverables.

10. World Class Standards ETSI Standards Engineering Process: OCG#32 (12 June 2007) 10 An Example of a Hierarchical Work Item IPv6 Testing Level 1 IPv6 Core WI Level 2 IPv6 Mobility WI Level 2 IPv6 to IPv4 WI Level 2 IPv6 Security WI Level 2 Requirements Catalogue WI Level 3 Interoperability WI Level 3 Conformance WI Level 3 TSS & TP WI Level 4 Test Suite WI Level 4 Core TSS & TP Deliverable Core TTCN-3 ATS Deliverable Core Catalogue Deliverable

11. World Class Standards ETSI Standards Engineering Process: OCG#32 (12 June 2007) 11 Recommendations (3) Recommendation 3: If the subject area of a new high-level work item does not fit clearly into the scope of a single existing technical committee, ETSI members should be encouraged to submit the work item request to OCG for approval and allocation of responsibilities Recommendation 4: The ETSI Secretariat should investigate the provision of an easy-to- use project management software package that can be used for small projects as well as large ones Recommendation 5: All technical committee support staff should be trained to be competent in project planning and management techniques and the associated software tools available in ETSI

12. World Class Standards ETSI Standards Engineering Process: OCG#32 (12 June 2007) 12 Recommendations (4) Recommendation 6: The current set of fixed milestones associated with a work item should be extended to include significant stages within the drafting process Recommendation 7: The ETSI Secretariat should investigate how the existing Work Programme Management application could be used to monitor the progress of work items upon which ETSI projects depend but which are developed in external SDOs. Recommendation 8: All technical committees should be encouraged to use a top-down and phased approach to the development of standards (particularly protocol specifications)

13. World Class Standards ETSI Standards Engineering Process: OCG#32 (12 June 2007) 13 Recommendations (5) Recommendation 9: All technical committees should identify for each work item a method that will be used for validation purposes. Recommendation 10: ETSI should ensure that the use of the TC-MTS validation techniques and guidelines is promoted to all technical committees Recommendation 11: "Specification Validated" should be included in the fixed set of WPM milestones Recommendation 12: All technical committees should be encouraged to incorporate Plugtest events into their standards validation processes.

14. World Class Standards ETSI Standards Engineering Process: OCG#32 (12 June 2007) 14 Recommendations (6) Recommendation 13: All technical committees should identify a method of change management which is appropriate to the nature of the specifications they produce Recommendation 14: The ETSI Secretariat should investigate the provision of an easy-to- use, web-based change request and management system and train all TB support staff in its application and use Recommendation 15: OCG and The ETSI Board should commission the production of a "Managing Standards Engineering" handbook to provide guidance and direction on all aspects ETSI's standards engineering process

15. World Class Standards ETSI Standards Engineering Process: OCG#32 (12 June 2007) 15 Conclusion DTR/MTS-00101 has started MTS approval by correspondence. Planned completion 20 April 2007 OCG review in June 2007 Approve Recommendations to present to The Board Board review September 2007 Approve Implementation Plan If required, a follow-up STF in 2008 to develop an implementation plan