Software Engineering Standards Prepared by: James W. Moore The MITRE Corporation SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 1 For presentation to DC/Baltimore SIGAda This presentation represents the opinion of the author and does not present positions of The MITRE Corporation or of the U.S. Department of Defense.

Software engineering: Definition (1) The application of a systematic, disciplined, quantifiable approach to the development, operation and maintenance of software, that is, the application of engineering to software. (2) The study of approaches as in (1). -- IEEE Std SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 2 © IEEE, used by permission

Software Engineering Standards Concern the responsible practice of software engineering Usually deal with process rather than product... … although they sometimes deal with generic product characteristics or supporting resources Deal with subjects like Configuration Management, Quality Assurance, Verification, and Validation SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 3

Importance of Software Engineering Standards They consolidate existing technology into a firm basis for introducing newer technology They increase professional discipline They protect the business They protect the buyer They improve the product SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 4

Software Engineering Standards: Roles Specify techniques to develop software faster, cheaper, better, IEEE (Measures for Reliable SW) Provide consensus validity for “best practices” that cannot be scientifically validated, IEEE 1008 (Unit Testing) Provide a systematic treatment of “ilities”, IEEE 730 (SW Quality Assurance) Provide uniformity in cases where agreement is more important than small improvements, IEEE P (IDEF0) Provide a framework for communication between buyer and seller, IEEE/EIA (SW Life Cycle Processes) Give precise names to concepts that are fuzzy, complex, detailed and multidimensional, IEEE 1028 (SW Reviews) SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 5 More exciting More effective

Organizational Goals for Software Engineering Standards Improve and evaluate software competence Framework for two-party agreements Evaluation of software products Assurance of high integrity levels for software products SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 6

Software Engineering Standards Approximately 315 software engineering standards, guides, handbooks, and technical reports are maintained by approximately 46 professional, sector, national, and international standards organizations. -- [Magee97] In 1981, IEEE had one software engineering standard. By year end 1997, the collection had grown to [SESC97a] The 1994 edition of IEEE Standards Collection: Software Engineering is 1300 pages long. The 1999 edition is in four volumes totaling 2400 pages. SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 7 Users need help in integrating all of this advice.

Developers of international standards related to SWE SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 8 ISOIEC JTC1TC176 SC22 Software EngineeringLanguage, OS SC7 TC56SC65A ITU QualityInformation TechnologyDependabilityFunctional Safety The focal point in international standards is ISO/IEC JTC1/SC7. Other committees, though, deal with related work. Members of these committees are “national bodies,” i.e. countries, represented by “national delegations.”...

SWE standards developers: US SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 9 About 550 organizations in the U. S. make standards. About half of them are accredited by ANSI, allowing them to participate in international standardization activity. The focal point (in the U. S.) is the Software Engineering Standards Committee of the IEEE Computer Society ANSI NCITSAIAAANSASTMEIA SESC IEEE INCOSE PMI...

International Software Engineering Standards SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 10

SC7 Plenary Attendance SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 11 Based on a chart by François Coallier, SC7 Chair

Scope of ISO/IEC JTC1/SC7 SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 12 Requirements Data Process TechnologiesResources (Human & Material) Tools Products Product Data Engineering Data “Standardization of processes, supporting tools and supporting technologies for the engineering of software products and systems.”

SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 13 Management Sciences Computer Sciences and Engineering Dependability Engineering (IEC TC 56) Safety (IEC TC65) Industrial Engineering Quality Management (ISO TC 176) APPLICATION DOMAINS (many TCs) SOFTWARE and SYSTEM ENGINEERING Based on a chart by François Coallier, SC7 Chair

ISO/IEC JTC1/SC7 Working Groups WG2: System software documentation WG4: Tools and environment WG6: Evaluation & metrics WG7: Life cycle management WG9: System & SW integrity WG10: Process assessment WG12: Functional size measurement WG13: Software measurement process WG17: ODP enterprise lang WG18: Quality mgmt sWG18: Revise ISO WG19: ODP and modelling languages SWG1: Planning SWG2: Vocabulary SWG3: Process architecture SWG4: Web and communication support Study Group: SE practices SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 14

SC7 Production (est.) (No new NWI assumed - exclude dependability) SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 15 Based on a chart by François Coallier, SC7 Chair

Current Standards of SC7 (1 of 2) Several “legacy” standards ISO/IEC 6592:2000, Guidelines for the documentation of computer-based application systems ISO/IEC 9126:1991, Product quality characteristics ISO 9127:1988, User documentation and cover information for consumer software packages ISO/IEC TR 9294:1990, Management of software documentation ISO/IEC 11411:1995, Representation of state transition diagrams ISO/IEC 12119:1994, Software packages: Quality requirements and testing ISO/IEC TR 12182:1998, Categorization of software ISO/IEC 12207:1995, Software life cycle processes ISO/IEC 14102:1995, Evaluation and selection of CASE tools ISO/IEC :1998, Functional size measurement SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 16

Current Standards of SC7 (2 of 2) ISO/IEC TR 14471:1999 Information technology -- Software engineering -- Guidelines for the adoption of CASE tools ISO/IEC 14568:1997, Diagram exchange language for tree charts ISO/IEC 14598:2000, Software product evaluation (6 parts) ISO/IEC 14756:1999, Measurement and rating of performance ISO/IEC TR 14759:1999, Mockup and prototype ISO/IEC 14764:1999, Software maintenance ISO/IEC 15026:1998, System and software integrity levels ISO/IEC TR 15271:1998, Guide for ISO/IEC ISO/IEC TR 15504:1998, Software process assessment (9 parts) ISO/IEC TR 15846:1998, SWLC processes - Configuration management ISO/IEC 15910:1999, Software user documentation process ISO/IEC TR 16326:1999, Software project management SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 17

ISO/IEC 9126:1991, Software Product Quality Characteristics 9126 defines the characteristics and subcharacteristics intended to cover all aspects of software quality resulting from the ISO definition of quality. SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 18

New Generation of 9126 Family SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification : General Overview : Developers Process : Documentation of Evaluation Module Quality in Use Char. and Metrics External Char. and Metrics Internal Char. and Metrics Quality Model : Acquirers Process : Evaluators Process : Planning and Management ISO/IEC 9126 Quality Model and Metrics ISO/IEC14598 Product Evaluation ISO/IEC Quality Requirement and Testing Adapted from a chart by Motoei Azuma

ISO/IEC 12207:1995, Software Life Cycle Processes To establish a common framework for the life cycle of software – Broad scope: Acquire, supply, develop, operate, and maintain software – Recognizes that software is part of a system and that a project is part of an enterprise To establish a basis for world trade in software Amendment underway to describe processes at level of purpose and outcome SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 20

Example Use of Processes SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 21 Supply Documentation Quality Assurance Verification Validation Configuration Mgmt Joint Review Audit Problem Resolution Management Infrastructure Training Improvement Acquisition An Enterprise

ISO/IEC TR 15504, Software Process Assessment A nine-part Technical Report -- not a standard Currently under revision to become a five-part Standard – Will have a “process dimension” provided by an externally supplied process reference model* – Will have a 6 point “capability dimension”: Incomplete through Optimizing What conforms to 15504? -- the assessment. SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 22 *Processes other than software may be supplied.

ISO/IEC 15026, System and Software Integrity Levels SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 23 System Design Risk Analysis Software Design Risks, threats, frequency, initiators, consequences System integrity level Risk containment, fault tolerance strategies Risk dimensions Software component integrity levels

Draft ISO/IEC 15939, Software Measurement Process Activities and tasks necessary to identify, define, select, apply and improve software measurement with a project or organization. Based on principles of Practical Software Measurement (PSM) Designed to fit with ISO/IEC 12207, ISO/IEC 15504, and ISO 9000 series. SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 24

How to Characterize Measurement as a Process? SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 25 Control Process Resources ProductNeeds MeasurementAction Goals Constraints Source: [SESC93]

Draft ISO/IEC SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 26 Experience Base Sustain Commitment Plan the Measurement Process Perform the Measurement Process Evaluate Measurement Tech & Mgmt Processes Core Measurement Process Scope of Standard OrganizationProject

Relationship to Quality Management Standards Current relationship will be made obsolete by the circa 2000 revision of the ISO 9000 series. SC7 will take responsibility for the replacement of ISO and other documents on SW QM SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 27 ISO 9001ISO ISO/IEC Current Situation

Scope of SC7 was Changed in 1997 Standardization of processes, supporting tools and supporting technologies for the engineering of software products and systems SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 28 SC7 JTC1 Scope: Information technology Scope: Engineering of software products and systems

Rationale Software generally is part of a system or product Tight integration of software and other component in many software based systems. Examples: – Telecommunication products (Hardware, Silicon, Software) – Fly-by-wire aircraft SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 29 Based on a chart by François Coallier, SC7 Chair

Working Groups with System Scope WG2: System software documentation WG4: Tools and environment WG6: Evaluation & metrics WG7: Life cycle management WG9: System & SW integrity WG10: Process assessment WG12: Functional size measurement WG13: Software measurement process WG17: ODP enterprise lang WG18: Quality mgmt sWG18: Revise ISO WG19: ODP and modelling languages SWG1: Planning SWG2: Vocabulary SWG3: Process architecture SWG4: Web and communication support Study Group: SE practices SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 30

How Can You Participate? US delegates to meetings of ISO/IEC SC7 and its working groups are selected from representatives to the US Technical Advisory Group. US positions are developed by the TAG. Any US-domiciled organization can join the TAG -- $300 per year. The TAG meets three times a year. SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 31

How the US is Represented in JTC1 and SC7 SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 32 JTC1 SC7 Working Groups JTC1 TAG: Members are US domiciled organizations SC7 TAG: Members are US domiciled organizations TAG Task Groups TAG Administrator ANSIANSI

Help Wanted! SW product quality models and metrics Systems processes and systems engineering practices Extension of process assessment to systems and business processes System/software dependability issues Non-traditional functional size measurement Comprehensive SW quality management Also... Documentation, CASE, electronic data interchange SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 33

IEEE as an Integrator of Software Engineering Standards SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 34

SW Engineering Standards Five Years from Now SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 35 SWE Body of Knowledge Fundamental Principles ISO/IEC Integrating Framework IEEE DoD ISO/IEC Umbrella Standard(s) IEEE ISO/IEC Integrated Collection of Practice Standards IEEE Motivate Curricula Licensing Exams Professional Practice Body of Standards Quality Management Project Management System Engineering Process Definition Process Assessment Process Measurement Best PracticesCompetency Certification Organizational Adoption Buyer/Seller Agreement Critical System Assurance

Guide to the Software Engineering Body of Knowledge Collaboration between IEEE CS and UQAM. International participation from industry, professional societies, standards bodies, academia, authors By the time the project is finished literally thousands of individuals will have touched it About to complete the middle of three phases SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 36

Guide to the Software Engineering Body of Knowledge Objectives – Better characterize the SWE discipline – Provide an index or taxonomy of the BOK – Promote a consistent view of SWE – Clarify relationship of SWE to other disciplines – Act as an indicator of maturity of the discipline – Provide a flagship document for SWE standards SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 37

Baseline List of Knowledge Areas Requirements Analysis Design Construction Testing Evolution and Maintenance Configuration Management Quality Analysis Engineering Infrastructure Engineering Process Engineering Management SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 38 Computer Science (CC2001) Mathematics (CC2001) Project Management (PMBOK) Computer Engineering Cognitive Sciences and Human Factors Systems Engineering Management and Management Science Related Disciplines

Knowledge Area Description SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 39 Classification of Topics Matrix of Topics & References References Topic Descriptions Classification by Vincenti’s Taxonomy Classification by Bloom’s Taxonomy References to Related Disciplines Not in Stone- man Version

SW Engineering Standards Five Years from Now SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 40 SWE Body of Knowledge Fundamental Principles ISO/IEC Integrating Framework IEEE DoD ISO/IEC Umbrella Standard(s) IEEE ISO/IEC Integrated Collection of Practice Standards IEEE Motivate Curricula Licensing Exams Professional Practice Body of Standards Quality Management Project Management System Engineering Process Definition Process Assessment Process Measurement Best PracticesCompetency Certification Organizational Adoption Buyer/Seller Agreement Critical System Assurance

Examples of Fundamental Principles Currently, 16 candidates. Examples are … – Since change is inherent in software, plan for it and manage it. Comme le logiciel est, de par sa nature même, sujet au changement, il faut planifier et gérer ce changement. – Since tradeoffs are inherent to SWE, make them explicit and document them. Les compromis étant inhérents au GL, il faut les rendre explicites et les documenter. SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 41 Source: [Dupuis97]

SW Engineering Standards Five Years from Now SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 42 SWE Body of Knowledge Fundamental Principles ISO/IEC Integrating Framework IEEE DoD ISO/IEC Umbrella Standard(s) IEEE ISO/IEC Integrated Collection of Practice Standards IEEE Motivate Curricula Licensing Exams Professional Practice Body of Standards Quality Management Project Management System Engineering Process Definition Process Assessment Process Measurement Best PracticesCompetency Certification Organizational Adoption Buyer/Seller Agreement Critical System Assurance

Organizing the SESC standards: Objects of software engineering SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 43 interacts with Agent CustomerProduct Process Resource performs uses produces Project transforms applies to aids Source: [SESC95]

Organizing the SESC standards: Relationship to other disciplines SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 44 Software Engineering Computer Science and Technology Application Domains Project Management Systems Engineering Quality Management Dependability Safety

Organization of SESC Collection SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 45 Overall Guide Terminology “Toolbox” of Technique Standards CustomerResourceProcessProduct Principles or Policies Element Standards Application Guides System Disciplines ISO SW Engineering Quality Management Project Management Contextual Standards IEEE SESC Standards Program Source: [SESC95]

Overall Guide to the IEEE Collection Rather than developing a consensus document, IEEE SESC “commissioned” the writing of an overall guide to the collection. Relevant standards are “sliced and diced” by: – Context – Object – Normative intent – Provider and subject Relationships are explained, even between standards from different providers. Published by IEEE Computer Society Press. 328 pages. 6" x 9" Softcover. ISBN $40.00 List SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 46

Relating the Four-Volume Edition to the SESC Model SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 47 CustomerProcessProductResource Techniques Layer Overall Guide Terminology Layer TechniquesTerminology Software Engineering, 1999 Edition (4 Volume Set) ISBN List: $ IEEE Mbr: $ g/catalog/softwareset.h tml

SW Engineering Standards Five Years from Now SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 48 SWE Body of Knowledge Fundamental Principles ISO/IEC Integrating Framework IEEE DoD ISO/IEC Umbrella Standard(s) IEEE ISO/IEC Integrated Collection of Practice Standards IEEE Motivate Curricula Licensing Exams Professional Practice Body of Standards Quality Management Project Management System Engineering Process Definition Process Assessment Process Measurement Best PracticesCompetency Certification Organizational Adoption Buyer/Seller Agreement Critical System Assurance

Relationship to other Standards IEEE SESC has adopted policy designating as a strategic, integrating standard for its collection. IEEE SESC will revise its other standards to improve their fit with Many of the other standards will detail the processes of SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 49

SWLC Process Standards: DoD Family Tree SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 50 DoD-Std 2167A DoD-Std 2167 Mil-Std 1679 Mil-Std 498 EIA/IEEE J-Std-016 NSA 1703 DoD-Std 7935A

SWLC process standards: Commercial family tree SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 51 Future revision ISO/IEC ISO/IEC IEEE/EIA Std EIA/IEEE J-Std-016 Mil-Std 498 IEEE Std 1074

Example Use of Processes SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 52 Supply Documentation Quality Assurance Verification Validation Configuration Mgmt Joint Review Audit Problem Resolution Management Infrastructure Training Improvement Acquisition An Enterprise Adding Reuse & Risk Mgmt

Improvements in IEEE/EIA 12207: Data Guidance Part 1 may be used as a Guide or a Standard Defines 84 information items related to data requirements of – Defines seven generic kinds of data that account for 44 of the information items – Defines 30 specific content guidelines that account for the other 40 information items The information items are data requirements … not documents SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 53

SW Engineering Standards Five Years from Now SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 54 SWE Body of Knowledge Fundamental Principles ISO/IEC Integrating Framework IEEE DoD ISO/IEC Umbrella Standard(s) IEEE ISO/IEC Integrated Collection of Practice Standards IEEE Motivate Curricula Licensing Exams Professional Practice Body of Standards Quality Management Project Management System Engineering Process Definition Process Assessment Process Measurement Best PracticesCompetency Certification Organizational Adoption Buyer/Seller Agreement Critical System Assurance

Overall SESC Collection SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 55 Terminology Overall Guide Principles Element Standards Application Guides and Supplements “Toolbox” of Techniques See figure below for Customer standards See figure below for Process standards IEEE glossary Road Map Book See figure below for Product standards See figure below for Resource standards 1044 Classification of anomalies Guide to 1044 Adapted from [Moore97a]

SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 56 Process Customer Product Resource

Process adoption: Is this the best way? SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 57 Project Manager Total Quality Management Process definition Corporate initiatives Contractual requirements Government regulations Safety standards Project manage- ment needs Systems engineering methods Capability evaluation Tooling needs“Best practices”

Traditional project-level compliance with standards SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 58 Enterprise Processes Project Processes Project Processes Software engineering standards Project claims compliance to standards Every project adopts a unique set of procedures, etc. “Message”: SW engineering standards are irrelevant to organizational capability and maturity. Procedures, practices, templates, etc. Procedures, practices, templates, etc.

Enterprise level adoption SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 59 Enterprise Processes Enterprise Processes Project Processes Project Processes Software engineering standards Enterprise claims compliance Project is able to use enterprise procedures, etc. “Message”: SW engineering standards are the basis for implementing repeatable, improving processes. Procedures, practices, templates, etc. Procedures, practices, templates, etc. Project complies with enterprise processes

Relationship of and CMM gives requirements for the implementation of processes -- definition CMM assesses the maturity of the implementation of the processes -- measurement SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 60 Control Process Resources ProductNeeds Evaluation / Assessment / Measurement Action Goals Constraints Source: [SESC93] Focus of Focus of CMM

Relationship of to SEI SW-CMM Key Process Areas SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 61

How Can You Participate? IEEE standards working groups are open to anyone. To vote on the standards, you must be a member of IEEE or IEEE Computer Society and a member of the IEEE Standards Association SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 62

Contact information Address for postal mail: The MITRE Corporation Attn: James W. Moore, M/S W Dolley Madison Blvd. McLean, VA Address for express delivery services: The MITRE Corporation Attn: James W. Moore, (703) Sunset Hills Road Reston, VA Electronic mail: Phone numbers: Office: Fax: SIGAda - JWM0012 © 2000, The MITRE Corporation. Permission is granted to reproduce without modification. 63