Credibility and Competitive Advantage - Thoughts on OLIF Conformance Assessment - Christian Lieske SAP AG - GBU Application Integration – MultiLingual Technology

SAP AG, Credibility and Competitive Advantage, C. Lieske, 12 Dec 2001, 2 Overview This presentation is meant to provide a foundation for discussing issues related to OLIF-conformity. The following topics will be addressed: 1.Certification 2.Conformance Assessment 3.Conformance Criteria and Test Harness 4.Suggested Next Steps and Discussion

SAP AG, Credibility and Competitive Advantage, C. Lieske, 12 Dec 2001, 3 OLIF-Certification – Why? OLIF is a means of interoperability for IT-related products, processes and services and thus is a pivotal element for economic success OLIF can only live up to its promise if OLIF-implementations are tested for quality (in the sense of conformance with the OLIF specification) OLIF certification helps to ensure and communicate quality (here: compliance with standard) OLIF itself gains credibility through high quality implementations OLIF vendors achieve competitive advantage via certified implementations

SAP AG, Credibility and Competitive Advantage, C. Lieske, 12 Dec 2001, 4 OLIF-Certification – Glossary ConformanceFulfillment of a product, process, or service of specified requirements Conformance ClauseSection in a specification or standard that states requirements (conformance criteria) Conformance Assessment Determines whether an implementation of a specification or standard conforms to the specification or standard Test HarnessCombination of test software, test data, test procedures and test documentation used to check an implementation for conformance to a specification ValidationProcess of testing an implementation for conformance by means of a test harness CertificationOfficial acknowledgment that validation has been completed and criteria for issuing a certificate have been met

SAP AG, Credibility and Competitive Advantage, C. Lieske, 12 Dec 2001, 5 OLIF-Certification – Possible Scenario Elements Actors Activities Achievements Buyer Seller Test Laboratory (TL) Certificate Issuer (CI) Control Board (CB) Require Certification Validate IUT Recognize Test Method Accredit TL Judge Validation Results Answer Programmatic Queries Answer Test Method Queries Resolve Test Method Disputes Issue Certificate Specification Certification Program Policy TL Criteria Certificate of Conformance Implementation Under Test (IUT) Test Method Test Report

SAP AG, Credibility and Competitive Advantage, C. Lieske, 12 Dec 2001, 6 OLIF-Certification – Prerequisites Certification Program Policy Clarifies responsibilities of all actors (e.g. the control board) Defines policies for activities (e.g. the accrediation of test laboratory) Lists achievements (e.g. the test suites) Conformance Criteria Features and level of quality Categories for different certificates of conformance Test Harness Adequate (provides measures to judge conformance) Appropriate (no undue requirement like injustifiable costs) Clarification of Business Issues Ramp-up costs and running costs Revenue distribution Concertation with TBX/LISA/SALT

SAP AG, Credibility and Competitive Advantage, C. Lieske, 12 Dec 2001, 7 OLIF-Certification – Possible Process

SAP AG, Credibility and Competitive Advantage, C. Lieske, 12 Dec 2001, 8 OLIF-Certification – Business Issues Who owns (in terms of copyright and intellectual property) the various elements of the certification? 1. OLIF logo 2. Overall certification process/programme 3. Logo for certification mark 4. Test harness How are costs and revenues distributed? 1. Costs for ramp-up of certification process 2. Costs for validation 3. Revenues for/from certification How should the relationship between OLIF-certification and TBX/LISA/SALT be defined?

SAP AG, Credibility and Competitive Advantage, C. Lieske, 12 Dec 2001, 9 Conformance Assessment – OLIF-Objects Conformance in the context of OLIF can be formulated for objects such as 1.Functional modules Import Export 2.Standalone programs Java package for checking canonical forms Command line term extraction program that generates OLIF 3.Files Monolingual Multilingual With or without use of extension mechanisms

SAP AG, Credibility and Competitive Advantage, C. Lieske, 12 Dec 2001, 10 Conformance Criteria – Examples For each type of OLIF-object conformance criteria like the following have to be formulated 1.Functional modules Produce XML that validates against the OLIF DTD 2.Standalone programs Apply mechanism for checking canonical forms to all canonical forms in an OLIF file (terms, transfers, synonyms …) 3.Files Does not contain any dangling references (e.g. between transfer and referenced entry) Conformance criteria should be measurable without undue effort

SAP AG, Credibility and Competitive Advantage, C. Lieske, 12 Dec 2001, 11 Conformance Criteria – Basic Categories Conformance criteria can be categorized in terms of value/difficulty. Example categories for OLIF files are Mandatory 1. Validates against OLIF DTD 2. Represents data in header correctly Optional 1. Uses extension mechanisms correctly 2. Represents terms according to guidelines for canonical forms Based on categorized conformance criteria, different flavours of certificates of conformance can be envisioned (examples pertain to files) Platinum (level 3) All mandatory and optional conformance criteria are met Gold (level 2) All mandatory and some optional conformance criteria are met Silver (level 1) All mandatory conformance criteria are met

SAP AG, Credibility and Competitive Advantage, C. Lieske, 12 Dec 2001, 12 Conformance Criteria – Dimensions for Categorization Ex: compliant to XML specification Valid Structure Ex: Quality of definition Accurate Content Conformant/adherent Extension processing Ex: Include check against data category specifications

SAP AG, Credibility and Competitive Advantage, C. Lieske, 12 Dec 2001, 13 Test Harness 1.Main ingredients a. Test software b. Test data c. Test procedures/test suites d. Test documentation 2.Traceability Each item in the test harness has to be connected to the conformance criteria to which it pertains 3.Catalogue of error codes Without such a catalogue, it is hard to improve an implementation under test in an effective way

SAP AG, Credibility and Competitive Advantage, C. Lieske, 12 Dec 2001, 14 Suggested Next Steps Establishment of working groups for 1.Certification Program Policy 2.Conformance Criteria 3.Test Harness 4.Clarification of Business Issues 5.Concertation with TBX/LISA/SALT

SAP AG, Credibility and Competitive Advantage, C. Lieske, 12 Dec 2001, 15 Discussion