M.-E. Bégin¹, S. Da Ronco², G. Diez-Andino Sancho¹, M. Gentilini³, E. Ronchieri ², and M. Selmi² ¹CERN, Switzerland, ² INFN-Padova, Italy, ³INFN-CNAF, Bologna, Italy ETICS Meta-Data Software Editing - From Check Out To Commit Operations References [1] M. E. Bégin, et al., Build, Configuration, Integration and Testing Tools for Large Software Projects: ETICS, In Proceeding of RISE 2006 International Workshop, September, 2006, University of Geneva, Switzerland [2] The Grid Quality Process, References [1] M. E. Bégin, et al., Build, Configuration, Integration and Testing Tools for Large Software Projects: ETICS, In Proceeding of RISE 2006 International Workshop, September, 2006, University of Geneva, Switzerland [2] The Grid Quality Process, etics-get-project P Scenario 1 A possible scenario consists of a user that downloads a project, called P. Then, the user creates a subsystem A that contains two components, called A-1 and A-2. Next, the user modifies subsystem and project configurations in order to link respectively the new component configurations (i.e., A-1.HEAD, A-2.HEAD) and subsystem configuration (i.e., A.HEAD). Finally, the user commits changes performed locally. Scenario 1 A possible scenario consists of a user that downloads a project, called P. Then, the user creates a subsystem A that contains two components, called A-1 and A-2. Next, the user modifies subsystem and project configurations in order to link respectively the new component configurations (i.e., A-1.HEAD, A-2.HEAD) and subsystem configuration (i.e., A.HEAD). Finally, the user commits changes performed locally. Scenario 2 Another scenario consists of a user that downloads a project, called P1. Then, the user modifies an existing component configuration (B.HEAD) changing a static dependency, then addes a new component configuration (C.HEAD), next modifies the previous configuration (B.HEAD) adding the new one as dynamic dependency. At this point the user commits changes performed locally. Scenario 2 Another scenario consists of a user that downloads a project, called P1. Then, the user modifies an existing component configuration (B.HEAD) changing a static dependency, then addes a new component configuration (C.HEAD), next modifies the previous configuration (B.HEAD) adding the new one as dynamic dependency. At this point the user commits changes performed locally. User 2 Get project P1 > etics-get-project P1; > etics-commit Commit Modify configuration B.HEAD > etics-configuration prepare -c B.HEAD B; > etics-configuration modify \ -i Configuration-B.HEAD.ini; Add configuration C.HEAD > etics-configuration prepare \ --new -c C.HEAD C; > etics-configuration add \ -i Configuration-C.HEAD.ini; Modify configuration B.HEAD > etics-configuration modify \ -i Configuration-B.HEAD.ini; User 1 Get project P > etics-get-project P; > etics-configuration prepare -c P.HEAD P; > etics-configuration modify \ -i Configuration-P.HEAD.ini; > etics-commit Commit Modify configuration P.HEAD > etics-configuration prepare -c A.HEAD A; > etics-configuration modify \ -i Configuration-A.HEAD.ini; > etics-module prepare \ --new --parent A --component A-2; > etics-module add -i Component-A-2.ini; Create component A-2 > etics-module prepare \ --new --parent A --component A-1; > etics-module add -i Component-A-1.ini; > etics-module prepare --new -s A; > etics-module add -i Subsystem-A.ini; Create subsystem A Create component A-1 Modify configuration A.HEAD Example of Component-A-1.ini file: [Component-A-1] licenceType = ETICS vendor = ETICS description = None repository = download = None packageName = A.1 homepage = None vcsroot = [Parent] Subsystem = A Example of Component-A-1.ini file: [Component-A-1] licenceType = ETICS vendor = ETICS description = None repository = download = None packageName = A.1 homepage = None vcsroot = [Parent] Subsystem = A Example of Configuration-B.HEAD.ini file: [Configuration-B.HEAD] profile = None status = None moduleName = P1.B description = P1.B v version = path = None age = 1 tag = B_branch_1_0_0 [Platform-default:BuildCommand] … [Platform-default:TestCommand] … [Platform-default:VcsCommand] checkout = echo ‘hi’ [Platform-default:Property] … [Platform-default:Environment] … [Platform-default:StaticDependency] externals|globus = globus v ,B [Platform-default:DynamicDependency] P1|C = C.HEAD,R Example of Configuration-B.HEAD.ini file: [Configuration-B.HEAD] profile = None status = None moduleName = P1.B description = P1.B v version = path = None age = 1 tag = B_branch_1_0_0 [Platform-default:BuildCommand] … [Platform-default:TestCommand] … [Platform-default:VcsCommand] checkout = echo ‘hi’ [Platform-default:Property] … [Platform-default:Environment] … [Platform-default:StaticDependency] externals|globus = globus v ,B [Platform-default:DynamicDependency] P1|C = C.HEAD,R It is important to observe that the ini file name contains as prefix one of the following values: Component, Subsystem, Project and Configuration. The prefix is followed by – plus a name. List of Operations handled by the module and configuration Commands add clone that creates a new data; that copies existing data (used for configuration only); modify rename that modifies existing data; that changes the name of existing data; preparethat generates a plain-text representation of existing data; removethat deletes existing data. Editing Commands etics-moduleused for project, component and subsystem data, generically called module; etics-configurationused for configuration; etics-commitused for committing changes performed locally. Details of Commands and Operations Web Application Web Service Via command-line tools Via browser Remote build/test infrastructure Database Figure 1: ETICS Architecture Introduction Experience in Grid projects, like EGEE, has shown that the use of version control and build systems is not able to support the development of the software efficiently, due to the large number of errors that cause the breaking of the build process. Common errors are, for example: the adoption of new libraries, libraries incompatibility, the extension of the current project in order to support new software modules. Solution ETICS, an integrated infrastructure for the automated configuration, build and test of Grid and distributed software [1, 2], has defined meta-data software abstractions, from which it is possible to download, build and test software projects. It is possible, for example, to set dependencies amongst components, and define what environment variables are necessary in order to execute a given component. Meta-data Meta-data is managed by ETICS in the same way than a version control system. This is possible thanks to the existence of a meta-data repository and the handling of a list of operations (e.g., check out, commit). Figure 1 synthesizes ETICS Architecture. Implementation A Project Metadata (e.g., project structure, VCS/Build/Test Commands, Environment Variables) is stored in a centralized relational database. In order to build or test, the clients will download an XML representation of the meta-data in their respective working areas. Introduction Experience in Grid projects, like EGEE, has shown that the use of version control and build systems is not able to support the development of the software efficiently, due to the large number of errors that cause the breaking of the build process. Common errors are, for example: the adoption of new libraries, libraries incompatibility, the extension of the current project in order to support new software modules. Solution ETICS, an integrated infrastructure for the automated configuration, build and test of Grid and distributed software [1, 2], has defined meta-data software abstractions, from which it is possible to download, build and test software projects. It is possible, for example, to set dependencies amongst components, and define what environment variables are necessary in order to execute a given component. Meta-data Meta-data is managed by ETICS in the same way than a version control system. This is possible thanks to the existence of a meta-data repository and the handling of a list of operations (e.g., check out, commit). Figure 1 synthesizes ETICS Architecture. Implementation A Project Metadata (e.g., project structure, VCS/Build/Test Commands, Environment Variables) is stored in a centralized relational database. In order to build or test, the clients will download an XML representation of the meta-data in their respective working areas. Table 1: Some editing commands Table 2: List of Operations (1) Work Area Web Service Non-Local Solution Local Solution For each operation commit (1) (2) operations Figure 2 shows the order of the actions performed by the local and non-local solutions. ETICS provides various commands to handle the metadata locally (some editing commands are described on Table 1). When a local editing command is executed, all the local changes are tracked in a local history file Different commands support different operations, as described in Table 2. ETICS provides a transaction mechanism, which avoids any possible intermediate state causing data corruption. Figure 2: Local and Non-local solutions actions. (1) and (2) represent the order of the actions.