1. Model-Independent Schema and Data Translation Paolo Atzeni Università Roma Tre Based on Tutorial at ICDE 2006 Paper in EDBT 2006 (with P. Cappellari and P. Bernstein) JISBD 2006 --- Sitges, 5 October 2006

2. P. AtzeniJISBD - October 5, 20062 Outline Introduction Model management Schema and data translation: the problem A metamodel based approach

3. P. AtzeniJISBD - October 5, 20063 A ten-year goal for database research The “Asilomar report” (Bernstein et al. Sigmod Record 1999 www.acm.org/sigmod): –The information utility: make it easy for everyone to store, organize, access, and analyze the majority of human information online

4. P. AtzeniJISBD - October 5, 20064 A general framework: cooperation "The capacity of a system to interact (effectively) with other systems, possibly managed by different organizations" Forms of cooperation –Process-centered cooperation: the systems offer services one another, by exchanging messages, or by triggering activities, without making remote data explicitly visible –Data-centered cooperation: the systems offer data one another; data is distributed, heterogeneous and autonomous

5. P. AtzeniJISBD - October 5, 20065 Databases in the Internet era Databases before the Internet –An internal infrastructure, a precious resource, but usually hidden, with some controlled cooperation Internet changes the requirements –More users (not only humans), more diverse cooperating systems (distributed, heterogeneous, autonomous), more types of data "Future" Internet changes more –New devices (embedded everywhere), even more users (many “per person”), real mobility, need for personalization and adaptation

6. P. AtzeniJISBD - October 5, 20066 The most studied form of data-centered cooperation: integration We are interested in data-centered cooperation, often referred to as integration “The unification of related, heterogeneous data from disparate sources, for example, to enable collaboration” (Hammer & Stonebraker 2005) Some "paradigms" …

7. P. AtzeniJISBD - October 5, 20067 Multidatabase Global Manager Local mgr DB Mediator Local mgr DB Mediator Local mgr DB Mediator

8. P. AtzeniJISBD - October 5, 20068 Data Warehousing System Mediator Local manager Mediator Local manager Mediator Local manager “Integrator” DB Data Warehouse DW manager

9. P. AtzeniJISBD - October 5, 20069 Intermediate solutions in practice Mediator Local manager DB Local manager DB Integrator Mediator Local manager Mediator Local manager DB Local Manager Application

10. P. AtzeniJISBD - October 5, 200610 Peer-based architecture Peer mgr Local mgr DB Local mgr DB Mediator Peer mgr Local mgr DB Local mgr DB Mediator Peer mgr Local mgr DB Local mgr DB Mediator

11. P. AtzeniJISBD - October 5, 200611 Data is not just in databases Mail messages Web pages Spreadsheets Textual documents Palmtop devices, mobile phones Multimedia annotations (e.g., in digital photos) XML documents

12. P. AtzeniJISBD - October 5, 200612 The same data in the same form? Adaptivity: –Personalization: content adapted to the user upon system's decision upon user's request –Customization: structure adapted to the user according to the user's role upon user's request –Context dependence User, Device, Network, Place, Time, Rate

13. P. AtzeniJISBD - October 5, 200613 Summarizing: a general need We have data at various places, and data has to be –exchanged –replicated –migrated –integrated –adapted

14. P. AtzeniJISBD - October 5, 200614 A major difficulty Heterogeneity –System level –Model level –Structural (different structure for similar data) –Semantic (different meaning for the same structure) Many efforts, but current techniques are mostly manual and ad hoc

15. P. AtzeniJISBD - October 5, 200615 A direction for the solutions Be general! –ad hoc solution could work in-the-small, but they are repetitive and time consuming do not scale are not maintainable Historical notes: –W. C. McGee: Generalization: Key to Successful Electronic Data Processing. J. ACM 1959 Indeed, databases are the result of generalization applied to secondary storage management!

16. P. AtzeniJISBD - October 5, 200616 Generality requires … … high-level descriptions of problems within the family of interest: –Metadata: “data about data” (formal or informal) description of structures and meaning General solutions leverage on metadata (and then operate on data as a consequence)

17. P. AtzeniJISBD - October 5, 200617 Outline Introduction and motivation Model management Schema and data translation: the problem A metamodel based approach

18. P. AtzeniJISBD - October 5, 200618 A wider perspective (Generic) Model Management: –A proposal by Bernstein et al (2000 +) –Includes a set of operators on schemas and mappings between schemas

19. P. AtzeniJISBD - October 5, 200619 Terminology: a warning Model Mgmt peopleTraditional DB people Meta-metamodelMetamodel Model Schema

20. P. AtzeniJISBD - October 5, 200620 Schemas and mappings A simplified approach: –Schema: a set of elements, related in some way to one another –Mapping: a set of correspondences (pair of elements) or its reification, a third schema related to the other two via two sets of correspondences

21. P. AtzeniJISBD - October 5, 200621 Model mgmt operators, a first set map = Match (S1, S2) S3 = Merge (S1, S2, map) S2 = Diff (S1, map) and more –map3 = Compose (map1, map2) –S2 = Select (S1, pred) –Apply (S, f) –list = Enumerate (S) –S2 = Copy (S1) –…

22. P. AtzeniJISBD - October 5, 200622 Match map = Match (S1, S2) –given two schemas S1, S2 –returns a mapping between them the “classical” initial step in data integration: –find the common elements of two schemas and the correspondences between them

23. P. AtzeniJISBD - October 5, 200623 Merge S3 = Merge (S1, S2, map) –given two schemas and a mapping between them –returns a third schema (and two mappings) the “classical” second step in data integration: –given the correspondences, find a way to obtain one schema out of two

24. P. AtzeniJISBD - October 5, 200624 Diff S2 = Diff (S1, map) –given a schema and a mapping from it (to some other schema, not relevant) –returns a (sub-)schema, with the elements that do not participate in the mapping

25. P. AtzeniJISBD - October 5, 200625 DW2 Example (Bernstein and Rahm, ER 2000) A database (a “source”), a data warehouse and a mapping between the two we get a second source, with some similarity to the first one and we want to update the DW DB1 DW1 DB2

26. P. AtzeniJISBD - October 5, 200626 DW2 Example, the "solution" DB1 DW1 DB2 m1 m2 m3 DB2’ m2 = Match(DB1,DB2) m3= Compose(m2,m1) DB2’=Diff(DB2,m3) DW2’, m4 user defined m5 = Match(DW1,DW2’) DW2 = Merge(DW1,DW2’,m5) DW2’ m4 m5

27. P. AtzeniJISBD - October 5, 200627 Magic does not exist Operators might require human intervention: –Match is the main case Scripts involving operators might require human intervention as well (or at least benefit from it): –a full implementation of each operator might not always available –a mapping might require manual specification –incomparable alternatives might exist

28. P. AtzeniJISBD - October 5, 200628 The “data level” The major operators have also an extended version that operates on data, and not only on schemas Especially apparent for –Merge

29. P. AtzeniJISBD - October 5, 200629 We also have heterogeneity Round trip engineering (Bernstein, CIDR 2003) –A specification, an implementation –then a change to the implementation: want to revise the specification We need a translation from the implementation model to the specification one S1 I1I2 S2

30. P. AtzeniJISBD - October 5, 200630 Model management with heterogeneity The previous operators have to be “model generic” (capable of working on different models) We need a “translation” operator – = ModelGen (S1)

31. P. AtzeniJISBD - October 5, 200631 ModelGen, an additional operator = ModelGen (S1) –given a schema (in a model) –returns a schema (in a different data model) and a mapping between the two A “translation” from a model to another I should call it “SchemaGen” … We should better write – = ModelGen (S1,mod2)

32. P. AtzeniJISBD - October 5, 200632 S2 Round trip engineering S1 I1 m1 I2 m2 = Match (I1,I2) m3 = Compose (m1,m2) I2’= Diff(I2,m3) = Modelgen(I2’) … Match, Merge m2 m3 I2’ S2’ m4

33. P. AtzeniJISBD - October 5, 200633 Outline Introduction Model management Schema and data translation: the problem A metamodel based approach

34. P. AtzeniJISBD - October 5, 200634 Schema and data translation, a long standing issue Schema translation: –given schema S1 in model M1 and model M2 –find a schema S2 in M2 that “corresponds” to S1 Schema and data translation: –given also a database D1 for S1 –find also a database D2 for S2 that “contains the same data” to D1

35. P. AtzeniJISBD - October 5, 200635 Schema and data translation, a long standing issue Translations from a model to another have been studied since the 1970’s Whenever a new model is defined, techniques and tools to generate translations are studied However, proposals and solutions are usually model specific

36. P. AtzeniJISBD - October 5, 200636 Model specific solutions Given an ER schema, find the suitable relational schema that “implements” it –the original paper (Chen 1976) contains the basics –further discussions by many (e.g. Markowitz and Shoshani 1989) –illustrated in every textbook Similarly with –any other conceptual model and any other logical one –XML and relational (or object)

37. P. AtzeniJISBD - October 5, 200637 Another problem in the picture: data exchange Given a source S1 and a target schema S2 (in different models or even in the same one), find a translation, that is, a function that given a database D1 for S1 produces a database D2 for S2 that “correspond” to D1 Often emphasized with reference to materialized solutions

38. P. AtzeniJISBD - October 5, 200638 Integration Given two or more sources, build an integrated schema or database

39. P. AtzeniJISBD - October 5, 200639 Schema translation Given a schema find another one with respect to some specific goal (better quality, another model, …)

40. P. AtzeniJISBD - October 5, 200640 Data exchange Given a source and a target schema, find a transformation from the former to the latter

41. P. AtzeniJISBD - October 5, 200641 Schema translation and data exchange Can be seen a complementary: –Data translation = schema translation + data exchange Given a source schema and database Schema translation produces the target schema Data exchange generates the target database In model management terms we could write –Schema translation: = ModelGen (S1,mod2) –Data exchange: i2 = DataGen (S1,i1,S2,map12)

42. P. AtzeniJISBD - October 5, 200642 Outline Introduction Model management Schema and data translation: the problem A metamodel based approach

43. P. AtzeniJISBD - October 5, 200643 The problem ModelGen –given two data models M1 and M2, and a schema S1 of M1 (the source schema and model), generate a schema S2 of M2 (the target schema and model), corresponding to S1 and, for each database D1 over S1, generate an equivalent database D2 over S2

44. P. AtzeniJISBD - October 5, 200644 We have been doing this for a while Initial work more than ten years ago (Atzeni & Torlone, 1996) Major novelty recently –translation of both schemas and data –data-level translations generated, from schema-level ones Moreover: –a visible, multilevel and (in part) self-generating dictionary –high-level, visible and customizable translation rules in Datalog with OID-invention –mappings between elements generated as a by-product

45. P. AtzeniJISBD - October 5, 200645 Heterogeneity We need to handle artifacts and data in various models –Data are defined wrt to schemas –Schemas are defined wrt to models –How models can be defined? Models Schemas Data

46. P. AtzeniJISBD - October 5, 200646 A metamodel approach The constructs in the various models are rather similar: –can be classified into a few categories (Hull & King 1986): Lexical: set of printable values (domain) Abstract (entity, class, …) Aggregation: a construction based on (subsets of) cartesian products (relationship, table) Function (attribute, property) Hierarchies … We can fix a set of metaconstructs (each with variants): –lexical, abstract, aggregation, function,... –the set can be extended if needed, but this will not be frequent A model is defined in terms of the metaconstructs it uses

47. P. AtzeniJISBD - October 5, 200647 The metamodel approach, example The ER model: –Abstract (called Entity) –Function from Abstract to Lexical (Attribute) –Aggregation of abstracts (Relationship) –… The OR model: –Abstract (Table with ID) –Function from Abstract to Lexical (value-based Attribute) –Function from Abstract to Abstract (reference Attribute) –Aggregation of lexicals (value-based Table) –Component of Aggregation of Lexicals (Column) –…

48. P. AtzeniJISBD - October 5, 200648 The supermodel A model that includes all the meta-constructs (in their most general forms) –Each model is subsumed by the supermodel (modulo construct renaming) –Each schema for any model is also a schema for the supermodel (modulo construct renaming)

49. P. AtzeniJISBD - October 5, 200649 A Multi-Level Dictionary Handles models, schemas and data Has both a model specific and a model independent component

50. P. AtzeniJISBD - October 5, 200650 Multi-Level Dictionary model schema model generic model specific description model independence Supermodel description (mSM) Model descriptions (mM) Supermodel schemas (SM) Model specific schemas (M) Supermodel instances (i-SM) Model specific instances (i-M) data

51. P. AtzeniJISBD - October 5, 200651 The supermodel description MSM-Construct OIDNameIsLex 1AggregationOfLexicalsF 2ComponentOfAggrOfLexT 3AbstractF 4AttributeOfAbstractT 5BinaryAggregationOfAbstractsF MSM-Property OIDNameConstr.Type 11Name1String 12Name2String 13IsKey2Boolean 14IsNullable2Boolean 15Type2String 16Name3String 17Name4String 18IsIdentifier4Boolean 19IsNullable4Boolean 20Type4String 21IsFunct15Boolean 22IsOptional15Boolean 23Role15String 24IsFunct25Boolean 25IsOptional25Boolean 26Role25String MSM-Reference OIDNameConstructTarget 30Aggregation21 31Abstract43 32Abstract153 33Abstract253 mSMmM SMM i-SMi-M

52. P. AtzeniJISBD - October 5, 200652 Schemas in the supermodel SM-Abstract OIDSchemaName 3011Employees 3021Departments 2013Clerks 2023Offices SM-AttributeOfAbstract OIDSchemaNameisIdentisNullableTypeAbstrOID 4011EmpNoTFInt301 4021NameFFText301 4041NameTFChar302 4051AddressFFText302 5013CodeTFInt201 ………………… MSM-Construct OIDNameIsLex ……… 3AbstractF 4AttributeOfAbstractT ……… Supermodel schemas mSMmM SMM i-SMi-M Employees Departments EmpNo Name Address

53. P. AtzeniJISBD - October 5, 200653 i-SM-Abstract OIDAbsOIDInstOID 30103011 30113011 ……… 30103012 i-SM-AttributeOfAbstract OIDAttrOfAbsOIDInstOIDValuei- AbsOID 40104011754323010 40204021John Doe3010 … 40214021Bob White3011 …………… Instances in the supermodel SM-Abstract OIDSchemaName 3011Employees 3021Departments 2013Clerks 2023Offices SM-AttributeOfAbstract OIDSchemaNameisIdentisNullableTypeAbstrOID 4011EmpNoTFInt301 4021NameFFText301 4041NameTFChar302 4051AddressFFText302 5013CodeTFInt201 ………………… Supermodel schemas Supermodel instances mSMmM SMM i-SMi-M

54. P. AtzeniJISBD - October 5, 200654 Multi-Level Repository model schema model generic model specific description model independence Supermodel description (mSM) Model descriptions (mM) Supermodel schemas (SM) Model specific schemas (M) Supermodel instances (i-SM) Model specific instances (i-M) data

55. P. AtzeniJISBD - October 5, 200655 Model descriptions MSM-Construct OIDNameIsLex 1AggregationOfLexicalsF 2ComponentOfAggrOfLexT 3AbstractF 4AttributeOfAbstractT 5BinaryAggregationOfAbstractsF MM-Construct OIDModelMSM-ConstrName 123ER_Entity 224ER_Attribute 325ER_Relationship 411Rel_Table 512Rel_Column 633OO_Class MM-Model OIDName 1Relational 2Entity-Relationship 3Object MSM-Property OIDNameConstructType ………… MSM-Reference OIDNameConstruct… ………… MM-Property ………… MM-Reference ………… mSMmM SMM i-SMi-M

56. P. AtzeniJISBD - October 5, 200656 Multi-Level Repository, generation and use model schema model generic model specific description model independence Supermodel description (mSM) Model descriptions (mM) Supermodel schemas (SM) Model specific schemas (M) Supermodel instances (i-SM) Model specific instances (i-M) data Structure fixed, content provided by tool designers Structure generated by the tool from the content of mSM Structure fixed, content provided by model designers out of mSM Structure generated by the tool from the content of mM

57. P. AtzeniJISBD - October 5, 200657 The metamodel approach, translations The constructs in the various models are rather similar: –can be classified into a few categories (“metaconstructs'') –translations can be defined on metaconstructs, and there are “standard”, accepted ways to deal with translations of metaconstructs they can be performed within the supermodel –each translation from the supermodel SM to a target model M is also a translation from any other model to M: given n models, we need n translations, not n 2

58. P. AtzeniJISBD - October 5, 200658 Generic translation environment 1. Copy 2. Translation 3. Copy Supermodel Source model Target model Translation: composition 1,2 & 3

59. P. AtzeniJISBD - October 5, 200659 Translations within the supermodel We still have too many models: –Just within simple ER model versions, we have 4 or 5 constructs, and each has several independent features which give rise to variants for example, relationships can be –binary or N-ary –with all possible cardinalities or without many-to- many –with or without the possibility of specifying optionality –with or without attributes –…–… –Combining all these, we get hundreds of models! –The management of a specific translation for each model would be hopeless

60. P. AtzeniJISBD - October 5, 200660 Translations, the approach Elementary translation steps to be combined Each translation step handles a supermodel construct (or a feature thereof) "to be eliminated" or "transformed" A translation is the concatenation of elementary translation steps

61. P. AtzeniJISBD - October 5, 200661 A complex translation, example (0,N) Eliminate N-ary relationships Eliminate attributes from relationships Eliminate many-to-many relationships Replace relationships with references Eliminate generalizations

62. P. AtzeniJISBD - October 5, 200662 Complex translations Binary ER w/o gen N-ary ER w/ gen Binary ER w/ gen Relational Bin ER w/ gen w/o attr on rel OO w/ gen N-ary ER w/o gen Bin ER w/o gen w/o attr on rel Bin ER w/o gen w/o M:N rel Elim. N-ary relationships Elim. Relationship attr.s Elim. M:N relationships Replace relationships with references Elim OO generalizations Elim ER generalizations Bin ER w/ gen w/o M:N rel OO w/o gen …

63. P. AtzeniJISBD - October 5, 200663 Translations Basic translations are written in a variant of Datalog, with OID invention –We specify them at the schema level –The tool "translates them down" to the data level –Some completion or tuning may be needed

64. P. AtzeniJISBD - October 5, 200664 A basic translation From (a simple) binary ER model to the relational model –a table for each entity –a column (in the table for E) for each attribute of an entity E –for each M:N relationship a table for the relationship columns … –for each 1:N and 1:1 relationship: a column for each attribute of the identifier …

65. P. AtzeniJISBD - October 5, 200665 A basic translation application Departments NameAddress Employees EmpNo Name Affiliation Departments 1,1 0,N Name Address Employees EmpNoNameAffiliation

66. P. AtzeniJISBD - October 5, 200666 A basic translation (in supermodel terms) From (a simple) binary ER model to the relational model –an aggregation of lexicals for each abstract –a component of the aggregation for each attribute of abstract –for each M:N aggregation of abstracts … … From (a simple) binary ER model to the relational model –a table for each entity –a column (in the table for E) for each attribute of an entity E –for each M:N relationship a table for the relationship columns … –for each 1:N and 1:1 relationship: a column for each attribute of the identifier …

67. P. AtzeniJISBD - October 5, 200667 "An aggregation of lexicals for each abstract" SM_AggregationOfLexicals( OID: #aggregationOID_1(OID), Name: name)  SM_Abstract ( OID: OID, Name: name ) ;

68. P. AtzeniJISBD - October 5, 200668 Datalog with OID invention Datalog (informally): –a logic programming language with no function symbols and predicates that correspond to relations in a database –we use a non-positional notation Datalog with OID invention: –an extension of Datalog that uses Skolem functions to generate new identifiers when needed Skolem functions: –injective functions that generate "new" values (value that do not appear anywhere else); so different Skolem functions have disjoint ranges

69. P. AtzeniJISBD - October 5, 200669 "An aggregation of lexicals for each abstract" SM_AggregationOfLexicals( OID: #aggregationOID_1(OID), Name: n)  SM_Abstract ( OID: OID, Name: n ) ; the value for the attribute Name is copied (by using variable n) the value for OID is "invented": a new value for the function #aggregationOID_1(OID) for each different value of OID, so a different value for each value of SM_Abstract.OID

70. P. AtzeniJISBD - October 5, 200670 "An aggregation of lexicals for each abstract" SM-Abstract OIDSchemaName 3011Employees 3021Departments ……… SM-AttributeOfAbstract OIDSchemaNameisIdentisNullableTypeAbstrOID 4011EmpNoTFInt301 4021NameFFText301 ………………… Employees EmpNo Name 11 Departments1002 Employees1001 SM_AggregationOfLexicals( OID: #aggregationOID_1(OID), Name: n)  SM_Abstract ( OID: OID, Name: n ) ; … 3021002 …… 1001 SM-AggregationOfLexicals SchemaNameOID SM-aggregationOID_1_SK absOIDOID 301 Employees

71. P. AtzeniJISBD - October 5, 200671 "A component of the aggregation for each attribute of abstract" SM_ComponentOfAggregation… ( OID: #componentOID_1(attOID), Name: name, AggrOID: #aggregationOID_1(absOID), IsNullable: isNullable, IsKey: isIdent, Type : type ) ← SM_AttributeOfAbstract( OID: attOID, Name: name, AbstractOID: absOID, IsIdent: isIdent, IsNullable: isNullable, Type : type ) ; Skolem functions –are functions –are injective –have disjoint ranges the first function "generates" a new value the second "reuses" the value generated by the first rule

72. P. AtzeniJISBD - October 5, 200672 A component of the aggregation for each attribute of abstract" SM-Abstract OIDSchemaName 3011Employees 3021Departments ……… Employees EmpNo Name 11 Departments1002 Employees1001 … 3021002 …… 1001 SM-AggregationOfLexicals SchemaNameOID SM-aggregationOID_1_SK absOIDOID 301 SM_ComponentOfAggregation… ( OID: #componentOID_1(attOID), Name: name, AggrOID: #aggregationOID_1(absOID), IsNullable: isNullable, IsKey: isIdent, Type : type ) ← SM_AttributeOfAbstract( OID: attOID, Name: name, AbstractOID: absOID, IsIdent: isIdent, IsNullable: isNullable, Type : type ) ; SM-componentOID_1_SK absOIDOID Employees EmpNoName SM-AttributeOfAbstract OIDSchemaNameisIdentisNullableTypeAbstrOID 4011EmpNoTFInt301 4021NameFFText301 ………………… SM-ComponentOfAggregationOfLexicals SchemaTypeAggrOIDisNullableisIdentNameOID Text1001FFName111004 1001IntFTEmpNo11 1003 401 1004402 1003

73. P. AtzeniJISBD - October 5, 200673 Generating data-level translations Schema translation Data translation Same environment Same language High level translation specification Supermodel description (mSM) Supermodel schemas (SM) Supermodel instances (i-SM)

74. P. AtzeniJISBD - October 5, 200674 Translation rules, data level i-SM_ ComponentOfAggregation… ( OID: #i-componentOID_1 (i-attOID), i-AggrOID: #i-aggregationOID_1(i-absOID), ComponentOfAggregationOfLexicalsOID: #componentOID_1(attOID), Value: Value ) ← i-SM_AttributeOfAbstract( OID: i-attOID, i-AbstractOID: i-absOID, AttributeOfAbstractOID: attOID, Value: Value ), SM_AttributeOfAbstract( OID: attOID, AbstractOID: absOID, Name: attName, IsNullable: isNull, IsID: isIdent, Type: type ) SM_ComponentOfAggregation… ( OID: #componentOID_1(attOID), Name: name, AggrOID: #aggregationOID_1(absOID), IsNullable: isNullable, IsKey: isIdent, Type : type ) ← SM_AttributeOfAbstract( OID: attOID, Name: name, AbstractOID: absOID, IsIdent: isIdent, IsNullable: isNullable, Type: type ) ;

75. P. AtzeniJISBD - October 5, 200675 Correctness Usually modelled in terms of information capacity equivalence/dominance (Hull 1986, Miller 1993, 1994) Mainly negative results in practical settings that are non-trivial Probably hopeless to have correctness in general We follow an "axiomatic" approach: –We have to verify the correctness of the basic translations, and then infer that of complex ones

76. P. AtzeniJISBD - October 5, 200676 Experiments A significant set of models –ER (in many variants and extensions) –Relational –OR –XSD –UML

77. P. AtzeniJISBD - October 5, 200677 Summary ModelGen was studied a few years ago New interest on it within the "Model management" framework New approach –Translation of schema and data –Visible (and in part self generated) dictionary –Visible and modifiable rules –Skolem functions describe mappings

78. P. AtzeniJISBD - October 5, 200678 Conclusion The ten-year goal of the Asilomar report: –The information utility: make it easy for everyone to store, organize, access, and analyze the majority of human information online A lot of interesting work has been done but … …integration, translation, exchange are still difficult… … 2009 is approaching … we are late!

79. P. AtzeniJISBD - October 5, 200679 GrazzieGraciesGracias Grazie  Thank you

80. P. AtzeniJISBD - October 5, 200680

81. Leftovers

82. P. AtzeniJISBD - October 5, 200682 i-SM-Abstract OIDAbsOIDInstOID 30103011 30113011 30123021 ……… i-SM-AttributeOfAbstract OIDAttrOfAbsOIDInstOIDValuei- AbsOID 40104011754323010 40204021John Doe3010 40214021Bob White3011 40224041CS3012 …………… Instances in our dictionary SM-Abstract OIDSchemaName 3011Employees 3021Departments SM-AttributeOfAbstract OIDSchemaNameisIdentisNullableTypeAbstrOID 4011EmpNoTFInt301 4021NameFFText301 4041NameTFChar302 ………………… mSMmM SMM i-SMi-M 75432 John Doe Bob White … CS

83. P. AtzeniJISBD - October 5, 200683 Translation of instances 75432 John Doe Bob White … CS Employees EmpNoNameAffiliation 75432John DoeCS Bob White Departments Name CS Departments NameAddress Employees EmpNo Name Affiliation Departments 1,1 0,N Name Address Employees EmpNoNameAffiliation

84. P. AtzeniJISBD - October 5, 200684 i-SM-Abstract OIDAbsOIDInstOID 30103011 30113011 30123021 ……… i-SM-AttributeOfAbstract OIDAttrOfAbsOIDInstOIDValuei- AbsOID 40104011754323010 40204021John Doe3010 40224041CS3012 …………… Instances in our dictionary 75432 John Doe Bob White … CS Employees EmpNoNameAffiliation 75432John DoeCS Departments Name CS i-SM-AggregationOfLexicals 1002 1001 AggOID 115011 115010 InstOIDOID 5010CS1110054030 5010754321110034010 5010John Doe1110044020 i-SM-ComponentOfAggregationOfLexicals i- AggOIDValueInstOIDCompOfAggOIDOID

85. P. AtzeniJISBD - October 5, 200685 i-SM-Abstract OIDAbsOIDInstOID 30103011 30113011 30123021 ……… i-SM-AttributeOfAbstract OIDAttrOfAbsOIDInstOIDValuei- AbsOID 40104011754323010 40204021John Doe3010 40224041CS3012 …………… Instances in our dictionary i-SM-AggregationOfLexicals 1002 1001 AggrOID 115011 115010 InstOIDOID 5010754321110036010 5010John Doe1110046020 5010CS1110056030 i-SM-ComponentOfAggregationOfLexicals i- AggrOIDValueInstOIDCompOfAggOIDOID i-SM_ComponentOfAggregation… (OID: #i-componentOID_1 (i-attOID), i-AggrOID: #i-aggregationOID_1(i-absOID), ComponentOfAggregationOfLexicalsOID: #componentOID_1(attOID),Value: Val ) ← i-SM_AttributeOfAbstract( OID: i-attOID, i-AbstractOID: i-absOID, AttributeOfAbstractOID: attOID, Value: Val ), SM_AttributeOfAbstract( OID: attOID, AbstractOID: absOID)

86. P. AtzeniJISBD - October 5, 200686 The problem ModelGen (a model management operator, [Bernstein 2003]) –given two data models M1 and M2, and a schema S1 of M1 (the source schema and model), generate a schema S2 of M2 (the target schema and model), corresponding to S1 and, for each database D1 over S1, generate an equivalent database D2 over S2 Generality: what are the models we consider? Correctness: what do corresponding and equivalent mean?

87. P. AtzeniJISBD - October 5, 200687 Many artifacts and models for them Many notations, each with variants and conventions: –Object diagrams –Interface definitions –Database schemas –Web site layouts –Control flow diagrams –XML schemas –Form definitions –…

88. P. AtzeniJISBD - October 5, 200688 Many models just in the database world With different features and goals –semantic models and logical models: E-R, functional, (conceptual) object relational, object-relational, object, network –general purpose models (for all seasons) and problem oriented models (for specific contexts: DW, statistical, spatial, temporal) Variations of models –models with different levels of abstraction –versions within a family (e.g. many versions of the ER model) More models recently with the Web and XML

89. P. AtzeniJISBD - October 5, 200689 What do we need to exchange and translate In design: –Artifacts (schemas and other descriptions) In operations: –Data (in databases, files, documents, …)

90. P. AtzeniJISBD - October 5, 200690 Elementary steps There can be a set of predefined basic translations, for example –eliminate n-ary aggregations; replace them with binary ones (and abstracts) –eliminate binary aggregations; replace them with functions –eliminate functions to abstracts; replace them with aggregations –eliminate complex attributes; replace them with simple attributes and abstracts Assumed to be correct and so complex translations built over them are correct by definition (an “axiomatic” approach)

91. P. AtzeniJISBD - October 5, 200691 A complex translation From an N-ary ER model with generalizations to a simple Object model with only single valued references and no generalizations –Eliminate N-ary relationships (replaced by binary ones and new entities) –Eliminate attributes from relationships –Eliminate many-to-many relationships –Transform relationships to references –Eliminate generalizations

92. P. AtzeniJISBD - October 5, 200692 "An aggregation of lexicals for each M:N aggregation of abstracts" SM_AggregationOfLexicals( OID: #aggregationOID_2(aggOID), Name: n )  SM_BinaryAggregationOfAbstracts( OID: aggrOID, Name: n, isFun1: "False", isFun2: "False") ; another Skolem function #aggregationOID_2(): –generates a "table" for a "relationship" the rule applies only to M:N "relationships", due to the conditions on isFun1 and isFun2

93. P. AtzeniJISBD - October 5, 200693 Translation rules, data level i-SM_ ComponentOfAggregation… ( OID: #i-componentOID_1 (i-attOID), i-AggrOID: #i-aggregationOID_1(i-absOID), ComponentOfAggregationOfLexicalsOID: #componentOID_1(attOID), Value: Value ) ← … SM_ComponentOfAggregation… ( OID: #componentOID_1(attOID), AggrOID: #aggregationOID_1(absOID), Name: name, IsNullable: isNullable, IsKey: isIdent, Type : type ) ← …

94. P. AtzeniJISBD - October 5, 200694 Translation rules, data level ← i-SM_AttributeOfAbstract( OID: i-attOID, i-AbstractOID: i-absOID, AttributeOfAbstractOID: attOID, Value: Value ), SM_AttributeOfAbstract( OID: attOID, AbstractOID: absOID, Name: name, IsNullable: isNullable, IsID: isIdent, Type: type ) ← SM_AttributeOfAbstract( OID: attOID, AbstractOID: absOID, Name: name, IsIdent: isIdent, IsNullable: isNullable, Type: type ) ;

95. P. AtzeniJISBD - October 5, 200695 Management of translations Basic properties: –Correctness, minimality, … Construction of complex translations by picking basic translations in the library

96. P. AtzeniJISBD - October 5, 200696 ModelGen: the architecture

97. P. AtzeniJISBD - October 5, 200697 A simple example An object relational database, to be translated in a relational one Source: the OR-model Target: the relational model System managed ids

98. P. AtzeniJISBD - October 5, 200698 Example, 2 Does the OR model allow for keys? Assume EmpNo and Name are keys

99. P. AtzeniJISBD - October 5, 200699 Example, 3 Does the OR model allow for keys? Assume no keys are specified