Import experiments in Protégé. October 10th © Raúl García-Castro Import experiments in Protégé Raúl García-Castro October 10th, 2005 Interoperability Working Days October 10th-11th, 2005

Import experiments in Protégé. October 10th © Raúl García-Castro Index Execution of the benchmark suite Comments on how Protégé imports Comments on the experimentation

Import experiments in Protégé. October 10th © Raúl García-Castro Execution of the benchmark suite The version of Protégé used was: Protégé 3.2 beta Build 228. The experiments were performed using the RDF backend of Protégé. No modifications were needed in the tool for performing the experimentation. Process followed for each benchmark: To define the expected Protégé ontology that should be obtained when importing the RDF(S) file. The expected ontology was not defined in Protégé, but in natural language. To open the Protégé application. To create a new project of type RDF Files, checking the option from Existing Sources. To select the RDF(S) file with the RDF(S) ontology and the namespace of the ontology. To compare the imported Protégé ontology with the expected Protégé ontology. This comparison was performed manually.

Import experiments in Protégé. October 10th © Raúl García-Castro Experimentation results IdDescriptionKnowledge AddedKnowledge LostResult Class benchmarks I01One class--OK I02Several classes--OK Metaclass benchmarks I03One class instance of another class that is instance of a third one -Classes being instances of other classes OK I04One class instance of several classes-IdemOK I05Several classes instance of the same class-IdemOK I06One class instance of another class and viceversa-IdemOK I07One class that is instance of itself-IdemOK Subclass benchmarks I08One class subclass of another class that is subclass of a third one --OK I09One class subclass of several classes--OK I10Several classes subclass of the same class--OK I11One class subclass of another class and viceversa-EverythingFAIL I12One class that is subclass of itself-EverythingFAIL

Import experiments in Protégé. October 10th © Raúl García-Castro Experimentation results 2 IdDescriptionAddedLost Class and property benchmarks I13One class that has a property with another class-Domain, rangeOK I14One class that has the same property with several classes-IdemOK I15Several classes that have the same property with the same class-IdemOK I16One class that has a property with itself-IdemOK I17One class that has a property with a literal-IdemOK I18One class that has the same property with several literals-IdemOK Property benchmarks I19One property--OK I20Several properties--OK Subproperty benchmarks I21One property subproperty of another property that is subproperty of a third one --OK I22One property that is subproperty of several properties--OK I23Several properties that are subproperty of the same property--OK I24One property that is subproperty of another property and viceversa-EverythingFAIL I25One property that is subproperty of himself-EverythingFAIL

Import experiments in Protégé. October 10th © Raúl García-Castro Experimentation results 3 IdDescriptionAddedLost Property with domain and range benchmarks I26One property that has as domain a class--OK I27One property that has as domain several classesIdem. Two anonymous concepts as subclass of the domain concepts. DomainFAIL I28Several properties that have as domain the same class --OK I29One property that has as domain rdfs:Class-DomainOK I30Several properties that have as domain rdfs:Class-IdemOK I31One property that has as range a class--OK I32One property that has as range several classes-RangeOK I33Several properties that have as range the same class --OK I34One property that has as range rdfs:Class--OK I35Several properties that have as range rdfs:Class--OK I36One property that has as range rdfs:Literal--OK I37Several properties that have as range rdfs:Literal--OK

Import experiments in Protégé. October 10th © Raúl García-Castro Experimentation results 4 IdDescriptionAddedLost Property with domain and range benchmarks (2) I38One property that has as domain a class and as range another class --OK I39One property that has as domain a class and as range several classes -RangeOK I40Several properties that have as domain several classes and as range a class -DomainFAIL I41Several properties that have as domain several classes and as range several classes -DomainFAIL I42Several properties that have as domain and range the same class --OK I43One property that has as domain a class and as range rdfs:Literal --OK I44One property that has as domain several classes and as range rdfs:Literal -DomainFAIL I45One property that has as domain and range rdfs:Class -DomainOK I46One property that has as domain rdfs:Class and as range rdfs:Literal -DomainOK

Import experiments in Protégé. October 10th © Raúl García-Castro Experimentation results 5 IdDescriptionAddedLost Instance benchmarks I47One instance of a class--OK I48One instance of several classes-One of the type properties OK I49Several instances of a class--OK Instance and property benchmarks I50One instance that has a property with another instance of the same class-PropertyOK I51One instance that has the same property with several instances of the same class-PropertyOK I52Several instances that have the same property with the same instance of the same class -PropertyOK I53One instance that has a property with himself-PropertyOK I54One instance that has a property with another instance of another class-PropertyOK I55One instance that has the same property with several instances of another class-PropertyOK I56Several instances that have the same property with the same instance of another class -PropertyOK I57One instance that has a property with a literal-PropertyOK I58One instance that has the same property with several literals-PropertyOK

Import experiments in Protégé. October 10th © Raúl García-Castro Experimentation results 6 IdDescriptionAddedLost Syntax and abbreviation benchmarks I59Several xlasses with absolute URI references--OK I61Several classes with URI references relative to a base URI --OK I61Several classes with URI references transformed from rdf:ID attribute values --OK I62Several classes with URI references relative to an ENTITY declaration --OK I63Several classes with empty nodes-EverythingFAIL I64Several classes with empty nodes shortened--OK I65Several classes with multiple properties--OK I66Several classes with multiple properties shortened--OK I67Several classes with typed nodes--OK I68Several classes with typed nodes shortened--OK I69Several classes with properties with string literals--OK I70Several classes with properties with string literals as XML attributes --OK I71Several classes with blank nodes with identifierAn anonym instance of :THINGPropertyFAIL I72Several classes with blank nodes shortened--OK

Import experiments in Protégé. October 10th © Raúl García-Castro Index Execution of the benchmark suite Comments on how Protégé imports Comments on the experimentation

Import experiments in Protégé. October 10th © Raúl García-Castro Benchmarks that Protégé doesn’t pass Cycles in class taxonomies Benchmarks: I11-I When Protégé imports an ontology containing a cycle in its class taxonomy it crashes, not importing anything. Example:

Import experiments in Protégé. October 10th © Raúl García-Castro Benchmarks that Protégé doesn’t pass Cycles in property taxonomies Benchmarks: I24-I When Protégé imports an ontology containing a cycle in its property taxonomy it crashes, not importing anything. Example:

Import experiments in Protégé. October 10th © Raúl García-Castro Benchmarks that Protégé doesn’t pass Properties with multiple domains Benchmarks: I27, I40, I41, I When Protégé imports an ontology containing a property with multiple domains it creates a slot with multiple domains, but in Protégé multiple domains in slots are considered as the union of all the domains and in RDF(S) multiple domains in properties are considered the intersection of all the domains. Example:

Import experiments in Protégé. October 10th © Raúl García-Castro Benchmarks that Protégé doesn’t pass Classes with empty nodes unshortened Benchmarks: I When Protégé imports an ontology containing a class definition with empty nodes unshortened, it crashes, not importing anything. Example:

Import experiments in Protégé. October 10th © Raúl García-Castro Benchmarks that Protégé doesn’t pass Blank nodes with identifier Benchmarks: I71 When Protégé imports an ontology containing blank nodes with identifiers, each time the blank node appears it is imported in Protégé as a new node. In this case, Protégé creates an instance for the blank node in class2 and, for the property of instance1, it creates another instance for the blank node as an instance of :THING. Example:... Literal 1...

Import experiments in Protégé. October 10th © Raúl García-Castro Comments on Protégé’s RDF(S) import Resources with multiple type definitions Benchmarks: I03-I09, I Protégé doesn't handle the import of resources with more than one type. In the case of metaclasses, when Protégé finds a class that is also an instance of another class it ignores this second type definition, losing the knowledge about the metaclass definition. In the case of instances of multiple classes, when Protégé finds an instance of two classes it ignores the second type definition, losing the knowledge about the multiple type definition. Example:

Import experiments in Protégé. October 10th © Raúl García-Castro Comments on Protégé’s RDF(S) import Classes with properties Benchmarks: I13-I18, I65, I66, I69-I Protégé does not allow to change its system classes. Therefore, when importing a class with a property, as it cannot create an own slot in the class' metaclass, it imports the property but doesn't define its domain and its range. Example:

Import experiments in Protégé. October 10th © Raúl García-Castro Comments on Protégé’s RDF(S) import Import of rdfs:Class Benchmarks: I34-35, I Protégé imports rdfs:Class as the system class :STANDARD_CLASS. Example:

Import experiments in Protégé. October 10th © Raúl García-Castro Comments on Protégé’s RDF(S) import rdfs:Class as the domain of a property Benchmarks: I29, I30, I45-I Protégé does not allow to change its system classes. Therefore, when importing a property with domain rdfs:Class, it cannot create a template slot in :STANDARD_CLASS (Protégé equivalent for rdfs:Class) for the property, it imports the property but doesn't define its domain. Example:

Import experiments in Protégé. October 10th © Raúl García-Castro Comments on Protégé’s RDF(S) import Properties with multiple ranges Benchmarks: I32, I39, I Protégé only allows one range when importing properties. Therefore, when importing a property with multiple ranges, it creates a slot with the range Any, losing the definition of the ranges. Although Protégé allows multiple ranges in a slot, they are considered as the union of all these ranges, in RDF(S) multiple ranges in a property are considered as the intersection of all these ranges. Example:

Import experiments in Protégé. October 10th © Raúl García-Castro Comments on Protégé’s RDF(S) import Properties with a range of rdfs:Literal Benchmarks: I43, I44, I46, I65, I66, I69-I Protégé imports rdfs:Literal in the range of a property to its own datatype String. Example:

Import experiments in Protégé. October 10th © Raúl García-Castro Comments on Protégé’s RDF(S) import Instances with undefined properties Benchmarks: I50-I When Protégé finds a property in an instance (an instance-property-object triple), and this property is not defined, it does not consider the property to belong to the domain class. Therefore, it creates the slot without domain and with a range of Any, and the property in the instance is lost. The range of the slot is Any even when the property relates an instance with a literal. If the property is defined in the ontology (benchmarks I50_bis-I58_bis), Protégé imports correctly properties in instances. Example:

Import experiments in Protégé. October 10th © Raúl García-Castro Index Execution of the benchmark suite Comments on how Protégé’s imports Comments on the experimentation

Import experiments in Protégé. October 10th © Raúl García-Castro Comments on improvements on the tool Protégé should not crash when importing RDF(S) files containing class or property taxonomies with cycles (benchmarks I11-I12, I24-I25), as it does not import anything from a file that contains them. Protégé should not crash when importing RDF(S) files containing classes with empty nodes unshortened (benchmark I63), as it does not import anything from a file that contains them. Protégé should import blank nodes with identifier, as it does not import anything from a file that contains them. Protégé should import properties with multiple domains creating slots with a domain of a common superclass of them, such as :STANDARD_CLASS (benchmarks I27, I40, I41, I44), as the semantics of multiple domains in Protégé and RDF(S) are different. When importing undefined properties in instances (benchmarks I50-I58), in some cases Protégé could work out the domain and the range of the property according to the instance-property-object triples. This way, the instance-property-object triples wouldn't be lost in the import.

Import experiments in Protégé. October 10th © Raúl García-Castro Comments on the benchmark suites To check the correct import of the RDF/XML syntax benchmarks, some RDF(S) files had to be modified: As Protégé does not import classes with properties, we had to change the RDF(S) files of some benchmarks in order to check the import of some ontologies that deal with the RDF/XML syntax (Benchmarks I65, I66, I69-I72).

Import experiments in Protégé. October 10th © Raúl García-Castro Import experiments in Protégé Raúl García-Castro October 10th, 2005 Interoperability Working Days October 10th-11th, 2005