ReQuest (Validating Semantic Searches) Norman Piedade de Noronha 16 th July, 2004

2 Limitations in searching for information on the Web Lack of syntax: Information is unstructured. Lack of semantics: Machine processes do not understand the meaning of information. Unable to properly filter information for users, leading to information overload.

3 The Semantic Web (SW) SW is a proposal for a Web of machine interpretable data. Purpose: Automate user and computer tasks. Goal: Add structure and semantics to the existing Web with metadata and ontologies.

4 Metadata Describes data about a web resource and not the actual content. Ex: Author, Date. Resource Description Format (RDF) is the SW standard for metadata representation.

5 Ontologies Ontologies provide a shared and common understanding of concepts that can be communicated between people and heterogeneous distributed systems. Ontologies are used in the SW as dictionaries, thesauri and vocabularies. OWL (Web Ontology Language) is the W3C standard for ontologies representation.

6 Semantic Web Searches Ex: Contacting an author of a certain article in a particular newspaper. Article  Article’s Author  Author’s Name  Author’s

7 Objectives 1. Build a Semantic Web search environment prototype: ReQuest. 2. Evaluate & compare searches against conventional search engines.

8 Hypothesis “Searches based on ontologies improve user satisfaction and reduce effort by eliminating irrelevant results.”

9 Outline Motivation & The Semantic Web ReQuest – Architecture – Prototype ReQuest for News Validation Conclusions & Future Work

10 The Semantic Web Stack

11 ReQuest & the SW XML + NS + xmlschema RDF Ontology ReQuest

12 ReQuest: Use Cases Request Search Engine

ReQuest: Architecture INTERFACE ENGINE REPOSITORY METADATA SUBSYSTEM ONTOLOGY SUBSYSTEM

14 Implementation Ontology Subsystem – Java, RDF Schema Metadata Subsystem – Java, RDF & RSS Repository – SQL (PosGreSQL) Interface Engine – HTML & Java Servlets

15 Outline Motivation & The Semantic Web ReQuest Architecture Prototype ReQuest for News Validation Conclusions & Future Work

16 Configure new domain in ReQuest 1. Selecting Input Data. 2. Configuring ReQuest. 3. Defining Equivalences. 4. Launching New Domain.

17 News Domain: Person Ontology

18 News Domain: Ontologies Newspaper – Created RDF Schema File Document – Imported from Dublin Core

19 News Domain: Metadata Files Newspaper – Created RDF File Person – Created RDF File Document – Imported from RSS News Feeds

20 News Domain: Setup Configuring ReQuest – Adding Ontology – Metadata Associations. – Defining periodicity of retrieval of metadata. Defining Equivalences Launching a New Domain – Retrieve and Process Ontologies – Generate template for interface.

21 ReQuest User Interface Person Newspaper Document

22 ReQuest User Interface Person Newspaper Document

23 ReQuest User Interface

24 Validation Survey 5 “volunteers”. 9 queries on news domain. ReQuest vs. Google.

25 Examples of search queries (Q1) Find the post office address for the publisher Público. (Q9) How many distinct articles were published by Público about Futebol between the 5th of January, 2004 and the 7th of January, 2004.

26 Individual Query Survey How hard was the query to formulate? Did the semantic links help find the information? How long did it take to find the information? How relevant was the obtained information for your need? How many results were not interesting in the first page? Which search system was easier to use?

27 User Feedback Improve interface. Rank search results. Reduce information by providing a reduced version of results. Search within results. Search with properties from different contexts. Domain search preferred to Global search.

28 Hypothesis Validation “Searches based on ontologies improve user satisfaction and reduce effort by eliminating irrelevant results.” Measurements: 1. Information Need Satisfaction. 2. Effort Reduction. 3. Irrelevant Results Reduction.

29 Measure 1: Information Need Satisfaction Only one user achieved greater success with Google. Google’s results better in only one out of nine queries.

30 Measure 2: Effort Reduction Majority of users were successfully aided by ReQuest approximately 7 times, while only 20% managed to solve more than half of the tests with less effort with Google. Some users did not produce a single test query where Google required less effort than ReQuest.

31 Measure 3: Irrelevant Results Reduction Only first page results were compared. 80% of users found fewer irrelevant results with ReQuest than with Google. ReQuest was more precise than Google for 48.9% of all questions, while Google was more precise for 24.4%.

32 Outline Motivation & The Semantic Web ReQuest Architecture Prototype ReQuest for News Validation Conclusions & Future Work

33 Conclusions Positive Feedback: Searches based on ontologies improved user satisfaction Reduced effort by eliminating irrelevant results. Offering users the ability to select the search context is a more exact method for expressing the information need than key words.

34 Conclusions Results not statistically significant due to: Population User Profile Queries

35 Future Work More Extensive Validation. Domain searches enhanced with ability to restrict values. Support OWL. Multilingual Searches.

36 References Presented at Interacção 2004 as Norman Noronha, Mário J. Silva Using the Semantic Web for Web Searches July Digital Deposit. TUMBA – A Portuguese Search Engine.

37 THANKS Thank you very much for your attention.