1. Compass
Semantic search

2. Basics Knowledge model based information retrieval
Fulltext search enhanced with Topic Maps = Semantic search
Search driven navigation
TMRA '06

3. Search technologies Semantic search Level of precision
("Intelligence")
Conceptual search
Full-text search
For å sette dette i sammenheng med forskjellige søke metoder:
Fulltekst: stort domene (Internett), rask og enkel implementasjon
Konseptuelle: Analyse av kombinasjon av ord
Semantiske: analyse av søkestreng og data basert på en kunnskapsmodell
Problemer med "tradisjonelle søk": basert på statistikk, for mange treff, upresise svar,lite fleksibel mhht formulering/feilstavelser/etc
Ex: Skiferie i Nordnorge?
Data volume
(Domain size)
Compass
TMRA '06

4. Given... a web site with a lot of text,
which is unstructured (no markup, no tags),
a controlled domain (we know what the discourse domain is), and
non-adequate search engine...
TMRA '06

5. We would like to... get relevant hits within a meaningful context,
spare the work of structuring our data,
add semantics to the content by defining a knowledge model.
TMRA '06

6. Compass-bowl: Take a fulltext search engine. Take a Topic Maps engine.
Add a hint of semantics.
Define the correct processes for orchestrating the components.
Mix them thoroughly.
Serve to public!
TMRA '06

7. Full text search engine
Apache Lucene (open source)
Possible to index most file formats
html, asp, php, jsp, pdf, rtf, txt, doc, ppt, xls, pst…
The index is independent of the model
No need to re-index when changes are made to the model
Small index size
typically less than 10% of the size of the data
Fast index lookup
less than 20 ms for index size >20000
TMRA '06

8. The knowledge model
Based on the ISO International Standard for Topic Maps
Semantic model of the discourse domain
Concept words = topic names/synonyms
Semantic relationships through associations
Compass Weight defines “closeness” between topics
property on association types
TMRA '06

9. Example Ovitas hasProduct hasEmployee Compass Christopher CW=0.8 type
TMRA '06

10. Compass orchestrator Guides the processes of the search:
Search for term in the topic map
Expand the map for relevant/related topics
Send all these terms off to a fulltext search
Calculates relevance (based on the combination of CW and Lucene weights) and prepares the result list as an XML instance
Render XML as wished
TMRA '06

11. Hits in the fulltext gruouped by the related topics
Search term
Hits in the fulltext gruouped by the related topics
Topic Map expansion
Relevant documents ranked by the weighting result

13. Search term in the topic map, but not in the text
Relevant information about ”Chris Searle”

14. Synonym search

15. Creating/maintaining the model
An MS Excel plug-in serves as the topic map editor
Can be put under version control
Import the model into the topic map engine: one click only
For complex topic maps a custom user interface can be used to enter instance data
TMRA '06

16. Navigation Navigation through the associations between topics
Navigation by search
TMRA '06

17. User configurations What pages to index What topic map to use
The number of hops to perform
The threshold for relevance
TMRA '06

18. Content lifecycle management
Easy to integrate with content repositories
A content management or publishing system can send a request to the indexer to re-index a particular resource
Incremental indexing: add, update or delete documents
HTTP is used as the basic mechanism to address content
TMRA '06

19. Architecture
SOA (service oriented architecture), no dependency on platform or components
Web service interface (HTTPRest)
.NET platform
Integrated components:
TMCore Topic Maps engine by NetworkedPlanet
Apache Lucene: full text engine
TMRA '06

20. Publishing System Services
Architecture diagram TM
Nav
TM Core
Full Text
Excel Editor
Compass Service
TM editor person
Publishing System Services
User
TMRA '06

21. Compass - Summary Semantic search based on Topic Maps
Search in any document formats
Organize information in a topic-oriented manner
Link to relevant information without touching the data content
Conceptual navigation by Topic Maps
Tools for maintaining/evolving the classification
Fast and easy implementation
TMRA '06