1 Oct 30, 2006 LogicSQL-based Enterprise Archive and Search System How to organize the information and make it accessible and useful ? Li-Yan Yuan

2 Oct 30, 2006 Projects n How to develop an enterprise search engine based on a database management system challenges: implementation of the inverted index

3 Oct 30, 2006 Projects n How to implement the TOP K query l Ranking formula l Inverted indexes are created with respect to frequences

4 Oct 30, 2006 Internet search n Search for relevant web pages l Good answers: à Relevant à Popular n Public domain knowledge, n Search engines are critical to Internet use l internal workings are secret l Tremendous political, economical, and cultural power

5 Oct 30, 2006 Enterprise search n Search the enterprise information systems for right information n Enterprise information l Internal web pages l Internal documentation systems l File systems l Databases l servers n The internet and enterprise domains differ fundamentally l Contents l User behavior l Economic motivations

6 Oct 30, 2006 Top-K Query n Objective l How to determine the top K objects that are most likely (approximately) related to the given query n Applications l Information retrieval l Internet and enterprise searches l Multimedia similarity search l Scheduling large scale on-demand data broadcase l ……

7 Oct 30, 2006

8

9 Development of Enterprise Search Systems

10 Oct 30, 2006 LogicSQL Enterprise information Archive and Search system n LogicSQL An object-relational database management system à New concurrency control algorithm à Staged database architecture l Developed in the University of Alberta l Commercialized by Shanghai Shifang Software Co.

11 Oct 30, 2006 Enterprise Archive and Search System n To archive all the enterprise information contents l File systems l Web pages l s l Internal documents l Database records? n To provide a web styled search engine n To support user-specified ranking algorithms l focus on the platform of archive and search l Easy implementation and test of various ranking algorithms

12 Oct 30, 2006 n Extend the database functionalities l Security model à Users, roles + security handle à Security primary key l New database objects à Inverted indexes F CREATE INVERTED INDEX F DROP INVESTED INDEX F Automatic population, similar to that of index F ORDER BY clause à User specified aggregate functions F CREATE AGGREGATE FUNCTION l Top-K query evaluation n Specified crawlers Enterprise Archive and Search System

13 Oct 30, 2006 n User configuration l Set up crawlers l Create a list of inverted indexes l Create one aggregate function for object ranking n Extend the query languages l Implement the top K query algorithm n Web based query pages Enterprise Archive and Search System