Restricted Search Engine Laurent Balat Christophe Decis Thomas Forey Sebastien Leclercq ESSI2 Project Supervisor: Johny BOND June 2002
Introduction(1) What is a search engine? 3 types: –disciplinary –global –thematic Internet users spend more than 50% of their time to search!
Introduction (2) Lots of pages can’t be reached. WEB Indexable WEB Google
How does it work ? The search engine is composed of two parts First processing : the WEB site spider WEBSpiderProcessing indexing PDF unit DOC unit HTML processing unit DATABASE Constraint
How does it work ? User part architecture DATABASE Query engine Query Interface User
Constraints Domain Restriction. Search depth. Theme: words accepted or not. Document type. Time delay.
The Spider Part Check if link already visited Check type data in constraints Error download HTTP HEAD link link priority queue Stack data page Push pageDownload
Document Processing Analyse of type Send to the appropriate unit. Extract words and links Trying to resolve bad links
Indexation Binary Search Tree: - quick building - efficient use Check constraints: - start list and stop list.
Database MySQL database. General Structure: Keywords Web links Correspondence between keywords and links
User interface and query engine The web page is generated by a script (cgi). The query engine questions the database Formatting the results
Demonstration (1) Fill the Database
Demonstration (2) How to search pages?
Conclusion Results and perspective –Original search engine. –Easy to improve by adding units to process differents file format (ps, doc, xls,…). Team working and repartition. This Project shows us how to use the different tools seen this year
References