Taxonomy & Ontology Impact on Search Infrastructure John R. McGrath Sr. Director, Fast Search & Transfer

2 Market Potential Search Innovation ConventionalInnovative Mature Market Growth Market General Site Search, Intranet eTailing Business Intelligence Enterprise eCommerce Premium Content Delivery Information Management Knowledge Management Compliance and Content Governance Detection, Surveillance and Enforcement Predictive Search alerting through business rules tracking and monitoring high order analytics pattern matching Next Generation Search 360º View of Enterprise Hyper-personalization Contextual synergy Technology transparency Central governance Conventional Search search bar w/ results fixed relevancy model static navigation few data sources Advanced Search tunable relevancy and navigation wide array of sources (structured, unstructured) static navigation Extended Search Extended platform (desktop, mobile) Intelligent use of context (Web, geospatial) rich media integration The Real Search Solution Space Evolving more powerful capability

3 Improving Search through Contextual Analysis The Importance of Context in Search Usage Patterns Queries Interest Profiles Location What does the data indicate is most important? What matters most to the user? What matters most to the business or organization providing the data? Statistics Metadata Organization Business Rules Editorial Control Program Control

4 Improving Search through Conceptual Analysis Embedded Application Semantics FAST Answers Extreme precision applications: – Self-service, NLP, Mobile, Compliance – Information discovery, Intelligence – Web 2.0 – The semantic web Rich Media The Adaptive Information Warehouse – Scene level discovery – Podcasts: Extreme precision access – Linguistic cleansing – On-the-fly fact mining – X query speed-ups – Low latency access to extreme volumes When was D-day? – Visualize implicit facts – Visualize uncertainty – Use embedded semantics: – Ex: Patent claims – Ex: Blogs

5 Applying Ontology to the Search Architecture COLLECTIONSCONNECTORSREFINEMENTSEARCH & ALERT ENGINESPROCESSINGSEARCH PROFILEUSER SEARCH & ADMINISTRATION MANAGEMENT ALERT CONTENT REFINEMENT QUERY PROCESSING RESULT PROCESSING SEARCH STRUCTURED DATA RICH MEDIA UNSTRUCTURED DATA SEARCH App. Logic Tagging Extraction Classification Relationship Personalization Term Expansion Query Control Source Selection Refinement Logic Presentation Security App logic Navigation

6 Primary applications today in search Classify documents – facilitate simple, keyword-based retrieval Provide a common language, or thesaurus – offer terms to refine a search from a consistent, controlled vocabulary Create browse-able directories – facilitate rapid navigation through defined hierarchies of information Promote meaningful clustering – establish fixed points for clustering results Generate pick-list elements – select or combine terms to limit/define your search domain Expedite query refinement – refine/exclude on similarly tagged items

7 FAST Relevancy Framework Business Rules User Profiles Core Algorithmic Model Application Model Sorting Navigation Feedback Accessible to…Control Mechanisms End Users Business Managers Alert Parameters Page boosting Administrator Rank Profile Concept Security DeveloperDynamic Algorithm weights Levels of control Multiple levels of control

8 Human Factors Considerations Limited user capacitance Most business users do not navigate deeper than 4 levels in a taxonomy More than 10 choices/nodes per level impacts willingness to move deeper, to next level Multiple perceptions of value Provides navigation for discovery (40%) Organizes disparate info (18%) Structures KM repository (16%) Automates classification & alerting (14%) Enhances searching(12%) Distribution of expected benefits Increased productivity (21%) Reduced search time ( 20% ) Increased knowledge sharing (18%) Shortened time to decision (16%) Improved collaboration (13%) Discover new opportunities (10%) Source : Delphi Group 2004 Survey – 300 respondents

9 Observed Trends Taxonomies and ontologies are expense to develop and maintain. Published works and services Social network product Automatic generation

10 Observed Trends Primary application is smart navigation. Simple knowledge bases Supervised clustering

11 Near Future? Application of Knowledge bases is increasing to support advanced features. Extraction and association Relationship analysis Advanced personalization