1. Enterprise Search – Where do we go from here? Aya Soffer, PhD DGM, Information and Interaction Technologies IBM Haifa Research Lab

2. IBM Research Lab in Haifa © Aya Soffer 25/2/082 The World of Search Content management, groupware, “Intranet search” E-commerce, News, public documentation, government,… Proprietary (owned & stored by the SE) “local” Market intelligence, news tracking, job data mining Private “inward facing” Web search Public “outward facing” Public (owned and stored by others) “global” Access Content

3. IBM Research Lab in Haifa © Aya Soffer 25/2/083 Enterprise Search

4. IBM Research Lab in Haifa © Aya Soffer 25/2/084 User Expectations Shape Products Global / Web search (public content) Users expect relevant answers to very low- content queries: elections, olympics Search engines deliver! Hence reinforcement of user expectations. The global context is extremely popular Shapes user expectations at the workplace Shapes enterprise search products Enterprise search (proprietary content) Users expect a similar interaction style Yet, users are more sophisticated

5. IBM Research Lab in Haifa © Aya Soffer 25/2/085 Enterprise Search — Harder than Web Search Far fewer resources — but high expectations Data is not “search friendly” Must index everything - find everything I can access Security - but show me only what I am allowed to see Link-based methods not as effective – not enough linkage Search is not cheap! - about 5-10 cents/ document/ year – yet ROI is hard to calculate

6. IBM Research Lab in Haifa © Aya Soffer 25/2/086 Smaller scale Corpus is much smaller Query load is much lower Less anarchic Central authority Data formats can be controlled More potential structure Data is better organized and richer Can tap into organizational knowledge No spam At least not intentional Enterprise Search — Easier than Web Search

7. IBM Research Lab in Haifa © Aya Soffer 25/2/087 Major Players

8. IBM Research Lab in Haifa © Aya Soffer 25/2/088 Trends

9. IBM Research Lab in Haifa © Aya Soffer 25/2/089 Enterprise Search Trends Information finding goes beyond simple keyword search in search bar Text analytics and semantic search allows searching by concepts instead of by keywords Understand content, understand user intent Collaboration technology is fundamentally changing how people organize and access content Democratic tagging of data as a bridge across multiple taxonomies Discover relationships, find experts, utilize wisdom of the crowds and social networks to find best matches Information finding isn’t enough – need to provide means to explore search results New visualizations for search results Combined search / browse paradigm will be pervasive Combination of search and BI  BI for the Masses Search goes mobile Access information in context from mobile devices

10. IBM Research Lab in Haifa © Aya Soffer 25/2/0810 Social Search

11. IBM Research Lab in Haifa © Aya Soffer 25/2/0811 Social Search With the advent and popularity of Web 2.0, the ability to share and find information is being expanded beyond keyword search mainly by use of tags While Google's page rank can be viewed as one of the first applications of the Web 2.0 concept of wisdom of crowds, Search has yet to fully harness the power of Web 2.0 Community influenced search is beginning to appear in niche search engines and as Beta’s in major search engines Wikia Search Yahoo! MyWeb Google Co-Op Eurekster

12. IBM Research Lab in Haifa © Aya Soffer 25/2/0812 1 billion global users80,000,000 web sites Content ConsumersContent Providers User-Generated Content Published Content CollaboratorsFacilitators Collective Intelligence Web 2.0: The wildly read-write Web 2.0

13. IBM Research Lab in Haifa © Aya Soffer 25/2/0813 Search Today – Web 2.0 phenomena indirectly influencing search results Indirect Influence Collective Intelligence User-Generated Content Published Content User-Generated Metadata Social Networks

14. IBM Research Lab in Haifa © Aya Soffer 25/2/0814 Evolution of Search Technologies Scope Technology Business Impact User World Wide Web (Html) Hyperlink analysis for relevance ranking (good for Web, not as good in enterprise) Categorization/ summarization eCommerce and consumer market Everyone Web-based Search (2 nd Generation) Small, closed collections String matching Boolean search Basic relevance ranking Vertical business domains (medical, legal) Trained specialists Information Retrieval (1 st Generation) Structured, semi- structured and unstructured information Text analytics with novel linguistic and semantic processing Ranking on structure Faceted navigation Pervasive use in business processes to realize value from unstructured information Everyone & applications Information Discovery (3 rd Generation ) Structured, semi- structured, unstructured information + networks Community input Incorporation of wisdom of crowds Tags, tag clouds, social network data Pervasive use in business processes to realize value from unstructured information Everyone & applications Social Information Discovery (4 th Generation)

15. IBM Research Lab in Haifa © Aya Soffer 25/2/0815 Trend - Mutually Reinforcing Relationship Data Metadata Author Mentioned Bookmark Reader Associated TaggedBy People tags folder label subject of email title of document anchor text query author document owner email address username code user profile email messages received documents authored documents calendar entries chat history

16. IBM Research Lab in Haifa © Aya Soffer 25/2/0816 Web 2.0 Unified Search The result set includes relevant documents, people, and tags. Ranking is affected by the volume of tags and comments that are associated with each document. Tags related to the result set – presented as a tag cloud People related to the result set – sorted by relevance

17. IBM Research Lab in Haifa © Aya Soffer 25/2/0817 Guided Navigation Meets Business Intelligence (BI for the Masses)

18. IBM Research Lab in Haifa © Aya Soffer 25/2/0818 Multifaceted search or guided navigation Allows the simultaneous exploration of many aspects of a topic, and the gradual “zooming in” on the information target Reduces frustration: ensures that only valid choices are presented, so zooming in never yields an empty result set Solution to the problem of “few terms  too many results, more terms  no results” Supports browsing when the user doesn’t really know what to ask for in a multi-dimensional information space Multifaceted search is very popular in e-Commerce solutions (Amazon, eBay, buy.com, …), but is also relevant to more traditional text search applications New applications are emerging every day

19. IBM Research Lab in Haifa © Aya Soffer 25/2/0819 Multifaceted Search – Example Categorie s- { Featured Dimensions Categories Category Counts Query Current Context Other Dimensions Search Results

20. IBM Research Lab in Haifa © Aya Soffer 25/2/0820 Guided Navigation Meets Business Intelligence Faceted Navigation Limitations Current faceted navigation interfaces enable drilling down one facet at a time. Counts and context are similarly presented for each facet separately Business Intelligence today is mainly for structured information Assumes structure is known in advance Offline processing Complex report generations New direction: faceted navigation as a front end for business intelligence applications. Facets can be defined on a combination of fields including aggregations and simple expressions.

21. IBM Research Lab in Haifa © Aya Soffer 25/2/0821 MultiFaceted Search with BI - Example Query For each author: counts (as before) but also calculated values on result set Average Rating per Sales Rank

22. IBM Research Lab in Haifa © Aya Soffer 25/2/0822 Correlating Facets – BI on search results Top-Ranked books do not have the highest ranking Older books have lower sales rank Clicking on the value will bring you to the books with high rating and top sales rank

23. IBM Research Lab in Haifa © Aya Soffer 25/2/0823 The World of Search Content management, groupware, “Intranet search” E-commerce, News, public documentation, government,… Proprietary (owned & stored by the SE) “local” Market intelligence, news tracking, job data mining Private “inward facing” Web search Public “outward facing” Public (owned and stored by others) “global” Access Content

24. IBM Research Lab in Haifa © Aya Soffer 25/2/0824 The World of Search The World of Search The boundaries are blurring Content management, groupware, “Intranet search” E-commerce, News, public documentation, government,… Proprietary (owned & stored by the SE) “local” Market intelligence, news tracking, job data mining Private “inward facing” Web search Public “outward facing” Public (owned and stored by others) “global” Access Content