2. ARC06 SharePoint Search Deployment Mike Fitzmaurice Senior Technical Product Manager Microsoft Corporation mikefitz@microsoft.com

3. Agenda SharePoint Version Overview Improvements to Search in MOSS 2007 Finding People and Expertise Indexing Unstructured Data Indexing Structured Data Customizing the Search UI Search Usage Reporting

4. SharePoint Version Overview SharePoint Products and Technologies Search features differ with specific version –Microsoft Office SharePoint Server 2007 Enterprise CAL Standard CAL –Microsoft Office SharePoint Server 2007 for Internet Sites –Microsoft Office SharePoint Server for Search Standard Edition –Microsoft Office SharePoint Server for Search Enterprise Edition –Windows SharePoint Services 3.0 A details of the features found in each version can be found on www.microsoft.com/office This session assumes all search functionality is available to you –(i.e., SharePoint Server 2007 w/Enterprise CAL)

5. Search in SharePoint Products and Technologies Windows SharePoint Services 3.0 –Subset of SharePoint Server search features –No index rebuild during upgrade to SharePoint Server –Indexing and query always on the same machine –Search scope limited to site and sub site content only –Only content stored in WSS 3.0 can be indexed Microsoft Office SharePoint Server 2007 –Substantial additional features and functionality (e.g. Indexing BDC) –Indexer and query server roles can be distributed –Custom Search scopes across multiple content sources –Managed via a Shared Service Provider –Local and remote content sources can be indexed

6. Indexer SharePoint Search Architecture Search topology model User Requests Load Balancer Web front ends Query servers Propagation of indexes Content Sources SQL Server Index Files Crawling Property Store

7. Search Internal Architecture

8. Relevancy Improvements Substantially improved relevancy was the number one goal for this release New algorithms for Web and Enterprise Content New relevancy elements including –Click Distance –Anchor Text –URL Depth –Metadata Extraction from documents Testing has shown significantly better relevancy –500% improvement on common queries –100% improvement on all queries

9. Security Improvements Query-time security trimming (Same as SPS 2003) –Custom security trimming API. Minimum crawler permission is “Full Read” –Still provides the same security trimming functionality –Automatically configured for new sites Search visibility options –Prevent sites or specific lists appearing in search results “Security only” crawl and single item removal

10. Shared Service Provider Unique to Microsoft Office SharePoint Server –i.e., not in Windows SharePoint Services by itself Groups a set of services –High-value –Resource intensive –Administration can be delegated Portals consume shared services associated with the SSP –Configurable at the site collection level to add scopes, keywords and best bets. –One index per SSP Multiple shared service providers in a farm

11. Scalability and Performance Improvements Tested to about 50 million documents Fast incremental crawls using the new change log Continuous Index Propagation –Items appear in results minutes after being indexed Crawler Impact Rules –Control the impact of crawling on remote sources

12. Improved Management Simplified and streamlined Admin UI Improved management and mapping of crawled properties One index per shared service  no need to worry about managing discrete indexes Multiple crawler start addresses for each content source Improved management of search scopes –Scopes decoupled from Content Sources –Scopes can span multiple content sources –Scope by Property, Site, Content Source, URL –Granular rules for scope creation

13. Improved Management Web Based Index Log –Browse and filter crawler log Complete admin API Microsoft Operations Manager Management pack

14. Indexing Management Streamlined experience and more control One index per shared service  no need to worry about managing discrete indexes Multiple start addresses per content source MOSS indexes can drive the WSS search experience –Allow upgrade from WSS to MOSS

15. Managing Search

16. Improved End User Experience Search Center –Clean UI, Search Site Template, Web Parts New Search Syntax –Implicit AND, Property Search, +/-, Advanced Search New Results Features –Context Summary, “Did you mean”, Duplicate Collapsing, Hit Highlighting, Best Bets, Auto Definitions Improved Alerting and Search RSS Feeds Customization –Web Parts, ASP.NET 2.0, Results with XSLT

17. Core Results Matching Keywords And Best Bets High Confidence Action Links Search Tabs Search Center Search Box Pagination Statistics

18. Improved End User Experience

19. Finding People and Expertise SharePoint Profile data –Built from AD, BDC, LDAP v3 and manually entered –Privacy controls My Colleagues –Stored in a list in My Site –Outlook 2007 suggested or manually entered –Privacy controls Results grouped by social relevancy –My colleagues, my colleagues colleagues, everyone else

20. People and Expertise Search

21. Indexing Unstructured Data Extensible Connectivity Protocol Handler –Connects to a content source and enumerates the documents –Ships with support for NTFS File System, Exchange Public Folders, SharePoint sites, HTTP, Lotus Notes, Active Directory –Partners providing support for Documentum, Hummingbird, FileNet, Interwoven and others iFilter –Reads a document and extracts the plain text content and properties for the indexer to index –Ships with support for over 200 file types –Partners/Vendors providing support for many more e.g. PDF

22. Protocol Handlers & IFilters Protocol Handlers Available with – Vista/Desktop Search – SharePoint Technologies Known Microsoft protocol handlers: – Vista – File, CSC, MAPI – SharePoint Server 2007 – HTTP, File Share, Notes, WSS – Others – One Note IFilters Available with – Indexing Service – Vista/Desktop Search – SharePoint technologies – SQL Server – Exchange Server Commonly Used IFilters: – Vista/WDS – Office, MIME, HTML, Text, RTF – Office 12 – Office, Office12, MSG (and uses OS filters) – Others – PDF, TIFF, Zip

23. Indexing Unstructured Data

24. Indexing Structured Data Business Data Catalog Index and Search Structured Data NO CODE! XML Application Definition –Sample SAP and Siebel definitions included OOB Access to data through ADO.NET or Web Service Render BDC Data in SharePoint Server –Business DataView Web Part –SharePoint Designer Surface Data in SharePoint Lists –Business Data Column Create Custom Actions Import data into User Profiles

25. Indexing Structured Data

26. Customizing the Search UI Customize the Query –Query Web Part Properties –Custom Search Scopes –Custom Search Center Tabs –Customize the Advanced Search Page –Object Model and Search Web Service Customize the Results –Core Results Web Part Properties –ASP.NET 2.0 Master Pages and Page Layouts –Apply custom XSL to results set –Custom page for BDC Search results –Object Model and Search Web Service

27. Customizing the Search UI

28. Search Usage Reporting MOSS 2007 now includes usage reporting out of the box. Best way to improve search is to understand current usage. Several built-in search activity reports –Search query reports: Number of queries and top queries. –Search result reports: Top Results, Queries With Zero Results, Results with low click-through and a few more. Available at both site collection level and SSP level. Export data for extended reporting in Excel. Respond to feedback with configuration changes or editorial results (Best Bets).

29. Usage Reporting

30. Search Performance Planning Things to keep in mind –Indexing is resource intensive – balance it… Think about freshness goals, change rate –Evaluate remote server responsiveness –Upgrade WSS 2003 servers with WSS 2007 to index them faster Scale strategies might include –Dedicated WFE for crawling –Separate indexer machine –Separate SQL DB for Search –Fast disk for SQL, fast CPU for Indexer, more memory

31. Additional Information SharePoint Team Blog –http://blogs.msdn.com/sharepointhttp://blogs.msdn.com/sharepoint SharePoint Server 2007 Developer Portal –http://msdn.microsoft.com/office/sharepointhttp://msdn.microsoft.com/office/sharepoint Product information –http://www.microsoft.com/sharepointhttp://www.microsoft.com/sharepoint