Solr Facets in Alfresco 4 Dan Tuffery

Who Am I? About Me Senior Software Engineer at Ixxus Working with Alfresco & Solr for 2.5 years Alfresco Certified Engineer and Solr Certified Winner of Alfresco’s 2012 Dashlet Challenge

Agenda Talk Outline Alfresco Solr facets demonstration Solr facets overview Walkthrough of implementation Plans for the module

Alfresco Solr Facets Demonstration

Solr Facets Overview 3 Different flavours of facets Field facets Date facets Number range facets

Indexing Facet Fields Indexed Not Tokenised Not Stored

Facet Field Parameters facet = enables faceting facet.field = the field that you want to facet on facet.mincount = constraint count >= facet.mincount facet.limit = limit the number of constraints that are displayed

Facet Field Parameters facet = enables faceting facet.field = the field that you want to facet on facet.mincount = constraint count >= facet.mincount facet.limit = limit the number of constraints that are displayed q=myfield:solr&facet=true&facet.field=myfield&facet.mincount= 1&facet.limit=20

Date Facet Parameters facet.date = name of date field facet.date.start = start of the date range facet.date.end = End of the date range facet.date.gap = period of time in between each date range

Date Facet Parameters facet.date = name of date field facet.date.start = start of the date range facet.date.end = End of the date range facet.date.gap = period of time in between each date range q=myfield:solr&facet=true&facet.date=created&facet.limit=20 &facet.date.start=2012-0101T00:00:00.0Z&facet.date.gap =1MONTH&facet.date.end=2012-10-22T00:00:00.0Z

Solr Facets In Alfresco 4 What Needed To Be Done?

Alfresco’s Facet Implementation Out Of The Box Facet field POJO Logic to add facet field params to Solr request URL Facet results are mapped to JSON object

My Implementation Five Main Tasks Add facets to Alfresco (Facet Manager) Solution to add facets to search request Solution to return facets in the search response Modify search results page to display facets Add support for filter queries

1)Add Facets to Alfresco Facet Manger Tool in Admin Console Based on current Tag Manager Create, Read, Update and Delete facets REST API in Alfresco (POST, GET, PUT, DELETE) Custom content model – ix:facetField

1)Add Facets to Alfresco What facets to display Content Type, Mime type, Property, Date How the facet should be displayed Display Name, Minimum Count, Limit When the facet should be displayed Individual Site, All Sites, Repository Search

Indexing Facet Fields in Alfresco Best Practices Indexed Not Tokenised Not Stored

Indexing Facet Fields in Alfresco Best Practices Indexed Not Tokenised Not Stored

Indexing Facet Fields in Alfresco Best Practices Indexed Not Tokenised Not Stored Indexed Field Name Indexed Value(s) @{http://www.ixxus.com/model/1.0}location {en}new {en}york @{http://www.ixxus.com/model/1.0}location.__ new york @{http://www.ixxus.com/model/1.0}location.u {en}New York @{http://www.ixxus.com/model/1.0}location.__.u New York @{http://www.ixxus.com/model/1.0}location.sort enNew York

Search Flow of Execution OOTB

Search Flow of Execution

2) Add Facets to Search Request Search request sent from Share search.lib.js var queryDef = { query: ftsQuery, language: "fts-alfresco", page: {maxItems: params.maxResults * 2}, templates: getQueryTemplate(), defaultField: "keywords", onerror: "no-results", sort: sortColumns, searchScope: searchScope, siteId: params.siteId, filterQueries: filterQueries }; nodes = search.query(queryDef);

2) Add Facets to Search Request Extended Search.java String searchScope = (String)def.get("searchScope"); String siteId = (String)def.get("siteId"); .. if(searchScope != null) { List<FieldFacet> fieldFacets = facetService.getFieldFacets(searchScope, siteId); searchParameters.setFieldFacets(fieldFacets); } results = this.services.getSearchService().query(searchParameters);

2) Add Facets to Search Request Extended SolrQueryHttpClient.java Generates Solr Request URL Add parameters (including facets) to URL List<FieldFacet> fieldFacets = searchParameters.getFieldFacets(); for(FieldFacet facet : fieldFacets) { url.append("&facet.field=" + facet.getField()); url.append("&facet.mincount=" + facet.getMinCount());, url.append("&facet.limit=" + facet.getLimit()); }

2) Add Facets to Search Request Extended SolrQueryHttpClient.java Generates Solr Request URL Add parameters (including facets) to URL List<FieldFacet> fieldFacets = searchParameters.getFieldFacets(); for(FieldFacet facet : fieldFacets) { url.append("&facet.field=" + facet.getField()); url.append("&facet.mincount=" + facet.getMinCount());, url.append("&facet.limit=" + facet.getLimit()); } facet.field=@{http://www.ixxus.com/model/1.0}location.__.u

3) Return Facets in Search Response Results mapped to SolrJSONResultSet.java Response returns set of ScriptNode objects Collection<Object> set = new LinkedHashSet<Object>(results.length(), 1.0f); .. //OOTB code to add scriptNodes to the set if(searchParameters.getFieldFacets().size() > 0) { List<FieldFacet> fieldFacets = sp.getFieldFacets(); for(FieldFacet facetField : fieldFacets) { List<Pair<String, Integer>> facets = ((SolrJSONResultSet)results).getFieldFacet(facetField.getField()); set.add(new ScriptFacet(facetField, facets, getScope(), this.services)); } return set;

3) Return Facets in Search Response ScriptFacet Implements Scopable Represents a facet field and its properties Constraint filter query is constructed here public ScriptFacet(IxxusFieldFacet facetField, List<Pair<String, Integer>> facetList, Scriptable scope, ServiceRegistry services) { this.services = services; this.facetField = facetField; this.scope = scope; this.constraints = getFacetResults(facetList); } public Boolean isFacet() { return isFacet;

3) Return Facets in Search Response Back up to search.lib.js to process results var results, i, j, facetResults; for (i = 0, j = nodes.length; i < j; i++) { if(nodes[i].isFacet) { var facet = nodes[i]; if(facet.facetHasAnyHits) { facetResults.push(facet); } } else {//it is a ScriptNode object, so process it in the usual way .. //OOTB code to add search results to results list return ({ items: results, facetItems: facetResults });

3) Return Facets in Search Response search.get.json.ftl "items": [ ... ], "facets": { "facetDisplayName":"Location", “constraints" : "constraint": "San Jose (7)", "fq":"@{http://www.alfresco.org/model/content/1.0}title.__.u:San Jose" }, "constraint":"Berlin (4)", "fq":"@{http://www.alfresco.org/model/content/1.0}title.__.u:Berlin" ] }

4) Modify Search Results Page Search Results page layout changed dynamically Nested divs to display facet column

4) Modify Search Results Page search.js (YUI) var facets = oFullResponse.facets, resultsContainer, facetsDiv, results, body; if(facets && facets.length > 0) { resultsContainer = new Element(document.createElement("div"), { id: “results-container" }).addClass("results-container"); facetsDiv = new Element(document.createElement("div"), { id: "facetsDiv" }).addClass("search-facets"); Ixxus.search.solr.generateFacets(me, facets, facetsDiv); body = new Element(Dom.get("body")); results = new Element(Dom.get("results")); results.addClass("results-right"); resultsContainer.appendChild(facetsDiv); resultsContainer.appendChild(results); body.appendChild(resultsContainer); }

5) Add Support For Filter Queries Each facet constraint is a link Original search is executed + filter query/queries Same flow of execution as facets Filter queries added to the Solr request URL

5) Add Support For Filter Queries SolrQueryHttpClient.java List<String> filterQueries = searchParameters.getFilterQueries(); for(String filterQuery : filterQueries) { filterQuery = filterQuery.replace("{", "\\{"); filterQuery = filterQuery.replace("http", "http\\"); filterQuery = filterQuery.replace("}", "\\}"); filterQuery = filterQuery.replace(" ", "*"); filterQuery = filterQuery.replace("TO", " TO "); url.append("&fq=").append(encoder.encode("{!lucene}" + filterQuery, "UTF8")); }

Module Plans Ixxus Solr Code has been contributed back to Alfresco Alfresco Enterprise 4.2 (Spring 2013) Maybe other community release before then

Module Plans TODO Number range facets Multi field faceting Hierarchical faceting

Additional Information Search Terms Dashlet Uses faceting in Alfresco 4 https://github.com/ixxus/ix-search-terms-dashlet Ixxus blog http://www.ixxus.com/blog/2012/10/solr-facets-alfresco-walkthrough/

Summary Easy and configurable way to add facet fields Solr facets Field Facets Date Facets Support for filter queries

Questions?