Presentation is loading. Please wait.

Presentation is loading. Please wait.

Getting started with ElasticSearch on Windows and .NET with NEST

Similar presentations


Presentation on theme: "Getting started with ElasticSearch on Windows and .NET with NEST"— Presentation transcript:

1 Getting started with ElasticSearch on Windows and .NET with NEST
A short introduction Present the topic Deployment Oslo/NNUG Meetup Tomas Jansson 29/01/2014

2 This is me Tomas Jansson Manager & Group Lead .NET BEKK Oslo
github.com/mastoj blog.tomasjansson.com Trenger en ny profilbilde

3 https://github.com/mastoj/NestDemo
TL;DR; https://github.com/mastoj/NestDemo

4

5

6 Audience N00b Expert N00b Expert

7 This is the data and we need this new application
BackgrounD This is the data and we need this new application

8 The masterplan

9 What I want to show you is...
Elasticsearch is awesome Indexing using NEST Querying using NEST ... not about advanced elasticsearch hosting

10

11 Powershell to the rescue
Installation Great news, install as a service added in Powershell to the rescue

12 Abstraction over Elasticsearch
NEST Abstraction over Elasticsearch One of the 108 projects that Karel mentioned There is an low level abstraction as well called RawElasticClient

13 Fluent & Strongly typed Abstraction over Elasticsearch
NEST Fluent & Strongly typed Abstraction over Elasticsearch

14 Strongly typed Elasticsearch Client
NEST What? How? Strongly typed Elasticsearch Client

15 Functional C# C:\Dev\git> scriptcs
scriptcs (ctrl-c or blank to exit) > using System.Linq.Expressions; > Func<int, int, int> add = (x, y) => x + y; > add(1, 3) 4 > Expression<Func<int, int, int>> addExpr = (x, y) => x + y; > addExpr(1, 3) (1,1): error CS1955: Non-invocable member 'addExpr' cannot be used like a method. > var body = addExpr.Body as BinaryExpression; > Console.WriteLine(body); (x + y)

16 FUNC DEMO C:\Dev\git> scriptcs scriptcs (ctrl-c or blank to exit) > Func<int, int, int> add = (x, y) => x + y; > add(1, 3) 4 Func  executable C:\Dev\git> scriptcs scriptcs (ctrl-c or blank to exit) > using System.Linq.Expressions; > Func<int, int, int> add = (x, y) => x + y; > add(1, 3) 4 > Expression<Func<int, int, int>> addExpr = (x, y) => x + y; > addExpr(1, 3) (1,1): error CS1955: Non-invocable member 'addExpr' cannot be used like a method. > var body = addExpr.Body as BinaryExpression; > Console.WriteLine(body); (x + y)

17 Simple Expression DEMO
> using System.Linq.Expressions; > Expression<Func<int, int, int>> addExpr = (x, y) => x + y; > addExpr(1, 3) (1,1): error CS1955: Non-invocable member 'addExpr' cannot be used like a method. > var binExpr = addExpr.Body as BinaryExpression; > Console.WriteLine(binExpr); (x + y) > var add2 = addExpr.Compile(); > add2(3, 1); 4 Expression  ”function description” C:\Dev\git> scriptcs scriptcs (ctrl-c or blank to exit) > using System.Linq.Expressions; > Func<int, int, int> add = (x, y) => x + y; > add(1, 3) 4 > Expression<Func<int, int, int>> addExpr = (x, y) => x + y; > addExpr(1, 3) (1,1): error CS1955: Non-invocable member 'addExpr' cannot be used like a method. > var body = addExpr.Body as BinaryExpression; > Console.WriteLine(body); (x + y)

18 More complex Expression DEMO
> public class SomeClass { public string MyString { get; set; } } > Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString; > var compExpr = propExpr.Compile(); > var obj = new SomeClass { MyString = "Hello world" }; > compExpr(obj) Hello worldHello world > var body = propExpr.Body as BinaryExpression; > Console.WriteLine(body); (y.MyString + y.MyString) > var left = body.Left as MemberExpression; > Console.WriteLine(left.Member.Name); MyString C:\Dev\git> scriptcs scriptcs (ctrl-c or blank to exit) > using System.Linq.Expressions; > Func<int, int, int> add = (x, y) => x + y; > add(1, 3) 4 > Expression<Func<int, int, int>> addExpr = (x, y) => x + y; > addExpr(1, 3) (1,1): error CS1955: Non-invocable member 'addExpr' cannot be used like a method. > var body = addExpr.Body as BinaryExpression; > Console.WriteLine(body); (x + y)

19 More complex Expression DEMO
> public class SomeClass { public string MyString { get; set; } } > Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString; > var compExpr = propExpr.Compile(); > var obj = new SomeClass { MyString = "Hello world" }; > compExpr(obj) Hello worldHello world > var body = propExpr.Body as BinaryExpression; > Console.WriteLine(body); (y.MyString + y.MyString) > var left = body.Left as MemberExpression; > Console.WriteLine(left.Member.Name); MyString Enables us to translate from one domain to another in an ”easy” manner C:\Dev\git> scriptcs scriptcs (ctrl-c or blank to exit) > using System.Linq.Expressions; > Func<int, int, int> add = (x, y) => x + y; > add(1, 3) 4 > Expression<Func<int, int, int>> addExpr = (x, y) => x + y; > addExpr(1, 3) (1,1): error CS1955: Non-invocable member 'addExpr' cannot be used like a method. > var body = addExpr.Body as BinaryExpression; > Console.WriteLine(body); (x + y)

20 Show me the code!

21 Elasticsearch connection
public class ElasticClientWrapper : ElasticClient { private static string _connectionString = Settings.ElasticSearchServer; private static ConnectionSettings _settings = new ConnectionSettings(new Uri(_connectionString)) //http://demoserver:9200 .SetDefaultIndex(Settings.Alias) //"customer_product_mapping" .UsePrettyResponses(); public ElasticClientWrapper() : base(_settings) } //usage var client = new ElasticClientWrapper();

22 Mapping public class Product { public double UnitPrice { get; set; } public int TotalQuantity { get; set; } [ElasticProperty(Index = FieldIndexOption.not_analyzed)] public string ProductName { get; set; } public string CategoryName { get; set; } } public class Customer public string CustomerID { get; set; } public string CompanyName { get; set; } public string Address { get; set; } public string City { get; set; } public string Country { get; set; } [ElasticProperty(Type = FieldType.nested)] public Product[] Products { get; set; }

23 Mapping & Indexing _client = new ElasticClientWrapper(); _client.CreateIndex("indexName", s => s.AddMapping<Customer>(m => m.MapFromAttributes())); var customers = _customerRepo.GetCustomers(); _client.IndexMany(customers, "indexName"); Mapping created from attributes Indexing will use the mapping for the specified index There is async versions of the methods

24 Alias _client = new ElasticClientWrapper(); _client.Alias("indexName", "aliasName"); Alias Index_01

25 Swapping _client = new ElasticClientWrapper(); _client.Swap("aliasName", new [] { "Index_01" }, new [] { "Index_02" } ); Alias Alias Create new index Swap Delete old index Index_01 Index_02

26 My query object (Will be used in the examples)
public class SearchModel { private int? _numberToTake; public string Query { get; set; } public Dictionary<string, IEnumerable<string>> Filter { get; set; } public int? NumberToTake get { return _numberToTake.HasValue ? _numberToTake.Value : 25; } set { _numberToTake = value; } }

27 QueryinG Elasticsearch NEST
{ "query": { "query_string": { "query": "tomas" } _client.Search<Customer>(sd => sd .QueryString(Input.Query));

28 Fuzzy Elasticsearch NEST
{ "query": { "fuzzy": { "_all": { "min_similarity": 0.6, "prefix_length": 1, "value": "tomas" } _client.Search<Customer>(sd => sd .Query(q => q .Fuzzy(fd => fd .OnField("_all") .MinSimilarity(0.6) .PrefixLength(1) .Value(Input.Query)))); Will enable us to search for both «Thomas» and «Tomas» when writing «Tomas»

29 Fuzzy Improved (using bool query) - Elasticsearch
{ "query": { "bool": { "should": [{ "match": { "_all": { "query": "tomas" } }, "fuzzy": { "boost": 0.1, "min_similarity": 0.6, "prefix_length": 1, "value": "tomas" }]

30 Fuzzy Improved (using bool query) - NEST
_client.Search<Customer>(sd => sd .Query(q => q .Bool(b => b .Should(new Func<QueryDescriptor<Customer>, BaseQuery>[] { _ => _.Match(m => m .OnField("_all") .QueryString(Input.Query)), _ => _.Fuzzy(fd => fd .MinSimilarity(0.6) .PrefixLength(1) .Value(Input.Query) .Boost(0.1)) }))));

31 Highlight result - Elasticsearch
{ "query": { // see previous example }, "highlight": { "pre_tags": [ "<span class='highlight'>" ], "post_tags": [ "</span>" "fields": { "companyName": { "fragment_size": 100, "number_of_fragments": 1 }

32 Highlight result - NEST
_client.Search<Customer>(sd => sd .Query( /* See previous example */ ) .Highlight(h => h .PreTags("<span class='highlight'>") .PostTags("</span>") .OnFields(new Action<HighlightFieldDescriptor<Customer>>[] { _ => _ .OnField(c => c.CompanyName) .NumberOfFragments(1) .FragmentSize(100) })));

33 Facets - Elasticsearch
{ "query": { /* See previous example */ }, "highlight": { /* See previous example */ }, "facets": { "products.productName": { "nested": "products", "terms": { "field": "products.productName", "size": 1000 } }, "products.categoryName": { "terms": { "field": "products.categoryName", "size": 1000 } "country": { "terms": { "field": "country", "size": 1000 } }

34 Facets - NEST _client.Search<Customer>(sd => sd .Query( /* See previous example */ ) .Highlight( /* See previous example */ ) .FacetTerm(f => f .Nested(c => c.Products) .OnField(c => c.Products[0].ProductName) .Size(1000)) .OnField(c => c.Products[0].CategoryName) .OnField(c => c.Country) .Size(1000)));

35 http://go-gaga-over-testing. blogspot

36 Filters - Elasticsearch
{ "query": { "filtered": { "query": { /* See previous example */ }, "filter": { "bool": { "must": [ "terms": { "country": ["usa"] } }, "nested": { "query": { "terms": { "products.categoryName": ["Condiments", "Seafood"] } }, "path": "products" } "query": { "terms": { "products.productName": ["Chai"] } }, ] "facets": { /* See previous example */}, "highlight": { /* See previous example */ }

37 Filters – NEST – Part 1, the Customers filter
private static BaseFilter AddCustomerFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Terms(propExpr, items.ToArray()); }

38 Filters – NEST – Part 1, the Products filter
private static BaseFilter AddProductsFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Nested(sel => sel .Path(c => c.Products) .Query(q => q.Terms(propExpr, items.ToArray()))); }

39 Filters – NEST – Part 1, the Magic Dictionary
public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc = new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>() { {"products.productName", ps => AddProductsFilter(ps, c => c .Products[0].ProductName)}, {"products.categoryName", cs => AddProductsFilter(cs, c => c .Products[0].CategoryName)}, {"country", cs => AddCustomerFilter(cs, c => c.Country)} };

40 Filters – NEST – Part 1, All the helpers
private static BaseFilter AddCustomerFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Terms(propExpr, items.ToArray()); } private static BaseFilter AddProductsFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) return Filter<Customer>.Nested(sel => sel .Path(c => c.Products) .Query(q => q.Terms(propExpr, items.ToArray()))); public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc = new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>() {"products.productName", ps => AddProductsFilter(ps, c => c .Products[0].ProductName)}, {"products.categoryName", cs => AddProductsFilter(cs, c => c .Products[0].CategoryName)}, {"country", cs => AddCustomerFilter(cs, c => c.Country)} };

41 Filters – NEST – Part 2, The query
_client.Search<Customer>(sd => sd .Query(q => q .Filtered(fq => { fq.Query(qs => if (!string.IsNullOrEmpty(Input.Query)) qs.Bool( /* See previous example */ )); } else qs.MatchAll(); return qs; }); if (Input.Filter.Count > 0) var filters = Input.Filter.Select(_ => FilterDesc[_.Key](_.Value)).ToArray(); fq.Filter(fs => fs.Bool(bf => bf.Must(filters))); })) .Highlight( /* See previous example */ ) .FacetTerm( /* See previous example */ ) .FacetTerm( /* See previous example */ );

42 Summary Elasticsearch NEST Easy installation Awesome search engine
Strongly typed client Fluent Abstraction over Elasticsearch

43 ResourceS Demo code: https://github.com/mastoj/NestDemo Nest documentation: Nest source code: https://github.com/Mpdreamz/NEST Slideshare: Sense (great tool to query elastic search in the browser): https://github.com/bleskes/sense

44 Questions?

45 Thank you! @TomasJansson


Download ppt "Getting started with ElasticSearch on Windows and .NET with NEST"

Similar presentations


Ads by Google