Presentation is loading. Please wait.

Presentation is loading. Please wait.

SPARK Search Engine. Martijn Harthoorn Programmer at Furore Implementer of the Search Engine of SPARK

Similar presentations


Presentation on theme: "SPARK Search Engine. Martijn Harthoorn Programmer at Furore Implementer of the Search Engine of SPARK"— Presentation transcript:

1 SPARK Search Engine

2 Martijn Harthoorn Programmer at Furore Implementer of the Search Engine of SPARK The work after the question mark. Who am I?

3 The place of Search RESTService Storage Index & Search MongoDB Spark

4 Paradigm FHIR client should be easy. FHIR server needs to solve the complex issues. Search Search has some…

5 First there was Storage Search Then there was Search

6 Connectathon To test a client – you must have a tested server To test a server – you must have a tested client “One fool can ask more questions than seven wise men can answer”

7 Connectathon “But what if you are wrong?”

8 History Version 1. -A Generics based implementation -On top of the FHIR data model. -Programmed per search parameter programming. -No meta data available yet. -No indexing. -Slow.

9 History Version 2. -Data Model independent, -Meta data not available - manually added -Lucene.NET as indexer (Index in Lucene, Database in Mongo) -Fast -Standardised all parameter specifics into standard “modifiers”. -All Code based on search parameter types. -Joins are client side

10 History Version 3. -Modified to store the Lucene index in Mongo -Index storage unreliable. -Never saw light of day

11 History Version 4. CURRENT -Index storage to a dedicated Mongo collection -Build expression tree from parameters -Chained parameters have full functionality (modifiers, operators) -Joins are client side

12 Indexing Why indexing?

13 Why indexing

14 Why indexing

15 Indexing. HOW-TO You DO want A de-serialized data to an object with all values strongly typed. You DON’T want to spend time analyzing and interpreting JSON and/or XML. 1.Harvest the Resource 2.Determine data type 3.Groom your data 4.Store data in Index

16 Indexing. 1. Harvesting Resource: Patient Search parameter: family Searches for the family name and prefix of every HumanName that is registered with a Patient. Usage:

17 Indexing. 1. Harvesting Patient List Name (HumanName) Family Prefix Given Suffix Resource: Patient Search parameter: family Using the Visitor pattern Path from Meta data: "patient.Name.Prefix" "patient.Name.Family"

18 Indexing. 2. Determine data type > patient (Patient) > Name (HumanName) > LastName (string) Data type: string Search parameter type: string Selected indexing method: -Single value – as string -More values – as string array

19 Indexing. 2. Determine data type > patient (Patient) > Gender (Coding) > Coding (List ) > Code (CodeableConcept) Data type: Code Search parameter type: Token Selected Indexing method: Store in an array each codeable concept -System (uri) -Code (string) -Display (string)

20 Indexing. 3. Groom your data -Remove dashes, dots, slashes from dates etc. -If you implement a like search from the left side, you might want to split names at the dash in to multiple hits.

21 Indexing. 4. Store in the index FieldValue Resource"Patient" Local IDpatient/1 Level0 Family["LaVaughn", "Robinson", "Obama"] Given"Michelle" Gender[ { System: “…”, Code: “..”, Display: “..” }, … … * Level The patient is not a contained resource (level 0) * Family In Mongo you can store an array that can be searched like a normal string.

22 Future Version 5. NEXT -All parameters based on FHIR data types? -Joins using Mongo Map-Reduce?

23 Complexity So what is the issue?

24 Complexity Include & Chained parameters -Joining over references return multiple resource types -Client side (not in Mongo database) joins

25 Complexity Transactions -FHIR has bulk POST -Split between Indexing and storage

26 Complexity Multiple types Some properties do not have a fixed type. Example: observation.value Can be a: -CodeableConcept -String -Quantity (number + unit)


Download ppt "SPARK Search Engine. Martijn Harthoorn Programmer at Furore Implementer of the Search Engine of SPARK"

Similar presentations


Ads by Google