1. Generating Medical Logic Modules for Clinical Trial Eligibility Craig Parker Brigham Young University

2. Clinical Trials Important for modern medical research Efficacy of therapies Safety of therapies Sponsored by Government (www.clinicaltrials.gov)www.clinicaltrials.gov Academic institutions Industry (especially pharmaceutical companies)

3. Clinical Trial Enrollment Power in numbers Traditional enrollment methods MethodsLimitations Regular office visitsLimited pool of patients AdvertisingLimited medical knowledge, Cost Chart reviewLimited information, Requires medical training

4. What if... we could automatically extract eligibility criteria from electronic medical records?

5. Electronic Medical Records Good News Large amounts of data being collected Efforts to standardize representations are well supported Bad News Most EMRs are far from complete Not all representations are in a standard form Legacy data Unstandardized realms

6. Thesis Statement Generate medical logic modules to represent eligibility criteria for clinical trials Start with criteria as first-order predicate logic Map concepts with medical vocabularies and ontologies Create medical logic modules Handle concepts that can’t be mapped Measure results

7. Medical Logic Modules (MLMs) An abstract term describing the knowledge necessary for making a medical decision A specific type of medical program Represented using the Arden Syntax ANSI standard Broad vendor acceptance Compiles to an executable form

8. MLM in Arden Syntax KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:...

9. Beginning...

10. Beginning and End KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:...

11. Overview Trial KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM

12. Overview Trial Predicates Logic KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM

13. Overview Trial Predicates Logic KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing

14. Overview Trial Predicates Logic KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing (P1) Gender: Female (P2) Pregnant (P3) Gestational age > 23.0 wks (P4) Gestational age < 31.6 wks (P5) Chorioamnionitis (P6) Non-reassuring fetal testing

15. Overview Trial Predicates Logic KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing (P1) Gender: Female (P2) Pregnant (P3) Gestational age > 23.0 wks (P4) Gestational age < 31.6 wks (P5) Chorioamnionitis (P6) Non-reassuring fetal testing P1  P2  (P3  P4)  NOT (P5  P6) Or in conjunctive normal form: P1  P2  P3  P4  NOT P5  NOT P6

16. Overview Trial Predicates Logic KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing (P1) Gender: Female (P2) Pregnant (P3) Gestational age > 23.0 wks (P4) Gestational age < 31.6 wks (P5) Chorioamnionitis (P6) Non-reassuring fetal testing P1  P2  (P3  P4)  NOT (P5  P6) Or in conjunctive normal form: P1  P2  P3  P4  NOT P5  NOT P6

17. Overview Trial Term Mapping Logic Handling Predicates Logic KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM

18. Overview Trial Term Mapping Logic Handling Predicates Logic UMLS Other Knowledge Target Schema KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM

19. Overview Trial Additional Information Term Mapping Logic Handling Predicates Logic UMLS Other Knowledge Target Schema KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM

20. Steps to Create MLMs Classify predicates Map concepts from trial to database Translate logic Generate MLM

21. Classifying Predicates Numeric comparisons e.g. gestational age > 23.0 wks time, lab values, physiologic measurements Single noun phrases e.g. pregnant diagnoses, observations Two noun phrases e.g. Gender: Female name-value pairs

22. Mapping Example 1 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing

23. Mapping Example 1 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing Single noun phrase – likely to be a diagnosis or observation

24. Mapping Example 1 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing pregnant: pregnancy pregnancy appointment type pregnant ambulatory status... has-parent: Diagnosis has-parent: Appointment type has-parent: Ambulatory status From vocabulary of target database

25. Mapping Example 1 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing pregnant: pregnancy pregnancy appointment type pregnant ambulatory status... has-parent: Diagnosis has-parent: Appointment type has-parent: Ambulatory status

26. Mapping Example 1 pregnant: pregnancy (83035) has-parent: diagnosis

27. Mapping Example 1 pregnant: pregnancy (83035) has-parent: diagnosis Looking for “pregnancy” in schema of target database.

28. Mapping Example 1 pregnant: pregnancy (83035) has-parent: diagnosis PregnancyObservation : Is-Subtype-Of : DiagnosisAndFindingObservation { value(codedTerm({pregnancy, 83035})); negation(boolean);... } Looking for “pregnancy” in schema of target database.

29. Mapping Example 1 pregnant: pregnancy (83035) has-parent: diagnosis Look for “PregnancyObservation” in vocabulary of target database. PregnancyObservation : Is-Subtype-Of : DiagnosisAndFindingObservation { value(codedTerm({pregnancy, 83035})); negation(boolean);... }

30. Mapping Example 1 pregnant: pregnancy (83035) has-parent: diagnosis PregnancyObservation (59665) has-parent: observation PregnancyObservation : Is-Subtype-Of : DiagnosisAndFindingObservation { value(codedTerm({pregnancy, 83035})); negation(boolean);... } Look for “PregnancyObservation” in vocabulary of target database.

31. Mapping Example 1 pregnant: pregnancy (83035) has-parent: diagnosis PregnancyObservation (59665) has-parent: observation PregnancyObservation : Is-Subtype-Of : DiagnosisAndFindingObservation { value(codedTerm({pregnancy, 83035})); negation(boolean);... }

32. Mapping Example 1 pregnant: pregnancy (83035) has-parent: diagnosis PregnancyObservation (59665) has-parent: observation PregnancyObservation : Is-Subtype-Of : DiagnosisAndFindingObservation { value(codedTerm({pregnancy, 83035})); negation(boolean);... } SELECT * FROM Observations WHERE ObsId = 59665 AND value = 83035

33. Mapping Example 2 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing

34. Mapping Example 2 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing Two noun phrases – likely to be a name- value pair

35. Mapping Example 2 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing Gender MaleFemale Demographics

36. Mapping Example 2 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing Gender MaleFemale Demographics Look in target database for a concept of ‘Gender’ with a value of ‘Female’.

37. Unmappable Concepts Concept or value not in target database Concept does not exist “Delivery intended outside center” Too Hard Inclusion: “Received full course of corticosteroids in the previous 7 days” Exclusion: “Corticosteroid therapy, other than qualifying course”

38. Unmappable Concepts Concept or value not in target database Concept does not exist “Delivery intended outside center” Too Hard Inclusion: “Received full course of corticosteroids in the previous 7 days” Exclusion: “Corticosteroid therapy, other than qualifying course” Solution Evaluate eligibility based on available data If eligibility is possible, present questionnaire to user for outstanding information needed

39. Evaluation Select ~25 trials (~200 predicates) from ClinicalTrials.gov Precision and recall of term mappings Precision and recall of predicate mappings Percentage of predicates that are mappable Correctness of logic in Arden Syntax modules