Generating Medical Logic Modules for Clinical Trial Eligibility Craig Parker Brigham Young University

Clinical Trials Important for modern medical research Efficacy of therapies Safety of therapies Sponsored by Government ( Academic institutions Industry (especially pharmaceutical companies)

Clinical Trial Enrollment Power in numbers Traditional enrollment methods MethodsLimitations Regular office visitsLimited pool of patients AdvertisingLimited medical knowledge, Cost Chart reviewLimited information, Requires medical training

What if... we could automatically extract eligibility criteria from electronic medical records?

Electronic Medical Records Good News Large amounts of data being collected Efforts to standardize representations are well supported Bad News Most EMRs are far from complete Not all representations are in a standard form Legacy data Unstandardized realms

Thesis Statement Generate medical logic modules to represent eligibility criteria for clinical trials Start with criteria as first-order predicate logic Map concepts with medical vocabularies and ontologies Create medical logic modules Handle concepts that can’t be mapped Measure results

Medical Logic Modules (MLMs) An abstract term describing the knowledge necessary for making a medical decision A specific type of medical program Represented using the Arden Syntax ANSI standard Broad vendor acceptance Compiles to an executable form

MLM in Arden Syntax KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:...

Beginning...

Beginning and End KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:...

Overview Trial KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM

Overview Trial Predicates Logic KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM

Overview Trial Predicates Logic KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing

Overview Trial Predicates Logic KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing (P1) Gender: Female (P2) Pregnant (P3) Gestational age > 23.0 wks (P4) Gestational age < 31.6 wks (P5) Chorioamnionitis (P6) Non-reassuring fetal testing

Overview Trial Predicates Logic KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing (P1) Gender: Female (P2) Pregnant (P3) Gestational age > 23.0 wks (P4) Gestational age < 31.6 wks (P5) Chorioamnionitis (P6) Non-reassuring fetal testing P1  P2  (P3  P4)  NOT (P5  P6) Or in conjunctive normal form: P1  P2  P3  P4  NOT P5  NOT P6

Overview Trial Predicates Logic KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing (P1) Gender: Female (P2) Pregnant (P3) Gestational age > 23.0 wks (P4) Gestational age < 31.6 wks (P5) Chorioamnionitis (P6) Non-reassuring fetal testing P1  P2  (P3  P4)  NOT (P5  P6) Or in conjunctive normal form: P1  P2  P3  P4  NOT P5  NOT P6

Overview Trial Term Mapping Logic Handling Predicates Logic KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM

Overview Trial Term Mapping Logic Handling Predicates Logic UMLS Other Knowledge Target Schema KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM

Overview Trial Additional Information Term Mapping Logic Handling Predicates Logic UMLS Other Knowledge Target Schema KNOWLEDGE: TYPE:... DATA: gestational_age := read last {SELECT gest_age FROM Observation}; ;; EVOKE:... LOGIC: IF gestational_age >= 73 days AND gestational_age <= 97 days THEN conclude true; ENDIF; ;; ACTION:... MLM

Steps to Create MLMs Classify predicates Map concepts from trial to database Translate logic Generate MLM

Classifying Predicates Numeric comparisons e.g. gestational age > 23.0 wks time, lab values, physiologic measurements Single noun phrases e.g. pregnant diagnoses, observations Two noun phrases e.g. Gender: Female name-value pairs

Mapping Example 1 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing

Mapping Example 1 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing Single noun phrase – likely to be a diagnosis or observation

Mapping Example 1 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing pregnant: pregnancy pregnancy appointment type pregnant ambulatory status... has-parent: Diagnosis has-parent: Appointment type has-parent: Ambulatory status From vocabulary of target database

Mapping Example 1 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing pregnant: pregnancy pregnancy appointment type pregnant ambulatory status... has-parent: Diagnosis has-parent: Appointment type has-parent: Ambulatory status

Mapping Example 1 pregnant: pregnancy (83035) has-parent: diagnosis

Mapping Example 1 pregnant: pregnancy (83035) has-parent: diagnosis Looking for “pregnancy” in schema of target database.

Mapping Example 1 pregnant: pregnancy (83035) has-parent: diagnosis PregnancyObservation : Is-Subtype-Of : DiagnosisAndFindingObservation { value(codedTerm({pregnancy, 83035})); negation(boolean);... } Looking for “pregnancy” in schema of target database.

Mapping Example 1 pregnant: pregnancy (83035) has-parent: diagnosis Look for “PregnancyObservation” in vocabulary of target database. PregnancyObservation : Is-Subtype-Of : DiagnosisAndFindingObservation { value(codedTerm({pregnancy, 83035})); negation(boolean);... }

Mapping Example 1 pregnant: pregnancy (83035) has-parent: diagnosis PregnancyObservation (59665) has-parent: observation PregnancyObservation : Is-Subtype-Of : DiagnosisAndFindingObservation { value(codedTerm({pregnancy, 83035})); negation(boolean);... } Look for “PregnancyObservation” in vocabulary of target database.

Mapping Example 1 pregnant: pregnancy (83035) has-parent: diagnosis PregnancyObservation (59665) has-parent: observation PregnancyObservation : Is-Subtype-Of : DiagnosisAndFindingObservation { value(codedTerm({pregnancy, 83035})); negation(boolean);... }

Mapping Example 1 pregnant: pregnancy (83035) has-parent: diagnosis PregnancyObservation (59665) has-parent: observation PregnancyObservation : Is-Subtype-Of : DiagnosisAndFindingObservation { value(codedTerm({pregnancy, 83035})); negation(boolean);... } SELECT * FROM Observations WHERE ObsId = AND value = 83035

Mapping Example 2 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing

Mapping Example 2 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing Two noun phrases – likely to be a name- value pair

Mapping Example 2 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing Gender MaleFemale Demographics

Mapping Example 2 Inclusion: Gender: Female Pregnant Gestational age > 23.0 wks and < 31.6 wks Exclusion: Chorioamnionitis Non-reassuring fetal testing Gender MaleFemale Demographics Look in target database for a concept of ‘Gender’ with a value of ‘Female’.

Unmappable Concepts Concept or value not in target database Concept does not exist “Delivery intended outside center” Too Hard Inclusion: “Received full course of corticosteroids in the previous 7 days” Exclusion: “Corticosteroid therapy, other than qualifying course”

Unmappable Concepts Concept or value not in target database Concept does not exist “Delivery intended outside center” Too Hard Inclusion: “Received full course of corticosteroids in the previous 7 days” Exclusion: “Corticosteroid therapy, other than qualifying course” Solution Evaluate eligibility based on available data If eligibility is possible, present questionnaire to user for outstanding information needed

Evaluation Select ~25 trials (~200 predicates) from ClinicalTrials.gov Precision and recall of term mappings Precision and recall of predicate mappings Percentage of predicates that are mappable Correctness of logic in Arden Syntax modules