ELLEN ZAGER HILL, MS TB PROGRAM EPIDEMIOLOGIST IDAHO DEPARTMENT OF HEALTH & WELFARE Refugee and Immigrant B Notification Program Evaluation— Idaho

Idaho Geography (1) Geographically large western state with a low population  19.2 people per square mile (ranked 44 th in US)  Population ≈ 1.58 million people  37% of population is in the Boise metro area (Ada & Canyon Counties)

Idaho Geography (2) 44 counties divided into 7 multi-county regional public health districts 95%+ of refugees and immigrants settle in Boise (Ada County) and Twin Falls (Twin Falls County).

Background For a state with the size population that Idaho has, the state receives an unusually large number of refugees and immigrants 2 local public health districts receive essentially all of the refugees arriving in the state: PHD4 (90%+) and PHD5 (~10%) Immigrants arriving in Idaho are pretty evenly distributed; in SFY 2007–2009, 83 immigrants with TB notifications arrived SFY Idaho Population No. of Refugees Arriving Per capita ratio (per 10,000 people) No. with TB notifications 20071,499, (3.5%) 20081,523,8161, (5.0%) 20091,545,8011, (5.8%)

Evaluation Plan

Logic Model Refugee/ Immigrant with a TB notification Inputs Private Provider State TB Program Staff LPHD Short term Long term Outcomes The LPHD will increase the proportions of completion of medical evaluations within 90 days. Increase the proportions of refugees/immigrants who started on LTBI treatment who complete treatment. Reduce the number of foreign born TB cases. Outputs Every LPHD has a process for contacting, evaluating, and treating refugees and immigrants with TB notification. Data for grant progress report. Refugees and immigrants with a TB notification are medically evaluated for TB. Refugees or immigrants treated who need to be treated. Activities Form turned into state TB program Fill out CDC TB Notification form Complete medical evaluation Complete treatment for LTBI OEFI receives notification from CDC Start treatment for LTBI District receives notification from OEFI Contact Refugee or immigrant Initiate medical evaluation

Evaluation Goal Evaluation Team Baseline analysis of medical screen activities of refugees and immigrants arriving in Idaho with a TB notification Lead Evaluator  Ellen Zager Hill – TB PEN Focal Point/TB Epi Team Members  Christine Hahn – TB Controller/State Epi  Adele Smith – MPH student from ISU Evaluation Plan (1)

Evaluation Plan (2) Big Picture QuestionsCorresponding Specific Questions Are the medical evaluations for refugees and immigrants with TB notifications done in a timely manner? What is the proportion of refugees or immigrants that were evaluated within 90 days of arrival? Are the PHDs sending notifications back to the OEFI in a timely manner? What is the proportion of TB notification forms that are received within 120 days of receiving the notification? What is the process that the PHDs go through to contact the refugees and immigrants with TB notifications for medical evaluations? 7 out of 7 PHDs have a process to contact refugees or immigrants with TB notifications

Evaluation Plan (3) Big Picture Questions (continued)Corresponding Specific Questions (continued) Do the PHDs have a different process for evaluating and treating refugees and immigrants with TB notifications? 7 out of 7 PHDs have no difference in evaluation or treatment of refugees and immigrants with TB notifications What is the process the PHDs perform to evaluate refugees and immigrants with TB notifications? 7 out of 7 PHDs have a process to evaluate refugees or immigrants with TB notifications. What is the process the PHDs perform to treat refugees and immigrants with TB notifications? 7 out of 7 PHDs have a process to treat the refugees or immigrants with TB notifications who have LTBI.

Data Sources Execution Needs Idaho Refugee & Immigrant Log (MS Excel spreadsheet) CDC’s Electronic Disease Notification System Interviews with PHD staff who coordinate TB activities (via phone) Time Survey questions Phone Software to map processes (MS Visio) (relatively) Clean TB notification data Data analysis tool (MS Excel and SAS) Evaluation Plan (3)

Carrying Out the Evaluation Plan

Carrying Out the Evaluation Plan (1) Developed a set of questions to ask the PHD staff who carry out TB activities Developed a script to go with the questions

Carrying out the Evaluation Plan (2) A flow of TB notification data from the state-level perspective was developed (shown at left) Based on survey questions, flow charts of the evaluation and treatment processes were developed for each PHD

Data Cleaning Data Analysis Started with our Idaho Refugee & Immigrant Log  Took out all records that were not for TB notifications Filled in what missing data we could from hard copy records and the EDN system Initially done with MS Excel Final analysis done with SAS  Benefit of using a program like SAS is that you have a record of exactly how the data was cleaned, what analysis was done, and how it was done. Carrying out the Evaluation Plan (3)

Carrying out the Evaluation Plan (4) Some lessons learned… Logs (especially those with no data standardization programmed in) are a difficult data source to analyze (A lesson re-learned) small denominators can be a problem Clearly document how you collect, clean, and analyze data  An evaluation report is like a lab report.  Your methods section may be your most important section. (Sometimes even more important than your results.)  SAS or other statistical analysis programs are a great way to document how you are cleaning and analyzing data

Evaluation Results

Evaluation Results (1) Are the medical evaluations for refugees and immigrants with TB notifications done in a timely manner?  What is the proportion of refugees or immigrants that were evaluated within 90 days of arrival? Overall (SFY2007–2009) 67% were evaluated within 90 days of arrival.

Evaluation Results (2) Are the PHDs sending notifications back to the OEFI in a timely manner?  What is the proportion of TB notification forms that are received within 120 days of receiving the notification? Overall 76% of TB notification forms were returned within 120 days of receiving the original notification.

Evaluation Results (3) Process QuestionsResults 7 out of 7 PHDs have a process to contact refugees or immigrants with TB notifications All 6 PHDs interviewed have a process (the 7 th is known (anecdotally) to have a process. Most have a process for contacting hard-to-reach R/Is and a defined point when they declare someone “lost-to-follow-up” 7 out of 7 PHDs have no difference in evaluation or treatment of refugees and immigrants with TB notifications Of the 6 PHDs interviewed only 4 have received notifications for both refugees and immigrants. Of these 4, only 1 reported a difference in protocol (a different staff member follows each group) 7 out of 7 PHDs have a process to evaluate refugees or immigrants with TB notifications All PHDs have a process for evaluating refugees and immigrants There is significant variation in these processes between PHDs 7 out of 7 PHDs have a process to treat the refugees or immigrants with TB notifications who are diagnosed with LTBI. Of the 6 PHDs interviewed, 5 actively follow R/Is on treatment for LTBI. The remaining PHD prefers that R/Is diagnosed with LTBI are treated and followed in their medical home.

Evaluation Results (4) Extra analysis tied to differences in processes… *Excludes data from 1 PHD (the PHD with the most notifications) because we were unable to interview them.

PHD-level State-level Ensure there is a system in place to track TB notifications Review process for coordinating medical evaluation on a semi- regular schedule Consult OEFI TB staff as needed when a problem related to medical evaluation arises Consider adding language to TB contracts stipulating a time frame for returning TB follow-up forms Update B Notifications chapter of Idaho TB manual Provide at least 1 NTIP-like report a year to each PHD that contains an analysis of their TB notification statistics Review data from calendar years 2009, 2010, and 2011 for improvement. If improvement is not seen, then consider re- evaluating this activity. Recommendations

What has been done as a result of the evaluation We are no longer using the Idaho Refugee and Immigrant log to monitor performance for this objective. We are using solely EDN data. We added language to contracts stipulating when TB follow-up data needs to be submitted. B Notifications chapter of TB manual has been reviewed an updated, but has not yet been finalized. Completion of medical evaluation statistics as of 1/31/2012: (calendar years) N Mean (days) Std Dev (days) Low CI (days) High CI (days) Min (days) Max (days) This data reflects only notifications for which data has been received. 2.Please note that data for all years are incomplete for various reasons. 3.The improvements in data are mainly due to process changes at the PHD that receives 90+% of refugees in the state. (This PHD was not interviewed for the evaluation.)

Questions