Evaluation Plan Development: CDC Evaluation Framework
Step 1: Engage Stakeholders Conducted internal and external stakeholder assessment Ultimately engaged internal stakeholders only in evaluation planning
Program Objectives Increase the proportion of contacts to sputum AFB smear-positive TB patients with newly diagnosed LTBI who start treatment to 75%. For above contacts who have started treatment for LTBI, increase the proportion of that complete treatment to 65%.
Step 2: Describe the Program Baseline 59 (32 sm+) cases in 2007 90 LTBI contacts 64 (71%) started treatment 39 (61%) completed treatment Baseline Year
Resources . 5 FTE graduate intern x 6 months .1 FTE prevention specialist .1 FTE Program Manager Access to TB medical and program team
Step 3: Focus Evaluation Design Evaluation Goals: Describe current processes for bringing contacts with LTBI to treatment initiation and completion; verify baseline findings; describe characteristics of contacts that successfully begin and finish treatment; suggest changes to current process to improved treatment initiation and completion.
Step 4: Gather Credible Evidence Methods Quantitative Review −Program policies and protocols −2008 LTBI* medical/case management charts Observation of staff/patient interaction −Physician, nurse case manager, TB investigator Qualitative Interviews −patients who started and did/did not complete treatment. *number of LTBI contacts was very small, so expanded to all persons who started LTBI treatment
Step 5: Justify Conclusions Results Observations No actual written protocol for treatment offer Treatment offer was purview of the physician; no Consistent review of patient information sheet by nurse on those who accepted Chart/Data Review – 2008 data 33% (102/309) of LTBI patients completed therapy Black race and being foreign-born were associated with not completing treatment History of incarceration was associated with successfully completing treatment
Qualitative Interviews 37 Patients contacted for interviews 6 completed 43% phone # wrong or disconnected 30% never answered Bust
Step 6: Ensure Use and Share Lessons Learned Writing protocols-review & reinforce w/staff Determining steps to enhance treatment offer beyond physician offer Language appropriate patient materials Considering: Increased case manager contact in first 2 weeks of starting treatment Pilot with community provider serving target group (foreign born, black) to conduct LTBI treatment follow up.
Lessons Learned (cont) High percentage of patients are transient Get multiple sources of locating information Check assumptions about what staff think is happening Be realistic about what can be accomplished Assure data sources are easily accessed Assure evaluation is properly resourced Stay focused Avoid letting evaluation purpose drift or languish