Importance of Data Quality for National HIV Prevention Program Monitoring and Evaluation Presented by: Guoshen Wang, MS Shubha Rao, MPH; Hui Zhao, MS; Wei Song, PhD, MS; Carolyn Wright, BS; and Marc V. Wiehn, MS 2019 Joint Statistical Meetings July 27, 2019 – August 1, 2019 Good morning everyone. My name is Guoshen Wang and I am a health scientist with the Program Evaluation Branch in the Division of HIV/AIDS Prevention. Today, I will present on the Importance of Data Quality for National HIV Prevention Program Monitoring and Evaluation. Before I proceed, I would like to thank my co-authors ….. Marc from LC…

Background CDC testing recommendations CDC-funded HIV Testing Everyone between the ages of 13 and 64 get tested for HIV at least once as part of routine health care For those with specific risk factors, CDC recommends getting tested at least once a year CDC-funded HIV Testing Under CDC’s high impact approach to HIV prevention, testing is a core activity supported through funding opportunities for health departments and community based organizations Funding recipients are expected to implement an HIV testing strategy that includes routine, opt-out HIV screening and targeted testing among priority populations in their local jurisdiction CDC recommends that everyone between the ages of 13 and 64 get tested at least once as a part of their routine health care. For those with specific risk factors, CDC recommends getting tested at least once a year. HIV testing is a core activity supported by CDC through funding opportunities for health departments and community based organizations. CDC’s high impact prevention approach emphasizes HIV testing targeted to the right populations in the right geographic areas. This includes opt-out, routine HIV screening most often associated with testing in health care settings and targeted testing that can occur in health care and non-health care settings. --------------------------------------------------- Additional information for you *If asked about specific risk factors - Are you a man who has had sex with another man? Have you had sex—anal or vaginal—with an HIV-positive partner? Have you had more than one sex partner since your last HIV test? Have you injected drugs and shared needles or works (for example, water or cotton) with others? Have you exchanged sex for drugs or money? Have you been diagnosed with or sought treatment for another sexually transmitted disease? Have you been diagnosed with or treated for hepatitis or tuberculosis (TB)? Have you had sex with someone who could answer yes to any of the above questions or someone whose sexual history you don’t know? 2019 Joint Statistical Meetings 2 of 17

Methods

Data collection flowchart This flowchart shows how HIV testing data are received by our health department monitoring and evaluation team. Starting from data collection requirements to reporting 2019 Joint Statistical Meetings 4 of 17

Assessing Data Quality - EvaluationWeb Control Process Data Freeze Generate all databases Check for irregularities such as duplicates, blank records, etc. & remove. Create a list of all removed records & reasons for removal Encrypt data and submit to CDC Run all required database checks & compare to expected results Generate & export database and lookup tables Generate balance sheets 2019 Joint Statistical Meetings 5 of 17

Assessing Data Quality - CDC Dataset Generation Process CDC processes relational SQL database to SAS HIV testing raw SAS format flat dataset Implement HIV indicator algorithm and generate provisional HIV testing dataset Finalized HIV test-level dataset Data correction based on recipients’ QA feedback Quality assurance, feedback to recipients 2019 Joint Statistical Meetings 6 of 17

2019 Joint Statistical Meetings 7 of 17 This is the cover of our latest HIV testing data quality assurance procedures guidance. We report on the completeness of each variable. 2019 Joint Statistical Meetings 7 of 17

Monitoring and Evaluation Feedback Reports I will now present examples of the monitoring and evaluation reports we produce.

PS12-1201 Rapid Feedback Report (RFR) Brief reports describing recipient’s program achievements and progress toward meeting funding performance standards Include data from selected key indicators of all funded recipients Intended to promote accountability for recipients and CDC Disseminated within 3 months of program data submission (i.e., June and December annually) Rapid feedback reports are brief reports describing program achievement and progress toward meeting program performance standards. The reports include data from all funded grantees and are intended to promote accountability for grantees and CDC. Our division uses rapid feedback reports as one of several tools to assess individual grantee performance and identify capacity-building assistance needs. The reports are disseminated no later than 3 months after program data submission. 2019 Joint Statistical Meetings 9 of 17

PS12-1201 Rapid Feedback Report (RFR) This is an example of three pages from one of the rapid feedback reports. 2019 Joint Statistical Meetings 10 of 17

PS12-1201 Individual Grantee Report (IGR) Each grantee also receives an Individual Grantee Report along with the rapid feedback report. The individual grantee report summarizes the grantee’s performance compared to all grantees and compared to program objectives or targets. 2019 Joint Statistical Meetings 11 of 17

Results: CDC-funded HIV Testing Trends, 2012-2017 I will now present examples of HIV testing data results for the period 2012-2017.

Newly diagnosed HIV positivity HIV testing and newly diagnosed HIV positivity, 2012-2017 19.7 million CDC-funded tests; 101,124 new diagnoses (0.51 positivity) # CDC-funded HIV tests Newly diagnosed HIV positivity EPAC (%): (-1.83*) (-0.76*) There was a total of about 19.7 million CDC-funded HIV tests conducted between 2012 and 2017. There were over 100,000 new diagnoses for a positivity of 0.51. There was a significant decrease in overall HIV testing and new positivity between 2012 and 2017. Testing decreased by nearly 2% and new positivity by almost 1%. Despite a decline of about 2%, there are still more than 3 million CDC-funded HIV tests conducted each year between 2012-2017. 2019 Joint Statistical Meetings 13 of 17

Linkage to HIV medical care within 90 days among newly diagnosed HIV positive persons, 2012-2017 Percent linked EAPC=0.39%* There was a significant increase in linkage to care among newly diagnosed HIV positive person from 2012 to 2017. The percentage in 2012 was 82.6% and it increased to 85.2% in 2017. However, the number of newly diagnosed HIV positive persons may be overestimated at all time periods since it is based on self-report. 2019 Joint Statistical Meetings * p < .01 14 of 17

Percentage Missing Data for Key HIV Testing Indicators Data Source: PS12-1201 HIV testing data 2012 2013 2014 2015 2016 2017 This slide shows improvements in data quality over the last 5-6 years. The percentage of records with missing or invalid information has decreased for linkage, partner services, and referral to prevention services. This highlights the efforts by health department staff and CDC mainly through integrated data quality calls and sustained technical assistance. 2019 Joint Statistical Meetings 15 of 17

Summary Quality of program evaluation data has been assessed at every step for accuracy, completeness, and logical patterns Data quality assurance process has also improved over years For key HIV testing indicators: Linkage to HIV medical care, missing data decreased from 37.2% in 2012 to 14.6% in 2017 Interview for partner service, missing data decreased from 41.5% in 2012 to 13.9% in 2017 Referral to HIV prevention service, missing data decreased from 25.3% in 2012 to 19.0% in 2017 In summary, the quality of program evaluation data has been assessed at every step. As a result, data quality for national HIV prevention program monitoring and evaluation has improved over years. For example, for the variable “Linkage to HIV medical care,” missing data decreased from 37.2% in 2012 to 14.6% in 2017 2019 Joint Statistical Meetings 16 of 17

Questions? Disclaimer: The findings and conclusions in this presentation are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.