ASSESSING THE QUALITY OF POPULATION SIZE ESTIMATES OF PEOPLE WHO INJECT DRUGS (PWID) Waimar Tun, Population Council 20 th International AIDS Conference Melbourne, Australia July 20 – 25, 2014

Background Current PWID population size estimations (PSE) in many countries are not based on strong data UNODC and World Bank requested a review of existing PWID population size estimates in 10 countries – Belarus, China, India, Libya, Myanmar, Philippines, Kazakhstan, Kyrgyzstan, Tajikistan, & Uzbekistan

Learning objectives Be able to critically review existing estimates and their methodology Be able to identify opportunities to improve the estimates (if required) Understand the strengths and weaknesses of methods of PSE data collection

Where do I get my data? Published and grey literature (including HIV- and drug- related country reports) Discussions with stakeholders in-country (NAP/NAC, UNAIDS, WHO, UNODC, CDC, USAID, PWID representatives, civil society organizations) Stakeholder meeting with representatives from MoH, drug control agencies, civil society, and implementing partners

Comment on data sources PSE methodologies not or poorly specified Reports not translated or translations of technical terms were ambiguous Stakeholders not always aware of PSE activities happening in their own country Time-consuming

What should I consider when I review the quality of the estimates? Are the underlying assumptions of the method met? What are the potential biases and how do they impact the estimate? Are multiple methods used? What is the quality of the data used?

Common methods to estimate PWID population size

Literature review/desk exercise Strengths: – Low cost – Little time/resources required Weaknesses: – Local contexts may be very different – Data sources may not be based on rigorous methods Review published and grey literature for similar context and geographic region The benchmark from literature is applied to the adult male and female population

Delphi Strengths: – Utilizes local expert views and experience – Does not require raw data capture – May be only option for countries with limited data sources Weaknesses: – Sometimes based only on qualitative or anecdotal information – Not good for identifying trends, comparing to other regions Systematically solicits and reviews selected experts’ estimates Iterative process through a series of feedback and revisions

Mapping/census and enumeration Strengths: – Is a real count, not an estimate – Can produce a credible lower limit Weaknesses: – PWID not always accessible, may not be exposed to census data collectors – Assumes you can locate all PWID – PWID may not wish to reveal drug use due to stigma and/or legal concerns – Highly time consuming and expensive – Safety issues; dangerous hotspots Venues where PWID congregate are identified Census counts all PWID at all hotspots Enumeration counts PWID at a sample of sites

Capture-recapture Strengths: – Fairly easy to implement in short period of time – Usually cost-efficient to implement – Can be done with multiple service sources Weaknesses: – Assumptions of method difficult to meet in reality (independent samples) – May be dangerous to implement (unsafe hotspots) PWID are counted and ‘tagged’; a 2 nd count is conducted An estimate is obtained through a formula that includes captures and overlaps between the 2 rounds

Service multipliers Strengths: – Uses existing and available data from service providers – Easily incorporated into IBBSS with minimal additional questions Weaknesses: – High-quality service data may not be available No duplicates Each target population member must have chance of being included in service data Requires two data sources: 1.Benchmark (service data such as drug treatment or HIV testing) 2.Population-based survey with PWID where you obtain info on the proportion who report that point of contact (‘benchmark) Estimate obtained from multiplying inverse of proportion to the benchmark

Wisdom of the crowd Strengths: – Very easy to implement. Only one question. – Easily incorporated into IBBSS or other size estimation methods Weaknesses: – May be biased if large segment of population is not well-networked or “hidden” – Bias if not implemented in a representative survey Ask PWID survey participants to estimate the number of PWID in a given location Assumes that the average response approximates the actual number

RDS † successive sampling size estimator ‡ Strengths: – Easily calculated with existing RDS survey data Weaknesses: – Statistical validity currently under debate – Not recommended as a an “only” method of estimation. – Results may be biased depending on number of people surveyed and actual population size In RDS-based survey, respondents indicate their network size Modelling is based on the assumption that those with large networks are sampled first and that the population will be depleted at a certain point † Respondent-driven sampling; ‡ Handcock, Gile, Mar (2012)

Network scale-up method † (NSUM) Strengths: – Does not ask sensitive questions directly to respondent – National level estimate Weaknesses: – Average personal network size difficult to estimate – Some PWID may not interact much with members of the general population – Respondent may not be aware that someone in their network engages in injection drug use Uses general population survey; questions about: Number of people they know of a known population Number of PWID they know † Bernard, Killworth, Johnsen, and Robinson (1991)

Triangulation of multiple methods Data points from multiple methods are desirable when possible – Reduces bias from any single method – May provide plausible lower and upper bounds – Informs stakeholder debate – Facilitates consensus on estimate ranges

PWID population size estimation (2011 IBBSS, Nairobi) Lower plausible 5,031 Upper plausible 10,937 (~0.5% adults) Source: Population Council, UCSF, NASCOP/Kenya, CDC (2011)

RDS SS-Size Added Source: Population Council, UCSF, NASCOP/Kenya, CDC (2011) Lower plausible 5,031 Median 6,107 Upper plausible 10,937 (~0.5% adults)

Cost Scientific rigor Straw man Conventional Wisdom Borrow from thy neighbor Soft modeling Consensus Wisdom of the crowd Delphi Registries, police, SHC, drug treatment, unions, workplace Discrepancies Place, RAP, ethnography Unique event multiplier Truncated Poisson Multipliers, multiple multipliers Multiple sample recapture Capture-recapture Network scale up Population-based survey Census Nomination counting Unique object multiplier Mapping, key informants, observation counting Scientific rigor and costs of methods Source: University of California, San Francisco RDS – Sequential Size

Cost Scientific rigor Straw man Conventional Wisdom Borrow from thy neighbor Soft modeling Consensus Wisdom of the crowds Delphi Registries, police, SHC, drug treatment, unions, workplace Discrepancies Place, RAP, ethnography Unique event multiplier Truncated Poisson Multipliers, multiple multipliers Multiple sample recapture Capture-recapture Network scale up Population-based survey Census Nomination counting Unique object multiplier Mapping, key informants, observation counting No resources or opportunity for data collection Source: University of California, San Francisco RDS – Sequential Size

Cost Scientific rigor Straw man Conventional Wisdom Borrow from thy neighbor Soft modeling Consensus Wisdom of the crowds Delphi Registries, police, SHC, drug treatment, unions, workplace Discrepancies Place, RAP, ethnography Unique event multiplier Truncated Poisson Multipliers, multiple multipliers Multiple sample recapture Capture-recapture Network scale up Population-based survey Census Nomination counting Unique object multiplier Mapping, key informants, observation counting Data collected directly from PWID for size estimation purposes only Source: University of California, San Francisco RDS – Sequential Size

Cost Scientific rigor Straw man Conventional Wisdom Borrow from thy neighbor Soft modeling Consensus Wisdom of the crowds Delphi Registries, police, SHC, drug treatment, unions, workplace Discrepancies Place, RAP, ethnography Unique event multiplier Truncated Poisson Multipliers, multiple multipliers Multiple sample recapture Capture-recapture Network scale up Population-based survey Census Nomination counting Unique object multiplier Mapping, key informants, observation counting Data collected from general population (DHS, AIDS Indicator Survey) Source: University of California, San Francisco RDS – Sequential Size

Cost Scientific rigor Straw man Conventional Wisdom Borrow from thy neighbor Soft modeling Consensus Wisdom of the crowd Delphi Registries, police, SHC, drug treatment, unions, workplace Discrepancies Place, RAP, ethnography Unique event multiplier Truncated Poisson Multipliers, multiple multipliers Multiple sample recapture Capture-recapture Network scale up Population-based survey Census Nomination counting Unique object multiplier Mapping, key informants, observation counting Data from PWID collected for other purposes (IBBSS, registries, service data) Source: University of California, San Francisco RDS – Sequential Size

IBBSS integration Size estimation methods increasingly being integrated worldwide – Leverages existing resources – Adds value to behavioral and seroprevalence data already being collected RDS increasingly used for IBBSS recruitment – “Population-based” estimates Forthcoming RDS software-based estimation (SS- Size) – Provides an estimate using existing IBBSS RDS network data – Has limitations and caveats, should not be used as a sole estimate source

Other considerations Ethical reviews Administrative approvals Safety of research assistants/study team Involvement local drug using community representatives

Conclusion Important to review how researchers arrived at the estimate since many are not grounded in quality data Multiple PSE methods should be used PWID size estimation should be a part of routine surveillance Stakeholder consensus on estimate ranges critical

Acknowledgement Scott Geibel (Population Council) Henry Fisher Raymond (UCSF) Abu Abdul-Quader (CDC) Pandu Harimurti (The World Bank) Riku Lehtovuori (UNODC)

GROUP DISCUSSION

Country A Epidemic concentrated in PWID (account for ~90% of HIV transmission) HIV prevalence in PWID: 15-30% (up to 87% in one city) Civil unrest has hindered PSE of PWID PSE (2,000) based on government registration of drug users † (0.05% of population nationally) RDS-based IBBSS was conducted in 2013 in one city; no PSE No IBBSS planned for near future † Registries based on treatment registers to arrest counts.

Country B Epidemic concentrated in PWID, FSWs and their clients; PWID HIV prevalence: 7% PSE available for 25 out of the country’s 28 states National PSE (177,000; 0.02% of population) obtained through: – District-level mapping/enumeration at hotspots – Data updated regularly (by NGOs that implement targeted intervention at hotspots (WOTC with PWID and gatekeepers at hotspots) IBBSS conducted every 2-3 years; currently being conducted but no PSE

Country C HIV epidemic concentrated in PWID (account for ~60% of HIV transmission) Prevalence of PWID: 100, ,000 (~0.9% of population) Has extensive epi-behavioral data, including PSE; regulated by government Latest published PSE available from 2010/11 – Methods and quality varies across region – National PSE obtained from summing regional results Current/Upcoming activities: – 2014 RDS-based IBBSS with service multiplier (6 sites); 7 multipliers being used – Some regions will include cap-recap with independent databases – NSUM (2012/13)

Country D Epidemic concentrated (PWID, MSM, FSW/clients) – PWID HIV prevalence: 18% Prevalence of PWID – Range: 60, ,000 – Stakeholder consensus (2002): 75,000 – Estimate based on 0.5% of the population being PWID; this may be based on registration of drug users IBBSS completed in May 2014; includes PSE using service and unique object multiplier – Conducted in 16 sites (14 are in high opium-growing states and two are major urban centers) – Injection drug use is believed to be occurring outside of these sites as well

Questions for Group Work What are the potential problems/biases with the current estimate? What kind of opportunity can you identify for improving the estimate? What are possible next steps?