Address-Based Sampling

Address-Based Sampling
The AAPOR Report on Address-Based Sampling Rachel Harter, Chair Rachel

Overview of Session Introduction and Background
Scripted Q&A with Panelists Open Q&A Floor Discussion Rachel

What is ABS? In broad terms, ABS (address-based sampling) refers to survey methodologies for samples selected from address frames; in the US, ABS frames are usually based, in part, on the US Postal Service mail delivery databases. Rachel

Why an ABS Task Force? Increased opportunities for ABS surveys
Importance of familiarity with frame and sample properties Evolving methodologies and complex tradeoffs Encourage consistent disclosure of ABS designs Rachel ABS is increasingly used for in-person, mail, and mixed mode surveys. ABS is an alternative to random digit dialing for household surveys. ABS requires knowledge of the frame and sample properties. Methodologies are evolving, and the tradeoffs may be confusing. Responsible reporting of address-based surveys should disclose aspects of the design that currently are not reported consistently.

Charge of the ABS Task Force
This AAPOR task force will supplement the existing AAPOR guidelines for ABS issues, consistent with existing standards for scientific surveys, with focus on: Standardization/clarification of ABS terminology Recommendations for: Frame-building and enhancements Sample selection possibilities Weighting adjustments Response rate calculations and reporting Rachel This AAPOR task force will supplement the existing AAPOR guidelines for address-based sampling issues, consistent with existing standards for scientific surveys. In particular: Standardization/clarification of ABS terminology. Recommendations for frame-building. Recommendations on frame enhancements. Recommendations for weighting and response rate calculations. Recommendations for reporting.

ABS Task Force Members Rachel Harter, RTI International, Chair
Michael P. Battaglia, Battaglia Consulting Group, LLC Trent D. Buskirk, Marketing Systems Group Don A. Dillman, Washington State University Ned English, NORC at the University of Chicago Mansour Fahimi, GfK Custom Research, LLC Martin R. Frankel, Baruch College Timothy Kennel, U.S. Census Bureau Joseph P. McMichael, RTI International Cameron Brook McPhee, American Institutes for Research Jill Montaquila DeMatteis, Westat Tracie Yancey, Nielsen Company Andrew L. Zukerberg, National Center for Education Statistics Rachel

Expert Consultants Anne Connelly, Cigna, formerly of Valassis, Inc.
Philip Faulstich, Valassis, Inc. David Malarek, Marketing Systems Group Missy Mosher, SSI Linda Piekarski, SSI Bonnie Shook-Sa, RTI International Rachel

Acknowledgements The task force gratefully acknowledges the helpful comments from J. Michael Brick, the AAPOR Standards Committee, and their designees. The task force also gratefully acknowledges the editorial support provided by RTI International and meeting support provided by AAPOR Headquarters. Rachel

Disclaimer The views expressed in the report and this presentation are those of the authors and do not represent the views of their agencies and employers. Rachel

The Process Build the team, with AAPOR Council approval
Hold series of informational sessions with expert consultants Prepare outline of report Assign sub-teams to write report sections Compile, review, and revise – ITERATE Obtain comments from AAPOR Standards Committee Solicit feedback from expert consultants based on early draft Revise, compile, review – ITERATE Obtain comments from outside expert and revise Obtain approval from AAPOR Standards Committee Obtain final approval from AAPOR Executive Council Rachel Build the team, with AAPOR approval. Hold series of informational sessions with expert consultants. Prepare outline of report. Assign sub-teams to write report sections. Compile, review, and revise. (Iterate) Obtain comments from AAPOR Standards Committee and expert consultants based on early draft. Revise, compile, review. (Iterate) Obtain comments from outside expert and revise. Obtain approval from AAPOR Standards Committee. Obtain final approval from AAPOR Executive Council.

The ABS Report port_1_7_16_CLEAN-COPY-FINAL.pdf Introduction Frame Creation Auxiliary Variables Designing and Implementing ABS Surveys Eligibility, Response Rates, and Weights Reporting Guidelines Quality and Cost Issues for ABS Samples Summary of the Current State of ABS References Appendix - Definitions Rachel

Today’s Panelists Jill Montaquila DeMatteis, Westat
Mansour Fahimi, GfK Custom Research, LLC Cameron Brook McPhee, American Institutes for Research Andrew L. Zukerberg, National Center for Education Statistics Jill Montaquila DeMatteis is an Associate Director in the Statistics Group and senior statistician at Westat, and Research Associate Professor in the Joint Program for Survey Methodology at the University of Maryland and University of Michigan. She is a Fellow of the American Statistical Association and an elected member of the International Statistical Institute. Jill’s research interests include nonresponse error, telephone survey methodology, and address based sampling. Mansour Fahimi of GfK Custom Research has 25 years of experience in survey and market research methods and applications, currently working on design and administration of complex surveys in the US and abroad. Mansour has worked on refinements to improve the efficiency of dual-frame RDD and ABS methodologies by minimizing their undercoverage and inconsistent practices for sampling and weighting applications. In recent years, he has been working on non-probability sampling alternatives to develop selection and weighting techniques that can improve the reliability of inferential statistics from such samples Cameron McPhee is a senior survey methodologist at the American Institutes for Research. Currently she leads AIR’s work on the National Household Education Surveys program in partnership with the National Center for Education Statistics. She is responsible for designing the sampling, mailing, and weighting plan for the 2016 and 2017 NHES data collections and leads the team developing mail and web-based survey instruments. Her expertise includes survey methodology, research design, data analysis, data file construction, and statistical software programming. Andy Zukerberg is Chief of the Cross- Sectional Surveys Branch at the National Center for Education Statistics. He was the Project Officer overseeing the conversion of a national household survey from an RDD frame and telephone collection to an ABS frame with data collection by mail starting in 2008.

Panel Format Scripted Q&A: Open to questions from the audience
Panelists will respond to prepared questions Convey the main points of the report Open to questions from the audience Handouts will be available at the end of the session Rachel Get somebody to put out handouts before end of session.

Why has ABS emerged? Mansour

Why Has ABS Emerged? Evolving coverage problems of telephone-based alternatives Complexities and inconsistencies of dual-frame RDD Eroding rates of response to telephone and single modes of contact Increasing costs and inefficiency of refusal conversions by phone alone Improvements in databases of household addresses ABS frame enhancement possibilities Increasing pressure for cost containments Mansour

When is ABS appropriate?
Jill

When is ABS Appropriate?
Two primary uses: Housing unit frames Alternative to RDD for mail only, mail-to-web, and multi-mode To address coverage, response rate, and cost issues Best frames for national US household surveys Not appropriate for all survey situations: Learning curve/infrastructure Length of field period Jill

What are the sources of ABS frames, and what is known about their coverage and quality?
Andy

ABS Sources & Coverage Primary vendors maintain and lease address lists for frame creation Secondary vendors subset address lists and select samples of addresses Frames have high coverage of households but can vary by address type: Rural addresses Some Native American reservations Drop points Coverage is improving as addresses are converted to city-style Most vendors supplement their coverage with data from other sources Andy Many address files used as sampling frames were developed primarily for mass marketing. Two types of vendors have been used in the survey industry. Primary vendors generally deal directly with USPS and must have a largely complete address frame from their own sources in the first place. These vendors often have a CDS license from the Post Office. Secondary vendors generally do not deal directly with USPS and may enhance a frame from a primary vendor with additional information for drawing samples. Some vendors do not use USPS sources at all. Some prefer to use USPS sources to maintain consistency with “mailable” addresses, but they must meet certain requirements to qualify for either the Computerized Delivery Sequence file updates or the DSF2 service that flags special types of addresses in vendor files. - In order to obtain the USPS addresses from the CDS the vendor must ‘own’ a zip code - This means they have between 90 percent and 110 percent of the addresses in the zip code on their file within the group (residential, business, rural, etc.) that they want to get updates for. - DSF has less stringent requirements but does not provide missing addresses Vendor address frames may have additional records and variables from the vendor’s own sources. Few vendors lease frames for the entire U.S., and the cost may be prohibitive unless a survey organization selects many samples for its own use. These vendors can summarize frame information to inform selections, or support two-phase sampling when subdomain sampling is required. Some secondary vendors may be more oriented to survey needs; certainly the smaller survey organizations are more likely to go this route. Go on to next slide

Potential Issues With ABS Frame Data
Over coverage and under coverage issues: Matching addresses between sources leads to over/under coverage Household members may receive mail at more than one address Some addresses are missing entirely from the files Misclassification of addresses as business or residential Important to have lists that are updated regularly Andy Over coverage can come from a variety of sources including if the same address is listed multiple times in the frame. This can happen because aspects of the address (e.g., unit number, street name, zip code can change) and the vendor pulls multiple files so the address may appear in one format on one file and another on a different file. Under coverage can occur because certain types of addresses (e.g. certain rural routes) are not included on the USPS file or multiunit placeholders (and drop points) – where the USPS delivers mail for multiunit buildings to one address (and the residents divide the mail themselves. Misclassification rates are generally low. For example in 2010 Census 7% of the 1.5 million business curb line delivery points and 2.3% of the 5.5 million business other delivery points were occupied housing units. Simplified addresses are missing entirely from USPS files. These are addresses that the only delivery information is city, state and zip code as well as the number of customers receiving mail at that zip code. To ensure good coverage is it is important to work with a vendor that regularly updates their frame. Go on to next slide.

Determining the Quality of ABS Frames
Somewhat difficult due to vendors’ proprietary processes Use checklist to screen vendors (hand out) Field-enumerated Lists vs. ABS Frames: Both subject to error due to omission and duplicates Can be used to complement each other: Rural routes can be hard to locate physically from an ABS frames ABS frames can be used to identify gaps in field enumerated lists Andy Many studies have been conducted to look at coverage of USPS driven frames. A consistent finding across studies is that the databases suffer from undercoverage of rural areas, in large part because of the type of addresses available in those areas (Dohrmann et al. 2006; Dohrmann et al. 2007; O’Muircheartaigh et al. 2007). City-style addresses consist of a house number, street name, state, and ZIP Code. Non–city-style addresses, common in rural areas, consist of box numbers rather than house numbers and delivery route numbers rather than street names; they include post office, rural route, and highway contract boxes. The correspondence between a mailbox and a housing unit are not necessarily clear in rural areas, especially if several mailboxes are clustered at the end of a lane leading to multiple housing units. For in-person surveys, interviewers must be able to locate each selected address; thus, only city-style addresses are useful (Eckman and English 2012a). We expect coverage to improve in rural areas as addresses convert to city style for 911 services. In urban and suburban areas, these frames are thought to perform as well as traditional listing methods. It is important to remember that traditional listing methods suffer from error as well. For example, listers sometimes make mistakes (Eckman 2010). Field methods to correct for errors or omission are also error prone (Eckman and Kreuter 2011).

What are key decisions to be made when using ABS?
Andy

Key Decisions When Using ABS
Delivery types to include on frame: Residential P.O. Boxes Vacant units Seasonal dwellings Drop points Supplemental variables: Phone number Append household characteristics from commercial sources Append Census geodemographics via geocoding Andy Inclusion of P.O. Boxes – can add to overcoverage if household receives mail at multiple locations. Can specify – only way to get mail (OWGM) to avoid this issue. Also it is important to note that OWGM PO boxes and regular PO boxes can also have a vacant flag and these are generally excluded from samples. Inclusion of vacant addresses (can be undercoverage if not occupied but increase cost and non response (in mail survey) if not occupied Educational and seasonal addresses – occupied certain times of the year – similar issues as vacant addresses Drop points ( where mail is delivered to a single location but multiple households live at the address) – an issue for mail surveys – some vendors use auxiliary data to try and expand these addresses. In person surveys will need an approach to do on site sampling of units.

How do I get a sample of households from addresses?
Jill

Sampling Households from Addresses
ABS provides a sample of delivery points, not households: Single address associated with multiple households (drop point) Single household associated with multiple addresses Both P.O. box and physical (city-style) address Vacation homes Handling households with multiple addresses: De-duplicate: Only operating “OWGM” P.O. boxes Eliminate “seasonal/educational” addresses from frame Screen out households at their seasonal/educational addresses Include questions about whether household can be reached at multiple addresses Jill

What methods can be used to sample individuals within households using ABS?
Cameron

Within-Household Subsampling
Single-phase design: All-adults approach Any-adult approach Next-birthday method Two-phase design: Screener to enumerate eligible persons Returned screener processed, eligible person randomly sampled Survey is administered in second phase Cameron Single-phase design (Battaglia et al. 2008) All-adults approach No selection All adults are asked to complete the survey Requires multiple survey forms; cost and resource implications Any-adult approach Household determines (purposively) which adult should respond Not a random (or even quasi-random) approach Next-birthday method Adult with next birthday is selected Quasi-random Battaglia et al. (2008) found All 3 methods resulted in comparable household-level response rates All-adults method resulted in lower person-level response rates Any-adult method yielded a set of adults with demographics less comparable to population distributions With next-birthday method, in HHs with 2+ adults, incorrect adult completed the questionnaire between 31% and 60% of the time Two-phase design (Brick, Williams, and Montaquila 2011; Montaquila et al. 2013) Description of approach Screener used to enumerate eligible persons within HH Returned screener is processed and an eligible person is randomly selected In second phase, survey is administered to the randomly selected person Benefits of two-phase approach Assurance that sampling is undertaken accurately Ability to personalize the questionnaire mailing based on information obtained on the screener form Easing respondents into the response process by starting with the shorter screener form before sending the larger survey questionnaire Drawbacks of this approach No guarantee that the sampled individual will, in fact, be the person who completes the questionnaire. Additional labor is needed to process the returned screener forms and conduct the within-household selection. The two phases can lead to the need for longer survey administration windows Second phase of data collection allows for a second opportunity for nonresponse But benefits listed above may have a positive effect on response rates At this point, for a given survey, it is unclear whether the two-phase approach or a single-phase approach is likely to yield higher response rates

What specific issues need to be considered when using ABS for local surveys?
Cameron

ABS Issues for Local Surveys
Prevalence of specific types of addresses: Drop points and educational addresses – variation from area to area Substantial group quarters population P.O. box addresses – will geocode based on location of post office Addressing potential effects of geocoding errors: Cast a wider net Include coverage enhancement methods Cameron ABS can be a good option for local surveys as the address frame can be filtered geographically. For the most part, the same issues apply locally as nationally. There are several things to keep in mind when using ABS for a local survey. If high proportion of drop points, educational addresses, or group quarters, will need to account for this in sampling PO boxes likely to geocode based on location of the post office, not necessarily the HH, which becomes particularly important if you are stratifying by local geography (e.g., tracks, blocks) Addressing geocoding problems: Cast a wider net than initially intended and verify that the person lives within the survey area Include coverage enhancement methods (such as linking methods such as Address Coverage Enhancement (ACE), but almost all require in-person listing component.

What is currently known about mixing modes in ABS studies?
Jill

Mixing Modes in ABS Studies
Mixed-mode study: Contact by one mode, request response by different mode Contact attempted by more than one mode Offer more than one mode response option (sequential or simultaneous) Popular approach: Contact by mail, try to elicit web response: Might offer mail/phone response as option Mail/web choice does not improve response Alternative strategy: Web-push Use of phone with mail and web (Finamore and Dillman 2013): Web-first/mail-first/phone-first Final response rates, respondent characteristics nearly the same Most respondents answered by the original mode Web-first least expensive; phone-first most expensive Jill Mixed-mode study: Contact made by one mode and people are requested to respond by a different mode Contact attempted by more than one mode Respondents offered more than one mode response option (either sequentially or simultaneously) Popular approach: Contact by mail, try to elicit web response Might offer mail/phone response as option to those unable/unwilling to respond by web Providing households a choice of whether to respond by mail versus Internet does not improve response rates, and may lower response rates (Medway and Fulton 2012). This may occur because offering choice encourages potential respondents to defer a decision (Tversky and Shafir 1992), or it encourages them to think of other options such as not responding at all (Schwartz 2004). people are far more likely to respond by mail (Smyth et al. 2010). Alternative strategy: Mail contact asking for response over the Internet (“Web-push”) Late in data collection, offer the possibility of responding by mail Washington State study: Offering web first, followed later by mail, decreased response rates by about 8 percentage points relative to mail-only (Dillman et al. 2014) Mail respondents different demographically than Web respondents (Messer and Dillman 2011) Mail-only treatment group similar to Web-plus-mail respondents in Web-push group Mail-push method (withholding web until late) was not effective Use of phone with mail and Web (Finamore and Dillman 2013) Three groups, with all 3 modes used for each group Web-first Mail-first Phone-first Final response rates nearly the same (75% to 77%) Respondent characteristics differed very little Most respondents answered by the original mode Per-respondent costs: Web-first: $48 Mail-first: $66 Phone-first: $75 Go on to next slide

Modes of Data Collection with ABS
Contact mode vs. data collection (response) mode: Mail, in-person via addresses (non-city-style issue) Matched phone numbers (non-matches and bad matches issues): Might use phone just for reminder calls, IVRs Addresses with phone number matches have higher response rates regardless of contact mode Data collection mode: In-person Mail Web Phone IVR Mixed-mode Jill

What are methods of selecting a stratified sample from an ABS frame?
Cameron

Stratification with ABS
Potential stratification variables: Postal variables Appended auxiliary variables from commercial sources Cautions! Appended geodemographic variables from the Census Stratification options: Stratify by characteristics available for all addresses on the frame Use two-phase (double) sampling Cameron Potential stratification variables Postal variables such as Address type (city-style, PO box, rural route, highway contract, vacancy, drop point) Multi-unit structure indicator State Auxiliary data that may be appended to ABS frame or sample Geographic-level auxiliary variables Obtained from the most recent decennial census, the ACS, or commercial data sources E.g., median household income, proportion of the adult population that graduated from college, and the race/ethnicity distribution of the population Household/person characteristics (matched to address) Typically appended from commercial sources E.g., tenure status, presence of children in the household, education of the head of the household. Not available for every sampling unit Caution using appended data Careful understanding of matching methods, completeness, and coverage of supplemental files is essential Ask vendor about match rates, accuracy of the variables In general, it is not recommended to use appended auxiliary data to filter the frame or sample Doing so may result in serious undercoverage errors Far better to stratify by auxiliary variables than to filter the frame or sample based on matched auxiliary variables Stratification options Stratify by characteristics available for all addresses on the frame (address type, census tract characteristics, etc.) Use two-phase (double) sampling Append auxiliary variables to phase 1 sample At second phase, stratify on these and subsample Need approach for dealing with missing data for auxiliary variables Might include as a separate stratum, because these addresses tend to differ demographically and socioeconomically from those with available data (Buskirk et al. 2014).

What are the key considerations for determining eligibility of addresses in ABS studies?
Jill

Determining Eligibility of Addresses
Need to consider both addresses and households residing at address Regardless of mode, the sampled unit is the address: Conflicting “signals” about the eligibility of an address is not uncommon Consider the source of the signals and their utility in assigning case dispositions Must have rules for reconciling conflicting dispositions Address confirmation question will be necessary for determination of eligibility Surveys with household- or person-level eligibility requirements: Unreturned mail questionnaire could be an indication of nonresponse or ineligibility May receive conflicting information about the household itself Eligibility criteria often need to be based on a single point in time Jill Often, for ABS samples, address eligibility is defined as an occupied residential address For single-phase studies that use mail as the only mode of data collection, and for multiphase studies that use mail as the only mode for the first phase of data collection, the only information available to inform whether the address corresponds to a household address is (1) whether a questionnaire was returned by the household and (2) whether a postmaster return (or Post Office Return) was received. For multi-mode studies (or studies in which mail is not used), researchers must be cognizant of the source of “signals” used to determine a sampled unit’s eligibility (e.g., ring-no-answer on appended phone number or bounce back on appended should not be used as indicators of address eligibility). Other important considerations include: timeline for receiving returns (from both households and postmaster returns) rules for handling questionnaires being returned from more than one mailing rules for handling seemingly conflicting information about whether the address corresponds to a household Need rules for reconciling conflicting dispositions from multiple modes. For studies using modes other than mail, the other modes may provide additional information about whether the address corresponds to a household. In these circumstances, it is very important to include an address confirmation in the questionnaire. If the respondent does not confirm the address, and the actual address (corresponding to the household) is sufficiently different from the sampled address, then the sampled case should be treated as if no contact was made (i.e., the completed questionnaire should not be processed, and the case disposition should be reassigned).

What are the special considerations for computing response rates in ABS studies?
Jill Often, for ABS samples, address eligibility is defined as an occupied residential address For single-phase studies that use mail as the only mode of data collection, and for multiphase studies that use mail as the only mode for the first phase of data collection, the only information available to inform whether the address corresponds to a household address is (1) whether a questionnaire was returned by the household and (2) whether a postmaster return (or Post Office Return) was received. For multi-mode studies (or studies in which mail is not used), researchers must be cognizant of the source of “signals” used to determine a sampled unit’s eligibility (e.g., ring-no-answer on appended phone number or bounce back on appended should not be used as indicators of address eligibility). Other important considerations include: timeline for receiving returns (from both households and postmaster returns) rules for handling questionnaires being returned from more than one mailing rules for handling seemingly conflicting information about whether the address corresponds to a household Need rules for reconciling conflicting dispositions from multiple modes. For studies using modes other than mail, the other modes may provide additional information about whether the address corresponds to a household. In these circumstances, it is very important to include an address confirmation in the questionnaire. If the respondent does not confirm the address, and the actual address (corresponding to the household) is sufficiently different from the sampled address, then the sampled case should be treated as if no contact was made (i.e., the completed questionnaire should not be processed, and the case disposition should be reassigned).

Response Rate Calculations with ABS
Handling the proportion of cases classified as unknown eligibility for good estimation of “e” Traditional methods for estimating e may not be appropriate for ABS surveys Good practice to report multiple response rates (including “best-” and “worst-case” estimates) For subpopulation surveys, multiple estimates of e should be computed for address- and household- or person-level eligibility Jill ABS surveys (excluding, perhaps, in-person modes) often have a large proportion of cases classified as having unknown eligibility (i.e., cases with no returns – complete, refused, or undeliverable). Must have a method for estimating eligibility (e) of unknown cases Traditional methods for estimating e may not be appropriate for ABS surveys, particularly general population surveys of unnamed individuals. Compared to RDD, where increased contact attempts can weed out additional eligible cases, increased contact attempts by mail, for example, may yield increased postmaster returns, suggesting remaining cases are likely to be eligible nonrespondents One suggestion is proportional allocation of eligibility within strata defined by auxiliary variables about the address. Often one may wish to report response rates that compute e using the minimum and maximum allocation as well as a proportional allocation or stratified proportional allocation to present best- and worst-case estimates of nonresponse. For subpopulation surveys, multiple estimates of e should be computed for address- and household- or person-level eligibility

What are the special considerations when calculating weights for an ABS study?
Cameron

Weighting Adjustment Issues
Steps in weighting similar to other list-frame surveys: Calculation of design weights to account for selection probabilities Eligibility adjustments Unit nonresponse adjustments Poststratification to control totals Weight components unique to ABS Designs: Address-level weights Household-level weights Person-level weights Cameron Go on to next slide

Weights Features Unique to ABS
Household-level weights: Addresses with multiple households (drop points) Households with multiple addresses: (PO Boxes and seasonal & educational units) Person-level weights will be needed to account for additional within- household selection Often eligibility and nonresponse adjustments are made at the household and/or person levels Cameron Household-level weights Addresses with multiple households (Drop points) - must account for the probability of a given unit within a drop point being sampled Households with multiple addresses: (PO Boxes, seasonal addresses, educational addresses) – must account for a household’s duplication on the frame in the form of multiple addresses reaching the same household Note that often only OWGM PO Boxes are sampled in order to reduce the likelihood of having households with multiple addresses May exclude these address types or ask a question about the household’s receipt of mail at multiple addresses

What types of variables are available for making nonresponse weighting adjustments for an ABS study?
Cameron

Nonresponse Weighting Adjustment with ABS
Require auxiliary variables available for both respondents and nonrespondents Several commonly used frame (or address-level) variables: Auxiliary variables available for Census geographies or ZIP codes Auxiliary variables that are available for all addresses in the sampling frame Auxiliary variables that are appended after the sample selection Ideal weighting class variables exhibit response rate & survey estimate variations across classes Cameron Walk through bullets, refer back to different frame-level variables from sampling slides Last bullet: In practice it is often easier to find variables related to response than to the survey variables Note that Missingess itself is often predictive of nonresponse

What special aspects should be reported for an ABS study?
Andy

Reporting AAPOR minimum standards for disclosure apply:
Definition of population under study and its geographic location Description of frame and its coverage of the target population (see AAPOR code) Following OMB and AAPOR guidelines: Vendor name Name of frame used if vendor offers different frames Date frame was constructed Source of address frame Methods vendor used to de-duplicate the file Sample design Comprehensive list of address types included Procedures to validate the representativeness of the sample Procedures to identify and resolve duplicate or invalid addresses Andy Replicability is a key component of surveys, The task force has identified recommendations for disclosure of methods when using an ABS frame based on AAPOR and OMB guidelines A clear description of the frame and its coverage of the target population is critical. Specifically, if any subgroup is not covered by the frame. For example if certain geographic areas or types of addresses (like P.O. boxes are excluded) When describing the sampling approach, researchers should discuss the types of addresses included or excluded (e.g. seasonal, vacant, OWGM, business or mixed use). Any methods to identify duplicate or invalid addresses should also be discussed. This would include methods used to assign disposition codes to sample records Since it is likely that an ABS survey will be a probability sample, a discussion of the method used to calculate weights should also be discussed. Response rate calculations should describe how e was calculated.

What are some areas for future research in ABS?
Mansour

Future Research Coverage and options for supplementing coverage
Impact of drop points and methods for working with them Ineligible addresses and postmaster returns Impact of diminishing landline matches Frame updates Auxiliary variables via multi-sourcing Two-phase methodologies ABS for business surveys Cost and quality tradeoffs Mansour Coverage has been studied extensively, but it constantly changes. Furthermore, coverage may not be well understood for local surveys. Methods have been developed to supplement coverage in the field, but more research could be done on how to estimate coverage and how to set thresholds for implementing supplemental methods. Drop points affect mail modes differently than in-person modes. The frequency and nature of updates can affect quality, but tend to be closely guarded vendor secrets. Two-phase methodologies can be used to increase response rates, decrease nonresponse bias, and decrease costs. How best to implement them? Very little is known about ABS for business surveys. Mixed mode surveys need more research on mode effects, best protocols, etc. Very little is known about returned mail and reliability of eligibility status. Cost and quality tradeoffs affect many aspects of ABS and mixed-mode surveys. See handout for more details.

Questions from the Floor
Panelists: Jill Montaquila DeMatteis Mansour Fahimi Cameron Brook McPhee Andrew L. Zukerberg Rachel

The ABS Report s/aapor_report_1_7_16_clean-copy-final.pdf More information: Rachel Harter RTI International Rachel

Address-Based Sampling

Similar presentations

Presentation on theme: "Address-Based Sampling"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Address-Based Sampling

Similar presentations

Presentation on theme: "Address-Based Sampling"— Presentation transcript:

Similar presentations

About project

Feedback