Survey of Tobacco Retail Outlets in 50 United States, DC, and Territories Lily Trofimovich, Karol Krotki RTI International RTI International is a trade name of Research Triangle Institute

2 Outline I. Introduction 1. Synar Amendment and Regulation, Key Players, GAO Report 2. Synar Survey II. Methodology of the Survey 1. Sampling Design 2. Sample sizes and precision 3. Inspector Characteristics (effect of gender and age on the survey results) III. Coverage Study 1. Introduction 2. Guidelines 3. States coverage studies: Results and Challenges IV. Results 1. RVRs and SE 2. Trends V. Conclusions

3 THE SYNAR AMENDMENT In July 1992 Congress enacted the Alcohol, Drug Abuse, and Mental Health Administration Reorganization Act, which included the Synar Amendment aimed at decreasing minors access to tobacco products Synar Amendment: Section 1926 of Title XIX of the Federal Public Health Service Act Sponsored by Mike Synar ( ) – late Representative (D) of Oklahoma It required states to pass and enforce laws that prohibit the sales of tobacco to individuals 18 years of age or younger

4 THE SYNAR AMENDMENT – cont. To determine compliance, the Amendment requires each State to: Enact State youth tobacco access control law Enforce State law Conduct annual random, unannounced inspections of retail tobacco outlets Report these findings to the Human Health and Services Secretary

5 THE SYNAR AMENDMENT – cont. Possible penalty for noncompliance with the Amendment: Potential loss of a portion of SAPT Block Grant funding.

6 Players The States Center for Substance Abuse Prevention (CSAP)/Substance Abuse and Mental Health Services Administration (SAMHSA) RTI provides technical assistance (statistical consulting) to CSAP: Verifying States results Evaluating sampling designs and coverage studies Other

7 THE SYNAR REGULATION January 1996: SAMHSA issued the final Synar Regulation to provide guidance to States on implementing the Synar Amendment.

8 General Accounting Office (GAO) Review GAO examined implementation of the Synar Amendment in 2000 in two key aspects: factors that can affect the quality and comparability of the retailer violation rates reported by the States whether the States are seeking penalties from retailers as part of their enforcement strategies GAO made 3 recommendations to improve Synar implementation and oversight: 1. Help States improve quality and comprehensiveness of tobacco outlet lists 2. Develop a more standardized inspection protocol that promotes more uniform implementation across States and that better reflects research results 3. Ensure that RVRs exclude invalid inspections

9 SAMHSA Actions to Improve Synar implementation and oversight 1. Help States improve quality and comprehensiveness of tobacco outlet lists by: Assisting States assess the accuracy and completeness of lists through site visits Requiring coverage studies every 2 years for States using list frames Develop new guidance document on conducting coverage survey (2005)

10 SAMHSA Actions to Improve Synar implementation and oversight 2. Develop a more standardized inspection protocol that promotes more uniform implementation across States and that better reflects research results by: Recommending that all States use youth inspectors aged 15 and 16 and monitoring age and gender breakdowns each year as reported in the Annual Synar Report Requiring States use a standard format for reporting inspection protocol activities

11 SAMHSA Actions to Improve Synar implementation and oversight 3. Ensure that RVRs exclude invalid inspections by: Instituting review protocols that require States to exclude inspections that do not include age or gender of minor inspectors or outcome of inspection Developing a Synar Survey Estimation System (SSES) to assist States estimate and report their annual RVRs (FYI: In FFY07, 48 States utilized the SSES)

12 Who is Required to Comply with the Synar Regulation? All 50 States, DC and 8 Territories: American Samoa Commonwealth of the Northern Mariana Islands Federated States of Micronesia Guam Marshall Islands Palau Puerto Rico US Virgin Islands

13 Objective of the Synar Survey To determine the retailer violation rate (RVR) for each State based on random, unannounced inspections of a sample of tobacco outlets accessible to youth. The sample of outlets inspected must be representative of the geographic distribution of tobacco outlets in the State. The RVR obtained over a number of years will be used to assess the States progress towards achieving the overall Synar goal of RVR<=20%

14 Sampling implementation steps Develop a sampling frame (which includes over-the counter and vending machines and includes at a minimum, 80% of the tobacco outlets in the state) and asses its quality (COVERAGE STUDY) Determine appropriate sample size for Synar survey to meet SAMHSAs precision requirement of having an RVR with no more than 3% margin of error with 95% confidence Select a random sample of outlets that reflects the geographic distribution of outlets accessible to youth Implement and monitor the Synar Survey within the federal fiscal year (FFY) Analyze the results of Synar Survey Report results to SAMHSA before January 1 of the same fiscal year

15 Eligibility Requirements An over-the-counter or vending machine outlet is eligible for the Synar Survey if it sells tobacco products and is accessible to youth under age 18. Ineligible tobacco outlets are bars, taverns, or other adult-only clubs that have enforced minimum age restriction for entry of 18 or older.

16 Sampling Frame Considerations Type of Frame List Frame (used by the majority of states) Business List License List Area Frame (used by two states: MA and IL) List – Assisted Area Frame (Hybrid Frame) (implemented by two states) Accuracy/Coverage

17 If SSES is used, Stratified Simple Random Sample with and without FPC (for census, simple or systematic random sample) Stratified Cluster Sample with and without FPC Synar Survey Design Options

18 Sample Size Determination Three sample size calculations are performed based on previous years survey Effective sample size Target sample size Original sample size

19 Effective Sample Size Effective sample size: where P is the previous years RVR s.e is the standard error of the estimate for 3% margin of error for one-sided confidence interval ( for one-sided) N is the total number of outlets in the sampling frame.

20 Target Sample Size Target Sample Size = Effective Sample Size * Design Effect Where design effect is the ratio of the variance under the chosen design over the variance under the SRS design The design effect is taken either from the previous years survey (SSES calculates the design effect) or hypothesized value is used

21 Original Sample Size The original sample size is determined by: where s is a safety margin (10% is recommended), rl is the expected eligibility rate, and rc is the expected completion rate.

22 Analysis of the inspection results Estimation of three essential items Retailer violation rate (RVR = number of violations over number of eligible completed inspections) Variance or standard error of RVR estimate 95% confidence interval Analysis of Survey Results

23 Analysis of the Inspection Results (continued) Estimation of other required items Coverage rate of the survey Accuracy rate of the list frame if appropriate Completion rate Analysis of Survey Results

24 Coverage Study - Background States can choose List frame Area frame Hybrid (lists within selected PSUs) HOWEVER, IF LIST FRAME IS ADOPTED, STATE MUST PROVIDE EVIDENCE FOR QUALITY OF LIST, ESPECIALLY COMPLETENESS, THAT IS, COVERAGE.

25 Characteristics of a List Frame Coverage - Crucial Accuracy – Less important A low coverage rate is a potential source of bias, because the unlisted outlets might be different from outlets on the list with respect to their likelihood of selling tobacco to youth.

26 SAMHSA Requirements Synar sampling frames must conform to the minimum coverage requirement of 80%. A frame coverage study should be repeated every 3 years. If a State provides compelling evidence to SAMHSA that the coverage rate is +90% and the list frame used is comprehensive and stable, the State may be allowed to implement the coverage study at 5-year intervals.

27 Coverage Study Guidelines Area frame - list of easily identifiable and manageable geographically designed areas that represent the full geographic extent of the State, without gaps or overlaps SAMHSA recommends using census tracts (ZIP Codes and census blocks/block groups could also be used) Rule of thumb: Area should contain 7 20 outlets on average It is important to select a coverage study sample and implement the sample as close as possible to the time when the Synar survey is conducted, because the outlet population keeps changing over time

28 Sample Design for a Coverage Study – Cont. To select a sample of areas from the area frame, a State must decide on the sample design: Design can range from SRS to a more complex design (stratified or a multi-stage design) depending on a States specific situations. Stratification always improves the quality of the design.

29 Corrective Actions for Low Coverage Improve frame to reach the minimum coverage rate prior to drawing the Synar sample Use additional data source in multi-frame mode Use area frame for the Synar sample itself

30 State-Level Statistics

31 State-Level Statistics

32 National Weighted* Average Retailer Violation Rate, FY 97–05 *Weighted by State population Note: All figures include data from the 50 States and the District of Columbia.

33 States are Achieving Very Low Retail Violation Rates

34 Coverage Rates as of FFY07 <80% - 6 States/Territories [80-90)% - 9 States/Territories [90-99)% - 20 States/Territories % - 10 States/Territories No results – 14 States/Territories

35 Conclusions/Next Steps The National Synar Retailer Violation Rate has significantly dropped since FY97; almost all States have achieved the 20% target Most States are verifying the coverage of the frames Most States implement efficient designs and analyze data using SSES Next Steps: Accuracy and Coverage of Frames should be assessed by all states with list frames Correct and efficient implementation and reporting of the survey

Acknowledgements Mel Tremper, Jennifer Wagner, Patty Martin JBS International RTI International is a trade name of Research Triangle Institute