Comprehensive Assessment: A Data Driven Decision Making Process

University of South Carolina Student Life VISION A caring community united for a safer Carolina. MISSION To create a campus-community environment that promotes healthy and safe behaviors among faculty, staff, students, and community organizations.

University of South Carolina Student Life PURPOSE To build capacity and to perform ongoing assessment, planning, implementation, and evaluation of substance abuse prevention strategies, and to make recommendations for policy change GOAL To contribute to the overall reduction of high risk substance use and its consequences at the University of South Carolina

The SPF process enables campus communities to: Plan and develop substance abuse prevention infrastructure Implement effective prevention strategies Environmental approaches and educational initiatives Achieve population level outcomes A Model for Reducing High Risk Drinking Behaviors on Campus: The Strategic Prevention Framework EVALUATION Monitor, evaluate, sustain, and improve or replace those that fail EVALUATION Monitor, evaluate, sustain, and improve or replace those that fail IMPLEMENTATION Implement evidence-based Prevention programs & activities IMPLEMENTATION Implement evidence-based Prevention programs & activities PLANNING Develop a Comprehensive Strategic Plan PLANNING Develop a Comprehensive Strategic Plan ASSESSMENT Profile population needs, resources, & readiness to address needs and gaps ASSESSMENT Profile population needs, resources, & readiness to address needs and gaps CAPACITY Mobilize and/or build capacity to address needs CAPACITY Mobilize and/or build capacity to address needs Sustainability & Cultural Competency

Our Coalition’s Assessment Process Created the Data, Assessment, and Evaluation Subcommittee Review all available data sources Identify High Risk Priority Areas Develop logic models Form subcommittees to address each problem priority area Set goals and objectives for each priority area Identify gaps in research, data, and resources Develop a comprehensive assessment toolkit Develop assessment protocols and data collection tools Collect comprehensive data Identify emerging trends Review environmental strategies to determine how best to address local conditions Set goals and develop a strategic action plan for the year Developed a comprehensive assessment toolkit Developed assessment protocols and data collection tools Collected comprehensive data Identified emerging trends Reviewed environmental strategies to determine how best to address local conditions Set goals and developed a strategic action plan for the year

Assessment Timeline August 2014: Review logic models and strategic plan September 2014: Present strategic plan to coalition October 2014: Implement strategies and monitor November 2014: Implement strategies and monitor December 2014: Implement strategies and monitor January 2015: Implement strategies and monitor February 2015: Review data for strategic planning session March 2015: Identify problems, resources, readiness, and priority areas April 2015: Annual Assessment report due May 2015: Host Annual Strategic Session June 2015: Review logic models July 2015: Develop strategic plans August 2015: Steering committee approves strategic plan Developed a comprehensive assessment toolkit Developed assessment protocols and data collection tools Collected comprehensive data Identified emerging trends Reviewed environmental strategies to determine how best to address local conditions Set goals and developed a strategic action plan for the year

Data, Assessment and Evaluation Subcommittee To inform and guide the coalition in problem and asset identification, strategic planning and implementation, and outcome evaluation Purpose Goal Objective PURPOSE To conduct ongoing, comprehensive community-campus assessment and process/outcome evaluation GOAL To To conduct assessment to determine the extent of: Risk factors associated with the problem Protective factors (assets and resource availability) Community-campus readiness to address the problem and fill needs and service gaps To conduct process and outcome evaluation of prevention strategies OBJECTIVE

Data, Assessment and Evaluation Subcommittee Tasks of the Subcommittee Determine the assessment’s purpose and scope Define the assessment’s goals and objectives Select the approach and methods for collecting data Design and pilot test the collection instruments and procedures Prepare assessment timelines Collect the data Analyze the data Prepare data in a way that describes the root causes and local conditions that contribute to identified community problems Develop a problem statement(s) that our coalition will address on a community wide, environmental level Source: The Fenway Institute

Comprehensive Assessment Three Areas: Problem Identification and Prioritization Resource Identification Readiness for Change

Comprehensive Assessment: #1: Problem Identification and Prioritization The subcommittee has identified many indicators that help to explain student substance use and patterns. Specifically we look at: prevalence of use locations of use drinking patterns consumptions levels student subpopulation(s) most affected types of substances being used/abused average age of use environmental factors, such as availability, accessibility, and cultural norms

Comprehensive Assessment: #1: Problem Identification and Prioritization Enterprise Risk Management Risk Assessment (probability, severity, velocity) Risk Control Assessment Final Risk Score and Ranking

Comprehensive Assessment: #2: Resource Identification Identify past and current organizational and individual resources, and categorize those resources by key community sectors. Identify future resources needed to achieve our desired outcomes Ask the following questions: o Who has the knowledge we need? o Who has the time to assist? o Who has the human resources we need? o Who can hold us accountable for outcomes? o Who can provide financial support? o Who can encourage us and recognize our successes? o Who has helped with this issue in the past? o Who has an interest in our outcome? o Who has decision making power? o Who cares about this issue? o Who has other resources like space, food, giveaways, that we may need?

Comprehensive Assessment: #2: Resource Identification Identify quantitative and qualitative data currently being gathered and analyzed. AlcoholEdu: Freshmen Student Survey CORE: Upperclassmen Student Survey NCHA STIR: Indicated Student Population OSC: Student violations and sanctions related to substance use USCPD: crime data related to substance use Health and Wellness Center: health consequences related to substance use Key Informant Interviews Focus Groups Environmental Scanning

Comprehensive Assessment: #3: Readiness for Change Determine important aspects to consider when assessing readiness to change. Considerations include: Examining multi-levels and multi-facets of campus and the surrounding community Determining political will for change Measuring shared resolve/commitment to implement change Determining if there is a shared belief in our capability to make change Determining if there is a shared value that change needs to occur Assessing the campus community’s ability to meet task and resource demands Concluded that the higher the readiness for change, the more likely change will be initiated, the greater the effort, persistence, and cooperation members will give

Drinking Location Fall 2013 vs. Fall 2014

Problem Identification and Prioritization Priority: Bars and nightclubs Priority: Off campus residences Priority: Other high risk substance use Priority: Community organizing and advocacy

Conduct Strategic Prevention Planning Steering Committee utilized assessment information to: Identify and prioritize the problem(s) Develop logic models to identify root causes and contributing local factors Coalition members then formed subcommittees to: Create SMART Goals Select the intended outcome(s) and measure(s) Identify intermediate variables and measures Identify strategies for changing intermediate variables

USC Problem Priority : Prevalent Drinking in Off Campus Residences Root CausesContributing Local Factors Perceived Risk Social Access & Availability Social Norms Prevalent Drinking Off Campus Residences Problem Social Hosting: Students over 21 provide alcohol to minors at parties/get togethers Social Hosting: Students over 21 are not award of social hosting laws Property management rules vary and do not get enforced “College” norm to drink alcohol Multiple jurisdictions and enforcement varies Students perceive low risk of harm in own apartment Students perceive low risk of getting caught in own apartment Underage drinking is acceptable Driving after Drinking Sexual Assaults Public Nuisance Property Damage Negative Consequences Inconsistent Enforcement

Logic Model: USC Problem Priority : Prevalent Drinking at Football Game Days Root CausesContributing Local Factors Insufficient Laws and Policies Social Access & Availability Social Norms Prevalent, High Risk Drinking On football Game Days Problem Tailgate Environment, social hosting, alcohol available with little monitoring Excessive amounts of alcohol and time periods available for drinking Large crowds vs. enforcement resources Perception of “no rules, no consequences” on Game Days Identification of illegal acts difficult to manage in large crowds Policies on tailgating vary due to mix of university/privately owned properties Tailgating restrictions are inadequate Alcohol central to tailgating culture Underage Drinking Binge Drinking Driving after Drinking Black Outs Alcohol Poisoning Injuries Public Intoxication Property Damage Negative Consequences Inconsistent Enforcement

USC Problem Priority : High Risk Drinking in Bars Root CausesContributing Local Factors Promotion Access & Availability Social Norms High Risk Drinking in Bars Problem Inconsistent Service Practices (ID checking, over-service) Extended Drink Hours Perceived safety: Safety risks are low with friends Underage drinking is acceptable Perceived getting caught: Easy to get in bars using fake ID and drink underage Social Networking and Social Media Advertising Inconsistent enforcement of pricing, drink specials, and happy hour laws. Drinking to intoxication is acceptable Binge Drinking Underage Drinking Black Outs Alcohol Poisoning Injuries Sexual Assaults Property Damage Arrests Negative Consequences

Bar Sales and Services Subcommittee Long Term Goal By March 2016, reduce high risk drinking at bars by 10%. Objectives Reduce access and availability, address alcohol promotion through social media, increase perception of risk (harm and getting caught), and change social norms. Strategies/Activities Collect data on sales and service practices, promotion and advertising through social media, in order to develop and implement a responsibility toolkit and provide merchant education.

Off Campus Safety Committee Long Term Goal By March 2016, reduce high risk drinking at off campus properties by 10%. Objectives Decrease availability and accessibility, increase perception of harm and getting caught, consistently enforce policies and laws, improve referrals, and change social norms. Strategies/Activities Collect data from off property managers, courtesy officers, and students to develop an incentive package, offer education and programming, and improve collaboration with properties, law enforcement and university.

Other High Risk Substance Use Subcommittee Long Term Goal By March 2016, reduce high risk drinking at off campus properties by 10%. Objectives Decrease availability and accessibility, increase perception of harm and getting caught, consistently enforce policies and laws, improve referrals, and change social norms. Strategies/Activities Collect data from off property managers, courtesy officers, and students to develop an incentive package, offer education and programming, and improve collaboration with properties, law enforcement and university.

Community Organizing and Advocacy Subcommittee Long Term Goal By March 2016, reduce high risk drinking at off campus properties by 10%. Objectives Decrease availability and accessibility, increase perception of harm and getting caught, consistently enforce policies and laws, improve referrals, and change social norms. Strategies/Activities Collect data from off property managers, courtesy officers, and students to develop an incentive package, offer education and programming, and improve collaboration with properties, law enforcement and university.

Game Day Task Force Long Term Goal By March 2016, reduce prevalence of drinking on game day by 5%. Objectives Reduce social access and availability, consistently enforce policies and laws, and change social norms. Strategies Collect data on tailgate policies and student drinking patterns on game day to inform and improve law enforcement and collaborations.

Data, Assessment, and Evaluation Subcommittee Developed the Comprehensive Assessment Tool Kit, a data collection protocol packet for each subcommittee Purpose: to assist the subcommittees with conducting assessment specific to their priority area Assessment Tool Kit included: o brief overview of assessment process o data collection guidelines o variety of data collection methods to consider o general timeline template to assist with planning o tools and resources to use for data collection University of South Carolina Student Life

Subcommittee Action Steps Review Assessment Tool Kit Select most appropriate data collection methods Choose or develop data collection tools and protocols Plan the data collection process and timeline Implement the data collection process

Subcommittees considered the following: What is the focus of our assessment? What concepts do we need to define to ensure clear understanding? (ie. How do we define “high risk drinking”?) What methods of data collection are best for our assessment’s focus? What resources do we need to collect the data? (financial, staff time, training, locations, etc.) Who will collect the data? How will we implement the data collection process? How will we address confidentiality? Who will our participants be? How will we recruit them? How many participants do we need for our data collection? Where will we conduct the assessment? How much time is needed to administer the data collection process? How will we measure the data? How will we ensure the tools we use are culturally appropriate? How will we pilot/adjust the tools with potential participants? Review Assessment Toolkit

Consider Collection Methods and Timelines Subcommittees considered the following: Key Informant Interviews Focus Groups Written Surveys Community Forums Environmental Scans Social Media Analysis Policy Review Benchmarking

Select or Create Data Collection Tools and Protocols Data, Assessment and Evaluation Subcommittee developed the following tools: Written surveys Focus group questionnaires and protocols Key informant interview questionnaires and protocols Social Media Collection Tools Environmental Scan Surveys

Development of Tools/Instruments to assess Over- service & Effectiveness of Responsibility Toolkit: – Establishment Observation Tool, Volunteer Observer Protocol, and Establishment Observation Supervisor Checklist; – Utilization Assessment Tool; and – Patron Survey Example: Bar Sales and Services Subcommittee

Responsibility Toolkit

Raise awareness of negative consequences associated with drinking & driving Increase perception of risk of getting arrested for DUI Raise awareness of SC law prohibiting over-service to intoxicated patrons Raise awareness of transportation options for intoxicated patrons Raise awareness of increased DUI enforcement Toolkit Objectives: Patron

Raise awareness of SC law prohibiting over-service to intoxicated patrons Raise awareness that law enforcement is tracking place of last drink Raise awareness of civil liability of establishment in event of tragedy Raise awareness of increased DUI enforcement Raise awareness of transportation options for intoxicated patrons Increase self-efficacy to refuse service to intoxicated patron Toolkit Objectives: Employees & Management

Increase implementation of model policies governing alcohol sales Increase establishment accountability to prevent driving after drinking Increase partnerships between alcohol retail establishments & Task Force Increase participation in responsible beverage service training (PREP) Toolkit Objectives: Management (behavior)

Patron/staff ratio Drink specials/promotions Age verification systems Inconsistent systems to designate underage patrons vs. 21 and older Staff drinking with patrons Patrons requesting non-alcohol drinks ridiculed, hassled Retailers see over-service as a personal responsibility issue, not a retailer issue Observational Scanning Findings

Conclusions from the Assessment Community Collaboration: Challenging issue – multiple systems, sectors Difficulty engaging alcohol retailers in substance use prevention efforts “No one size fits all” – community demographics Responsibility Toolkit: No good evidence of effects Observation Tool: Provided documentation for both (a) pre intervention baseline and (b) post intervention Manageable and easy to administer Observer agreements (reliability) from 70-99%-- EXCELLENT

Implications for Future Coalition Prevention Efforts Continue assessment activities (bar sales and services observations, key informant interviews with bartenders and shuttle drivers; frequent, brief student surveys, etc.) Continue collaboration efforts, particularly with retail establishments Establish a student substance abuse cost analysis for key stakeholders Utilize science concerning key variables and effective interventions Establish a city-college data monitoring system (MIS)—reflect USC plus city. Publish article “Observational Documentation in Evaluating Alcohol Over- service in Bars and Restaurants” for professional prevention journals to share findings with other college communities

Rhonda DiNovo Phone- (803) Questions & Answers