Presentation on theme: "EUROPEAN DRUG PREVENTION QUALITY STANDARDS EQUS conference, Brussels, 15-17 June 2011."— Presentation transcript:
EUROPEAN DRUG PREVENTION QUALITY STANDARDS EQUS conference, Brussels, June 2011
Synergy between two projects Prevention part of the EQUS standards was developed through an earlier, separate EC co-funded project Project started 2008, completed in 2010 It was recognised that it was more efficient to adapt the existing prevention standards than to develop a new set of prevention standards synergy between both projects However: different structure of standards, different methodologies etc.
Outline of the presentation 1. Introduction to the prevention standards project 2. Methodology of standards development 3. Structure/content of the prevention standards (brief!) 4. Adaptation of the standards to the EQUS project 5. Structure/content of the EQUS prevention standards If you have any questions, please feel free to ask at any time
Introduction to the Standards project
Project beneficiaries Project partners: Liverpool John Moores University (LJMU), United Kingdom (Project lead) Azienda Sanitaria Locale della Città di Milano (ASL), Italy Consejeria de Sanidad - Servicio Gallego de Salud (Xunta de Galicia) (CS-SERGAS), Spain Azienda Sanitaria Locale n. 2 - Savonese (ASL2), Italy Institute for Social Policy and Labour (SZMI-NDI), Hungary National Anti-Drug Agency (NAA), Romania National Bureau for Drug Prevention (NBDP), Poland European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) Adaptation of standards for EQUS project carried out by LJMU
Background & Aims At the time of starting the project: No EU-level guidance on evidence-based drug prevention National or regional guidance available in some countries – applicable to wider EU? USA standards of evidence – applicable to European context? Lack of guidance for policy makers and practitioners Aims: To bridge the gaps between science, policy and practice To produce a set of evidence-based drug prevention standards for use in the EU To provide a checklist for policy makers and practitioners
Potential Impact of Standards Policy and practice: Improve drug prevention practice (e.g. increase sustainability of interventions) Improve efficiency of funding (better outcomes) Target populations: Reduce likelihood of implementation of ineffective or iatrogenic interventions Increase relevance and acceptability of interventions for target populations Research and evidence base: More evidence-based and scientifically sound interventions Improve European evidence base for prevention by promoting research methodology Interested? European Society for Prevention Research (EUSPR) (http://euspr.org)
Methodology of developing the Standards
Collecting information about available drug prevention guidance 12 EU countries with drug prevention standards or guidance (9 already on EMCDDA website, 3 new) 9 EU countries: no standards (yet) 6 EU countries: no information received 27 EU countries; EMCDDA mailing 12 Form returned 6 No standards 6 Standards (5 known) 15 Form not returned 5 No follow-up 6 Follow-up 1 Not returned 3 No standards 2 Standards (4 known)
Available guidance – EU countries (spring 2009) Drug prevention standards and/or guidelines available No standardsNo information received Czech Republic Denmark Finland Germany Ireland Italy (regionally) Lithuania Poland Portugal Romania Spain (Galicia) United Kingdom Austria Cyprus (in progress)* France Greece Hungary (in progress) Latvia Netherlands (in progress) Slovenia Sweden Belgium Bulgaria Estonia Luxembourg Malta Slovakia * Cyprus: standards completed in late 2009
Methodology MethodAimsImplementationTimeline Collation and review of existing guidance To produce a long list of standards; to identify a common structure that will synthesise existing standards 77 documents retrieved, 19 documents selected March-September 2009 First draft of standards
Methodology Field testing Usability and feasibility of standards 72 professionals took part August-September 2010 MethodAimsImplementationTimeline Delphi survey Perceived priority of standards 423 professionals completed both rounds January-February 2010 Focus groups (Cultural) relevance of standards 14 focus groups heldMarch-April 2010 First draft of standards Second draft of standards Final standards
Participants in consultations In six countries: Galicia (Spain), Hungary, Italy, Poland, Romania, UK Sampling frame covered ten professional backgrounds: Regional drug teams or networks Education Health Mental Health Social services/ Children, young people, families Criminal Justice Voluntary/ Community sector Government representatives Prevention consultants Media
Online Delphi survey Aim: to validate draft structure and to identify priority items Two rounds, with 2-3 weeks per round (January and February 2010) First round: Participants rated the priority of each component from High Priority to Not a priority at all Additionally, possibility to choose whether standards should be made mandatory or not Text comments possible Second round: Same as above, but – Also information on how all participants had rated the components in the first round, and examples of text comments from the first round Only results from Round 2 are shown in this presentation
Response rates EUESHUITPLROUK Recruitment: Persons contacted Round 1: complete and valid responses % of contacted49%95%80%56%30%65%37% Round 2: complete valid responses % of Round 1 responses87%100%91%80%88%87%82%
Delphi survey findings (excerpt) PriorityConsensus High priority Mandatory yes Priority Rank 1. Situation Analysis 1.1. Assessing drug use and community needs %87.7 % Defining the problem - Justifying the intervention %87.3 % Understanding the target population %92.9 %1 2. Resource Assessment 2.1. Knowing policies and legislation %80.9 % Identifying funding opportunities2.35< 50 %43.4 %72.1 % Mapping existing stakeholders and services %83.4 % Assessing target population and community resources %84.9 % Assessing internal capacities %81.3 %24 3. Programme Formulation 3.1. Defining the target population %94.5 % Defining goals and objectives %90.0 % Defining the setting %78.4 % Summarising the evidence base %57.5 % Illustrating the theoretical framework %60.0 % Developing or selecting an effective intervention %89.5 % Determining the timeline2.28< 50 %40.4 %76.3 % Ensuring an ethical intervention %85.5 %20 Note: Some UK results not included under "Mandatory", as missing values > 50%. First draft standards
Delphi findings - summary Across all partner countries, 23 out of 46 components obtained consensus and a higher priority rating Components with highest / lowest priority ratings: ComponentsPriority Consensu s High priority Mandatory yes Priority Rank 1.3. Understanding the target population %92.9 % Defining the target population %94.5 % Adapting the intervention to the target population %89.7 % Summarising the evidence base %57.5 % Conducting a pilot intervention2.05< 50 %30.1 %42.4 % Illustrating the theoretical framework %60.0 %46 Low ratings – standards not necessarily for exclusion, but for discussion and modification
Focus groups: Relevance, usefulness, feasibility Relevance: What is the current practice of professionals working in the area, and how do the draft standards relate to it? Are the draft standards appropriate for these professional groups and for their everyday work/projects? Usefulness: How would the standards improve current practice? What are potential unintended consequences of the draft standards (e.g. economic, ethical)? Are the standards clear to understand and without ambiguity? Are the draft standards likely to be accepted by intended users? Feasibility: How can the standards be integrated with current practice? Which standards are already common practice in most projects; which standards are new and yet to be achieved? Overlap with existing statutory frameworks? What further resources, workforce development/ training or support might be needed to implement the standards? What are the organisational, professional and other barriers to / opportunities for implementation?
Field testing (further focus groups) Aims: To discuss second draft standards Content of introduction and appendix Layout To discuss possibilities for implementation Practical use of the standards Implementation tools
Consultations: Emerging challenges Issues identified included: Knowledge on how to use standards, and what for - Promote the use of quality standards Lack of content guidance (i.e. what to do) - Develop content guidelines Funding (e.g. for outcome evaluations, staff development), knowledge on how to attract funding - Attract funding Prevention work is given less priority than treatment - Prioritise and support prevention Lack or weakness of governmental prevention agencies - Establish and strengthen central prevention agencies Diversity of prevention work - Consider differences in prevention practice Duplication of work - Create synergies Scepticism among professional groups - Develop professional attitudes and skills - Will be discussed by Dr Rachele Donini in tomorrows session
Using the Standards
Applicability of the standards The standards are applicable to a wide range of drug prevention activities, for example: universal, selective, indicated or environmental prevention; preventing initiation of drug use, reducing the frequency of use and/or reducing drug-related harms; targeting legal and/or illegal drugs; structured manualised programmes or ongoing participant- and needs-led services; short- or long-term projects; different methods, target populations, settings … The standards give advice on how to plan, implement, and evaluate interventions, and they can be used to reflect on new, ongoing, or completed activities.
Although the standards refer to programmes, they can be used to reflect on prevention work at several levels of delivery, including: People: individual staff members or teams. Activities: singular interventions or wider programmes comprising several interventions. Organisations: organisations involved in drug prevention, such as service providers or schools. Strategies: priorities, action plans and tenders set out by government or funding bodies. Applicability of the standards
Who contributes to drug prevention? Policy- and decision-making: those who work on a strategic level, for example government representatives, commissioners, regional planning teams, funders. Service management: those who are in charge of managing drug prevention activities at an organisational level within prevention providers. Front-line work / work in direct contact with the target population: those who conduct, or contribute to, drug prevention in (direct) contact with the target population, such as psychologists, youth workers, social workers, outreach workers, teachers, pharmacists.
Who contributes to drug prevention? Training: those who provide training in drug prevention. Training may be directed at practitioners (e.g. social workers), or it may take place in a higher education setting (i.e. university students). Supervision: those who supervise and support drug prevention professionals (particularly practitioners). Programme development: those who design and develop drug prevention interventions. Consultancy / evaluation / academic research: those who provide consultancy on prevention issues, for example with regard to effective drug prevention, prevention policy development, adaptation of programmes, and/or evaluation.
Recommended uses of Standards PurposeRecommended Information, education and guidance Developing or updating quality criteria Self-reflection checklist Discussion in group settings Performance appraisals
Dissemination and evaluation plan PurposeRecommended Formal self-assessmentNot yet Funding decisionsNot yet External accreditationNot yet Follow-on projects: Produce user-friendly implementation tools/manuals Trial standards with real programmes Develop accreditation system for model programmes Provide training/education to relevant target audiences International Standards initiative with CCSA, CICAD, EMCDDA, UNODC (in early planning stage)
Importance of additional materials Example scenarios of how the standards could be used General and country-specific conditions for (and potential barriers to) implementation Implementation considerations for each of the 35 standard components References to additional literature, materials, websites Bespoke glossary developed for use with the standards (over 100 terms) Self-reflection checklist allowing users to reflect upon their current position in relation to the standards
The Prevention Standards
The drug prevention project cycle - a model to be adopted and adapted
Content of Standards Cross-cutting Considerations A: Sustainability and funding B: Communication and stakeholder involvement C: Staff development D: Ethical drug prevention 1 Needs Assessment 1.1 Knowing drug-related policy and legislation 1.2 Assessing drug use and community needs 1.3 Describing the need – Justifying the intervention 1.4 Understanding the target population 2 Resource Assessment 2.1 Assessing target population and community resources 2.2 Assessing internal capacities
Content of Standards 3 Programme Formulation 3.1 Defining the target population 3.2 Using a theoretical model 3.3 Defining aims, goals, and objectives 3.4 Defining the setting 3.5 Referring to evidence of effectiveness 3.6 Determining the timeline 4 Intervention Design 4.1 Designing for quality and effectiveness 4.2 If selecting an existing intervention 4.3 Tailoring the intervention to the target population 4.4 If planning final evaluations
Content of Standards 5 Management and Mobilisation of Resources 5.1 Planning the programme - Illustrating the project plan 5.2 Planning financial requirements 5.3 Setting up the team 5.4 Recruiting and retaining participants 5.5 Preparing programme materials 5.6 Providing a programme description 6 Delivery and Monitoring 6.1 If conducting a pilot intervention 6.2 Implementing the intervention 6.3 Monitoring the implementation 6.4 Adjusting the implementation
Content of Standards 7 Final Evaluations 7.1 If conducting an outcome evaluation 7.2 If conducting a process evaluation 8 Dissemination and Improvement 8.1 Determining whether the programme should be sustained 8.2 Disseminating information about the programme 8.3 If producing a final report
Example layout Level 2: Component title Implementation considerations Glossary terms Level 3: Attributes (basic) Examples to clarify meaning Level 3: Attributes (expert)
Three levels of detail 1. Project stages: major areas of prevention practice covered by the standards 2. Components: actions to be taken at that point in the project 3. Attributes: show how achievement of the standards can be evidenced
Two quality levels Basic standards: applicable to all drug prevention work, regardless of its particular circumstances. Additional basic standards (basic standard if): reflect the basic level under particular circumstances. Expert standards: in addition to the basic standards, representing a higher level of quality; desirable but may not always be applicable or feasible.
Adaptation to EQUS project
Adaptation of the standards 1)Description and re-assessment of original standards documents in line with template used for treatment and harm reduction areas Development, applicability, legal status etc. of original standards All original standards deemed relevant for EQUS project 2)Coding of individual basic standards and components in line with standards structure developed by EQUS project coordinators i.e. structure/ process/ outcome standards + sub-categories No need for new data extraction
Adaptation of the standards 3) Developing prevention standards for EQUS project Adaptation at component rather than attribute level Reduction of 35 components to 30 EQUS prevention standards Phrasing changed in order to correspond to treatment and harm reduction standards, but meaning has been preserved Proposed list of minimal quality standards reflects a summary of the basic standards submitted for EMCDDA publication
Adaptation of the standards 4)Revisiting results from Delphi survey to determine level of consensus for standards in line with methods used in treatment and harm reduction areas Cut-off levels (> 80%, >50-80%, 50% or less) Acceptability defined as percentage of those who thought that a standard would represent a mandatory part of good quality drug prevention work (based on first draft standards) Remember that Delphi survey conducted with first draft standards – likely that final version would receive higher levels of acceptability
EQUS prevention standards The standards can consequently be distinguished as follows: standards with high acceptability (rated as mandatory by > 80% of responding participants) standards with medium acceptability (rated as mandatory by % of responding participants consensus) – for discussion standards with low acceptability (rated as mandatory by less than 50% of responding participants) (already excluded/modified through previous work). Lists were established for structural standards at service level (5 + 3) outcome standards at system level (4 + 0) process standards at service/intervention level (11 + 7)
High acceptability – for information Discussion: Are the proposed lists of minimum quality standards (high consensus in surveys) acceptable? For which types of services / interventions are they accepted? EQUS prevention standards
Structural Standards of Services 1.Ethical drug prevention: A code of ethics is defined. Participants rights are protected through informed consent. The programme has clear benefits for participants, and will not cause them any harm. Participant data is treated confidentially. The physical safety of participants and staff members is protected. 2.Staff composition: The staff required for successful implementation is defined and (likely to be) available (e.g. type of roles, number of staff). The set-up of the team is appropriate for the programme. Staff selection and management procedures are defined. 3.Policy and legislation: The knowledge of drug-related policy and legislation is sufficient for the implementation of the programme. The programme supports the objectives of local, regional, national, and/or international priorities, strategies, and policies.
Structural Standards of Services 4. Staff composition: Internal resources and capacities are assessed (e.g. human, technological, financial resources). The assessment takes into account their current availability as well as their likely future availability for the programme. 5.Financial requirements: A clear and realistic cost estimate for the programme is given. The available budget is specified and adequate for the programme. Costs and available budget are linked. Financial management corresponds to legal requirements.
Outcome Standards at the System Level 6. Goal: It is clear what is being prevented (e.g. what types of drug use?). The programmes aims, goals, and objectives are clear, logically linked, and informed by the identified needs. They are ethical and useful for the target population. Goals and objectives are specific and realistic. 7.Outcome evaluation: Evaluation is seen as an integral and important element to ensuring programme quality. It is determined what kind of evaluation is most appropriate for the intervention, and a feasible and useful evaluation is planned. Relevant evaluation indicators are specified, and the data collection process is described.
Outcome Standards at the System Level 8.Monitoring the implementation: Monitoring is seen as an integral part of the implementation phase. Outcome and process data are collected during implementation and reviewed systematically. The project plan, resources, etc. are also reviewed. The purpose of monitoring is to determine if the programme will be successful and to identify any necessary adjustments. 9.Outcome evaluation: The sample size on which the outcome evaluation is based is given, and it is appropriate for the data analysis. An appropriate data analysis is conducted, including all participants. All findings are reported in measurable terms. Possible sources of bias and alternative explanations for findings are considered. The success of the programme is assessed.
Process Standards of Interventions 10.Assessment procedures: The needs of the community (or environment in which the programme will be delivered) are assessed. Detailed and diverse information on drug use is gathered. The study utilises existing epidemiological knowledge as possible, and adheres to principles of ethical research. 11.Assessment procedures: Sources of opposition to, and support of, the programme are considered, as well as ways of increasing the level of support. The ability of the target population and other relevant stakeholders to participate in the programme is assessed. 12.Assessment procedures: Justifying the intervention: The need for an intervention is justified. The main needs are described based on the needs assessment, and the potential future development of the situation without an intervention is indicated. Gaps in current service provision are identified.
Process Standards of Interventions 13.Assessment procedures: The target population is chosen in line with the needs assessment. The chosen target population(s) can be reached. The needs assessment considers the target populations culture and its perspectives on drug use. 14.Intervention design: The programme is adequate for the specific circumstances of the programme (e.g. target population characteristics), and tailored to those if required. Elements to tailor include: language; activities; messages; timing; number of participants 15.Intervention design: The programme builds on positive relationships with participants by acknowledging their experiences and respecting diversity. Programme completion is defined.
Process Standards of Interventions 16.Intervention design: If selecting an existing intervention, benefits and disadvantages are considered, as well as the balance between adaptation, fidelity, and feasibility. The interventions fit to the needs assessment and other local circumstances are also examined. The chosen intervention is adapted carefully, and changes are made explicit. 17.Implementation: The programme is implemented according to the written project plan. The implementation is adequately documented, including details on failures and deviations from the original plan. 18.Process evaluation: The implementation of the programme is documented and explained. The following aspects are evaluated: target population involvement; activities; programme delivery; use of financial, human, and material resources.
Process Standards of Interventions 19.Staff development: It is ensured prior to the implementation that staff members have the competencies which are required for a successful programme implementation. If necessary, high quality training based on a training needs analysis is provided. During implementation, staff members are supported in their work as appropriate. 20.Dissemination: The final report documents all major elements of programme planning, implementation, and (where possible) evaluation in a clear, logical, and easy-to-read way. Details on implementation experiences and unintended outcomes are included. Legal aspects of reporting on the programme are included (e.g. copyright).
Moderate acceptability – for discussion Discussion: Which standards from the presented additional lists of quality standards (medium consensus in surveys) should be included in the definite lists of minimum standards? For which types of services / interventions? EQUS prevention standards
Structural Standards of Services 1.Sustainability: The programme promotes a long-term view on drug prevention and is not a fragmented short-term initiative. The programme is coherent in its logic and practical approach. 2.Planning the programme: A systematic programme plan is constructed. A written project plan outlines the main programme elements and procedures. Contingency plans and risk management strategies are developed. 3.Sustainability: A programme is continued on the basis of evidence provided by monitoring and/or final evaluations. If it is to be continued, opportunities for continuation are outlined. The lessons learnt from the implementation are used to inform future activities.
Outcome Standards at the System Level (none – all high acceptability)
Process Standards of Interventions 4.Communication and stakeholder involvement: The multi-service nature of drug prevention is considered. All stakeholders relevant to the programme (e.g. target population, other agencies) are identified, and they are involved as required for a successful programme implementation. The organisation cooperates with other agencies and institutions. 5.Physical environment: The setting(s) for the activities is (are) described. It matches the aims, goals, and objectives, available resources, and is likely to produce the desired change. Necessary collaborations for implementation of the programme in this setting are identified. 6.Recruiting and retention: It is clear how participants are drawn from the target population, and what mechanisms are used for recruitment. Specific measures are taken to maximise recruitment and retention of participants.
Process Standards of Interventions 7.Intervention design: The programme is based on an evidence-based theoretical model that allows an understanding of the specific drug-related needs and shows how the behaviour of the target population can be changed. Scientific literature reviews and/or essential publications on the issues relating to the programme are consulted. The reviewed information is of high quality and relevant to the programme. The main findings are used to inform the programme. 8.Intervention design: Materials necessary for implementation of the programme are specified. If intervention materials (e.g. manuals) are used, the information provided therein is factual and of high quality. 9.Intervention design: A written, clear programme description exists and is (at least partly) accessible by relevant groups (e.g. participants). It outlines major elements of the programme, particularly its possible impact on participants.
Process Standards of Interventions 10.Implementation: Flexibility is possible if required for a successful implementation. The implementation is adjusted in line with the monitoring findings, where possible. Issues and problems are dealt with in a manner that is appropriate for the programme. Adjustments are well-justified, and reasons for adjustments are documented. (Not rated in Delphi survey. Included as a result of target group consultation)
Next steps For European drug prevention quality standards project Publication of standards by EMCDDA For EQUS project To determine which of the adapted EQUS prevention standards are acceptable/feasible as EU minimum quality standards – AT THIS MEETING Suggestions for better adaptation and integration into the EQUS project? – AT THIS MEETING LJMU will submit revised list of standards to project coordinator following meeting
EMCDDA Manual Full list of prevention standards will be available as EMCDDA manual (publication expected with 3-6 months).
Contacts Angelina Brotherhood Public Health Researcher Dr Harry Sumnall Reader in Substance Use Centre for Public Health Liverpool John Moores University, UK