Presentation on theme: "EUROPEAN DRUG PREVENTION QUALITY Standards"— Presentation transcript:
1 EUROPEAN DRUG PREVENTION QUALITY Standards EQUS conference, Brussels, June 2011
2 Synergy between two projects Prevention part of the EQUS standards was developed through an earlier, separate EC co-funded projectProject started 2008, completed in 2010It was recognised that it was more efficient to adapt the existing prevention standards than to develop a new set of prevention standards synergy between both projectsHowever: different structure of standards, different methodologies etc.
3 Outline of the presentation Introduction to the prevention standards projectMethodology of standards developmentStructure/content of the prevention standards (brief!)Adaptation of the standards to the EQUS projectStructure/content of the EQUS prevention standardsIf you have any questions, please feel free to ask at any time
5 Project beneficiaries Project partners:Liverpool John Moores University (LJMU), United Kingdom (Project lead)Azienda Sanitaria Locale della Città di Milano (ASL), ItalyConsejeria de Sanidad - Servicio Gallego de Salud (Xunta de Galicia) (CS-SERGAS), SpainAzienda Sanitaria Locale n. 2 - Savonese (ASL2), ItalyInstitute for Social Policy and Labour (SZMI-NDI), HungaryNational Anti-Drug Agency (NAA), RomaniaNational Bureau for Drug Prevention (NBDP), PolandEuropean Monitoring Centre for Drugs and Drug Addiction (EMCDDA)Adaptation of standards for EQUS project carried out by LJMU
6 Background & Aims At the time of starting the project: No EU-level guidance on evidence-based drug preventionNational or regional guidance available in some countries – applicable to wider EU?USA standards of evidence – applicable to European context?Lack of guidance for policy makers and practitionersAims:To bridge the gaps between science, policy and practiceTo produce a set of evidence-based drug prevention standards for use in the EUTo provide a checklist for policy makers and practitioners
7 Potential Impact of Standards Policy and practice:Improve drug prevention practice (e.g. increase sustainability of interventions)Improve efficiency of funding (better outcomes)Target populations:Reduce likelihood of implementation of ineffective or iatrogenic interventionsIncrease relevance and acceptability of interventions for target populationsResearch and evidence base:More evidence-based and scientifically sound interventionsImprove European evidence base for prevention by promoting research methodologyInterested? European Society for Prevention Research (EUSPR) (http://euspr.org)
9 Collecting information about available drug prevention guidance 27EU countries;EMCDDA mailing12Form returned6 No standards6 Standards(5 known)15Form not returned5 No follow-up6 Follow-up1 Not returned3 No standards2 Standards(4 known)12 EU countries with drug prevention standards or guidance (9 already on EMCDDA website, 3 new)9 EU countries: no standards (yet)6 EU countries: no information received
10 Available guidance – EU countries (spring 2009) Drug prevention standards and/or guidelines availableNo standardsNo information receivedCzech RepublicDenmarkFinlandGermanyIrelandItaly (regionally)LithuaniaPolandPortugalRomaniaSpain (Galicia)United KingdomAustriaCyprus (in progress)*FranceGreeceHungary (in progress)LatviaNetherlands (in progress)SloveniaSwedenBelgiumBulgariaEstoniaLuxembourgMaltaSlovakia* Cyprus: standards completed in late 2009
11 Methodology Method Aims Implementation Timeline Collation and review of existing guidanceTo produce a long list of standards; to identify a common structure that will synthesise existing standards77 documents retrieved, 19 documents selectedMarch-September 2009First draft of standards
12 Methodology First draft of standards Method Aims Implementation TimelineDelphi surveyPerceived priority of standards423 professionals completed both roundsJanuary-February 2010Focus groups(Cultural) relevance of standards14 focus groups heldMarch-April 2010Second draft of standardsField testingUsability and feasibility of standards72 professionals took partAugust-September 2010Final standards
13 Participants in consultations In six countries:Galicia (Spain), Hungary, Italy, Poland, Romania, UKSampling frame covered ten professional backgrounds:Regional drug teams or networksEducationHealthMental HealthSocial services/ Children, young people, familiesCriminal JusticeVoluntary/ Community sectorGovernment representativesPrevention consultantsMedia
14 Online Delphi survey Aim: to validate draft structure and to identify priority itemsTwo rounds, with 2-3 weeks per round (January and February 2010)First round:Participants rated the priority of each component from “High Priority” to “Not a priority at all”Additionally, possibility to choose whether standards should be made “mandatory” or notText comments possibleSecond round:Same as above, but –Also information on how all participants had rated the components in the first round, and examples of text comments from the first roundOnly results from Round 2 are shown in this presentation
15 Response rates EU ES HU IT PL RO UK Recruitment: Persons contacted 987 EUESHUITPLROUKRecruitment: Persons contacted9875611421031982206Round 1: complete and valid responses48753911189577% of contacted49%95%80%56%30%65%37%Round 2: complete valid responses4238394844663% of Round 1 responses87%100%91%88%82%
16 Delphi survey findings (excerpt) PriorityConsensusHigh priorityMandatory yesPriority Rank1. Situation Analysis1.1. Assessing drug use and community needs2.73ü74.6 %87.7 %121.2. Defining the problem - Justifying the intervention2.6366.4 %87.3 %161.3. Understanding the target population2.8888.7 %92.9 %12. Resource Assessment2.1. Knowing policies and legislation2.3642.3 %80.9 %362.2. Identifying funding opportunities2.35< 50 %43.4 %72.1 %372.3. Mapping existing stakeholders and services2.4853.7 %83.4 %302.4. Assessing target population and community resources2.7476.1 %84.9 %102.5. Assessing internal capacities2.5455.5 %81.3 %243. Programme Formulation3.1. Defining the target population88.8 %94.5 %23.2. Defining goals and objectives2.8383.5 %90.0 %43.3. Defining the setting2.3240.8 %78.4 %383.4. Summarising the evidence base2.1026.9 %57.5 %443.5. Illustrating the theoretical framework1.9720.1 %60.0 %463.6. Developing or selecting an effective intervention2.7578.6 %89.5 %83.7. Determining the timeline2.2840.4 %76.3 %423.8. Ensuring an ethical intervention2.6169.0 %85.5 %20Note: Some UK results not included under "Mandatory", as missing values > 50%.First draft standards
17 Delphi findings - summary Across all partner countries, 23 out of 46 components obtained consensus and a higher priority ratingComponents with highest / lowest priority ratings:ComponentsPriorityConsensusHigh priorityMandatory yesPriority Rank1.3. Understanding the target population2.88ü88.7 %92.9 %13.1. Defining the target population88.8 %94.5 %24.2. Adapting the intervention to the target population2.8687.9 %89.7 %33.4. Summarising the evidence base2.1026.9 %57.5 %446.3. Conducting a pilot intervention2.05< 50 %30.1 %42.4 %453.5. Illustrating the theoretical framework1.9720.1 %60.0 %46Low ratings – standards not necessarily for exclusion, but for discussion and modification
18 Focus groups: Relevance, usefulness, feasibility Relevance: What is the current practice of professionals working in the area, and how do the draft standards relate to it? Are the draft standards appropriate for these professional groups and for their everyday work/projects?Usefulness: How would the standards improve current practice? What are potential unintended consequences of the draft standards (e.g. economic, ethical)? Are the standards clear to understand and without ambiguity? Are the draft standards likely to be accepted by intended users?Feasibility: How can the standards be integrated with current practice? Which standards are already common practice in most projects; which standards are “new” and yet to be achieved? Overlap with existing statutory frameworks? What further resources, workforce development/ training or support might be needed to implement the standards? What are the organisational, professional and other barriers to / opportunities for implementation?
19 Field testing (further focus groups) Aims:To discuss second draft standardsContent of introduction and appendixLayoutTo discuss possibilities for implementationPractical use of the standardsImplementation tools
20 Consultations: Emerging challenges Issues identified included:Knowledge on how to use standards, and what for - Promote the use of quality standardsLack of content guidance (i.e. what to do) - Develop content guidelinesFunding (e.g. for outcome evaluations, staff development), knowledge on how to attract funding - Attract fundingPrevention work is given less priority than treatment - Prioritise and support preventionLack or weakness of governmental prevention agencies - Establish and strengthen central prevention agenciesDiversity of prevention work - Consider differences in prevention practiceDuplication of work - Create synergiesScepticism among professional groups - Develop professional attitudes and skills- Will be discussed by Dr Rachele Donini in tomorrow’s session
22 Applicability of the standards The standards are applicable to a wide range of drug prevention activities, for example:universal, selective, indicated or environmental prevention;preventing initiation of drug use, reducing the frequency of use and/or reducing drug-related harms;targeting legal and/or illegal drugs;structured manualised programmes or ongoing participant- and needs-led services;short- or long-term projects;different methods, target populations, settings …The standards give advice on how to plan, implement, and evaluate interventions, and they can be used to reflect on new, ongoing, or completed activities.
23 Applicability of the standards Although the standards refer to programmes, they can be used to reflect on prevention work at several levels of delivery, including:People: individual staff members or teams.Activities: singular interventions or wider programmes comprising several interventions.Organisations: organisations involved in drug prevention, such as service providers or schools.Strategies: priorities, action plans and tenders set out by government or funding bodies.
24 Who contributes to drug prevention? Policy- and decision-making: those who work on a strategic level, for example government representatives, commissioners, regional planning teams, funders.Service management: those who are in charge of managing drug prevention activities at an organisational level within prevention providers.Front-line work / work in direct contact with the target population: those who conduct, or contribute to, drug prevention in (direct) contact with the target population, such as psychologists, youth workers, social workers, outreach workers, teachers, pharmacists.
25 Who contributes to drug prevention? Training: those who provide training in drug prevention. Training may be directed at practitioners (e.g. social workers), or it may take place in a higher education setting (i.e. university students).Supervision: those who supervise and support drug prevention professionals (particularly practitioners).Programme development: those who design and develop drug prevention interventions.Consultancy / evaluation / academic research: those who provide consultancy on prevention issues, for example with regard to effective drug prevention, prevention policy development, adaptation of programmes, and/or evaluation.
26 Recommended uses of Standards PurposeRecommendedInformation, education and guidanceDeveloping or updating quality criteriaSelf-reflection checklistDiscussion in group settingsPerformance appraisals
27 Dissemination and evaluation plan PurposeRecommendedFormal self-assessmentNot yetFunding decisionsExternal accreditationFollow-on projects:Produce user-friendly implementation tools/manualsTrial standards with real programmesDevelop accreditation system for model programmesProvide training/education to relevant target audiencesInternational Standards initiative with CCSA, CICAD, EMCDDA, UNODC (in early planning stage)
28 Importance of additional materials Example scenarios of how the standards could be usedGeneral and country-specific conditions for (and potential barriers to) implementationImplementation considerations for each of the 35 standard componentsReferences to additional literature, materials, websitesBespoke glossary developed for use with the standards (over 100 terms)Self-reflection checklist allowing users to reflect upon their current position in relation to the standards
30 The drug prevention project cycle - a model to be adopted and adapted
31 Content of Standards Cross-cutting Considerations A: Sustainability and fundingB: Communication and stakeholder involvementC: Staff developmentD: Ethical drug prevention1 Needs Assessment1.1 Knowing drug-related policy and legislation1.2 Assessing drug use and community needs1.3 Describing the need – Justifying the intervention1.4 Understanding the target population2 Resource Assessment2.1 Assessing target population and community resources2.2 Assessing internal capacities
32 Content of Standards 3 Programme Formulation 3.1 Defining the target population3.2 Using a theoretical model3.3 Defining aims, goals, and objectives3.4 Defining the setting3.5 Referring to evidence of effectiveness3.6 Determining the timeline4 Intervention Design4.1 Designing for quality and effectiveness4.2 If selecting an existing intervention4.3 Tailoring the intervention to the target population4.4 If planning final evaluations
33 Content of Standards 5 Management and Mobilisation of Resources 5.1 Planning the programme - Illustrating the project plan5.2 Planning financial requirements5.3 Setting up the team5.4 Recruiting and retaining participants5.5 Preparing programme materials5.6 Providing a programme description6 Delivery and Monitoring6.1 If conducting a pilot intervention6.2 Implementing the intervention6.3 Monitoring the implementation6.4 Adjusting the implementation
34 Content of Standards 7 Final Evaluations 7.1 If conducting an outcome evaluation7.2 If conducting a process evaluation8 Dissemination and Improvement8.1 Determining whether the programme should be sustained8.2 Disseminating information about the programme8.3 If producing a final report
35 Example layout Level 2: Component title Level 3: Attributes (basic) Examples to clarifymeaningImplementationconsiderationsLevel 3: Attributes (expert)Glossary terms
36 Three levels of detailProject stages: major areas of prevention practice covered by the standardsComponents: actions to be taken at that point in the projectAttributes: show how achievement of the standards can be evidenced
37 Two quality levelsBasic standards: applicable to all drug prevention work, regardless of its particular circumstances.Additional basic standards (“basic standard if”): reflect the basic level under particular circumstances.Expert standards: in addition to the basic standards, representing a higher level of quality; desirable but may not always be applicable or feasible.
39 Adaptation of the standards 1) Description and re-assessment of original standards documents in line with template used for treatment and harm reduction areasDevelopment, applicability, legal status etc. of original standardsAll original standards deemed relevant for EQUS project2) Coding of individual ‘basic’ standards and components in line with standards structure developed by EQUS project coordinatorsi.e. structure/ process/ outcome standards + sub-categoriesNo need for new data extraction
40 Adaptation of the standards 3) Developing prevention standards for EQUS projectAdaptation at component rather than attribute levelReduction of 35 components to 30 EQUS prevention standardsPhrasing changed in order to correspond to treatment and harm reduction standards, but meaning has been preservedProposed list of minimal quality standards reflects a summary of the basic standards submitted for EMCDDA publicationIt is likely that the final standards would receive a greater level of support in a new Delphi survey as some of the standards have been modified and rephrased.
41 Adaptation of the standards 4) Revisiting results from Delphi survey to determine level of consensus for standards in line with methods used in treatment and harm reduction areasCut-off levels (> 80%, >50-80%, 50% or less)‘Acceptability’ defined as percentage of those who thought that a standard would represent a ‘mandatory’ part of good quality drug prevention work (based on first draft standards)Remember that Delphi survey conducted with first draft standards – likely that final version would receive higher levels of acceptabilityIt is likely that the final standards would receive a greater level of support in a new Delphi survey as some of the standards have been modified and rephrased.
42 EQUS prevention standards The standards can consequently be distinguished as follows:standards with high acceptability (rated as mandatory by > 80% of responding participants)standards with medium acceptability (rated as mandatory by % of responding participants consensus) – for discussionstandards with low acceptability (rated as mandatory by less than 50% of responding participants) (already excluded/modified through previous work).Lists were established forstructural standards at service level (5 + 3)outcome standards at system level (4 + 0)process standards at service/intervention level (11 + 7)
43 EQUS prevention standards High acceptability – for informationDiscussion:Are the proposed lists of minimum quality standards (high consensus in surveys) acceptable?For which types of services / interventions are they accepted?
44 Structural Standards of Services 1. Ethical drug prevention: A code of ethics is defined. Participants’ rights are protected through informed consent. The programme has clear benefits for participants, and will not cause them any harm. Participant data is treated confidentially. The physical safety of participants and staff members is protected.2. Staff composition: The staff required for successful implementation is defined and (likely to be) available (e.g. type of roles, number of staff). The set-up of the team is appropriate for the programme. Staff selection and management procedures are defined.3. Policy and legislation: The knowledge of drug-related policy and legislation is sufficient for the implementation of the programme. The programme supports the objectives of local, regional, national, and/or international priorities, strategies, and policies.
45 Structural Standards of Services 4. Staff composition: Internal resources and capacities are assessed (e.g. human, technological, financial resources). The assessment takes into account their current availability as well as their likely future availability for the programme.5. Financial requirements: A clear and realistic cost estimate for the programme is given. The available budget is specified and adequate for the programme. Costs and available budget are linked. Financial management corresponds to legal requirements.
46 Outcome Standards at the System Level 6. Goal: It is clear what is being ‘prevented’ (e.g. what types of drug use?). The programme’s aims, goals, and objectives are clear, logically linked, and informed by the identified needs. They are ethical and ‘useful’ for the target population. Goals and objectives are specific and realistic.7. Outcome evaluation: Evaluation is seen as an integral and important element to ensuring programme quality. It is determined what kind of evaluation is most appropriate for the intervention, and a feasible and useful evaluation is planned. Relevant evaluation indicators are specified, and the data collection process is described.
47 Outcome Standards at the System Level 8. Monitoring the implementation: Monitoring is seen as an integral part of the implementation phase. Outcome and process data are collected during implementation and reviewed systematically. The project plan, resources, etc. are also reviewed. The purpose of monitoring is to determine if the programme will be successful and to identify any necessary adjustments.9. Outcome evaluation: The sample size on which the outcome evaluation is based is given, and it is appropriate for the data analysis. An appropriate data analysis is conducted, including all participants. All findings are reported in measurable terms. Possible sources of bias and alternative explanations for findings are considered. The success of the programme is assessed.
48 Process Standards of Interventions 10. Assessment procedures: The needs of the community (or environment in which the programme will be delivered) are assessed. Detailed and diverse information on drug use is gathered. The study utilises existing epidemiological knowledge as possible, and adheres to principles of ethical research.11. Assessment procedures: Sources of opposition to, and support of, the programme are considered, as well as ways of increasing the level of support. The ability of the target population and other relevant stakeholders to participate in the programme is assessed.12. Assessment procedures: Justifying the intervention: The need for an intervention is justified. The main needs are described based on the needs assessment, and the potential future development of the situation without an intervention is indicated. Gaps in current service provision are identified.
49 Process Standards of Interventions 13. Assessment procedures: The target population is chosen in line with the needs assessment. The chosen target population(s) can be reached. The needs assessment considers the target population’s culture and its perspectives on drug use.14. Intervention design: The programme is adequate for the specific circumstances of the programme (e.g. target population characteristics), and tailored to those if required. Elements to tailor include: language; activities; messages; timing; number of participants15. Intervention design: The programme builds on positive relationships with participants by acknowledging their experiences and respecting diversity. Programme completion is defined.
50 Process Standards of Interventions 16. Intervention design: If selecting an existing intervention, benefits and disadvantages are considered, as well as the balance between adaptation, fidelity, and feasibility. The interventions’ fit to the needs assessment and other local circumstances are also examined. The chosen intervention is adapted carefully, and changes are made explicit.17. Implementation: The programme is implemented according to the written project plan. The implementation is adequately documented, including details on failures and deviations from the original plan.18. Process evaluation: The implementation of the programme is documented and explained. The following aspects are evaluated: target population involvement; activities; programme delivery; use of financial, human, and material resources.
51 Process Standards of Interventions 19. Staff development: It is ensured prior to the implementation that staff members have the competencies which are required for a successful programme implementation. If necessary, high quality training based on a training needs analysis is provided. During implementation, staff members are supported in their work as appropriate.20. Dissemination: The final report documents all major elements of programme planning, implementation, and (where possible) evaluation in a clear, logical, and easy-to-read way. Details on implementation experiences and unintended outcomes are included. Legal aspects of reporting on the programme are included (e.g. copyright).
52 EQUS prevention standards Moderate acceptability – for discussionDiscussion:Which standards from the presented additional lists of quality standards (medium consensus in surveys) should be included in the definite lists of minimum standards?For which types of services / interventions?
53 Structural Standards of Services 1. Sustainability: The programme promotes a long-term view on drug prevention and is not a fragmented short-term initiative. The programme is coherent in its logic and practical approach.2. Planning the programme: A systematic programme plan is constructed. A written project plan outlines the main programme elements and procedures. Contingency plans and risk management strategies are developed.3. Sustainability: A programme is continued on the basis of evidence provided by monitoring and/or final evaluations. If it is to be continued, opportunities for continuation are outlined. The lessons learnt from the implementation are used to inform future activities.
54 Outcome Standards at the System Level (none – all high acceptability)
55 Process Standards of Interventions 4. Communication and stakeholder involvement: The multi-service nature of drug prevention is considered. All stakeholders relevant to the programme (e.g. target population, other agencies) are identified, and they are involved as required for a successful programme implementation. The organisation cooperates with other agencies and institutions.5. Physical environment: The setting(s) for the activities is (are) described. It matches the aims, goals, and objectives, available resources, and is likely to produce the desired change. Necessary collaborations for implementation of the programme in this setting are identified.6. Recruiting and retention: It is clear how participants are drawn from the target population, and what mechanisms are used for recruitment. Specific measures are taken to maximise recruitment and retention of participants.
56 Process Standards of Interventions 7. Intervention design: The programme is based on an evidence-based theoretical model that allows an understanding of the specific drug-related needs and shows how the behaviour of the target population can be changed. Scientific literature reviews and/or essential publications on the issues relating to the programme are consulted. The reviewed information is of high quality and relevant to the programme. The main findings are used to inform the programme.8. Intervention design: Materials necessary for implementation of the programme are specified. If intervention materials (e.g. manuals) are used, the information provided therein is factual and of high quality.9. Intervention design: A written, clear programme description exists and is (at least partly) accessible by relevant groups (e.g. participants). It outlines major elements of the programme, particularly its possible impact on participants.
57 Process Standards of Interventions 10. Implementation: Flexibility is possible if required for a successful implementation. The implementation is adjusted in line with the monitoring findings, where possible. Issues and problems are dealt with in a manner that is appropriate for the programme. Adjustments are well-justified, and reasons for adjustments are documented. (Not rated in Delphi survey. Included as a result of target group consultation)
58 Next steps For European drug prevention quality standards project Publication of standards by EMCDDAFor EQUS projectTo determine which of the adapted EQUS prevention standards are acceptable/feasible as EU minimum quality standards – AT THIS MEETINGSuggestions for better adaptation and integration into the EQUS project? – AT THIS MEETINGLJMU will submit revised list of standards to project coordinator following meeting
59 EMCDDA ManualFull list of prevention standards will be available as EMCDDA manual (publication expected with 3-6 months).
60 Contacts Angelina Brotherhood Public Health Researcher Dr Harry SumnallReader in Substance UseCentre for Public HealthLiverpool John Moores University, UK