1. Evaluation of substitution maintenance programmes in Slovenia Bob Keizer, Franz Trautmann Trimbos Institute, Utrecht (Nl) Nino Rode Faculty of Social Work, Ljubljana (Si) ‏

2. Evaluation of SMT in Slovenia: Original request of MoH Evaluation of quality of services delivered (including patient satisfaction) cost-effectiveness of the programme, assessing the input (human resource, financial) – output (client numbers, interventions / services) relation impact of the programme on patients, assessing the development of a selection of clients in a specified time interval starting at the time they enter the programme

3. Realisation / Adaptations Focus of quality interviews (using internationally agreed standards) on: – Staff (quantity and quality: professions, skills, knowledge, attitude) – Team meetings – Treatment (intake, treatment services offered) – Client approach – Client registration – Accessibility of programme – Monitoring – Guidelines and protocols – Process management – Cooperation with other services

4. Realisation / Adaptations From cost-effectiveness to an analysis of the system – Lack of (standards for relationship) hard input and output data – lack of comparability with other forms of treatment (reference standards) – Focus on management and organisational structure, financing system, client numbers, staff number and structure, etc. Impact on patients: – Lack of ‘hard’ patient data at intake – no RCT – Option of retrospective analysis – Self-report (‘soft’ data, but perception of well-being important notion in health concept) – Recommendation to develop tool based on EuropASI for the future

5. Realisation / Adaptations Presenting rationale, aims and evidence for effectiveness of SMT Collecting context information Including force field analysis to reflect highly politicised debate / surroundings in Slovenia Comparing Slovenian practice against a selection of reference countries – 2 old (D, NL) and 2 new EU Member States (CZ, LT)

6. C ontext I nput P rocess P roduct model Context: What needs to be done? – Background information – Force field analysis – Comparing with reference countries Input: How should it be done? – Quality intreview: –Staff availability, expertise and training, motivation, – System analysis questionnaire: –Efficiency of a system –Comparison of the SMT with other services –Management organisation and cost of SMT –Clients data (number, retention), –Staff structure –Financing

7. C ontext I input P rocess P roduct model Process: Is it being done? – quality intreview: –services: patient intake and treatment, procedures, availability of services, accessibility of the centre –organization: team-meetings, client registration system, monitoring, guidelines, process management, cooperation with other organizations Product: Did it succeed? – Client satisfaction intreview: –Satisfaction with the services –Impact of the services

8. Data sources Rationale, aims and evidence: – Desk research Context information: – Interviews with key stakeholders – Desk research Force field analysis: – Interviews with key stakeholders (political parties, involved Ministries, NGOs, media) Reference countries: – Interviews with 3 key experts p.c. (Focal Point, research, harm reduction experience) – Desk research

9. Data sources Quality – Interviews with directors + staff (18 SMTCs) – Interviews with directors PHCCs (6 SMTCs: Kočevje, Koper, Ljubljana, Logatec, Nova Gorica, Velenje) – Patient satisfaction interviews with 75 patients (6 SMTCs: Kočevje, Koper, Ljubljana, Logatec, Nova Gorica, Velenje) Efficiency of system – Interviews: directors + staff (18 SMTCs) – Interviews with directors PHCCs – Desk research

10. Data analysis Mixed methods (qualitative and quantitative) to get the full picture: – Largely qualitative analysis –force field analysis –analysis of open questions and remarks –Additional probing for clarification of inconsistencies – Quantitative analysis –analysis of frequencies (SPSS) –comparison with the qualitative data

11. Relevance of this type of evaluation Serving a mix of quantitative and qualitative information Functioning as thorough SWOT analysis – Strong points - weak points: serving information for improving SMT – Opportunities – threats: how to deal with threats Direct indications for adaptations Offering the possibility of discussion / supporting to find a consensus

12. Findings: Weak Points Treatment data collection in use does not allow formulating an individual treatment plan and monitoring SMT on individual, centre and national level; Management data collection in use does not serve reliable and accurate data for thorough auditing; There are general guidelines but no (basic) protocols resulting in substantial differences in treatment policy and practice between the SMTCs; The regular funding is limited regarding staff capacity for regular SMT services and insufficient for offering additional services like counselling or social work (in the SMTC);

13. Findings: Weak Points SMTCs are rather medically oriented, psycho- social aspects of problem drug use do not get sufficient attention (at least partly to be explained by insufficient staff capacity); SMTCs lack autonomy with regards to staff and financial management; Working in SMTCs (like in other drug services) has a relatively low status which makes it difficult to find appropriate staff.

14. Findings: Strong Points The practise of the prescription of substitution maintenance treatment in Slovenia is of a relative high standard if compared to other countries. It scores well on issues like different approaches for specific target groups, etc.; SMT has a high coverage; it covers nearly the whole country and around one third of the estimated total of problem heroin users. Access to SMT is good (no waiting lists, appropriate opening hours, no exceptional criteria for entering). Also in this respect SMT is scoring well compared to the reference countries;

15. Findings: Strong Points Monthly meetings of SMTC staff facilitates cooperation between SMTCs and expert exchange on among others SMT developments, treatment issues and individual cases; Diversity of substitution substances prescribed (taking into account individual needs); Following and picking up latest developments in SMT quickly (e.g. introduction of Suboxone in Slovene SMT); Overall consensus between staff and clients on being treated with respect or ‘normally’ by the other party is an important indicator for quality.

16. Recommendations Standardising work of SMTCs – Internal communication – Treatment policy through guidelines and checklists (e.g. take-home medication) Standardised and computerised treatment data collection – Facilitating on centre level treatment planning, monitoring and evaluation – Facilitating in anonymised, aggregated form monitoring and evaluation of SMTC work – Standardised recording of treatment services provided reflecting caseload and case severity

17. Recommendations Management data collection – Standardised and computerised (central) medication registration – Standardised and computerised financial reporting system Shift management responsibility from PHCC to SMTC: Stronger focus on psycho-social aspects – Through additional staff capacity or – Cooperation with other services

18. Recommendations Political responsibility: – Creating conditions for standardising treatment (guidelines and protocols, etc.) (not only SMTC) – Creating conditions for a financing system (taking into account the factual caseload and case severity) – Auditing (monitoring and evaluation) – Inter-ministerial cooperation to develop a consistent and coherent demand reduction policy – Inter-ministerial guidance to facilitate cooperation between the different demand reduction services