1. Seventh Framework Programme Grant Agreement No. 207727 Risk Based Inspections in organic farming Raffaele Zanoli Università Politecnica delle Marche, IT Improving the Organic Certification System Workshop in Brussels, October 14, 2011

2. A working definition of a RBI The goal of Risk Based Inspections (RBIs) is to develop a cost-effective inspection and maintenance program that provides assurance of acceptable integrity and reliability of a control system A risk based approach to inspection planning is used to: Ensure risk is reduced as low as reasonably practicable Optimize the inspection schedule Focus inspection effort onto the most critical areas Identify and use the most appropriate methods of inspection

3. Modelling RBI systems: Objectives 1.Assessment and description of the current inspection practices in terms of risk and efficiency 2.Define a probabilistic model to increase the efficiency of the system based on probability theory 3.Optimisation of enforcement measures to reduce the occurrence of objectionable organic production

4. Modelling RBI systems: Data required In order to predict the risk of non-compliance At farmer/operator level Depending on crop type, farm type, geographic location, operators characteristics, etc. We need data on: detected non-compliances; structural, financial and managerial information at operator level

5. Modelling RBI systems: Data available Collected during CERTCOST EU project Data from 6 different European CBs (from CH, CZ, DE, DK, IT, UK) Three years covered (2007-2009) We used standard data that is routinely recorded by inspection bodies

6. Available data do not match the requirements Databases mainly contain structural data CBs collect NC data with non-homogenised textual descriptions: hard to rank NC severity Sanction data are more standardised, but: they are only a proxy of NC no common definition of sanctions across CBs / countries no clear relationship between NC and sanctions (with some exceptions); no information available about why an operator receives a sanction (e.g. use of pesticides in wheat production, use of unauthorised feed for livestock, etc.)

7. Homogenisation of sanctions across CBs and countries IT, CZ (and UK) CBs use a similar 4 sanction category (UK: NC)classification Further aggregation in terms of slight and severe sanction categories IT, CZ, UK straightforward interpretation; DE, DK, CH: input from CBs to correctly classify sanctions

8. Distribution of farms, by sanction category, country, and year

9. Modelling RBI systems: Analytical tools Econometric models Parametric approach: a-priori distributional assumptions Statistical testing of each risk factor, count models Good for standardised analysis across countries/CBs Bayesian Networks Semiparametric approach: more flexible, no individual factor testing allowed Single and multiple risk factors impact evaluation Good for farm type simulations

10. Potential risk factors 46 hypothesis concerning factors affecting the probability for an operator to get a sanction has been generated with collaboration from partners The hypothesis refers to the following aspects: general risk, structural / managerial for farms, structural/managerial for processors, specific crop, livestock and product variables, control related issues Some of the hypothesis cannot be tested for all countries/years due to missing data (eg processor turnover, risk class)

11. Factors increasing/decreasing risk

14. Few risk factors found relevant for all countries: Past behaviour, Farm Size, Bovine livestock History dependence: operators who are not compliant tend to continue to be so if one operator has been non compliant the previous year is more likely to be non compliant in the next year If one operator has committed minor irregularities is more likely to be found to have committed major infringements No overall risk pattern for crop types, though country specific risks For livestock, bovines and pigs entail higher risk In countries where (slight) non compliances are more numerous (DK, UK, partly CH) there might be a higher farms homogeneity, hence lower discrimination effects of explanatory variables Personal, farmer-specific variables are probably crucial in explaining risk but we have VERY limited data on these

15. General conclusions We can say with some confidence which factors contribute to risk, but we cannot rule out those who don’t As a consequence, we cannot define low risk operator types To implement more efficient Risk Based Inspection procedures CBs would need better or different datasets RBI based on past experience can limit predictable risk, but cannot avoid potential ‘catastrophic’ events uncertainty is an essential factor that should inform inspection procedures (black swans): think what can impact (the sector, the consumer, the CB, etc.) most, even if the risk (probability) of occurrence is low (but maybe the cost of detection is also low)

16. Some statements to open discussion Harmonised RBI is fundamental to guarantee integrity, improve efficiency and reduce the cost of inspection: a growing body of small “organic” farmers and growers are refusing certification and inspection schemes and selling on alternative short supply-chains – this creates further confusion among consumers Without clear and uniform criteria for classifying non-compliances as irregularities or infringement AND without better data and better information systems, no RBI system can work on a global scale Without global trust on certification and inspection procedures no global organic trade can survive

17. Seventh Framework Programme Grant Agreement No. 207727 Grazie! Thank you! zanoli@agrecon.univpm.it

18. Limitations of the study Data issues: Data suffer from censoring (i.e. missing data): we only have information on NCs that were detected by the CBs, but we have no idea how many and what kind of NCs have NOT been detected Inspection data contain varying quality/quantity of management & structural data, but little/no personal information on operators All operators should be inspected at least once per year (legal requirement), but the share of subsequent inspections (either unannounced or follow ups) varies across countries and CBs Data are little/no harmonised both within a country and across various countries

19. Limitations of the study (2) Epistemological/methodological issues: What is the data generating process (DGP)? Since CBs are actually using some form of internal RBI protocol to inform timing of compulsory announced inspections as well as follow-ups and unannounced inspections, the risk factors that we have observed may simply depend on their inspection planning and NOT actual risk (confirmation bias) Due to limited amount of severe NCs and related sanctions in the database, the reliability of the analysis of factors influencing severe risks is limited by statistical reasons