Presentation on theme: "Daniela Gatti Lombardy Region ["— Presentation transcript:
1Daniela Gatti Lombardy Region [e-mail: firstname.lastname@example.org] Network: The better future for social economySeminar on March, WarsawEvaluation of socio-economic value of social enterprises A model for measuring of social added valueDaniela GattiLombardy Region[
2ObjectivesThe general goal of the evaluation model is to measure the social value produced by a social enterprise as a whole or by a specific project.The primary goal of the system is to give funding entities, particularly public bodies, a tool for evaluating the social value created by social enterprises.allowing institutions to make informed decisions and better monitor the use of resourcesusing parameters that are as objective as possible and applicable on large scale
3BeneficiariesPublic bodies: public bodies play a key role in the process. Their job is to monitor and verify the results and social- environmental impacts produced by the allocated funds.Social enterprises: social enterprises are the object of the evaluation. They needs to be aware of the generated social value and to develop monitoring, evaluating and accounting tools.Associations of social enterprises: The associations of social enterprises are responsible for ensuring the quality of their enterprises and developing their entrepreneurial skills.Banks and financial institutions: credit institutions, according to the new BASEL II rules, can also evaluate enterprises according to their social performance.Private providers (e.g. foundations): foundations for territorial social policies have become interesting providers of financing.
4Starting pointCo-design the research with project partners > choosing the reasearch fieldDuring the meeting held in Warsaw last year it was decided to create a tool for evaluating the social value produced by social enterprises by using two main instruments:Social Accounting SROI–Social Return on Investment
5Methodology The research outline is based on two principles: the importance of the experience aspectthe participatory approachcollecting as many experiences as possible on local, national and international levelsinvolving all stakeholders: project partners, subjects who in the future will use the evaluation tool, but also social enterprises that will be the object of evaluationIt should lead to the implementation of a tool that is as close as possible to the actual needs of the players involved and applicable to/in the everyday life of organisations.
6Work phasesThe research work has implemented the following specific interlinked phases which are being carried out partially in parallel:Desk researchQuestionnaire for project partnersConsultation of relevant stakeholdersEditing the model for measuring social added valuePilot projectFinal steps
71. Desk research (1)Survey of documents and site-bibliography: to identify sensitive areas and aspects to be considered, approaches and models for quality and social responsibility were examined.Subsequently, the research has focused the analysis on:International systems of quality evaluation (and particularly the ISO, SA8000 and EFQM rules)Evaluation systems promoted by EU projects within the last few years (e.g. several projects under EQUAL Initiative)Experiences of social accounting developed in Italy and Europe during the last twenty years.SROI models and tools
81. Desk research (2) Public Administration Social enterprise Goals System governanceManagement controlObjectivity and comparability of choicesQualitySocial legitimacy and affirmation of identityLearningStrengthening of relationshipsPrevalent criteriaEconomic quantitative criteria defined ex anteQualitative and economic criteria defined ex ante and ex postOpen questionsDefinition of output and performance standardsMeasurement of production factors and results in economic dimensionsSimplicity in the application to ensure homogeneous distributionFinding quality criteriaMeasurement of outcomesStakeholder engagementProcedures for determiningevaluation modelsUnder the government controlNegotiated
91. Desk research (3) Links between evaluation and Social accounting they are closely linked by a common threadthey share certain characteristics of social research, such as repeatability, transparency, controllabilitythey are both processes of search for meaning, i.e. a pluralistic process of meaning construction that allows room for discrepancies and different points of viewLinks between SROI and Social accountingthey are both approaches to help organisations understand and account for the impact of their work.they recognise that the true social, economic and environmental value achieved by organisations is not fully understood or adequately reported.Both are founded on similar principles.
102. Questionnaire for project partners (1) The questionnaire tackles the following items :Context analysis: describing the main programmes or funding lines dedicated to social enterprises in each country)Currently used evaluation systems: in this part the partners were asked to describe in depth the currently used evaluation systemsObjectives and perspectives for a new evaluation tool: the aim is to understand the main objectives that each partner has in relation to evaluation of social enterprises accessing public funding. Moreover the partners were asked to describe the features which should characterize a new evaluation system.Questions concerning a possible evaluation tool integrating Social Accounting and SROI. In this sections there are questions concerning Social Accounting and SROI, in relation to their main strengths and weakness, their features, the need for indipendent evaluation and the applicability of the two instruments within the social enterprises context of each country.
112. Questionnaire for project partners (2) The partners agree on the following items:The system has to consider economic, social and environmental results (completeness)The system should be relatively easy to carry out (no experts needed)The system should be clear enough for those who are supposed to use the results of the evaluationThe system should take into account small social enterprises, which lack human and economic resourcesThe results of social evaluation should be comparable between organizationsThe systems should support social enterprises’ empowerment.Agreement on a system which uses the methodology of both social accounting and SROI, but the partners generally ask for a simplification of the two toolsThe partners have different ideas on:indipendent assessment /evaluation : it is a guarantee for process and data correctness; on the other hand it is an additonal cost.
123. Consultation of relevant stakeholders (1) focus group with selected local stakeholders:representatives of the Lombardy Region, of Italian Cooperatives’ National Agencies and of Finlombarda. Aim: identifying relevant and significant dimensions of the social added value analysisone-to-one interviews :Interviews with representatives of banks working closely with the Third Sector both directly and through initiatives and agreements with the Lombardy Region and other public investors (3 banks). Confcooperative)Interviews with representatives of social cooperatives (i.e. Legacoop and Confcooperative)Interview with a representative of a guarantee funds Institution: Cooperfidi
133. Consultation of relevant stakeholders (2) The one-to-one interviews have tackled the following themes:Definition of social enterprise and of social valueMain elements which could be considered indicators of a virtuous social enterprisesMain elements which could be considered indicators of problematic aspects within social enterprisesEvaluation tools already used to take decisions concerning the allocation of funding or loansMain characteristics of a system which could help public investors in evaluating the social added value of social enterprisesAnalyse a selection of indicators to provide comments, changes and additions
143. Consultation of relevant stakeholders (3) Main results form one-to-one interviews:necessity to elaborate simple tools which could be easily and effectively used within daily activities.the importance of the relation between the evaluator and the evaluated enterprises, which is the necessary basis for a coherent and adequate evaluation.Evaluation systems should represent a first step of analysis but not the only one.the importance of training and competence acquisition for evaluatorsa the widespread introduction of evaluation systems could be very useful in promoting the organizational empowerment of social enterprises
154. Editing the model for measuring social added value The evaluation system consists of two partsThe first evaluation system is designed to "measure" the social value produced by the social enterprise as a wholeThe second system is intended for evaluating mainly single projects ore interventions (even if it can be applied to the whole organisation). It is based on SROI analysisEach part could be applied at different levels of depth analysis, based on the context (i.e. amount of the financial grant, size of the social enterprise, etc..)
16Evaluation of the overall social added value produced by the enterprise (1) The model has been built and elaborated largely on the basis of the accounting reporting guidelines and other experiences concerning the generation and distribution of “value” developed in recent years.Main goal: to evaluate social, economic and environmental value pruduced by the social enterprise in its whole.
17Evaluation of the overall social added value produced by the enterprise (2) PrinciplesConnection: analysis of elements which could be attributed directly to the social enterpriseCompleteness: information presenting a complete picture (triple bottom line) are requested.Significance and relevance: the relevance needs to be evaluated on the basis of two main factors: 1-the importance given by the stakeholders involved (i.e. Public Administrations and social enterprises); 2-the nature, scale and size of the elements involved (i.e. amount of funding involved, size of the social enterprise, relevance of the project etc.)Applicability: the evaluation tool and the results from its implementation need to be understandable by a sizeable range of stakeholdersComparability: the results obtained from the implementation of the tool need to be comparableCredibility: the information is reliable and credible only when it is unbiased and error-free.Verification: Data and information (or at least the process of implementation and the compliance with the basic principles) will need to be verified by a third independent party. Currently, the issue of verification concerning the tool is still open: there are difficulties related to costs and timing as well as to a lack (in several countries) of professionals accredited to this end
18The system takes into considerations the following dimensions: Evaluation of the overall social added value produced by the enterprise (3)The system takes into considerations the following dimensions:Financial and economic soundnessDemocracy and governanceOrganizational functioningProfessional resourcesEqual opportunitiesSocio-occupational integrationClientsNetworks and partnersProject design and innovation abilitiesEnvironmental sustainabilityServices and interventionsEach of the above dimensions is made up of further sub-dimensions.
19Regarding the scoring model and calculation methods: Evaluation of the overall social added value produced by the enterprise (4)To simplify the collection and analysis of data; to facilitate comparison and make it possible to build a concise index for each area of evaluation:Numeric indicators are always indexes.Qualitative indicators are presented as logical indicators, with closed questions requiring the answer 'yes' or 'no'.It will be possible to identify two types of indicators, core and addedRegarding the scoring model and calculation methods:a relative weight will be given to the various dimensions of the analysis in order to ensure the efficacy and compliance of the evaluation system.Scores will need to be determined for each index. The objective is to achieve an overall score for each dimension in order to build a dashboard for instant reading.The (Italian) scores will be defined through the method of percentiles based on the weighted average number of responses received from various social enterprises
20Evaluation of social value produced in projects: calculation of the SROI (1) SROI is an analytic tool developed recently in the United States for measuring and accounting for the social, economic and environmental impact created by organisations.This approach aims to reveal the value of changes experienced by stakeholders whether those changes have been priced in market transaction or not and helps identify which changes are significant. It uses financial proxies to help reveal the value and as part of the process of deciding which changes are significant.Thanks to funding from the Cabinet Office, a consortium led by SROI Network prepared a new Guide to SROI. The SROI Network is an international membership body supporting the consistent use and development of SROI. The model proposed is based on this Guide
21Evaluation of social value produced in projects: calculation of the SROI (2) Principles1- Involve stakeholders: Stakeholders are those people or organisations that experience change as a result of the activity and they will be best placed to describe the change. This principle means that stakeholders need to be identified and then involved in consultation throughout the analysis,2- Understand what changes:. This principle requires the theory of how these changes are created to be stated and supported by evidence. These changes are the outcomes of the activity, made possible by the contributions of stakeholders, and often thought of as social, economic or environmental outcomes. It is these outcomes that should be measured in order to provide evidence that the change has taken place.3- Value the things that matter: Many outcomes are not traded in markets and as a result their value is not recognised. Financial proxies should be used in order to recognise the value of these outcomes and to give a voice to those excluded from markets but who are affected by activities.(…)
22Evaluation of social value produced in projects: calculation of the SROI (2) 4- Only include what is relevant and appropriate:. The relevance of an activity or of a change will depend from the organisation and its stakeholders: they are the actors who identify what is relevant so as to provide a reasonable picture of the impacts produced.5- Do not over-claim: Only claim the value that organisations are responsible for creating. This principle requires reference to trends and benchmarks to help assess the change caused by the activity, as opposed to other factors, and to take account of what would have happened anyway or happened with the help of other organisations.6- Be transparent: This principle requires that each decision relating to stakeholders, outcomes, indicators and benchmarks; the sources and methods of information collection; the difference scenarios considered should be explained and documented..7- Verify the result: Although a SROI analysis provides the opportunity for a more complete understanding of the value being created by an activity, it inevitably involves subjectivity. Appropriate independent assurance is required to help stakeholders assess whether or not the decisions made by those responsible for the analysis were reasonable.
23Evaluation of social value produced in projects: calculation of the SROI (3) There are 5 work phases:Identifying aims and scope of the SROI and identifying the stakeholders. Its important to identify aims, subjects, targets, resources of the analysis. it is necessary to identify the key stakeholders. The stakeholders are those affected by the changes and the impacts produced by the activities analysed.Identifying the impacts of the project/intervention/service. We can develop the “map of impacts”, i.e. describe what changes and impacts the project activities might cause through its activities, starting with the incoming resources. The following factors must be taken into account: input, activities, output, outcome, attributions.Identifying indicators, economic values and proxies. Meaningful indicators must be identified in order to measure the outcomes achieved. Once the outcomes, the significant indicators and the data have been identified, it is necessary to express the outcomes in economic terms. When the outcomes dont’have an economic value, it is necessary to develop proxies.
24Evaluation of social value produced in projects: calculation of the SROI (3) Defining Impacts. In this workphase it will be necessary to verify whether the outcomes produced are indeed directly linked to the activities carried out by the social enterprise. This way, it is possible to avoid an over-estimation of the project value and hence increase the credibility and the reliability of the calculations carried out. We have to consider: deadweight, attribution, negative effects, drop off.Calculating the SROI: The SROI measures the value created in relation to the costs sustained, thereby acknowledging the social return on investment.SROI =Social valueInvestments (input)
25To carry on… Pilot Projects Within the Italian pilot project: identifying the key indicators (both compulsory and voluntary)identifying a system of weights to establish the relative scoring of each area of analysisTesting the evaluation system and modifying it if necessaryWithin the Italian pilot project:the first part of the evaluation tool will be applied to an adequate number of social enterprises within the Lombardy Region. This application will allow for the identification of values which will be used to calculate a weighted averagethe Italian tool (first and second part), will be applied and tested so as to be able to verify the actual functioning of the toolThe results from this final testing phase will be used to introduce final modifications to the model (if necessary)