1. Daniela Gatti Lombardy Region [e-mail: d.gatti@pares.it]
Network: The better future for social economy
Seminar on March, Warsaw
Evaluation of socio-economic value of social enterprises A model for measuring of social added value
Daniela Gatti
Lombardy Region [

2. Objectives
The general goal of the evaluation model is to measure the social value produced by a social enterprise as a whole or by a specific project.
The primary goal of the system is to give funding entities, particularly public bodies, a tool for evaluating the social value created by social enterprises.
allowing institutions to make informed decisions and better monitor the use of resources
using parameters that are as objective as possible and applicable on large scale

3. Beneficiaries
Public bodies: public bodies play a key role in the process. Their job is to monitor and verify the results and social- environmental impacts produced by the allocated funds.
Social enterprises: social enterprises are the object of the evaluation. They needs to be aware of the generated social value and to develop monitoring, evaluating and accounting tools.
Associations of social enterprises: The associations of social enterprises are responsible for ensuring the quality of their enterprises and developing their entrepreneurial skills.
Banks and financial institutions: credit institutions, according to the new BASEL II rules, can also evaluate enterprises according to their social performance.
Private providers (e.g. foundations): foundations for territorial social policies have become interesting providers of financing.

4. Starting point
Co-design the research with project partners > choosing the reasearch field
During the meeting held in Warsaw last year it was decided to create a tool for evaluating the social value produced by social enterprises by using two main instruments:
Social Accounting SROI–Social Return on Investment

5. Methodology The research outline is based on two principles:
the importance of the experience aspect
the participatory approach
collecting as many experiences as possible on local, national and international levels
involving all stakeholders: project partners, subjects who in the future will use the evaluation tool, but also social enterprises that will be the object of evaluation
It should lead to the implementation of a tool that is as close as possible to the actual needs of the players involved and applicable to/in the everyday life of organisations.

6. Work phases
The research work has implemented the following specific interlinked phases which are being carried out partially in parallel:
Desk research
Questionnaire for project partners
Consultation of relevant stakeholders
Editing the model for measuring social added value
Pilot project
Final steps

7. 1. Desk research (1)
Survey of documents and site-bibliography: to identify sensitive areas and aspects to be considered, approaches and models for quality and social responsibility were examined.
Subsequently, the research has focused the analysis on:
International systems of quality evaluation (and particularly the ISO, SA8000 and EFQM rules)
Evaluation systems promoted by EU projects within the last few years (e.g. several projects under EQUAL Initiative)
Experiences of social accounting developed in Italy and Europe during the last twenty years.
SROI models and tools

8. 1. Desk research (2) Public Administration Social enterprise Goals
System governance
Management control
Objectivity and comparability of choices
Quality
Social legitimacy and affirmation of identity
Learning
Strengthening of relationships
Prevalent criteria
Economic quantitative criteria defined ex ante
Qualitative and economic criteria defined ex ante and ex post
Open questions
Definition of output and performance standards
Measurement of production factors and results in economic dimensions
Simplicity in the application to ensure homogeneous distribution
Finding quality criteria
Measurement of outcomes
Stakeholder engagement
Procedures for determining
evaluation models
Under the government control
Negotiated

9. 1. Desk research (3) Links between evaluation and Social accounting
they are closely linked by a common thread
they share certain characteristics of social research, such as repeatability, transparency, controllability
they are both processes of search for meaning, i.e. a pluralistic process of meaning construction that allows room for discrepancies and different points of view
Links between SROI and Social accounting
they are both approaches to help organisations understand and account for the impact of their work.
they recognise that the true social, economic and environmental value achieved by organisations is not fully understood or adequately reported.
Both are founded on similar principles.

10. 2. Questionnaire for project partners (1)
The questionnaire tackles the following items :
Context analysis: describing the main programmes or funding lines dedicated to social enterprises in each country)
Currently used evaluation systems: in this part the partners were asked to describe in depth the currently used evaluation systems
Objectives and perspectives for a new evaluation tool: the aim is to understand the main objectives that each partner has in relation to evaluation of social enterprises accessing public funding. Moreover the partners were asked to describe the features which should characterize a new evaluation system.
Questions concerning a possible evaluation tool integrating Social Accounting and SROI. In this sections there are questions concerning Social Accounting and SROI, in relation to their main strengths and weakness, their features, the need for indipendent evaluation and the applicability of the two instruments within the social enterprises context of each country.

11. 2. Questionnaire for project partners (2)
The partners agree on the following items:
The system has to consider economic, social and environmental results (completeness)
The system should be relatively easy to carry out (no experts needed)
The system should be clear enough for those who are supposed to use the results of the evaluation
The system should take into account small social enterprises, which lack human and economic resources
The results of social evaluation should be comparable between organizations
The systems should support social enterprises’ empowerment.
Agreement on a system which uses the methodology of both social accounting and SROI, but the partners generally ask for a simplification of the two tools
The partners have different ideas on:
indipendent assessment /evaluation : it is a guarantee for process and data correctness; on the other hand it is an additonal cost.

12. 3. Consultation of relevant stakeholders (1)
focus group with selected local stakeholders:
representatives of the Lombardy Region, of Italian Cooperatives’ National Agencies and of Finlombarda. Aim: identifying relevant and significant dimensions of the social added value analysis
one-to-one interviews :
Interviews with representatives of banks working closely with the Third Sector both directly and through initiatives and agreements with the Lombardy Region and other public investors (3 banks). Confcooperative)
Interviews with representatives of social cooperatives (i.e. Legacoop and Confcooperative)
Interview with a representative of a guarantee funds Institution: Cooperfidi

13. 3. Consultation of relevant stakeholders (2)
The one-to-one interviews have tackled the following themes:
Definition of social enterprise and of social value
Main elements which could be considered indicators of a virtuous social enterprises
Main elements which could be considered indicators of problematic aspects within social enterprises
Evaluation tools already used to take decisions concerning the allocation of funding or loans
Main characteristics of a system which could help public investors in evaluating the social added value of social enterprises
Analyse a selection of indicators to provide comments, changes and additions

14. 3. Consultation of relevant stakeholders (3)
Main results form one-to-one interviews:
necessity to elaborate simple tools which could be easily and effectively used within daily activities.
the importance of the relation between the evaluator and the evaluated enterprises, which is the necessary basis for a coherent and adequate evaluation.
Evaluation systems should represent a first step of analysis but not the only one.
the importance of training and competence acquisition for evaluators
a the widespread introduction of evaluation systems could be very useful in promoting the organizational empowerment of social enterprises

15. 4. Editing the model for measuring social added value
The evaluation system consists of two parts
The first evaluation system is designed to "measure" the social value produced by the social enterprise as a whole
The second system is intended for evaluating mainly single projects ore interventions (even if it can be applied to the whole organisation). It is based on SROI analysis
Each part could be applied at different levels of depth analysis, based on the context (i.e. amount of the financial grant, size of the social enterprise, etc..)

16. Evaluation of the overall social added value produced by the enterprise (1)
The model has been built and elaborated largely on the basis of the accounting reporting guidelines and other experiences concerning the generation and distribution of “value” developed in recent years.
Main goal: to evaluate social, economic and environmental value pruduced by the social enterprise in its whole.

17. Evaluation of the overall social added value produced by the enterprise (2)
Principles
Connection: analysis of elements which could be attributed directly to the social enterprise
Completeness: information presenting a complete picture (triple bottom line) are requested.
Significance and relevance: the relevance needs to be evaluated on the basis of two main factors: 1-the importance given by the stakeholders involved (i.e. Public Administrations and social enterprises); 2-the nature, scale and size of the elements involved (i.e. amount of funding involved, size of the social enterprise, relevance of the project etc.)
Applicability: the evaluation tool and the results from its implementation need to be understandable by a sizeable range of stakeholders
Comparability: the results obtained from the implementation of the tool need to be comparable
Credibility: the information is reliable and credible only when it is unbiased and error-free.
Verification: Data and information (or at least the process of implementation and the compliance with the basic principles) will need to be verified by a third independent party. Currently, the issue of verification concerning the tool is still open: there are difficulties related to costs and timing as well as to a lack (in several countries) of professionals accredited to this end

18. The system takes into considerations the following dimensions:
Evaluation of the overall social added value produced by the enterprise (3)
The system takes into considerations the following dimensions:
Financial and economic soundness
Democracy and governance
Organizational functioning
Professional resources
Equal opportunities
Socio-occupational integration
Clients
Networks and partners
Project design and innovation abilities
Environmental sustainability
Services and interventions
Each of the above dimensions is made up of further sub-dimensions.

19. Regarding the scoring model and calculation methods:
Evaluation of the overall social added value produced by the enterprise (4)
To simplify the collection and analysis of data; to facilitate comparison and make it possible to build a concise index for each area of evaluation:
Numeric indicators are always indexes.
Qualitative indicators are presented as logical indicators, with closed questions requiring the answer 'yes' or 'no'.
It will be possible to identify two types of indicators, core and added
Regarding the scoring model and calculation methods:
a relative weight will be given to the various dimensions of the analysis in order to ensure the efficacy and compliance of the evaluation system.
Scores will need to be determined for each index. The objective is to achieve an overall score for each dimension in order to build a dashboard for instant reading.
The (Italian) scores will be defined through the method of percentiles based on the weighted average number of responses received from various social enterprises

20. Evaluation of social value produced in projects: calculation of the SROI (1)
SROI is an analytic tool developed recently in the United States for measuring and accounting for the social, economic and environmental impact created by organisations.
This approach aims to reveal the value of changes experienced by stakeholders whether those changes have been priced in market transaction or not and helps identify which changes are significant. It uses financial proxies to help reveal the value and as part of the process of deciding which changes are significant.
Thanks to funding from the Cabinet Office, a consortium led by SROI Network prepared a new Guide to SROI. The SROI Network is an international membership body supporting the consistent use and development of SROI. The model proposed is based on this Guide

21. Evaluation of social value produced in projects: calculation of the SROI (2)
Principles
1- Involve stakeholders: Stakeholders are those people or organisations that experience change as a result of the activity and they will be best placed to describe the change. This principle means that stakeholders need to be identified and then involved in consultation throughout the analysis,
2- Understand what changes:. This principle requires the theory of how these changes are created to be stated and supported by evidence. These changes are the outcomes of the activity, made possible by the contributions of stakeholders, and often thought of as social, economic or environmental outcomes. It is these outcomes that should be measured in order to provide evidence that the change has taken place.
3- Value the things that matter: Many outcomes are not traded in markets and as a result their value is not recognised. Financial proxies should be used in order to recognise the value of these outcomes and to give a voice to those excluded from markets but who are affected by activities.
(…)

22. Evaluation of social value produced in projects: calculation of the SROI (2)
4- Only include what is relevant and appropriate:. The relevance of an activity or of a change will depend from the organisation and its stakeholders: they are the actors who identify what is relevant so as to provide a reasonable picture of the impacts produced.
5- Do not over-claim: Only claim the value that organisations are responsible for creating. This principle requires reference to trends and benchmarks to help assess the change caused by the activity, as opposed to other factors, and to take account of what would have happened anyway or happened with the help of other organisations.
6- Be transparent: This principle requires that each decision relating to stakeholders, outcomes, indicators and benchmarks; the sources and methods of information collection; the difference scenarios considered should be explained and documented..
7- Verify the result: Although a SROI analysis provides the opportunity for a more complete understanding of the value being created by an activity, it inevitably involves subjectivity. Appropriate independent assurance is required to help stakeholders assess whether or not the decisions made by those responsible for the analysis were reasonable.

23. Evaluation of social value produced in projects: calculation of the SROI (3)
There are 5 work phases:
Identifying aims and scope of the SROI and identifying the stakeholders. Its important to identify aims, subjects, targets, resources of the analysis. it is necessary to identify the key stakeholders. The stakeholders are those affected by the changes and the impacts produced by the activities analysed.
Identifying the impacts of the project/intervention/service. We can develop the “map of impacts”, i.e. describe what changes and impacts the project activities might cause through its activities, starting with the incoming resources. The following factors must be taken into account: input, activities, output, outcome, attributions.
Identifying indicators, economic values and proxies. Meaningful indicators must be identified in order to measure the outcomes achieved. Once the outcomes, the significant indicators and the data have been identified, it is necessary to express the outcomes in economic terms. When the outcomes dont’have an economic value, it is necessary to develop proxies.

24. Evaluation of social value produced in projects: calculation of the SROI (3)
Defining Impacts. In this workphase it will be necessary to verify whether the outcomes produced are indeed directly linked to the activities carried out by the social enterprise. This way, it is possible to avoid an over-estimation of the project value and hence increase the credibility and the reliability of the calculations carried out. We have to consider: deadweight, attribution, negative effects, drop off.
Calculating the SROI: The SROI measures the value created in relation to the costs sustained, thereby acknowledging the social return on investment.
SROI =
Social value
Investments (input)

25. To carry on… Pilot Projects Within the Italian pilot project:
identifying the key indicators (both compulsory and voluntary)
identifying a system of weights to establish the relative scoring of each area of analysis
Testing the evaluation system and modifying it if necessary
Within the Italian pilot project:
the first part of the evaluation tool will be applied to an adequate number of social enterprises within the Lombardy Region. This application will allow for the identification of values which will be used to calculate a weighted average
the Italian tool (first and second part), will be applied and tested so as to be able to verify the actual functioning of the tool
The results from this final testing phase will be used to introduce final modifications to the model (if necessary)

26. THANK YOU!