Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Understanding Evaluation of STI Policy in a Multi-level Governance Context Karlsruhe, 13th June 2012.

Similar presentations


Presentation on theme: "1 Understanding Evaluation of STI Policy in a Multi-level Governance Context Karlsruhe, 13th June 2012."— Presentation transcript:

1 1 Understanding Evaluation of STI Policy in a Multi-level Governance Context Karlsruhe, 13th June 2012

2 2 STI Policy itself has increased in complexity due to: 1.Broadening of the innovation concept 2.Evolution from linear to systemic views of innovation (Lundvall, 1992; Nelson, 1993) 3.Inclusion of the policy mix concept in innovation field (Flanagan et al., 2011) Regions can be considered policy spaces, in which policies administered at different levels are being felt (Uyarra and Flanagan, 2010) In this context, multi-level governance of STI Policy constitutes a key issue and evaluation an important stage for policy-learning purposes. Motivation

3 3 Policy evaluation can be used for understanding this complexity through the concept of additionality We can broadly distinguish between 3 different types of additionalities: 1.input 2.output 3.behavioural Most existing STI policy evaluation doesn’t capture all of the policy complexity. Therefore regions or any other policy space might be defining STI policies without taking into account potential effects derived from interactions among rationales, domains and instruments from different administrative levels. Policy evaluation and additionality concepts

4 4 The Basque region is a very interesting case study and a good example of complexity Analysis of a specific regional R&D programme with impact on firms, versus analysis of the impacts of the whole funding system on Basque firms Measurement of input, output and behavioural additionality Basque region as policy space

5 5 Triangulation of methods : Matching approach and semi-structured interviews o Matching approach: Comparison of two groups: firms that have received an R&D subsidy with innovative firms that haven’t received any. o Semi-structured interviews with some beneficiaries as a complementary technique Focus of the analysis: Firms that have received any subsidy between 2001 and 2004 and their results from 2007-onwards Methodology

6 6 Use of propensity score matching to find similar firms in both the treated group (subsidized firms) and the control group (innovative but non-subsidized firms according to a set of individual characteristics) (Rosenbaum and Rubin, 1983; Dehejia and Wahba, 2002) Use the Nearest Neighbour Matching to match similar units in both groups according to their propensity score Calculate the ATT (Average Effect of Treatment on the Treated) through the difference between the average of the dependent variable in both groups. Given this, it is possible to calculate the causal effect of the treatment: We have run the model twice: for analysing the impact of the regional programme in isolation and the impact of the whole funding system (for the same amount of subsidies received)

7 7 Results for the whole population of firms Controlling for other R&D programmes (national & local)Without controlling for other R&D programmes Variable(log) Expected coefficient sign Expected mean from the matched sample (SD. in parentheses) (* significant at 95%) Differences in averages (firms subsidized by Intek- control firms) Mean (SD. in parentheses) (*significant at 95%) Expected coefficient sign Expected mean from the matched sample (SD. in parentheses) (* significant at 95%) Differences in averages (firms subsidized by Intek- control firms) Mean (SD. in parentheses) (*significant at 95%) Total R&D expenditure (*) (1.0350) ( ) (*) (0.3669) (*) ( ) R&D intensity ( ) (*) ( ) ( ) ( ) R&D employees in the Basque Country (FTE) (0.3625) (*) ( ) (*) ( ) (*) (0.2818) Number of PhDs in the whole firm ( ) ( ) ( ) (*) ( ) Input additionality: Output additionality: Controlling for other R+D programmes (national & local)Without controlling for other R&D programmes Variable Expected coefficient sign Expected mean from the matched sample (SD. in parentheses) (* significant at 95%) Differences in averages (firms subsidized by Intek- control firms) Mean (SD. in parentheses) (*significant at 95%) Expected coefficient sign Expected mean from the matched sample (SD. in parentheses) (* significant at 95%) Differences in averages (firms subsidized by Intek- control firms) Mean (SD. in parentheses) (*significant at 95%) Productivity (log) (*) (0.5403) (*) ( ) (*) (0.3835) (*) ( ) Patents (num) ( ) (*) ( ) ( ) (*) ( ) Patentes (probability of patent) ( ) (*) ( ) ( ) (*) ( )

8 8 Controlling for other R+D programmes (national & local)Without controlling for other R&D programmes Variable Expected coefficient sign Expected mean from the matched sample (SD. in parentheses) (* significant at 95%) Differences in averages (firms subsidized by Intek- control firms) Mean (SD. in parentheses) (*significant at 95%) Expected coefficient sign Expected mean from the matched sample (SD. in parentheses) (* significant at 95%) Differences in averages (firms subsidized by Intek- control firms) Mean (SD. in parentheses) (*significant at 95%) R&D systematization ( ) ( ) ( ) (*) ( ) EU(log€) (0.1815) ( ) ( ) (*) ( ) PhDs/total employment ( ) (*) ( ) ( ) (*) ( ) Results for the whole population of firms Behavioural additionality:

9 9 Results: Additionalities achieved in collaborative firms Controlling for other R+D programmes (national & local)Without controlling for other R&D programmes Variable Expected coefficient sign Expected mean from the matched sample (SD. in parentheses) (* significant at 95%) Differences in averages (firms subsidized by collaborative Intek projects- control firms) Mean (SD. in parentheses) (*significant at 95%) Expected coefficient sign Expected mean from the matched sample (SD. in parentheses) (* significant at 95%) Differences in averages (firms subsidized by collaborative Intek projects- control firms) Mean (SD. in parentheses) (*significant at 95%) Total R&D expenditure (*) ( ) ( ) (0.9482) ( ) R&D employees in the Basque Country (FTE) (0.4442) ( ) (0.2838) ( ) Number of PhDs in the whole firm ( ) ( ) ( ) ( ) R&D intensity (0.1386) ( ) (0.1281) ( ) Productivity (log) (*) (0.8843) (0.1494) (*) (0.7021) ( ) Patents (num) ( ) ( ) ( ) ( ) Patents (prob to patent) ( ) ( ) ( ) ( ) R&D systematizati on (0.2933) ( ) (0.2530) (*) ( ) EU(log€) ( ) (*) ( ) ( ) (*) ( ) PhDs/ total employment ( ) ( ) ( ) 1.708e-02 (8.937e-03) Higher effect of the regional programme in isolation

10 10 Synthesis of Results Regional PolicyMulti-level Policies All Firms Low input additionality or crowding out effects Output additionality Low Behavioural additionality Multiplier effects on input, output and behavioural additionalities Firms that collaborate No input and output additionality Behavioural additionality No input and output additionality Low behavioural additionality or crowding out effects

11 11 Conclusions and implications for public policy Evaluation needs to evolve in order to capture complexity at policy level perspective Interaction of policies administered at different levels have a multiplier effect on regions so it is appropriate to consider regions as policy spaces. STI policy has different effects depending on the administrative level it has been designed: Input and output additionality could be better achieved by other STI policies different to regional ones (mainly national policies) Behavioural additionality are achieved by STI regional policies, which maximise the value of proximity for innovation In consequence, multi-governance of STI policies is necessary to take the best from each level Regions have to develop capacities to adequately design and implement regional STI policies to avoid risks of considering policy effects in isolation and not from a systemic perspective

12 12 Thank you


Download ppt "1 Understanding Evaluation of STI Policy in a Multi-level Governance Context Karlsruhe, 13th June 2012."

Similar presentations


Ads by Google