1. Bogota, August 2011 Innovation surveys and innovation policy: the European experience Anthony Arundel UNU-MERIT, The Netherlands & University of Tasmania, Australia

2. Why survey? Case studies insufficient and can be dominated by the ‘loudest voices’ Surveys give an accurate picture of an entire sector or economy But, results only as good as the relevance of the questions and the accuracy of the data

3. CIS-4 (2000) Added organisational innovation CIS-2006 All new questions undergo cognitive testing CIS-2008 Addition of one-off modules MEPIN, NESTA pilot studies on innovation in the public sector 2010 Innobarometer survey, 4,000 responses from public sector agencies OECD/Eurostat project on developing a model questionnaire Public sector innovation Business sector innovation CIS-1 (1992) CIS-2 (1996 ) CIS-3 (2000) Added service sector CIS-2010 New questions

4. How public sector organisations innovate

5. What leads to innovative & effective public services? Regression model results (controlling for size, regional responsibility & country) for Professional organisations User firms, Staff Citizen users Supplier firms Foreign info sources Involve users Collaboration Staff incentives Evaluations Workgroups Staff training Policy drivers Budget decreases Laws or policies Information sources Strategies Note: Excludes management as an information source or as a player in strategies – too important!

6. What have we learnt from business sector innovation surveys? 1.Firms compete in sectors. 2.Collaboration and using a variety of knowledge sources increases innovation outcomes, but effect on economic outcomes is ambiguous. 3.Patents are a minor incentive for innovation. 4.Half of innovative firms do not perform R&D. 5.Firms innovate in many different ways. 6.Very few firms (approx 5%) are ‘pure’ technology adopters.

7. Bogota, August 2011 Has what we have learnt about innovation influenced European innovation policy? R&D data Patent data Innovation survey data Most commonly used indicators by the European policy community between 2005 and 2007

8. Bogota, August 2011 European policy instruments focus on R&D 95% of financial support in Europe for ‘innovation’ is for R&D.

9. Bogota, August 2011 2007 Innobarometer survey: 4,395 responses from innovative firms (all EU 27 countries)

10. What the role of innovation surveys should be: R&D data Patent data Innovation survey data

11. Bogota, August 2011 Barriers to the policy relevance of Innovation Surveys

12. Bogota, August 2011 … Lack of policy relevance of academic research based on innovation surveys.

13. Bogota, August 2011 Example Galia et al, Complementarities between obstacles to innovation: Evidence from France, 2004. –Obstacles are complementary (occur together), ‘implies a need to adopt a package of policies in order to help firms’.

14. Concerns over data reliability for: Innovation expenditures and sales from innovative products (innovative sales share) Effect of different markets on data for innovative sales shares Comparability of leading indicators by country and sector

15. Example of poor comparability In 2000, 45% of firms in Portugal innovated compared to 46% of firms in Finland. On the European Innovation Scoreboard, Finland is usually in first or second place, while Portugal is between 22 and 23 place out of 27 countries.

16. Bogota, August 2011 Comparability

17. Bogota, August 2011

18. Other barriers 1.Indicators often outdated – not timely. 2.Lack of adequate detail and trend data. 3.Innovation survey questions not relevant to policy needs.

19. Bogota, August 2011 Survey results contradict existing beliefs of policy makers.

20. Bogota, August 2011 Result: underuse of innovation survey data 1.Only a small number of indicators were in wide use. 2.Main policy use of innovation indicators was for benchmarking. 1.Rarely used to develop specific policies. 2.Occasional use for policy evaluation (collaboration)

21. Bogota, August 2011 Increasing the policy relevance of an innovation survey

22. Bogota, August 2011 Improve data reliability Increase survey frequency Reduce time between survey and release of results

23. Bogota, August 2011 Provide more detail - Question modules - Results by sector - Additional surveys

24. Bogota, August 2011 Improve question relevance Involve users in questionnaire development Policy makers Academics Business managers

25. Bogota, August 2011 Establish a strong working relationship with data users –Frequent meetings –Produce results tailored to user needs

26. Innovative performance of the Aquaculture sector in Tasmania

27. Innovative performance of the marine manufacturing sector

28. Bogota, August 2011 Improve awareness European Innovation Scoreboard: 6 of 29 indicators from CIS

29. Bogota, August 2011 What still needs to be done to improve the relevance of the CIS

30. Bogota, August 2011 Improve relevance of academic research I mprove data access Insist that academics evaluate the policy significance of their results

31. Bogota, August 2011 Develop a ‘science’ of innovation policy Address a lack of interest in results that contradict ‘perceived wisdom’: –Careful empirical work to change policy views. –Support a ‘science’ of innovation policy –Replication of results (increases credibility) –More evaluation of policy relevance

32. Improve relevance of innovation surveys to policies to increase performance Main policy goal that is also relevant to businesses Contrast between CIS and survey of public sector innovation: –Shift from asking about barriers to questions on drivers

33. Conclusions The CIS is expensive –To justify its cost, we need to increase its usefulness to policy, academics and businesses Long, slow process to improve the relevance of the CIS