1. Interpreting Innovation Surveys 2
Globelics Tampere 2008

2. 1. A note on interaction effects
Serious issue that affects use of sector, size, or other dummy variables.
Assumption is that sector effects are parallel – no difference in effect of sector on dependent variable across other variables such as firm size.
Often not the case – need to introduce interaction variables and/or calculate separate regressions by major criteria of interest: ie separate regressions for small and large firms.
Note: Very few econometric studies using innovation survey data include interaction effects
Globelics Tampere 2008

3. 2. Econometric research
Globelics Tampere 2008

4. 2.1 Main Econometric research topics
Who innovates?
….at what level (novelty, in-house…)
If a firm innovates, what are its innovation strategies?
What effect do these strategies have on performance outcomes?
Globelics Tampere 2008

5. The number of econ0metric papers on the effects of innovation strategies has been increasing substantially after 1999, along with research on innovation strategies (after 2001). There are still a lot of papers on who innovates or how much, but increasingly, after 2000, these are due to the inclusion of this information in tobit models, with this topic no longer forming the main topic of the paper.
Globelics Tampere 2008

6. 2.2 Who innovates Determinants of innovative status
What differentiates innovative from non-innovative firms?
or firms that introduce different types of innovations?
Products
Processes
Organisational methods
Research on the determinants of innovative status is declining because we have a good idea of the factors that influence this – firm size, sector, markets, and to a small extent country. An pharmaceutical firm in Slovakia, for example, is probably less innovative than an ICT firm in the UK. Of course, the real question is how to get the pharmaceutical sector in Slovakia to move towards higher innovative capabilities, and so far innovation surveys have not collected the right kind of data to look at this problem, partly because of a lack of reliable time series, and partly because it is difficult to identify and follow non-innovative firms (most are small, inconsistently picked up in samples).
Globelics Tampere 2008

7. 2.3 Innovation strategies
Innovation objectives and obstacles.
Little work on this topic, even though obstacles should be of great interest to policy
Effect of public funding on innovation intensity – additive or substitutes?
External sourcing of knowledge.
Collaboration and innovation
Appropriation and intellectual property rights.
Globelics Tampere 2008

8. 2.4 Innovation and performance
Three common measures:
Change in employment
innovation sales share for products (best)
productivity (sales per employee – no value added data; requires linking to other data files)
Plus, patenting status sometimes used (Don’t do it for all sectors – OK within sectors)
Globelics Tampere 2008

9. 2.5 CDM model (Crepon-Duguet-Mairesse)
3-stage model for solving cross-sectional nature of CIS data in analyses of performance
Probit selection model to identify determinants of innovative status (did they innovate, yes or no, but usually R&D)
Research equation (Tobit) for the intensity of innovation (almost always R&D)
Innovation output equation: dependent variable can be innovative sales share, patents, etc
Note: the model works best with R&D as the measure of innovative status, since R&D data are available in an interval form.
Globelics Tampere 2008

10. 2.6 Some performance results
Knowledge management policies improve patent rates (Kremp & Mairesse, 2004)
Innovation intensity determined by share of highly-skilled employees (Janz & Peters, 2002)
Labour productivity increases with innovation intensity (Janz et al, 2003; Loof, 2004)
Labour productivity in the service sector decreases with innovation intensity (Ferreira and Mira, 2005) .
These are four selected examples from 16 studies. The first finding is interesting, since there is a lot of discussion over knowledge management. The second is no surprise, we would expect innovation intensity to be determined by the share off high skilled employees.
The last two – I have included these because they run in opposite directions. This is frequently the case with econometric results, and we don’t know if the cause is differences in country – the increases are observed in Germany, Austria and Sweden while the decrease is in Portugal; sector differences, the last focuses on the service sector, or to problems with data sets, model formulations etc.
Globelics Tampere 2008

11. 2.7 Appropriation example
Does the use of formal appropriation methods such as patents improve the probability of collaborating?
Illustrates the problems of using an innovation survey to address a policy issue.
This has large policy implications within Europe, because current policy encourages the use of patents. This is particularly relevant for policies that subsidize collaborative research between firms and PRIs – the belief is that patents will encourage this.
Globelics Tampere 2008

12. 2.8 Appropriation options
Use strong formal protection methods (patents, copyright, design registration)
Use ‘strategic’ or non-formal protection methods such as secrecy, first mover advantages, and technical complexity.
This question is of policy interest. There has been a concerted effort, for instance, to strengthen formal IPR rights and also to encourage PRIs to patent more. The latter is thought to improve the ability of firms to collaborate with PRIs, since ownership of the results of such research will be clarified and potentially passed to the collaborating firm. But, there are other alternatives to formal protection methods.
Globelics Tampere 2008

13. Relationship between appropriation methods & collaboration
More studies than not find a positive relationship between formal protection methods (usually patents) and collaboration, suggesting that patents permit collaboration, possibly by providing a means to share ownership of discoveries. However, the results are not consistent. They vary by country and by study. For instance, two studies find that formal protection methods have no effect on collaboration in Germany, whereas the two studies by Czarnitzki find a positive relationship for Germany.
In any case, it is not clear that strong formal protection methods are necessary, since not all studies looked at strategic and formal methods separately. An exception is the study by Bonte and Keilbach, which use a multinomial approach with four different cooperation categories as the dependent variable and which includes both strategic and formal protection methods in each model. This study finds no effect for formal protection methods and a positive effect for strategic methods.
The differences are partly due to the model specifications and partly due to how collaboration is defined – ie. Any collaboration, or limited to specific types of collaboration (with PRIs or customers/suppliers). For example, Cassiman and Veugelers find a positive effect of strategic methods and collaboration with customers and suppliers, but no effect when the collaboration is with PRIs.
Globelics Tampere 2008

14. 2.9 Formal or strategic?
Not clear that strong formal protection methods are promoters of collaboration, since most studies did not look at both methods.
Exception: study by Bonte and Keilbach:
A multinomial model with three different cooperation categories with suppliers or customers as the dependent variable.
Informal cooperation
Formal cooperation
No cooperation
Globelics Tampere 2008

15. Bonte and Keilbach (cont)
Includes both strategic and formal protection methods in each model:
Strategic protection methods (aggregation of ordinal scores for secrecy, lead time, complexity)
Industry level measure of scores for formal protection method (patents, copyright, brand names)
Compared to ‘no cooperation’, formal protection methods has no effect on either formal or informal collaboration.
Compared to no cooperation, strategic protection methods have a strong positive effect on collaboration.
Globelics Tampere 2008

16. Bonte and Keilbach (cont)
Comments:
Results only applicable to vertical supply chains (suppliers and customers). These could differ from cooperation with competitors, universities, etc.
Not clear why variable for formal protection methods (patents) is entered as an industry dummy.
No data on if value of formal or informal protection methods are directly linked to cooperation – only firm averages.
Globelics Tampere 2008

17. Veugelers & Cassiman, 2005
Focus on collaborations between firms and universities
Same construction of formal and strategic variables as with Bonte & Keilbach (strategic at firm level, formal at industry level)
Strategic protection methods have no effect, whereas formal protection methods have a strong positive effect
Globelics Tampere 2008

18. Veugelers & Cassiman, 2005 Comments:
Gives opposite result, but for collaboration with universities – is this the reason?
Using industry level variables for formal protection methods creates a confounding problem:
Sectors with high importance given to patent protection have a high probability of collaborating with universities: pharmaceuticals and ICT
Do not know if formal protection actually used with universities
Is the driver of collaboration related to formal protection, or is formal protection really relevant?
Globelics Tampere 2008

19. 2.10 Why do we find conflicting results?
Country effects (National innovation systems)?
Over-interpretation of the data
No direct link between the value of appropriation to the firm (a market variable) and the use of patents and strategic methods as enablers of collaboration
Different model formulations? – problem with using cross-sectional data to explore causation.
Both effects could be occurring, depending on sample, model, control variables, etc.
Novel process innovations are NOT captured by the CIS – a difficult problem, since we do not know if firm managers know if their innovation is novel.
Question 6 on knowledge sourcing does not assume informal sources – it could include information obtained from collaboration.
Globelics Tampere 2008

20. 3. Lessons for survey-based research
What do these two examples teach us about research methods or how to interpret published papers using survey data?
Globelics Tampere 2008

21. 3.1 Main problem
Conflicting results from study to study on some issues of major interest (appropriation, performance, etc).
Creates ‘unease’ among policy makers – what to believe?
Globelics Tampere 2008

22. 3.2 Limitations of surveys
There is a “long standing tension between [innovation surveys], with their advantage of generality but lack of depth, versus case study methods, which offer richness at the expense of generalizability”.
Keith Smith (2004)
Need in-depth research to explain the ‘why’ of innovation survey results
CAUTION: No one would claim that innovation surveys are ever enough.
Often the best approach is to combine both statistical analysis, including innovation survey results, with in-depth case studies that can explain the ‘why’. In the two examples given here, we know that there is a correlation between patenting and collaboration in many, but not all studies, but the CIS does not tell us why. The ‘why’ is answered through assumptions about firm motivations. This is often inadequate, since these assumptions can be wrong.
To answer the ‘ why’, we often need more focused surveys or use a combination of survey research and in-depth case studies, with the latter based on face-to-face interviews with the management of firms.
Globelics Tampere 2008

23. 3.3 Hazard of over interpreting data
Trying to fit a square peg into a round hole is a common problem with innovation survey research – theory driven analysis can conflict with the actual question.
If you use published papers on survey data in your research, find the original survey questionnaire and check how the author’s have interpreted the questions.
Also look for data on the reliability of the survey.
The most common problem I faced when asked to review papers for publication that are based on survey data.
Example of research on informal versus formal knowledge flows from universities to firms:
Formal – collaboration
Informal – importance of universities as a source of knowledge, but the CIS question is NOT limited to informal methods – responses will also include perceptions on formal methods.
Globelics Tampere 2008

24. 3.4 Carefully sift through data
Both positive and negative results can be limited to specific sectors or conditions.
Break up analyses to identify the drivers of specific results – are they consistent across sectors or firm size classes?
Note: Controlling for sectors or size classes in a model assumes no interaction effects, which may not be true.
What methods did the authors use to evaluate their data?
Interaction effects can be controlled in data analysis, but the results are simpler to interpret through separate regressions or cross-tabulations – for instance, conduct separate analyses of low, medium and high tech sectors, or for small, medium and large firms.
Globelics Tampere 2008

25. 4. Future research areas
Globelics Tampere 2008

26. 4.1 Heavily explored topics:
Knowledge sourcing
Collaboration
Exception –with competitors
Effect of R&D status on performance
Appropriation (patents, etc)
Role of ‘public science’
If you look at these topics, you need something new!
Globelics Tampere 2008

27. 4.2 ‘Unexplored’ topics
What factors hinder innovative firms in their ability to innovate?
Effects of innovation on “non economic” outcomes:
Environmental
Market strategies
Process flexibility
Interaction between different types of innovative activities (product, process, organisational) on firm performance
Process innovation and performance
Innovation among non-R&D performing innovators
Role of markets – domestic, international etc.
Globelics Tampere 2008

28. A note on environmental innovation
Are firms reacting to policy signals?
Globelics Tampere 2008

29. A few slides on neglected innovators
Main results of study for non-R&D innovators:
Their activities are similar to R&D performers: 71% develop innovations in-house compared to 91% of R&D performers.
They are less likely to benefit from innovation support programmes including programmes that do not require R&D.
There is no revenue penalty for not performing R&D.
Globelics Tampere 2008

30. R&D status by change in turnover 2004 to 2006
Source: Arundel, Bordoy & Kanerva, 2008
Results confirmed in an econometric model that controls for size, sector, country, total innovation expenditures, innovative capabilities.
Globelics Tampere 2008

31. Conclusions
Globelics Tampere 2008

32. Five conclusions
Survey data requires appropriate statistical models – be careful about causality and interaction effects!
Do not over interpret your data – assume that respondents answer your questions literally.
Often helpful to supplement survey research with in-depth interviews.
If you are interested in ‘heavily explored’ topics, find something new and of interest.
Still many ‘unexplored’ areas, even using existing survey data.
Globelics Tampere 2008