Presentation on theme: "Www.uis.unesco.org Innovation data collection: Methodological procedures & basic forms Regional Workshop on Science, Technology and Innovation (STI) Indicators."— Presentation transcript:
Innovation data collection: Methodological procedures & basic forms Regional Workshop on Science, Technology and Innovation (STI) Indicators for Gulf countries Doha, Qatar 15 to 17 October 2012
Ch. 8 OM - Survey procedures Guidelines - collection and analysis of innovation data; Comparable results over time and across countries; Particular circumstances may require other methodology comparability.
The survey approach The subject approach: Innovative behaviour and activities of the firm as a whole; The object approach: Specific innovations (significant innovation of some kind, firms main innovation).
Populations (1) The target population: The target population: Innovation activities in the business enterprise sector (goods-producing and services industries); Minimum: all statistical units with at least ten employees: Small: 10-49; Medium: ; Large: 250 and above.
Populations (2) The frame population: The frame population: Units from which a survey sample or census is drawn; Based on the last year of the observation period for surveys; Ideal frame = up-to-date official business register NSOs; If the register forms the basis for several surveys (innovation, R&D, general business), the information can be restricted to innovation.
Survey methods (1) Mandatory Mandatory surveys increase response rates; Census or sample surveys? Census or sample surveys? Sample surveys - representative of target population (industry, size, region) stratified sample; Census - costly but unavoidable in some cases.
Survey methods (2) Domains (sub-populations): Domains (sub-populations): Subsets of the sampling strata; Potential sub-populations: industry groupings, size classes, regions, units that engage in R&D and innovation-active; Guidelines: » Same statistical units and classifications; » Consistence of the methods for results calculation; » Documentation of deviations in data treatment or differences in the quality of the results from the domains.
Survey methods (3) Sampling techniques: Sampling techniques: Stratified sample surveys: size and principal activity; Sampling fractions should not be the same for all strata; Cross-sections: Cross-sections: standard approach - new random sample for each innovation survey; Panel data: Panel data: alternative/supplementary approach.
Survey methods (4) Suitable respondents: Suitable respondents: Methods: e.g., postal surveys, web-based questionnaires, personal interviews; Units most suitable respondent - very specialised questions that can be answered by only a few people; Try to identify respondents by name before data collection starts. Try to identify respondents by name before data collection starts.
Survey methods (5) The questionnaire: The questionnaire: Pre-test; Simple and short; Order of the questions; Questions on qualitative indicators - binary or ordinal scale; International innovation surveys - attention to translation and design; Short-form questionnaires - units with little/no innovation activity previously reported.
Survey methods (6) Combination of Innovation and R&D surveys: Combination of Innovation and R&D surveys: Reduction in the overall response burden; Scope for analysing the relations between R&D and innovation activities; Increase in the frequency of innovation surveys; Country experiences - it is possible to obtain reliable results for R&D expenditures; Longer questionnaire; Units not familiar with the concepts of R&D and innovation may confuse them; Different frames for the two surveys.
Survey methods (7) Guidelines for conducting combined surveys: Guidelines for conducting combined surveys: Questionnaire: two distinct sections; Smaller individual sections; Comparison of results from combined and stand-alone surveys should be done with care - surveying methods should be reported; Samples extraction from a common business register.
Estimation of results (1) Weighting methods: Weighting methods: Weighting by the inverse of the sampling fractions of the sampling units, corrected by the unit non- response; If a stratified sampling technique with different sampling fractions is used, weights should be calculated individually for each; Based on the number of enterprises in a stratum; International and other comparisons: same weighting method.
Estimation of results (2) Non-response: Non-response: Unit non-response: reporting unit does not reply at all; Item non-response: response rate to a specific question - % of blank or missing answers; biased results » Disregarding missing values + applying simple weighting procedures based on the responses received assumes that respondents and non-respondents are distributed in the same way biased results; imputation methods » Possibility: imputation methods.
Presentation of results Descriptive analysis: no generalisation of results; Inferential analysis: conclusions about target population; Variance for the results: (average) values for innovation indicators and their coefficients of variation and/or confidence intervals; Results presentation: metadata (data collection procedure, sampling methods, procedures for dealing with non-response, quality indicators).
Frequency of data collection Every 2 years; If not economically feasible frequency of 3 or 4 years; Specify an observation period; The length of the observation period for innovation surveys should not exceed 3 years nor be less than 1 year.
Annex A - 5. Methodological issues for developing country contexts (1) Information system specificities: Relative weakness of statistical systems: » Absence of linkages between surveys and data sets; » Lack of official business registers; Involvement of NSOs; When lacking, basic variables about firms performance can be included in the innovation survey.
Annex A - 5. Methodological issues for developing country contexts (2) General methodological considerations: Survey application: » In-person; » Trained personnel; Questionnaire design: » Sections can be separated to allow different persons in the firm to reply them; » Guidance/definitions; » Language and translation of technical terms.
Annex A - 5. Methodological issues for developing country contexts (3) General methodological considerations: Frequency: CIS CIS » Every 3 to 4 years (e.g., timed to CIS rounds);CIS » Update a minimum set of variables every year; Purpose of surveys; Clear questions; Adequate legislative base; The results should be published and distributed widely. The results should be published and distributed widely.
Example - product innovation/degree of novelty
Example - innovation activities and expenditures for product and process innovations
Example - co-operation
Example - hampering factors
Example - organisational innovation
Basic forms - CIS The Community Innovation Survey (2010) (CIS 2010): The Community Innovation Survey (2010) (CIS 2010): 1. General information about the enterprise 2. Product innovation 3. Process innovation 4. Ongoing or abandoned innovation activities for pdt/pcs innovation 5. Innovation activities and expenditure s for pdt/pcs innovation 6. Sources of information and co-operation for pdt/pcs innovation 7. Objectives for pdt/pcs innovation 8. Factors hampering pdt/pcs innovation activities 9. Organisational innovation 10. Marketing innovation 11. Creativity and skills 12. Basic economic information on your enterprise
Basic forms - AU/NEPAD AU/NEPAD Standard Innovation Questionnaire: AU/NEPAD Standard Innovation Questionnaire: 1. General information about the enterprise 2. Product innovation 3. Process innovation 4. Ongoing or abandoned innovation activities for pdt/pcs innovation 5. Innovation activities and expenditure s for pdt/pcs innovation 6. Sources of information and co-operation for pdt/pcs innovation 7. Effects/Objectives of pdt/pcs innovation 8. Factors hampering (pdt/pcs) innovation activities 9. IPRs 10. Organisational and m arketing innovation 11. Specific innovations by your enterprise