s t a r t Session 4 Methodologies and model questions: households Methodological recommendations for household/individual surveys on ICT usage Christophe Demunter Hartmut Buchow EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services WSIS Thematic Meeting « Measuring the Information Society » February 2005 Geneva IndicatorsIndicatorsIndicatorsIndicators

EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services Information Society Statistics in Internet: EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society Background of the survey  Since 2002, Eurostat coordinates across the EU (and candidate countries) surveys on households’ and individuals’ usage of ICT.  Data collected by national statistical institutes following a Eurostat model questionnaire and methodology  Member States take part on a voluntary basis with financial support from the EU to cover part of the survey costs (2005: 25 European countries)  From 2006 onwards, a European legal act will guarantee full coverage in all EU countries

EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services Information Society Statistics in Internet: EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society Contents of the survey Model questionnaire (2005) includes questions on: available ICT devices at home; frequency and location of computer and internet use; purpose and nature of internet activities; ICT security; e-skills; barriers to computer and internet access. Broken down by: age gender employment situation educational level (ISCED) occupation (ISCO) household type urban / rural area, …

EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services Information Society Statistics in Internet: EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society Contents of the survey: future steps  Move from general questions covering a wide range of topics to more detailed modules on a few selected topics (e.g. for the 2006 survey, a module on e-government may be included)  Lower the frequency of data collection for some questions (e.g. every two years instead of annually)  Remove some questions that have proved to be difficult to collect  Structure of the future surveys (2007 and later) may partly be defined by the new benchmarking indicators for the EU (the list of indicators will be finalised by the European Commission by the end of 2005).

EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services Information Society Statistics in Internet: EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society Some general aspects Length of the survey:  Questionnaire consists of ± 25 questions  Survey has ± 125 observation variables  Because of routing and filtering, respondents only need to answer a subset of the questions Type of questions:  Simple: most questions/items only require a “Yes” or “No” answer Major strengths:  Harmonised data collection across Europe leads to highly comparable data (also with third countries via OECD)  Sample is representative for the whole population (aged 16 to 74)

EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services Information Society Statistics in Internet: EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society Scope of the survey  Questions relating to the available equipment are directed at all households with at least 1 member in the age group  All other questions are directed at individuals, where the scope consists of all individuals aged 16 to 74  Lower limit = 16 years  Reason: legal objections to surveying children in some countries  Disadvantage: -16 is an interesting study group for this domain  Upper limit = 74 years  Reason: practical difficulties in collecting data from 74+, sometimes linked to the survey vehicle  Disadvantage: 74+ is an interesting study group for e-Inclusion ?? Future ??- Expand scope to include more age groups? - Move household questions to individual level?

EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services Information Society Statistics in Internet: EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society Data collection method: survey vehicle Stand-alone survey ? or Questions embedded in an existing survey ?  About half of the countries collect the data in a separate survey  About half of the countries add the ICT usage questions to an existing questionnaire (e.g. Labour Force Survey; Income and Living Conditions Survey; general purpose Omnibus Survey, Micro-Census) + ask background variables (age, education, …) only once + reduces costs (e.g. travel costs for interviewers) - very long interviews (high respondent burden, affects quality)

EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services Information Society Statistics in Internet: EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society Data collection method: collection mode Face-to-face interview, telephone interview or postal survey ? 2003 surveyCollection mode DenmarkCATI GermanyPostal survey, self administered. Exception Saxony: CATI GreeceCATI SpainCATI / face to face IrelandFace-to-face ItalyCATI LuxembourgFace-to-face AustriaFace-to-face PortugalFace-to-face FinlandCATI SwedenCATI UKFace-to-face IcelandCATI NorwayCATI

EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services Information Society Statistics in Internet: EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society Sample sizes  Relatively low sample sizes because of the type of questions (Yes/No answers) and the limited level of detail  Does not always give reliable results for very detailed breakdowns or specific sub-groups (due to low cell frequencies)  Some countries increase sample size for national reasons (e.g. policy need for very detailed regional data)  Net sample sizes for individuals in 2004 range from 1455 (Slovenia) to (Italy)  Sampling rates for individuals in 2004 range from 0,007% (1 in 14000) in the UK to 0,822% (1 in 120) in Iceland  Average sampling rate for individuals (2004): 0,053% (1 in 1900 persons)

EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services Information Society Statistics in Internet: EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society Sample sizes (2004 survey) CountryPopulation size (16-74)Sample size Sampling rate DK (Denmark) % DE (Germany) % EE (Estonia) % EL (Greece) % IT (Italy) % CY (Cyprus) % LV (Latvia) % LT (Lithuania) % LU (Luxembourg) % HU (Hungary) % AT (Austria) % PL (Poland) % PT (Portugal) % SI (Slovenia) % FI (Finland) % SE (Sweden) % UK (United Kingdom) % IS (Iceland) % TR (Turkey) % Total for the above countries %

EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services Information Society Statistics in Internet: EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society Response and non-response  Countries’ experience show that the final response rate is on average around 75%  When aiming at a net sample (= number of elements in the final database) of e.g individuals, it may be necessary to select 4000 individuals.  Main reasons for non-response are refusals to participate and non-contacts (respondents cannot be reached)  Risk of biased results: non-users of ICT may be more inclined to refuse participation because they do not feel concerned by the subject of the interview  Importance of imputation (e.g. post-stratification methods to improve the representativeness of the estimates)  Item non-response is low (excepting for a few complex questions)

EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services Information Society Statistics in Internet: EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society Response and non-response Overview of response patterns for some participating countries: Source: 2004 Methodology Reports (2004 survey) DKLTHUSIFISEUKISNO A. Gross sample size a) Ineligible: out-of-scopen/a b) Other ineligible B. Number of eligible elements = (A – a – b) c) Non-contact d) Refusal e) Inability to respondn/a f) Rejected interviews g) Other non-responsen/a C. Net sample size = (B – c – d – e – f – g) D. Unit response rate = (C / B)72%89%61%81%76%67%60%84%64%

EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services Information Society Statistics in Internet: EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society Methodological reporting  Participating countries are requested to send a methodological report to Eurostat  Since 2004, a template is used which makes it easy to compare the methodologies across countries  Contents:  Survey period, reference period, survey vehicle, survey mode, etc.  Statistical units used, age groups used, territorial coverage  Comparison of national questionnaire to Eurostat model questionnaire  Discussion of sampling frame and sampling design  Response and non-response (see previous slide)  Grossing-up procedures  Sampling error: coefficients of variation for selected indicators  Annexes: analysis of results, questionnaires, interviewer instructions, …

EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services Information Society Statistics in Internet: EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society Methodological Manual  Eurostat is currently preparing a manual for the NSI’s who already carry out the survey or for NSI’s who wish to collect ICT usage statistics  Improve harmonisation by offering recommended guidelines  First “edition” to be released in Spring 2005 (electronic format)  Contents:  Detailed description and explanatory notes for the questions in the models, definitions used, expected output, etc.  General outline of the survey: reference and survey periods used, scope of the survey (target population), etc.  Methodological recommandations in terms of sampling design, data collection methods, questionnaire structure, data processing, estimation methods, non-response treatment, quality measurement, etc.

EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-7: Information Society and Services Information Society Statistics in Internet: EUROSTAT website -> Data -> Database -> Themes -> Industry, trade and services -> Information society ICT usage data for the European Union  Since October 2004, the results of the ICT usage surveys are available via the New Cronos database which can be accessed free of charge on Eurostat’s website:  Go to:  Then go to « Data » - « Database » - « Themes »  Then select « Information Society Statistics » under one of the following themes:  Population and social conditions  Industry, trade and services  Science and technology