1. Photis Stavropoulos AGILIS SA
Draft quality report of the implementation of the 2003 LFS ad hoc module on lifelong learning
Photis Stavropoulos
AGILIS SA
ETS-WG meeting, 31/01-02/02, 2005

2. Introduction
30 countries included an ad hoc module on Lifelong Learning (LLL) in the 2003 Labour Force Survey (LFS)
A draft quality report about the implementation of the module has been prepared, based on information provided by the countries until 7/1/2005
This talk presents the main findings contained in the draft report
The final report is scheduled for 31/3

3. Overview
Collection of information about the implementation of the module
Main aspects of the implementation of the module
Draft quality assessment
Planned improvements for the final quality report

4. Information collection
Countries submitted MEGs, Grant reports, Self evaluation reports, etc.
A metadata questionnaire was prepared
Questionnaires were pre-filled with submitted information and sent back to countries
Questionnaires returned until 7/1/2005 and available reports were used for the preparation of the draft quality report

5. Implementation of the module
Adoption of the questionnaire
Most variables were adopted as defined in Commission Regulation 1313/2002
A few of them however were either modified or not included at all, e.g.
HATFIELD, LLLFIELD were considered covered by HATFIELD and EDUCFIELD of the core questionnaire (ES, FI, SE, DK, HU, IT)
Registers were used for LLLLEVEL and LLLFIELD (NO) and for HATFIELD (SE and FI)
UK constrained questions about non-regular education to the most recent activity and Norway to the most recent job-related activity
Most extended modifications in Norway, UK and France

6. Implementation of the module
Target population
Regulation 1313/2002: persons aged 15 years and over
Exceptions
15-74: DK, EE, FI, HU, LV, NL SE,
16-74: CH, IS,
16 - … : ES, UK,
15-64: BE
Norway used a target population of 15 to 59 years for questions on regular education (registers for older persons) and 16 to 74 for learning outside the regular education system.
Sampling scheme
Two-stage stratified sampling of individuals or households in the majority of cases

7. Implementation of the module
Mode of data collection
PAPI or CAPI, often with CATI
Only CATI in the Nordic countries
Postal survey (Denmark, in addition to CATI)
Proxy interviews allowed in most countries
Response rates

8. Implementation of the module
Reported users and uses of the module’s results
Users: Ministries, media, academia, research organizations
Uses: Indicators on LLL, linkage of information on training activities with information on their cost, comparisons at international level
Dissemination of results
Publications in paper and electronic format, mailing lists, provision of anonymized microdata to researchers

9. Implementation of the module
Main reported problems
Recalling of short term learning activities
Inefficient coding of field of training
High percentage of “General programs” in field-of education questions
Difficulty in distinction between formal and non-formal education
High non-response about taught activities
Inaccurate information
Lack of coherence of results with other sources
High cost of the survey

10. Implementation of the module
Suggestions for improvement, made by countries
Reduce reference period
Limit detailed investigation of taught activities to the most recent one
Ask about duration of taught activities in closed form (pre-defined time intervals)
Include additional checks during data entry
Improve training of interviewers on coding of field of education
Provide more explanations in the questionnaire about the questions

11. Quality assessment Relevance
LFS regulation restricts the size of the module
Consequence: not all potential needs can be covered
Coverage of needs for indicators of LLL
Participation rates and time spent in LLL (covered)
Perceived interest in learning (partly covered - LLLCOURPURP)
Sources of financing and employer support for learning (partly covered - LLLCOURWORH)
Outcome of learning activities (not covered)
Barriers to learning (not covered)
Investment in LLL (not covered)
Conformity of countries with Commission Regulation
Modification or non-inclusion of certain variables
Reduction of target population in Belgium (15-64)

12. Quality assessment Accuracy Direct assessment (variability of results)
Sweden reports rates with s.e. < 1.5% in most cases
Latvia reports main rates of participation in LLL with c.v < 5.85%
Indirect assessment
Coverage errors: reported at below 2%, except EE (5%) and ES (17%)
Measurement error sources: difficulty to distinguish formal from non-formal education, comprehension problems for informal training activities, long reference period, proxy interviews, coding errors during the interview
Processing errors: Coding errors at the statistical agency
Non-response: the rate of non-response was non-negligible in several countries, no information about differences between respondents and non-respondents

13. Quality assessment Comparability Concepts Measurement processes
Variables used: Countries with greatest deviation from the regulation were France, UK and Norway
Statistical unit: Individual and educational activity (common for all countries)
Target population: Major discrepancies were found in Belgium(15-64)
Reference period of results: Sweden, France and Iceland implement the module throughout 2003, extending the reference period of results
Measurement processes
Various sampling schemes, not common data collection mode, difficulties in education field coding

14. Quality assessment Coherence
HATFIELD is in agreement with data from administrative registers in Denmark
LLLLEVEL compared with data from educational institutes in Bulgaria
Overestimates participants up to and including upper secondary education  due to different reference periods
Underestimates participants to higher education  difference possibly within limits of sampling error
LLLCOURWORH compared with Staff Training Survey (STS) in Sweden
Underestimates training financed by the employers
Consistent patterns of participation in employer-financed activities by sex and level of education
Inconsistent differences in rates between subpopulations
LLLLEVEL, LLLCOURLENP, LLLCOURPURP incoherent with core LFS questionnaire

15. Quality assessment Timeliness and punctuality
Distance between end of reference period and time of delivery to Eurostat ranges from 6 to 11 months
Punctuality: Sweden delivered data 2 months after scheduled time, Hungary was on time

16. Accessibility and clarity
Quality assessment
Accessibility and clarity
Users can obtain statistics in the form of results publications (paper and electronic) and anonymized microdata (Ireland, Hungary, Latvia)
Methodological information (Ireland and Hungary)
Information in MEGs, Grant reports, self evaluation reports  not widely available

17. Planned improvements for final quality report
Presentation of data processing and analysis
Detailed description on data editing, treatment of non-response, presentation of the results
Problems and suggestions
The section will be revised with new information
Relevance
More information on user needs, users and uses of the results of the module
Accuracy
Coefficients of variation and non-sampling errors
Comparability
Visual representations of comparability using multidimensional scaling and clustering techniques on the complete metadata
Coherence
Comparison of the EU results and similar EU-wide results

18. … your comments are welcome