Meeting of TF1 "Input Harmonisation" 24-25 April 2017 Agenda Item 2.1 Main conclusions from the test results Riccardo.GATTO@ec.europa.eu Eurostat

Overview Tests run before 2015 Results from 2015 tests Conclusions Eurostat

Tests run before 2015 2012 – first tests on the Model Questionnaire (MQ) proposed by TF HMEU: in DE, RO and CZ 2013 – second test exercise on modified MQ Main problems related to translation and wording ES: length and complexity of 1st question of "At work" AT: results positive; "At work" similar to proposed FC HU: any particular problems with "At work"; better catch marginal employment NL: proposals for splitting answers, adding explanatory texts and rephrasing SE: problems with the 1st question of "At work"; "work" difficult to be fully covered by just 1 question Eurostat

2015 EU grant - Romania Test on: V2 vs. V3 vs. current national questionnaire Method: field test Main results: Adjusting the results for the new classification of self-producers, comparison shows that V3 better catchs marginal employment “without doubts, V3 was the preferred version due to its simplicity, shortness and ease of administration. Questionnaire V2 was considered too complicated, with too many filters and questions that are perceived as useless or even annoying” Eurostat

2015 EU grant - Spain Test on: V2 vs. current national questionnaire (V3) Method: cognitive interviewing, focus group Main results: “we can conclude that no evidence has been found that this formulation from the “At work” section works less effectively than the EPA formulation” In the other sub-modules modifications introduced after former tests work generally better Translation and wording should be very well studied at national level Eurostat

2016 without EU grant - France Test on: V2 vs. current national questionnaire (V3) Method: field test Main results: Even if no significant differences in terms of resulting employment rate, clear preference for the V3 approach Question on small jobs works better if used as catching-up question with V3 Better to add response items to ‘Absences’ and ‘Job search methods’ French questionnaire works better for the order of the first 2 questions of ‘Search for work’ Eurostat

2016 EU grant – version to be tested The version tested in the sub-action 4 of the EU grant 2016 exercise is the one developed after the discussion that took place at the June 2016 LAMAS meeting (V4) Questions sequence determined through a written consultation after the June 2016 LAMAS meeting This version has the form of a flowchart (FC) The FC is anyway complemented by a MQ that should not be considered as compulsory, but as guidelines for the translation in national language All NSIs decided to test the MQ as provided Eurostat

2016 EU grant - Germany Method: cognitive interviewing and expert review Main results: Sub module "At work": main issue concerned the 1st question The term “pay or profit” very difficult to translate; preferred “no matter if employee or self-employed”. In contrast to earlier pretests, no signs that this question could fail to cover small or side jobs. All people indicated the correct status according their situation. The 2nd question shows no problem. The 3rd question should be complemented by the word “paid” in German. Sub module "Absences", no major problems were found Eurostat

2016 EU grant - Finland Method: cognitive interviewing and field test Main results: Cognitive test: 1st question of ‘At work’: all respondents seemed to understand the question correctly. Recommendation is to change even if for one hour into at least one hour Field test: 1st question of ‘At work’: confusing mainly due to the sentence “even if it was only for one hour”. No feedback from interviewers asked for AW2 and no problems reported with AW3. Question AW4 often felt as burdening, but 0.8% of population has been retrieved to employment thanks to this question. Eurostat

2016 EU grant - Latvia Method: cognitive and personal interviewing Main results: 1st question of ‘At work’ showed language problems, translation of pay and profit is problematic; rewording needed. Terms 'pay or profit 'and time reference should be better explained; some explanations help in receiving correct answers. But, AW1 works better than the current LV question. AW2 works well. AW3 is too long. AW4 is clear but often considered redundant. Suggestions: add examples; split some questions to get more information on occasional and short-term works; mention for the reference period “during the week from [date] to [date]” (without "Monday" and "Sunday") Eurostat

Conclusions Outcomes are generally positive Flexibility is sometimes needed as regards the number of questions to cover one concept Other problems can be solved at national level by fine tuning the wording Suggestions include examples and explanations, given by interviewers or written in questionnaire Sequence of questions, information contents, routing in questionnaire and filtering did not show any problems Recommendations taken into account to build the new simplified FC approach Eurostat

TF1 is invited to take note of: the results of the tests, which can serve as working tool for the detailed discussion on the sub modules in following agenda items Eurostat