1. © Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Business Surveys – Testing Strategies in the FSO (GER) QUEST WORKSHOP 2007, Ottawa 23th-26th April Karen Blanke

2. © Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Folie 2 Survey on Trade Statistics Sample size: 55.000 enterprises Data collection mode: paper/pencil or electronical questionnaire, self-administered Topics: Structure and amount of employees Stock and purchase of goods Investment and subsidies Turnover of the financial year

3. © Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Folie 3 Survey Form: Trade Statistics

4. © Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Folie 4 Feedback on Trade Statistics: By the enterprises: Burdensome Useless – does not reflect reality Relevance of data By the public: Timeliness Inconsistency with other findings

5. © Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Folie 5 Basic problems of business surveys: Availability of data (record formation) Willingness to participate Respondent selection Retrieval and judgement (knowledge of data sets and access) Reporting – (statistical and accounting categories) Authority to provide data

6. © Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Folie 6 Testing principles: Aim of testing: No total revision Resources: 2 collaborateurs Focus: Check different perspectives of users Use what is available Focus on respondents‘ views

7. © Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Folie 7 Conceptual frame of testing: Conceptual testing: Comparison of definitions (accounting system and official statistics) Quantitative methods: Evaluation of data sets Main Hotline requests Qualitative methods Discussions with experts and data entry staff Company-site-visits

8. © Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Folie 8 Preliminary findings (I): Data sets: comparison raw data vs. final data: Highest rate of correction does not correspond with problems noticed by respondents Over and underestimation of corrections No evidence of the reason of unreliable data

9. © Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Folie 9 Preliminary findings (II): Problems encountered via callbacks and hotline: Interrelated errors Implausible data Via hotline: case-specific problems, less general problems

10. © Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Folie 10 Preliminary findings (III): Discussion with experts: Experts are quite convinced of their concepts They provide feedback on general concepts, need of specific items Provide helpful knowledge on organisational issues, layout aspects for data entry

11. © Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Folie 11 Preliminary findings (IV): Discussion with data entry staff : Sensitive on layout issues Order and structure of questionnaire Accepted level of detail by respondents Check of interrelated errors

12. © Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Folie 12 Preliminary findings (V): Company-site-visits : Companies are difficult to recruit and less is more Main problem: Companies are overloaded by surveys Not only companies are different but also the respondent in the companies Instructions are rarely in use; respondents prefer their own definitions However: Instructions and definitions are needed

13. © Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Folie 13 Discussion Relevance of quantitative and qualitative testing in business surveys What are the most helpful business survey testing methods? Recruitement of enterprises Laboratory tests in business surveys Aspects of visual design, e.g. 3 columns design (NL) Treatment of instructions (length, location )

14. © Statistisches Bundesamt, Institute for Research and Development in Federal Statistics Statistisches Bundesamt Thanks for your attention Karen.Blanke@destatis.de