European Module Field Trial outcomes and Plans for Main Study National Research Coordinators Meeting Windsor, June 2008

NRC Meeting Windsor 9-12 June 2008 Content of presentation Purpose and background Field trial participation Field trial outcomes Draft questionnaire for the MS

NRC Meeting Windsor 9-12 June 2008 Purpose The purpose of the European Regional Module is to investigate specific Europe-related aspects to be derived from the domains of the IEA ICCS Civic and Citizenship Framework

NRC Meeting Windsor 9-12 June 2008 Background (1) European Module paper (NRC1) European Module Expert Group Paper (EC/CRELL/CoE) (concept map) Country contributions Langley consultation meeting (Version 1)

NRC Meeting Windsor 9-12 June 2008 Background (2) NRC Rome meeting (Version 2 review) Small-scale pilots and pilot review (Version 3) Field trial preparation (Version 4) Field trial Field trial outcomes (Version 5)

NRC Meeting Windsor 9-12 June 2008 EM – Field trial participation Field trial analysis: data sets from 20 countries Students with data 458 Schools 25 Students per school 21 Countries participated

NRC Meeting Windsor 9-12 June 2008 EM – Field trial participation Country Nr of students with valid data Number of schools Students per school BFL BGR CHE DNK ENG ESP62625 EST FIN GRC IRL ITA LTU LUX LVA MLT NLD POL SVK SVN SWE Total

NRC Meeting Windsor 9-12 June 2008 EM test – Field trial outcomes Overall findings: - no single underlying dimension of civics and citizenship knowledge (p.6) Recommendations made: -are based on merits of individual items considered independently -acknowledge the issue of overlong instructions

NRC Meeting Windsor 9-12 June 2008 EM test – Main recommendations Reduced set of items –Knowledge about the EU –Wording, relevance, content validity Re-wording -eg. meaning of ‘European’ issue

NRC Meeting Windsor 9-12 June 2008 EM test – Main recommendations Revised instructions, inc. knowledge self-assessment battery now in questionnaire Revised, shorter introduction about content → Detailed explanations about Europe to precede questionnaire

NRC Meeting Windsor 9-12 June 2008 EM test – Main recommendations PAC discussions Valuable to have items about basic knowledge CU205C1 not so valuable The official language of the European Union is English.

NRC Meeting Windsor 9-12 June 2008 EM test – Main recommendations PAC discussions Retain CU201C4, if re-worded: All EU countries have signed up to the European Convention on Human Rights. Retain CU201C4 One aim of co-operation between European countries is to ensure that all people in Europe can live in peace.

NRC Meeting Windsor 9-12 June 2008 EM questionnaire – FT Outcomes Instrument worked well, generally - Most batteries → reliable scales Informed the (de-)selection of items for Main Study

NRC Meeting Windsor 9-12 June 2008 EM quest. – Main recommendations Reduced sets of items Cautious reduction Aim for Main Study: reliable scales constructs measured by good number of items

NRC Meeting Windsor 9-12 June 2008 EM quest. – Main recommendations Participation in European-related activities √ Keep all items Openness to other European countries/cultures √ Keep all items

NRC Meeting Windsor 9-12 June 2008 EM quest. – Main recommendations Interpersonal communication behaviours √ Keep items that scaled PAC discussions ► Keep also ESP03D: Discussing European sports events with your friends and family.

NRC Meeting Windsor 9-12 June 2008 EM quest. – Main recommendations Knowledge of European foreign languages ► Simplify language proficiency scale to 1 item Attitudes to learning modern European foreign languages √ Keep items that scaled

NRC Meeting Windsor 9-12 June 2008 EM quest. – Main recommendations European self-identity √ Keep items that scaled and add I feel part of Europe from international instrument PAC discussions ► Have as first battery

NRC Meeting Windsor 9-12 June 2008 EM quest. – Main recommendations Freedom of travel, settlement and work within Europe √ Keep all items

NRC Meeting Windsor 9-12 June 2008 EM quest. – Main recommendations Equal opportunities √ Items scale, but keep all? PAC discussions ► Keep at least: a) ethnic and racial background b) religion or beliefs c) language spoken f) rich/poor country origin h) level of education (but lowest loading)

NRC Meeting Windsor 9-12 June 2008 EM quest. – Main recommendations Organisation of European countries √ Keep all items that scale ► Rephrase environment item PAC discussions Shorten item ES2P11B: All European countries should have the same approach when it comes to their relationships with countries outside Europe.

NRC Meeting Windsor 9-12 June 2008 EM quest. – Main recommendations EU enlargement √ Keep items that scale Self-confidence ► Deleted Knowledge self-assessment ► Updated to reflect test content

NRC Meeting Windsor 9-12 June 2008 EM instruments – Layout Field Trial Issue (SAQ) Layout back-to-front & upside down (8 countries) PAC discussions Main Study Ordinary layout with administration note about students not being allowed to revisit test

NRC Meeting Windsor 9-12 June 2008 EM – Time TestQuestionnaire Field Trial 30 mins. 10 minutes 36 items 20 minutes 101 items Main Study 20 mins. 5 minutes circa 15 items 15 minutes circa 75 items

NRC Meeting Windsor 9-12 June 2008 EM test – additional items European Parliament (CU207C1-6) Every European Union country has the same number of representatives in the European Parliament. European Parliament groups are organised by political background/affiliation, rather than by country. OR Members of the European Parliament ( ) are organised in groups. What are these groups based on? country region policy areas political party

NRC Meeting Windsor 9-12 June 2008 EM test – additional items EU membership – historic (CU205C7) How many countries was the original group of EU members composed of? OR The European integration process started with the European Coal and Steel community. How many countries signed this agreement in 1952?

NRC Meeting Windsor 9-12 June 2008 EM test – additional items Keep, re-word, replace: CU207C3  European Union member states have signed up to a European Constitution. CU201C2  If you have a passport of a country of the European Union, it states both your country of citizenship and ‘European Union’. CU201C4: aims of co-operation between EU countries

NRC Meeting Windsor 9-12 June 2008 EM test – additional items Re-instate CU206C1: about car number plates having EU symbol CU206C4: about environmentally friendly farming methods

NRC Meeting Windsor 9-12 June 2008 EM instruments - Discussion Your views on proposed: -Test focus & items -Questionnaire scales & items -Instructions -Layout EM instruments - Discussion Y -T-T -Questionnaire scales & items -Instructions -Layout

NRC Meeting Windsor 9-12 June 2008 SUMMARY additional test items European Parliament: number of country seats and parliamentary organisation. EU membership – historic European Constitution Passports of European Union countries Aims of co-operation between EU countries Car number plates having EU symbol Environmentally friendly farming methods

NRC Meeting Windsor 9-12 June 2008 Questions and comments?