Download presentation

Presentation is loading. Please wait.

Published bySkyler Savage Modified over 2 years ago

1
22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit van Amsterdam AMSTEL Instituut

2
2 International comparison Use identical instruments in different countries and look at differences between countries. My own international experience: Went to school in NL, D and F worked in NL, Zimbabwe, Mozambique & South Africa Worked at the Dutch research centre for TIMSS (University of Twente) PhD on mathematics curriculum in NL using TIMSS-instruments

3
3 Many variations Bilateral studies Affect in science: ROSE (Relevance of Science Education), 40 countries Teacher competencies: TEDS(M), 17 countries

4
4 Two abbreviations… PISA = Project for International Student Assessment organised by OECD (Organisation for Economic Cooperation and Development) TIMSS = Trends in International Maths and Science Study previously: Third Internat. Maths & Science Study organised by IEA (Internat. Assoc. for the Evaluation of Educational Achievement)

5
5 Translate and define Mathematics Basic calculations and further Science physics, chemistry, biology, physical geography, astronomy, ecology, etc Students all students (not just pre-university)

6
6 NLUSAFlaJapMarocRuss

7
7 NLUSAFlaJapMarocRuss

8
8 NLUSAFlaJapMarocRuss

9
9 NLUSAFlaJapMarocRuss

10
10 NL36 USA19 Fla21 Jap44 Maroc 2 Russ. 9

11
11 NLUSAFlaJapMarocRuss

12
12 Some history… 1964 A comparison of 12 countries, Husén & Bloom 1971 FISS SIMS 1984 SISS 1995 TIMSS (later known as TIMSS-1995 ) 1999 TIMSS-Repeat (later known as TIMSS-1999) 2003 TIMSS TIMSS TIMSS Advanced TIMSS PISA PISA PISA-2006 PISA-2009

13
22 April 2009 To what extent do countries differ in the education of science and mathematics? How can you do research on the question:

14
14 Hours of instruction for science (grd 4)

15
15 Teacher characteristics math (grd 8)

16
22 April 2009 Both studies focus on a test

17
17 Complex studies.. Assessment framework Large number of items Long design process for instruments Large random sample Quality control trend items (retained) coding and re-coding excluding countries when criteria were not met

18
18 Pisa framework for science

19
19 Pisa framework for science competencies

20
20 Pisa framework for science

21
21 Pisa framework for science

22
22 Pisa science item Acid Rain

23
23 Pisa science item Acid Rain

24
24 Pisa science item Acid Rain

25
25 Complex studies… (1) Not just measuring country scores in one discipline, but also: Sub-disciplines (algebra, ecology,..) Sub-competencies (reproduce, explain, communicate,..) Background variables Not just measuring students scores, but also trying to explain: Differences between countries Differences between schools Differences between students

26
26 Complex studies… (2) PISA: 85 exercises distributed over 13 test booklets PISA-2000 Main topic: Literacy (reading) Additional: science, maths PISA-2003: Main topic: Mathematical Literacy Additional: literacy, science, problem solving PISA-2006 Main topic: Science Additional: literacy, maths

27
27 Complex studies… (3) TIMSS: 300 exercises distributed over 12 booklets TIMSS-95: grades 3, 4, grades 7, 8 Performance Assessment (= test in a lab-like environment) grade 12: math/science literacy; advanced maths/science video study (USA, Jap, Germ.) TIMSS-99: grade 8 TIMSS-R Video study (USA, Jap, NL, Cz, Switz, Austr, HongK.) TIMSS-2003 grade 4, grade 8 TIMSS-2007 grade 4, grade 8 TIMSS-2008 grade 12

28
28 Percentage Distribution Math Items PISA - TIMSS 15-year-olds PISA Grade 8 TIMSS Number 22%33% Measurement 18%14% Geometry 12%18% Data 40%11% Algebra 11%23% Classified to multiple strands 2%0%

29
29 Math Item Formats Multiple Choice Short Constructed Response Extended Constructed Response TIMSS71%16%13% PISA*33%42%25% * PISA definitions: Multiple Choice includes regular and complex multiple-choice items. Short Constructed Response includes short answer and closed response items. Extended Constructed Response includes open response items.

30
30 Mathematical Complexity LowModerateHigh TIMSS51%46%3% PISA29%64%7%

31
31 PISA-2006 science country ranking

32
32 PISA-2006 science country ranking

33
33 TIMSS country ranking Maths grade 4

34
34 TIMSS-2007 multi-country comparison

35
35 Horse race - thinking both in PISA and TIMSS Ranking is simplistic Sample: intrinsically inaccurate Scores of some countries are very close. With s.e.>3 are small differences not significant NL position in 2003: math: on shared positions 2-7 in PISA and shared positions 6-9 in TIMSS science: on shared positions 4-8 in PISA and shared positions 7-11 in TIMSS

36
36 Differences PISA and TIMSS Population: PISA: age cohort: 15-year olds (indep. of grade) TIMSS: grade (indep. of age) Countries: PISA: economically developed (OECD) TIMSS: whole world (incl. Saudi Arabia, Botswana) Cyclus: PISA: 3-yearly with different thematic foci TIMSS: 4-yearly with different populations Content: PISA: applicable knowledge (situations): reading, math. literacy, scientific literacy TIMSS: school knowledge (US curriculum): mathematics, science Justification of the study: PISA: knowledge has economical importance TIMSS: knowledge about education is needed

37
37 Country score and GDP

38
38

39
39 TIMSS-1999 Video Study Grade 8 seven countries: Australia, Hong Kong, Japan, NL, Czech Rep, USA, Switzerland Aim: describe patterns in lessons question: what is typical for the math lessons in a certain country? in each country 85 random lesson filmed, transcribed & coded Points of analysis: Structure of the lesson: e.g. who speaks? How many exercises? Mathematical content: complexity, procedures Char of exercise: symbols, contexts

40
40 TIMSS-Video study: use of materials: Percentage lessons in which a calculator is used

41
41 TIMSS-Video study: structure of lessons Percentage lessons in which a goal is stated

42
42 TIMSS-Video study: structure of lessons Percentage lessons, in which a summary statement is made

43
43 TIMSS-Video study: exercises Percentage exercises per lesson with a context of symbols only

44
44 TIMSS-1999 Video Study USA video USA Grade 8 (14 yrs) Heterogeneous non-problem segment: the teacher shows the class examples of angles formed by secants and tangents intersecting inside, on, or outside a circle.

45
45 TIMSS-1999 Video Study Netherlands video NL Grade 8 (14 yrs) Homogeneous (pre-university) non-problem segment: the teacher briefly presents historical background related to the Pythagorean theorem (i.e., Pythagoras was a Greek gentleman). She also connects ideas learned previously (i.e., the definitions of various types of triangles) to the present lesson.

46
46 TIMSS-1999 Video Study Hong Kong: grd 8, heterogeneous Explanation of a new concept. video HongKong Pre-knowledge: students can solve linear equations What preceded the scene: Solving 2x+10=2(x+5) 2x+10=2x+10 0 = 0 Then different values were insterted for x. left hand side right hand side expanded form factorized form

47
47 Next in this theatre… TEDS(M) report in December 2009 TIMSS-2008 Advanced report in December 2009 Pisa 2009 report in 2010

48
48 References for PISA and TIMSS

49
49 remains the question…. what are the benefits and limitations?

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google