1. Establishing educational standards and monitoring student performance Directions for methodological improvements in international assessments

2. Key methodological features 1.Focus on literacy and using knowledge 2.Collection of range of information (attitudes, school-level factors, etc) other than performance data 3.Use of IRT to construct achievement variables 4.Multiple matrix sampling of items 5.Use of a range of item types and authentic assessment tasks 6.Standards-referenced reporting using 6 levels 7.Multilevel modelling to identify student/school/system effects 8.Two-stage approach to reporting results

3. Areas for improvement What/who is assessed? How it is assessed? What other information is collected? How the data are analyzed? How the results are reported, disseminated and used?

4. What/who is assessed? What other information to be collected? More open- ended items; Importance of trend and monitoring progress over time; More number of items which requires various type of learning strategies; Classroom environment: “what happen in the classrooms?” (e.g. use Video); Students’ social skill, emotional aspects, creativity, social-cooperation skill; Teachers’ perception of students performance; Teachers’ classroom teaching strategy; Testing 1year younger or older students: Value-added of 1 year of schooling; Longitudinal study; Targeting different age group; Foreign language skill; “How much and what skill/ability retain after few years?”; Conception of teaching: “what is teaching?” (e.g. guiding or facilitating); Conception of learning: “what is learning?”.

5. How the data are analysed Scaling: need caution for bottom and top ends which are instable; The relationship between combined-scale score and sub-scale scores.

6. Reporting, dissemination and use of results Feedback to teachers and schools to improve classroom teaching`; Media makes the most use of PISA results. But, often only focus on ranking. Need to avoid over simplified interpretation; Concrete national analysis/reports which goes beyond the international report; Need secondary study and in-depth study (e.g. case study or ethnography); Further involvement t of high level officials; Importance of dissemination seminars to understand other countries reaction to PISA;. Further use of PISA database and technical reports for capacity building of researchers and students; Reaction of students who participated in PISA;. Feedback to students and schools which participated in PISA; PISA reports are often too intensive. More concise results need to be prepared to disseminate to teachers, schools and parents.

7. Others Link the PISA scale/proficiency level to the national assessments; Link to other international studies such as PIACC.