Download presentation
Presentation is loading. Please wait.
Published byCynthia Byrd Modified over 10 years ago
1
Assessment and Evaluation of the Israeli Education System
Michal Beller Director-General of RAMA The National Authority for Measurement and Evaluation in Education, Israel April 2013
2
Assessment and Evaluation Should Provide Support for an Ongoing Process
Baker, 2005
3
Intended Instruction Hours
EAG 2012
4
Teaching Hours EAG 2012
5
Class Size EAG 2012
6
Ministerial Goals
7
Ministerial Goals (2009-2013) Promote educational values
Combat violence and create an optimal educational climate Improve achievement Best utilize available study time Narrow the educational gaps Strengthen the status of teachers and improve the quality of instruction Strengthen the status of principals Focus on core subjects Suit curricula to changing educational reality Raise the number of students in professional-technological education Align the education system to the digital age in the 21st Century Mainstream special needs students into regular education by enhancing the ability to retain them in classes and provide a range of solutions for them
8
Assessment for Learning
RAMA National Authority for Measurement and Evaluation in Education Assessment for Learning
9
RAMA An independent statutory unit - the National Authority for Measurement and Evaluation – RAMA - shall be established as the entity that leads and provides professional guidance to the education system with respect to measurement and evaluation. RAMA will conduct periodic evaluations of the education system and evaluations in schools, and will publish its findings in an annual report submitted to the National Council for Education. From the Dovrat Commission (2004)
10
Improving Student Learning
Ultimately, accountability is not only about measuring student learning but actually improving it Consequently, genuine accountability involves supporting changes in teaching and schooling that can heighten the probability that students meet standards There are at least three major areas where attention is needed: Ensuring that teachers have the knowledge and skills they need to teach to the standards Providing school structures that support high quality teaching and learning Creating processes for school assessment that can evaluate students’ opportunities to learn and can leverage continuous change and improvement Darling Hammond, 2004
11
Campbell's Law "The more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” "achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.” Campbell, Donald T., "Assessing the Impact of Planned Social Change“. The Public Affairs Center, Dartmouth College, December, 1976.
12
Unintended Negative Effects of External High Stakes Testing
Narrowing the Curriculum Between-Subject Reallocation of time The Impact of Setting a Specific Target Teaching to the Test; Test Inflation Assessment is Only a Sample of the Curriculum “Most Children are Left Behind…”
13
Real Estate Map (based on Meitzav)
14
Meitzav League Tables
15
Updating the Israeli Assessment System
The main goals of the recent update: Implement a culture of “assessment for learning” Mitigate the threats of external exams (including gaming the system) Effective integration of internal and external evaluation Decentralization of the evaluation process along with the use of centrally designed rigorous tools Empowerment of teachers and principals Professional design of assessments and scoring (including equating, alignment to curriculum and standards and more)
16
Use of Multiple Indicators
Sample-based Assessments (National & International exams) Internal External Internal admin. of external tests Matriculation exams, Meitzav Formative Assessments, Assignment Data- Base, “Off-the-shelf” Assessments Schools & pupils are being examined
17
Large Scale Assessment in Israel
18
Large Scale Assessment in Israel
Matriculation Exams (“Bagrut”) – end of high school Meitzav – National Assessments (Native Language, Math, Science, English) – 5th and 8th grades (+2nd grade in Language) + Climate & Pedagogical Survey to students and teachers (primary and secondary students, teachers and principals) International Comparisons: TIMSS - Math and Science Study – 9th Grade (1995, 1999, 2003, 2007, 2011, 2015) PIRLS - Reading Assessment – 4th Grade (2001, 2006, 2011, 2016) PISA - Reading, Mathematical & Scientific Literacy – 15+ Years Old (2000/2, 2006, 2009, 2012, 2015) TALIS - Teaching and Learning International Survey - Teachers (2013) PIAAC - Programme for the International Assessment of Adult Competencies – (2013) Information Technology - SITES (1997, 2006); ICILS (2012 pilot) Sample-based Assessments
19
Matriculation Exams (Bagrut)
20
Upper Secondary Graduation Rates (2010)
EAG 2012
21
~52% Do Not Matriculate Approximately 20% do not study in 12th grade
Approximately 9% of those studying in 12th grade do not take the Matriculation exams Approximately 23% of those who take the Matriculation exams are not eligible for Matriculation Certificate
22
Only 48% Matriculate (on Average)
23
School Climate Surveys
Meitzav National Assessments and School Climate Surveys
24
School Climate & Pedagogical Environment Surveys
Meitzav המיצ"ב - מהו? National Assessments School Climate & Pedagogical Environment Surveys 24
25
Meitzav – School Report
25
26
Achievement on Meitzav Assessments – 2007-2012 Multi-Year Comparison Across Schools (Grades 5&8)
26
27
Achievement on Meitzav Assessments in 5th Grade according to SES (high-middle-low) amongst Hebrew Speakers 27
28
School Performance on Meitzav by SES
29
School Climate and Pedagogical Environment
Teachers Students Questionnaires Principals School Climate Pedagogical Environment
30
Meitzav Surveys The surveys of school climate and pedagogical environment on the Meitzav are designed to provide a detailed picture of the school climate and pedagogical processes that occur within it, as depicted in the information gathered from questionnaires administered to students and teachers The questionnaires provide comprehensive and relevant information on important dimensions in this area, including: student motivation; the relationship between teachers and students; team work among faculty; Violence and more
31
Monitoring School Violence School & National Levels
32
Monitoring Level of Violence at the National Level
In order to get a more detailed picture of school violence, RAMA developed the ‘Violence Monitoring Questionnaire’ (VMQ) (based on previous work by Benbenishty & Astor and others) The VMQ focuses on a range of expressions of violent and dangerous behaviors within schools and on transport to and from school The goal: Collect systematic bi-annual comparable information, based on student self-response Identify trends over time with regard the level of violence in the education system In both 2009 and RAMA administered the VMQ among a representative national sample of students (around 24,000) in grades 4 to 11 Next survey
33
Topics Investigated on the VMQ
Severe violence Moderate violence Social violence Violence using digital media Verbal violence Violent gangs and bullying in schools Sexual violence Alcohol and drug abuse Violence by the school staff Violence towards the school staff Possession of cold weapons (knives/penknives) Violence on transport to and from school School absenteeism due to a fear of harm Student feelings of safety Students perception of the school's efforts in preventing violence
34
Student Reporting of Violence (1/2)
Students reporting (in %) different types of violence according to grade level in 2009 and 2011, and the rate of the change (in %)
35
Student Reporting of Violence (2/2)
36
International Comparisons
37
Reading Assessment – 4th grade
38
PIRLS 2006: Results for Israel
Average of the leading country – Russian Federation - 565 "PIRLS Scale Average" Rank from 45 countries 31 11 40
39
Percentage of students reaching the PIRLS 2006 benchmark, by sector
40
Arabic Diglosia Diglosia refers to a situation in which closely related languages are used by a single language community. Usually it involves a spoken language and a highly codified variety used in formal settings Diglosia is characteristic of Arabic – Arab children acquire their Spoken Arabic as first language (L1) at home When they attend the Arab school system they begin to learn to speak and to read Modern Standard Arabic (MSA) in first grade, to speak Hebrew (L2) in second grade, and to read and write in Hebrew and in English in third grade This in turn led the Ministry of Education to design a radically new Arabic language acquisition program starting from first grade
41
PISA
42
A rise of 22 points in total from 2002 to 2009
Trends of Change in Reading Literacy in Israel- PISA Between the Years 2002, 2006, and 2009 +22 Mean A significant rise in achievement (35 points) in reading achievement in Israel between 2006 and This improvement brings Israel closer to the OECD average (493) A rise of 22 points in total from 2002 to 2009 42
43
Trends of Change in Reading Literacy - PISA between the years 2006 and 2009
Between 2006 and 2009 Israel rose by 35 points in reading, and is positioned at third place with regard the size of improvement 43
44
Score Variance - PISA 2009 The countries are ranked according to size of the score variance (the gap between the 10th and 90th percentile) 44
45
Between and Within Variance of Science Scores on PISA 2006
Finland excels in personalized learning: 30% of instruction hours are spent by teacher outside the classroom with individual students
46
Release of PISA 2012 Results in December 2013
47
PISA - Pushing the Envelope
Literacy of Reading, Math, Science Financial Literacy Problem Solving Digital Assessments Collaborative Learning Innovation
48
Digital PISA Assessments - March 2012
PISA 2015 – fully computerized
50
PISA IN FOCUS 1: Does participation in pre-primary education translate into better learning outcomes at school?
51
General Results at a Glance)
PISA in Focus 1 reported that significant differences in educational achievements exist between students who had attended pre- primary education (PPE) ten years earlier, and those who had not (even after accounting for ESCS background) Of all participating countries, this difference was the most prominent in Israel Although a casual conclusion cannot be drawn from these results, they are still of high importance to educators, policy makers and researchers
52
Results for Israel for PISA In Focus 1
Israel manifested the largest difference in reading achievements between students who had attended PPE and those who had not
53
Reanalyzing the Data A cautious analysis revealed that this large difference is mainly explained by a sector variable: The majority of those who didn’t attend PPE were Arabic-speaking students (mainly Bedouins) The average score on PISA of Arabic- speaking students is relatively low Within each language sector, the difference in PISA achievements between students who had and had not attended PPE is about 45 points, and it is stable along the entire ESCS range
54
PPE Attendance in Israel)
In Israel, the attendance rate of PPE for more than one year is very similar to the equivalent rate in OECD countries (74% and 72%, respectively) However, PPE attendance for more than one year is much more prevalent among Hebrew-speaking students than among Arabic-speaking students (86% and 35%, respectively) Not attending PPE is rare in the Hebrew- speaking sector (2%), while in the Arabic- speaking sector it is more common and reaches 16%
55
PISA 2009 Achievements and Ethnic Background in Israel
On average, the Arabic-speaking students scored about 100 points less than the Hebrew-speaking students These results are consistent throughout the PISA studies (and other studies) and apply to the three fields of literacy measured in PISA
56
Actions Taken In the wake of the results presented, Israel's MoE decided to take steps to reduce the number of children who do not attend PPE at all, mainly in the Arab sector The results also shed light upon a possible vulnerable population, i.e. students who did not attend PPE. Teachers and educators should be aware of such students when they enter school
57
Students Achievements in Israel International and National Views
57 57
58
What do the TIMSS and PIRLS Studies test?
TIMSS Study in Mathematics and Science (Grade 8) Mathematics Content Domains: Numbers (29%), Algebra (33%), Geometry (19%), Data and probability (19%) Science Content Domains: Biology (37%), Chemistry (20%), Physics (25%), Earth Science (18%) Cognitive Domains: Knowing - Mathematics (36%), Science (32%) Applying - Mathematics (39%), Science (44%) Reasoning - Mathematics (25%), Science (24%) PIRLS Study in Reading (Grade 4) The study includes two types of text (literary and informational) For each of the text types, four processes of comprehension are assessed: Retrieving Explicitly Stated Information (22%) Straightforward Inferences (28%) Interpreting and Integrating (37%) Evaluating Text (13%) 58 58
59
Israel’s Participation in IEA Studies
PIRLS 2011 TIMSS 2011 Reading Mathematics and Science Subject area Grade 4 Grade 8 Grade level 45 42 Participating countries Once every 5 years Once every 4 years Research cycle 2001, 2006, 2011 1999, 2003, 2007, 2011 Research years 80 min. + questionnaire 90 min. + questionnaire Length of test Students, parents, teachers and principals Students, teachers, principals Questionnaire participants 4,186 students in 152 schools 4,699 students in 151 schools Test participants
60
TIMSS and PIRLS 2011: Results and Ranks in Israel
7 13 18 Rank among 42 participating countries Rank among 45 participating countries
61
TIMSS and PIRLS 2011: Inter-percentile Range of “Participating Countries”
TIMSS - Mathematics TIMSS - Science PIRLS - Reading
62
TIMSS and PIRLS 2011: % of Students Reaching the International Benchmarks in Israel and in the “Participating Countries”
63
TIMSS and PIRLS: Trends in Achievements 1999 - 2011
The number and composition of participating countries vary across cycles TIMSS - Mathematics TIMSS - Science PIRLS - Reading +32 +50 +48 28 19 24 7 26 23 25 13 23 31 18 Rank No. of participating countries
64
TIMSS and PIRLS: Trends in Achievements 2006/2007 - 2011
+48 +29 +53 24 7 25 13 31 18 Rank No. of participating countries
65
TIMSS and PIRLS: Trends in % of Students Reaching the International Benchmarks 2006/2007 – 2011
66
TIMSS and PIRLS: Trends in Achievements 2006/2007 – 2011
Among Hebrew Speakers TIMSS - Mathematics TIMSS - Science PIRLS - Reading +20 +52 +45 Placement Based on Int’l Rank No. of participating countries 19 7 19 10 11 2
67
TIMSS and PIRLS: Trends in Achievements 2006/2007 – 2011
Among Arabic Speakers TIMSS - Mathematics TIMSS - Science PIRLS - Reading +51 +59 +57 Placement Based on Int’l Rank No. of participating countries 34 22 37 22 40 35
68
TIMSS and PIRLS: Trends in Mathematics Achievements by Socioeconomic Status 2007 - 2011
Hebrew Speakers Arabic Speakers +36 +68 51+ 78+ +73
69
Formative Assessments
70
Formative Assessments
First Grade Assessment in Reading in Native Language Assessment of initial reading in first grade – Hebrew assessment for Hebrew speakers and Arabic assessment for Arabic speakers. The main aim is to assess students’ initial literacy in their native language. ABLE - Assessment of Basic Literacy in English Assessment of initial reading ability in EFL at beginning fifth grade. The main aim is to identify students who are struggling with the initial stages of reading. AMIT Kit for Student Mapping Assessment of reading and writing ability at beginning of seventh grade. The main aim is to assess the extent to which students can cope with academic texts (similar to those they encounter in course books). Assessing Speaking Kit in Hebrew Assessment of speaking in English for seventh, eighth and ninth grades. The main aim is to assess the following skills: reading aloud, reporting and discussing.
71
Formative Assessments
Assessing Speaking Kit (ASK) in English (EFL) Assessment of speaking in English for seventh, eighth and ninth grades. The main aim is to asses the speaking ability of students studying English as a foreign language (EFL). Language Assessment for New Immigrants Assessment of language skills (reading, listening, speaking, writing) among new immigrant students, designed for students from third grade through to ninth grade. The main aim is to assess the ability of students to function in regular classes with additional support. Banks of Performance Tasks Performance tasks for assessing students in school subjects (Math, Science, History, etc.) at all grade levels.
72
Program Evaluation
73
Types of Projects Evaluated
Wide range programs – Systemic reforms and/or wide range projects. Usually imply wide public interest Local programs with high potential – Locally initiated programs that exhibit high potential for systemic expansion Educational projects – Evaluation of Projects of limited scale, requested by initiators of the projects Pooled evaluation – A combined evaluation of several programs with shared goals major features
74
Wide Range of Programs “New Horizon” Reform: Study of the implementation of the reform in primary and secondary schools, as well as the analysis of its added value to the school climate and students’ scholastic achievements. Tools of evaluation: National test grades, questionnaires and in-depth interviews “The Power to Change (‘Oz Letmura’)” Reform: A major reform in high schools, focused on lengthening teachers’ working days and improving their wages and benefits. A formative evaluation had started concurrently with the reform itself. A summative evaluation will be implemented later, depending on the reform’s progress. “Advancement of 21st Century Skills”: This program focuses on introducing and implementing ICT technologies in classrooms. The evaluation combines formative measures with an experimental design which randomly allocates schools to experiment and control groups.
75
Local Programs With High Potential
“Ad Halom” - A program for promoting literacy among adults: Studying the effect of literacy centers for adults on their children’s scholastic achievements. An experimental design which randomly allocates schools to experiment and control groups. “Personalized Education in Bat-Yam”: A program aimed at Facilitating improved school climate and scholastic achievement through personal tutoring. The evaluation design included both quantitative and qualitative methods, as well as a quasi- experimental study of the program’s effects. “Recruitment of university graduates to teaching”: A study of several programs that were aimed at recruiting university graduates by schools. The evaluation team accompanied the graduates during their initial training and the first stages of their new career
76
Educational Projects “Ogen (Anchor)” - Reducing drop-out rates among immigrant students: The major feature of this program: personal tutoring of immigrant students in order to improve their sense of belonging to school and reduce drop-out rates among them. In-depth Interviews were conducted among students and teachers “Healthy Lifestyle”: A program for advancing healthy nutrition and physical activity among students. The program, aimed at primary school students, is accompanied by formative evaluation, using both qualitative and quantitative methods “Amirim”: A program for advancing gifted children. This program is widely implemented in primary schools and focuses on developing creative and investigative thinking. accompanied by formative evaluation, using both qualitative and quantitative methods
77
“Pooled” Program Evaluation
Mapping and analyzing programs for advancement of achievements in matriculation exams: Facing a plethora of programs in high schools aimed at advancing students’ levels of achievement on matriculation exams, this study mapped and characterized their common features Programs for advancing literacy in kindergartens: A study of Kindergarten teachers’ and parents’ attitudes towards programs that seek to advance literacy among children. A combination of qualitative and quantitative methods A series of studies and analyses concerning Ethiopian-born students: Results in national and international scholastic aptitude tests; investigation of the “Mediation” and “Feuerstein” programs; analysis of results achieved by schools that received enlarged budgets
78
Evaluations currently in process
“New Horizon” Reform in primary and secondary schools “New Horizon” Reform in Kindergartens “Personalized Education” in secondary schools “The Power to Change (Oz Letmura)” Reform in high schools “Advancement of 21st century skills” – integration of ICT in schools The program for advancement of scholastic achievements in secondary schools Evaluation of teachers, principals and other staff members Programs for advancing literacy in kindergartens “Amirim” - a program for advancing gifted children “Social Matriculation Diploma” – advancing community involvement among high school students A unified program for schools’ field trips “Healthy lifestyle” - a program for advancing healthy nutrition and physical activity among students Development of emotional strength among students “AHAM” – improving school climate Interconnections between school climate and scholastic achievements Mapping and assessing Youth movements
79
Evaluations currently in process
“Nahshon” Project – Distant Learning “Michael” Project – empowerment and Improvement of cognitive skills among students “Masa” program – English speaking volunteers teaching english in primary schools “Feuerstein Program” – cognitive advancement of Ethiopian primary schools students Programs for advancement of Ethiopian - born students A program of Mediation between Immigrant students, their families and school staff Evaluation of “Avney Rosha” institute – an institute for advancing school principals “Ad Halom” – a program for advancing literacy among adults “The transparency project” – connecting inputs and outcomes Autonomous conduct in schools Principals’ leadership Personalized education in high schools Advancement of achievements in mathematics in middle schools Advancement of achievements in geography in middle schools Mapping and analyzing programs for advancement of achievements in matriculation exams
80
Teacher & Principal Evaluations
81
Teacher Evaluation in Israel
Teacher evaluation in primary and secondary schools has become a ministry policy and part of the official contracts with the unions. In primary schools it is required for: determining tenure promotion to the highest levels during service professional development In high schools it is required for: individual award for distinguished teachers accumulation of points for advancement of distinguished specific role holders in schools
82
Formative vs Summative Evaluation
Different tools?? Different goals Formative Professional Development Summative Tenure Promotion Rewards High Stakes
83
Teacher Evaluation in “Ofek Hadash”
Teacher evaluation is required for: Determining teacher tenure Promotion on the teachers’ rating scale (top ranks): From grade 6 to 7 From grade 7 to 8 From grade 8 to 9 Professional development
84
Collaboration with all Stakeholders to Build the Evaluation Process
The Ministry of Education: The pedagogic administration: Elementary education and Secondary education The training and professional development department The pedagogical secretariat The religious education department The social and youth department The Arab, Bedouin, Druse and Circassian departments Educational psychological department Special education department District managers and inspectors/supervisors School principals and teachers The Israeli Institute for Educational Leadership Teacher unions
85
Basic Assumptions in the Evaluation Process
The process of teacher evaluation, their active participation and involvement in the process, will contribute to the improvement of teaching and promotion of student learning and well-being. Teacher self-evaluation and the evaluation by the principal of teachers' work will contribute to the image of the fitting teacher and to fostering learning processes and continual improvement Constant and continual improvement of teachers is based on respect, trust, fairness, reflective dialogue and cooperation, constitutes a central element in constructing a professional community for school improvement and the advancement of the education system Teacher evaluation is an important component of the role of the school principal and in establishing administrative quality and accountability for the development of human capital in his/her school Teacher evaluation will strengthen the status of teachers and the teaching profession through the design of professional identity, continued professional development, examining the suitability to teaching and provision of tenure, promotion in terms of rank and roles.
86
Central Points in the Process of Teacher Evaluation
The principal presents the process of teacher evaluation to the teaching staff The principal presents the tool for teacher evaluation to the evaluated teacher at the beginning of the year, including the evaluation process, its stages, and time line throughout the year Teacher evaluation will be based on additional evidence that attests to the teacher's work (e.g., planning a study unit, instructional materials, personal work plan, class file, etc.) During the year the principal will carry out at least two documented observations of the teacher's lessons and also an observation of an individual instructional session.
87
Central Points in the Process of Teacher Evaluation
Prior to the lesson observation the principal will hold a discussion with the teacher in order to collect information about the context in which the lesson is taking place and the lesson's goals. Following the lesson observation the principal will hold a feedback discussion with the teacher, as close to the time of the observation as possible. The teacher will also provide a self evaluation based on the four dimensions of evaluation Together goals will be set for the professional development of the teacher
88
Domains for Teacher Performance
89
Domains for Teacher Performance
90
The Teachers’ Professional Development Scale
The evaluation scale indicates the level of expected teacher performance on different development stages and within different domains: Domains for Teacher Performance Unsatisfactory Basic Skilled Distinguished Master
91
Tool for Teacher Performance Assessment
The tool enable drawing a teacher profile on different domains.
92
A Teachers’ Profile
93
Domains for Teacher Performance
94
RAMA: Assessment in the Service of Learning
for Assessment Learning as of Accountability begins from school self-evaluation
95
Trust and Shared Responsibility
Wider Community Students Parents Trust and Shared Responsibility Principals Teachers Politicians External Stakeholders Active Partners Administrators Local Education Units
96
mbeller.rama@education.gov.il Comments ? Questions?
Thank You Comments ? Questions?
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.