1. Assessment and Evaluation of the Israeli Education System
Michal Beller
Director-General of RAMA
The National Authority for Measurement and Evaluation in Education, Israel
April 2013

2. Assessment and Evaluation Should Provide Support for an Ongoing Process
Baker, 2005

3. Intended Instruction Hours
EAG 2012

4. Teaching Hours
EAG 2012

5. Class Size
EAG 2012

6. Ministerial
Goals

7. Ministerial Goals (2009-2013) Promote educational values
Combat violence and create an optimal educational climate
Improve achievement
Best utilize available study time
Narrow the educational gaps
Strengthen the status of teachers and improve the quality of instruction
Strengthen the status of principals
Focus on core subjects
Suit curricula to changing educational reality
Raise the number of students in professional-technological education
Align the education system to the digital age in the 21st Century
Mainstream special needs students into regular education by enhancing the ability to retain them in classes and provide a range of solutions for them

8. Assessment for Learning
RAMA
National Authority for Measurement and Evaluation in Education
Assessment for Learning

9. RAMA
An independent statutory unit - the National Authority for Measurement and Evaluation – RAMA - shall be established as the entity that leads and provides professional guidance to the education system with respect to measurement and evaluation. RAMA will conduct periodic evaluations of the education system and evaluations in schools, and will publish its findings in an annual report submitted to the National Council for Education. From the Dovrat Commission (2004)

10. Improving Student Learning
Ultimately, accountability is not only about measuring student learning but actually improving it
Consequently, genuine accountability involves supporting changes in teaching and schooling that can heighten the probability that students meet standards
There are at least three major areas where attention is needed: 
Ensuring that teachers have the knowledge and skills they need to teach to the standards
Providing school structures that support high quality teaching and learning 
Creating processes for school assessment that can evaluate students’ opportunities to learn and can leverage continuous change and improvement
Darling Hammond, 2004

11. Campbell's Law
"The more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
"achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence.
But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.”
Campbell, Donald T., "Assessing the Impact of Planned Social Change“. The Public Affairs Center, Dartmouth College, December, 1976.

12. Unintended Negative Effects of External High Stakes Testing
Narrowing the Curriculum
Between-Subject Reallocation of time
The Impact of Setting a Specific Target
Teaching to the Test;
Test Inflation
Assessment is Only a Sample of the Curriculum
“Most Children are Left Behind…”

13. Real Estate Map (based on Meitzav)

14. Meitzav League Tables

15. Updating the Israeli Assessment System
The main goals of the recent update:
Implement a culture of “assessment for learning”
Mitigate the threats of external exams (including gaming the system)
Effective integration of internal and external evaluation
Decentralization of the evaluation process along with the use of centrally designed rigorous tools
Empowerment of teachers and principals
Professional design of assessments and scoring (including equating, alignment to curriculum and standards and more)

16. Use of Multiple Indicators
Sample-based Assessments
(National & International exams)
Internal
External
Internal admin. of external tests
Matriculation exams,
Meitzav
Formative Assessments,
Assignment Data- Base, “Off-the-shelf” Assessments
Schools & pupils are being examined

17. Large Scale Assessment in Israel

18. Large Scale Assessment in Israel
Matriculation Exams (“Bagrut”) – end of high school
Meitzav – National Assessments (Native Language, Math, Science, English) – 5th and 8th grades (+2nd grade in Language) + Climate & Pedagogical Survey to students and teachers (primary and secondary students, teachers and principals)
International Comparisons:
TIMSS - Math and Science Study – 9th Grade
(1995, 1999, 2003, 2007, 2011, 2015)
PIRLS - Reading Assessment – 4th Grade
(2001, 2006, 2011, 2016)
PISA - Reading, Mathematical & Scientific Literacy – 15+ Years Old
(2000/2, 2006, 2009, 2012, 2015)
TALIS - Teaching and Learning International Survey - Teachers
(2013)
PIAAC - Programme for the International Assessment of Adult
Competencies – (2013)
Information Technology - SITES (1997, 2006); ICILS (2012 pilot)
Sample-based Assessments

19. Matriculation Exams
(Bagrut)

20. Upper Secondary Graduation Rates (2010)
EAG 2012

21. ~52% Do Not Matriculate Approximately 20% do not study in 12th grade
Approximately 9% of those studying in 12th grade do not take the Matriculation exams
Approximately 23% of those who take the Matriculation exams are not eligible for Matriculation Certificate

22. Only 48% Matriculate (on Average)

23. School Climate Surveys
Meitzav
National Assessments
and
School Climate Surveys

24. School Climate & Pedagogical Environment Surveys
Meitzav
המיצ"ב - מהו?
National Assessments
School Climate & Pedagogical Environment Surveys 24

25. Meitzav – School Report25

26. Achievement on Meitzav Assessments – 2007-2012 Multi-Year Comparison Across Schools (Grades 5&8)26

27. Achievement on Meitzav Assessments in 5th Grade according to SES (high-middle-low) amongst Hebrew Speakers 27

28. School Performance on Meitzav by SES

29. School Climate and Pedagogical Environment
Teachers
Students
Questionnaires
Principals
School Climate
Pedagogical Environment

30. Meitzav Surveys
The surveys of school climate and pedagogical environment on the Meitzav are designed to provide a detailed picture of the school climate and pedagogical processes that occur within it, as depicted in the information gathered from questionnaires administered to students and teachers
The questionnaires provide comprehensive and relevant information on important dimensions in this area, including:
student motivation;
the relationship between teachers and students;
team work among faculty;
Violence
and more

31. Monitoring School Violence School & National Levels

32. Monitoring Level of Violence at the National Level
In order to get a more detailed picture of school violence, RAMA developed the ‘Violence Monitoring Questionnaire’ (VMQ) (based on previous work by Benbenishty & Astor and others)
The VMQ focuses on a range of expressions of violent and dangerous behaviors within schools and on transport to and from school
The goal:
Collect systematic bi-annual comparable information, based on student self-response
Identify trends over time with regard the level of violence in the education system
In both 2009 and RAMA administered the VMQ among a representative national sample of students (around 24,000) in grades 4 to 11
Next survey

33. Topics Investigated on the VMQ
Severe violence
Moderate violence
Social violence
Violence using digital media
Verbal violence
Violent gangs and bullying in schools
Sexual violence
Alcohol and drug abuse
Violence by the school staff
Violence towards the school staff
Possession of cold weapons (knives/penknives)
Violence on transport to and from school
School absenteeism due to a fear of harm
Student feelings of safety
Students perception of the school's efforts in preventing violence

34. Student Reporting of Violence (1/2)
Students reporting (in %) different types of violence according to grade level in 2009 and 2011, and the rate of the change (in %)

35. Student Reporting of Violence (2/2)

36. International
Comparisons

37. Reading Assessment – 4th grade

38. PIRLS 2006: Results for Israel
Average of the leading country – Russian Federation - 565
"PIRLS Scale Average"
Rank from 45 countries 31 11 40

39. Percentage of students reaching the PIRLS 2006 benchmark, by sector

40. Arabic Diglosia
Diglosia refers to a situation in which closely related languages are used by a single language community. Usually it involves a spoken language and a highly codified variety used in formal settings
Diglosia is characteristic of Arabic –
Arab children acquire their Spoken Arabic as first language (L1) at home
When they attend the Arab school system they begin to learn to speak and to read Modern Standard Arabic (MSA) in first grade, to speak Hebrew (L2) in second grade, and to read and write in Hebrew and in English in third grade
This in turn led the Ministry of Education to design a radically new Arabic language acquisition program starting from first grade

41. PISA

42. A rise of 22 points in total from 2002 to 2009
Trends of Change in Reading Literacy in Israel- PISA Between the Years 2002, 2006, and 2009
+22
Mean
A significant rise in achievement (35 points) in reading achievement in Israel between 2006 and This improvement brings Israel closer to the OECD average (493)
A rise of 22 points in total from 2002 to 2009 42

43. Trends of Change in Reading Literacy - PISA between the years 2006 and 2009
Between 2006 and 2009 Israel rose by 35 points in reading, and is positioned at third place with regard the size of improvement 43

44. Score Variance - PISA 2009
The countries are ranked according to size of the score variance (the gap between the 10th and 90th percentile) 44

45. Between and Within Variance of Science Scores on PISA 2006
Finland excels in personalized learning:
30% of instruction hours are spent by teacher outside the classroom with individual students

46. Release of PISA 2012 Results in December 2013

47. PISA - Pushing the Envelope
Literacy of Reading, Math, Science
Financial Literacy
Problem Solving
Digital Assessments
Collaborative Learning
Innovation

48. Digital PISA Assessments - March 2012
PISA 2015 – fully computerized

50. PISA IN FOCUS 1: Does participation in pre-primary education translate into better learning outcomes at school?

51. General Results at a Glance)
PISA in Focus 1 reported that significant differences in educational achievements exist between students who had attended pre- primary education (PPE) ten years earlier, and those who had not (even after accounting for ESCS background) Of all participating countries, this difference was the most prominent in Israel Although a casual conclusion cannot be drawn from these results, they are still of high importance to educators, policy makers and researchers

52. Results for Israel for PISA In Focus 1
Israel manifested the largest difference in reading achievements between students who had attended PPE and those who had not

53. Reanalyzing the Data
A cautious analysis revealed that this large difference is mainly explained by a sector variable:
The majority of those who didn’t attend PPE were Arabic-speaking students (mainly Bedouins)
The average score on PISA of Arabic- speaking students is relatively low
Within each language sector, the difference in PISA achievements between students who had and had not attended PPE is about 45 points, and it is stable along the entire ESCS range

54. PPE Attendance in Israel)
In Israel, the attendance rate of PPE for more than one year is very similar to the equivalent rate in OECD countries (74% and 72%, respectively)
However, PPE attendance for more than one year is much more prevalent among Hebrew-speaking students than among Arabic-speaking students (86% and 35%, respectively)
Not attending PPE is rare in the Hebrew- speaking sector (2%), while in the Arabic- speaking sector it is more common and reaches 16%

55. PISA 2009 Achievements and Ethnic Background in Israel
On average, the Arabic-speaking students scored about 100 points less than the Hebrew-speaking students These results are consistent throughout the PISA studies (and other studies) and apply to the three fields of literacy measured in PISA

56. Actions Taken
In the wake of the results presented, Israel's MoE decided to take steps to reduce the number of children who do not attend PPE at all, mainly in the Arab sector
The results also shed light upon a possible vulnerable population, i.e. students who did not attend PPE. Teachers and educators should be aware of such students when they enter school

57. Students Achievements in Israel International and National Views57 57

58. What do the TIMSS and PIRLS Studies test?
TIMSS Study in Mathematics and Science (Grade 8)
Mathematics Content Domains:
Numbers (29%), Algebra (33%), Geometry (19%), Data and probability (19%)
Science Content Domains: Biology (37%), Chemistry (20%), Physics (25%), Earth Science (18%)
Cognitive Domains:
Knowing - Mathematics (36%), Science (32%)
Applying - Mathematics (39%), Science (44%)
Reasoning - Mathematics (25%), Science (24%)
PIRLS Study in Reading (Grade 4)
The study includes two types of text (literary and informational)
For each of the text types, four processes of comprehension are assessed:
Retrieving Explicitly Stated Information (22%)
Straightforward Inferences (28%)
Interpreting and Integrating (37%)
Evaluating Text (13%) 58 58

59. Israel’s Participation in IEA Studies
PIRLS 2011
TIMSS 2011
Reading
Mathematics and Science
Subject area
Grade 4
Grade 8
Grade level 45 42
Participating countries
Once every 5 years
Once every 4 years
Research cycle
2001, 2006, 2011
1999, 2003, 2007, 2011
Research years
80 min. + questionnaire
90 min. + questionnaire
Length of test
Students, parents, teachers and principals
Students, teachers, principals
Questionnaire participants
4,186 students in 152 schools
4,699 students in 151 schools
Test participants

60. TIMSS and PIRLS 2011: Results and Ranks in Israel7 13 18
Rank among 42 participating countries
Rank among 45 participating countries

61. TIMSS and PIRLS 2011: Inter-percentile Range of “Participating Countries”
TIMSS - Mathematics
TIMSS - Science
PIRLS - Reading

62. TIMSS and PIRLS 2011: % of Students Reaching the International Benchmarks in Israel and in the “Participating Countries”

63. TIMSS and PIRLS: Trends in Achievements 1999 - 2011
The number and composition of participating countries vary across cycles
TIMSS - Mathematics TIMSS - Science PIRLS - Reading
+32
+50
+48 28 19 24 7 26 23 25 13 23 31 18
Rank
No. of participating countries

64. TIMSS and PIRLS: Trends in Achievements 2006/2007 - 2011
+48
+29
+53 24 7 25 13 31 18
Rank
No. of participating countries

65. TIMSS and PIRLS: Trends in % of Students Reaching the International Benchmarks 2006/2007 – 2011

66. TIMSS and PIRLS: Trends in Achievements 2006/2007 – 2011
Among Hebrew Speakers
TIMSS - Mathematics TIMSS - Science PIRLS - Reading
+20
+52
+45
Placement Based on Int’l Rank
No. of participating countries 19 7 19 10 11 2

67. TIMSS and PIRLS: Trends in Achievements 2006/2007 – 2011
Among Arabic Speakers
TIMSS - Mathematics TIMSS - Science PIRLS - Reading
+51
+59
+57
Placement Based on Int’l Rank
No. of participating countries 34 22 37 22 40 35

68. TIMSS and PIRLS: Trends in Mathematics Achievements by Socioeconomic Status 2007 - 2011
Hebrew Speakers Arabic Speakers
+36
+68
51+
78+
+73

69. Formative Assessments

70. Formative Assessments
First Grade Assessment in Reading in Native Language Assessment of initial reading in first grade – Hebrew assessment for Hebrew speakers and Arabic assessment for Arabic speakers. The main aim is to assess students’ initial literacy in their native language. ABLE - Assessment of Basic Literacy in English Assessment of initial reading ability in EFL at beginning fifth grade. The main aim is to identify students who are struggling with the initial stages of reading. AMIT Kit for Student Mapping Assessment of reading and writing ability at beginning of seventh grade. The main aim is to assess the extent to which students can cope with academic texts (similar to those they encounter in course books). Assessing Speaking Kit in Hebrew Assessment of speaking in English for seventh, eighth and ninth grades. The main aim is to assess the following skills: reading aloud, reporting and discussing.

71. Formative Assessments
Assessing Speaking Kit (ASK) in English (EFL) Assessment of speaking in English for seventh, eighth and ninth grades. The main aim is to asses the speaking ability of students studying English as a foreign language (EFL). Language Assessment for New Immigrants Assessment of language skills (reading, listening, speaking, writing) among new immigrant students, designed for students from third grade through to ninth grade. The main aim is to assess the ability of students to function in regular classes with additional support. Banks of Performance Tasks Performance tasks for assessing students in school subjects (Math, Science, History, etc.) at all grade levels.

72. Program
Evaluation

73. Types of Projects Evaluated
Wide range programs – Systemic reforms and/or wide range projects. Usually imply wide public interest
Local programs with high potential – Locally initiated programs that exhibit high potential for systemic expansion
Educational projects – Evaluation of Projects of limited scale, requested by initiators of the projects
Pooled evaluation – A combined evaluation of several programs with shared goals major features

74. Wide Range of Programs
“New Horizon” Reform: Study of the implementation of the reform in primary and secondary schools, as well as the analysis of its added value to the school climate and students’ scholastic achievements. Tools of evaluation: National test grades, questionnaires and in-depth interviews
“The Power to Change (‘Oz Letmura’)” Reform: A major reform
in high schools, focused on lengthening teachers’ working days
and improving their wages and benefits.
A formative evaluation had started concurrently with the reform itself.
A summative evaluation will be implemented later, depending on the
reform’s progress.
“Advancement of 21st Century Skills”: This program focuses on
introducing and implementing ICT technologies in classrooms.
The evaluation combines formative measures with an experimental
design which randomly allocates schools to experiment and control
groups.

75. Local Programs With High Potential
“Ad Halom” - A program for promoting literacy among adults:
Studying the effect of literacy centers for adults on their children’s
scholastic achievements. An experimental design which randomly
allocates schools to experiment and control groups.
“Personalized Education in Bat-Yam”: A program aimed at
Facilitating improved school climate and scholastic achievement
through personal tutoring.
The evaluation design included both quantitative and qualitative
methods, as well as a quasi- experimental study of the program’s
effects.
“Recruitment of university graduates to teaching”: A study of
several programs that were aimed at recruiting university graduates
by schools. The evaluation team accompanied the graduates during
their initial training and the first stages of their new career

76. Educational Projects
“Ogen (Anchor)” - Reducing drop-out rates among immigrant
students: The major feature of this program: personal tutoring of
immigrant students in order to improve their sense of belonging to
school and reduce drop-out rates among them. In-depth Interviews
were conducted among students and teachers
“Healthy Lifestyle”: A program for advancing healthy nutrition
and physical activity among students. The program, aimed at
primary school students, is accompanied by formative
evaluation, using both qualitative and quantitative methods
“Amirim”: A program for advancing gifted children. This
program is widely implemented in primary schools and focuses
on developing creative and investigative thinking. accompanied
by formative evaluation, using both qualitative and quantitative
methods

77. “Pooled” Program Evaluation
Mapping and analyzing programs for advancement of
achievements in matriculation exams: Facing a plethora of
programs in high schools aimed at advancing students’ levels of
achievement on matriculation exams, this study mapped and
characterized their common features
Programs for advancing literacy in kindergartens: A study of
Kindergarten teachers’ and parents’ attitudes towards programs that
seek to advance literacy among children. A combination of
qualitative and quantitative methods
A series of studies and analyses concerning Ethiopian-born
students: Results in national and international scholastic aptitude
tests; investigation of the “Mediation” and “Feuerstein” programs;
analysis of results achieved by schools that received enlarged
budgets

78. Evaluations currently in process
“New Horizon” Reform in primary and secondary schools
“New Horizon” Reform in Kindergartens
“Personalized Education” in secondary schools
“The Power to Change (Oz Letmura)” Reform in high schools
“Advancement of 21st century skills” – integration of ICT in schools
The program for advancement of scholastic achievements in secondary schools
Evaluation of teachers, principals and other staff members
Programs for advancing literacy in kindergartens
“Amirim” - a program for advancing gifted children
“Social Matriculation Diploma” – advancing community involvement among high school students
A unified program for schools’ field trips
“Healthy lifestyle” - a program for advancing healthy nutrition and physical activity among students
Development of emotional strength among students
“AHAM” – improving school climate
Interconnections between school climate and scholastic achievements
Mapping and assessing Youth movements

79. Evaluations currently in process
“Nahshon” Project – Distant Learning
“Michael” Project – empowerment and Improvement of cognitive skills among students
“Masa” program – English speaking volunteers teaching english in primary schools
“Feuerstein Program” – cognitive advancement of Ethiopian primary schools students
Programs for advancement of Ethiopian - born students
A program of Mediation between Immigrant students, their families and school staff
Evaluation of “Avney Rosha” institute – an institute for advancing school principals
“Ad Halom” – a program for advancing literacy among adults
“The transparency project” – connecting inputs and outcomes
Autonomous conduct in schools
Principals’ leadership
Personalized education in high schools
Advancement of achievements in mathematics in middle schools
Advancement of achievements in geography in middle schools
Mapping and analyzing programs for advancement of achievements in matriculation exams

80. Teacher & Principal
Evaluations

81. Teacher Evaluation in Israel
Teacher evaluation in primary and secondary schools has become a ministry policy and part of the official contracts with the unions.
In primary schools it is required for:
determining tenure
promotion to the highest levels during service
professional development
In high schools it is required for:
individual award for distinguished teachers
accumulation of points for advancement of distinguished specific role holders in schools

82. Formative vs Summative Evaluation
Different tools??
Different goals
Formative
Professional
Development
Summative
Tenure
Promotion
Rewards
High Stakes

83. Teacher Evaluation in “Ofek Hadash”
Teacher evaluation is required for:
Determining teacher tenure
Promotion on the teachers’ rating scale (top ranks):
From grade 6 to 7
From grade 7 to 8
From grade 8 to 9
Professional development

84. Collaboration with all Stakeholders to Build the Evaluation Process
The Ministry of Education:
The pedagogic administration: Elementary education and Secondary education
The training and professional development department
The pedagogical secretariat
The religious education department
The social and youth department
The Arab, Bedouin, Druse and Circassian departments
Educational psychological department
Special education department
District managers and inspectors/supervisors
School principals and teachers
The Israeli Institute for Educational Leadership
Teacher unions

85. Basic Assumptions in the Evaluation Process
The process of teacher evaluation, their active participation and involvement in the process, will contribute to the improvement of teaching and promotion of student learning and well-being.
Teacher self-evaluation and the evaluation by the principal of teachers' work will contribute to the image of the fitting teacher and to fostering learning processes and continual improvement
Constant and continual improvement of teachers is based on respect, trust, fairness, reflective dialogue and cooperation, constitutes a central element in constructing a professional community for school improvement and the advancement of the education system
Teacher evaluation is an important component of the role of the school principal and in establishing administrative quality and accountability for the development of human capital in his/her school
Teacher evaluation will strengthen the status of teachers and the teaching profession through the design of professional identity, continued professional development, examining the suitability to teaching and provision of tenure, promotion in terms of rank and roles.

86. Central Points in the Process of Teacher Evaluation
The principal presents the process of teacher evaluation to the teaching staff The principal presents the tool for teacher evaluation to the evaluated teacher at the beginning of the year, including the evaluation process, its stages, and time line throughout the year Teacher evaluation will be based on additional evidence that attests to the teacher's work (e.g., planning a study unit, instructional materials, personal work plan, class file, etc.) During the year the principal will carry out at least two documented observations of the teacher's lessons and also an observation of an individual instructional session.

87. Central Points in the Process of Teacher Evaluation
Prior to the lesson observation the principal will hold a discussion with the teacher in order to collect information about the context in which the lesson is taking place and the lesson's goals. Following the lesson observation the principal will hold a feedback discussion with the teacher, as close to the time of the observation as possible. The teacher will also provide a self evaluation based on the four dimensions of evaluation Together goals will be set for the professional development of the teacher

88. Domains for Teacher Performance

89. Domains for Teacher Performance

90. The Teachers’ Professional Development Scale
The evaluation scale indicates the level of expected teacher performance on different development stages and within different domains:
Domains for Teacher Performance
Unsatisfactory
Basic
Skilled
Distinguished
Master

91. Tool for Teacher Performance Assessment
The tool enable drawing a teacher profile on different domains.

92. A Teachers’ Profile

93. Domains for Teacher Performance

94. RAMA: Assessment in the Service of Learning
for
Assessment
Learning as of
Accountability begins from school self-evaluation

95. Trust and Shared Responsibility
Wider Community
Students
Parents
Trust and Shared Responsibility
Principals
Teachers
Politicians
External Stakeholders
Active Partners
Administrators
Local Education
Units

96. mbeller.rama@education.gov.il Comments ? Questions?
Thank You
Comments ? Questions?