1. YES SEMINAR Dubrovnik , June 4, 2017
ZURAB ABRAMISHVILI MASS REPLACEMENT OF SCHOOL PRINCIPALS IN GEORGIA Comment by : Oleh Havrylyshyn
YES SEMINAR
Dubrovnik , June 4, 2017

2. HIGHLIGHTS OF PAPER
UNIQUE ISSUE OF A RADICAL REFORM METHOD –DISMISS ALL PRINCIPALS ; REDO HIRING
GOOD USE OF QUANTITATIVE METHODS FOR A NON-ECONOMIC POLICY ISSUE
ADDS TO UNDERSTANDING OF THE OVERALL GEORGIAN ROSE-REVOLUTION RADICAL APPROACH TO ENTRENCHED VESTED INTERESTS (?good lessons for e.g. Ukraine ?)
STATISTICAL ANALYSIS BROADLY SUPPORTS HYPOTHESIS THAT MASS DISMISSAL REFORM IMPROVED EDUCATION RESULTS

3. WHAT I LIKE BEST …
THOUGHTFUL DEVELOPMENT OF AN UNCOMMON , BUT INTERESTING RESEARCH TOPIC
SOLID,SIMPLE ,AND CLEVER USE OF
STAT. ANALYSIS FOR A COMPLEX CAUSE-EFFECT LINK
CONSIDERABLE EFFORTS TO CROSS-CHECK, VERIFY, SUPPORT RESULTS, AND TEST ROBUSTNESS

4. WHAT I FIND PROBLEMATIC &/or INCOMPLETE…
Lack of data on actual performance/tests is a concern - is university acceptance ratio correlated ?
Presentation needs more clarity–( good e.g. is it % or percentage points?)
Margin of 4% ( perc.points?) compared to 63% average enrollment not obviously large or significant :IS IT ?
no information on the ratio of (Applications / Graduates) did this change, differ among schools ?
Less important but interesting : 53% of schools got replacement principal , 47 % same one as before : is this high ? low ? relevant? irrelevant ? (for Police I believe replacement rate was much higher, no?)

5. NO TESTS DATA
University acceptance a reasonable proxy BUT need more support : is it really correlated with academic results? Other countries’ evidence ?
Why not use ratio (acceptance/ total cohort)?
Why not use 2011 general tests? Only 3 yrs. time lag
P.14 refers to mandatory entrance exams since 2005 – are these data not available ?

6. PRESENTATION: MORE CLARITY
More data on absolute levels for applications, acceptances etc.-some in Fig. 4- USE it in text
p.12 in same sentence you refer to “academic standing” and “enrollment rate” –are they the same or not ?
P : non-replacement schools enrollment declined, replacement schools enrollment increased by more than 4% ??-- not clear – show absolutes to clarify ? Text here compared to Fig.4 sometimes hard to follow

7. IS 4% DIFFERENCE SIGNIFICANT
First, this means 4 pp. better, right ?
Second, actual comparisons in Tab.2,3 are against private , not between different public categories , some results significant some not --- not clear on significance of the 4% number emphasised.
Can you provide more time comparisons? international benchmarks ? qualitative supporting evidence for main hypothesis
Arguably the strongest result from stats. may be that lottery-assignment in this reform gave best results – perhaps that should be your “main finding “ in Abstract and Conclusion

8. MY MAIN SUGGESTION USE FIG.4 AS CENTRAL FOCUS OF PAPER
THE ABSOLUTE LEVELS OF ENROLLMENT SHOW STRONG UPWARD TREND FOR ALL –FIRST CLEAR CONCLUSION IS THAT ALL IMPROVED REGARDLESS OF REFORMS AND DESPITE 2010 DECLINE --PAPER MUST NOT IGNORE THIS
FIG. 4 ALSO SHOWS NICELY HOW REPLACEMENTS DIFFERED—USE REGRESSION RESULTS TO DISCUSS THESE DIFFERENCES, CONCLUSION THAT REPLACEMENT BETTER THAN NON-REPLACEMENT etc.
THIS MAY MEAN YOU WILL CHANGE MAIN CONCLUSION TO: LOTTERY BEST