1. A comparison of self- vs
A comparison of self- vs. tutor assessment among undergraduate business students
András István Kun
University of Debrecen
Faculty of Economics and Business Administration

2. "The whole problem with the world is that fools and fanatics are always so certain of themselves, and wiser people so full of doubts."
(Bertrand Russel)
… about their own skills, knowledge, preparedness.

3. "The whole problem with the world is that fools and fanatics are always so certain of themselves, and wiser people so full of doubts."
(Bertrand Russel)
… about their own skills, knowledge, preparedness.
Where?
How much?
What does it depend on?
Why?
What are the consequences (costs)?
Can we do anything?

4. Self- vs. tutor assessment in higher education
Self-evaluation for this research means ONLY: students’ ability to predict and to evaluate their examination performance relative to their externally assessed achievement.
Accuracy: how strongly related it is to the real – tutor assessed – performance of the student.
Direction of self-assessment: tendency of students to over- or underrate themselves.

5. Structure of the presentation
Introducing the research problem (done)
Research motivation
Brief overview of the literature
Empirical findings
Conclusions

6. Research motivation
To help tutors and institutions to facilitate students to manage their own learning. Nicol and Macfarlene-Dick (2006): students already assess their own work, thus higher education institutions could build on this ability.
Mismanaged learning can lead to suboptimal allocation of time and efforts (lower overall performance of students and institutions).
Several researches have previously shown that students’ self- assessment ability is learnable (e.g. Everett 1983; Pintrich 1995; Zimmerman and Schunk 2001; Ross 2006; Baartman and Ruijs 2011).

7. Literature review
Are higher-achieving students, on average, more accurate in their self-assessment than low achievers?
No: Lynn, Holzer and O’Neill (2006)
Yes: Boud and Falchikov (1989); Krueger and Dunning (1999); Sundström (2005); Tejeiro et al. (2012); Karnilowicz (2012)
Do high-achieving students tend to overestimate their own performance less than their low-achieving fellows?
Yes: every article (Boud and Falchikov 1989; Fitzgerald et al. 1997; Krueger and Dunning 1999; Hodges, Regehr and Martin 2001; Lejk and Wyvill 2001; Edwards et al. 2003; Gramzow et al. 2003; Karnilowicz 2012)

8. Literature review
Are pre-assignment self-predictions less accurate than post-assignment self-evaluations?
Yes: Tausignant and DesMarchais (2002), Edwards et al. (2003), and Eva et al. (2004)
Is there a general tendency to overassess?
No: Boud and Falchikov (1989), Mehrdad, Bigdeli and Ebrahimi (2012)
Yes: Krueger and Dunning (1999), Basnet et al. (2012), and Tejeiro et al. (2012)
Is there a difference in the direction of the self-estimation errors between the two sexes?
No: Boud és Falchikov (1989); Krueger and Dunning (1999); Lynn, Holzer and O’Neill (2006); Basnet et al. (2012)
Yes, and men tend to overestimate themselves more than women: Edwards et al. (2003) and Macdonald (2004)

9. Hypotheses of the empirical research
H1: Higher-achieving students assess their examination results more accurately (measured with the absolute value of the assessment error) than their lower- achieving fellows.
H11: Higher achieving students predict their examination results more accurately (measured with the absolute value of the pre-examination assessment error) than their lower achieving fellows.
H12: Higher achieving students evaluate their examination results more accurately (measured with the absolute value of the post-examination assessment error) than their lower achieving fellows.
H2: High-achieving students tend to over-assess their examination results less than low-achieving students.
H3: Compared to female students, males tend to overestimate their own performance more.
H4: Ceteris paribus students tend to overrate their performance and this overrating is greater in pre-examination than in post-examination

10. Samples 163 bachelor level business students (University of Debrecen)
28 master level business students (UD)
35 bachelor level international business students (UD)

11. Sample 1 details
163 business students from the University of Debrecen, Hungary
13 (2 men and 11 women) vocational higher education students
150 bachelor students: 70 (24 men and 46 women) in the Business Administration and Management 80 (21 men and 59 women) on the International Business Economics major
Mid-term test:
2013/14 Spring semester
Management of Value Creating Processes class (= introduction to operations management)
6 test versions (controlled with dummy variables) at 2 optional times
2 consecutive examination session (dummy controlled)
2 types of exercises:
Multiple choices (20)
Calculation problems (pcs.: 3, max. score: 50)

12. Sample 2 details
28 business students from the University of Debrecen, Hungary
12 men and 16 women
Leadership and organization MSc major
Mid-term test:
2014/15 Fall semester
Research Methodology class
2 test versions (controlled with standardization)
Multiple choices (15)

13. Sample 3 details
35 business students from the University of Debrecen, Hungary
17 men and 18 women
Business Administration and Management major
International students from various countries
Language: English
Mid-term test:
2014/15 Fall semester
Human Resource Management class
2 test versions (controlled with dummy variables)
2 types of exercises:
Multiple choices (15)
Calculation problem (pc.: 1, max. score: 15)

14. Data collection Before and after the examination
They were not informed in advance about the self estimation
Motivation of students to estimate more accurately: multiplying their results with if the SECOND estimation was correct

15. The results

16. Descripive statistics of sample 1

17. Suggestions from the descriptives
both sexes overestimated their test scores,
the overestimations were higher in the pre- than in the post- examination evaluation,
the self-assessment scores of female students were higher before the test and slightly lower after it than those of their male counterparts.
Are these significant? Is it connected directly to students’ performance?
Linear regression models are created (depenent is the diffeence between estimation and real score)

18. Testing accuracy on Sample 1
Pre-examination
Post-examination

19. Pre-exam VHE, BAM = major dummies Vi = test variant dummy
Absolute difference Multiple Choice
Absolute difference Calculation
Absolute difference Total test
Pre-exam
VHE, BAM = major dummies
Vi = test variant dummy
MCSCORE = multiple choice score
CPSCORE = calculation score
TTSCORE = total test score
Model1 = every measured variables included
Model2 = only the significant variables

20. Thus…
H11 is supported: a higher score predicts more accurate predictions.

21. Post-exam VHE, BAM = major dummies Vi = test variant dummy
Multiple Choice
Calculation
Total test
Post-exam
VHE, BAM = major dummies
Vi = test variant dummy
MCSCORE = multiple choice score
CPSCORE = calculation score
TTSCORE = total test score
Model1 = every measured variables included
Model2 = only the significant variables

22. Thus…
H12 is ‚mostly’ supported. In case of calculations higher achievment tends to decrease accuracy between 0 and
Estimation accuracy on calculation scores is significantly connected to tutor assessment via a cubic function, that decreases accuracy up to CPSCORE ≤ , increases it when ≤ CPSCORE ≤ , and decreases it again if ≤ CPSCORE. Maximum score was 50, thus, within the ‚normal range’ it is increasing accuracy.

23. Testing overestimation tendency on Sample 1
Binary logistic regression models are used.
Dependent is the is the likeliness that a student over-assesses him/herself.
Those cases where the tutor-assigned score was 0 or maximal, leaving no chance for under- or over-assessment error, are left out of the sample.
Cases where the student estimated his/her own performance without error are also neglected, being unimportant for this question.

24. Pre-exam Dependent: possibility to overestimate
Multiple Choice
Calculation
Total test
Pre-exam
Dependent: possibility to overestimate
Model1 = every measured variables included
Model2 = only the significant variables
VHE, BAM = major dummies
Vi = test variant dummy
MCSCORE = multiple choice score
CPSCORE = calculation score
TTSCORE = total test score

25. Post-exam Dependent: possibility to overestimate
Multiple Choice
Calculation
Total test
Post-exam
Dependent: possibility to overestimate
Model1 = every measured variables included
Model2 = only the significant variables
VHE, BAM = major dummies
Vi = test variant dummy
MCSCORE = multiple choice score
CPSCORE = calculation score
TTSCORE = total test score

26. Thus…
The H2 hypothesis is thus mostly supported by the binary logistic regression analysis, with the only exception being that in Model 2 for the pre-examination total scores, the negative relationship between self-overassessment of total scores and TTSCORE is not supported for lower-achieving students (below app. 44 points form 70) but only for high-achievers.

27. Testing H3 (gender differences in overestimation)
DIFMC1 = pre-exam multiple choice self- vs. tutor assigned score differences DIFCP1 = pre-exam calculation self- vs. tutor assigned score differences DIFTT1 = pre-exam total score self- vs. tutor assigned score differences DIFMC2 = post-exam multiple choice self- vs. tutor assigned score differences DIFCP2 = post-exam total score self- vs. tutor assigned score differences DIFTT2 = post-exam total score self- vs. tutor assigned score differences
H3 is rejected

28. Testing H4 on Sample 1 (pre vs. post)
H4 is supported (with the exemption of the single case of males and multiple choices, but even in these cases the direction of the difference is fitting H4)

29. Testing H4 on Sample 1 (pre vs. post)
H4 is supported with the only exemption of multiple choice estimations of males (the sign of the difference is fitting even here).

30. Analysing H1 & H2 with the small samples
Simpler analysis: calculating the groups of the lowest and highest thirds of the samples according to their tutor assessed achievement, and comparing the accuracy and overestimation tendency of these sample-thirds.

31. Raw data from Sample 2 and 3 crosstabs

32. Testing H1 & H2
H1 is supported only for calculations on Sample3. H2 is supported with the only exemption of multiple choices, pre-examination on Sample3.

33. Testing H3 on the small samples
H3 is supported only for the calculation on Sample3, where males tend to be less accurate and also more overestimating.

34. Testing H4 on the small samples

35. Conclusions High achievers tend to: (compared to low achievers.)
predict and evaluate their own performance better according to the largest sample for both multiple choice and calculation exercises, and according to the calculation exercises on international students’ sample. But no such effect could be identified for multiple choice questions on the smaller samples. This suggests that the effect depends on the sample and also on the type of the assessment.
overestimate themselves less frequently. This was not identified only in one case: multiple choices for the international students (where the foreign language can bias the test results).
(compared to low achievers.)
These differences are mostly independent of their gender (females are overestimating more on the largest sample in the case of predicting multiple choice scores and males are overestimating stronger in the international students’ sample in the case of evaluating calculation exercise performance).
Predictions are less accurate and more likely to be overestimated than evaluations. On the largest sample it was insignificant only for males’ multiple choice results. Smaller samples were more confusing: the effect is significant for the master level sample’s signed differences, for the international student sample’s multiple choice signed mistakes and calculation absolute mistakes.

36. Possible areas for further research
Many (possible) fields of focus:
Formal education: primary, secondary, tertiary
Company (internal) trainings
Sports
etc.
Differences among majors
Intercultural differences (samples from different countries)

37. The presentation was based on the following papers
Kun, András István (2015): A comparison of self versus tutor assessment among Hungarian undergraduate business students, Assessment & Evaluation in Higher Education, DOI: /
Kun András István (2015): Önértéklés és teljesítmény az üzleti felsőoktatásban (Self-assessment and performance in business higher education). Manuscript.

38. Comments and questions are welcome andras.kun@econ.unideb.hu
András István Kun
Associate professor
University of Debrecen, Hungary
Faculty of Economics and Business
Institute of Leadership and Management
H-4032 Böszörményi út 138.