IDENTIFYING GREAT TEACHERS THROUGH THEIR ONLINE PRESENCE Evanthia Faliagka, Maria Rigou, Spiros Sirmakessis

Qualities of good teachers  Teachers are distinguished as liked or disliked based on three criteria: academic qualifications, relationship with students and personality traits  Traits that yield positive educational results  Conscientiousness (efficient, not lazy, thorough)  Agreeableness (warm, forgiving, sympathetic)  Openness to experience (curious, imaginative, excitable)  Extroverted (sociable, enthusiastic, forceful, positive)  Emotionally stable (calm, self-confident, not shy)  Big-Five personality model (NEO PI-R)

Previous work  Academic qualifications of teachers come from:  CVs and accompanying documents  LinkedIn  Teacher personality is typically evaluated through  special purpose questionnaires/tests  Interviews  Recruiting teachers could be performed online provided that we can have some unbiased feedback on teacher personality  Social web data (from Facebook, Twitter, LinkedIn, etc.) can be the source of such feedback especially in the case of active users  if we can interpret social web activities in terms of personality traits demonstrated.

Proposed system  A system that automates candidate teacher pre- screening process providing an overall candidate ranking  based on supervised learning  and automatically extracting applicant personality measures from tweets and Fb posts  This approach has been implemented as a web based teacher evaluation system

Architecture Candidate teachers School director CV data Login to Position requirements Extracting candidate’s skills Calculating personality traits Data mining algorithms Ranked list of teachers Applicant education, work experience and loyalty are directly extracted from LinkedIn Personality traits of the candidate are assessed by analyzing posts to Twitter and Fb

Academic qualifications  Education (in years of formal academic training)  Work experience (in years of working at relevant job positions  Loyalty (average number of years spent per job)

Personality mining  Judging a teacher’s personality (or ANY personality) is a hard problem for automated e-recruitment systems  We focus on the extroversion personality trait  It is reflected through language use in written speech  It is discriminated through text analysis  It is a crucial characteristic in teacher personality  The emotional positivity and social orientation of a person, both directly extracted from LIWC frequencies, can act as predictors of the extroversion trait Linguistic Inquiry and Word Count system

Calculation of teacher extroversion  To find which words are mentioned most frequently by the candidate we analyze the raw text of tweets and Facebook posts  The words identified are input to the TreeTagger tool for lexical analysis and lemmatization  Then using the LIWC dictionary the system classifies the canonical form of word output by the TreeTagger  A dictionary of word stems classified in certain psycholinguistic categories  We calculate the LIWC extroversion score E  E is estimated directly from LIWC scores, by summing the emotional positivity score and the social orientation score

Calculation of teacher extroversion  Finally, we use the regression model which was trained in a previous work of ours that predicts the candidates’ extroversion from their LIWC scores in the {posemo, negemo, social} categories E = S *P *N Where:  E is the extroversion score  S the frequency of social words  P the frequency of positive emotion works  N the frequency of negative emotion words

Login page

Experience mining

Ranking  Our system uses machine learning algorithms  It requires a training set as an input  It automatically builds the ranking model  It calculates the final scoring function h(x)  It returns the final ranked list of teachers by applying the learned function to sort them

Learning-to-rank

Pilot Scenario  42 teachers logged in to our system as candidates for a job position in a private elementary school  The job position was also announced through the system  Teachers were also evaluated manually on their academic qualifications and interviewed for assessing their personality by the school director  Automated extroversion scores were compared to the interview results referring to each teacher’s extroversion  The data collected are to be used as the training set for the ranking algorithm

Pilot Scenario  Grading scale for the personality extroversion score: 0-5  We used Weka to test the correlation of the scores output from the system (i.e. model predictions) with the actual scores assigned by the director  Comparison metrics (system vs director):  Overlap size of the top-k list  Correlation coefficient of the top-k candidates  Mean absolute difference of top-k candidates’ ranks k=8Top-kCorrelationRanking error Candidates6 (75%)0.722,6

Conclusions  The proposed system could be of practical value in speeding up the teacher recruitment process  Automating qualifications and personality assessment  Future work:  Use larger training sets  Instead of manual character assessment, use special questionnaires and train the system with their results  Use additional social network metrics (LinkedIn endorsements and recommendations, no of re-tweets, Fb likes and shares, etc)  Incorporate automated assessment of teacher scores in more personality traits (agreeableness, openness to experience, etc)

Thank you!