Presentation on theme: "Roles, functions and evaluative language in PhD examiners’ reports Professor Brian Paltridge A/Professor Sue Starfield Dr Robert McMurtrie."— Presentation transcript:
Roles, functions and evaluative language in PhD examiners’ reports Professor Brian Paltridge A/Professor Sue Starfield Dr Robert McMurtrie
Outline The project and our role Part 1 Methods, resources, tools, frameworks and theories SFL lens: APPRAISAL, TRANSITIVITY Part 2 10 roles institution, supervisor, expert, reporter, peer, viva examiner, evaluator, editor, commentator and examiner not isomorphic v. clustering Final comments
Roles [T]he real role of the examiner is to judge whether the student has mastered appropriate indeterminate skills and displayed the right indeterminate qualities. (Delamont et al. 2000, 41, emphasis added) Resources from FOCUS sharpen and soften the boundaries between things, grading something that is inherently non- gradable (Martin and Rose, 2007: 46)
Previous research : function/roles Previous research on the language of examiners’ reports Praising, hedging (Hyland 1998), criticising, sugar coating the pill (Hyland and Hyland 2001) Previous research on the roles examiners adopt Responsibilities, duties, job descriptions Teacher, proof-reader, facilitator, gatekeeper, evaluator, knowledge provider and reader (Hyland and Hyland 2001) “schizophrenic” role of the teacher when marking assignments Leki (1990: 59)
Our contribution: Identifying various roles Making the correlation between the linguistic features and the roles TRANSITIVITY = what is being evaluated APPRAISAL framework to see how “the what” is being appraised/evaluated Synthesis of experiential and interpersonal
APPRAISAL framework AFFECT: how we feel dis-satisfaction un-happiness in-security JUDGEMENT: how we evaluate the behaviour of another person normalcy capacity tenacity veracity Propriety APPRECIATION: how we evaluate an object reaction impact quality composition balance complexity valuation
Transitivity framework The studenthas performedan in-depth study. Actor Process:materialGoal The findingswere evaluatedcritically. GoalProcess:material Circumstance The thesisisa fine example of a good PhD. Carrier Process:relational: attributive:intensive Attribute Ilikethe engagement with Popper. SenserProcess:mental Phenomenon
Role of the PhD thesis A successful PhD thesiswill demonstratethe candidate’s ability to conduct original research and to present the findings of that research to a professional standard TokenProcess:relational: intensive Value
Criteria Does the thesis comprise a coherent investigation of the chosen topic? (APPRECIATION:composition:balance) Does the thesis deal with a topic of sufficient range and depth to meet the requirements of the degree? (APPRECIATION:valuation:standard:depth) Does the thesis make an original contribution to knowledge in its field (APPRECIATION:valuation:standard:original) and contain material suitable for publication in an appropriate academic journal ? (APPRECIATION:valuation:standard:publishable) Does the thesis meet internationally recognised standards for the conduct and presentation of research in the field? (APPRECIATION:valuation:standard/propriety APPRECIATION:reaction:quality/composition:complexity) Does the thesis demonstrate both a thorough knowledge of the literature relevant to its subject and general field? (APPRECIATION:valuation:capacity:cognition) and the candidate's ability to exercise critical and analytical judgement of that literature? (covert JUDGEMENT:capacity) Does the thesis display mastery of appropriate methodology and/or theoretical material? (APPRECIATION:valuation:standard:appropriate)
Transitivity The candidatehas presentedthe thesisgenerallywell ActorProcess:materialGoalCircumstance The thesisisgenerallypresentedwell GoalCircumstanceProcess: material Circumstance The candidatehas presentedthe thesisgenerallywell ActorProcess:materialGoalCircumstance
JUDGEMENT v. COVERT JUDGEMENT APPRECIATIONJUDGEMENTCOVERT JUDGEMENT This study is well- presented (+ve APPRECIATION: reaction:quality) The candidate has presented the thesis well (+ve JUDGEMENT:capacity). The thesis is generally presented well (+ve COVERT JUDGEMENT:capacity) This thesis fails to justify its claims (-ve APPRECIATION:valuation) The candidate has failed to substantiate her claims (-ve JUDGEMENT:capacity) The failure to substantiate her claims … (-ve COVERT JUDGEMENT:capacity)
ENGAGEMENT Monoglossic – no room for negotiation (prescriptive) Heteroglossic – room for negotiation Other voices – reporting, quoting and paraphrasing, modality Related to stance, voice, position, hedging
Register (style) Holbrook et al (2012, 3, emphasis added) ‘[e]xaminers write their report in a sequence and style that reflects their discipline, their unique interests and expertise’ Johnston (1997, 336, emphasis added) ‘showed a wide variation in style’ Register Field What is going on (participants and processes) Tenor What are the social roles of the participants Mode How is the discourse presented (as formal written or causal conversation)
Unit of information: clause Overview: This is an original, well thought out, methodical, detailed and concise PhD thesis 5x APPRECIATION (one role – examiner) The nature of this colour variation is interesting and important 2x APPRECIATION (two roles – commentator, evaluator)
Method Clause Passive Positive/ negative Role APPRECIATION JUDGEMENT COVERT JUDGEMENT AFFECT COVERT AFFECT Overview: This is an original, well thought out, methodical, detailed and concise PhD thesis [[that addresses a poorly understood topic]]. positive examiner √ The results are fascinating positive commentator √ √ and are a major contribution to our understanding of fragmentation in Surtseyan eruptions. positive examiner √ X has clearly put a big effort into this positive evaluator √ and she should be congratulated for a super piece of work. positive evaluator √ √ I recommend that the thesis be accepted and the degree awarded after amendments have been made positive examiner √
The “real” role of the examiner APPRECIATION : Impact: quality Composition: balance/complexity Valuation However, because the thesis and the candidate are in a Token/Valued relationship: COVERT JUDGEMENT Which is APPRECIATION : normalcy/capacity/tenacity/ veracity/propriety Passive voice Nominalisation Field: recommendations, award, degree, emendations I recommend that the thesis be accepted and the degree awarded after amendments have been made
Roles institution ????????
Institution Institution: Does the thesis make an original contribution to knowledge in the field and contain material suitable for publication in an appropriate academic journal? Examiner: Yes
Roles institution supervisor ???????
Supervisor “I felt throughout reading the thesis that my comments on the work were more akin to those of a supervisor rather than that of an examiner.” The actual role of supervisor: formative feedback (Holbrook et al 2004, 9) Gap in knowledge and how to close the gap and improve the thesis. Dialogic (Kumar and Stracke 2007, 467). Tenor: close, friendly
Supervisor Why do you think this is the case? Cf. Why is this the case? … but can you say a little more about why you prefer it and did not use the EM algorithm? You might also consider some of the earlier work of Valière A useful start is his 2002 JBR paper (for an overview see Dixon, Donald F. (2002) "Emerging Macromarketing Concepts: From Socrates to Alfred Marshall." Journal of Business Research, 55 (2), 87-95) You should consider stating here that this piece of research focuses on this subtype I would make (even) clearer where this emerges from, and take full credit.
Comparisons ExaminerSupervisor The candidate needs to explain exactly what these numbers represent You might also consider some of the earlier work of Valière Page 44. (Do) As with Chapter, voucher specimens should be lodged in a National Collection and voucher numbers given Page 77. (Consider) Could there be some thermoregulation function to retention of wings but loss of flight?
Roles institutionsupervisor editor ??????
Editor Prescriptive x should be y P14 loose should be lose Usually at the end of the report P60 "to small" should be "too small" P64 expression in the sentence "In winter " p71 "an person" should be "a person" p73 expression "There have been maximum..." P76 describe above should be described above P input station should be stations P89 expression "The outgoing solar..." P94 Fig 4.2 is referred to before Fig 4.1 P97 expression "the ELA was with " P97 expression 2000 msl has an massive
Reporter Synoptic overview of Thesis Chapters Sections This thesis reports the results of what is essentially a single study with boys with ADHD and a control group of normally developing boys Uses verbal processes, quotes, paraphrases “These organic compounds are known as siderophores." This sentence suggests that all Fe (III)-binding compounds in seawater are siderophores. Monoglossic Several well-known motifs (such as the 5' UTR TATA box) are low complexity elements To position her or himself in relation to the content Heteroglossic contracting – closing space down for alternatives The author has demonstrated that putative toxin/antitoxin pairs can act as such on when found on the chromosomes of M. smegmatis… Heteroglossic expanding – distancing formulation In particular, it is claimed (p. 56) that the research in the thesis is based on the work of Tripp and Alsop (1999, 2001), but the details of those studies and their theoretical impact is not fully explained (p.44).
Peer Encouragement/improvement beyond the confines of the thesis Field: publish, paper, articles Tenor: close relationship; modality: should, could X should consider publishing the introduction, in a shortened version, as a review paper Some of the suggestions listed are identified as not necessary for the thesis but desirable prior to publication.
Evaluator Evaluation beyond the criteria: APPRECIATION and JUDGEMENT (overt) No intrusion on the message, i.e., “I” She is extremely passionate about blogging (+ve JUDGEMENT:normalcy) This is an early study opening up a very complex system
Expert “Examiners should be suitably qualified to undertake the task. Suitably qualified examiners: preferably have a PhD (or if not, some other higher degree but with appropriate research experience at a high level.” Authoritative declaratives Also direct mass balance measurements cannot a priori claim to exactly catch the end of the ablation season. Call into doubt their expertise The statistical analysis seems sound, although I am not an expert in statistics.
Expert Examiners exert their knowledge in the field in question and in terms of the genre/conventions of a thesis. Claims must be evidence based A strong structure allows readers to know where to look for particular information and makes the reader more confident that all important information has been included… Distinguishing between reporter and expert when there is no attribution: Page 15 Free rotation of carbons adjacent to single bond in lycopene polyene carbon chain occurs only in those bonds which are not conjugated…
Imperatives / roles EditorExaminerExpert Pg line 14— change 'the' to 'these' and 'to' to 'in’ … and write in full sentences, explaining these observations 17. (2.1.2, Page 11, section 3, last line): Add e.g. "weathering rates and geomorphological activity", as the amount of rock walls alone are not decisive
Commentator According to the OED, a comment is ‘a remark expressing an opinion or reaction […] especially of a critical nature’. Not prescriptive, formative or summative Explicit intrusion on the message “I” Formulaic expressions, attitude markers, personal musings What on earth has Donald Rumsfeld to do with marketing (quoted twice). I wonder if reciprocal blasts were used,… AFFECT, COVERT AFFECT I worry (-ve AFFECT :insecurity) about your mention of analysing relations among concepts This is an interesting (+ve APPRECIATION :reaction:impact) […] thesis This thesis interested me (+ve COVERT AFFECT :satisfaction)
Realised as grammar: Third conditional
3 rd v. 2 nd conditional/roles Commentatorsupervisor I assume that they are the extracted cells, this could have been explained in the methods (or perhaps I have missed it). You could explain that exchange value is only a limited form of value creation Also, more detail on how specific toxins act should have been given. You should consider stating here that this piece of research focuses on this subtype; It might have been useful here to compare the results obtained to those recently reported by Nauseef et al, and Palazzolo-Balance et al. …. You might also consider some of the earlier work of Valière.
Other multimodal realisations From the perspective of future work – 3 rd conditional is formative = peer role Marked punctuation Very interesting!! P55. What is DRE?? I'm curious -???
Viva examiner Actual primary role: to ask questions at the viva. The role ↘ interrogative (technical term F^S) Prepare for the viva voce. Many types of questions – not all viva examiners Field: Thus, these questions are largely unanswered in the thesis and are exactly the questions I would like to discuss with the candidate during an oral exam.
Different types of questions Viva examinerSupervisorCommentator how was this controlled for?Normal human neonatal foreskin or adult skin? Page 40, para 2: laboratory Page 51, para 1: is there really an apostrophe after its? what difference could this make?2.5, p48, end of para 1: Should it be 'bone marrow cells' rather than 'white blood cells'? What on earth has Donald Rumsfeld to do with marketing (quoted twice). Page 67. (Clarify) What is a geomorphological region? Why were cell types chosen? How likely is this to relate to real cancer? Were all arrays exposed on the same piece of x-ray film? Are these separated by major rivers, faults or mountain ranges? 4. Can the candidate outline/ suggest from the studies and their findings some quite direct applications in the joint flight crew and flight attendant training activities that would contribute to safety and efficiencies? Can you speculate on how a particular type of siderophore would respond to such a pH change based on their acid base chemistry? …could you have used another site with a longer term record, such as Craigieburn, to test the hypothesis that the ERA dataset is better than the Tait?
FINAL REMARKS 1.The various roles and linguistic features are not isomorphic; rather roles are ↘ by a clustering of features. 2.The complexity of interpreting the evaluation when there are a number of roles to grapple 3.As Sadler (2010: 537) states, ‘students cannot convert feedback statements into actions for improvement without sufficient working knowledge of some fundamental concepts’ 4.A candidate might ask: Which comments do I need to act upon to get this thesis passed? Which comments might I consider in order to improve the thesis? Which comments can I return to later when considering publications? Which comments can I simply disregard?” 5.Are all roles necessary/appropriate? 6.Can some comments be reworded so that they are given by a more authoritative role (examiner, editor) 7.Should there be a feedback rubric that ties the comments in with APPRAISAL and TRANSITIVITY. 8.Can discourse analysts collaborate with educators to create a metalanguage to share amongst examiners, candidates, and institutions.
Is a more robust PhD assessment regime actually possible?
Reference List Bednarek, M. (2008). Emotion Talk Across Corpora. New York: Palgrave MacMillan. Delamont, S., Atkinson, P. and Parry, O. (2000). The Doctoral Experience: Success and Failure in Graduate School. Rouledge-Falmer: London and New York Hyland, K Hedging in scientific research articles. Amsterdam: Benjamins. Hyland, F. and Hyland, K Sugaring the pill: Praise and criticism in written feedback. Journal of Second Language Writing 10: 185—212. Johnston, S. (1997a). Examining the examiners: An analysis of examiners’ reports on doctoral theses, Studies in Higher Education, 22, 3: Kress, G. (2010). Multimodality: A Social Semiotic Approach to Contemporary Communication. London and New York: Routledge. Kumar V. and Stracke, E. (2007). An analysis of written feedback on PhD thesis. Teaching in Higher Education 12(4): 461—470. Kumar V. and Stracke, E. (2011). Examiners’ reports on theses: Feedback or assessment? Journal of English for Academic Purposes 10: Leki, I. (1990). Coaching form the margins: Issues in written response. In B. Kroll (ed.), Second Language Writing: Research Insights for the Classroom, 57—68. Cambridge: Cambridge University Press. Martin, J.R. and Rose, D Working with Discourse: Meaning beyond the Clause. London: Continuum. Martin, J.R. and White, P.R.R. (2005). The language of evaluation: Appraisal in English. New York: Palgrave Macmillan.
Further research Genre analysis Stages Roles in stages Clustering of roles Micro structures More transitivity analysis
Our role Project Examination of examiners’ reports Linguistic perspective Language of evaluation, roles and function