Presentation on theme: "Specialist Registrar in Occupational Medicine"— Presentation transcript:
1 Specialist Registrar in Occupational Medicine Extent and determinants of agreement between Medical Assessors in the assessment of seafarer fitnessDr Isam RustomSpecialist Registrar in Occupational Medicine22nd November 2011
2 RESEARCH QUESTIONSWhat is the extent of agreement between Approved Doctors (ADs) and Medical Referees (MRs) in assessing the fitness of seafarers?What is the extent of agreement between the MRs in assessing the fitness of seafarers?What are the determinants of agreement between the MRs in assessing the fitness of seafarers?
3 METHODSSeafarer Medical Review- Medical Referee’s Report (MSF 4108) forms were collected for the period between May 2003 and May 2009.MRs were assigned individual codes (1-12) to ensure confidentiality.The medical categories of those seeking review were coded using the 16 categories on the (MSF 4108) forms.
4 METHODSType of work has been categorised as Deck, Engine, Catering and Others.The AD’s and MR’s decisions were categorised into one of the four fitness categories.The reasons of grading or restrictions change were coded into four categories:-More Information-New Event-GP/Specialist Report-Other (which includes experience, knowledge and different interpretation of the standards by the MRs)
5 STATISTICAL ANALYSISStandard statistical tools were used: Chi-square, P values < 0.05, inter-rater reliability Kappa statistic and the ration between upgraded/downgraded cases for each MR.The outcome of the appeal process (result of appeal) for the purposes of analysis is taken to be the difference between the AD’s decision and the MR’s decision which were classified as:- No change (same category, but could alter the imposed restrictions);- Upgrade (MR’s decision gave the seafarer more flexibility to work at sea);- Downgrade (MR’s decision was more strict than the AD’s decision, further limiting the seafarer work at sea).
8 AD’s decision vs. MR’s decision NumberPercentageMR’s decisionFit0.06614.2Fit withrestrictions11123.9Fit with restrictions25254.2Temporarily unfit8618.56213.3Permanently unfit26857.68518.3Total465100.0
9 Reasons for grading and restrictions change Reasons for grading / restrictions change n (%)MedicalcategoryMoreinformationNew eventGP/SpecialistreportOtherTotalCardiovascular65 (45.4)18 (12.6)27 (18.9)33 (23.1)143Endocrine20 (30.3)8 (12.1)5 (7.6)33 (50.0)66Sensory10 (16.7)3 (5.0)37 (61.6)60Neurological15 (34.1)1 (2.2)5 (11.4)23(52.3)44Respiratory10 (23.8)1 (2.4)3 (7.1)28 (66.7)42Mental17 (44.7)0 (0.0)7 (18.5)14 (36.8)38Musculoskeletal12 (33.4)4 (11.1)3 (8.3)17 (47.2)36149 (34.7)42 (9.8)53 (12.4)185 (43.1)429*Genitourinary, digestive, blood and cancer (36) cases were excluded from analysis, see Methods section
10 Extent of agreement between the MRs The workload varied considerably between MRs as they were distributed across the UK.Referee 9 saw (95 cases),Referee 6 (68 cases),Referee 10 (61 cases),Referee 7 (46 cases),These Referees dealt with more than half (270 cases; 58%) of all appeal cases over the 6-year period of the study.
11 Ratio of upgraded/downgraded cases by MRs Dove traitHawk traitReferee codea,b6109135Ratio(upgraded/downgraded)3830201813.512.5a Referee 2,4,7 and 11 had no downgraded cases, so their ratios were 0.b Referee 8 and 12 were excluded from the analysis as they only considered 5 and 1 respectively and had no downgraded cases, so their ratios were 0.
12 Determinants of agreement between the MRs Age groupMedical categoryReferee codeType of work
13 There was significant association between age group and “Fit with restrictions” fitness category except under 20s age group.There was significant association between medical category and “Fit with restrictions” fitness category except Mental ill health cases.There was No significant association between MR’s decision and Referee code.There was No significant association between MR’s decision and type of work.
14 Discussion of extent of agreement between ADs and MRs. This study identified poor agreement between the MR’s decision and the AD’s decision among those seafarers who decided to appeal against the AD’s initial decision [kappa=0.18; poor].The most frequent category for a grading or restrictions change was “Other reasons” (43.1%), in all medical categories apart from Cardiovascular 65 (45.4%) and Mental ill health 17 (44.7%) cases which "More Information" was the most frequent category when reviewing fitness decisions.
15 Outcome of the appeal process 58% of the appellant seafarers benefited from the process and were granted an upgrade in their fitness category.40% of the appellant seafarers who had no change in their fitness category often benefited from less or different restrictions which allowed them to continue in employment and working at sea at some capacity.Only the 2% of the appellant seafarers who had been downgraded were disadvantaged by the process.
16 This study attempted to determine the factors that may have affected the outcome of the appeal process. The relationships with measurable variables like age group, medical category, Referee code, and type of work were studied.It appears that the outcome of the appeal process was multi-factorial and was affected by a combination of the above variables.
17 CONCLUSIONSFirst research question: this study found that there was poor agreement between the MR’s decision and the AD’s decision regarding the fitness of those seafarers who decided to appeal against the AD’s initial assessment.Second research question: MRs tended to agree with each other as they upgraded the fitness category of seafarers to allow more of them to work at sea.Third research question: MRs decisions on the fitness of seafarers were affected by the age group and the medical category and MRs preferred to use “Fit with restrictions” fitness category more than any other fitness categories in order to impose more or less restrictions whilst keeping the seafarer in employment.
18 The way forwardIn 2008 the MCA decided to review the 2002 fitness standards in the light of changes in medical diagnosis, treatment, disability legislation and decision taking process especially apparent in the reviews undertaken by MRs. As a result more seafarers would be allowed to work at sea with restrictions and reasonable adjustments which were taken into consideration in the new 2010 fitness standards.
19 FURTHER RESEARCHThere is a need to explore other factors which might affect the ADs and MRs decisions when assessing the fitness of seafarers. For example, the hypothesis being that ADs with occupational medicine qualifications would be less likely to have their verdicts set aside on appeal. There might be a gradient, with higher levels of qualifications more likely to produce a supported verdict.