Presentation on theme: "Extent and determinants of agreement between Medical Assessors in the assessment of seafarer fitness Dr Isam Rustom Specialist Registrar in Occupational."— Presentation transcript:
Extent and determinants of agreement between Medical Assessors in the assessment of seafarer fitness Dr Isam Rustom Specialist Registrar in Occupational Medicine 22 nd November 2011
RESEARCH QUESTIONS 1.What is the extent of agreement between Approved Doctors (ADs) and Medical Referees (MRs) in assessing the fitness of seafarers? 2.What is the extent of agreement between the MRs in assessing the fitness of seafarers? 3.What are the determinants of agreement between the MRs in assessing the fitness of seafarers?
METHODS Seafarer Medical Review- Medical Referees Report (MSF 4108) forms were collected for the period between May 2003 and May 2009. MRs were assigned individual codes (1-12) to ensure confidentiality. The medical categories of those seeking review were coded using the 16 categories on the (MSF 4108) forms.
METHODS Type of work has been categorised as Deck, Engine, Catering and Others. The ADs and MRs decisions were categorised into one of the four fitness categories. The reasons of grading or restrictions change were coded into four categories: -More Information -New Event -GP/Specialist Report -Other (which includes experience, knowledge and different interpretation of the standards by the MRs)
STATISTICAL ANALYSIS Standard statistical tools were used: Chi-square, P values < 0.05, inter-rater reliability Kappa statistic and the ration between upgraded/downgraded cases for each MR. The outcome of the appeal process (result of appeal) for the purposes of analysis is taken to be the difference between the ADs decision and the MRs decision which were classified as: - No change (same category, but could alter the imposed restrictions); - Upgrade (MRs decision gave the seafarer more flexibility to work at sea); - Downgrade (MRs decision was more strict than the ADs decision, further limiting the seafarer work at sea).
MRs referrals - age profile
Medical categories and mean age
ADs decision vs. MRs decision ADs decision Number Percentage MRs decision Number Percentage Fit 0 0.0 Fit 66 14.2 Fit with restrictions 111 23.9 Fit with restrictions 252 54.2 Temporarily unfit 86 18.5 Temporarily unfit 62 13.3 Permanently unfit 268 57.6 Permanently unfit 85 18.3 Total 465 100.0 Total 465 100.0
Reasons for grading and restrictions change *Genitourinary, digestive, blood and cancer (36) cases were excluded from analysis, see Methods section
Extent of agreement between the MRs The workload varied considerably between MRs as they were distributed across the UK. –Referee 9 saw (95 cases), –Referee 6 (68 cases), –Referee 10 (61 cases), –Referee 7 (46 cases), These Referees dealt with more than half (270 cases; 58%) of all appeal cases over the 6-year period of the study.
Ratio of upgraded/downgraded cases by MRs a Referee 2,4,7 and 11 had no downgraded cases, so their ratios were 0. b Referee 8 and 12 were excluded from the analysis as they only considered 5 and 1 respectively and had no downgraded cases, so their ratios were 0.
Determinants of agreement between the MRs Age group Medical category Referee code Type of work
There was significant association between age group and Fit with restrictions fitness category except under 20s age group. There was significant association between medical category and Fit with restrictions fitness category except Mental ill health cases. There was No significant association between MRs decision and Referee code. There was No significant association between MRs decision and type of work.
Discussion of extent of agreement between ADs and MRs. This study identified poor agreement between the MRs decision and the ADs decision among those seafarers who decided to appeal against the ADs initial decision [kappa=0.18; poor]. The most frequent category for a grading or restrictions change was Other reasons (43.1%), in all medical categories apart from Cardiovascular 65 (45.4%) and Mental ill health 17 (44.7%) cases which "More Information" was the most frequent category when reviewing fitness decisions.
Outcome of the appeal process 58% of the appellant seafarers benefited from the process and were granted an upgrade in their fitness category. 40% of the appellant seafarers who had no change in their fitness category often benefited from less or different restrictions which allowed them to continue in employment and working at sea at some capacity. Only the 2% of the appellant seafarers who had been downgraded were disadvantaged by the process.
This study attempted to determine the factors that may have affected the outcome of the appeal process. The relationships with measurable variables like age group, medical category, Referee code, and type of work were studied. It appears that the outcome of the appeal process was multi-factorial and was affected by a combination of the above variables.
CONCLUSIONS First research question: this study found that there was poor agreement between the MRs decision and the ADs decision regarding the fitness of those seafarers who decided to appeal against the ADs initial assessment. Second research question: MRs tended to agree with each other as they upgraded the fitness category of seafarers to allow more of them to work at sea. Third research question: MRs decisions on the fitness of seafarers were affected by the age group and the medical category and MRs preferred to use Fit with restrictions fitness category more than any other fitness categories in order to impose more or less restrictions whilst keeping the seafarer in employment.
THE WAY FORWARD In 2008 the MCA decided to review the 2002 fitness standards in the light of changes in medical diagnosis, treatment, disability legislation and decision taking process especially apparent in the reviews undertaken by MRs. As a result more seafarers would be allowed to work at sea with restrictions and reasonable adjustments which were taken into consideration in the new 2010 fitness standards.
FURTHER RESEARCH There is a need to explore other factors which might affect the ADs and MRs decisions when assessing the fitness of seafarers. For example, the hypothesis being that ADs with occupational medicine qualifications would be less likely to have their verdicts set aside on appeal. There might be a gradient, with higher levels of qualifications more likely to produce a supported verdict.