Presentation on theme: "MIMCentre …thinking healthcare solutions Telemedicine Evaluation and the M 2 DM Project Mike Hughes 1,2, Riccardo Bellazi 3, Ewart Carson 1, Derek Cramp."— Presentation transcript:
MIMCentre …thinking healthcare solutions Telemedicine Evaluation and the M 2 DM Project Mike Hughes 1,2, Riccardo Bellazi 3, Ewart Carson 1, Derek Cramp 1, Fiona Harvey 1, Til Rendschmidt 4  MIM Centre, City University, London, United Kingdom;  CodeGen.IT, Thames Gateway Technology Centre, London, United Kingdom;  Dipartimento di Informatica e Sistematistica, University di Pavia, Italy;  Munchen Bogenhausen Hospital, Munchen, Germany.
MIMCentre …thinking healthcare solutions Objective Development of a systemic and systematic method for the evaluation of telemedicine systems –Overview of the M 2 DM Project –Comparison of Telemedicine System Evaluation with Treatment Evaluation –Identification of Evaluation Parameters –Selection of Evaluation Method –Identification and Recognition of Experimental Artefacts or Biases
MIMCentre …thinking healthcare solutions Multi-Access-Services for Diabetes Management (M 2 DM) Pan-European multi-accessible telemedicine system for the management of Diabetes Mellitus –EPR Module –Educational Module –Data Analysis Module Pilot system to be deployed and evaluated in 2002 –Padova, Italy –Barcelona, Spain –München, Germany
MIMCentre …thinking healthcare solutions M 2 DM System Architecture DB AS1AS2 MAS Web/WAPTelephoneGlucometer
MIMCentre …thinking healthcare solutions Treatment Evaluation Multiple Evaluation Parameters Randomised Test/Control Groups Double Blindness Placebo Control
MIMCentre …thinking healthcare solutions Medical Information System Evaluation Multiple Evaluation Parameters Randomised Test/Control Groups Single Blindness (at most) No Equivalent to a Placebo Control
MIMCentre …thinking healthcare solutions Medical Information System Evaluation Multiple Evaluation Parameters –Clinical Effectiveness; –Organisational Efficiency; –Information System Cost; –Accuracy/Completeness of Information; –Usability and Accessibility of Information System; –Information System Reliability; –Information System Processing Speed Randomised Test/Control Groups Single Blindness (at most) No Placebo Control
MIMCentre …thinking healthcare solutions Economic Efficiency Clinical Effectiveness Organisation Efficiency Usability and Accessibility Accuracy and Completeness Processing Speed Information System Cost Reliability (+) (-)
MIMCentre …thinking healthcare solutions M 2 DM Evaluation Method Four different evaluation studies: –Clinical Effectiveness Evaluation Study; Clinical Effectiveness –Organisational Efficiency Evaluation Study; Organisational Efficiency; Information System Cost –Economic Cost-Effectiveness Evaluation Study; Economic Cost-Effectiveness –User Satisfaction Evaluation Study; Information System Usability/Accessibility; Information Accuracy/Completeness; Information System Reliability; Information Processing Speed.
MIMCentre …thinking healthcare solutions M 2 DM Evaluation Timeline Jan 02Apr 02July 02Oct 02Jan 03 Ongoing scheduled patient-diabetologist consultations and delivery of evaluation study questionnaires Jan 02: Selection and randomisation of subjects
MIMCentre …thinking healthcare solutions Problems with M 2 DM Evaluation Method 1.Limited Extrapolation of Results to Non-Experimental Scenario but… All experimental studies rely on the assumption of the replication of some experimental conditions in non- experimental scenarios but… This is a matter of degree! 2.Single/Zero Blindness but… This is unavoidable!
MIMCentre …thinking healthcare solutions Conclusions Telemedicine Systems are Information Systems, not drugs or surgical procedures, and should be evaluated according to their own set of evaluation standards, rather than the standards used in clinical trials. In clinical evaluation of telemedicine systems, there is a possibility of experimenter/clinician bias (no experimenter blindness) and a possibility of subject bias (no subject blindness). There can be no equivalent to a placebo control, and classification and identification of different control information systems is difficult and the evaluation parameters difficult to calibrate and measure.