Fundamentals of Clinical Research for Radiologists Presented by: Reema Al-Shawaf

Introduction In recent years the evaluation of diagnostic technologies has become more demanding In recent years the evaluation of diagnostic technologies has become more demanding

The new trend of clinical practice and decision making is towards the evidence based Dentistry The new trend of clinical practice and decision making is towards the evidence based Dentistry

Outline : Introduction Introduction Concepts of research Concepts of research Methods of evaluating diagnostic procedures Methods of evaluating diagnostic procedures

Definition of research The systemic collection, description, analysis and interpretation of data to answer a certain question or solve a problem.

Characteristics of good research Proper planning Proper planning Accuracy in data collection Accuracy in data collection Proper unbiased interpretation Proper unbiased interpretation

Research requires TEaM ? Research requires TEaM ?

Research requires TEaM Time + Effort + Money Time + Effort + Money

Ethical considerations throughout the research process!!

Ethics in health research No one shall be subjected to torture or to cruel, inhuman or degrading treatment or punishment. In particular, no one shall be subjected without his free consent to medical or scientific experimentation. No one shall be subjected to torture or to cruel, inhuman or degrading treatment or punishment. In particular, no one shall be subjected without his free consent to medical or scientific experimentation. International Covenant on Civil and Political Rights, Article 7, 1966 International Covenant on Civil and Political Rights, Article 7, 1966

Concepts of research

The research process Choice of the research topic Choice of the research topic Planning the research Planning the research Development of the research protocol Development of the research protocol Writing and submitting a research proposal Writing and submitting a research proposal Implementing the study Implementing the study Description and analysis of the research results Description and analysis of the research results Interpretation of the research results Interpretation of the research results Communicating the research Communicating the research Publication Publication

What research to do??

Defining the question A key features in defining the question to be addressed are: value of the research value of the research Motivation Motivation

What drives the health research? - Curiosity driven - Needs-driven - Profit-driven - Opportunity-driven

Radiology research questions categories: Evaluation of equipment Evaluation of equipment Discovery and evaluation of techniques Discovery and evaluation of techniques Re-evaluation of old techniques or procedures Re-evaluation of old techniques or procedures Application of Radiologic techniques to investigate changes in treatment Application of Radiologic techniques to investigate changes in treatment

Planning the research Time spent on proper planning is never lost.

Planning the research Feasibility Feasibility Own capabilities Own capabilities Availability of material or subjects for research Availability of material or subjects for research Availably of resources Availably of resources

Planning the research Type of research design Type of research design Define and Refine the question Define and Refine the question Hypothesis Hypothesis sampling sampling

Defining and refining the research question In planning the research: In planning the research: The more refined the question, the better will be the plan The more refined the question, the better will be the plan

Hypothesis A hypothesis is a fundamental basis for A hypothesis is a fundamental basis for generating a successful research project generating a successful research project

The question of interest is divided into two competing hypothesis. The null hypothesis (H0) There is no difference The null hypothesis (H0) There is no difference The alternative hypothesis (H1) there is difference The alternative hypothesis (H1) there is difference

Conventionally the scientific community has used a p value less than 0.05 as sufficiently small to call a result statistically significant Conventionally the scientific community has used a p value less than 0.05 as sufficiently small to call a result statistically significant

Types of Errors Type I error: Type I error: occurs in a hypothesis test when a true null hypothesis is rejected = false positive occurs in a hypothesis test when a true null hypothesis is rejected = false positive Type II error Type II error occurs in when the null hypothesis is not rejected when it should be= false negative occurs in when the null hypothesis is not rejected when it should be= false negative

Reality Reality Conclusion drawn form study Test A No Better Than Test B Test A Better Than Test B False-positive Type I error True-positive correct Test A Better Than Test B True-negativecorrectFalse-negative Type II error Test A No Better Than Test B Labeling the Erroneous Conclusions from a study

Which error is more serious and why?

Study sample In planning the research, a crucial question is the type and size of sample to be studied. In planning the research, a crucial question is the type and size of sample to be studied.

Sample size Sample size appropriate to provide the answer to the research question should be defined. Sample size appropriate to provide the answer to the research question should be defined.

Larger sample: Waste of resources Waste of resources Affects the results Affects the results Type I error Type I error Smaller sample Waste of effort and money Waste of effort and money Affects the results Affects the results Type II error Type II error

description and analysis of research. Descriptive statistics are useful to summarize and present the data in a way that allows subsequent correlation Descriptive statistics are useful to summarize and present the data in a way that allows subsequent correlation Tools of descriptive statistics include Tools of descriptive statistics include Tabulations, calculations graphs and correlation Tabulations, calculations graphs and correlation

Why statistics ? Although statistics will not reveal the absolute truth about the whole it will allow the estimation of the truth Although statistics will not reveal the absolute truth about the whole it will allow the estimation of the truth

Statistical methods Methodologies used to interpret quantities data with special calculated values that describe a collection of data and then to assess errors In these values. Methodologies used to interpret quantities data with special calculated values that describe a collection of data and then to assess errors In these values.

Defining a protocol Identify the problem Identify the problem Answer the question of whether it is generalized or specific Answer the question of whether it is generalized or specific Evaluate the existing evidence Evaluate the existing evidence Construct an appropriate hypothesis Construct an appropriate hypothesis Establish one or more aims to test the hypothesis Establish one or more aims to test the hypothesis

Scientific inquiry loop The loop is the foundation for our research work The loop is the foundation for our research work The selection of the right hypothesis would lead directly to the formulation of a new testable hypothesis The selection of the right hypothesis would lead directly to the formulation of a new testable hypothesis The loop of science continues in these repeated small steps The loop of science continues in these repeated small steps

Methods of evaluating diagnostic procedures

Critical evaluation is necessary so that the use of given diagnostic procedures can be justified.

The Six levels model for assessing the efficacy of diagnostic technologies The Six levels model for assessing the efficacy of diagnostic technologies Various study designs that can be used to evaluate the different levels of efficacy Various study designs that can be used to evaluate the different levels of efficacy

Defintions Efficacy: The benefit from technology applied under ideal circumstances. Efficacy: The benefit from technology applied under ideal circumstances. Effectiveness: The use of technology in everyday usual circumstances. Effectiveness: The use of technology in everyday usual circumstances. Efficacy must be shown before effectiveness. Efficacy must be shown before effectiveness.

Six levels model for Dx efficacy Stage of efficacy Definition Technical Capacity Ability to produce an image Resolution, sharpness, reliability Dx Accuracy The ability to differentiate between normal and abnormal Sensitivity, Specificity, predictive values, OCR curves Dx impact Ability of a Dx test to affect the Dx workup Therapeutic impact Ability of a Dx test to affect the therapeutic choices Pt outcomes Ability of a Dx test to increase the length or quality of life Social outcomes Cost-effectiveness and cost utility

1 st level: Technical Efficacy The ability to produce an image (measured through the physical characteristics of the image The ability to produce an image (measured through the physical characteristics of the image This stage is similar to lab phase were technical parameters are refined. This stage is similar to lab phase were technical parameters are refined.

2 nd level: Dx accuracy The ability of a Dx test to differentiate normal from abnormal. The ability of a Dx test to differentiate normal from abnormal. Dx tests are compared with a gold standard truth. Dx tests are compared with a gold standard truth. 2X2 tables are the standard way of displaying the comparison between a new test (index test) and the gold standard (reference test). 2X2 tables are the standard way of displaying the comparison between a new test (index test) and the gold standard (reference test).

2X2 table Reference test (gold standard) PositiveNegative Positive TPTPTPTP FPFPFPFP Negative FNFNFNFN TNTNTNTN 2 nd level: Dx accuracy Index test (new test)

Reference test (gold standard) PositiveNegative Positive TPTPTPTP FPFPFPFP Negative FNFNFNFN TNTNTNTN Index test (new test) 2X2 table

Sensitivity & Specificity True Positive (TP). True Negative (TN). False Positive (FP). False Negative (FN).

The value of a diagnostic test lies in its sensitivity and specificity : The value of a diagnostic test lies in its sensitivity and specificity : Sensitivity : the ability of the diagnostic test to detect patients with disease. Sensitivity : the ability of the diagnostic test to detect patients with disease. Specificity : the ability of the diagnostic test to exclude patients with out disease. Specificity : the ability of the diagnostic test to exclude patients with out disease.

Sensitivity: Sensitivity: TP (detected disease) / TP+FN (all disease) Specificity: Specificity: TN (detected normal) / TN + FP (all normal) Sensitivity & Specificity

TOTALSNO DISEASELIVER DISEASE PRESENT SCAN Abnormal Normal Total

TOTALSNO DISEASE (D-) LIVER DISEASE PRESENT (D+) SCAN Abnormal (T+) Normal (T-) Total TP ratio = P( T+/D+)=TP ratio = P( T+/D+)= 231/ = / = 0.90 FP ratio= P( T+/D-)=FP ratio= P( T+/D-)= 32/ = / = 0.37 TN ratio= P( T+/D-)= (1-FP ratio) =54 / 54+32= 0.63= ( )TN ratio= P( T+/D-)= (1-FP ratio) =54 / 54+32= 0.63= ( ) FN ratio= P( T-/D+)= ( 1-TP ratio) = 27/ = 0.10= (1-0.90)FN ratio= P( T-/D+)= ( 1-TP ratio) = 27/ = 0.10= (1-0.90)

Receiver Operating Curve (ROC) Most tests yield continuous scale of values rather than a binary outcomes Most tests yield continuous scale of values rather than a binary outcomes Cut off points are selected on the continues scale to differentiate between patient with and without the disease. Cut off points are selected on the continues scale to differentiate between patient with and without the disease. cut off points will depend on the costs associated with classifying patient with disease as normal or classifying normal as diseased cut off points will depend on the costs associated with classifying patient with disease as normal or classifying normal as diseased

ROC Curve The ROC curve is a plot of sensitivity versus 1-specificty for a family of cutoff points that define positive and negative for a test The ROC curve is a plot of sensitivity versus 1-specificty for a family of cutoff points that define positive and negative for a test

Hypothetical ROC Curve Hypothetical ROC Curve

Selection of cutoff points is aided by the knowledge of : Selection of cutoff points is aided by the knowledge of : Knowledge of probability of Disease in patient population Knowledge of probability of Disease in patient population Cost associated with errors in Diagnosis Cost associated with errors in Diagnosis ROC Curve

Screen for fatal disease with safe treatment increase false positive Screen for fatal disease with safe treatment increase false positive Screening for less serious conditions with more dangerous treatment reduce false positive Screening for less serious conditions with more dangerous treatment reduce false positive

3- Diagnostic Impact Efficacy Measures of diagnostic impact attempt to quantify the importance of diagnostic thinking. Measures of diagnostic impact attempt to quantify the importance of diagnostic thinking. This is usually assessed using a questionnaire This is usually assessed using a questionnaire

4-Therapeutic impact efficacy Therapeutic impact assesses the degree to which a diagnostic test influences subsequent therapeutic choices. Therapeutic impact assesses the degree to which a diagnostic test influences subsequent therapeutic choices.

5-Patient outcome efficacy The patients perspective of the success or failure of the intervention The patients perspective of the success or failure of the intervention There are a verity of methods to evaluate this There are a verity of methods to evaluate this QALY quality adjust life QALY quality adjust life

6-Societal Efficacy Cost effectiveness analysis are now commonly incorporated into the evaluation of new technologies Cost effectiveness analysis are now commonly incorporated into the evaluation of new technologies

In Conclusion

Health is wealth and that research is important for overall development Health is wealth and that research is important for overall development Commit ourselves to ensure that health research improves health and quality of life of all people Commit ourselves to ensure that health research improves health and quality of life of all people health research is essential for improvement not only in health but also in social and economic development health research is essential for improvement not only in health but also in social and economic development