Randomized Clinical Trials: The Versatility and Malleability of the “Gold Standard” Wing Institute Jack States Ronnie Detrich Randy Keyworth “Get your facts first, then you can distort them as you please.” (Mark Twain)

Randomized Clinical Trials (RCT) History  The use of RCT in agricultural experiments was pioneered by R. A. Fisher in the 1920’s.  The Tuberculosis Trials evaluating the effectiveness of streptomycin, is generally accepted as the first RCT - Post World War II

Randomized Clinical Trial Definition A research study in which the participants are assigned groups to objectively compare different interventions.  The use of chance to assign groups results in groups being similar  The effects of the intervention can be evaluated based upon the statistical assumption that any error (a). can be calculated; and (b) can be made very small

Why Use Randomized Clinical Trial? “To make statements of cause and effect regarding an intervention” RCT is recognized as a sound scientific method By the 1960’s RCT had established a track record as the most accepted methodology

How Does Randomized Clinical Trial Achieve It’s Designed Purpose? Control of Bias 1.Subject Selection Bias Increases internal validity by minimizing subject differences. 2.Study Evaluation Bias Limits inadvertent tampering with results

Strengths of RCT Design Statements of internal validity Statements of external validity (Limited to the populations studied) Demonstrates strong social validity as the most commonly accepted method for evaluating research  Governmental entities - World Health Organization and US Department of Education  Professional associations - AMA and APA  Advocacy organizations - Cochrane and Campbell Collaborations

Benefits of Randomized Clinical Control Design Valuable tool for making policy decisions in the selecting interventions with a strong evidence base When combined with a public health approach RCT offers a cost effective strategy to influence behavior on a broad scale through selection of effective universal interventions A way to avoid politically motivated, fad-of-the- month educational practices

Limitations Philosophical and Ethical Issues “RCT creates a moral dilemma” When the need to advance science is weighed against the obligation to offer each student optimal treatment  Practitioner has an obligation to chose an intervention in the best interest of subjects  A process must exist to protect subjects Achieving cultural acceptance

Limitations Strategic Issues Single studies are not sufficient to establish causation Difficult with low incident populations Requires specialized expertise to obtain approval, implement, and analyze data “If you torture data sufficiently, it will confess to almost anything” Dr.Fred Menger

Limitations Strategic Issues Limited ability for use making conclusions for individuals Substantial time required to complete studies Limited number of data points per condition (you can’t see how independent variable “works” only that it works)

Limitations Tactical Issues Issues of treatment fidelity and integrity Timely response to proliferation of new practices Logistical Issues Challenges finding adequate numbers of subjects Expensive requiring substantial resources Limited number of studies currently available

Issues for Behavior Analysis (BA) RCT is considered a social science methodology vs. the natural science methodology preferred by BA BA’s focus is on the individual and the methodology of choice is Single Subject Design Question: Should BA adopt RCT as a part of it’s research methodology?

Recommendations BA needs to participate in use of RCT  To remain a player in discussion and policy making  Increase the acceptance of BA “To make the general contributions of which our science is capable, behavior analysts will have to use methods of wider generality, in the sense they affect many people at the same time.” Murray Sidman The Behavior Analyst - Fall 2006

Recommendations Increase the number of studies Increase education and training of consumers and practitioners in strength and weakness of RCT Integrate RCT into an effective “Hierarchy of Evidence” model that takes advantage of different methodologies.