1. Development and Evaluation of a Study Design Typology
for Human Subjects Research
Simona Carini, MA1, Brad H. Pollock, PhD2,
Harold P. Lehmann, MD, PhD3, Suzanne Bakken, RN, DNSc4,
Edward M. Barbour, MS5, Davera Gabriel, RN6, Herbert K. Hagler, PhD7,
Caryn R. Harper, MS, CCRC7, Shamim A. Mollah, MA5, Meredith Nahm, MS8,
Hien H. Nguyen, MD6, Richard H. Scheuermann PhD7, Ida Sim, MD, PhD1
1 University of California, San Francisco, CA; 2 University of Texas Health Science Center, San Antonio, TX; 3 Johns Hopkins University, Baltimore, MD;
4 Columbia University, New York, NY; 5 The Rockefeller University, New York, NY; 6 University of California, Davis, CA; 7 UT Southwestern Medical Center, TX;
8 Duke University, Durham, NC
Funded in part by UL1RR (Columbia), UL1RR (Duke), UL1RR (Johns Hopkins),
UL1RR (The Rockefeller University), UL1RR (UC Davis), UL1RR (UCSF), UL1RR (UTHSC-San Antonio),
and UL1RR (UT Southwestern)‏ from the National Center for Research Resources (NCRR),
a component of the National Institutes of Health (NIH) and NIH Roadmap for Medical Research.

2. Outline Classification of study designs
The Human Studies Database Project
The Study Design Typology
Evaluation and results
Current Work
Conclusions

3. Classification of Study Designs (I)‏
Study designs are complex, with attributes such as populations, eligibility criteria, interventions, outcomes, study analyses.
Each of these attributes is crucial
in establishing what can be learned from the data- collection process,
in making a study relevant to a researcher or decision maker,
in permitting aggregation of data across studies.
An ontology, an abstract representation, of clinical research study designs is required.

4. Classification of Study Designs (II)‏
There are textbooks and articles about study types in a wide array of disciplines, each grounded in their own field of application and using their own terminology for common concepts.
What is required is an integrated typology that spans all study design types.
We set out to define and evaluate a general integrated typology of study design type.

5. Outline Classification of study designs
The Human Studies Database Project
The Study Design Typology
Evaluation and results
Current Work
Conclusions

6. The Human Studies Database (HSDB) Project
Context for needing and using a Study Design Typology is the Human Studies Database Project, a multi-institutional project among 14 Clinical and Translational Science Award (CTSA) institutions to federate study design descriptors of human research portfolio over a grid-based architecture.
Central to goal is the clear definition and classification of human research study types.

7. Outline Classification of study designs
The Human Studies Database Project
The Study Design Typology
Evaluation and results
Current Work
Conclusions

8. The Study Design Typology (I)‏
Use cases for using a Human Studies Database were first elicited from CTSA colleagues.
One of those use cases, “Research Characterization—Report research design,” had a high priority, motivating the need for a Study Design Typology.
Typology defined through an iterative process. The goal was to identify the most parsimonious set of factors that will correctly classify any study on individual humans into a set of common study design types.

9. The Study Design Typology (II)‏
The Typology scopes Human Studies as any study collecting or analyzing data about individual humans, whole or in part, living or dead. (Studies on populations, organisms, decision strategies, etc. are not considered human studies.)
Human studies are next classified as primarily Qualitative or Quantitative.

11. The Study Design Typology (III)‏
Quantitative studies are split into
Interventional Studies, where the investigator assigns one or more interventions to a study participant.
Observational Studies, where the investigator has no role in what treatment or exposures a study participant receives, but only observes participants for outcomes of interest.

12. Interventional Studies (I)‏
We defined 4 study design types for, based on factors deemed factual and independent of judgement, with permissible values exhaustive and mutually exclusive.
These high-level study types represent distinct approaches to human investigations.
Additional descriptors elaborate on secondary design and analytic features (some descriptors apply only to some study types).

14. Interventional Studies (I)‏
Four study design types, based on factors deemed factual and independent of judgement, with permissible values exhaustive and mutually exclusive.
These high-level study types represent distinct approaches to human investigations.
Additional descriptors elaborate on secondary design and analytic features (some descriptors apply only to some study types).

15. Interventional Studies (II)‏

17. Observational Studies
Five study design types, again based on factors deemed factual and independent of judgement, with permissible values exhaustive and mutually exclusive.
After the pilot evaluation, we removed one type.
These high-level study types represent distinct approaches to human investigations.
Additional descriptors elaborate on secondary design and analytic features (e.g., timing of measurements to participant selection).

18. Outline Classification of study designs
The Human Studies Database Project
The Study Design Typology
Evaluation and results
Current Work
Conclusions

19. Pilot Evaluation (I)‏ Evaluation focused on Quantitative Studies.
Four institutions (Columbia, Duke, UC Davis, UT Southwestern) provided the protocol or equivalent document describing the study design for 7-10 studies.
The Study Typology was translated into a SurveyMonkey survey and deployed online.
One respondent per institution classified the protocols from his/her institution and from two others.

21. Pilot Evaluation (II)‏

22. Pilot Evaluation (III)‏

23. Pilot Evaluation (IV)‏

24. Pilot Evaluation (V)‏
35 protocols classified into the 9 high-level designs in the quantitative sub-tree; some were non-human or qualitative studies.
For all 35 protocols, the Fleiss’ kappa, an overall measure of agreement, was
For the subset of 23 quantitative studies, the kappa was
A recurring theme in comments was the need for better definitions.
Evaluation results and subsequent discussion led to changes to the Study Typology.

25. Outline Classification of study designs
The Human Studies Database Project
The Study Design Typology
Evaluation and results
Current Work
Conclusions

27. Current Work
The Study typology is being incorporated into the study design module of the Ontology of Clinical Research (OCRe).
Plan for a larger evaluation with a broader sample and a wider range of users.
Use of a different tool for designing a user interface that would provide a more articulated path through the typology.
Incorporation of hybrid study designs in the classification.

28. Outline Need for a classification of study designs
The Human Studies Database Project
The Study Design Typology
Evaluation and results
Current Work
Conclusions

29. Conclusions
Definition of a typology of study design types for human research.
For quantitative studies, this typology defines discriminatory factors that partition quantitative research studies into one of eight high-level study design types.
Pilot evaluation demonstrated initial promise for reliably classifying study design types in human research.