Presentation on theme: "A Selected Look at the Literature Base on Vocational Rehabilitation and Implications for Future Research Institute for Community Inclusion University of."— Presentation transcript:
A Selected Look at the Literature Base on Vocational Rehabilitation and Implications for Future Research Institute for Community Inclusion University of Massachusetts Boston 100 Morrissey Boulevard Boston, Massachusetts July 2010
Study Overview and Goals The National Institute for Disability and Rehabilitation Research (NIDRR) and Rehabilitation Services Administration (RSA) charged the Vocational Rehabilitation Research and Training Center (VR-RRTC) to conduct a systematic review of existing empirical research on vocational rehabilitation (VR) The goal of the systematic review was to characterize the public VR program in terms of its: Programmatic and systematic features. Types of customers served, and Kinds of outcomes achieved as reported in research studies. Overall, the review aimed to describe the VR research base including existing gaps in literature, and to make recommendations for future research investment.
Study Methods Over 12,000 possible documents identified through extensive literature search process 550 studies met criteria for study inclusion: Produced (published/unpublished) between 1970 and 2008 Written in English Carried out in the United States Focus on working age adults (22 and older) Empirical basis to research (qualitative or quantitative) Focus on the VR program in terms of system, services, and / or outcomes at an individual or program level Researchers reviewed and coded the 550 studies using a coding tool, and then synthesized the findings.
Findings: Description of Studies CharacteristicFinding Publication Year More than two-thirds of studies produced in last two decades Publication Type Most studies in form of peer-reviewed journal articles Funding Source More than half of the studies did not report funding source; of those reporting, most were NIDRR funded (116 studies) Geographic Scope Largest number of studies conducted at state level Study Scope Largest number of studies related to VR service delivery process, followed by VR performance and structure
Findings: Target Populations The majority of studies focused on programs and services targeting the general VR population 40% of studies focused on programs and services targeting sub- populations Sub-populations Particular types of disabilities Mental illness/emotional problems Visual impairments Mental retardation/developmental disabilities Other special populations Ethnic and racial minorities Persons with low income Older people Individuals who receive disability benefits or specialized services Supported employment Public benefits (e.g., Social Security, welfare) Workforce Investment Act
Findings: Study Topics Nearly 70% of studies examined VR services in general Remaining studies focused on ‘other’ types of programs or examined employment and job-related services Studies were divided into four general categories according to programs and services: CategoryNumber of studies Employment513 Health18 Independent Living37 Policy and Administration Action55 Note: A study could belong to more than one category
Findings: Outcomes Nearly 60% of studies reported at least one individual-level outcome Type of employment Wages Other employment related outcomes (e.g. quality of life, social participation) The majority of studies reported at least one agency- or program-level outcome Agency / program effectiveness Agency / program efficiency Staff capacity Agency / program cooperation
Findings: Study Design and Data Collection Study Design Secondary data analysis (SDA) most common design Followed by cross-sectional study and case study SDA studies most commonly relied on RSA data Less than 5% of studies used experimental (11) or quasi- experimental (13) designs Data Collection Quantitative survey most common data collection method Followed by SDA and qualitative interviews Slightly more than 20% of studies used more than one method of data collection More studies used SDA as a research design and data collection method between 2001 and 2008 than previous decades.
Findings: Study Quality Studies were assessed for quality based on aggregating several components including: Appropriate choice of research design Attempts to establish reliability and validity of data collection and analysis Ability to rule out sources of bias Generalizibility of findings Trustworthiness of conclusions Nearly half of studies were rated as “high quality” by reviewers Note: The study quality determination is only an indicator of the degree to which the research was reported in the study; it does not represent a judgment of the rigor or quality of the research itself.
Conclusions The observation of a wide range of topics, populations and outcomes covered by VR research is consistent with previous studies 1 stressing the need for more replication and expansion to build a knowledge base to better inform policy and practice Studies were found to be high in quality, but very reliant on administrative data such as RSA 911 data, as well as survey research Methodologies used to build evidence-based practice, increasingly found in social science research, are not currently widely utilized in VR research VR research has been limited in terms of depth of knowledge relating to “what works” in terms of practice, process or strategy 1 Saunders, J.L., Leahy, M.J., McGlynn, C., & Estrada-Hernandez, N. (2006). Predictors of employment outcomes for persons with disabilities: An integrative review of potential evidence-based factors. Journal of Applied Rehabilitation Counseling, 37(2), 3–20.
Study Implications A greater variety of research methods should be explored including: The use of additional datasets (beyond RSA 911) RSA data in combination with other sources of information such as qualitative interviews, focus groups or participant observation More utilization of participatory action research and other methods consistent with rehabilitation philosophy 2 VR community of researchers and practitioners should: Identify critical areas of need and invest in those areas Seek to generate knowledge that results in the development and adoption of best practices with the field of VR Utilize rigorous research methodology to address questions that are relevant to VR practitioners and customers 3 2 Balcazar, F.E., Keys, C.B., Kaplan, D.L., & Suarez-Balcazar, Y. (1998). Participatory action research and people with disabilities: Principles and challenges. Canadian Journal of Rehabilitation, 12, 105– Johnston, M.V., Vanderheiden, G.C., Farkas, M.D., Rogers, E.S., Summers, J.A., & Westbrook, J.D. for the NCDDR Task Force on Standards of Evidence and Methods (2009). The challenge of evidence in disability and rehabilitation research and practice: A position paper. Austin, TX: Southwest Educational Development Laboratory (SEDL).
Study Limitations Selection criteria created a subset of studies, not all studies pertaining to VR were included Review of studies across the methodology spectrum Studies were reviewed using a specific coding tool Reviewers had varying levels of research experience which may account for some differences in the study screening and coding process
Review Team Review Team: Heike Boeltzig, Ph.D. Research Associate, Institute for Community Inclusion (ICI) at the University of Massachusetts Boston (UMB) Martha Klemm Research Study Coordinator, ICI/UMB Allison Fleming Doctoral Student, Michigan State University Julisa Cully Program Manager, ICI/UMB Tina Mullins Reference Librarian, UMB Graduate Assistants: Paolo Infante Keith Lewandowski Christine Gottshall Kate Szenamici Melissa Manninen Luse For free copies of the report visit the Vocational Rehabilitation Research and Training Center website:
Input on Synthesis Study and Report Provided By: Senior ICI Research and Program Management Staff: –Susan Foley, Ph.D., Principal Investigator –William Kiernan, Ph.D. –John Halliday –Joe Marrone Senior Researchers from InfoUse: –Susan Stoddard, Ph.D. –Lita Jans, Ph.D. Advisory Board members: Michael Leahy, Ph.D., Professor and Director of the Office of Rehabilitation and Disability Studies, Department of Counseling, College of Education, Michigan State University, Barbara Elliott, Senior Researcher at the Research Triangle Institute, and John Harper, Assistant Director of Mental Health Services and Data Reporting, Missouri Division of Vocational Rehabilitation
Project Publications Executive Summary Final Report: A Selected Look at the Literature Base on Vocational Rehabilitation and Implications for Future Research. Reference list of 550 studies included in final report Available at