Presentation is loading. Please wait.

Presentation is loading. Please wait.

Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University.

Similar presentations


Presentation on theme: "Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University."— Presentation transcript:

1 Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University of Connecticut Presentation at the Association on Higher Education and Disability Annual Conference Baltimore, MD July 11, 2013 Center on Postsecondary Education and Disability, ; AHEAD

2 About Us Allison – Assistant Professor, Department of Educational Psychology, Neag School of Education, UConn – Research Associate, Center on Postsecondary Education and Disability (CPED) Adam – Doctoral Student, Department of Educational Psychology, Neag School of Education, UConn Joe – Associate Professor, Department of Educational Psychology, Neag School of Education, UConn – Director of CPED, Neag School of Education, UConn – Member, NPSO Advisory Board Center on Postsecondary Education and Disability, ; AHEAD

3 Special Thanks Lyman L. Dukes III Michael Faggella-Luby Nicholas Gelbar Jennifer S. Kowitt Melissa Root Center on Postsecondary Education and Disability, ; AHEAD

4 Session Objectives To explain the rationale for conducting this comprehensive literature review To explain the background and methods used To present specific key findings to date To present suggestions for future research To facilitate discussion regarding future research directions Center on Postsecondary Education and Disability, ; AHEAD

5 Project Background The Vocational Rehabilitation Act of 1973 required all institutions of higher education to ensure access to qualified students with disabilities (SWD). Forty years after the passage of the Act, 11% of college freshmen report having a disability (U.S. G.A.O., 2009). The profession of disability services is now longer a nascent field in higher education Center on Postsecondary Education and Disability, ; AHEAD

6 Project Background To date, a comprehensive analysis of the literature dealing with disability and higher education has not been conducted This literature is broad in scope and dispersed across a variety of disciplines (e.g., special education, higher education, psychology, sociology) The 40-year anniversary of the passage of Section 504 provides an anniversary to review the field’s literature: – What topics have been studied? – What populations have been studied? – What methodologies have been employed? – How much of the literature is research? How much is not research? – What aspects of the field have substantial evidence and support? – What aspects are lacking substantial evidence and support? – What research areas within the field are likely to receive greater attention in coming years? Relevance to practitioners, researchers, policy makers Center on Postsecondary Education and Disability, ; AHEAD

7 Project Background Initially looked to a process used in secondary transition by the National Secondary Transition Technical Assistance Center (NSTTAC: Test, Fowler, Kohler & Kortering, 2010) To review evidence-based practices in secondary transition based on quality experimental studies Method: – Procedures: (a) electronic search, (b) reviewing reference lists, (c) hand searches of journals, and (d) updating by replicating the initial procedures – Criteria: (a) publ. after 1984, (b) SWD in subjects yrs, (c) IV or DV aligned to five areas of Kohler’s Taxonomy – Focus: Included systematic reviews and group or single subject design studies – Lens: Applied NSTTAC decision rules for determining levels of evidence (Strong, Moderate, Potential) – Total: 240 reviews and intervention studies Center on Postsecondary Education and Disability, ; AHEAD

8 8 Are based on rigorous research designs Have demonstrated a record of success for improving student outcomes Have undergone systematic review process using quality indicators to evaluate level of evidence Evidence- Based Practices Are based on rigorous research designs Have demonstrated a record of success for improving student outcomes Research- Based Practices Are based on research Have demonstrated limited success Have used a ‘weak’ research design Promising Practices Are not based on research Have no data to support effectiveness Based on anecdotal evidence and/or professional judgment Unestablished Practices (Helsel, Hitchcock, Miller, Malinow, & Murray, 2006; Twyman, 2008) Broad Definitions From Cameto, Mazzotti, & Test (2011) Center on Postsecondary Education and Disability, ; AHEAD 2013

9 Findings Identified 33 evidence-based practices in secondary Transition Categorized using Kohler’s Taxonomy for Transition Programming – 3 in Student-Focused Planning (e.g., involving students in IEP) – 26 in Student Development (e.g., life skills, purchasing skills) – 1 in Family Involvement (training parents about transition) – 3 in Program Structure (extending services beyond secondary school) – No practices identified in the area of Interagency Collaboration Only 2 evidence-based practices have a strong level of evidence: – teaching life skills, teaching purchasing skills 28 practices had a moderate level of evidence For more information, see: Test, D. W., Fowler, C. H., Richter, S. M., White, J., Mazzotti, V., Walker, A. R., & Kortering, L. (2009). Evidence-based practices in secondary transition. Career Development for Exceptional Individuals, 32(2), Center on Postsecondary Education and Disability, ; AHEAD

10 Project Background Genesis was a request from NSTTAC to present information about evidence-based practices regarding: – Successful transition to postsecondary education – Success in postsecondary education Our plan: – Initially, to follow the NSTTAC meta-analysis procedures – But, postsecondary education lacks a taxonomy for the literature – Postsecondary education does not use the evidence based practice standards required in secondary education – No prior sorting of the literature, either by topical or research categories – Required a regrouping and new direction Center on Postsecondary Education and Disability, ; AHEAD

11 Our Method Began with review of 80+ JPED articles from – Identifying common themes and topics – Development of broad content “domains” Identification of respective sub-domains Domains sent to past two JPED editors for feedback An electronic rating form was developed and revised JPED articles from 10 issues reviewed by four coders Reliability determined, team debriefing, further refinement of domains, rating form Center on Postsecondary Education and Disability, ; AHEAD

12 Initial Domain Descriptions Domain NameDomain Description Student Support and Services Transition into college, student retention, access to accommodations, access to assistive technology, determining eligibility, assessment, evaluation, and diagnosis, coaching Student Learning and Experiences Teaching students study skills, learning strategies, teaching self-determination, teaching self-advocacy, teaching students about legal rights and responsibilities, knowledge attitudes and beliefs (KAB), experiences as person with disability Postsecondary program structure Program development, program evaluation, policies and procedures, fit within the institution, collaboration with other campus services, legal compliance, determining eligibility Postsecondary Outcomes Transition to career or graduate school, employment outcomes Faculty/Staff Support Faculty development and training, faculty KAB; Staff development and training, staff KAB No Fit Studies that do not relate to any of the above domains Center on Postsecondary Education and Disability, ; AHEAD

13 Our Method Discussion revealed overlaps, determination of key terms – e.g., “policies and procedures”, “experiences” – Where does eligibility “belong”? – Difference between institutional and program legal compliance? – What about studies of instruments and proposed constructs? Domains collapsed and updated: – Student level – Program level – Faculty/staff level – Construct level JPED articles from an additional 5 issues reviewed by four coders Reliability determined at 75%-85% Debriefing lead to 100% agreement; refinement of terms, inclusion and exclusion criteria Center on Postsecondary Education and Disability, ; AHEAD

14 Our Method Concurrently, 500 articles from other sources collected Sorted into domains; reliability measured – 88% - 96% for sorting Articles provided a broader perspective and lead to further refinement of the subdomains Validity check by 8 former editors or co-editors of JPED – Measured the clarity of domain definitions all were strongly agree or agree that the definition is clear – Requested suggestions for missing domains – Fit of the subdomains – Suggestions for missing subdomains and clarification of subdomains (e.g., legal compliance at the program or institutional level) Center on Postsecondary Education and Disability, ; AHEAD

15 Inclusion and Exclusion Criteria Inclusion criteria: 1. The article is about Postsecondary Education for Students with Disabilities (broadly considered to include faculty, disability services, etc.) 2. The article is about one of the following topics/populations: a. Programs for accepted students into degree granting programs at a 2 or 4 year college or university b. Programs, services, or experiences of matriculated students c. Articles about the experiences of students with disabilities who have dropped out of degree granting programs at a 2 or 4 year college or university d. Articles about the experiences of students with disabilities who are graduates of degree granting programs at a 2 or 4 year college or university Exclusion criteria 1. Articles that are primarily about secondary students in transition or transition aged programs. Center on Postsecondary Education and Disability, ; AHEAD

16 Domain Descriptions Domain NameDomain Description Student-Level Studies Experiences and perceptions of students with disabilities in and after higher education. Program or Institution-Level Studies Service provision by the disability services office in a higher education institution. Can also relate to institutional policies and procedures pertaining to students with disabilities. Faculty/Non- Disability Support Staff-Level Studies Knowledge, attitudes, beliefs of faculty and non-disability services personnel to enhance access to higher education for students with disabilities. Also education or support for faculty and staff in this practice. Construct Development-Level Studies Development, evaluation, or validation of a variable, including development/validation of assessment instruments, evaluation metrics, theoretical models of service delivery, standards of practice, or ethics. The variable must be under proposal, in development, or being used in practice to gather empirical evidence. No Fit Studies that do not relate to any of the above domains. Center on Postsecondary Education and Disability, ; AHEAD

17 Sub Domains Student Level Studies Access (physical, cognitive, attitudinal) Assistive technology use Career development Experiences, perceptions, knowledge, attitudes, or beliefs of students with disabilities Learning/using study skills, learning strategies Mainstream technology use Meeting institutional requirements (e.g., degree requirements, foreign language requirements, math requirements) Post-undergraduate program experiences and/or outcomes (e.g., graduate school, employment) Profiles of students (e.g., diagnostic profiles, profiles of successful and/or unsuccessful students) Requesting or using accommodations (e.g., assistive technologies, separate testing location, course substitutions) Self-determination skills (e.g., self-advocacy, student goal attainment, self- disclosure, self-management, legal rights and responsibilities) Statistics on students with disabilities (e.g., rate of access to postsecondary education, student retention, graduation rate, statistics on accommodation use) 17

18 Sub Domains Program/Institutional Level Studies Collaboration with faculty or academic departments Collaboration with other campus services Experiences, perceptions, knowledge, attitudes, or beliefs of disability service providers General or specific descriptions of disability programs and resources and/or recommended program components Institutional Policies/Procedures Legal compliance (institutional specific) Legal compliance (program specific) Program development Programs for incoming students (e.g., freshmen, transfer students) Programs for students transitioning to graduate school or employment Programs for specific cohorts of students (e.g., LD, Aspergers, etc) Policies and procedures (e.g., determining student eligibility for services, determining reasonable accommodations, determining access to assistive technology) Professional development/training for disability services staff Program evaluation (e.g., student retention, student use of program related services, student graduation rates) Program fit within the institution (e.g., student affairs v. academic affairs) 18

19 Sub Domains Faculty/Non-Disability Support Level Studies Campus staff development and training Campus staff knowledge, attitudes and beliefs (e.g., about students with disabilities) Campus staff practices Faculty development and training Faculty knowledge, attitudes and beliefs (e.g., about students with disabilities; about providing accommodations) Faculty teaching practices 19

20 Sub Domains Construct Development Studies Assessment instruments (development, validation, use to develop diagnostic profiles) Conceptual models or discussion of issues in disability services (e.g., eligibility for services) Conceptual models of service delivery (e.g., Universal Design, other models) Conceptual models of instruction/assessment of learning Evaluation metrics or methods Instructional practices Standards of practice, performance or ethics. Other (including disability studies) 20

21 Method Search Terms Included: Academic Accommodation Accommodation ADD ADHD Blind College College Admission College Student Deaf Disabilities Disability Disabled Dyslexia Dyslexic Handicap Handicapped Hearing Impairment Postsecondary Education Mental Illness Mobility Impairment Postsecondary Education Student Affairs Student Personnel Student Services University University Student Visual Impairment Continued literature review and collection of articles 21

22 Method 1,210 articles identified by searches of multiple data bases (e.g., Academic Search Premier, EBSCO, PSYCinfo) Published between 1955 and 2012 Articles grouped into domains, reliability measured – Coding resulted in some articles shifting domains Will focus on student-level, program or institution-level, faculty/non-disability-level studies today Future steps will focus on coding construct development studies Center on Postsecondary Education and Disability, ; AHEAD

23 Method An electronic database was developed that included the reference citation for each article and unique alpha-numeric codes (e.g., Shaw4). The codes were designed to be entered into the electronic coding instrument, so that coding could be linked back to the reference citation. Each article randomly assigned to two coders Reliability determined for each domain Discrepancies discussed Center on Postsecondary Education and Disability, ; AHEAD

24 Instrument An electronic coding instrument was designed and refined with two pilots, multiple coders. The instrument allowed for the researchers to code: – Did the article meet inclusion criteria? – Did the article present original data? – If not research based, what type? (e.g., lit review, legal analysis) – If research based, what type? (with multiple layers) – What was the setting? (US, Canada, international, 2- or 4-year) – Sample information? (numbers, gender, disability, race, etc..) – Domain and sub-domain Across coding sheet, 148 choices were possible To achieve agreement, coders selections must be exact Center on Postsecondary Education and Disability, ; AHEAD

25 Inter-Rater Reliability For today’s presentation, three subsets of articles (Domains 1, 2, and 3) were analyzed. Each article coded twice to check for inter-rater reliability. Discrepancies discussed and reconciled Frequency and Reliability by Domain DomainnOverall Reliability Domain 1: Student-Level Studies 37682% Domain 2: Program or Institution-Level Studies 23586% Domain 3: Faculty/Non- Disability Support Staff-Level Studies 11086% Center on Postsecondary Education and Disability, ; AHEAD

26 Articles by Domain (Initial Sort) Domain NameN Student-Level Studies 500 Program or Institution-Level Studies 290 Faculty/Non-Disability Support Staff-Level Studies 125 Construct Development-Level Studies 134 No Fit 161 Center on Postsecondary Education and Disability, ; AHEAD

27 Frequency of Articles by Domain Over Time Center on Postsecondary Education and Disability, ; AHEAD

28 Journals with the Highest Frequency of Articles About Higher Education and Disability Unique Journals: 305 Center on Postsecondary Education and Disability, ; AHEAD

29 Journals with the Highest Frequency of Student- Level Articles Unique Journals: 172 Center on Postsecondary Education and Disability, ; AHEAD

30 Journals with the Highest Frequency of Program/Institutional-Level Articles Unique Journals: 109 Center on Postsecondary Education and Disability, ; AHEAD

31 Journals with the Highest Frequency of Faculty/Non-Disability Staff-Level Articles Unique Journals: 71 Center on Postsecondary Education and Disability, ; AHEAD

32 Journals with the Highest Frequency of Construct Development-Level Articles Unique Journals: 70 Center on Postsecondary Education and Disability, ; AHEAD

33 Proportion of Data-Based vs. Non-Data-Based Articles for Student-Level Studies Center on Postsecondary Education and Disability, ; AHEAD

34 Frequency of Data-Based vs. Non-Data-Based Student-Level Studies Over Time Center on Postsecondary Education and Disability, ; AHEAD

35 Proportion of Student-Level Studies by Research Methodology Center on Postsecondary Education and Disability, ; AHEAD

36 Proportion of Student-Level Studies With and Without Control/Comparison Groups Center on Postsecondary Education and Disability, ; AHEAD

37 Proportion of Student-Level Studies Containing a Control/Comparison Group by Experimental vs. Quasi-Experimental Methodology Center on Postsecondary Education and Disability, ; AHEAD

38 Proportion of Student-Level Studies by Location Note: 17 studies include multiple locations Center on Postsecondary Education and Disability, ; AHEAD

39 Location of Student-Level Studies Over Time Center on Postsecondary Education and Disability, ; AHEAD

40 Proportion of Student-Level Studies Including Clear Sample Size Data Center on Postsecondary Education and Disability, ; AHEAD

41 Proportion of Student-Level Studies Including Data on the Race/Ethnicity of Participants Mean = 6,136 SD = 91,021 Min = 1 Max = 1,502,658 Center on Postsecondary Education and Disability, ; AHEAD

42 Proportion of Student-Level Studies Including Disability-Related Demographic Information Center on Postsecondary Education and Disability, ; AHEAD

43 Proportion of Student-Level Articles Including Data About the Gender of Participants Center on Postsecondary Education and Disability, ; AHEAD

44 Proportion of Student-Level Articles Including Data on the Class Standing of Participants Center on Postsecondary Education and Disability, ; AHEAD

45 Proportion of Student-Level Articles Including Data About the Non-Student Participants Center on Postsecondary Education and Disability, ; AHEAD

46 Thirteen Subdomains of Student-Level Studies and Their Frequencies (Articles could be coded as multiple subdomains) Experience, perception, knowledge, attitude of SWD (n = 161) Profiles of SWD (n = 54) Learning/using learning strategies (n = 30) 20 ≤ Statistics on SWD (n = 19) Requesting or using accommodations (n = 18) Self-determination (n = 17) Access (physical/cognitive/attitudinal) (n = 14) Assistive technology use (n = 11) 10 – 19 Career development (n = 9) Mainstream tech use (n = 6) Other (n = 6) Meeting institutional requirements (n = 5) Post-undergraduate outcomes (n = 1) ≥ 9 Center on Postsecondary Education and Disability, ; AHEAD

47 Proportion of Data-Based vs. Non-Data-Based Program/Institution-Level Studies Center on Postsecondary Education and Disability, ; AHEAD

48 Frequency of Data-Based vs. Non-Data-Based Program/Institution-Level Studies Over Time Center on Postsecondary Education and Disability, ; AHEAD

49 Proportion of Program/Institution-Level Studies by Research Methodology Center on Postsecondary Education and Disability, ; AHEAD

50 Proportion of Program/Institution-Level Studies With and Without Control/Comparison Groups Center on Postsecondary Education and Disability, ; AHEAD

51 Proportion of Program/Institution-Level Studies by Location Note: 42 studies include multiple locations Center on Postsecondary Education and Disability, ; AHEAD

52 Location of Program/Institution-Level Studies Over Time Center on Postsecondary Education and Disability, ; AHEAD

53 Proportion of Program/Institution-Level Studies Including Data About the Non-Student Participants Center on Postsecondary Education and Disability, ; AHEAD

54 Sixteen Subdomains of Program/Institution- Level Studies and Their Frequencies (Articles could be coded as multiple subdomains) Institutional policies/procedures(n = 51) Prof. development/training for DSPs (n = 40) Program policies and procedures(n = 35) Legal compliance (Program specific) (n = 23) Experience, knowledge, attitudes,, beliefs of DSPs (n = 20) 20 ≤ 10 – 19 Legal compliance (Institution specific) (n = 9) Programs for incoming students (n = 8) Progs for students transitioning to grad school/employment (n = 8) Collaboration with other campus services (n = 7) Programs for specific cohorts of SWD (n = 7) Descriptions/recommendations of disability programs/resources ( n = 5) Program evaluation (n = 5) Program fit within institution (n = 3) Program development (n = 2) Other (n = 1) ≥ 9 Center on Postsecondary Education and Disability, ; AHEAD

55 Proportion of Data-Based vs. Non-Data-Based Faculty/Non-Disability Support Staff-Level Studies Center on Postsecondary Education and Disability, ; AHEAD

56 Frequency of Data-Based vs. Non-Data-Based Faculty/Non-Disability Support Staff-Level Studies Over Time Center on Postsecondary Education and Disability, ; AHEAD

57 Proportion of Faculty/Non-Disability Support Staff-Level Studies by Research Methodology Center on Postsecondary Education and Disability, ; AHEAD

58 Proportion of Faculty/Non-Disability Support Staff- Level Studies With and Without Control/Comparison Groups Center on Postsecondary Education and Disability, ; AHEAD NOTE: The 1 disagreement exists because one coder indicated that the study was an article without a control/comparison group while the other coder left the data point blank.

59 Proportion of Faculty/Non-Disability Support Staff-Level Studies by Location Note: 7 studies include multiple locations Center on Postsecondary Education and Disability, ; AHEAD

60 Location of Faculty/Non-Disability Support Staff- Level Studies Over Time Center on Postsecondary Education and Disability, ; AHEAD

61 Proportion of Faculty/Non-Disability Support Staff-Level Studies Including Data About the Non-Student Participants Center on Postsecondary Education and Disability, ; AHEAD

62 Six Subdomains of Faculty/Non-Disability Staff- Level Studies and Their Frequencies (Articles could be coded as multiple subdomains) Faculty knowledge, attitudes and beliefs (n = 45) Faculty teaching practices (n = 23) 20 ≤ Faculty development and training (n = 17) Campus staff knowledge, attitudes, and beliefs (n = 15) 10 – 19 Campus staff practices (n = 9) Campus staff development and training (n = 6) ≥ 9 Center on Postsecondary Education and Disability, ; AHEAD

63 Discussion Articles on higher education and disability have been published in 305 unique journals These journals have a range of purposes, styles, level of rigor Articles at the Student-Level are by far the most common (n = 376) followed by Program/Institutional-Level (n = 235) The overall number of published articles in the field increased through the ‘90’s and early ‘00’s, but is declining in past 5 year period Center on Postsecondary Education and Disability, ; AHEAD

64 Discussion –Good News JPED leads the way, but the field is represented in a multi- disciplinary range of journals 85% of the articles at the Student-Level are data-based The data-based studies in this area increased steadily over the past 30 years (e.g., less than 5 in early 80’s to 100 in ) Number of studies at international institutions have steadily increased over time Student-Level studies feature a range of ethnicities A range of methods are used: – Quantitative – 54% – Qualitative – 32% Center on Postsecondary Education and Disability, ; AHEAD

65 Discussion –Concerns (or opportunities!) Very limited number of experimental or quasi- experimental studies (n = 8) Limited number of studies with control groups (n = 8) Only 8% of the Student-Level studies are at 2-year institutions – Trend line has not increased over time Better descriptions of samples needed – Size, race, class standing Need more data-based studies at the Program/Institutional level (61% are non data-based) – Trend increased in the 1990’s, then dropped through the 2000’s Center on Postsecondary Education and Disability, ; AHEAD

66 Discussion –More Opportunities… Most studies at the Student-Level are on experiences, perceptions, KAB’s (n = 161) More articles needed on: – Learning strategies (n=30) – Self-determination (n=17) – Access (n=14) More studies needed: – Program/Institutional-Levels – Faculty/Staff-Level Center on Postsecondary Education and Disability, ; AHEAD

67 Discussion –Limitations Not possible to gather every published article – Search terms as broad as possible – Use of a range of data-bases Domains and codes for data-collection determined by the research team – Iterative process – Examined multiple journals – Feedback from outside experts Coding errors – Each article double coded – Reconciliations Center on Postsecondary Education and Disability, ; AHEAD

68 Next Steps Code articles for Domain 4 Additional data-cleaning Additional analysis to observe frequencies, trends over time, trends by journals, by locations, etc… Deep drill downs into specific areas to identify evidence-based practices, promising practices, etc… Center on Postsecondary Education and Disability, ; AHEAD

69 References Cameto, R., Mazzotti, V. L., & Test, D.W. (2011, April). High-quality research in secondary transition: Current status and future need. DCDT Showcase presented at the Council for Exceptional Children Annual Convention, Nashville, TN. Helsel, F. K. I, Hitchcock, J. H., Miller, G., Malinow, A., & Murray, E. (2006). Identifying evidence-based, promising and emerging practices that use screen-based technology to teach mathematics in grades K- 8: A research synthesis. Presented at AERA 2006 Meeting, San Francisco, CA. Rehabilitation Act of 1973, 29 U.S.C. § 701 et seq. Test, D. W., Fowler, C., Kohler, P., & Kortering, L. (2010, August). Evidence-based practices and predictors in secondary transition: What we know and what we need to know, Executive Summary.(Revised). Charlotte, NC: National Secondary Transition Technical Assistance Center. Available athttp://www.nsttac.org/sites/default/files/assets/pdf/pdf/ebps/ExecsummaryPPs.pdfhttp://www.nsttac.org/sites/default/files/assets/pdf/pdf/ebps/ExecsummaryPPs.pdf Test, D. W., Fowler, C. H., Richter, S. M., White, J., Mazzotti, V., Walker, A. R., & Kortering, L. (2009). Evidence-based practices in secondary transition. Career Development for Exceptional Individuals, 32(2), U.S. Department of Education, National Center for Education Statistics. (2012). Digest of Education Statistics, 2011 ( ) Center on Postsecondary Education and Disability, ; AHEAD

70 Q & A Thank you!! Center on Postsecondary Education and Disability, ; AHEAD


Download ppt "Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University."

Similar presentations


Ads by Google