1. Development of an Online Suite of Career Exploration, Substance Abuse and Mental Health Assessments in American Sign Language for use when working with Deaf and Hard of Hearing(d/hh) students. Deb Guthmann, Ed.D. Wright State University & Josephine Wilson D.D.S., Ph.D Wright State University The Substance Abuse Resources and Disability Issues(SARDI) Program, Boonshoft School of Medicine

2. Learning Objectives 1. Participants will become familiar with the process being used to translate into American Sign Language career exploration, mental health and substance abuse assessments that transition staff, college counselors or others in higher education could use when working with d/hh students. 2. Participants will become familiar with the wide range of assessments being translated into ASL i.e. O*NET Interest Profiler, Satisfaction with Life Scale (SWLS), Rosenberg Self-Esteem Scale(RSES), Beck Depression Inventory(BDI), Alcohol Use Disorders Identification Test (AUDIT), Drug Abuse Screening Test (DAST), Directions for the Wide Range Interest and Occupation Test - Second Edition (WRIOT-2), Global Appraisal of Individual Needs Short Screener (GAIN-SS). 3. Participants will see a demonstration of the online suite of instruments being translated and validated in ASL for use with d/hh students. 4. Using examples from the presentation, participants will be able to identify how they may be able to use the ASL assessments with d/hh students in transition programs or higher education settings.

3. Wright State University- Federal Grant Updates 2011- 2014 - Field-Initiated Program grant through The National Institute on Disability Independent Living and Rehabilitation Research (NIDILDR). The goals were to investigate the Effectiveness of DODA based on five outcome measures, administered at the beginning of treatment and six months after treatment ended as compared with two specialized inpatient treatment programs for deaf and hard of hearing people. 2014-2019 - National Institute of Disability, Independent Living and Rehabilitation Research, (NIDILRR) Disability and Rehabilitation Research Projects (DRRP). Develop & validate a suite of web-based assessments for administration and scoring in American Sign Language (ASL) that will reduce barriers for those working in higher education, vocational rehabilitation, mental health and substance abuse.

4. Online 12 Step Meetings & Other Resources https://doda.omnijoin.com/ 1. Online 12 step meetings in text & ASL Last Monday of month, 7:00pm (EST) (Men Only, all signers welcome Deaf and hearing. Wednesday, 8:00pm (EST) AA - Text Chat (All signers welcome Deaf and hearing). Friday, 8:00pm (EST) (Women Only, all signers welcome Deaf and hearing). 2. Daily Thought/Daily Meditation/daily Prayer in ASL

5. CONTACT INFORMATION Deb Guthmann – dguthmann@aol.comdguthmann@aol.com

6. Grant Components 1. Norming and validation of The O*NET Short Form Interest Profiler (web-based 60-item), Work Importance Profiler, and Ability Profiler. 2. Norming and validation of 7 mental health and substance abuse assessment tools. 3. Training on the use of the online suite of assessment tools.

7. Disability and Rehabilitation Research Project(DRRP) Wright State University (WSU) partnered with: 1. An Advisory Board consisting of professionals who work with deaf and hard of hearing clients in VR, mental health, and other social services areas from around the U.S. 2. An Expert panel made up of individuals who; –work with deaf and hard of hearing clients in VR, higher education, mental health and other social service areas from around the US. –have expertise in assessment and who have been involved in previous translation projects.

8. Grant Components Instruments being translated and validated into ASL include: 1. Alcohol Use Disorders Identification Test (AUDIT)(10 items) 2. Beck Depression Inventory(BDI)(21items), 3. Drug Abuse Screening Test (DAST)(10 items), 4. Directions for the Wide Range Interest and Occupation Test - Second Edition (WRIOT-2), 5. Rosenberg Self-Esteem Scale (RSES)(10 items), 6. The Satisfaction with Life Scale (SWLS)(5 items), 7. The O*NET Short Form Interest Profiler (web-based 60-item), Work Importance Profiler, and Ability Profiler, 8. Global Appraisal of Individual Needs Short Screener (GAIN- SS)(23 items), (field testing)

9. ASL Translation Process Step 1 >> Forward Translation Original Language Target Language (English) (American Sign Language) Moo! ! Outcome is ASL version #1

10. ASL Translation Process Step 2 >> Back Translation Original Language v2 Target Language (English-back translated) (American Sign Language) Moo! !

11. ASL Translation Process Step 3 >> Back Translation Reconciliation Original Language Original Language v2 (English) (English-back translated) This is what we hope to see… Compare We are focusing on the meaning – are the English meanings the same? If there was another commonly-used, equivalent term in English it would be acceptable too. Moo!

12. ASL Translation Process We don’t want to see this… Original Language Original Language v2 (English) (English-back translated) Moo! Compare Inconsistencies in the English versions are like “red flags" – they signal a possible problem. Possible problems include mistranslated words, omissions, sentences that lose their meaning during translation, cultural differences between languages (literal, implied) Moo! Mau mau! Quack! Moooooo! Outcome is ASL version #2

13. Cognitive Interviewing 1. Input was obtained from potential consumers. –If the item content is not meaningful to consumers or they do not understand the language of the items, the instrument will not serve its purpose. 2. Cognitive interviews were conducted individually with 20 deaf adults to get their feedback on the signed items each instrument being validated. 3. Individuals watched each item and then provided feedback on the clarity of the signing and their understanding of the meaning of each item.

14. Field Testing  Each instrument completed on line by 300 VR and 150 non VR deaf individuals(each compensated for their time).  Field testing doesn’t occur until we satisfactorily complete:  Forward Translation,  Back Translation,  Back Translation Reconciliation,  Cognitive Interviewing.

15. 1. F ORWARD T RANSLATION. 2. B ACK T RANSLATION ( N = 30) 3. B ACK T RANSLATION R ECONCILIATION - B ASED ON THE RECONCILIATION OF BACK TRANSLATION, VIDEO CLIPS ARE REDONE AS NEEDED AND BACK TRANSLATION REPEATED FOLLOWED BY RECONCILIATION AND RE - TAPING AS NEEDED. 5. C OGNITIVE I NTERVIEWING ( N = 20) - RECONCILIATION OF COGNITIVE INTERVIEWING, REDO OF VIDEO CLIPS AS NEEDED. 6. F IELD T ESTING (300 MEMBERS OF THE DEAF COMMUNITY / 150 VR CONSUMERS ). 7. P SYCHOMETRICS ARE RUN AS PART OF THE VALIDATION PROCESS. ASL Translation Process in a Nutshell

16. Back Translation N= 30 native ASL users i.e. deaf or CODAs from diverse parts of the United States Based on back translation feedback, revisions to the ASL for some questions occurred and were re-taped for the SWLS, RSES, AUDIT, O*NET, & BDI-II. Each time re-taping occurs, back translation is redone before moving to the next step which is cognitive interviewing.

17. Back Translation

19. Beck Depression Inventory(BDI-II)  21 questions + directions = Over 100 video clips  Based on feedback from back translation and cognitive interviewing, some questions re-taped three times.

21. Beck Depression Inventory(BDI-II) 21 questions + directions Over 100 video clips 1.Sadness 0 I do not feel sad. 1 I feel sad much of the time. 2 I am sad all the time. 3 I am so sad or unhappy that I can't stand it. 2. Pessimism 0 I am not discouraged about my future. 1 I feel more discouraged about my future than I used to be. 2 I do not expect things to work out for me. 3 I feel my future is hopeless and will only get worse.

22. Beck Depression Inventory(BDI-II) 21 questions + directions Over 100 video clips 1.Sadness 0 I do not feel sad. 1 I feel sad much of the time. 2 I am sad all the time. 3 I am so sad or unhappy that I can't stand it. 2. Pessimism 0 I am not discouraged about my future. 1 I feel more discouraged about my future than I used to be. 2 I do not expect things to work out for me. 3 I feel my future is hopeless and will only get worse.

23. Beck Depression Inventory(BDI-II) Issues that came up during reconciliation 1.Pick out the statement in each group that best represents the way you have been feeling for the past 2 weeks including today: ASL GLOSS: Understand recent two-weeks-til-today, past-til- today. 2.I don't enjoy things as much, as I used to. ASL Gloss: Compare 2 weeks up til-now me enjoy less. 3.I feel more discouraged about my future than I used to be. ASL Gloss: Compare, past, my future, I feel now, more negative, soft thumb down, less hopeful (hope sign and less sign)

24. BDI-II- Issues that came up during reconciliation 4. How best to explain some word concepts clearly in ASL(example): Agitation 0 I am no more restless or wound up than usual. 1 I feel more restless or wound up than usual. 2 I am so restless or agitated that it's hard to stay still 3 I am so restless or agitated that I have to keep moving or doing something. Agitation - itself both body, mind, restless, 5-nervous, upset, 5-irritated Restless or wound up - my body mind both restless, 5-nervous, upset

25. BDI-II- Issues that came up during reconciliation 5. All of the BDI items have 4 possible responses with the exception of #16 and #18 that has 7 possible responses. 16. Changes in Sleeping Pattern 0 I have not experienced any change in my sleeping pattern. la I sleep somewhat more than usual. Ib I sleep somewhat less than usual. 2a I sleep a lot more than usual. 2b I sleep a lot less than usual. 3a I sleep most of the day. 3b I wake up 1-2 hours early and can't get back to sleep. Included a video clip indicating directions prior to #16, after #16 prior to #18 and after #18 Example: ASL GLOSS: lately, you see 4 sentences 4-4-4. now this #16 you will see 7 sentences, understand you same lately you need pick one answer, same. you pick One. ASL GLOSS: now #17, you will see 4 sentences again. you pick which one sentence best fit-you what you feel, understand two-weeks up-til-now, now. You pick one.

27. Satisfaction With Life (SWLS)

28. Back Translation Reconciliation-SWLS (Example) In most ways my life is close to my ideal. So far, my life feels perfect. My life from past to present is somewhat perfect. My life so far seems to be okay. My life, overall, seems to be satisfactory. My life to this date seems to be satisfactory. Overall, my life has been adequate. My experience with life seem content-normal. My whole life up until now is good I feel my life has been perfect all of my life.

29. Back Translation Reconciliation-SWLS (Example) In most ways my life is close to my ideal. The basic meaning of the item appears to have been captured well given the closeness in meaning between the English and ASL across all the respondents. ……..However…. There is one exception. The last concept- close to ideal/perfect is missed by some respondents- why?  The signer was lacking facial expression to match "perfect", viewers expressed confusion as to whether the intended meaning was perfect/ideal, just pretty good, or satisfactory/adequate.

31. Rosenberg Self Esteem Scale (RSES)

32. RSES: I feel that I am a person of worth, at least on an equal plane with others. A lot of focus in on self-esteem for the person of worth, but does it get the message across of being equal to others? Most people translated it as: “I have strong self-esteem, “Others around me have strong self-esteem”, “We have the same self- esteem" – A fair number of others did not get that. Does the ASL translation fits the intended English meaning... seeing oneself as a "person of worth" and being on equal footing with others... it's more like saying I am just as important as other people. It does not have to relate to "self-esteem" specifically. More like I am no less important than anybody else

34. Translation Challenges AUDIT(Audience Participation) How often do you have six or more drinks on one occasion? How would you sign: on one occasion How would you sign: six or more drinks How would you sign the whole sentence: How often do you have six or more drinks on one occasion?

35. Translation Challenges AUDIT(Audience Participation) How often do you have six or more drinks on one occasion? 1. ASL Gloss: You drink example.. Beer, wine or hard alcohol like vodka or whiskey. Happen how often? You with friends group at event or out good time or sit alone, drink flat flat flat 6 or more beer, wine or whiskey. 2. ASL Gloss: How often, you alone at home, with friends at party, out good time doesn't matter, do you drink 6 or more drinks like beer, wine or hard alcohol like vodka or whiskey flat, flat flat ? 3. ASL Gloss: Before, one time, Either alone or with friends, it doesn't matter, do you drink beer, wine, alcohol 6 or more drinks flat, flat, flat all one time. 4. ASL Gloss: How often, if(suppose sign) you alone at home for a few hours, with friends at party, out good time doesn't matter, do you drink 6 or more drinks like beer, wine or hard alcohol like vodka or whiskey flat, flat flat (drinking the cup sign) 5. ASL Gloss: One time, you go one event, you with friends, you alone, no matter? you finish drink 6 above beer, wine, whiskey, flat, flat, flat, understand 6 above drinks? how often that happen?

36. Translation Challenges AUDIT(Audience Participation) How often do you have six or more drinks on one occasion? Video Clip feedback: I would like to see "beer, wine, or hard alcohol like vodka, whiskey etc" Leaving out hard alcohol isn't realistic if you really want to know about a person's drinking problem. Implied in the statement of "6 or more drinks" is that this person drink 6 drinks over a short span of time so I think using the sign that looks like a "flat tire" to show drink after drink, would make the message clearer. Video Clip feedback: I don't like "one time one day". In the back translation the "one time one day" doesn't show because it is clarifier for the ASL message, but it doesn't accurately clarify. It appears to conflict with the intended message because it gives the appearance that you are asking for that one time that most anyone has drank too much. It doesn't show the implication that this could be a recurring behavior. Suggestion: Talk now about drinking. Beer, wine, hard alcohol (alcohol sign) like vodka whisky etc. How often, you with friends or you alone sit flat, flat, flat beer or wine or hard alcohol(alcohol sign)l 6 or more drinks.

37. O*NET O*NET = https://www.onetonline.org/

38. O*NET

39. O*Net  Comprehensive database system for collecting, organizing, describing, & disseminating data on job characteristics & worker attributes.  Employability tool.  Helps organizations recruit & select workers.  Helps workers choose organizations.  Helps career counselors provide information

40. O*Net Interest Profiler  Can be self- administered and interpreted with student input during all stages.  Can be used on a stand-alone basis or with other O*NET Career Exploration Tools or with privately developed instruments  Results can be directly linked to over 800 occupations in O*NET on-line  Approximately 30 minute completion time

41. Career Exploration Tools O*NET Ability Profiler O*NET Interest Profiler O*NET Computerized Interest Profiler O*NET Interest Profiler Short Form (online) O*NET Work Importance Locator O*NET Work Importance Profiler

42. Implications O*Net Job Descriptions linked to: Wages Video clips of occupations Occupational Outlook (National and State) Incumbent’s education levels Specific job openings

43. Tools to measure Interests Abilities Importance (values)

44. O*NET Career Exploration Tools Computerized versions Software Materials Free downloads Online version Paper and pencil versions Order from Department Of Labor Download print shop files for onsite printing

45. O*NET Interest Profiler (IP) Self-assessment career exploration tool with resources available on line. Helps students discover work activities and occupations of interest. Students identify and learn about broad interest areas most relevant to themselves. Use as stand-alone or with other O*NET Career Exploration Tools or other instruments. Results directly linked to >800 occupations in O*NET.

46. Interest Profiler (IP) Process Select 8th grade reading level, >14 years old 60 items (short form; 6 scales with 10 items) Reliable and valid Test-retest reliability =.78 -.86 for short form High convergent validity Administer Self administered or in a group or with assistance 20 to 60 minutes (computerized in one session)

47. Holland Codes (RIASEC) used in IP

48. O*NET Online Home Page

50. IP Short (My Next Move)

51. Interest Profiler Results This section displays your individual results

52. Interests+Job zones=career Information page explaining how to "focus your search"- displayed before the job zones section

53. Select a Job Zone This allows the user to select a job zone to see the results.

54. Job Zone-my results Shows career possibilities within each zone.

63. O*Net Interest Profiler PRO Free Holland Codes ◦ RIASEC (Realistic, Investigative, Artistic, Social, Enterprising, and Conventional) Direct link to the O*Net Useful for career counseling CONCERN 8 th grade reading level Client computer sills (mouse and navigation)

64. Item Development  O*Net Interest Profiler

67. O*NET – ASL Glosses

68. Forward Translation-O*Net Interest Profiler Teach an individual an exercise routine ASL GLOSS: teach person how exercise Help people with personal or emotional problems ASL GLOSS: help people they have either internal, emotional issues, struggles. Example people themselves inside feel bad, sad, confused. Proofread records or forms ASL GLOSS: you look, edit papers, forms Calculate the wages of employees ASL GLOSS: people they work how-much they earn, you figure (?)

69. Forward Translation-O*Net Interest Profiler YOUR TURN Manage a department within a large company ASL GLOSS??? Drive a truck to deliver packages to offices and homes ASL GLOSS??? Start your own business ASL GLOSS??? Negotiate business contracts ASL GLOSS???

70. Forward Translation-O*Net Interest Profiler YOUR TURN Manage a department within a large company ASL GLOSS: Business big OR Business--many people groups. Point to one group. You boss that one. Drive a truck to deliver packages to offices and homes ASL GLOSS: you drive TRUCK, you deliver boxes where? Offices, homes Start your own business ASL GLOSS: set-up your OWN business Negotiate business contracts ASL GLOSS: boss you. You-two negotiate, discuss. Agree cost.

71. Global Appraisal of Individual Needs(GAIN) Short Screener in ASL (GAIN-ASL-SS) Used in diverse settings (e.g., schools, mental health clinics, substance abuse programs, workplaces, health clinics, child welfare, justice) The GAIN-ASL-SS needs to be field tested to complete the validation process.

72. What is the GAIN-SS?  Global Appraisal of Individual Needs – Short Screener (GAIN- SS)  23 item screening instrument used to quickly identify or rule-out general behavioral health disorders  Identifies problems along 4 dimensions: Internalizing Disorder Externalizing Disorder Substance Use Disorder Crime/Violence Problems  There is also a Total Symptom Score

73. What is the GAIN-SS?  Items are answered on a “recency” scale… “When was the last time you had this problem?” –In the past month –2-3 months ago –4-12 months ago –1 or more years ago –Never  Staff or self-administered  Administer with paper & pen OR via computer administration OR via web-based administration  Takes about 10 minutes in English and 10-20 minutes in ASL.

75. Project Goal The goals of this NIDRR(DRRP) grant are to:  Develop a suite of web-based assessments for administration and scoring in American Sign Language (ASL) that will reduce barriers for those working in the fields of vocational rehabilitation, mental health and substance abuse.  Enhance the employment outcomes of individuals who are deaf and hard of hearing.

76. Global Appraisal of Individual Needs(GAIN) Short Screener in ASL

78. Reliability/Internal Consistency Cronbach’s alpha will be calculated for each scale or subscale. Equivalence For each scale, 100 bilingual (ASL/English) individuals will be randomly assigned to one of two groups that receive either: 1) the instrument in ASL only; or 2) the instrument in English only. Means and standard deviations for the groups will be compared, and a t test for independent samples will be conducted for each instrument. Psychometric Analyses

79. Evidence for Validity  Validity is the extent to which a test measures what it claims to measure. It is vital for a test to be valid in order for the results to be accurately applied and interpreted.  Validity isn’t determined by a single statistic, but by a body of research that demonstrates the relationship between the measure and the behavior it is intended to measure.  There are 3 types of validity: –Construct –Criterion-related (predictive, concurrent) –Construct (convergent, discriminant)

80. For the GAIN-SS: Correlation between the Externalizing Disorders Screener (EDScr) and a subscale of 4 items from the Behavior Complexity Scale, a GAIN-I measure that both the EDScr and the 4 items come from. Correlation between the Crime & Violence Screener (CVScr) and a subscale of 4 items from the General Crime Scale, a GAIN-I measure that both the CVScr and the 4 items come from. For the BDI: correlation with GAIN-SS Internalizing Disorders Screener (IDScr) For the SWLS: correlation with GAIN-SS Internalizing Disorders Screener (IDS) For the RSES: correlation with GAIN-SS Internalizing Disorders Screener (IDS) For the AUDIT: correlation with GAIN-SS Substance Disorders Screener (SDScr) For the DAST: correlation with GAIN-SS Substance Disorders Screener (SDScr) Convergent Validity

81. Externalizing Disorders Screener (EDScr) and a subscale of 4 items from the Behavior Complexity Scale During the past 12 months, have you done the following things 2 or more times? 1) Been unable to pay attention when other things were going on. 2) Felt like you were always on the go, restless, or couldn’t sit still. 3) Been physically cruel to other people. 4) Taken money or things from a house, building, or car. Crime & Violence Screener (CVScr) and a subscale of 4 items from the General Crime Scale 1) During the past 12 months, did you do anything that would have gotten you into trouble with the police if they had known about it? 2) During the past 12 months, have you taken money or property that did not belong to you? 3) During the past 12 months, during an argument, did you try to hurt someone by throwing an object at them? 4) During the past 12 months, did you hit someone or get into a physical fight? Items Used in Convergent Validity Analysis – EDScr & CVScr

82. Items Used in Convergent Validity Analysis – IDScr & SPScr For the Internalizing Disorders Screener Items (IDScr) and the Substance Problems Screener Items (SPScr), their scores will be correlated with the scores from the RSES/SWLS/BDI or the AUDIT/DAST. Internalizing Disorders Screener Items(IDScr) When was the last time that you had significant problems with… –feeling very trapped, lonely, sad, blue, depressed, or hopeless about the future? Substance Problems Screener Items(SPScr) When was the last time that… –you had withdrawal problems from alcohol or other drugs like shaky hands, throwing up, having trouble sitting still or sleeping, or that you used any alcohol or other drugs to stop being sick or avoid withdrawal problems?

83. Reliability: Cronbach’s alpha will be conducted for items on each scale used. Psychometric Analyses: Reliability

84. GAIN- Reliability GAIN SS-ASL(23 item, ASL) Internal consistency (using all cases with item data) -Total Disorder Screener:  =.93 (N=583) -Internalizing Disorder Screener:  =.77 (N=591) -Externalizing Disorder Screener:  =.81 (N=588) -Substance Problems Screener:  =.88 (N=588) -Crime & Violence Screener:  =.84 (N=584) GAIN SS(23 item, English) Internal consistency – Adults (N=10,175) -Total Disorder Screener: a=.88 -Internalizing Disorder Screener: a=.78 -Externalizing Disorder Screener: a=.80 -Substance Problems Screener: a=.88 -Crime & Violence Screener: a=.66

85. Current Status & Next Steps 1. Forward translation and taping has been completed for the AUDIT, DAST, WRIOT directions, BDI, SWLS, RSES, & O*NET. 2. Back Translation has been completed for the AUDIT, DAST, WRIOT directions, BDI, SWLS, RSES & O*NET. 3. Reconciliation of back translation and re-taping(as needed) has been completed for the BDI, SWLS, RSES, AUDIT, DAST & O*NET. 4. Cognitive Interviewing is complete for the BDI, SWLS, RSES, AUDIT, DAST & O*NET. 5. Field testing has been completed for the GAIN, will soon begin for the O*NET and has been completed for the AUDIT, DAST, SWLS, and RSES. 6. During year 3&4 we will do training counselors related to the use of the online suite of assessments. 7. Psychometric Analyses will be run on each assessment i.e. Validity, Reliability and Equivalence.

86. CONTACT INFORMATION Deb Guthmann – dguthmann@aol.comdguthmann@aol.com

88. DEMONSTRATION http://130.108.114.24/limesurvey205/index.php/433252/lang-ase https://doda.omnijoin.com/join?dn=5MWM996 Username: dguthmann Password: Omnijoin123