Download presentation
Presentation is loading. Please wait.
Published byBrian Gilmore Modified over 8 years ago
1
1 Lessons Learned using Responsive Virtual Humans for Assessing Interaction Skills R. Hubal & D. Fishbein, RTI International M.J. Paschall, Pacific Inst. for Research and Evaluation
2
2 Overview of Talk Problem Experimental Setting Solution Methods Results Implications This study was performed under awards # 5-R01-DA14813-02 from the National Institute on Drug Abuse and # EIA-0121211 from the National Science Foundation.
3
3 Problem Many inner-city adolescents are at risk during social situations that require: Emotional control. Conflict resolution through negotiation, seeking information, expressing preferences. Adverse consequences include violence, school suspension, criminal activity. Training on these skills is well studied, but… Assessment is unsatisfactory: Want to assess adolescent’s learning before he’s “on the street”. Existing methods use surveys or text-based cases, not “assessment-by-doing”. Problem Experimental Setting Solution Methods Results Implications
4
4 Experimental Setting Part of a larger study: Examining neurocognitive-emotive predictors of behavioral problems among minority adolescents in high-risk urban settings. Identifying promising approaches to improve social-cognitive skills and reduce problem behaviors such as interpersonal violence and drug abuse. Participants part of an ongoing longitudinal study. Research team includes RTI, JHU, PIRE, others. Problem Experimental Setting Solution Methods Results Implications
5
5 Solution Role playing is known to be effective in providing adolescents with social competency skills. Use responsive virtual characters to simulate role-play scenarios for assessment: Interactive 3D. Natural language processing. Cognitive, social, emotional models. Problem Experimental Setting Solution Methods Results Implications
6
6 Solution, continued Responsive virtual human training applications: Law officers learning to manage encounters with the mentally ill. Medical practitioners practicing on virtual adult and pediatric standardized patients. Survey field interviewers learning to avert non- response and to gain informed consent to participate. Army ground tracked vehicle mechanics using a maintenance training system. These were not intended for assessment… Problem Experimental Setting Solution Methods Results Implications
7
7 Solution, continued Vignette storylines designed to target skills (engaging in negotiation, maintaining emotional control and being non-provocative, seeking information). Language models designed to interpret participant input as reflecting one or more social skill. Behavior models designed so that the virtual character attempts different strategies to require participants to demonstrate skills. Methods devised to measure participant engagement. Problem Experimental Setting Solution Methods Results Implications
8
8 Methods Participants: Black male inner-city 10 th graders. Approx. half diagnosed with Conduct Disorder. Materials: PACT video “Workin’ It Out”. Virtual character vignettes: – 3 scenarios (stolen goods, invitation, confrontation). – 2 variations (pre/post). Behavioral reports, teacher ratings, self-reports. Problem Experimental Setting Solution Methods Results Implications
9
9 Problem Experimental Setting Solution Methods Results Implications Methods, continued Positive Adolescents Choices Training (PACT) content: Social skills training program developed specifically for black youth. Uses videotapes containing vignettes of a conflict situation and demonstrating skills that could defuse a potentially violent situation. Target skills include: – Giving negative feedback or expressing criticism calmly (“Givin’ It”). – Reacting appropriately to the criticism or anger of others (“Takin’ It”). – Identifying problems and solutions (“Workin’ It Out”).
10
10 Methods, continued Procedure: Participant given pre-test set of vignettes. – Wizard-of-Oz methodology. – Monitor how participants handle vignettes designed to elicit risky behavior. Participant given standard text-based assessments. Participant shown PACT video in presence of facilitator. After up to 6 month delay, participant given post-test set of vignettes (again WoZ). Behavioral reports, teacher ratings, self-reports collected. Problem Experimental Setting Solution Methods Results Implications
11
11 Methods, continued Measures: Engagement with vignettes. Body language. Verbalizations. Number of conversational turns. Response time. Outcome: – Positive outcome is to decline or back away. – Negative outcome is to agree or escalate confrontation. Problem Experimental Setting Solution Methods Results Implications
12
12 Results Scoring procedures used during pre/post test with vignettes identify: Level of emotional control. Interpersonal communication skills. Analysis against established measures provides some support for construct and criterion validity. Nearly all queried participants stated their virtual decision mirrored what would be their real-life decision. Note: acceptability / usability not different among groups. Problem Experimental Setting Solution Methods Results Implications
13
13 Participants agree to help/go along with/escalate confrontation with the synthetic character one- third to half of time. Simulation effective in differentiating: Adolescents with clinical Conduct Disorder. Adolescents who had participated in live training sessions on key skills. Results, continued Problem Experimental Setting Solution Methods Results Implications
14
14 Implications Virtual characters can be useful for assessment as well as training in some contexts. Military applications: Greater understanding of signs and symptoms associated with PTSD. Managing encounters with the mentally ill. Interaction with child casualties and combatants. Crowd control or checkpoint management. Essentially, where need is to assess interaction skills in environment that is costly, dangerous, or difficult to replicate. Problem Experimental Setting Solution Methods Results Implications
15
15 Questions? This study was performed under awards # 5-R01-DA14813-02 from the National Institute on Drug Abuse and # EIA-0121211 from the National Science Foundation.
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.