iTechnology and Communicative Drawing for Individuals with Aphasia

iTechnology and Communicative Drawing for Individuals with Aphasia
Dr. O Jennifer A. Ostergren, PhD, CCC-SLP Jennifer Raminick, B.A.

Welcome Thank We hope you enjoy 3 hours, etc. Much ground to cover Drawing on an iPad by an individual with aphasia (Ostergren, 2012)

Thank You To our clients (our wonderful teachers)
Thank you for your inspiration and willingness to participate in our research To my students (future SLPs) Thank you for your help in preparing this presentation. In particular thank you Kristen Churney for her guest appearance in describing the CDP Kristen Churney is a graduate students at CSULB, study to become an SLP…….

House Keeping A handout for this presentation, including links to the technology discussed is available at: __________ (insert CD page web address once loaded)

Agenda Multi-Modality Communication and Aphasia Drawing and Aphasia
Correlations with Neuroplasticity Drawing and Aphasia iTechnologies for Communicative Drawing Examples of available iApps Features of available iApps Recommendations for Selection Communicative Drawing and iTechnology in a Clinical Setting New Directions and Future Research Questions and Answers Dr. O

Correlations with Neuroplasticity Drawing and Aphasia iTechnologies for Communicative Drawing Examples of available iApps Features of available iApps Recommendations for Selection Communicative Drawing and iTechnology in a Clinical Setting New Directions and Future Research Questions and Answers

What is Multimodality Communication?
Using different modes of communication to meet the needs of the individual and to allow them the opportunity to participate in their communities (ASHA, 2004) Writing Gesturing Drawing AND Speech Won’t spend time on aphasia As Neuro-typical communicators we use multimodal communication in our speech/communication constantly – by using gestures such as “talking with your hands”, and here in this presentation we are supplementing our discussion with written notes on the power-point. These are all examples of how we use multimodal communication everyday. Multimodality communication has been shown to be an effective means for aphasic individuals to access residual language capabilities and increase communication effectiveness (Purdy, Duffy, & Coelho, 1994). Often, therapy focuses on retreiving a lexicon and decreasing anomia through verbal output. however we are discussing therapy that makes use of more than one communication system to increase overall communication output and increase verbal output.

Total Communication Use in PWA (Rautakoski, 2011)
Many people with aphasia (PWA) use nonverbal methods of communication spontaneously, including: Pointing Gesturing Facial expressions Pantomime Drawing Concrete signs Some PWA also use low-tech devices Not used as much as the spontaneous nonverbal communication Some PWA need special training in use of total communication Based on survey of PWA and their families about PERCEIVED communication strategies Low tech devices (e.g., writing, using picture boards, picture books, and readymade wordlists) Authors recommend that……Using different nonverbal techniques to enable communication and participation in social interaction.

Varied Approaches Given:
Key Elements to Multimodality Communication (Beukelman & Mirenda, 2012) Co-Construction Partner Training Varied Approaches Given: Partner Dependent Communicators Partner Independent Communications Direct training and generalization to real-world settings is KEY!

AS: Example of Multimodality Communication
What did you do yesterday? Drew a hamburger Said, “Hamburger. Big” Put up four fingers Wrote : UCLA, DDS, 2 Viet, 1 Japan, and 2 white The client had gone out to eat hamburgers with four of his friends from dental school at UCLA; two friends were Vietnamese and two were Caucasian, and the “1 Japan” was referring to himself.

Neuroplasticity Rich body of research on neuroplasticity in the past decade (Saur & Hartwigsen, 2012) Neuroplasticity is defined as, “ ... functional reorganization/ compensation within residual neural tissue, mediated by changes in neural circuitry” (Gonzalez Rothi, Musson, Rosenbeck, & Sapienza, 2008) Be sure to connect back to multimodality and drawing Good news for our clients is that changes in neuroorganization have been noted in the chronic phase – and with TX!

Educating Clients about Neuroplasticity
Empowering Individuals with Aphasia with Neuroplastic Principles: From Theory to Practice. Birtler, E., Hayes, M., Kim, J., Sibby, K., Weltz, K., Navarro, A., & Ostergren, J. (2012). California Speech and Hearing Association (CSHA) Magazine, 42 (1), July 2012. Share with you briefly how we educate clients on neuroplasticity and how it applies to the use of drawing and total communication by PWA Based on recent article published last summer in the CSHA magazine.

Explaining Neuroplasticity to PWA
WHAT YOUR BRAIN IS DOING FOR YOU…. Researchers have identified four strategies your brain uses to compensate for damaged areas: 1. Homologous area adaptation 2. Cross-modal reassignment 3. Map expansion 4. Compensatory masquerade No matter how long since your stroke, your brain is still working to get the job done! Homologous area adaptation: Similar area on the opposite side takes over ….Just like the right-handed pitcher who pitches left handed when he breaks his right arm, the brain can compensate for damage on one side by training the area on the opposite side to do the same job. 2. Cross-modal reassignment: Using a new area of the brain, not previously involved, changing functional organization and inputs (e.g., blind person who registers touch from braille in visual cortex – while reading) ……Ever heard how little of your brain you actually use? Sometimes, after a brain injury, your brain assigns the duties of the damaged area to an area that was not being used. 3. Map expansion: Areas around the damaged area previously responsible for function expand ….Like asking your neighbor for a cup of sugar, the brain expands into neighboring areas to “borrow” undamaged neurons to complete tasks. 4. Compensatory masquerade: Brain re-routes to figuring out an alternative strategy for carrying out a task …..(e.g., like after a RCVA when a persons sense of direction is impaired and they navigate their environment instead using landmarks) Ever taken a surface street to avoid a traffic jam? Your brain does the same thing -- since damaged areas are too jammed up to use any more, your brain finds a different pathway to get the same destination! Grafman (2000). Conceptualizing Neuroplasticity

WHAT YOU CAN DO FOR YOUR BRAIN…. Repeat, Repeat, Repeat Why does your clinician make you keep repeating the same thing over and over? Raymer et. al (2008). Translational research in aphasia

Raymer et. al (2008). Translational research in aphasia

Pulvermüller & Berthier (2008). Aphasia therapy on a neuroscience basis

Outcome of Education Clients and family members are very receptive to information about neuroplasticity Visual aids and reducing jargon critical Qualitative changes were noted immediately: Increased motivation/participation More attempts to verbalize More willingness to try alternative modalities Seeking areas of change in their own environments EDUCATING CLIENTS ABOUT DRAWING, as part of multimodality communication, is critical!

Effects of Brain Damage on Drawing
Left hemisphere lesions May have difficulty depicting internal details of a target picture, but little difficulty depicting external shape Some are unable to generate a mental image of the target items Some may have difficulty choosing the correct color for images Right hemisphere lesions May retain ability to produce internal details but fail to correct spatial relations or relative size May omit all or the left half of external configuration May interfere with ability to discriminate between colors Jenn (Helm-Estabrooks & Albert, 2004)

Left or Right Hemisphere Lesion?
Left hemisphere ? (Helm-Estabrooks & Albert, 2004)

History of Drawing and Aphasia
Long history in the research literature (Lyons, 1995) Two primary areas of emphasis in the research: Realm of AAC and improving functional communication (Beeson & Ramage, 2000; Sacchett, 2002; Sacchett & Lindsay, 2007; Helm-Estabrook & Albert, 2004) For its impacts on verbal output as a source of de-blocking and semantic access (Davis, Farias, Baynes, 2005; Farias, Davis, Harrington, 2006) Describe results of each

Drawing and AAC Drawing can be used to compensate when a PWA is not able to use written or verbal language (Lyons, 1995) Advantage - no limitations caused by deficits in short-term memory and/or sequential ability (Sacchett, 2002) A move toward interactive (Sacchett, 2002) Supplementing Language Communication is enhanced by the drawings, so the emphasis is on the exchange of information and ideas Substituting Language Drawings are the communication, so emphasis is on producing recognizable content units. Supplementing – therapy focuses on the interactive message and using drawing and graphic symbols such as arrows to add to the message Substituting – therapy focuses on improving the content of the drawing. Burden of communication is on the PWA

De-blocking, Semantic Access, and Drawing
Drawing vs. writing as a cue for verbal expression Drawing more effective in increasing verbal naming (Davis, Farias, Baynes, 2005; Farias, Davis, Harrington, 2006) Quality of the drawings has no impact on the ability to access the semantic features of a word (Farias, Davis, Harrington, 2006) Possible Explanation? One plausible explanation for these findings is that act of drawing accesses the semantic features of a word, which are in turn are connected to the phonological representation needed for naming. Findings from fMRI studies have suggested that during drawing tasks, increase right hemispheric activation occurs, particularly in the right frontal lobe, and as such, homologous right-hemisphere activation of the left-hemisphere language areas may promote improved naming ability in individuals with aphasia during drawing tasks (Farias, Davis, Harrington, 2006; Harrington, Farias, Davis, Cortex, 2009). Transition to CDP – one program available to target drawing is the CDP – Kristen Farias (2006) found that drawing, as opposed to writing, was a more effective modality when used as a cueing strategy for increasing speech production.

Communicative Drawing Program (CDP)
“The functional goal of the CDP is for patients with aphasia who are unable to convey desired messages through speech or writing to communicate instead through drawing.” (Helm-Estabrooks & Albert, 2004, pg. 273)

CDP Candidate Profile (Helm-Estabrooks & Albert, 2004)
Inability to communicate information through speech or writing Ability to use a medium-point felt-tip pen to copy one dimensional shapes Relatively intact visual memory Good visual attention skills Alert, cooperative, and willing to pursue a drawing program to improve functional communication Pre-morbid artistic skills are not required, nor it is necessary to draw with the dominant hand * Comprehension of the client should be functional.

Overview of Steps Basic semantic-conceptual knowledge
Knowledge of object color properties Outlining pictures of objects with distinct shape properties Copying geometric shapes Completing drawings with missing internal and external features Drawing objects with characteristic shapes from memory Drawing objects to command from stored representations Drawing objects within superordinate categories Generative drawing- animals and modes of transportation Drawing cartooned scenes (Helm-Estabrooks & Albert, 2004)

Step 1 Basic Semantic-Conceptual Knowledge
“Circle the objects that belong together.” Criterion: 100% identification of the 5 items in 5 different superordinate categories

Step 2 Knowledge of Object Color Properties
“Select the correct color (from 12 colored markers) and color that item.” Criterion: Correct color choices for all nine items.

Step 3 Outlining Pictures of Objects with Distinct Shape Properties
“Take this pen and draw an outline around each object without touching the lines of the pictures” Criterion: Each object outline conforms to shape and does not touch or trace the lines of printed stimuli

Step 4 Copying Geometric Shapes
“Take this pen and copy each figure. Your drawings should not touch the examples.” Criterion: For each shape, all lines are present, in correct proportion and in proper relation to one another

Step 5 Completing Drawings with Missing External and Internal Features
“Show me what is missing in this picture…Okay, now take this pen and fill in the missing part.” Criterion: All completions are successfully rendered, complete, and conceptually reasonable

Step 6 Drawing Objects with Characteristic Shapes from Memory
“Look at this object. (Present single card.) Remember what it looks like, because I’m going to ask you to draw it. Okay, are you ready? Take this pen and draw the object here.” Criterion: For each shape, all lines are present, in correct proportion and in proper relation to one another

Step 7 Drawing Objects to Command from Stored Representations
“Look at this word. It says____ Can you draw____right here?” ice cream Criterion: Drawings of each object can be correctly identified by a naïve judge who is not shown the target words

Step 8 Drawing Objects Within Superordinate Categories
“See this word? It says ___. I want you to think of a _____ and draw it right here.” tool Criterion: All responses belong to the named superordinate category and can be identified correctly by a naïve judge not provided with the target words.

Step 9 Generative Drawing: Animals and Modes of Transporting People
“On this page, I want you to draw as many types of animals/modes of transportation as you can. Make your drawings clear enough so that anyone could name the animals/modes of transportation that you draw.” Criterion: All responses are correct exemplars of the superordinate categories and are correctly identified by a naïve judge.

Step 10 Drawing Cartooned Scenes
“Show me what is funny about this picture…Now study the picture well because I’m going to ask you to draw it from memory.” Criterion: Each panel should be accurate enough for a naïve judge to identify all elements necessary for understanding and stating the message (joke) of each cartoon.

CDP: Case Study (A.S) 45-yr-old male
Status post left basal ganglia intracranial hemorrhagic CVA 10/03/2008 Dentist Bilingual in English (primary) and Japanese Mixed , non-fluent aphasia - Verbal expression characterized by: - 1-word utterances - verbal apraxia - decreased loudness and pitch range - confrontational naming relatively intact - Written expression characterized by 1-word phrases - Severe auditory and reading comprehension deficits (1-2 word phrases) - Used writing and gestures to supplement verbal expression

Sample CDP Goals (A.S.) Goal: To improve multimodality communication, the client will draw three-panel drawings from memory with enough detail so that a naive judge can identify the message. Objective 1: In the clinical setting, when given the name of an object with distinct visual features (i.e., ice cream cone, palm tree, etc), the client will draw the object with enough detail so that a naive judge can identify the object with 100% accuracy in a minimum of 10 trials. Objective 2: In the clinical setting, when given a superordinate category (e.g., tool, animals, vehicle, etc.) and no prompts, the client will draw at least one object within the specified category with enough detail so that a naive judge can identify the object with 100% accuracy in a minimum of 10 trials. Objective 3: In the clinical setting, when given a three-panel cartooned story, the client will draw the story from memory with enough detail so that a naive judge can identify the message of the cartoon in 2 out of 3 trials.

CDP Training/Tasks (A.S.)
Began with Step 3 of the CDP During Step 7 Modeled adding more detail to unrecognizable drawings Client began to spontaneously add more detail Clinician provided the client with a puzzled look whenever his drawings were not recognizable Homework given for Steps 8-9 CDP supplemented by multimodality communication Barrier tasks encouraged all modalities Step 3 = (outlining objects with distinct outer configurations) Step 7 = drawing objects to command from stored representations Once the client began to spontaneously add more detail to his drawings, the clinician provided the client with a puzzled look whenever his drawings were not recognizable in order to encourage the addition of more detail Homework tasks were given for Steps 8-9 CDP tasks were supplemented by multimodality communication training requiring the client to use all modalities (verbal expression, written expression, gestures, and drawing) to communicate pictures in a barrier task

CDP Outcome Measure The client will draw one-, two-, and three-panel cartooned stories from memory. Panel should be accurate enough for a naïve judge to identify all elements necessary for understanding and stating the message (joke) of each cartoon. (Helm-Estabrooks & Albert, 2004)

1-Panel Model Copyright Helm-Estabrooks, N. Communicative Drawing Program Cartoons

Pre-Therapy Post-Therapy

Maintenance/Generalization Data
Copyright Helm-Estabrooks, N. Communicative Drawing Program Cartoons

Maintenance/Generalization Data
1 2 3 Caregiver interview and the Communicative Effectiveness Index (CETI) indicate that A.S. primarily communicates via verbal and written expression and drawing

Potential Supplemental Practice - Game Apps
Draw Something Social drawing and guessing game Whiteboard Collaborative drawing app These game apps are popular social games where you can connect with friends to create interactive drawings. While these apps are not therapeutic in nature, they emphasize the therapy activities and create opportunities to generalize

New Frontier The role of new technologies in multimodal communication, including drawing. More questions than answers Is new technology applicable for drawing purposes? What technology is best? Any hidden pitfalls? Any potential advantages? Where to begin??????

Technology for PWA Most often technology is used in speech therapy for: stimulus material, data tracking, and AAC (ASHA, 2011) Technology-aided therapy effective, due to the increased intensity of treatment (van de Sandt-Koenderman, 2011) Likely due to potential for use with multiple communication partners and use across multiple settings Apps and tech used in ST for children – especially Autism & Child Language Therapy apps for target drills and practice, social apps to use outside of therapy for reinforcement of skills Van de Sandt-Koenderman found that computer-aided therapy for people with aphasia was more effective due to the increased intensity of treatment as technology improved performance of functional goals and increased social participation. Tactus Therapy Solutions – Comprehension therAppy which works on auditory and reading comprehension, Naming Therappy which provides target drills for naming AAC apps Dragon Diction – text to speech Notes + - allows for drawing, writing and typing

What is a Smartphone? “Smartphone is a mobile phone that includes software that a user is able to modify and update. The user controlled software must be able to transfer information to and from external systems” (Toyysy & Helenius, 2006, p. 110).

Smartphone Options Blackberry Android $250-$500 Windows Phone
$300-$400 Windows Phone $400-$500 Here are some examples of smartphones. These are intuitive devices that offer direct touch interface These can access thousands of apps. They also come with built in apps which are useful such as calendars, alarms, timer, notepads, voice recorder, and navigation.

iPhone $400-$600 HP Palm $400-$600 Nokia $400-$600
A few more smartphones – the iPhone has access to 350,000 +apps Some smartphones require set-up which can be difficult – the store can set-up and provide support if needed

Alternate Formats - Tablets
iPad Android Tablet $500 - $750 $480-$750 There are now other ways to access iTechnology – the tablet Here is the iPad from Apple and the Android tablet Another advantage of the tablet is the size – easier to read, larger work space Also, some of the newer e-readers (i.e., Kindle) offer access to apps

Alternate Formats – iPod Touch
The iTouch is the same size as the smartphone and has the same capabilities to access the apps. There is just no phone access. $299-$399

What is an App? “Apps” is an abbreviation for application
Piece of software that can run on the Internet, on a computer, on a phone or other electronic devices Variety of available on different smartphones Some require internet connectivity, while others do run off-line These are able to be directly downloaded to your device. The average price of an app is $2 – but many social/game apps are free Most apps are paid with a 1 time fee – some run on a monthly service.

What is iTechnology? Technology manufactured by
Apple, collectively referred to as: iTechnology or iDevices For example: iPads iPod Touch iPhones Selection of iTechnology for this presentation and current research isn’t value judgment, just starting point. There are other manufacturers, such as Android

Potential Benefits of Drawing Apps
Unlimited numbers of pages accessible without erasing Save and organize drawings created for future access/modification Enhance drawings with colors, stored symbols, and backgrounds. Incorporate real photographs Possibility? GPS Picture Prediction (e.g., Locabulary for picture) Export and send drawings via Some apps include keyboard function to augment drawing with typed letters, words, or phrases. Possibility? Text-to-speech Possibility? Interactive Alphabet Board/Phoneme cue Dr. O. iTechnology allows drawing to be even more interactive and allow for more communication opportunities – not just face-to-face. Now we can import pictures from the internet and incorporate them in our drawings, save and the drawings or save and access for future conversations. GPS Picture Prediction Guessing photos to import, based on location. Locabulary is an iphone app that does this with text (word or phrases). Based on GPS – guesses vocabulary (e.g., in coffee house, suggests words associated with coffee). Picture prediction = suggest images from Google images associated with location Text-to-speech Click on word written or typed to say it? Interactive Alphabet Board/Phoneme cue Click on letter written or typed to say sound

iTechnology: A Pilot Study
Ostergren, J., & Raminick, J. (n.d.) Drawing and iTechnology: A pilot study in the use of a drawing iApp by individuals with aphasia (n.d.). Unpublished manuscript. Methods 4 PWA Ages 20 – 80 years old Non-fluent forms of aphasia Participants were evaluated on their ability to: Access and navigate within a drawing app (NotesPlus) Intelligibility of their drawings (as compared to traditional paper and pencil methods) Jenn Participants were four individuals with non-fluent aphasia who were given initial training in the use of an iApp for communicative drawing purposes, followed by practice opportunities within bi-weekly therapy sessions. Participants were evaluated on their ability to access and navigate within this iApp, as well as the intelligibility of the drawings rendered using this app, as compared to traditional paper and pencil methods.

CDP Cartoon Copyright Helm-Estabrooks, N. Communicative Drawing Program Cartoons

Paper/Pencil vs. iPad Intelligible to a naive judge using pen and paper Intelligible to a naive judge using the iPad PT1HPR One-Panel Yes Two-Panel Three-Panel No PT2 PT3 PT4HPL Jenn We had participants draw the 1,2, and 3 panel cartoons from the CDP – shown previously using paper and pen and also the iPad Most participants drawings were comparable with both the ipad and paper/pen PT __: Premorbid _____hand dominant Hemiparesis in _____hand Used ______hand to draw PT 4: Premorbid _____hand dominant Add MC handedness

Example (PT 1): Paper/Pencil vs. iPad
Here is an example of one participant’s drawings. This is the one-panel drawing with the paper/pen drawing on the left and the iPad drawing on the right. They are very comparable in detail and both intelligible from the model

App Use: Ease of Use After Training
In the pilot study, the participants used Notesplus which had the capability for individuals to draw, write and type all on the same screen. The app was used in therapy as a platform for multimodal communication. One of the research questions we had was, will the participants be able to access and navigate the app and the iTechnology. All but one participant (PT4) was rated as either completely independent or needing minimal assistance in effectively opening the app, opening a new page within the app, and retrieving a previously drawn page. PT4’s frustration with touch screen use (as described above) appeared to carry over into her overall desire to use this technology, such that she often selected to utilize paper and pencil format for communicative drawing purposes. This may have resulted in less practice time and less motivation in learning to navigate within this iApp. The use of iTechnology was found to be successful for most participants as an efficient and effective platform for multimodality communication with minimal training and instruction required for the app/iPad. Two participants in this study did have upper extremity hemiparesis of the dominant hand. PT4 utilized her non-dominant hand for drawing as did PT2. In contrast to PT4 though, PT2 was not rated as having significant difficulty in touch screen use. This warrants further investigation. All participants were given the option of utilizing either the iPad or paper and pencil during communicative drawing tasks. Interestingly, at times, even those rated as independent in iPad use were observed to select paper and pencil over technology use. This was an unanticipated observation and was not formally tracked throughout this study. Qualitatively, it was noted, however, that these instances appeared most frequently when short, quick responses were needed or after long periods without communication when the iPad “slept” and required a touch screen swipe before accessing the drawing app. Communication by nature takes place rapidly. This additional step in accessing the app, may have added time to the communication task, and for some individuals this may have been a barrier to its use for communicative purposes. Additional research is needed in this area, including the potential for eliminating this automatic “sleep” function so that the drawing app is consistently accessible without an additional step in re-activating it for use in communication.

PT 4: Words of Caution Still obstacles to over-come
Non-dominant hand use/Limited control Registering drawing contact (false starts) No different in stylus vs. finger-tip Difficulty and frustration with touch screen use (with and without stylus) No previous touch screen use Lower levels of use and confidence with computers and technology in general Participant 4 had hemiparesis, so she was using her non-dominant hand in drawing. Because it was difficult to use that hand and she had never used a touch screen, she had increased levels of frustration. We surveyed all of the participants prior to using the iPad on their comfort and frequency of use of computers/technology and this participant had lower levels of comfort with technology than the rest of the group.

Finding the Right App: Recommended Feature Analysis -
ASHA defines feature matching as: “…devices are selected based on relationships between an individual's strengths… capabilities and communication needs in relation to various features of a device (ASHA, 2004, page 9). Dr. O

Best predictor of long-term success with AT is careful selection of aids to ensure they are well-matched to the user and the environment (Scherer et al., 2007)

Which Apps to Use: App Analysis
Ostergren, J. & Gastelum, M. (n.d.). An Exploration iTechnology Drawing Apps for Individuals with Aphasia. Unpublished manuscript. Evaluation of Drawing and Writing apps for communicative drawing purposes, given: Phase 1 Phase 2 Phase 3 As researchers have pointed out and anyone who uses this technology knows ….numerous good, great, and not so good apps available So after the initial pilot study, we endeavored to identify which might work _ NO EASY TASK!

Phase 1 – App Identification (Part A)
METHODS Search Terms: Apps with “drawing” or “handwriting” in search terms Consumer Ratings: Apps highly rated by users on the Apple Market place as a 4 or higher stars Device Mode: Apps that could be utilized on an iPad. Category: “Productivity” apps RESULTS 22 drawing apps 112 writing apps identified TOTAL = 134 apps Start Dr. O. Search Terms These search terms were utilized to yield apps which potentially contained drawing functions. All apps with instructions and marketing information in a language other than English were also removed from this list. A rating of 3+ star (on a scale of 0 to 4+ stars) was utilized to identify apps that a wide range of individuals found helpful, suggesting that the app was user friendly, contained more features, and/or had initial bugs addressed. Consumer Ratings Given the nature of drawing, it was felt that the larger workspace provided on an iPad would facilitate the greatest potential for effective drawing, both in future use by individuals with aphasia and given pilot studies to assess drawing use of each app. Device Mode Removed apps in non-applicable categories such as “reference,” “books,” “games,” “music,” and “sports” apps. “Education**,” “entertainment” or “photo & video” apps also Category: Eliminate duplicates Removed drawing apps that were for professional use (e.g., computer assisted drawing (CAD) apps, architectural and blue print apps, etc.) NOTE: Several of the apps in “education” category could be revisited to incorporate within CDP or other drawing programs, including drawing games, such as Draw Something

Phase 1 – App Identification (Part B)
METHODS Line Sizes. Apps with at least 8 line size options Color Choices. Apps with at least 8 color options Background Choices. Apps with the ability to modify the background in some fashion beyond strictly a white canvas were selected. Saving Capacity. Apps with the ability to save drawings for future retrieval and easily modification. Ability to . Apps with an function. RESULTS 5 drawing apps 9 writing apps identified TOTAL = 14 apps were selected. It was felt that this would give individuals with varying visual and motor dexterity more choices in apps use and functionality. Line Size Still needed to narrow it down…. This criterion was utilized given that color enhancement for communicative drawing purpose can be a valuable feature, for example, in distinguishing drawing of similar shape/features, but unique color representation (e.g., apple vs. orange). In addition, research suggested that the act of drawing in individuals with aphasia may strengthen the activation of the associated lexical-phonological processes (Farias, Davis, and Harrington, 2005). Hence more color options in this respect would be a valuable feature for individuals with aphasia who wanted to incorporate color in this fashion. Color Choices Background Choices. Certain apps included a variety of backgrounds to choose from, including: 1) the option to change the type of “paper”, such as white paper, colored paper, or line paper options, 2) the option to use a set of pre-stored backgrounds, such as nature scenes, textured or abstract backgrounds, and 3) the option to import photos as a background options. These varied options permit the drawing canvas to be individually adapted to the patient’s needs. Background Ability to – form of communication some individuals with aphasia can’t do Saving Capacity – add to drawings and co-construction

Phase 3 – App Evaluation METHODS
Apps from Phase 2 rated using 5 point scale, as follows Ease of Navigation: 1 = Complex navigation………………5 = Minimal/Easy Navigation Visual Distraction 1 = Maximal Distraction… = Minimal Distraction Workspace: 1 = Small Icons/Workspace…………..5 = Large Icons/Workspace Iconicity: 1 = Opaque Icons……………………………………5 = Transparent Icons Ease of Navigation. number of steps needed to draw within the app (e.g., layers to draw, change color, erase, etc.) and the number layers to save and a drawing Visual Distraction. Amount of visual distractors, such as advertisements common of “free” or “lite” versions Visual Demand. size and placement of visual icons and the size and orientation of the drawing field (e.g., drawing space that was large and adjustable in orientation) for use by individuals with potentially limited vision or other impairments in, such as visual field cuts Iconicity. This criteria consisted of the researchers ranking the app given the transparency in visual icons for us by someone potentially unfamiliar with common technology symbols, such as buttons for deleting, ing, changing settings, etc.

Drawing Apps

Writing Apps Now we would like to demonstrate the top rated apps in each category – plug in iPad – Jenn demo

Looking Deeper: Additional Trends Uncovered
Options for setting hand use (right vs. left) Moveable palm rests Icon knowledge required Layers in saving – Less is more Remote access/additional accounts (e.g., EverNote) Start position important Access knowledge required Tapping, swiping, and finger expansion Now we would like to demonstrate the top rated apps in each category – plug in iPad – Jenn demo

SketchTime $1.99 Language: English Developer: Hansol Huh
Rated 4+ Together

Jotter $0.99 Languages: English, Korean Seller: groosoft
Rated 4+

Recent Replication: Pilot Study
13 additional PWA Varying forms of aphasia (fluent and non-fluent) Varying ages All in chronic stage of recovery from a CVA All receiving services at CSULB for communication impairments Asked to draw a 1-panel CDP cartoon using paper and pen and one (1) of the following apps: Sketch Time Jotter Absolute Board Noteshelf Dr. O.

Intelligible to a naive judge
Sketch Time Intelligible to a naive judge using pen/paper Intelligible to a naive judge using the iPad PT 1 Yes No PT 2HPL PT 3

Jotter Intelligible to a naive judge using pen/paper Intelligible to a naive judge using the iPad PT 1 No PT 2HPL Yes No* PT 3HPL *NOTE: Patient experienced notable difficulty app registering finger placement (both with stylus and finger tip). – frustrated and appeared to abandon task early

Absolute Board Intelligible to a naive judge using pen/paper Intelligible to a naive judge using the iPad PT 1 Yes No PT 2HPR PT 3HPL PT 4

Noteshelf Intelligible to a naive judge using pen/paper Intelligible to a naive judge using the iPad PT 1HPL No PT 2HPR PT 3

RESULTS 5/17 PWA had intelligible drawings in the paper and pencil mode (29%) Two (2) had equally intelligible app drawings (Notesplus) Three (3) had less intelligible app drawings (Jotter, Absolute Board, and Sketch-Time)* Noteshelf N/A because no intelligible

Ad Hoc and Qualitative Motor control and app use:
9/17 individuals had hemiparesis (52%) 3 of these individuals had intelligible drawings in one mode (33%) 1 = equally intelligible drawings in both modes 1 = less intelligible app drawing 1 = less intelligible paper/pencil Most individuals in our study with hemiparesis had unintelligible drawings in both modes (6/9, 67%) Consider OT and CDP?

Conclusion Interpret with CAUTION – small sample!
For some individuals, drawing using an app was less intelligible, with some apps But………

The Devil’s in the Details – Aim for Total Communication

Sketch Time

Absolute Board – PA?

Pointed to spot highlighted and said: “Water” and “Hot”
Pointed to spot highlighted and said: “Water” and “Hot”. Also, gestured drinking. What is this? Is that a tree? What is this? Jotter

Big Picture: Lessons Learned
Aim for communication, not perfection in drawing. Incorporate within multimodality system (e.g., speech, writing, gestures, AND drawing) Provide education in neuroplasticity and the positive impact of drawing on verbal expression Consider pre-morbid technology use and comfort Increase acceptance by minimizing client frustration Frustration – at least two of the clients abandon due to difficulty in access

Additional Lessons Learned: Experiment/Match to Client
Compare Drawings on Different Apps Explore Palm Rest/Hand Use Settings Evaluate Stylus vs. Finger-tip Individual preference a factor Assess with different “pen” settings on the app (fine vs. large) Assess Case Withstand pressure from resting palm while drawing Adjust to different positions

Additional Lessons Learned: Therapy Tips
Incorporate within Multimodality/Total Communication Training Speaking Gestures Drawing Writing Use Barrier Tasks Train Partners Use CDP to Enhance Details (Internal/External Features) Assess with Referential Communication Tasks

Referential Communication Task
Based on Purdy and VanDyke (2009) CSULB Modifications: 15 real color photographs of common everyday activities Unfamiliar listener: Instructions: A PWA is going to be describing to you pictures shown to them. You will not be able to see these pictures. The PWA may speak, write, draw, gesture (or use a variety of methods) in describing these pictures to you. Please write down what you think the picture represents.

Future Research: Information Needed
Develop and test frameworks for effectively and efficiently evaluating apps Explore communicative drawing and technology Influence of technology in communication dynamics Collect actual efficacy data Just because it is new, doesn’t make it better. Move toward dedicate apps for PWA that allow for fluidity in modalities PWA input critical

Q & A For additional information about this presentation contact:
Dr. Jennifer A. Ostergren Assistant Professor California State University, Long Beach Department of Communicative Disorders

Reference ASHA. (2004). Roles and responsibilities of speech-language pathologists with respect to augmentative and alternative communication: technical report [Technical Report]. Available from Beeson, P.M., & Ramage, A.E. (2000). Drawing from experience: The development of alternative communication strategies. Topics in Stroke Rehabilitation, 7(2), Beukelman, D., & Mirenda, P. (2012). Augmentative and Alternative Communication: Supporting children and adults with complex communication needs (4th Ed.) Baltimore, MD: Brookes Publishing Co. Davis, C.H., Farias, D., Baynes, K. (2005). Understanding the effects of cuing strategies through error analysis. Brain and Language, 95, Farias, D., Davis, C., & Harrington, G. (2006). Drawing: Its contribution to naming in aphasia. Brain and Language , 97, Gonzalez Rothi, L. J., Mussi, N., Rosenbeck, J. C., & Sapieza, C. M. (2008). Neuroplasticity and rehabilitation research for speech, language, and swallowing disorders. Journal of Speech, Language, and Hearing Research, 51(1), S222-S224. doi: / (2008/017).

Reference Grafman, J. (2000). Conceptualizing functional neuroplasticity. Journal of Communication Disorders, July-August, (Posted on Beachboard) Helm-Estabrooks, N. & Albert, M. (2004). Manual of Aphasia and Aphasia Therapy (2nd Ed.). Austin, TX: Pro-Ed. Koul, R. K, & Corwin, M. (2011). Augmentative and alternative communication intervention for persons with chronic severe aphasia: Bringing research to practice. EBP Briefs 6(2), 1–8. Bloomington, MN: Pearson Lyons, J. (1995). Drawing: Its value as a communication aid for adults with aphasia. Aphasiology, 9, Purdy, M., & VanDyke, J. (2009). Intermodal training to facilitate communication in aphasia: A pilot study. In Clinical Aphasiology Conference: Clinical Aphasiology Conference (2009 : 39th : Keystone, CO : May 26-30, Retrieved from:

Reference Pulvermüller, F., & Berthier, M. L. (2008). Aphasia therapy on a neuroscience basis. Aphasiology, 22(6), doi: / Raymer, A. M., et. al (2008). Translational research in aphasia: From neuroscience to neurorehabilitation. Journal of Speech, Language, and Hearing Research, 51, S259-S275. Rautakoski, P. (2011). Training total communication. Aphasiology, 25, Sacchett, C., & Black, M. (2011). Drawing as a window to event conceptualization: Evidence from two people with aphasia. Aphasiology, 25(1), 3 – 26. Sacchett, C. (2002). Drawing in aphasia: moving towards the interactive. International Journals of Human-Computer Studies, 54(4), Sacchett, C., & Lindsay, J. (2007). Revealing competence and rethinking identify in severe aphasia using drawing and a communication book. In S. Byng, J. Duchan, & C. Pound (Eds). The Aphasia Therapy File, Volume 2. Hove, UK: Pyschology Press

Reference Saur, D., & Hartwigsen, G. (2012). Neurobiology of language recovery after stroke: Lessons from neuroimaging studies. Archives of Physical Medicine and Rehabilitation, 93(1), Supplement, S15-S25 Toyssy, S. and Helenius, M. (2006). About malicious software in smartphones. Journal in Computer Virology, 2, van de Sandt-Koenderman WM. (2011). Aphasia rehabilitation and the role of computer technology: can we keep up with modern times?. International Journal of Speechlanguage Pathology 13(1):21-7.

iTechnology and Communicative Drawing for Individuals with Aphasia

Similar presentations

Presentation on theme: "iTechnology and Communicative Drawing for Individuals with Aphasia"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

iTechnology and Communicative Drawing for Individuals with Aphasia

Similar presentations

Presentation on theme: "iTechnology and Communicative Drawing for Individuals with Aphasia"— Presentation transcript:

Similar presentations

About project

Feedback