Presentation on theme: "iTechnology and Communicative Drawing for Individuals with Aphasia"— Presentation transcript:
1iTechnology and Communicative Drawing for Individuals with Aphasia Dr. OJennifer A. Ostergren, PhD, CCC-SLP Jennifer Raminick, B.A.
2WelcomeThankWe hope you enjoy3 hours, etc.Much ground to coverDrawing on an iPad by an individual with aphasia (Ostergren, 2012)
3Thank You To our clients (our wonderful teachers) Thank you for your inspiration and willingness to participate in our researchTo my students (future SLPs)Thank you for your help in preparing this presentation. In particular thank you Kristen Churney for her guest appearance in describing the CDPKristen Churney is a graduate students at CSULB, study to become an SLP…….
4House KeepingA handout for this presentation, including links to the technology discussed is available at:__________ (insert CD page web address once loaded)
5Agenda Multi-Modality Communication and Aphasia Drawing and Aphasia Correlations with NeuroplasticityDrawing and AphasiaiTechnologies for Communicative DrawingExamples of available iAppsFeatures of available iAppsRecommendations for SelectionCommunicative Drawing and iTechnology in a Clinical SettingNew Directions and Future ResearchQuestions and AnswersDr. O
6Agenda Multi-Modality Communication and Aphasia Drawing and Aphasia Correlations with NeuroplasticityDrawing and AphasiaiTechnologies for Communicative DrawingExamples of available iAppsFeatures of available iAppsRecommendations for SelectionCommunicative Drawing and iTechnology in a Clinical SettingNew Directions and Future ResearchQuestions and Answers
7What is Multimodality Communication? Using different modes of communication to meet the needs of the individual and to allow them the opportunity to participate in their communities (ASHA, 2004)WritingGesturingDrawingAND SpeechWon’t spend time on aphasiaAs Neuro-typical communicators we use multimodal communication in our speech/communication constantly – by using gestures such as “talking with your hands”, and here in this presentation we are supplementing our discussion with written notes on the power-point. These are all examples of how we use multimodal communication everyday.Multimodality communication has been shown to be an effective means for aphasic individuals to access residual language capabilities and increase communication effectiveness (Purdy, Duffy, & Coelho, 1994). Often, therapy focuses on retreiving a lexicon and decreasing anomia through verbal output. however we are discussing therapy that makes use of more than one communication system to increase overall communication output and increase verbal output.
8Total Communication Use in PWA (Rautakoski, 2011) Many people with aphasia (PWA) use nonverbal methods of communication spontaneously, including:PointingGesturingFacial expressionsPantomimeDrawingConcrete signsSome PWA also use low-tech devicesNot used as much as the spontaneous nonverbal communicationSome PWA need special training in use of total communicationBased on survey of PWA and their families about PERCEIVED communication strategiesLow tech devices (e.g., writing, using picture boards, picture books, and readymade wordlists)Authors recommend that……Using different nonverbal techniques to enable communication and participation in social interaction.
9Varied Approaches Given: Key Elements to Multimodality Communication (Beukelman & Mirenda, 2012)Co-ConstructionPartner TrainingVaried Approaches Given:Partner Dependent CommunicatorsPartner Independent CommunicationsDirect training and generalization to real-world settings is KEY!
10AS: Example of Multimodality Communication What did you do yesterday?Drew a hamburgerSaid, “Hamburger. Big”Put up four fingersWrote : UCLA, DDS, 2 Viet, 1 Japan, and 2 whiteThe client had gone out to eat hamburgers with four ofhis friends from dental school at UCLA; two friends wereVietnamese and two were Caucasian, and the “1 Japan” wasreferring to himself.
11NeuroplasticityRich body of research on neuroplasticity in the past decade (Saur & Hartwigsen, 2012)Neuroplasticity is defined as,“ ... functional reorganization/ compensation within residual neural tissue, mediated by changes in neural circuitry” (Gonzalez Rothi, Musson, Rosenbeck, & Sapienza, 2008)Be sure to connect back to multimodality and drawingGood news for our clients is that changes in neuroorganization have been noted in the chronic phase – and with TX!
12Educating Clients about Neuroplasticity Empowering Individuals with Aphasia with Neuroplastic Principles: From Theory to Practice.Birtler, E., Hayes, M., Kim, J., Sibby, K., Weltz, K., Navarro, A., & Ostergren, J. (2012). California Speech and Hearing Association (CSHA) Magazine, 42 (1), July 2012.Share with you briefly how we educate clients on neuroplasticity and how it applies to the use of drawing and total communication by PWABased on recent article published last summer in the CSHA magazine.
13Explaining Neuroplasticity to PWA WHAT YOUR BRAIN IS DOING FOR YOU….Researchers have identified four strategies your brain uses tocompensate for damaged areas:1. Homologous area adaptation2. Cross-modal reassignment3. Map expansion4. Compensatory masqueradeNo matter how long since your stroke, your brain is stillworking to get the job done!Homologous area adaptation:Similar area on the opposite side takes over….Just like the right-handed pitcher who pitches left handed when he breaks his right arm, the brain can compensate for damage on one side by training the area on the opposite side to do the same job.2. Cross-modal reassignment:Using a new area of the brain, not previously involved, changing functional organization and inputs (e.g., blind person who registers touch from braille in visual cortex – while reading)……Ever heard how little of your brain you actually use? Sometimes, after a brain injury, your brain assigns the duties of the damaged area to an area that was not being used.3. Map expansion:Areas around the damaged area previously responsible for function expand….Like asking your neighbor for a cup of sugar, the brain expands into neighboring areas to “borrow” undamaged neurons to complete tasks.4. Compensatory masquerade:Brain re-routes to figuring out an alternative strategy for carrying out a task …..(e.g., like after a RCVA when a persons sense of direction is impaired and they navigate their environment instead using landmarks)Ever taken a surface street to avoid a traffic jam? Your brain does the same thing -- since damaged areas are too jammed up to use any more, your brain finds a different pathway to get the same destination!Grafman (2000). Conceptualizing Neuroplasticity
14Explaining Neuroplasticity to PWA WHAT YOU CAN DO FOR YOUR BRAIN….Repeat, Repeat, RepeatWhy does your clinician make you keep repeating the same thing over and over?Raymer et. al (2008). Translational research in aphasia
15Explaining Neuroplasticity to PWA Raymer et. al (2008). Translational research in aphasia
16Explaining Neuroplasticity to PWA Pulvermüller & Berthier (2008). Aphasia therapy on a neuroscience basis
17Outcome of EducationClients and family members are very receptive to information about neuroplasticityVisual aids and reducing jargon criticalQualitative changes were noted immediately:Increased motivation/participationMore attempts to verbalizeMore willingness to try alternative modalitiesSeeking areas of change in their own environmentsEDUCATING CLIENTS ABOUT DRAWING, as part ofmultimodality communication, is critical!
18Agenda Multi-Modality Communication and Aphasia Drawing and Aphasia Correlations with NeuroplasticityDrawing and AphasiaiTechnologies for Communicative DrawingExamples of available iAppsFeatures of available iAppsRecommendations for SelectionCommunicative Drawing and iTechnology in a Clinical SettingNew Directions and Future ResearchQuestions and Answers
19Effects of Brain Damage on Drawing Left hemisphere lesionsMay have difficulty depicting internal details of a target picture, but little difficulty depicting external shapeSome are unable to generate a mental image of the target itemsSome may have difficulty choosing the correct color for imagesRight hemisphere lesionsMay retain ability to produce internal details but fail to correct spatial relations or relative sizeMay omit all or the left half of external configurationMay interfere with ability to discriminate between colorsJenn(Helm-Estabrooks & Albert, 2004)
20Left or Right Hemisphere Lesion? Left hemisphere?(Helm-Estabrooks & Albert, 2004)
21History of Drawing and Aphasia Long history in the research literature (Lyons, 1995)Two primary areas of emphasis in the research:Realm of AAC and improving functional communication (Beeson & Ramage, 2000; Sacchett, 2002; Sacchett & Lindsay, 2007; Helm-Estabrook & Albert, 2004)For its impacts on verbal output as a source of de-blocking and semantic access (Davis, Farias, Baynes, 2005; Farias, Davis, Harrington, 2006)Describe results of each
22Drawing and AACDrawing can be used to compensate when a PWA is not able to use written or verbal language (Lyons, 1995)Advantage - no limitations caused by deficits in short-term memory and/or sequential ability (Sacchett, 2002)A move toward interactive (Sacchett, 2002)Supplementing LanguageCommunication is enhanced by the drawings, so the emphasis is on the exchange of information and ideasSubstituting LanguageDrawings are the communication, so emphasis is on producing recognizable content units.Supplementing – therapy focuses on the interactive message and using drawing and graphic symbols such as arrows to add to the messageSubstituting – therapy focuses on improving the content of the drawing. Burden of communication is on the PWA
23De-blocking, Semantic Access, and Drawing Drawing vs. writing as a cue for verbal expressionDrawing more effective in increasing verbal naming (Davis, Farias, Baynes, 2005; Farias, Davis, Harrington, 2006)Quality of the drawings has no impact on the ability to access the semantic features of a word (Farias, Davis, Harrington, 2006)PossibleExplanation?One plausible explanation for these findings is that act of drawing accesses the semantic features of a word, which are in turn are connected to the phonological representation needed for naming. Findings from fMRI studies have suggested that during drawing tasks, increase right hemispheric activation occurs, particularly in the right frontal lobe, and as such, homologous right-hemisphere activation of the left-hemisphere language areas may promote improved naming ability in individuals with aphasia during drawing tasks (Farias, Davis, Harrington, 2006; Harrington, Farias, Davis, Cortex, 2009).Transition to CDP – one program available to target drawing is the CDP – KristenFarias (2006) found that drawing, as opposed to writing, was a more effective modality when used as a cueing strategy for increasing speech production.
24Communicative Drawing Program (CDP) “The functional goal of the CDP is for patients with aphasia who are unable to convey desired messages through speech or writing to communicate instead through drawing.”(Helm-Estabrooks & Albert, 2004, pg. 273)
25CDP Candidate Profile (Helm-Estabrooks & Albert, 2004) Inability to communicate information through speech or writingAbility to use a medium-point felt-tip pen to copy one dimensional shapesRelatively intact visual memoryGood visual attention skillsAlert, cooperative, and willing to pursue a drawing program to improve functional communicationPre-morbid artistic skills are not required, nor it is necessary to draw with the dominant hand* Comprehension of the client should be functional.
26Overview of Steps Basic semantic-conceptual knowledge Knowledge of object color propertiesOutlining pictures of objects with distinct shape propertiesCopying geometric shapesCompleting drawings with missing internal and external featuresDrawing objects with characteristic shapes from memoryDrawing objects to command from stored representationsDrawing objects within superordinate categoriesGenerative drawing- animals and modes of transportationDrawing cartooned scenes(Helm-Estabrooks & Albert, 2004)
27Step 1 Basic Semantic-Conceptual Knowledge “Circle the objects that belong together.”Criterion: 100% identification of the 5 items in 5 different superordinate categories
28Step 2 Knowledge of Object Color Properties “Select the correct color (from 12 colored markers) and color that item.”Criterion: Correct color choices for all nine items.
29Step 2 Knowledge of Object Color Properties “Select the correct color (from 12 colored markers) and color that item.”Criterion: Correct color choices for all nine items.
30Step 3 Outlining Pictures of Objects with Distinct Shape Properties “Take this pen and draw an outline around each object without touching the lines of the pictures”Criterion: Each object outline conforms to shape and does not touch or trace the lines of printed stimuli
31Step 4 Copying Geometric Shapes “Take this pen and copy each figure. Your drawings should not touch the examples.”Criterion: For each shape, all lines are present, in correct proportion and in proper relation to one another
32Step 5 Completing Drawings with Missing External and Internal Features “Show me what is missing in this picture…Okay, now take this pen and fill in the missing part.”Criterion: All completions are successfully rendered, complete, and conceptually reasonable
33Step 6 Drawing Objects with Characteristic Shapes from Memory “Look at this object. (Present single card.) Remember what it looks like, because I’m going to ask you to draw it. Okay, are you ready? Take this pen and draw the object here.”Criterion: For each shape, all lines are present, in correct proportion and in proper relation to one another
34Step 7 Drawing Objects to Command from Stored Representations “Look at this word. It says____ Can you draw____right here?”ice creamCriterion: Drawings of each object can be correctly identified by a naïve judge who is not shown the target words
35Step 8 Drawing Objects Within Superordinate Categories “See this word? It says ___. I want you to think of a _____ and draw it right here.”toolCriterion: All responses belong to the named superordinate category and can be identified correctly by a naïve judge not provided with the target words.
36Step 9 Generative Drawing: Animals and Modes of Transporting People “On this page, I want you to draw as many types of animals/modes of transportation as you can. Make your drawings clear enough so that anyone could name the animals/modes of transportation that you draw.”Criterion: All responses are correct exemplars of the superordinate categories and are correctly identified by a naïve judge.
37Step 10 Drawing Cartooned Scenes “Show me what is funny about this picture…Now study the picture well because I’m going to ask you to draw it from memory.”Criterion: Each panel should be accurate enough for a naïve judge to identify all elements necessary for understanding and stating the message (joke) of each cartoon.
38CDP: Case Study (A.S) 45-yr-old male Status post left basal ganglia intracranial hemorrhagic CVA 10/03/2008DentistBilingual in English (primary) and JapaneseMixed , non-fluent aphasia- Verbal expression characterized by:- 1-word utterances- verbal apraxia- decreased loudness and pitch range- confrontational naming relatively intact- Written expression characterized by 1-word phrases- Severe auditory and reading comprehension deficits(1-2 word phrases)- Used writing and gestures to supplement verbal expression
39Sample CDP Goals (A.S.)Goal: To improve multimodality communication, the client will draw three-panel drawings from memory with enough detail so that a naive judge can identify the message.Objective 1: In the clinical setting, when given the name of an object with distinct visual features (i.e., ice cream cone, palm tree, etc), the client will draw the object with enough detail so that a naive judge can identify the object with 100% accuracy in a minimum of 10 trials.Objective 2: In the clinical setting, when given a superordinate category (e.g., tool, animals, vehicle, etc.) and no prompts, the client will draw at least one object within the specified category with enough detail so that a naive judge can identify the object with 100% accuracy in a minimum of 10 trials.Objective 3: In the clinical setting, when given a three-panel cartooned story, the client will draw the story from memory with enough detail so that a naive judge can identify the message of the cartoon in 2 out of 3 trials.
40CDP Training/Tasks (A.S.) Began with Step 3 of the CDPDuring Step 7Modeled adding more detail to unrecognizable drawingsClient began to spontaneously add more detailClinician provided the client with a puzzled look whenever his drawings were not recognizableHomework given for Steps 8-9CDP supplemented by multimodality communicationBarrier tasks encouraged all modalitiesStep 3 = (outlining objects with distinct outer configurations)Step 7 = drawing objects to command from stored representationsOnce the client began to spontaneously add more detail to his drawings, the clinician provided the client with a puzzled look whenever his drawings were not recognizable in order to encourage the addition of more detailHomework tasks were given for Steps 8-9CDP tasks were supplemented by multimodality communication training requiring the client to use all modalities (verbal expression, written expression, gestures, and drawing) to communicate pictures in a barrier task
41CDP Outcome MeasureThe client will draw one-, two-, and three-panel cartooned stories from memory.Panel should be accurate enough for a naïve judge to identify all elements necessary for understanding and stating the message (joke) of each cartoon.(Helm-Estabrooks & Albert, 2004)
421-Panel ModelCopyright Helm-Estabrooks, N. Communicative Drawing Program Cartoons
48Maintenance/Generalization Data Copyright Helm-Estabrooks, N. Communicative Drawing Program Cartoons
49Maintenance/Generalization Data 123Caregiver interview and the Communicative Effectiveness Index (CETI) indicate that A.S. primarily communicates via verbal and written expression and drawing
50Potential Supplemental Practice - Game Apps Draw SomethingSocial drawing and guessing gameWhiteboardCollaborative drawing appThese game apps are popular social games where you can connect with friends to create interactive drawings. While these apps are not therapeutic in nature, they emphasize the therapy activities and create opportunities to generalize
51Agenda Multi-Modality Communication and Aphasia Drawing and Aphasia Correlations with NeuroplasticityDrawing and AphasiaiTechnologies for Communicative DrawingExamples of available iAppsFeatures of available iAppsRecommendations for SelectionCommunicative Drawing and iTechnology in a Clinical SettingNew Directions and Future ResearchQuestions and Answers
52New FrontierThe role of new technologies in multimodal communication, including drawing.More questions than answersIs new technology applicable for drawing purposes?What technology is best?Any hidden pitfalls?Any potential advantages?Where to begin??????
53Technology for PWAMost often technology is used in speech therapy for: stimulus material, data tracking, and AAC (ASHA, 2011)Technology-aided therapy effective, due to the increased intensity of treatment (van de Sandt-Koenderman, 2011)Likely due to potential for use with multiple communication partners and use across multiple settingsApps and tech used in ST for children – especially Autism & Child LanguageTherapy apps for target drills and practice, social apps to use outside of therapy for reinforcement of skillsVan de Sandt-Koenderman found that computer-aided therapy for people with aphasia was more effective due to the increased intensity of treatment as technology improved performance of functional goals and increased social participation.Tactus Therapy Solutions – Comprehension therAppy which works on auditory and reading comprehension, Naming Therappy which provides target drills for namingAAC appsDragon Diction – text to speechNotes + - allows for drawing, writing and typing
54What is a Smartphone?“Smartphone is a mobile phone that includes software that a user is able to modify and update. The user controlled software must be able to transfer information to and from external systems” (Toyysy & Helenius, 2006, p. 110).
55Smartphone Options Blackberry Android $250-$500 Windows Phone $300-$400Windows Phone$400-$500Here are some examples of smartphones. These are intuitive devices that offer direct touch interfaceThese can access thousands of apps. They also come with built in apps which are useful such as calendars, alarms, timer, notepads, voice recorder, and navigation.
56iPhone $400-$600 HP Palm $400-$600 Nokia $400-$600 A few more smartphones – the iPhone has access to 350,000 +appsSome smartphones require set-up which can be difficult – the store can set-up and provide support if needed
57Alternate Formats - Tablets iPad Android Tablet$500 - $750 $480-$750There are now other ways to access iTechnology – the tabletHere is the iPad from Apple and the Android tabletAnother advantage of the tablet is the size – easier to read, larger work spaceAlso, some of the newer e-readers (i.e., Kindle) offer access to apps
58Alternate Formats – iPod Touch The iTouch is the same size as the smartphone and has the same capabilities to access the apps. There is just no phone access.$299-$399
59What is an App? “Apps” is an abbreviation for application Piece of software that can run on the Internet, on a computer, on a phone or other electronic devicesVariety of available on different smartphonesSome require internet connectivity, while others do run off-lineThese are able to be directly downloaded to your device.The average price of an app is $2 – but many social/game apps are freeMost apps are paid with a 1 time fee – some run on a monthly service.
60What is iTechnology? Technology manufactured by Apple, collectively referred toas: iTechnology or iDevicesFor example:iPadsiPod TouchiPhonesSelection of iTechnology for this presentation and current research isn’t value judgment, just starting point.There are other manufacturers, such as Android
61Potential Benefits of Drawing Apps Unlimited numbers of pages accessible without erasingSave and organize drawings created for future access/modificationEnhance drawings with colors, stored symbols, and backgrounds.Incorporate real photographsPossibility? GPS Picture Prediction (e.g., Locabulary for picture)Export and send drawings viaSome apps include keyboard function to augment drawing with typed letters, words, or phrases.Possibility? Text-to-speechPossibility? Interactive Alphabet Board/Phoneme cueDr. O.iTechnology allows drawing to be even more interactive and allow for more communication opportunities – not just face-to-face. Now we can import pictures from the internet and incorporate them in our drawings, save and the drawings or save and access for future conversations.GPS Picture PredictionGuessing photos to import, based on location. Locabulary is an iphone app that does this with text (word or phrases). Based on GPS – guesses vocabulary (e.g., in coffee house, suggests words associated with coffee).Picture prediction = suggest images from Google images associated with locationText-to-speechClick on word written or typed to say it?Interactive Alphabet Board/Phoneme cueClick on letter written or typed to say sound
62iTechnology: A Pilot Study Ostergren, J., & Raminick, J. (n.d.) Drawing and iTechnology: A pilot study in the use of a drawing iApp by individuals with aphasia (n.d.). Unpublished manuscript. Methods4 PWAAges 20 – 80 years oldNon-fluent forms of aphasiaParticipants were evaluated on their ability to:Access and navigate within a drawing app (NotesPlus)Intelligibility of their drawings (as compared to traditional paper and pencil methods)JennParticipants were four individuals with non-fluent aphasiawho were given initial training in the use of an iApp for communicative drawing purposes, followed by practice opportunities within bi-weekly therapy sessions.Participants were evaluated on their ability to access and navigate within this iApp, as well as the intelligibility of the drawings rendered using this app, as compared to traditional paper and pencil methods.
63CDP CartoonCopyright Helm-Estabrooks, N. Communicative Drawing Program Cartoons
64Paper/Pencil vs. iPadIntelligible to a naive judge using pen and paperIntelligible to a naive judgeusing the iPadPT1HPROne-PanelYesTwo-PanelThree-PanelNoPT2PT3PT4HPLJennWe had participants draw the 1,2, and 3 panel cartoons from the CDP – shown previously using paper and pen and also the iPadMost participants drawings were comparable with both the ipad and paper/penPT __: Premorbid _____hand dominantHemiparesis in _____handUsed ______hand to drawPT 4: Premorbid _____hand dominantAdd MC handedness
65Example (PT 1): Paper/Pencil vs. iPad Here is an example of one participant’s drawings. This is the one-panel drawing with the paper/pen drawing on the left and the iPad drawing on the right. They are very comparable in detail and both intelligible from the model
66App Use: Ease of Use After Training In the pilot study, the participants used Notesplus which had the capability for individuals to draw, write and type all on the same screen. The app was used in therapy as a platform for multimodal communication. One of the research questions we had was, will the participants be able to access and navigate the app and the iTechnology.All but one participant (PT4) was rated as either completely independent or needing minimal assistance in effectively opening the app, opening a new page within the app, and retrieving a previously drawn page. PT4’s frustration with touch screen use (as described above) appeared to carry over into her overall desire to use this technology, such that she often selected to utilize paper and pencil format for communicative drawing purposes. This may have resulted in less practice time and less motivation in learning to navigate within this iApp.The use of iTechnology was found to be successful for most participants as an efficient and effective platform for multimodality communication with minimal training and instruction required for the app/iPad.Two participants in this study did have upper extremity hemiparesis of the dominant hand. PT4 utilized her non-dominant hand for drawing as did PT2. In contrast to PT4 though, PT2 was not rated as having significant difficulty in touch screen use. This warrants further investigation.All participants were given the option of utilizing either the iPad or paper and pencil during communicative drawing tasks. Interestingly, at times, even those rated as independent in iPad use were observed to select paper and pencil over technology use. This was an unanticipated observation and was not formally tracked throughout this study. Qualitatively, it was noted, however, that these instances appeared most frequently when short, quick responses were needed or after long periods without communication when the iPad “slept” and required a touch screen swipe before accessing the drawing app. Communication by nature takes place rapidly. This additional step in accessing the app, may have added time to the communication task, and for some individuals this may have been a barrier to its use for communicative purposes. Additional research is needed in this area, including the potential for eliminating this automatic “sleep” function so that the drawing app is consistently accessible without an additional step in re-activating it for use in communication.
67PT 4: Words of Caution Still obstacles to over-come Non-dominant hand use/Limited controlRegistering drawing contact (false starts)No different in stylus vs. finger-tipDifficulty and frustration with touch screen use (with and without stylus)No previous touch screen useLower levels of use and confidence with computers and technology in generalParticipant 4 had hemiparesis, so she was using her non-dominant hand in drawing. Because it was difficult to use that hand and she had never used a touch screen, she had increased levels of frustration. We surveyed all of the participants prior to using the iPad on their comfort and frequency of use of computers/technology and this participant had lower levels of comfort with technology than the rest of the group.
68Finding the Right App: Recommended Feature Analysis - ASHA defines feature matching as:“…devices are selected based on relationships between an individual's strengths… capabilities and communication needs in relation to various features of a device (ASHA, 2004, page 9).Dr. O
69Best predictor of long-term success with AT is careful selection of aids to ensure they are well-matched to the user and the environment (Scherer et al., 2007)
70Which Apps to Use: App Analysis Ostergren, J. & Gastelum, M. (n.d.). An Exploration iTechnology Drawing Apps for Individuals with Aphasia. Unpublished manuscript.Evaluation of Drawing and Writing apps for communicative drawing purposes, given:Phase 1Phase 2Phase 3As researchers have pointed out and anyone who uses this technology knows ….numerous good, great, and not so good apps availableSo after the initial pilot study, we endeavored to identify which might work _ NO EASY TASK!
71Phase 1 – App Identification (Part A) METHODSSearch Terms: Apps with “drawing” or “handwriting” in search termsConsumer Ratings: Apps highly rated by users on the Apple Market place as a 4 or higher starsDevice Mode: Apps that could be utilized on an iPad.Category: “Productivity” appsRESULTS22 drawing apps112 writing apps identifiedTOTAL = 134 appsStart Dr. O. Search TermsThese search terms were utilized to yield apps which potentially contained drawing functions. All apps with instructions and marketing information in a language other than English were also removed from this list.A rating of 3+ star (on a scale of 0 to 4+ stars) was utilized to identify apps that a wide range of individuals found helpful, suggesting that the app was user friendly, contained more features, and/or had initial bugs addressed.Consumer RatingsGiven the nature of drawing, it was felt that the larger workspace provided on an iPad would facilitate the greatest potential for effective drawing, both in future use by individuals with aphasia and given pilot studies to assess drawing use of each app.Device ModeRemoved apps in non-applicable categories such as “reference,” “books,” “games,” “music,” and “sports” apps. “Education**,” “entertainment” or “photo & video” apps alsoCategory:Eliminate duplicatesRemoved drawing apps that were for professional use (e.g., computer assisted drawing (CAD) apps, architectural and blue print apps, etc.)NOTE: Several of the apps in “education” category could be revisited to incorporate within CDP or other drawing programs, including drawing games, such as Draw Something
72Phase 1 – App Identification (Part B) METHODSLine Sizes. Apps with at least 8 line size optionsColor Choices. Apps with at least 8 color optionsBackground Choices. Apps with the ability to modify the background in some fashion beyond strictly a white canvas were selected.Saving Capacity. Apps with the ability to save drawings for future retrieval and easily modification.Ability to . Apps with an function.RESULTS5 drawing apps9 writing apps identifiedTOTAL = 14 appswere selected. It was felt that this would give individuals with varying visual and motor dexterity more choices in apps use and functionality.Line SizeStill needed to narrow it down….This criterion was utilized given that color enhancement for communicative drawing purpose can be a valuable feature, for example, in distinguishing drawing of similar shape/features, but unique color representation (e.g., apple vs. orange). In addition, research suggested that the act of drawing in individuals with aphasia may strengthen the activation of the associated lexical-phonological processes (Farias, Davis, and Harrington, 2005). Hence more color options in this respect would be a valuable feature for individuals with aphasia who wanted to incorporate color in this fashion.Color ChoicesBackground Choices. Certain apps included a variety of backgrounds to choose from, including: 1) the option to change the type of “paper”, such as white paper, colored paper, or line paper options, 2) the option to use a set of pre-stored backgrounds, such as nature scenes, textured or abstract backgrounds, and 3) the option to import photos as a background options. These varied options permit the drawing canvas to be individually adapted to the patient’s needs.BackgroundAbility to – form of communication some individuals with aphasia can’t doSaving Capacity – add to drawings and co-construction
73Phase 3 – App Evaluation METHODS Apps from Phase 2 rated using 5 point scale, as followsEase of Navigation:1 = Complex navigation………………5 = Minimal/Easy NavigationVisual Distraction1 = Maximal Distraction… = Minimal DistractionWorkspace:1 = Small Icons/Workspace…………..5 = Large Icons/WorkspaceIconicity:1 = Opaque Icons……………………………………5 = Transparent IconsEase of Navigation. number of steps needed to draw within the app (e.g., layers to draw, change color, erase, etc.) and the number layers to save and a drawingVisual Distraction. Amount of visual distractors, such as advertisements common of “free” or “lite” versionsVisual Demand. size and placement of visual icons and the size and orientation of the drawing field (e.g., drawing space that was large and adjustable in orientation) for use by individuals with potentially limited vision or other impairments in, such as visual field cutsIconicity. This criteria consisted of the researchers ranking the app given the transparency in visual icons for us by someone potentially unfamiliar with common technology symbols, such as buttons for deleting, ing, changing settings, etc.
75Writing AppsNow we would like to demonstrate the top rated apps in each category – plug in iPad – Jenn demo
76Looking Deeper: Additional Trends Uncovered Options for setting hand use (right vs. left)Moveable palm restsIcon knowledge requiredLayers in saving – Less is moreRemote access/additional accounts (e.g., EverNote)Start position importantAccess knowledge requiredTapping, swiping, and finger expansionNow we would like to demonstrate the top rated apps in each category – plug in iPad – Jenn demo
77SketchTime $1.99 Language: English Developer: Hansol Huh Rated 4+Together
82Recent Replication: Pilot Study 13 additional PWAVarying forms of aphasia (fluent and non-fluent)Varying agesAll in chronic stage of recovery from a CVAAll receiving services at CSULB for communication impairmentsAsked to draw a 1-panel CDP cartoon using paper and pen and one (1) of the following apps:Sketch TimeJotterAbsolute BoardNoteshelfDr. O.
83Intelligible to a naive judge Sketch TimeIntelligible to a naive judgeusing pen/paperIntelligible toa naive judgeusing the iPadPT 1YesNoPT 2HPLPT 3
84Intelligible to a naive judge JotterIntelligible to a naive judgeusing pen/paperIntelligible toa naive judgeusing the iPadPT 1NoPT 2HPLYesNo*PT 3HPL*NOTE: Patient experienced notable difficulty app registering finger placement (both with stylus and finger tip). – frustrated and appeared to abandon task early
85Intelligible to a naive judge Absolute BoardIntelligible to a naive judgeusing pen/paperIntelligible toa naive judgeusing the iPadPT 1YesNoPT 2HPRPT 3HPLPT 4
86Intelligible to a naive judge NoteshelfIntelligible to a naive judgeusing pen/paperIntelligible toa naive judgeusing the iPadPT 1HPLNoPT 2HPRPT 3
87RESULTS5/17 PWA had intelligible drawings in the paper and pencil mode (29%)Two (2) had equally intelligible app drawings (Notesplus)Three (3) had less intelligible app drawings (Jotter, Absolute Board, and Sketch-Time)*Noteshelf N/A because no intelligible
88Ad Hoc and Qualitative Motor control and app use: 9/17 individuals had hemiparesis (52%)3 of these individuals had intelligible drawings in one mode (33%)1 = equally intelligible drawings in both modes1 = less intelligible app drawing1 = less intelligible paper/pencilMost individuals in our study with hemiparesis had unintelligible drawings in both modes (6/9, 67%)Consider OT and CDP?
89Conclusion Interpret with CAUTION – small sample! For some individuals, drawing using an app was less intelligible, with some appsBut………
90The Devil’s in the Details – Aim for Total Communication
94Pointed to spot highlighted and said: “Water” and “Hot” Pointed to spot highlighted and said: “Water” and “Hot”. Also, gestured drinking.What is this? Is that a tree?What is this?Jotter
95Big Picture: Lessons Learned Aim for communication, not perfection in drawing.Incorporate within multimodality system (e.g., speech, writing, gestures, AND drawing)Provide education in neuroplasticity and the positive impact of drawing on verbal expressionConsider pre-morbid technology use and comfortIncrease acceptance by minimizing client frustrationFrustration – at least two of the clients abandon due to difficulty in access
96Additional Lessons Learned: Experiment/Match to Client Compare Drawings on Different AppsExplore Palm Rest/Hand Use SettingsEvaluate Stylus vs. Finger-tipIndividual preference a factorAssess with different “pen” settings on the app (fine vs. large)Assess CaseWithstand pressure from resting palm while drawingAdjust to different positions
97Additional Lessons Learned: Therapy Tips Incorporate within Multimodality/Total Communication TrainingSpeakingGesturesDrawingWritingUse Barrier TasksTrain PartnersUse CDP to Enhance Details (Internal/External Features)Assess with Referential Communication Tasks
98Referential Communication Task Based on Purdy and VanDyke (2009)CSULB Modifications:15 real color photographs of common everyday activitiesUnfamiliar listener:Instructions: A PWA is going to be describing to you pictures shown to them. You will not be able to see these pictures. The PWA may speak, write, draw, gesture (or use a variety of methods) in describing these pictures to you. Please write down what you think the picture represents.
99Agenda Multi-Modality Communication and Aphasia Drawing and Aphasia Correlations with NeuroplasticityDrawing and AphasiaiTechnologies for Communicative DrawingExamples of available iAppsFeatures of available iAppsRecommendations for SelectionCommunicative Drawing and iTechnology in a Clinical SettingNew Directions and Future ResearchQuestions and Answers
100Future Research: Information Needed Develop and test frameworks for effectively and efficiently evaluating appsExplore communicative drawing and technologyInfluence of technology in communication dynamicsCollect actual efficacy dataJust because it is new, doesn’t make it better.Move toward dedicate apps for PWA that allow for fluidity in modalitiesPWA input critical
101Agenda Multi-Modality Communication and Aphasia Drawing and Aphasia Correlations with NeuroplasticityDrawing and AphasiaiTechnologies for Communicative DrawingExamples of available iAppsFeatures of available iAppsRecommendations for SelectionCommunicative Drawing and iTechnology in a Clinical SettingNew Directions and Future ResearchQuestions and Answers
102Q & A For additional information about this presentation contact: Dr. Jennifer A. OstergrenAssistant ProfessorCalifornia State University, Long BeachDepartment of Communicative Disorders
103ReferenceASHA. (2004). Roles and responsibilities of speech-language pathologists with respect to augmentative and alternative communication: technical report [Technical Report]. Available fromBeeson, P.M., & Ramage, A.E. (2000). Drawing from experience: The development of alternative communication strategies. Topics in Stroke Rehabilitation, 7(2),Beukelman, D., & Mirenda, P. (2012). Augmentative and Alternative Communication: Supporting children and adults with complex communication needs (4th Ed.) Baltimore, MD: Brookes Publishing Co.Davis, C.H., Farias, D., Baynes, K. (2005). Understanding the effects of cuing strategies through error analysis. Brain and Language, 95,Farias, D., Davis, C., & Harrington, G. (2006). Drawing: Its contribution to naming in aphasia. Brain and Language , 97,Gonzalez Rothi, L. J., Mussi, N., Rosenbeck, J. C., & Sapieza, C. M. (2008). Neuroplasticity andrehabilitation research for speech, language, and swallowing disorders. Journal ofSpeech, Language, and Hearing Research, 51(1), S222-S224.doi: / (2008/017).
104ReferenceGrafman, J. (2000). Conceptualizing functional neuroplasticity. Journal of Communication Disorders, July-August, (Posted on Beachboard) Helm-Estabrooks, N. & Albert, M. (2004). Manual of Aphasia and Aphasia Therapy (2nd Ed.). Austin, TX: Pro-Ed. Koul, R. K, & Corwin, M. (2011). Augmentative and alternative communication intervention for persons with chronic severe aphasia: Bringing research to practice. EBP Briefs 6(2), 1–8. Bloomington, MN: Pearson Lyons, J. (1995). Drawing: Its value as a communication aid for adults with aphasia. Aphasiology, 9, Purdy, M., & VanDyke, J. (2009). Intermodal training to facilitate communication in aphasia: A pilot study. In Clinical Aphasiology Conference: Clinical Aphasiology Conference (2009 : 39th : Keystone, CO : May 26-30, Retrieved from:
105ReferencePulvermüller, F., & Berthier, M. L. (2008). Aphasia therapy on a neuroscience basis. Aphasiology, 22(6), doi: / Raymer, A. M., et. al (2008). Translational research in aphasia: From neuroscience to neurorehabilitation. Journal of Speech, Language, and Hearing Research, 51, S259-S275. Rautakoski, P. (2011). Training total communication. Aphasiology, 25, Sacchett, C., & Black, M. (2011). Drawing as a window to event conceptualization: Evidence from two people with aphasia. Aphasiology, 25(1), 3 – 26. Sacchett, C. (2002). Drawing in aphasia: moving towards the interactive. International Journals of Human-Computer Studies, 54(4), Sacchett, C., & Lindsay, J. (2007). Revealing competence and rethinking identify in severe aphasia using drawing and a communication book. In S. Byng, J. Duchan, & C. Pound (Eds). The Aphasia Therapy File, Volume 2. Hove, UK: Pyschology Press
106ReferenceSaur, D., & Hartwigsen, G. (2012). Neurobiology of language recovery after stroke: Lessons from neuroimaging studies. Archives of Physical Medicine and Rehabilitation, 93(1), Supplement, S15-S25 Toyssy, S. and Helenius, M. (2006). About malicious software in smartphones. Journal in Computer Virology, 2, van de Sandt-Koenderman WM. (2011). Aphasia rehabilitation and the role of computer technology: can we keep up with modern times?. International Journal of Speechlanguage Pathology 13(1):21-7.