1. Use Of Electronic Media To Promote Acquisition of Action Verbs
and Prepositions
Created in:
Autism Language Program,
Children’s Hospital Boston
Howard Shane, Ph.D.

2.
This work is funded by the National Institute on Disability and Rehabilitation Research of the U.S. Department of Education, under grant number H133E The opinions contained in this presentation are those of the grantee and do not necessarily reflect those of the U.S. Department of Education.

3. Contributors to this presentation
Acknowledgement
Contributors to this presentation
Patients and parents in the Autism Language Program
Monarch School students, parents, and staff
Model Autism Program, Boston Public Schools
Sharon Shaham, James Sorce, Meghan O’Brien, Marie Duggan & John Costello (Center for Communication Enhancement, Children’s Hospital Boston)

4. Today’s Agenda Overview of Teaching Language Concepts Program (TLC)
Why is learning language concepts difficult for some persons with ASD?
Why are visuals needed?
Three Phases of Instruction
Dynamic Scene Cues
Static Scene Cues
Element Cues
Developing Effective and Efficient Implementation Procedures for Teaching Element Cues
Using typically developing children
Adapting and applying findings to individuals with autism

5. Overview of VIP
Goal
Utilize visual supports to teach learners on the autism spectrum how to better communicate during everyday home, school and community interactions.
Key ingredients
Visual language for comprehension and expression
A ‘closed language’ with restricted pragmatic opportunity -- targeting basic communicative operations
Mentors and learners use the same visual symbols
Use of advanced computer and video technologies

6. What is Visual Immersion?
A symbol-rich environment
An environment where visual symbols clarify spoken language, support expressive communication, assist with transitions, and facilitate learning.
Used in home, school, and community
Easy access to
Non-electronic symbols
Electronic Screen Media (Monitors and TVs)

7. What Does VIP Target? Seven concrete Communicative Operations :
Protests/Refusals
Requests
Directives
Comments
Questions
Social conventions
Transitions
VIP is not intended to teach advanced communicative operations:
Abstract concepts: “…with liberty and justice for all.”
Passive voice: “The book was read by the boy.”
Complex syntactic structures: “If he hadn’t checked the weather in the morning, then he would have forgotten to bring his umbrella.”
Figurative language: “She flew to the bookstore.”
Humor : “Why did the chicken cross the road?”
VIP provides foundation for more advanced language-based intervention programs

8. VIP: Three Instructional Settings
Natural setting:
Familiar, everyday people,objects, actions, locations and events as opportunities to overlay language. Takes advantage of adding language to already learned behavioral skills:
Daily living/self-help routines
Operational knowledge
Preferred play routines
Tabletop setting:
A controlled environment involving one-to-one instruction
Virtual setting:
Electronic (television or computer) display for presentation of materials
Setting selection varies as a function of child and the skill to be learned.

9. Overview of Teaching Language Concepts Program (TLC)

10. Why Teach Language Concepts?
Autism Language Program (ALP):
Population: Moderate to severe autism (from initial diagnosis to adulthood)
Core deficit: Difficulties with language comprehension and expression, in addition to pragmatic language skills
Comprehension:
Children on the spectrum often experience difficulty comprehending spoken language and/or auditory processing impairment (Novick et al., 1980; Boddaert et al., 2003).
Many children are able to follow routine-based, familiar directives, but have not yet demonstrated comprehension of more abstract linguistic concepts (e.g., verbs, prepositions, attributes)
Expression:
50% of individuals with autism spectrum disorders do not use speech functionally (Wetherby and Prizant, 2005; Lord & Paul, 1997; Rutter, 1978)
Many children are able to reliably use PECS to request and label, but have not yet developed the ability to describe, comment, or ask questions

11. Model of Language Comprehension (Typical Development)
Short-Term
Memory
Language
Processor
Spoken
Language
Comprehension

12. Model of Language Comprehension (Moderate-Severe ASD)
Short-Term
Memory
Language
Processor
Verbal
Information
Comprehension
= Clinical insight suggests these impairments are due to:
Attention to auditory stimuli
Not understanding language as meaningful/symbolic
Fleeting nature of spoken language
Language processor ‘broken’
Combination of these
Language comprehension is often compromised
Occasional comprehension of noun labels and familiar directives (in context)
Impaired comprehension of relational linguistic concepts (e.g., verbs, prepositions,
attributes)

13. ‘Teaching Language Concepts’ (TLC) Program
A visual instruction system for hard to learn language concepts (e.g., verbs, prepositions and attributes).
TLC is a closed visual language
Instruction in the virtual environment (video clips), tabletop environment (photographs, toy figurines and miniature objects), and natural environments (home, school and community)

14. Three Phases of TLC Instruction

15. TLC Instructional Phases:
Learner progresses through all/some of three phases of visual language symbols, from concrete to abstract representations:
Dynamic Scene Cues: full-motion video clips of action scenes

16. Bypassing Spoken Language Comprehension Using Dynamic Scene Cues (Moderate-Severe ASD)
Verbal
Information
Imitation
(as inferred from
dynamic scene cues)
Short-Term
Memory
Visual
Information
(dynamic scene cues)
*Note: No involvement of language processor required
for comprehension of dynamic scene cues

17. TLC Instructional Phases:
Learner progresses through all/some of three phases of visual language symbols, from concrete to abstract representations:
Dynamic Scene Cues: full-motion video clips of action scenes
Static Scene Cues: photographs that capture a prototypical moment in the action scene

18. Bypassing Spoken Language Comprehension Using Static Scene Cues (Moderate-Severe ASD)
Verbal
Information
Imitation
(as inferred from
static scene cues)
Short-Term
Memory
Visual
Information
(static scene cues)
*Note: No involvement of language processor required
for comprehension of static scene cues

19. Implications of Static Scene Cue Mastery
Use of dynamic and static cues bypasses language processor
Due to load on language processor when using elements, some children may use static scenes as their communication system
Mastery of static scene cues is a significant accomplishment
Can be used to promote general understanding and communication related to:
Daily Living Skills
Play
Transitions
Requesting
Commenting
Clarifying
Directives

20. TLC Instructional Phases:
Learner progresses through all/some of three phases of visual language symbols, from concrete to abstract representations:
Dynamic Scene Cues: full-motion video clips of action scenes
Static Scene Cues: photographs that capture a prototypical moment in the action scene
Language Element Cues: graphic icons representing each of the individual linguistic components that comprise an action scene (e.g., subject, object, verb, preposition, adjective, etc.)

21. TLC Phase 3: Element Cues

22. Symbolate: Language Comprehension?
Stringing symbols together does not automatically result in comprehension.
Learners must first have a knowledge of language elements and semantic relationships, as provided in VIP.
Paradoxically, stringing symbols together may actually interfere with comprehension.

23. Model of Visual and Spoken Language Comprehension (Moderate-Severe ASD)
Comprehension of
Verbal
Representation
Verbal
Information
Short-Term
Memory
Language
Processor
Visual
Information
(element cues)
Comprehension of
Visual
Representation
(element cues)

24. Clinical Observations About Element Understanding
Some label elements without meaning attached (echolalic-like)
Some comprehend agent and object elements -- struggle with relational elements
Some comprehend isolated elements -- difficulty interpreting element strings
What improves element comprehension?
Immersion in symbol-rich environment
Mass trials across multiple settings and communication partners
Computer-based instruction
Persistence over extended period
Bottom line: A more effective, efficient way of teaching language elements must be developed.

25. TLC 3 Phases: Clinical Observations
Progressing from
Dynamic Scene Cues to Static Scene Cues -- is easy
No involvement of language processor required
Static Scene Cues to Element Cues -- is difficult
Requires involvement of language processor
But
The payoff is worth it -- foundation for generative language
Goal: Combine language element cue vocabulary with their understanding of semantic relationships to generate novel sentences for expressing requests, comments, replies, etc. in natural settings.

26. The Empirical Challenge
Can we better understand the process?
Can we expedite the process?

27. Developing Effective and Efficient Implementation Procedures for Teaching Element Cues

28. Top Down Approach

29. Example of Mixed Display - with Referencing

30. The TLC Approach
Strategy: First study typical children, then apply findings to children with autism
Rationale:
Not yet literate
Able to verbalize thought process
Compliance
Subjects: Ages 3 yrs, 11 mos to 6 yrs, 4 mos
Procedure: General task - sequence three linguistic elements: left-to-right reading order
agent + action + object
agent + preposition + object
Experimenter provides no spoken language to label elements
Learner demonstrates comprehension of element string by acting out directives
Multiple design-prototype-test cycles to refine TLC instructional approach

31. Iteration 1: Top-Down Approach - Sequentially Through Three Instructional Phases
Iteration 1 Procedure
Learner presented with
Dynamic scene cues; imitate action
Static scene cues; imitate action
Element cues; demonstrate comprehension by acting out directives
For each instructional phase, learner viewed the visual cues and was provided with spoken prompt, “Now you do it!”

32. Iteration 1: Top-Down Approach Sequentially Through Three Instructional Phases
Outcome:
Able to accurately imitate dynamic and static cues
Difficulty understanding task when presented with element cues (often attempted to match objects to elements rather than carry out directive)
Difficulty comprehending left-to-right sequence of ‘reading’ elements
Difficulty understanding meaning of verb/preposition (abstract) symbols
Difficulty comprehending de-contextualized element strings.
Tendency to act on objects in a familiar manner (e.g., ‘man on ladder’ rather than ‘man climbs ladder’)

33. Iteration 2: Modified Top-Down Approach
Modifications to Iteration 1 Procedure:
Introduced visual template to focus attention on left-to-right order of elements, and presented elements in left-to-right sequence
Introduced Mixed Display (Static scene cue along with Element cues) and demonstrated association between scene cue and its elements
To enhance understanding of task expectation:
Presented task as ‘game’ to discover meaning of the relational (verb or preposition) symbol
Experimenter modeled the task

34. Iteration 2: Visual Template

35. Iteration 2: Modified Top-Down Approach
Outcome:
Initially learners did not appear to attend to the relational element -- verb or preposition (e.g., “You don’t need that one”)
Still initially attempted to match the agent and object elements to their physical objects
Given repeated trials, demonstrated comprehension of the individual elements, but continued to experience difficulty with the left-to-right ‘reading’ order for elements

36. Iteration 3: Bottom-Up Approach
Modifications to Iteration 2 Procedure
Instruction started at element rather than dynamic phase
Presented elements on computer monitor
Experimenter modeled the task
Guided discovery to select combination of element strings to play corresponding video clip
Then moved to tabletop
Presented non-electronic task with identical element cues and physical objects;
Experimenter modeled the task
Learner encouraged to create element strings to direct the actions of communication partners

37. Software Applications
Learning Together with Music
Puddingstone Place

39. Iteration 3: Bottom-Up Approach

40. Iteration 3: Bottom-Up Approach
Outcome:
Older learners were able to execute each directive presented and label each element
Younger learners had difficulty attending to screen and task; more likely to engage in their own independent play with the materials

41. Iteration 4: Modified Bottom-Up Approach with Compelling Scenes
Modifications to Iteration 3 Procedure:
Created ‘fun’ scenes depicting engaging, motivating activities using human models and life-sized objects
Clicked on message window to activate video clip of corresponding element string (in hopes of enhancing attention to relational element)
Presented learner with an ‘agent’ element cue representing him/herself
Encouraged learner to use elements to direct his/her own play and actions of communication partners

42. Iteration 4: Modified Bottom-Up Approach

43. Iteration 4: Modified Bottom-Up Approach with Compelling Scenes
Outcome:
Actively engaged
Successfully combined elements - although still initial tendency not to attend to relational element (verb or preposition)
Prepositions more difficult than verbs

44. Iteration 5 Modifications to Iteration 4 Procedure:
Focus attention to relational element (verb or preposition):
Animated relational element
Click on relational element to play video

45. Key Ingredients to Teaching Element Cues
Present elements electronically
Guided discovery with element combinations (to play video)
Provide children with symbols representing themselves and familiar communication partners to allow them to direct the activity
Begin instruction with fun, motivating tasks. Gradually expand to language-arts activities.
Animate center elements to direct attention to element and enhance likelihood of comprehension

46. Directions for future TLC research
Improve symbols
Animated symbols (verbs, preposition, attributes)
Generalization of language skills targeted in TLC to natural setting
Facilitating generative language