1. 8th Annual Post-Graduate Research Symposium
Robert Smith
School of Informatics and Engineering
Institute of Technology, Blanchardstown
Irish Sign Language Research Forum
Human Language Technologies Research Group
8th Annual Post-Graduate Research Symposium
Exploring the relationship between depicting verbs, constructed actions and non-manual features in Irish Sign Language
11th Dec 2017, DIT

2. Contents Motivation Short overview of topic area Research questions
Methodological approach
Findings
Knowledge contribution
Research Impact
11/11/2018
Robert Smith

3. Motivation Fidelity of Sign Language generation avatar performance
Findings of Smith, R.G., (2014).
Access for Deaf Community
11/11/2018
Robert Smith

4. Short Overview Irish Sign Language Often referenced with acronym “ISL”
Unique indigenous language used by the Irish Deaf Community (45,000 users approx.)
Not English, Not Irish. Unique.
Most countries have their own signed language
Limited research and poorly resourced -> cannot avail of language technologies.
11/11/2018
Robert Smith

5. Short Overview (2)
Visual gestural modality results in multichannel articulation.
Manual Features
Articulated by hands and arms
Include handshape, position, hand orientation and movement
Nonmanual Features
Articulated by head, parts of the face, shoulders and torso
Consider body lean, chin up/down, eyebrows up/down etc.…
Basic ISL example (negation)
GO | DON’T GO
11/11/2018
Robert Smith

6. Research questions
(RQ) 1: What semantic meaning and grammatical function do NMFs add to ISL?
(RQ) 2: What is the NMF meaning contribution to ISL?
[SHAPE-OF-JAR]
INDEX+fr
[DOG] CLIMB-UP
SLEEP
11/11/2018
Robert Smith

7. Research questions
(RQ) 3: How can the SOI corpus be extended to represent NMFs and their relation to other multilevel linguistic phenomena?
How does the SOI corpus compare to international norms? Are there NMFs missing? Etc.…
11/11/2018
Robert Smith

8. Research questions
(RQ) 4: How are spatially modified verbs utilised by ISL signers in comparison to the lexical form, in the SOI corpus and what are the drivers of this?
(1) Depicting verb
(2) Indicating verb
[DOG] CLIMB-UP
SEE-SOMETHING
11/11/2018
Robert Smith

9. Research questions
(RQ) 4: How are spatially modified verbs utilised by ISL signers in comparison to the lexical form, in the SOI corpus and what are the drivers of this?
(3) Plain verb
(4) Constructed Action
HEAR
TIRED LIE-BACK SLEEP
11/11/2018
Robert Smith

10. Methodology Dataset: SOI corpus (Leeson et al., 2006)
Developed in order to capture a snapshot of ISL as it was used during that time.
Natural ISL narratives across 3 different genre
Demographically representative:
63 videos
40 signers
aged between 18 and 65
from 5 locations across the Republic of Ireland
It includes male and female signers
11/11/2018
Robert Smith

11. Step 1: Annotate a subset
11/11/2018
Robert Smith

12. Analysis Subset of the corpus to be annotated by January 11 videos
3,500 sign tokens approx.
22 minutes of video approx.
Analyses of dataset
Comparative frequency/collocation analysis – specific NMFs with specific grammatical class
Market basket analysis – Which NMFs occur most commonly with specific grammatical class
N-Gram analysis – What is the most common word order?
11/11/2018
Robert Smith

13. Some prelim indicators
11/11/2018
Robert Smith

14. Some prelim indicators
11/11/2018
Robert Smith

15. Some prelim indicators
11/11/2018
Robert Smith

16. Original contributions
ISL technical analysis
Description of grammatical classes in ISL
Including an analysis on how these are used in the language.
Description of NMFs, their form and function in ISL
Description of depicting verbs in ISL and how they correlate with constructed actions.
Digital corpus building - SOI corpus
Additional annotations for future linguistic research
Focus is on improving the quality of computer generated signing avatar performance.
11/11/2018
Robert Smith

17. Research impact Access for Deaf community
The average reading age of Deaf school leavers is comparable to that of an 8-9 year old hearing child(Irish National Rehabilitation Board, 1991; Conrad, 1979).
The ratio of qualified interpreters to Deaf people in Ireland equates to 250:1 (Leeson, 2003).
Focus on sign Language generation and recognition technologies to improve access.
This must begin at the language level – currently, not enough is known about ISL to build these technologies.
Improves understanding of ISL for future research, teaching and interpreting.
11/11/2018
Robert Smith

18. Questions ?
11/11/2018
Robert Smith

19. References
Conrad, R., The Deaf School Child: Language and Cognitive Function. London: Harper Collins Publishers.
Irish National Rehabilitation Board, Access to colleges: A survey of third level centres. Dublin: National Rehabilitation Board, National Advisory Committee on Training and Employment.
Leeson, L., Sign Language Interpreters: Agents of Social Change in Ireland. s.l., Languages ofIreland, Dublin, Ireland: Four Courts Press.
Leeson, L., Saeed, J., Macduff, A. & Byrne-Dunne, D., Moving Heads and Moving Hands: Developing a Digital Corpus of Irish Sign Language. Carlow, Ireland, Information Technology and Telecommunications, pp
Smith, R.G., (2014). The role of emotional and facial expression in synthesised sign language avatars. Thesis. Institute of Technology Blanchardstown.
11/11/2018
Robert Smith