1. EE3P BEng Final Year Project – 1 st meeting SLaTE – Speech and Language Technology in Education Martin Russell m.j.russell@bham.ac.uk

2. EE3P – BEng Final Year Project 2009-2010 - Slide 2 Introduction  SLaTE  Resources  Some possible projects  Phase 1 of project  Things that you need to know  Any questions?

3. EE3P – BEng Final Year Project 2009-2010 - Slide 3 SLaTE  Speech and Language Technology in Education (SLaTE)  Concerned with any aspect of applying speech and language technology in education  Most common applications are language related: –Interactive intelligent second language (L2) learning –Interactive reading tutors –Interactive pronunciation tutors (L1 or L2)

4. EE3P – BEng Final Year Project 2009-2010 - Slide 4 SLaTE  But, SLaTE is not restricted to language learning: –Spoken language interaction with any educational system (e.g. dictating mathematical formulae)

5. EE3P – BEng Final Year Project 2009-2010 - Slide 5 Resources  SLaTE is a relatively new topic, but there are some resources:  BEng FYP 2009-2010 web page –http://www.eee.bham.ac.uk/russellm/ee3p-fyp09_files/EE3P-BEng- FYP2009.pdfhttp://www.eee.bham.ac.uk/russellm/ee3p-fyp09_files/EE3P-BEng- FYP2009.pdf  Proceedings of the SLaTE 2007 and 2009 workshops –SLaTE 2007SLaTE 2007 –SLaTE 2009SLaTE 2009

6. EE3P – BEng Final Year Project 2009-2010 - Slide 6 Timetable  Autumn term –Group meetings –Learn background –‘Mini-project’ –Choose main project –First ‘bench inspection’ (5 minutes)  Spring term –Individual project –Individual meetings –Final bench inspection (weeks 10/11) –Final report  Summer term –Present poster at ‘Project Open Day’

7. EE3P – BEng Final Year Project 2009-2010 - Slide 7 Possible Projects  Measuring Goodness of Pronunciation (GoP)  Dictation of mathematical expressions  Selection of appropriate audio material for SLaTE based on acoustic analysis  Measuring reading fluency and its relationship with reading ability  Selection of appropriate audio material for SLaTE based on lexical analysis

8. EE3P – BEng Final Year Project 2009-2010 - Slide 8 1. Measuring Goodness of Pronunciation (GoP)  Use Automatic Speech Recognition technology to decide whether a pronunciation of a given word by a learner of English as a second language is acceptable. The learner’s utterance is compared with an acoustic statistical model and accepted or rejected based on a score. By restricting the vocabulary appropriately it will be possible to develop a simple GoP system using the HTK speech recognition toolkit. This project should include a serious evaluation of the system.

9. EE3P – BEng Final Year Project 2009-2010 - Slide 9 2. Dictation of mathematical expressions  This project involves the use of automatic speech recognition technology to dictate mathematical equations. The project will involve collecting data to determine how engineers say mathematical expressions, and then developing techniques to parse this data. For example, do engineers always say when ‘brackets’ are needed, or can the need for brackets be inferred from pause duration? A simple real-time demonstrator will be developed using a commercial speech recognition system and evaluated using carefully prepared test data.

10. EE3P – BEng Final Year Project 2009-2010 - Slide 10 3. Selection of appropriate audio material for SLaTE based on acoustic analysis  Different voices are appropriate for different applications. For example, a voice which is suitable for reading stories to children would be inappropriate for communicating instructions to soldiers on a parade ground! The goal of this project is to find out if it is possible to use techniques from automatic speech recognition to differentiate between audio material on the web which is suitable or unsuitable for SLaTE applications with children. The decision should be based on properties of the acoustic signal rather than lexical or syntactic content. An important part of the project is the evaluation and critical analysis of the system

11. EE3P – BEng Final Year Project 2009-2010 - Slide 11 4. Selection of appropriate audio material for SLaTE based on lexical analysis  This project is related to Project 3, but in this case the goal is to determine the suitability of a piece of audion for SLaTE applications with children using lexical (word content) and syntactic (grammatical) analysis. This will involve passing the audio through a commercial speech recognition system and analyzing the result. Important questions will include whether or not current speech recognition technology is sufficiently accurate to support this application, and the effect of speech recognition accuracy on system performance.

12. EE3P – BEng Final Year Project 2009-2010 - Slide 12 5. Measuring reading fluency and its relationship with reading ability  The goal of this project is to design, implement and test a system for measuring, automatically, reading fluency. The system might measure factors such as the number of words spoken per minute, or the average length of pauses. The system will be based on a commercial automatic speech recognition system. The student will collect a set of recordings of people reading with different levels of proficiency, and this data will be used to test the system. In addition, the student will obtain human judgments of the reading proficiency shown in each of the recordings and discover whether these subjective judgments correlate with the measures of fluency.

13. EE3P – BEng Final Year Project 2009-2010 - Slide 13 Phase 1 of project (autumn term)  Understand basic speech and language technology used in SLaTE  Implement a simple SLaTE system (see next slide)  Choose individual project for the Spring term  ‘Deliverables’ –Knowledge of speech and language technology –Simple software demonstration (of something) –Individual project specification

14. EE3P – BEng Final Year Project 2009-2010 - Slide 14 Phase 1 – possible project  Implement (and test) a system that takes texts from the internet and decides whether they are appropriate for a particular level of learner of English as a second language.  See the paper by Heilman, Zhao, Pino and Eskenazi (FYP 2009/10 web page) ...or you could choose something which is the first phase of your own project

15. EE3P – BEng Final Year Project 2009-2010 - Slide 15 Things you need to know  Automatic speech recognition: –How does a basic system work? –What is a hidden Markov model (HMM)? –What is HTK?  We will discuss this at the next meeting

16. EE3P – BEng Final Year Project 2009-2010 - Slide 16 Any Questions?