Designing a Multi-Lingual Corpus Collection System Jonathan Law Naresh Trilok Pace University 04/19/2002 Advisors: Dr. Charles Tappert (Pace University) Dr. Zhong-hua Wang (IBM) Dr. Fred Grossman (Pace University)

What is a Corpus ? Any collection of more than one text can be called a corpus, (corpus being Latin for "body", hence a corpus is any body of text). Corpus in modern linguistics must have these properties –Sampling and representation –Finite Size –Machine-readable form –A standard reference

Importance of a Corpus for Automatic Speech Recognition (ASR) To Provide Training data for Speech Recognition To supply Training data for Automatic Language Identification To offer body of language to Research Community To enable analysis of language at all levels. To support transcription and labeling document for linguistic research

Sample Corpus Speech Wave File

Corpus Collection System Overview

Major Components –Corpus Collection Module via telephone Native speakers –Corpus Verification Module via Web or telephone Native speakers

Data Recordings Process –Via toll-free number on Tellme platform Caller select native language Prompt for general attributes (for naming convention) Prompt with pre-defined scripts (for short utterance) Prompt with open set responses (for long utterance)

Corpus Collection Protocol “Script” of questions and prompts for user responses Reproduced in language by native speaker (all in wav files) Prompts and Questions are all the same in all languages –Are you male or female (gender) ? –What day is today (date)? –What time is it (time)? –Please say all the days of the week ? –Describe the route you take to work or school (route)? –Describe the climate today (climate) ?

Corpus Collection Module Add Language or Prompts

Corpus Verification Module

Corpus Verification Module (cont.)

System Demonstration