1. Dr. Babasaheb Ambedkar Marathwada University, Aurangabad
Department of Computer Science & IT, Dr. B.A.M.U., Aurangabad

2. Content What is Speech Recognition ? How human produces Speech
Application Areas
What you can do with Speech Recognition
Applications related to Speech Recognition
Steps of Signal Processing
Speech Recognition
1. Analysis
2. Feature Extraction
3. Modelling
4. Matching
Database Statistic of Department

3. What is Speech Recognition ?
Speech recognition is the process of converting an acoustic signal, captured by a microphone or a telephone, to a set of words.
The recognised words can be an end in themselves, as for applications such as commands & control, data entry, and document preparation.
They can also serve as the input to
further linguistic processing in order
to achieve speech understanding.

4. How human produces Speech:

5. Physically Handicapped
Application Areas
Application
Areas
Education
Domestic
Military
Robotics
Medical
Physically Handicapped

6. What you can do with Speech Recognition:
Transcription
Command and control
Information access
Problem solving
Call Centers

7. Cont.. Transcription Dictation, information retrieval
Medical reports, court proceedings, notes
Indexing (e.g., broadcasts)
Command and control
Data entry, device control, navigation, call routing
Information access
Airline schedules, stock quotes, directory assistance

8. Cont.. - Automate services, lower payroll Problem solving
- Travel planning, logistics
Call Centers
- Automate services, lower payroll
- Shorten time on hold
- Shorten agent and client call time
- Reduce fraud
- Improve customer service

9. Applications related to Speech Recognition:
Figure out what a person is saying.
Speaker Verification
Authenticate that a person is who she/he claims to be.
Limited speech patterns
Speaker Identification
Assigns an identity to the voice of an unknown person.
Arbitrary speech patterns

10. Steps of Signal Processing
Digitization
Converting analogue signal into digital representation
Signal processing
Separating speech from background noise
Phonetics
Variability in human speech
Pragmatics
Filtering of performance errors (disfluencies)
Syntax and pragmatic
Interpreting prosodic features

11. Digitization Analogue to digital conversion Sampling and quantizing
Use filters to measure energy levels for various points on the frequency spectrum
Knowing the relative importance of different frequency bands (for speech) makes this process more efficient
Separating speech from background noise
Noise cancelling microphones
Two mics, one facing speaker, the other facing away
Ambient noise is roughly same for both mics
Knowing which bits of the signal relate to speech

12. Interpreting prosodic features
Identifying phonemes
Differences between some phonemes are sometimes very small
May be reflected in speech signal
Often show up in articulation effects (transition to next sound)
Interpreting prosodic features
Pitch, length and loudness are used to indicate “stress”

13. Performance errors Performance “errors” include, Non-speech sounds
Hesitations
False starts, repetitions
Filtering implies handling at syntactic level or above
Some disfluencies are deliberate and have pragmatic effect – this is not something we can handle in the near future.

14. Speech Recognition
1. Analysis 2. Feature Extraction 3. Modelling 4. Matching

15. Speech Recognition Techniques

16. Speech Recognition 1. Analysis 2. Feature Extraction 3. Modelling
4. Matching

17. 1. Analysis
The first stage is analysis. When the speaker speaks, the speech includes different types of information that help to identify a speaker. The information is different because of the vocal tract, the source of excitation as well as the behavior feature.
Segmentation Analysis
Sub-segmental Analysis
Supra-segmental Analysis

18. Segmentation Analysis:
In segmentation analysis, the testing to extort the information of speaker is done by utilizing the frame size as well as the shift which is in between 10 to 30 milliseconds (ms).
b. Sub-segmental Analysis:
In this analysis technique, the testing to extract the information of speaker is done by utilizing the frame size as well as the shift which is in between 3 to 5 milliseconds (ms). The features of excitation state are analyzed and extracted by using this technique.

19. c. Supra-segmental Analysis:
In Supra-segmental analysis, the analysis to extract the behavior features of the speaker is done by utilizing the frame size as well as the shift size that ranges in between 50 to 200 milliseconds.

20. Speech Recognition 1. Analysis 2. Feature Extraction 3. Modelling
4. Matching

21. 2. Feature Extraction It is considered as the heart of the system.
The work of this is to extract those features from the input speech (signal) that help the system in identifying the speaker.
Feature extraction compresses the magnitude of the input signal (vector) without causing any harm to the power of speech signal. There are many feature extraction techniques.
Some feature extraction techniques are mentioned in table.

22. Fig: Feature Extraction Diagram

23. Table: Some Feature Extraction Techniques
Sr.
No.
Method
Property 1
Principal Component analysis (PCA)
Eigenvector-based method.
Nonlinear feature extraction method
Supported to Linear map.
Faster than other technique.
It is good for Gaussian data. 2
Linear Discriminate Analysis(LDA)
Linear feature extraction method
Supported to supervised linear map.
Faster than other technique,
Better than PCA for classification. 3
Independent Component Analysis (ICA)
Blind course separation method
Support to Linear map
It is iterative in nature
It is good for non- Gaussian data.

24. 4
Linear Predictive coding
Static feature extraction Method.
It is used for feature Extraction at lower order coefficient. 5
Cepstral Analysis
Static feature extraction method.
Power spectrum method.
Used to represent spectral envelope 6
Mel-Frequency Scale Analysis
Spectral analysis method.
Mel scale is calculated. 7
Filter Bank Analysis
It required frequencies possible
Used for filter based feature extraction.

25. 8
Mel-Frequency Cestrum Coefficient (MFFCs)
Power spectrum is computed by performing Fourier Analysis,
Robust and dynamic method for speech feature extraction 9
Kernel Based Feature Extraction Method
Nonlinear transformations method 10
Wavelet Technique
Better time resolution than Fourier Transform, Real time factor is minimum 11
Dynamic Feature Extractions i)LPC
ii)MFCCs
Acceleration and delta coefficients
II and III order derivatives of Normal LPC and MFCCs coefficients

26. 12
Spectral Subtraction
Robust Feature extraction method 13
Cepstral Mean Subtraction
Robust Feature extraction method for small vocabulary based system 14
RASTA Filtering
Used for Noisy speech recognition 15
Integrated Phoneme Subspace Method (Compound Method)
A transformation based on PCA + LDA + ICA.
It gives Higher Accuracy than the existing Methods.

27. Feature Extraction Techniques
Linear Predictive coding
Mel-frequency Cepstrum
RASTA filtering
Probabilistic Linear Discriminate Analysis

28. 1. Linear Predictive Coding
LPC is based on an assumption: In a series of speech samples, we can make a prediction of the nth sample which can be represented by summing up the target signal’s previous samples (k).
The production of an inverse filter should be done so that is corresponds to the formant regions of the speech samples. Thus the application of these filters into the samples is the LPC process.

29. Technique
Characteristics
LINEAR PREDICTIVE CODING
Provides auto regression based speech features.
Is a formant estimation technique
A static technique.
The residual sound is very close to the vocal tract input signal.

30. Advantages Disadvantages
Is a reliable, accurate and robust technique for providing parameters which describe the time varying linear system which represent the vocal tract.
Computation speed of LPC is good and provides with accurate parameters of speech.
Useful for encoding speech at low bit rate.
Is not able to distinguish the words with similar vowel sounds.
Cannot represent speech because of the assumption that signals are stationary and hence is not able to analyze the local events accurately.
LPC generates residual error as output that means some amount of important speech gets left in the residue resulting in poor speech quality.

31. 2. Mel-frequency Cepstrum
The main purpose of the MFCC processor is to copy the behavior of human ears. The derivation of MFCCs is done by the following steps.
Fig: MFCCs Derivation

32. Technique
Characteristics
MEL – FREQUENCY CEPSTRUM (MFCC)
Used for speech processing tasks.
Mimics the human auditory system
Mel frequency scale: linear frequency spacing below 1000Hz & a log spacing above 1000Hz.

33. Advantages Disadvantages
The recognition accuracy is high. That means the performance rate of MFCC is high.
MFCC captures main characteristics of phones in speech.
Low Complexity.
In background noise MFCC does not give accurate results.
The filter bandwidth is not an independent design parameter.
Performance might be affected by the number of filters.

34. 3. RASTA filtering RASTA is short for RelAtive SpecTrAl.
It is a technique which is used to enhance the speech when recorded in a noisy environment.
The time trajectories of the representations of the speech signals are band pass filtered in RASTA.
Initially, it was just used to lessen the impact of noise in speech signal but know it is also used to directly enhance the signal.

35. Fig: Process of RASTA Technique

36. Technique
Characteristics
RelAtive SpecTrAl (RASTA Filtering)
Is a band pass filtering technique.
Designed to lessen impact of noise as well as enhance speech. That is, it is a technique which is widely used for the speech signals that have background noise or simply noisy speech.

37. Advantages Disadvantages
Removes the slow varying environmental variations as well as the fast variations in artefacts.
This technique does not depend on the choice of microphone or the position of the microphone to the mouth, hence it is robust.
Captures frequencies with low modulations that correspond to speech.
This technique causes a minor deprivation in performance for the clean information but it also slashes the error in half for the filtered case.
RASTA combined with PLP gives a better performance ratio.

38. 4. Probabilistic Linear Discriminate Analysis
This technique is an extension for linear probabilistic analysis (LDA).
Initially this technique was used for face recognition but now it is used for speech recognition.

39. Technique
Characteristics
Probabilistic Linear Discriminate Analysis (PLDA)
Based on i-vector extraction. The vector is one which is full of information and is a low dimensional vector having fixed length.
This technique uses the state dependent variables of HMM.
PLDA is formulated by a generative model.

40. Advantages
Disadvantages
Is a flexible acoustic model which makes use of variable number of interrelated input frames without any need of covariance modelling.
High recognition accuracy
The Gaussian assumption which are on the class conditional distributions.
This is just an assumption and is not true actually.
The generative model is also a disadvantage.
The objective was to fit the date which takes class discrimination into account.

41. Speech Recognition 1. Analysis 2. Feature Extraction 3. Modelling
4. Matching

42. 3. Modelling
The goal of the modeling techniques is to produce speaker models by making use of the features extracted (feature vector).
Modeling techniques are further categorized into speaker recognition & identification.
Speaker recognition can be further classified into speaker dependent and speaker independent.
Speaker identification is a process in which the system is able to identify who the speaker is on the basis of the extracted information from the speech signal.

43. Modeling Approaches: Acoustic-Phonetic approach
Pattern recognition approach
Dynamic Time Warping (DTW)
Artificial Intelligence Approach (AI)

44. 1. Acoustic-Phonetic approach
The basic principle that this approach follows is identifying the speech signals and then providing these speech signals with apt labels to these signals.
Thus the acoustic phonetic approach postulates that there exists finite number of phonemes of a language which can be commonly described by acoustic properties.

45. 2. Pattern recognition approach
It involves two steps: Pattern Comparison and Pattern Training.
It is further classified into Template Based and Stochastic approach.
This approach makes use of robust mathematical formulas and develops speech pattern representations.

46. 3. Dynamic Time Warping (DTW)
DTW is an algorithm which measures whether two of the sequences are similar that vary in time or even in speed.
A good ASR system should be able to handle the different speeds of different speakers and the DTW algorithm helps with that.
It helps in finding similarities in two given data keeping in mind the various constraints involved.

47. 4. Artificial Intelligence Approach (AI)
In this approach, the procedure of recognition is developed in the same way as a person thinks, evaluates (or analyzes) and thereafter makes a decision on the basis of uniform acoustic features.
This approach is the combination of acoustic phonetic approach and pattern approach.

48. Speech Recognition 1. Analysis 2. Feature Extraction 3. Modelling
4. Matching

49. 4. Matching Sub word matching:
Phonemes are looked up by the search engine on which the system later performs pattern recognition.
These phonemes are the sub words thus the name sub word matching.
The storage that is required by this technique is in the range 5 to 20 bytes per word which is much less in comparison to whole word matching but it takes a large amount of processing.

50. 2. Whole word matching:
In this matching technique there exists a pre-recorded template of a particular word according to which the search engine matches the input signal.
The processing that this technique takes is less in comparison to sub word matching.
A disadvantage that this technique has is that we need to record each and every word that is to be recognized beforehand in order for the system to recognize it and thus it can only be used when we know the vocabulary of recognition beforehand.
Also these templates need storage that ranges from 50 bytes to 512 bytes per word which very large as compared to sub word matching technique.

51. Database Statistic of Department
All developed Speech Database are according to Linguistic Data Consortium for Indian Languages (LDC-IL) standards.
Marathi Speech Database
RRD-IMSDN (8000)
RRD-RMSDA-I (30000)
RRD-IMSDA-II (30000)
RRD-IMSDTA (37200)
RRD-CMSDA (38000)

52. Emotional Speech Database
RRD-EMSID (3800)
Artificial Emotional Marathi Speech Database (250)
Swahili Speech Database
RRD-IS2DN (3000)
RRD-IS2DA (25000)
RRD-GS3D (15000)