Controlling Computer Using Speech Recognition (CCSR) Creative Masters Group Supervisor : Dr: Mounira Taileb

Outline Introduction. Problem definition. Suggested solutions. CCSR architecture. Implementation tools. Testing : performance testing. Usability testing. Implementation difficulties. Future work.

Computer is considered as an important tool in different fields and for different age categories. As we know, the number of computer users is increasing year after another. For this increasing many technologies are provided to facilitate the use of this device Introduction

Problem Definition A computer user may need to control his device while he is busy with other job: - Having a meal. - Presenting a lecture. - Physically challenged.

Suggested Solutions 1-Using hardware attached to the wheelchair of the physically challenged person. 2-Using glasses with attached video camera. 3-Using speech recognition software.

CCSR is an application that enable normal and physically challenged people to control their computer using speech recognition instead of using mouse and key board. CCSR is implemented in Arabic language. Proposed Application

CCSR Architecture Figure1: CCSR Architecture

Implementation Tools 1- HTK toolkit for building recognition model. 2- C# for GUI and API functions to deal with system files. 3- Audacity for sample recording.

Implementation of CCSR - Explain CCSR interface and our commands. - Start application running. - First scenario is: Desktop–Right- open. - Explain batch file and the comparison with training result. - Second scenario is: favorite. - Third scenario close the application.

Testing 1-Unit testing. 2-Integration testing. 3-Performance testing. 4-Usability testing.

Performance Testing (1) Figure2: Recognition rate, 7 samples in the training Figure3: Recognition rate with 30 samples in the training

Performance Testing (2) Figure5: Performance testing of all commands, 12 commands, 51 samples in the training, tested by 6 users

Performance Testing (3) Figure 6: Performance testing for 3 consecutive commands, 8 users

Performance Testing (4) Figure8: Performance testing of 4 consecutive commands, 8 users

Usability Testing CCSR GUI tested on 6 users and it gives an accepted response time. Because there is only one way to execute each function (only one command to activate each function as CCSR tutorial) so the users are interact easily with CCSR interface. When user requests the command by his speech, the result (for example computer shutdown) will show on the screen immediately. All tabs and buttons in CCSR GUI are designed based on HCI guidelines.

Implementation Difficulties (1) 1- In data preparation (Grammar and Dictionary building) Figure 9: Grammar file Figure 10: Dictionary file

Implementation Difficulties (2) 2- Item selection function and getting item path.

Implementation Difficulties (3) 3- Noise cancelation in Visual Studio environment. Tried solutions: 1-Change default visual studio recorder. 2- C# audacity component. 3- Using Sox in command prompt.

Future Work In the future work developers will improve the application by: 1- Solve noise cancellation problem in visual studio. 2- Add voice print feature that enables user to log in to his computer account using his voice. 3- Start CCSR application when Windows is booted.