SpeechClipse v 1.0 “An Effective Plug-In for the Eclipse IDE” Team Members: Faizan, Javed Qasim, Ijaz Rajesh, Sudarsan Raymond, Corvin Shairaj, Shaik Suman, Roychoudhury Advisors: Dr.Jeff Gray & Dr.Barrett Bryant http://www.cis.uab.edu/info/Eclipse/SpeechClipse/ Computer and Information Sciences Department University Of Alabama at Birmingham

Motivation Speech plug-in to positively affect individuals who are physically challenged or diagnosed with Carpal Tunnel Syndrome (CTS) Increased productivity and ease of IDE navigation by concurrent control Customizable and Taylor-Made speech control Contribution towards Eclipse community 9-Jun-19 http://www.cis.uab.edu/info/Eclipse/SpeechClipse/

How does a Typical Speech Engine works ? Background How does a Typical Speech Engine works ? Picture - Courtesy Microsoft Corporation 9-Jun-19 http://www.cis.uab.edu/info/Eclipse/SpeechClipse/

Implementation The user defined “Speech-Robot” Class to translate speech commands into keyboard events Mapping Eclipse events to speech commands through rule based grammar rules Extensible design to accommodate future IDE events Complete control of keyboard through used defined KeyMap Class JSGF (Java Speech Grammar Format) to specify rules 9-Jun-19 http://www.cis.uab.edu/info/Eclipse/SpeechClipse/

Control Flow Diagram SpeechClipse TOKEN 9-Jun-19 RECOGNIZER SYNTHESIZER Rule-Based Grammar TOKEN NAVIGATOR SpeechClipse 9-Jun-19 http://www.cis.uab.edu/info/Eclipse/SpeechClipse/

CloudGarden’s Speech Engine JSAPI based implementation Consists of both Recognizer and Synthesizer Supports both Rule-Based and Dictation grammar 9-Jun-19 http://www.cis.uab.edu/info/Eclipse/SpeechClipse/

Grammar SpeechClipse Control: JSAPI Specifications: Grammar rules and event properties are the primary ways of controlling the speech recognizer’s activity JSAPI Specifications: JSGF (Java Speech API Grammar Format) Currently Used In SpeechClipse JSML (Java Speech API Markup Language) Will be used in future versions of SpeechClipse 9-Jun-19 http://www.cis.uab.edu/info/Eclipse/SpeechClipse/

Advantages of SpeechClipse Increase productivity to users by allowing multi-tasking Speech enabled programming environment for various users groups Better recognition by adapting Speech-engine with user voice Highly Customizable Rich set of Speech Commands: Hand’s Free Refactoring Hand’s Free Debugging Code templates Speech Wizards 9-Jun-19 http://www.cis.uab.edu/info/Eclipse/SpeechClipse/

Current Status Efficient navigation of Eclipse IDE Code-Templates: Automate the typing of standard programming constructs Speech-Hot-Keys: Accomplish the effect of multiple keyboard or mouse clicks Access to standard external applications An extensible design Ability to dynamically update the SpeechClipse grammar Harness the powerful refactoring and debugging capabilities of Eclipse through Speech commands 9-Jun-19 http://www.cis.uab.edu/info/Eclipse/SpeechClipse/

SpeechClipse Video Demonstration Project creation Standard keyboard capabilities Project compilation and building Hand’s free Debugging Hand’s free Refactoring Generating Java-Documentation SpeechClipse 9-Jun-19 http://www.cis.uab.edu/info/Eclipse/SpeechClipse/

Future Work Improve the Grammar using of all the capabilities of rule-based grammars. Implement Lazy-Typing using Dictation grammar Emulate mouse movements Ability to effectively code and store identifier-names Speech synthesis (Futuristic) for visually impaired 9-Jun-19 http://www.cis.uab.edu/info/Eclipse/SpeechClipse/

Thank You 9-Jun-19 http://www.cis.uab.edu/info/Eclipse/SpeechClipse/