Robotic Assistance

The PROBLEM Providing assistance for the Blind –What do we mean by “Blind?” Stereotypical blindness Visually impaired What assistance is necessary? –Many things we would consider simple tasks become very difficult

The PROBLEM – Types of Challenges Environmental –These are the ones you think of first, getting around –Don’t move those tables! –In a new place or a busy city, more likely to have either sighted assistance or guide dog –These issues more typical of severe blindness, though those with low vision have similar problems

The PROBLEM – Types of Challenges Social –Frequent (sometimes subconscious) prejudice against the blind and visually impaired –Imagine taking classes here if you were suddenly blinded –Hard to find/keep job –Hard to play sports/other social activities –Can lead to lowered self esteem

The PROBLEM – Types of Challenges Technological –Those with severe blindness cannot use computers, smartphones in the same way we do –Visually impaired people need modified tech, especially bigger screens or text-to-speech

The PROBLEM – So What Can We Do? The blind face many problems beyond navigation and movement –We can only provide so much assistance with turtlebots –Limited time –Extremely limited resources We chose to focus on assistance in the navigational sense

Navigation The main function of our project Pre-mapping –Not ideal, almost useless in real life, but without it, too much trial and error GPS, Autonomous discovery of space –Would be more useful, but limited time and budget Hybridize these!

Navigation Hybrid—Best choice for now –Instead of following blundering bot, told to wait –Once robot finds path, leads person through it –This obstacle avoidance can be something we code from scratch without use of other APIs

Speech Recognition and Text To Speech Initially wanted to use HARK –Problems with “flowdesign,” we ed the support team but no idea how long it could be Sphinx was another option –Is part of ROS already –Unfortunately, tutorial is out of date –Voice Commands to Turtlebot—VideoVoice Commands to Turtlebot First step is now to use sound_play package

Speech Recognition and Text To Speech Sound_Play provides node that translates commands into sounds –Node supports built-in sounds,.OGG and.WAV files, and speech synthesis Basically, take in text, say that text.

Speech Recognition and Text To Speech Three steps –Figure out robot speech with sound_play This can also play music and tell time, useful for technological aspect of blind difficulties –Have robot accept voice commands, translate to text Cannot be done without external software (HARK, Sphinx) If we have problems with this, instead use PS3 controller –Connect input command and output audio

Localization Interface Important part of the puzzle—need an interface that doesn’t rely on vision –First thought? Use a leash! Not as good an option as we thought—turtlebot is touchy –What now? Want robot to slow down or even backtrack to user –Hands-Free, Low Strength Radio Transmitter Omni-directional, can go through/around objects

Localization Interface – Implementation Have user wear ‘beacon,’ attach transmitter to robot through DB-25 connector Read analog inputs and use them to estimate distance –These inputs aren’t actually distance values… –Intensity values from 0 to 4095 To find actual distance, we would have to play around a little with the system, see what numbers occur at what distances

The Initial Design Navigation First, we figure out the obstacle avoidance code, and make sure it works. Localization We must then program the localization interface, so the person is not left behind. Text-To-Speech Next, we must attempt to have the robot speak, and tell directions for the user. Speech Software Finally, if we have time and the proper software, we program certain commands that can be used by the person.

The Initial Design