1. TURTLEBOT ROBOTIC GUIDE

2. Project Description Teach a robot to guide a person to a predefined destination. General requirements: 1. Use a Turtlebot as the robot 2. The robot must use a vision algorithm(s) as the main guidance. 3. Sensors such as bump, infra-red and ultra-sound could provide direction and safety for the robot. 4. First get the robot to follow someone somehow "remembering" the route so that it can take a turn leading. 5. The solution should be easily modified for alternative routes. 6. Check out Joseph Stawicki's 2013 project for techniques about driving the robot.Joseph Stawicki's 2013 project

3. Meet Turtlebot

4. Hardware Breakdown Kinect and Mobile Base Gyro combined so all sensor data can be read from one place iRobot create listens for commands on a serial port, so serial port to USB converter required for communication. Sensors send data to laptop, laptop sends commands to the mobile base. Hardware requires software framework in order to receive, process, and send data

5. The Robot Operating System(ROS) Services Provided by ROS: Hardware abstraction Low-level device control Message passing service Name and Parameter service Package management + Laptop’s main OS is Ubuntu which is then overlaid with ROS. ROS also allows for Android connectivity to the robot

6. Developing Apps with ROS Workspace allows for packages to be installed and edited easily Very easily pulls from repositories for quick installs Symbolically linked to main ROS install Workspace set up based on directory structure ROS incorporates a lot of different programming languages C, C++, Python, Java, and more The catkin workspace allows for building and debugging across all languages ROS created ROS specific API’s for Python and C++ (rospy & roscpp) Makes it extremely easy to call ROS functions Also allows for cross-language communication

7. ROS Limitations and Frustrations  Open Source – does not necessarily mean “FUNCTIONING”  Sources, Documentation, and Support not easily obtained  Newest ROS distribution designed for newest Turtlebot base (Kobuki)  The ROS Wiki – instructions misleading  For the first time ever, Google.com was not my friend

8. How it Works  Remote communication to the Turtlebot required  Workstation Computer connects via SSH  Applications are also run locally on Workstation Computer for monitoring, information processing, and visuals  These applications receive info via HTTP  Android connectivity via HTTP Incorporated the Wifi Pineapple Allows for a portable and private Local Area Network

9. Wireless Communication

10. Services, Nodes, Parameters, Messages, Topics  Nodes are processes that perform computation  Communicate with one another via topics, services, and parameters  Topics are named busses over which nodes exchange messages  Nodes either subscribe or publish to relevant topics  Services handle requests and replies  Defined by a pair of messages  Messages are simply data structures comprising of typed fields  ROS Parameter server is a dictionary accessible via network APIs  Nodes use this to retrieve parameters at runtime

11. Software & Hardware Communication

12. Kinect Example

13. Sound Demo Keyboard inputs command Keyboard node processes command Parameters obtained from server Word node processes the sound to play based on parameters Sound node processes sound Output to speakers

14. The Puzzle Pieces  Teleoperation  Room mapping (SLAM Algorithm)  Autonomous Navigation of Known Map  Following  Voice Commands  Color Blob Tracking Its all here! But why is it in pieces? Applications are limited by the way they interact with the robot Publishing/Receiving information cannot be done on the same node at the same time

15. Room Mapping with the SLAM Algorithm

16. Autonomous Navigation of a Known Map

17. The Problem: Communication Interrupts  Both Nodes need to publish movement commands to the mobile base  The Voice Command Node is designed to continually publish directly to the mobile base node  The mobile base node sees the Voice Command messages as priority and blocks movement commands from the Follower node

18. The Pivot Node/Dynamic Subscriber  Could also be considered ‘nested nodes’  Subscribing node is dynamic  If multiple messages received simultaneously then prioritize

19. My Solution (Follower with Voice Commands)  The node needs help prioritizing  Processing and movement become slowed when subscribing to commands from multiple nodes  Using multiple pivot nodes helps speed and processing  Created new voice commands that correspond to OS commands  Bringing the process down to the OS level allows for universal actions across all applications  The use of shell scripts allows the ability to talk to ROS via command line  OS helps flag enabled nodes  Mobile base will then only listen for commands from enabled nodes

20. Command Recognition Process for enabling follow mode This particular command effects the follower node the most. Little is done inside the Voice Command Node.

21. Command Recognition Process for disabling follow mode This particular command corresponds to actions in both the Voice Command Node and the Follower Node

22. Testing/Methodology  Alter launch files to see how nodes react  On screen messages are given when things go wrong  Adjust Node publishing target  Shows which nodes are capable of handling certain applications or combinations of applications  Try as much hardware as possible  Different laptops, headsets, graphics cards, etc. respond and process very differently from one another  Experiment on EVERYTHING  If I didn’t know what it was, I played with it until I understood how it worked

23. Follower with Voice Commands Live Demo

24. Learning and Development Process  RESEARCH – find as much information from as many different sources as possible  Run it and see what happens  Try everything that’s available, watch and see how the Robot reacts and then connect what’s seen to what sources say  Guess, check, and actually read the error messages  Make alterations and see what kind of errors come up, helps to correlate what broke to what was changed

25. Project Continuation & Advice  UNDERSTAND ROS!!!  Not understanding how ROS is working makes it extremely difficult to try and make any changes  Follow the link trail  Google.com is not your friend for this project. Scimming google search results will get you almost no where. In order to find good information follow links in forums and on web pages that indicate they might be helpful. Eventually you will find something actually useful  Consider upgrading mobile base to the Kobuki  Gyro is built in allowing for more efficient processing and little calibration  Attempt to break Kinect data apart  The Kinect has many separate sensors, learning how to get data from each sensor individually would allow for efficient visual application multitasking