1. Introduce about sensor using in Robot NAO Department: FTI-FHO-FPT Presenter: Vu Hoang Dung

2. OUTLINE Introduce Sensors 1 Audio 2 Vision 3 Other Sensors 4

3. Introduce Sensors  Sensor is a converter that measures a physical quantity and converts it into a signal which can be read by an observer or by an instrument. A sensor’s sensitivity indicates how much the sensor’s output changes when the measured quantity changes.  Senses of the NAO, the physical devices in robots that sense the environment are called sensor.  NAO sensors can divide into 3 types main sensors: Audio: include devices which listen to sound and play to sound from microphone.(Example TTS and STT) Vision: include devices which capture image from camera. (Example Face detect, Hand detect, Line detect …) Other sensor: include devices which get value from environment. (Example touch sensor, battery sensor, infrared sensor, laser sensor, sonar sensor).

4. Introduce Sensors

5. Audio  Audio NAO has 4 microphones built into its head that it uses to listen to sound around it  Module in Audio Audio Device: manage audio inputs and outputs Audio Player: play audio file on the robot Audio Recorder: record audio file on the robot Sound Detection: detect sound events Audio Source Localization: localize sounds detected Speech Recognition: make the robot understand what a human says Text To Speech: make the robot speak

6. Audio  Example Sound detection and source localization Value return: Number of sounds detected Index of the sound start(=1), sound end(= 0) Detection time Gives an estimate of the probability[0,1] that the sound detected by the module corresponds to a real sound

7. Vision  Vision NAO has two cameras in its head. The cameras are vision sensors and provide camera images for the NAO’s computer to process. This is different from the human “sensor of sight” as sight would include processing the image To understand what the robot sees

8. Vision  Module in Vision Backlighting: enables to check if the image of the camera is backlit. Darkness: gives NAO the ability to detect if it is in a dark environment. Face detection: gives NAO the ability to detect and recognize human faces. Landmark detection: gives NAO the ability to detect specific visual landmarks. Movement detection: gives NAO the ability to detect some movement and tell where it comes from. Photo capture: provides tools to take pictures and save them on disk. Red ball detection: gives NAO the ability to detect red and circular objects. Video device: manages video inputs. Video recorder: provides video recording tools. Vision recognition: provides learning and recognition capabilities of visual patterns. Visual compass: gives NAO the ability to use an image as a compass. Beside: Hand detection, line detection, flashcard….

9. Vision  Example Landmark detection: use to get position of NAO with environment around Value return: Landmark ID Landmark’s location in camera angles The Landmark’s size in camera angles Landmark’s is oriented about the vertical axis with NAO’s head

10. Vision  Example Red ball detection: Value return: CenterX, centerY are center of the ball in angles SizeX, sizeY are the ball “horizontal and vertical radius” Position of the camera in Torso frame Position of the camera in robot frame

11. Other Sensors  Besides above sensors, NAO also uses other sensors as touch, press, balance for robot, measure level power in robot Sensors: deals with bumpers, tactile hands, tactile head and press chest button. Battery: manage level power and warn the user Fsr: generates the event whenever the foot contact changes Infrared: allows IR communication with NAO Laser: retrieve data sent by the Laser head(optional device) Sonar: retrieves ultrasonic sensor.  Module in other Sensors

12. Other Sensors  Example Sensor: get value follow bumper, tactile, chest button Bumper Left bumper Right bumper Tactile head Front Back Rear Tactile hand Back Left Right Chest button Simple click Double click Triple click

13. Other Sensors  Example Sonar: 2 emitters, 2 receives Mode operation Frequency: 40kHz Sensitivity: -86dB Resolution: 1cm Detection range: 0.25 – 2.55 Effective cone: 60o

14. Other Sensors  Example Event sensor : Sonar Left Detected Sonar Right Detected Sonar Left Nothing Detected Sonar Right Nothing Detected Sonar Left Nothing Detected and Sonar Right Nothing Detected

15. Other Sensors  Example IR: Use case Use NAO as a remote control Set NAO to receive orders from a remote control Make several NAOs communicate together

16. Other Sensors  Example Led Sensor : allows you to control NAO’s LEDs

17. Example - Subscribe  Example Subscribe event:

18. Example - Subscribe  Example Subscribe event:

19. Example - Subscribe  Example unSubscribe event:

20. Example - Subscribe  Example Get value:

21. Questions and Comments Thank for Your Attention