Special Interest Group on NETworking SIGNET Discovery, localization, and recognition of smart objects by a mobile robot UNIVERSITY OF PADUA Dept. of Information Engineering E. Menegatti M. Danieletto, M. Mina, A. Pretto, A. Bardella, A. Zanella, P. Zanuttigh Intelligent Autonomous Systems Lab University of Padua

Recognition of smart objects Goal: A robotic system exploiting Wireless Sensor Network (WSN) technologies for implementing an ambient intelligence scenario. We address the problems of object discovery, localization, and recognition in a fully distributed way. The robot does not have any a priori information! Neither on the number nor on the kind of objects in the environment.

3 RAMSES2 - Project RAMSES 2 : integRation of Autonomous Mobile robots and wireless SEnsor networks for Surveillance and reScue A utonomous M obile R obot Wireless network channel W ireless S ensors N etwork Laptop eyesIFX motes from Infineon b wireless channel

4 Experimental Set up EyesIFX sensor nodes –Infineon Technologies. –19.2 kbps bit 868 MHz –Light, temperature, RSSI sensors AMR Bender – self-made, based on Pioneer 2 ActivMedia platform – Linux OS with Miro middleware – ATX motherboard – 1,6 GHz Intel Pentium 4, 256 MB RAM, 160 GB HD EyesIFX connected to ATX via USB + EyesService class added to Miro – Omnidirectional camera, odometers

5 Step 1 - DISCOVERY Allow a bidirectional serial communication (ASCII chars) Allow robot’s applications to interact with the WSN  Physical connection between robot and mote  Serial port emulation over USB (VCP)  Standard commands for eyesIFX sensor  Predefined actions to access to the WSN Input/OutputFunctions

6 Middleware Miro The robot is programmed exploiting the framework Miro Miro is a framework for mobile robot programming developed by Gerd Mayer and Gerhard Kraetzschmar at Ulm University Miro is a middleware based on CORBA architecture for creating and managing distributed services. Miro is based on TAO libraries of the ACE framework. We interact with the eyesIFX mote on board of the robot through a Miro service we created, called EyesService. Range-only SLAM with a Mobile Robot and a Wireless Sensor Networks 6

Why Localization? WSN deploying is an annoying and time consuming task. Moreover, motes can be attached to objects that are moved around Step 2 - LOCALIZATION First goal of the project localize WSN nodes spread in unknown positions inside a building using a mobile robot.

Localization Approaches Three main ranging approaches: Angle of Arrival Time of Arrival Received Signal Strength Indicator (RSSI) Focus on RSSI: No specific Hardware required Poor outdoor ranging performance  Very poor indoor ranging performance 

Our Solution SLAM (Simultaneous Localization And Mapping), for a mobile robot moving in an unknown environment in which there is a WSN (Wireless Sensor Network). We use only: robot’s odometry; range measurements from the nodes to the robot Step 2 - LOCALIZATION

For the same range, we can measure very different RSSIWe measure the RSSI to estimate the range, then... RSSI based ranging is VERY noisy! Random variations due to shadowing and fading obscure the log-decreasing law for the received power vs distance How harsh is the indoor radio channel?

Experiments EyesIFX v2 Mote Robot “Bender” 10 x 6 m environment

Results (1/4) - SLAM Much better that classical static WSN localization algorithm Large variance on residual error for motes locations Slightly better results taking only highest RSSI measurements (Elab 2) Fig. 1 residual mean error on robot and motes position

Much better that classical static WSN localization algorithm Large variance on residual error for motes locations Results (2/4) - SLAM

If we correctly initialize the mote position in the EKF...( Elab 5 & 6 ) Results: Slight improvements on robot residual error Large improvements on mote residual error Fig. 2 Residual mean error on robot and motes position Where does the error come from?

Results (4/4) - SLAM

16 Extended Kalman Filter OdometryRSSI Measures Initialization Particle Filter Motes pose and robot position estimation Localization with delayed particle filter Localization Algorithm:

Delayed Initialization based on Particle Filter 15

18 Step 3 - RECOGNITION Goal: the robot should be able to recognize the objects from a description of the objects’ appearance. The object appearance is stored inside the object (in the motes). The appearances is coded by scale invariant feature descriptors robust also to motion blur (MoBIF) Feature matching under motion blur (Pretto et al. ICRA 2009)

19 MoBIF Matching The object in the robot’s camera image are identified matching the MoBIF descriptors (robust to motion blur) transmitted by the smart objects with the MoBIF descriptor extracted from the robot image; A minimum number of matching is needed to correctly identify an object; matching is robust to scale, occlusion, illumination and rotation;

20 Storing the appearance To achieve robustness to scale: the object is imaged at distances: near, medium, far and the MoBIF descriptor are merged To achieve robustness to rotation: the object is imaged every 20 deg. and the MoBIF descriptor are merged The merging of the MoBIF removes redundancy! 5 meters 2 meters 1 meter

21 The robot uses a simple visual-servoing image map to approach the object. An approaching direction is identified for each of the 9 regions of the image (the red arrow). Future work: IF the robot camera is calibrated the object location in the image can be feed to the SLAM algorithm to further refine the object pose estimation. Step 4 - APPROACHING

Special Interest Group on NETworking SIGNET Discovery, localization, and recognition of smart objects by a mobile robot UNIVERSITY OF PADUA Dept. of Information Engineering E. Menegatti M. Danieletto, M. Mina, A. Pretto, A. Bardella, A. Zanella, P. Zanuttigh Intelligent Autonomous Systems Lab University of Padua