1. Using the Web to Interactively Learn to Find Objects
ΕΘΝΙΚΟ ΚΑΙ ΚΑΠΟΔΙΣΤΡΙΑΚΟ ΠΑΝΕΠΙΣΤΗΜΙΟ ΑΘΗΝΩΝ
ΜΕΤΑΠΤΥΧΙΑΚΟ ΜΑΘΗΜΑ : ΠΡΟΗΓΜΕΝΗ ΤΕΧΝΗΤΗ ΝΟΗΜΟΣΥΝΗ
ΚΟΚΚΙΝΟΥ ΑΝΝΑ AM :Μ1253
Using the Web to Interactively Learn to Find Objects
Mehdi Samadi, Thomas Kollar, Manuela Veloso

2. INTRODUCTION AIM: make mobile robots that are able to
intelligently perform tasks
APPROACHES:
Manually define the knowledge of the robot
Enable robot to actively query the WWW

3. RELATED WORK Yamauchi 1997 (Baseline search)
Stachniss,Grisetti and Bulgar (focus on balancing the trade-off between localization quality and the need to explore new areas)
Meger 2008 (describe an integrated robotic platform)
Kollar and Roy 2009 (integration between robots and Web)
Sjoo 2009, Aydemir 2011, Velez 2011 ,Joho- Senk and Bulgar 2011 (focus on visual object search that does not leverage the Web)

4. Object Eval (OE)
An approach ,which addresses the challenges of the find and deliver task by querying the Web.
(stages of find and deliver task: receive command, go to a location, ask for object, get object ,deliver object)
PROCCESS:
Downloading and categorizing a set of web pages
Learning the probability that a location (ex.kitchen) will contain an object (ex. Coffee)
Probability => Utility Function

5. FIND AND DELIVER TASK

6. UTILITY FUNCTION (part1)
Input: Object Name (ex. papers) and Destination Room
Output: A plan consisting of locations that robot should visit
U( plan | O)= ∑ p( plani |O) × R ( plani|O) (1)
U: Utility function
O: Object name
p(plani|O) : probability of finding an object in a location
R: the reward that robot receives when it executes i-th step

7. UTILITY FUNCTION (part2)
The problem can be formulated as finding the plan that maximizes the utility
argplanmax U(plan|O) (2)
R (plani|O) = D (plani) × I (plani) × F (plani) (3)
R: the reward that robot receives when it executes i-th step
D: the reward which is dependent on the distance
I: the reward which is dependent on the number of interactions
F: the reward which uses previous searches to help search for objects ( F = 1 or F = 0.5 or F = 0 )

8. UTILITY FUNCTION (part3)
The probability will be computed as p(plani |O) ≈ [ ∏ ( 1 – p(lj | O))] × p(li | O) (4) where, p(lj =kitchen| O=coffee)=p(locationHasObject(kitchen,coffee)

9. QUERYING THE WEB
Objects present in a location will be found frequently on the Web
Object Eval converts predicate instances into a search query
(ex: { papers, printer room} )
The text on the web that is most relevant to a predicate instance will be near the search terms
OE extracts text snippets from each of the web pages ( up to 10 words before, after and between the terms).
Multiple text snippets from the same page are merged into one
Each text snippet is transformed into a feature vector

10. To find which location type should be assigned to a page, we train an SVM classifier (input :feature vector, output :location type)
OE only needs positive examples of the predicate

11. OE EVALUATION-SIMULATION
OE searches over candidate plans to maximize the utility function (beam search ,W=10 ,N=10 no loops)
At each step a new plani is added to overall plan
Query the Web to predict the probability distribution p(li | O)
A set of simulated commands were executed for three floors of an office building (80 object types)

13. RESULTS
OE achieves a high F1 value even when it uses a few training examples
The search for 20 objects is repeated 5 times starting from different initial location to obtain 100 runs.

15. Thank you!