1. v k equals the vector difference between the object and the block across the first and last frames in the image sequence or more formally: Toward Learning to Detect and Use Containers Shane Griffith, Jivko Sinapov, and Alexander Stoytchev, Developmental Robotics Lab, Iowa State University 2. Related Work Grasp points are frequently found on the rim of containers 4. Experimental Setup For each object q in Q 100 experiments K are recorded – 50 for each block p in P; each with the drop point d k Objects are tracked with color during the push state 5. Learning and Prediction K-means with k = 2 is applied to V to separate containment points from non-containment points. 7. Preliminary Learning Results Results for containment prediction with novel drop points among all three containers: 8. Future Work Apply prediction model to cluster containers from non-containers, adding to the current scheme that identifies the container drop area. Process the dataset for other features including: 1. Motivation Kemp et al. (2007) propose a grand challenge for robots to clean and organize a house; a task that relies heavily on a robot’s ability to understand the affordances of containers. Some work has focused on developing accurate 3D models of objects without considering the affordances of them. Saxena et al. (2008) have applied machine learning techniques to find grasp locations on non-3D models. Sinapov and Stoytchev (2008) introduced a framework in which a robot learns tool affordances from interactions between a tool and a puck. Containers are fundamental to almost every aspect of daily life. Little previous work address the problem of learning the properties of containers Containers offer a generalizable paradigm and good starting point toward achieving this goal. Programming a robot with the ability to discover congruency between two objects and how the objects bind to each other is a challenging task. A superquadric The features used to find grasp points may supplement other sensory data for accurate container classification. This framework leads to an accurate and compact model for possible outcomes with each tool used, which can be used to find similarities with other tools. A strict separation between containers and non- containers may be possible with this framework. 3. Infants’ Understanding of Containment The early formation of these (conceptual) object-kinds in infants is theorized to emerge from mechanical properties of objects and events (Leslie, 1994) Characterization of container affordances is a fundamental ability that infants learn through interaction and observation of objects in their environment. (Hespos and Baillargeon, 2005) Casasola et al. (2003) show that infants as young as six months conceptualize containment. Casasola et al.’s results support the argument that infants acquire containment as one of their earliest spatial concepts. 7-dof Barrett WAM arm with the Barrett hand. Sony EVI-D30 camera beside the WAM facing the table. Sequence of behaviors with four states: Three objects, Q, to test for containment and two blocks, P, to grab and drop: Recorded with robot position every 500 Hz is time in nanoseconds and experiment state. Given the output o = {d k, v k } for each experiment performed with an object, apply 10-fold cross-validation and process the data with a linear RBF Network. A value of v k near zero indicates the object and block moved together during the push interaction Robot learns a mapping m to predict the outcome v given a novel d 6. Clustering Results K-means clustering applied to the output metric v shows the area of containment for each container: v k = | Δ X q – Δ X p | + | Δ Y q – Δ Y p | An outcome v k in V is computed for each experiment k For each image i, object q has position X q, Y q and block p has position X p, Y p Histograms for each object show the results of the output metric v. Noise in the clustering data has a few explanations: 1. The block fell to a position outside of the container where the push behavior made the block move with the object 2. The block hit the rim of the container 3. The container occluded the block Black bucket Red bucketPurple bucket Fig. 1: Vector difference histograms for each object q in Q. A high peak near zero indicates containment. The other high peaks indicate only the container moved during the push state. Black bucketRed bucket Blue bucketSquare block Cross block 1. Grab block p3. Push object q2. Drop block p near object q 4. Idle 1. Distances between object, block, and robot 2. Spatiotemporal information 3. Start time of movements 4. Shape and size of both objects Modify the experimental setup in the following ways: 1. Record sound during the drop state 2. Record the amount of force required to push the object 3. Add to the dataset more containers and blocks with a wider range of shapes and sizes For further investigation: Identify strengths and weaknesses of the learning model with the “Mega Blocks” toy set, further improving the learning technique. Give robot the “Baby’s First Blocks” toy when the robot performs well with containment prediction and container classification among dynamic objects. Containment prediction with only a novel drop point, where the robot was trained with data clustered by a simple vector difference metric, is close to 80% accurate. Error is accounted for during the clustering step – many points are grouped with the wrong cluster due to the limited amount of information the k-means algorithm receives. The robot can learn and predict a containment relation significantly better than chance using a small repertoire of babbling interactions. Fig. 1: The block drop points associated with clustered vectors are shown. The green cluster indicates a containment relation; the red cluster indicates non-containment.