1. Bryan Willimon IROS 2011 San Francisco, California Model for Unfolding Laundry using Interactive Perception

2. Using a PUMA 500 and 3D simulation software, a piece of laundry is pulled in different directions at various points of the cloth in order to flatten the laundry. Overview

3. D. Katz and O. Brock. Manipulating articulated objects with interactive perception. ICRA 2008 Previous Related Work on Unfolding Cusano-Towner et al. were aimed at flattening a piece of crumpled clothing by implementing a disambiguation phase and a reconfiguration phase.

4. First Phase Remove any minor wrinkles and / or folds Pulling the cloth at individual corners every 'd' degrees Second Phase Define cloth model Calculate various components needed for the cloth model Model to Unfold Laundry into a Flat Canonical Position

5. First Phase Each step of the process is numbered along with each orientation that is used to go from step i to step, as i goes from. In each step, the outer edge of the piece of clothing is grasped and pulled away from the center of the object.

6. Second Phase Cloth Model Peak RidgeCorner ContinuityCorner LocationsPeak ContinuityDiscontinuity Check

7. Second Phase Cloth model equation is as follows: represent the coordinates of grasping on the image and represents the orientation at which to pull the object.

8. Second Phase Peak ridge equation is as follows: where is the value on the depth map ( is the image), and HP is the highest point (i.e. largest intensity value) on the depth map. The equation returns an point (centroid) and the orientation (major/minor vectors) of the peak ridge.

9. Second Phase Corner continuity equation is as follows: where is a window around pixel of size that finds corner locations, with This equation finds locations of all detected corners and returns the locations in terms of position,, and the orientation,, in which the corner is angled,

10. Second Phase Corner locations equation is as follows: where “ ” means ”to be on the same continuous surface as”. This equation locate corners on the top surface that is continuous with the and returns a list of corners that are a subset of

11. Second Phase Peak continuity equation is as follows: where if the corner is on the peak region, and otherwise. The equation determines if a detected corner has a smooth surface or discontinuity close to the peak surface and returns a subset of that contains the locations of corners.

12. Second Phase Discontinuity equation is as follows: where is the intensity value of a binary image with the locations of discontinuity labeled as 1 and everything else labeled as 0. Discontinuity locations are found by using, a 3 by 3 window surrounding

13. Second Phase If the difference in intensity change is greater than some predefined threshold,, then the pixel is labeled as an edge (in our experiments, we use pixels). The depth map may also contain areas with continuity regions with a steep slope. The way to get around this problem is to double check all discontinuous areas with a larger window, If the slope is consistent, then it is not discontinuous, otherwise it is.

14. Experimental Results The proposed approach was applied to a variety of initial configurations of cloths to test its ability to perform under various scenarios using 3D simulation software. We tested our approach on a single washcloth to demonstrate how our algorithm works on a piece of laundry. Five experiments were conducted: 1)Differences between Models of the Nearest Neighbors 2)Experimental Test of Algorithm 3)Taxonomy of Possible Starting Configurations 4)Test to Fully Flatten the Cloth 5)Experiment using PUMA 500

15. Differences between Models of the Nearest Neighbors : The eight different orientations are consistent of starting from degrees, considering that 0 degrees is pointing to the bottom of the image moving in a counter-clockwise direction, every 45 degrees intervals (i.e. 0, 45, 90, 135, 180, 225, 270, 315 degrees). Experimental Results

16. Differences between Models of the Nearest Neighbors : The lower the difference value, the more in common the two configurations share in terms of shape space. This histogram is to illustrate how much the cloth configurations can change from pulling from a single point. Experimental Results

17. Experimental Test of Algorithm : This experiment tested the first phase of the proposed algorithm and monitored the process from eight iterations of pulling the cloth. The models continually change configurations in a manner that flattens and unfolds larger areas of the cloth as the iterations increase. Eventually, the cloth is mostly flattened out to a more recognizable shape in the final iteration. Experimental Results

18. Experimental Test of Algorithm : The following equation describes how the percentage of flatness is calculated: where is the value on the depth map The overall goal is to achieve 100% flatness for the next step in the laundry process. Experimental Results

19. Taxonomy of Possible Starting Configurations : The initial and final configurations of three different starting configurations after going through the first phase of the proposed algorithm in eight iterations. Experimental Results The dropped cloth was created by dropping the cloth onto the table from a predefined height. The folded cloth was created by sliding the article across the corner of the table and allowing it to fold on top of itself. The placed cloth was placed on the table from the same position as the dropped cloth.

20. Test to Fully Flatten the Cloth : This experiment tested the proposed algorithm in determining if this approach would completely flatten a piece of clothing. The test used the first and second phase of the algorithm to grasp the cloth at various locations and moved the cloth at various orientations until the cloth obtained a flattened percentage greater than 95%. Experimental Results

21. Test to Fully Flatten the Cloth : The percentages of flatness range from The figure below shows the percentage of flatness against all iterations of the algorithm. Experimental Results

22. Experiment using PUMA 500 : The goal of this experiment is to test the performance of our algorithm in a real world environment using a PUMA 500 manipulator. We used a Logitech QuickCam 4000 for an overhead view to capture the configuration of the cloth. Experimental Results

23. Experiment using PUMA 500 : The four steps illustrated below show how the robot interacted with the cloth in each iteration. Experimental Results

24. Conclusion  We have proposed an approach to interactive perception in which a piece of laundry is flattened out into a canonical position by pulling at various locations of the cloth.  The algorithm is shown to provide an initial step in the process of unfolding / flattening a piece of laundry by using features of the cloth.

25. Questions?