1. Using Computer Vision to Test Web Display Xu Liu liuxu@cs.umd.edu

2. Why test web display?  Display Bugs Bug “Opera 7.54” Normal IE6

3. More bugs BUG - Opera Normal - Firefox Normal - IE Normal - Mozilla

4. Where comes these bugs?  Different parsers on DHTML, CSS  Web designer doesn’t follow W3C standard  IE tolerant buggy HTML  Java Script, ActiveX, Flash

5. Which kinds of bugs do we have?  Text/Image Overlap  Incorrect blank area  Missing Text/Image Generally they are all layout problems

6. How do we detect these bugs  Is HTML source helpful?  Yes, but we need a correct parser which is being tested  HTML does not have straight forward relationship with display  Eyes always tell the truth – directly use the image of snapshot

7. Find the outlier  Let them vote:  IE, Firefox, Mozilla, Netscapte, Opera, MyIE…  Anyone seems distinct from others is probably an outlier  Assumption : Major are correct, Minor are incorrect

8. Basic Question  How do we compare 2 images? Are they same? Missing Front and end These 2 look different but they should be considered the same

9. Simpler Question  How do we compare 2 sequences? S1: 1 2 3 4 5 6 8 7 S2: 1 2 4 5 5 5 6 7 Direct Compare |S1-S2|=0+0+1+1+0+1+2+0=5 Dynamic Time Warping !! In fact ||S1-S2||=0, they should be considered the same

10. Dynamic Time Warping (DTW)  A widely used technique in signal processing  Speech recognition, Image matching Compare S1,S2

11. Can we directly apply DTW to compare web pages?  No! Segmentation First!! If we directly compare 2 pages: Most of the error will be omitted Local VS Global

12. A Segmented Page

13. How do we derive segmentations Edge Detection First Over Segment Merge

14. Next  For every page  We have all its segments  For every segment  If it CANNOT be found in other pages, possibly it is an error

15. Result http://www.microsoft.com/smallbusiness/default.mspxhttp://www.microsoft.com/smallbusiness/default.mspx by Opera 7.54

16. Result http://www.microsoft.com/learning/default.asphttp://www.microsoft.com/learning/default.asp by Opera 7.54

17. Shortage and Future Work  Make segmentation more accurate  Make the system completely automatic