1. Enhancing the Web With End-User Programming Tak Yeon Lee, Ben Bederson

2. Outline Problem Inefficiencies on the Web Challenge Building extension is very hard Our goal Expressive and Easy Programming Env. - Improves a wide range of tasks - No programming skills required

3. Snapshot ANY WEBSITE

4. Inefficiencies on the Web 1. Missing relevant information “I want to see the pictures.”  Detail page “I want to see the pictures.”  Detail page “Is it a good value?”  Kelly’s Blue book “Is it a good value?”  Kelly’s Blue book “How far is this place?”  Google Map “How far is this place?”  Google Map “Is the model reliable?”  Auto forums “Is the model reliable?”  Auto forums

5. 2. Removing unwanted information AD

6. 2. Removing unwanted information

7. 3. Repetitive Operations Howe-TheRiseOfCrowdsourcing-2006.pdf Kittur-CrowdsourcingUserStudies-2008.pdf Brabham-CrowdsourcingAsAModelFor-2008.pdf Alonso-CrowdsourcingForRelevanceEvaluation-2008.pdf Kittur-08-001.pdf 60947(1).pdf Alonso2008.pdf 10.1.1.154.444.pdf “I want to download all PDFs named as [first author]-[title]-[year].pdf ” “I want to download all PDFs named as [first author]-[title]-[year].pdf ”

8. 1. Finding mistakes 2. Add comments 3. Calculate the total points 4. Submit to grade server Grading Procedure Takes 4-8 hours to grade 70 students 4. Finding a needle in a haystack

9. MyDouble.zero Grading with an extension Summary of Potential Mistakes 1. Click to scroll to the code line of the mistake 2. Click to add predefined comment 3. Setting partial credit 4. Report total score to another server “Previously it took several hours, but now I can finish in 40 minutes.” - Lecturer “Three times faster than before, at least!” -Grading TA “Previously it took several hours, but now I can finish in 40 minutes.” - Lecturer “Three times faster than before, at least!” -Grading TA

10. Browser Extensions add new features and personalize experience. Over 50,000 on Chrome Web Store AdBlock >15 million users Photo Zoom >2 million users Print All E-mails in a folder Building an extension requires Programming knowledge (HTML, CSS, and Ajax) Time and effort of development The grading example took >40 hours

11. OUR GOAL IS TO BUILD Expressive and Easy programming env. The toolkit should support tasks like... Automating repetitive jobs Integrating data from multiple sources Custom filtering Information Summary We defined a domain language model having Pick, Change, Add operations as generic functionalities of the extensions.

12. Pick extract information from web pages or elements. Pick Change Add model The rise of crowdsourcing.pdf Crowdsourcing user studies... Crowdsourcing as a model... http://sistemas- humano.com... http://dl.acm.org/citation.cfm ?id=1357127 http://con.sagepub.com/... tags URLs

13. Change manipulates dataset. Pick Change Add model Arithmetic operations +, -, /, %,... sum, avg, count,... String manipulation concatenate, split, find, replace,... Filter Sort 1 2 3 3 3 6 9 [a] [b] [a] * [b]

14. Change manipulates dataset. Pick Change Add model Arithmetic operations +, -, /, %,... sum, avg, count,... String manipulation join, split, find, replace,... Filter Sort I am a boy - I-am a-boy [a] [b] Join [a] with [b]

15. Change manipulates dataset. Pick Change Add model Arithmetic operations +, -, /, %,... sum, avg, count,... String manipulation join, split, find, replace,... Filter Sort a.jpg b.gif c.png png c.png [a] [b] Filter [a] having [b]

16. Change manipulates dataset. Pick Change Add model Arithmetic operations +, -, /, %,... sum, avg, count,... String manipulation join, split, find, replace,... Filter Sort Toronto Seoul Paris [a] Sort [a] Paris Seoul Toronto

17. Add modify the original web pages. Pick Change Add model Create new elements image, text, button,... Modify element’s style Show / Hide, changing color, fonts,... Triggering events click,... Type text

18. [1]. Pick ad banner elements [2]. Hide [1] AD Pick Change Add model e.g. Removing AD banners

19. e.g. Downloading files with custom name pattern The rise of crowdsourcing.pdf Crowdsourcing user studies with Mechanical Turk.pdf Crowdsourcing as a model for.pdf Crowdsourcing for relevance evaluation.pdf [3]. Pick download links [1]. Pick title text [2]. Merge [1] with name pattern “.pdf” [2]. Merge [1] with name pattern “.pdf” [4]. Add [2] to ‘download’ attribute of [3] Pick Change Add model

20. OUR GOAL IS TO BUILD Expressive and Easy programming env. Requires no programming knowledge Programming-by-Example techniques Telling computer what I want, without worrying how to do.

21. ? ? PROGRAM OUTPUT INPUT Traditional Programming Programming-by-Example int sum=0; for(int v in input) { sum=sum+v; } return sum; 1, 2, 3 6 ! Programming-by-Example technique

22. Paris NewYork Seoul Tokyo.jpg Paris.jpg NewYork.jpg Seoul.jpg Tokyo.jpg [1][2] Merge [1] with [2] Programming-by-Example technique Programs with multiple inputs e.g. Adding suffix to strings Input Output

23. Paris, France Istanbul, Turkey Seoul, Korea Osaka, Japan Korea Japan [1][2] Seoul, Korea Tokyo, Japan 2 Filter [1] exist in [2] [3] Count [3] Programming-by-Example technique Multi-step programming e.g. Counting items of [1] that exist in [2]

24. London, UK LA, US Beizing, China Toronto, Canada UK Canada China [1][2] London, UK Beizing, China Toronto, Canada 3 [3] Count [3] Programming-by-Example technique Multi-step programming e.g. Counting items of [1] that exist in [2] Filter [1] exist in [2]

25. Demo time

26. Conclusion A new opportunity for end-users to personalise the web by creating their own extensions. Programming-by-Example techniques with Pick, Change, Add model Future work includes Usability Tune up Evaluation of the system’s learnability Please email us if you have an idea for extensions Tak Yeon Lee