Extracting Valuable Information Lazily Shiry Ginosar
One picture is worth a thousand words
If you have never seen an image, a description might help your imagination
Imagine a picture. It is a painting of two lovers. A man and a woman. Only their faces and necks are shown but they are covered with white sheets. In the background there is greenery and blue skies with white clouds.
The lovers Rene Magritte, 1928
Most of the web is not accessible to the visually impaired Why? 1. Images do not have proper captions or are not captioned at all. 2. There are images that are required for navigation.
Our task is to attach a description to every image on the web Wait… Didn’t already do that!? Not really…
only provides word labels. These are great for image search but not enough for accessibility
Labels: Eat, Drink, Man, Woman
Labels: Man, Woman
Our task is to attach a description to every image on the web Our task is to attach a descriptive paragraph to every image on the web
Following the EVIL tradition, this will be done in the form of a FUN GAME
Treasure Hunt Picture Hunt
Input Output behavior image Descriptive paragraph Game
The game is an internet race between multiple players all aiming to find the same randomly chosen image. They are the “seekers”
The only information they have about this image comes from a written description that is given to them
We provide them with an image search engine Because searching is cool
A typical seeker screen The image is a drawing of a man and a woman… drawing man woman
One of the players is the “describer”. He is the only one who gets to see the image. He will provide us with the description we are looking for
A typical describer screen The image is a drawing of a man and a woman…
Once a seeker finds the correct image, he becomes the next describer and a new image is chosen
Points The seeker that finds the image first The describer, in proportion to the time it took the seekers to find the image
We are testing other ideas to make this more fun Taboo words Intentional corruption of search results A mega puzzle for each game
This form of a FUN GAME will hopefully provide us with descriptions of all images on the web Thus contributing to the accessibility of the web
One picture is worth a thousand words
Shiry Ginosar Advisor: Professor Manuel Blum Graduate Mentor: Luis von Ahn Brought to you by EVIL
Labels: Man, Woman
Labels: Dali, portrait, photograph