Location-Aware Image Database Yung-Hsiang Lu Center for Wireless Systems and Applications (CWSA) School of Electrical and Computer Engineering Purdue University Joint Work with Edward Delp and 495M team Spring 03-

2 Motivation Many people are carrying small cameras + displays + wireless networks. Potential applications: send pictures to friends, see videos, surf web … Big idea: turn every camera to a general search machine, no keyboard, no keywords. –If you can see it, we can find it. –Unfortunately, this is a very hard problem.

3 Image-Assisted Navigation Images are used to assist navigation by constructing an image database. We use Purdue campus as the environment. –Most streets have no signs. –Building names are often too small to see. Most Purdue buildings have similar styles and colors.  Add location information: GPS and compass heading.

4 Limitation of GPS If we have GPS, do we still need images?  GPS signal strength is affected by weather and surrounding buildings.  A user may stand at the same location looking at different buildings (in particular, remote landmarks)  GPS provides no information on orientation unless the user moves.  Images provide direct visual feedback to the user for navigation.

5

6

7 Image-Assisted Localization information about the objects in the image query through wireless network GPS and compass information image direction + Navigation thumbnail + description

8 Location-Aware Image Database Each image has annotations –GPS location –compass heading –day and time –weather Potential applications: –helping users to determine the current location –providing images for navigation

Demonstration available during this workshop Please visit our poster tomorrow.

10 Building and Using the Database

11 Current Status 477 images 25 buildings included taken on different sides, angles, time, weather … multiple resolutions approximately 115 m 2 per image excluding building areas, 34.5 m 2 per image

12 Sample Images

13 Image Comparison Results without GPS (Mechanical) input (Chemical Engineering) with GPS (Chemical Engineering)

14 Image Comparison Results without GPS (BRWN) input (MSEE) with GPS (MSEE)

15 Image Comparison Results 60 input images not in the database: 57% accurate results with GPS, 5% without GPS control comparison: 100% accuracy with and without GPS if the input images are from the database

16 Applications with the Database providing virtual campus tours assisting actual campus tours (without getting lost!) improving image comparison algorithms –mostly brown, green, or gray  color histograms are insufficient –time of a day  sunlight, shadow –weather –snow –…

17 Future Work perform more testing currently, tests are performed in the lab due to poor wireless signals outdoors improve user interface add more images into the database extend beyond Purdue campus include other types of objects Krannert new Computer Science

18 Education Contributions Built completely by undergraduates through a course project (495M) in 4 semesters. + Team work + Wireless communication (both client and server) + Database + Image processing + Software engineering + Purdue history

19 Conclusions We construct an image database annotated with location, orientation, time, and weather information. It assists users in finding their locations and guiding them to the destinations. It offers a framework for future studies on location- aware services, image processing, and image databases. Acknowledgments: CWSA, ECE 495M sponsors, Intel

20