1. Interactive Visual System
This presentation will probably involve audience discussion, which will create action items. Use PowerPoint to keep track of these action items during your presentation
In Slide Show, click on the right mouse button
Select “Meeting Minder”
Select the “Action Items” tab
Type in action items as they come up
Click OK to dismiss this box
This will automatically create an Action Item slide at the end of your presentation with your points entered.
Interactive Visual System By
Arthur Evans, John Sikorski and Patricia Thomas

2. What is Interactive Visual System
Flower recognition application that runs on a handheld computer
Based on CAVIAR desktop system developed by Dr George Nagy and Jie Zou at RPI
Leverage the strong points of humans and computers
Written in Java - runs equally well on a handheld or desktop computer
IVS is a technology, not just a flower identification application – any number of object could be identified with the proper database

3. Motivation Image recognition is difficult
Modern AI and pattern recognition aims to automate process – remove human from the equation
Humans and computers have different strong points
Computers excel at large scale memory and computations
Humans excel at segmentation
Combine the two for optimal performance with the least amount of time

4. Activity Diagram

5. Handheld Choice Sharp Zaurus SL-5500 Sharp CE-AG06 camera attachment
200 MHz Intel StrongArm processor
64 MB memory
1 Compact Flash port and 1 SD port
Linux OS
Personal Java support
Sharp CE-AG06 camera attachment
Plugs into Compact Flash port

6. Java 2 Micro Edition (J2ME)
Provides Java functionality to a broad range of devices
Uses concept of Configurations and Profiles
Configurations consider system specific issues: Memory, CPU, etc
Profiles sit on top of configurations and provide API used by programmer

7. Personal Java Predates current J2ME specifications
Available on many handhelds
Provides most Java desktop API
Most similar to Personal Profile
Personal Profile is the successor to Personal Java

8. IVS User Interface Load Flower Select Feature Identify Previous 3 Hits
Color 1
Color 2
Stamen color
Petal count
Petal outline
Crop
Identify
Previous 3 Hits
Next 3 Hits
Store new Flower
Auto feature extract
List extracted features

9. Methods for Feature Entry
Point to an object
Outline an object
Crop an object
Enter count

10. Segmentation Uses Sobel edge detect algorithm
Crop box around flower greatly increases accuracy

11. Classification Uses a k-nearest neighbor algorithm.
Allows user to add new flowers to the database with no systematic changes.
Handles variation within a species since sample is compared to each exemplar equally – only need a single similar exemplar in the database make an accurate classification

12. K-Nearest Neighbor
Assumes all instances correspond to points in n-dimensional space
Nearest neighbor defined as instance closest in Euclidean space
(Ax-Bx)2+(Ay-By)2… D=

13. Calculate Distances
Petal count
Color
Species A
Species B
Query

14. Testing IVS Users classified 20 flowers under 5 different conditions
Using IVS interactively on the handheld.
Using IVS interactively on a desktop.
Using IVS with automatic feature extraction on the handheld.
Using IVS with automatic feature extraction on the desktop.
Searching through a field guide.

15. Results
Using Handheld
Using Desktop

16. Conclusion
Designed with extensibility in mind. Only need to extend a few classes to identify other objects
Written as a proof of concept, showing human interaction with an image on a handheld and porting a desktop application design to a handheld
Improves both speed and accuracy of flower identification