Technion Faculty of Electrical Engineering Project A Summer 2001 Israel Institute of Technology

3D Geometric Objects Search Project team: Lyakas Alexander Instructor: Dr. Sigal Ar

The Main Idea Given a collection (database) of objects Choose a search object Find objects that are similar to the search object The search is iterative and interactive A user marks some objects as ‘GOOD’ or ‘BAD’ The search program tries to refine the search by considering the user’s feedback

The Main Idea – Cont.

An Example

An Example – Cont.

The following components available from Starting the Project Iterative and Interactive Search for Objects by Moty Golan & Oren Kerem based on Similarity Between Three-Dimensional Objects – An Iterative and Interactive Approach by Michael Elad, Ayellet Tal, Sigal Ar. Two databases: 3D colorless models & 2D images A search program

Test the approach with 3D colored models Design & perform system tests with real users Project Requirements Improve the search program

Project Requirements - Cont. Build a database of 3D colored models Gather 3D colored objects from the WWW Perform preprocessing calculations, i.e. present each object in a way that will enable searching

Project Requirements - Cont. Improve the search program Adding a new database must not influence the search program’s code Add features needed for testing

Project Requirements - Cont. Test the system with real users Design the tests Perform tests with volunteers Draw conclusions

Working with Objects Each object is presented as a numerical vector, AKA ‘feature vector’ To calculate feature vectors we used moments of different orders on colored points in 3D colored normals in 3D colorless points in 3D colorless normals in 3D

Comparing Objects Consider two objects represented as feature vectors: We can compare them using the (square of) standard Euclidean distance: By adding weights and a bias value we can refine the distance function:

Data Preprocessing Convert the objects to the format convenient to be sampled Perform sampling Correct normals directions Normalize rotation and scale Create icons for all objects Calculate features vectors

Before sampling each object is presented as colored triangular mesh Sampling The sampling workflow: Choose a triangle to sample Sample a point, normal and color from the chosen triangle Do this as many times as needed (10,000 in our case) Ensure uniform sampling

Calculating features We approximate moments as: The pqr-th moment (of a 3D object) is defined as: The order of the moment is p+q+r Feature vector of level 3 in ‘colorless 3D’ look like:

The Search Program The extendibility requirement – adding new database must not influence the search program code The object-based solution introduces the DBLINK class Database-specific information is stored inside DBLINK objects only One DBLINK object for each database – stored on disk

The Search Program – Cont.

Saving Test Sessions Results Automatic Screen Shooting Before search refinements – with user’s ‘GOOD/BAD’ markings When the new results are displayed Manual Screen Shooting

Testing the system Several volunteers that had no previous knowledge about how the system works Tests were done for several test objects For each test object – all search configurations were tried The testers gave feedback on the search results

Test Example 1

Test Example 1 - Cont.

Test Example 2 - Cont.

Testing Results Example Object №TesterConfiguration Feedback across iterations Final feedback 595 Shpitser Boris ‘Pixels’, level ‘Results do not improve. Few good results.’ 595 Brouk Aleksey ‘Pixels’, level ‘Many good results.’ 595 Lyakas Igor ‘Pixels’, level ‘Found everything I meant.’ 890 Shpitser Boris ‘Colorless Pixels’, level ‘The search did not converge.’ 890 Brouk Aleksey ‘Colorless Pixels’, level ‘Good results in first two rows.’ 890 Lyakas Igor ‘Colorless Pixels’, level ‘Results do not change. Very few good results.’

Conclusions No search configuration worked well for all objects ‘Normals’, level 4 worked good but slow… In most cases the search converged not always with good search results… So should the colors be considered? … Searching for objects having a ‘family’ was successful with most configurations

The End See the project book for many skipped details