How Do Humans Sketch Objects? SIGGRAPH 2012 Mathias Technische Universität Berlin ( 柏林工业 )Technische Universität Berlin James Brown UniversityBrown.

Slides:

Advertisements

Similar presentations

Location Recognition Given: A query image A database of images with known locations Two types of approaches: Direct matching: directly match image features.

Advertisements

Incentivize Crowd Labeling under Budget Constraint

Technische Universität München Face Model Fitting based on Machine Learning from Multi-band Images of Facial Components Institute for Informatics Technische.

Large-Scale Image Retrieval From Your Sketches Daniel Brooks 1,Loren Lin 2,Yijuan Lu 1 1 Department of Computer Science, Texas State University, TX, USA.

Learning Globally-Consistent Local Distance Functions for Shape-Based Image Retrieval and Classification Computer Vision, ICCV IEEE 11th International.

Presented by Relja Arandjelović The Power of Comparative Reasoning University of Oxford 29 th November 2011 Jay Yagnik, Dennis Strelow, David Ross, Ruei-sung.

Information Retrieval: Human-Computer Interfaces and Information Access Process.

CS395: Visual Recognition Spatial Pyramid Matching Heath Vinicombe The University of Texas at Austin 21 st September 2012.

Query Specific Fusion for Image Retrieval

1 Overview of Image Retrieval Hui-Ying Wang. 2/42 Reference Smeulders, A. W., Worring, M., Santini, S., Gupta, A.,, and Jain, R “Content-based.

Data-driven Visual Similarity for Cross-domain Image Matching

Ziming Zhang *, Ze-Nian Li, Mark Drew School of Computing Science, Simon Fraser University, Vancouver, B.C., Canada {zza27, li, Learning.

SUPER: Towards Real-time Event Recognition in Internet Videos Yu-Gang Jiang School of Computer Science Fudan University Shanghai, China

Data Visualization STAT 890, STAT 442, CM 462

Landmark Classification in Large- scale Image Collections Yunpeng Li David J. Crandall Daniel P. Huttenlocher ICCV 2009.

Effective Image Database Search via Dimensionality Reduction Anders Bjorholm Dahl and Henrik Aanæs IEEE Computer Society Conference on Computer Vision.

Generic Object Recognition -- by Yatharth Saraf A Project on.

Crowdsourcing research data UMBC ebiquity,

Object-based Image Representation Dr. B.S. Manjunath Sitaram Bhagavathy Shawn Newsam Baris Sumengen Vision Research Lab University of California, Santa.

ACM Multimedia th Annual Conference, October , 2004

1 Integrating User Feedback Log into Relevance Feedback by Coupled SVM for Content-Based Image Retrieval 9-April, 2005 Steven C. H. Hoi *, Michael R. Lyu.

Recap: HOGgles Data, Representation, and Learning matter. – This work looked just at representation By creating a human-understandable HoG visualization,

Predicting Matchability - CVPR 2014 Paper -

A String Matching Approach for Visual Retrieval and Classification Mei-Chen Yeh* and Kwang-Ting Cheng Learning-Based Multimedia Lab Department of Electrical.

Computational Analysis of USA Swimming Data Junfu Xu School of Computer Engineering and Science, Shanghai University.

ImageNet: A Large-Scale Hierarchical Image Database

Global and Efficient Self-Similarity for Object Classification and Detection CVPR 2010 Thomas Deselaers and Vittorio Ferrari.

Large-Scale Content-Based Image Retrieval Project Presentation CMPT 880: Large Scale Multimedia Systems and Cloud Computing Under supervision of Dr. Mohamed.

Presenting by, Prashanth B R 1AR08CS035 Dept.Of CSE. AIeMS-Bidadi. Sketch4Match – Content-based Image Retrieval System Using Sketches Under the Guidance.

Wang, Z., et al. Presented by: Kayla Henneman October 27, 2014 WHO IS HERE: LOCATION AWARE FACE RECOGNITION.

Students: Ilya Paskhover, Itay Gal Supervisors: Oleg Rokhlenko, Nadav Golbandi.

Internet-scale Imagery for Graphics and Vision James Hays cs195g Computational Photography Brown University, Spring 2010.

Problem Statement A pair of images or videos in which one is close to the exact duplicate of the other, but different in conditions related to capture,

Semantic Similarity over Gene Ontology for Multi-label Protein Subcellular Localization Shibiao WAN and Man-Wai MAK The Hong Kong Polytechnic University.

Christopher Harris Informatics Program The University of Iowa Workshop on Crowdsourcing for Search and Data Mining (CSDM 2011) Hong Kong, Feb. 9, 2011.

3D Sketch-based 3D Model Retrieval With Kinect Natacha Feola 1, Azeem Ghumman 2, Scott Forster 3, Dr. Yijuan Lu 4 1 Department of Computer Science, University.

Andrew Brasher Andrew Brasher, Patrick McAndrew Userlab, IET, Open University Human-Generated Learning.

Marcin Marszałek, Ivan Laptev, Cordelia Schmid Computer Vision and Pattern Recognition, CVPR Actions in Context.

Learning Models for Object Recognition from Natural Language Descriptions Presenters: Sagardeep Mahapatra – Keerti Korrapati

Why Categorize in Computer Vision ?. Why Use Categories? People love categories!

 Text Representation & Text Classification for Intelligent Information Retrieval Ning Yu School of Library and Information Science Indiana University.

Sketch-Based Shape Retrieval M. Eitz, R. Richter, K. Hildebrand, M. Alexa, TU Berlin; T. Boubekeur, Tele ParisTech – CNRS;

Yao, B., and Fei-fei, L. IEEE Transactions on PAMI(2012)

Crowdsourcing for Spoken Dialogue System Evaluation Ling 575 Spoken Dialog April 30, 2015.

Beauty is Here! Evaluating Aesthetics in Videos Using Multimodal Features and Free Training Data Yanran Wang, Qi Dai, Rui Feng, Yu-Gang Jiang School of.

Online Kinect Handwritten Digit Recognition Based on Dynamic Time Warping and Support Vector Machine Journal of Information & Computational Science, 2015.

Human Computation and Computer Vision CS143 Computer Vision James Hays, Brown University.

Technische Universität München A User Perspective on Typification of Facade Features of 3D Building Models Mathias Jahnke Hongchao Fan Liqiu Meng ICC 2009.

Prof. Thomas Sikora Technische Universität Berlin Communication Systems Group Thursday, 2 April 2009 Integration Activities in “Tools for Tag Generation“

APPLICATIONS OF LIGHT FIELDS IN COMPUTER VISION WEEK 2 REU STUDENT: AMARI LEWIS P.H.D STUDENT: AIDEAN SHARGHI.

INTERACTIVELY BROWSING LARGE IMAGE DATABASES Ronald Richter, Mathias Eitz and Marc Alexa.

Improving Search Results Quality by Customizing Summary Lengths Michael Kaisser ★, Marti Hearst  and John B. Lowe ★ University of Edinburgh,  UC Berkeley,

Gang WangDerek HoiemDavid Forsyth. INTRODUCTION APROACH (implement detail) EXPERIMENTS CONCLUSION.

CoCQA : Co-Training Over Questions and Answers with an Application to Predicting Question Subjectivity Orientation Baoli Li, Yandong Liu, and Eugene Agichtein.

Click to Add Title A Systematic Framework for Sentiment Identification by Modeling User Social Effects Kunpeng Zhang Assistant Professor Department of.

Image Classification over Visual Tree Jianping Fan Dept of Computer Science UNC-Charlotte, NC

26/01/20161Gianluca Demartini Ranking Categories for Faceted Search Gianluca Demartini L3S Research Seminars Hannover, 09 June 2006.

Describing People: A Poselet-Based Approach to Attribute Classification.

SUN Database: Large-scale Scene Recognition from Abbey to Zoo Jianxiong Xiao *James Haysy Krista A. Ehinger Aude Oliva Antonio Torralba Massachusetts Institute.

CENG 789 – Digital Geometry Processing 10- Data-Driven Shape Analysis

The SUN Database Slides by Jennifer Baulier. What is the SUN database? Scene Understanding Database Scene = a place humans could act within Full database.

Machine Learning Lecture 4: Unsupervised Learning (clustering) 1.

A content-based System for Music Recommendation and Visualization of User Preference Working on Semantic Notions Dmitry Bogdanov, Martin Haro, Ferdinand.

1 Bilinear Classifiers for Visual Recognition Computational Vision Lab. University of California Irvine To be presented in NIPS 2009 Hamed Pirsiavash Deva.

Cheng-Ming Huang, Wen-Hung Liao Department of Computer Science

Basic Intro Tutorial on Machine Learning and Data Mining

Attributes and Simile Classifiers for Face Verification

IEEE ICIP Feature Normalization for Part-Based Image Classification

Objects as Attributes for Scene Classification

THE ASSISTIVE SYSTEM SHIFALI KUMAR BISHWO GURUNG JAMES CHOU

Presentation transcript:

How Do Humans Sketch Objects? SIGGRAPH 2012 Mathias Technische Universität Berlin ( 柏林工业 )Technische Universität Berlin James Brown UniversityBrown University Marc Technische Universität BerlinTechnische Universität Berlin Speaker: Xiaohua Xie

Mathias Eitz M.Eng. degree from Shanghai Jiao Tong UniversityShanghai Jiao Tong University PhD student at Technische Universität BerlinTechnische Universität Berlin Research on sketch: -How Do Humans Sketch Objects? SIGGRAPH Sketch-based Shape Retrieval. SIGGRAPH Learning to classify human object sketches. ACM SIG. Talks -Photosketcher: interactive sketch-based image synthesis. IEEE CGA -Sketch-based image retrieval: benchmark and bag-of-features descriptors. IEEE TVCG -An evaluation of descriptors for large-scale image retrieval from sketched feature lines. CG ……..

#1:A large dataset of human object sketches 20,000 sketches of 250 object categories Using Amazon Mechanical Turk (AMT) - paid “Human Intelligence Tasks (HIT)” to work

#2:Human Recognition AMT again Each HIT identifies 4 sketches. Each worker takes no more than 100 HITs. Categories are organized in 3-level hierarchy. Average accuracy: 73.1%. Usually confusions between semantically similar categories (e.g. ‘panda’ and ‘bear’) Easy tasks Difficult tasks

#3:Machine Recognition “ Bag-of-word” + ”SIFT” + “soft SVM” works best! Average accuracy: 56%. Extension to semantic sketch-based retrieval and the recognition of artistic sketches. Semantic search Recognizing artistic sketch

Some middle conclusions Humans tend to follow a coarse-to-fine drawing strategy: -first outlining with longer strokes and then adding detail. People seem to have more than one iconic (representation) for each category of objects.

Conclusions Pay for collecting a sketch dataset. Pay for a study for human recognition on sketches. Study the computer recognition for sketches. Title question: How do humans sketch objects? How? How? How?.....

Open questions How to exploit the temporal order of strokes. How could the computer generate distinctive sketches that are immediately recognizable by humans? How universal sketching and sketch recognition is among humans. (different cultures, ages, genders, artistic expertise, etc.)

Thank you!!