Probabilistic Color-by-Numbers: Suggesting Pattern Colorizations Using Factor Graphs Sharon Lin, Daniel Ritchie, Matthew Fisher, Pat Hanrahan.

Slides:

Advertisements

Similar presentations

Basic Steps 1.Compute the x and y image derivatives 2.Classify each derivative as being caused by either shading or a reflectance change 3.Set derivatives.

Advertisements

Data Mining and the Web Susan Dumais Microsoft Research KDD97 Panel - Aug 17, 1997.

Designing Services for Grid-based Knowledge Discovery A. Congiusta, A. Pugliese, Domenico Talia, P. Trunfio DEIS University of Calabria ITALY

Machine Learning for Vision-Based Motion Analysis Learning pullback metrics for linear models Oxford Brookes Vision Group Oxford Brookes University 17/10/2008.

Pseudo-Relevance Feedback For Multimedia Retrieval By Rong Yan, Alexander G. and Rong Jin Mwangi S. Kariuki

Animosaics Kaleigh Smith, Yunjun Liu, and Allison Klein McGill University Eurographics Symposium on Computer Animation 2005.

The Layout Consistent Random Field for detecting and segmenting occluded objects CVPR, June 2006 John Winn Jamie Shotton.

Ying Cao Antoni B. ChanRynson W.H. Lau City University of Hong Kong.

Introduction Distance-based Adaptable Similarity Search

1 Autocompletion for Mashups Ohad Greenshpan, Tova Milo, Neoklis Polyzotis Tel-Aviv University UCSC.

Split Questionnaire Designs for Consumer Expenditure Survey Trivellore Raghunathan (Raghu) University of Michigan BLS Workshop December 8-9, 2010.

From Interactive to Semantic Image Segmentation Varun Gulshan Supervisors: Prof. Andrew Blake Prof. Andrew Zisserman 20 Jan 2012.

Color Compatibility From Large Datasets Peter O’Donovan University of Toronto Aseem Agarwala Adobe Systems, Inc. Aaron Hertzmann University of Toronto.

Training Manual Aug Probabilistic Design: Bringing FEA closer to REALITY! 2.5 Probabilistic Design Exploring randomness and scatter.

Qualifying Exam: Contour Grouping Vida Movahedi Supervisor: James Elder Supervisory Committee: Minas Spetsakis, Jeff Edmonds York University Summer 2009.

A Probabilistic Model for Component-Based Shape Synthesis Evangelos Kalogerakis, Siddhartha Chaudhuri, Daphne Koller, Vladlen Koltun Stanford University.

Automatic in vivo Microscopy Video Mining for Leukocytes * Chengcui Zhang, Wei-Bang Chen, Lin Yang, Xin Chen, John K. Johnstone.

Department of Mechanical Engineering 1 Meshing of 2-D Cross Section Mesh Tool Comparison Group: Graden Hardy, Patrick Lewis Date:22 April 2009 ME 501 –

Relevance Feedback Content-Based Image Retrieval Using Query Distribution Estimation Based on Maximum Entropy Principle Irwin King and Zhong Jin Nov

Search and Retrieval: More on Term Weighting and Document Ranking Prof. Marti Hearst SIMS 202, Lecture 22.

Background Removal of Multiview Images by Learning Shape Priors Yu-Pao Tsai, Cheng-Hung Ko, Yi-Ping Hung, and Zen-Chung Shih.

Berenzweig - Music Recommendation1 Music Recommendation Systems: A Progress Report Adam Berenzweig April 19, 2002.

Automatic Image Annotation and Retrieval using Cross-Media Relevance Models J. Jeon, V. Lavrenko and R. Manmathat Computer Science Department University.

Self-Supervised Segmentation of River Scenes Supreeth Achar *, Bharath Sankaran ‡, Stephen Nuske *, Sebastian Scherer *, Sanjiv Singh * * ‡

Recognition Of Textual Signs Final Project for “Probabilistic Graphics Models” Submitted by: Ezra Hoch, Golan Pundak, Yonatan Amit.

Evaluating the Quality of Image Synthesis and Analysis Techniques Matthew O. Ward Computer Science Department Worcester Polytechnic Institute.

Relevance Feedback Content-Based Image Retrieval Using Query Distribution Estimation Based on Maximum Entropy Principle Irwin King and Zhong Jin The Chinese.

Using Image Priors in Maximum Margin Classifiers Tali Brayer Margarita Osadchy Daniel Keren.

RL via Practice and Critique Advice Kshitij Judah, Saikat Roy, Alan Fern and Tom Dietterich PROBLEM: RL takes a long time to learn a good policy. Teacher.

Creating and Exploring a Large Photorealistic Virtual Space INRIA / CSAIL / Adobe First IEEE Workshop on Internet Vision, associated with CVPR 2008.

Educational Data Mining and DataShop John Stamper Carnegie Mellon University 1 9/12/2012 PSLC Corporate Partner Meeting 2012.

Midterm Presentation Undergraduate Researchers: Graduate Student Mentor: Faculty Mentor: Jordan Cowart, Katie Allmeroth Krist Culmer Dr. Wenjun (Kevin)

Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC Jeany Son.

LÊ QU Ố C HUY ID: QLU OUTLINE  What is data mining ?  Major issues in data mining 2.

The Three R’s of Vision Jitendra Malik.

Interactive Discovery and Semantic Labeling of Patterns in Spatial Data Thomas Funkhouser, Adam Finkelstein, David Blei, and Christiane Fellbaum Princeton.

Improving Web Search Ranking by Incorporating User Behavior Information Eugene Agichtein Eric Brill Susan Dumais Microsoft Research.

Learning Models for Object Recognition from Natural Language Descriptions Presenters: Sagardeep Mahapatra – Keerti Korrapati

Universit at Dortmund, LS VIII

Style Compatibility For 3D Furniture Models Tianqiang Liu 1 Aaron Hertzmann 2 Wilmot Li 2 Thomas Funkhouser 1 1 Princeton University 2 Adobe Research.

Astro / Geo / Eco - Sciences Illustrative examples of success stories: Sloan digital sky survey: data portal for astronomy data, 1M+ users and nearly 1B.

Attention-Directing Composition of Ying Cao Rynson W.H. Lau Elements Look Over Here: Antoni B. Chan City University of Hong Kong.

Visual Abstraction and Exploration of Multi-class Scatterplots Haidong Chen, Wei Chen, Honghui Mei, Zhiqi Liu, Kun Zhou, Weifeng Chen, Wentao Gu, Kwan-Liu.

Module networks Sushmita Roy BMI/CS 576 Nov 18 th & 20th, 2014.

Nature Reviews/2012. Next-Generation Sequencing (NGS): Data Generation NGS will generate more broadly applicable data for various novel functional assays.

C OMPUTING E SSENTIALS Timothy J. O’Leary Linda I. O’Leary Presentations by: Fred Bounds.

Look Over Here: Attention-Directing Composition of Manga Elements Ying Cao Rynson W.H. Lau Antoni B. Chan SIGGRAPH

VisDB: Database Exploration Using Multidimensional Visualization Maithili Narasimha 4/24/2001.

Image Classification for Automatic Annotation

Digital Video Library Network Supervisor: Prof. Michael Lyu Student: Ma Chak Kei, Jacky.

ASSOCIATIVE BROWSING Evaluating 1 Jinyoung Kim / W. Bruce Croft / David Smith for Personal Information.

Evaluation of DBMiner By: Shu LIN Calin ANTON. Outline  Importing and managing data source  Data mining modules Summarizer Associator Classifier Predictor.

Unsupervised Relation Detection using Automatic Alignment of Query Patterns extracted from Knowledge Graphs and Query Click Logs Panupong PasupatDilek.

Automatic Labeling of Multinomial Topic Models

POSTER TEMPLATE BY: Background Objectives Psychophysical Experiment Smoothness Features Project Pipeline and outlines The purpose.

Change Blindness Images Li-Qian Ma 1, Kun Xu 1, Tien-Tsin Wong 2, Bi-Ye Jiang 1, Shi-Min Hu 1 1 Tsinghua University 2 The Chinese University of Hong Kong.

Selective Sampling for Information Extraction with a Committee of Classifiers Evaluating Machine Learning for Information Extraction, Track 2 Ben Hachey,

Learning Photographic Global Tonal Adjustment with a Database of Input / Output Image Pairs.

A Scalable Machine Learning Approach to Go Pierre Baldi and Lin Wu UC Irvine.

Learning Hierarchical Features for Scene Labeling Cle’ment Farabet, Camille Couprie, Laurent Najman, and Yann LeCun by Dong Nie.

Software Testing By Souvik Roy. What is Software Testing? Executing software in a simulated or real environment, using inputs selected somehow.

Shape2Pose: Human Centric Shape Analysis CMPT888 Vladimir G. Kim Siddhartha Chaudhuri Leonidas Guibas Thomas Funkhouser Stanford University Princeton University.

Structure-Aware Shape Processing Part III Example Applications Martin Bokeloh Niloy Mitra Michael Wand Hao Zhang Daniel Cohen-Or Martin Bokeloh.

Dieudo Mulamba November 2017

Project Implementation for ITCS4122

Cheng-Ming Huang, Wen-Hung Liao Department of Computer Science

Designing Search for Humans

By Charlie Fractal Mentor: Dr. Vignesh Subbian

Automated Software Integration

François Guimbretière

Presentation transcript:

Probabilistic Color-by-Numbers: Suggesting Pattern Colorizations Using Factor Graphs Sharon Lin, Daniel Ritchie, Matthew Fisher, Pat Hanrahan

Colored Patterns Are Everywhere Flickr: Rowena of the Rants

Coloring Patterns Can Be Challenging Hard to mentally visualize coloring Template by COLOURLover Any Palacios

Coloring Patterns Can Be Challenging Difficult to explore other options Such as:

Output Suggested Colorings Suggest Pattern Colorizations to Facilitate the Process ? ? User preferences Input Template

Output Suggested Colorings Suggest Pattern Colorizations to Facilitate the Process ? ? User preferences Input Template Suggest diverse colorings Allow refinement Accommodate stylistic preferences Suggest diverse colorings Allow refinement Accommodate stylistic preferences

Pattern Template Anatomy Color Groups Colored Template COLOURLovers Nickity Split & ivy21

Related Work: Color Compatibility What combinations of colors do people find appealing? ( Goethe 1810; Itten 1974; Matsuda 1995; Cohen-Or et Al 2006)

Related Work: Color Compatibility What combinations of colors do people find appealing? Low compatibilityHigh compatibility (ODonovan et al. 2011)

Color Compatibility for Patterns Need to take into account 2D arrangement Template by COLOURLover jilbert loud backgroundleaves blending into background What about personal preferences?

Look at Examples for Guidance COLOURLovers AlineDam, Any Palacios, wondercake, bhsav

Example-Based Color Suggestion Model Suggeste r (optional) user constraints Input Template Output Suggested Colorings Examples COLOURLovers AlineDam, Any Palacios, wondercake, bhsav …

Can Change Style Based on Examples Model Suggeste r (optional) user constraints Input Template Output Suggested Colorings Examples … COLOURLovers AlineDam, Any Palacios, praxicalidocious, bhsav

Dataset: COLOURLovers Many patterns available: Collected 8200 from 82 artists For our tests: Trained on up to 913 patterns

MODEL

Scoring a Coloring Unary Factors

Scoring a Coloring Goo d

Scoring a Coloring Poor ? ? ? ? ? ?

PairwiseFactors

Color Theme: Global Color Compatibility [ODonovan et al. 2011]

Scoring a Coloring Color Theme: Global Color Compatibility [ODonovan et al. 2011]

Scoring a Coloring Color Theme:

Scoring a Coloring

Modeling Unary Color Factors Property of Regions Color Features of Regions Shape … Lightness Saturation Name Saliency [Heer & Stone 2012]

Modeling Unary Color Factors Property of Regions Color Features of Regions Shape … Size Elongation Centrality

Learning Factor Distributions Predictor = Size Elongation Centrality … Learner Saturation Distribution =

Learning Factor Distributions 01 Lightness

Learning Factor Distributions … Lightness

Learning Factor Distributions … Lightness

Learning Factor Distributions Lightness …

… Learning Factor Distributions Lightness

Learning Factor Distributions Lightness …

Learning Factor Distributions Lightness …

Learning Factor Distributions Lightness …

Learning Factor Distributions Classifier ? 0 1 Lightness 7 …

Learning Factor Distributions Lightness Classifier ? …

Learning Factor Distributions Lightness Classifier ? … [Charpiat et al. 2008]

Example Learned Factors

Scoring a Coloring (Revisited) Unary Factors PairwiseFactors Global Color Compatibility [ODonovan et al. 2011]

Scoring a Coloring (Revisited) Score = Product of Factors (Factor Graph)

Generating Coloring Suggestions Metropolis Hastings (MH) Parallel Tempering Maximum Marginal Relevance REJEC T ACCEPT

RESULTS

Exploratory Suggestions

Refinement: Nearby Colorings

Refinement: Hard Constraints Unconstrained Flower Stem Color =

Style Simulation Light DarkBoldMellow

Application: Web Design

Application: Fashion Design

EVALUATION

4 x Uniform Random 4 x Color Compatibility Only 4 x Full Model 4 x Hand-Colored 4 x Uniform Random 4 x Color Compatibility Only 4 x Full Model 4 x Hand-Colored

Better Than Other Automatic Methods (but not hand-colored patterns)

People Make Bad Colorings Just as Often

FUTURE WORK

Limitation: Semantics sky

Limitation: Known Color Groups ??

Integration into Interactive Tools

Looking Forward

THANKS! Support for this research provided by: Intel (ISTC-VC) SAP (Stanford Graduate Fellowship) Support for this research provided by: Intel (ISTC-VC) SAP (Stanford Graduate Fellowship)