1. Probabilistic Color-by-Numbers: Suggesting Pattern Colorizations Using Factor Graphs Sharon Lin, Daniel Ritchie, Matthew Fisher, Pat Hanrahan

2. Colored Patterns Are Everywhere Flickr: Rowena of the Rants

3. Coloring Patterns Can Be Challenging Hard to mentally visualize coloring Template by COLOURLover Any Palacios

4. Coloring Patterns Can Be Challenging Difficult to explore other options Such as:

5. Output Suggested Colorings Suggest Pattern Colorizations to Facilitate the Process ? ? User preferences Input Template

6. Output Suggested Colorings Suggest Pattern Colorizations to Facilitate the Process ? ? User preferences Input Template Suggest diverse colorings Allow refinement Accommodate stylistic preferences Suggest diverse colorings Allow refinement Accommodate stylistic preferences

7. Pattern Template Anatomy Color Groups Colored Template COLOURLovers Nickity Split & ivy21

8. Related Work: Color Compatibility What combinations of colors do people find appealing? ( Goethe 1810; Itten 1974; Matsuda 1995; Cohen-Or et Al 2006)

9. Related Work: Color Compatibility What combinations of colors do people find appealing? Low compatibilityHigh compatibility (ODonovan et al. 2011)

10. Color Compatibility for Patterns Need to take into account 2D arrangement 3.75 3.743.70 3.67 Template by COLOURLover jilbert loud backgroundleaves blending into background What about personal preferences?

11. Look at Examples for Guidance COLOURLovers AlineDam, Any Palacios, wondercake, bhsav

12. Example-Based Color Suggestion Model Suggeste r (optional) user constraints Input Template Output Suggested Colorings Examples COLOURLovers AlineDam, Any Palacios, wondercake, bhsav …

13. Can Change Style Based on Examples Model Suggeste r (optional) user constraints Input Template Output Suggested Colorings Examples … COLOURLovers AlineDam, Any Palacios, praxicalidocious, bhsav

14. Dataset: COLOURLovers Many patterns available: Collected 8200 from 82 artists For our tests: Trained on up to 913 patterns

15. MODEL

16. Scoring a Coloring Unary Factors

17. Scoring a Coloring Goo d

18. Scoring a Coloring Poor ? ? ? ? ? ?

22. PairwiseFactors

24. Color Theme: Global Color Compatibility [ODonovan et al. 2011]

25. Scoring a Coloring Color Theme: Global Color Compatibility [ODonovan et al. 2011]

26. Scoring a Coloring Color Theme:

27. Scoring a Coloring

28. Modeling Unary Color Factors Property of Regions Color Features of Regions Shape … Lightness Saturation Name Saliency [Heer & Stone 2012]

29. Modeling Unary Color Factors Property of Regions Color Features of Regions Shape … Size Elongation Centrality

30. Learning Factor Distributions Predictor 0.2 0.7 0.4 = Size Elongation Centrality … 0.8 0.92 Learner Saturation Distribution =

31. Learning Factor Distributions 01 Lightness

32. Learning Factor Distributions 0 0 9 9 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0.4 0.1 0.5 0.1 0.2 0.1 0.4 0.2 0.1 0.9 0.7 … Lightness

33. Learning Factor Distributions 0 0 9 9 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 0.4 0.1 0.5 0.1 0.2 0.1 0.4 0.2 0.1 0.9 0.7 … Lightness

34. Learning Factor Distributions 0 0 9 9 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 1 Lightness 0.4 0.1 0.5 0.2 0.1 0.4 0.2 0.1 0.9 0.7 …

35. … Learning Factor Distributions 0 0 9 9 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 1 Lightness 0.4 0.1 0.5 0.2 0.1 0.4 0.2 0.1 0.9 0.7

36. Learning Factor Distributions 0 0 9 9 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 2 Lightness … 1 0.4 0.1 0.5 0.2 0.1 0.4 0.1 0.9 0.7

37. Learning Factor Distributions 0 0 9 9 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 Lightness … 0.1 0.9 0.7 21 0.4 0.1 0.5 0.2 0.1 0.4

38. Learning Factor Distributions 0 0 9 9 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 7 Lightness 0.1 0.9 21 0.4 0.1 0.5 0.2 0.1 0.4 …

39. Learning Factor Distributions 0 0 9 9 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 Classifier 0.2 0.7 0.4 ? 0 1 Lightness 7 … 0.1 0.9 21 0.4 0.1 0.5 0.2 0.1 0.4

40. Learning Factor Distributions 0 1 0 1 Lightness Classifier 0.2 0.7 0.4 ? 7 0.1 0.9 21 0.4 0.1 0.5 0.2 0.1 0.4 …

41. Learning Factor Distributions 0 1 1 0 1 Lightness Classifier 0.2 0.7 0.4 ? 7 0.1 0.9 21 0.4 0.1 0.5 0.2 0.1 0.4 … [Charpiat et al. 2008]

42. Example Learned Factors

43. Scoring a Coloring (Revisited) Unary Factors PairwiseFactors Global Color Compatibility [ODonovan et al. 2011]

44. Scoring a Coloring (Revisited) Score = Product of Factors (Factor Graph)

45. Generating Coloring Suggestions Metropolis Hastings (MH) Parallel Tempering Maximum Marginal Relevance REJEC T ACCEPT

46. RESULTS

47. Exploratory Suggestions

48. Refinement: Nearby Colorings

49. Refinement: Hard Constraints Unconstrained Flower Stem Color =

50. Style Simulation Light DarkBoldMellow

51. Application: Web Design

52. Application: Fashion Design

53. EVALUATION

55. 4 x Uniform Random 4 x Color Compatibility Only 4 x Full Model 4 x Hand-Colored 4 x Uniform Random 4 x Color Compatibility Only 4 x Full Model 4 x Hand-Colored

58. Better Than Other Automatic Methods (but not hand-colored patterns)

59. People Make Bad Colorings Just as Often

60. FUTURE WORK

61. Limitation: Semantics sky

62. Limitation: Known Color Groups ??

63. Integration into Interactive Tools

64. Looking Forward

65. THANKS! Support for this research provided by: Intel (ISTC-VC) SAP (Stanford Graduate Fellowship) Support for this research provided by: Intel (ISTC-VC) SAP (Stanford Graduate Fellowship)