1. Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory

2. Metin Sezgin Outline Motivation and problem statement Feature point detection -Curvature and speed as information sources -Average based filtering; Scale space filtering -Hybrid fit generation -Handling strokes with curves Evaluation Related work Future work

3. Metin Sezgin Terminology Sketch: Informal, messy diagrams consisting of several strokes Stroke: Array of timestamped points Feature Point: Corners of a stroke (vertices)

4. Metin Sezgin Task Given a freehand stroke, generate a description of the stroke in terms of geometric primitives such as circles, lines, and curves

5. Metin Sezgin Lifecycle of a stroke Data Collection Data Collection Vertex Detection Vertex Detection Polyline Fit Polyline Fit Oval Fit Curve Detection Curve Detection Line Fit Line Fit Complex Fit (lines and curves) Complex Fit (lines and curves) Best Fit Selection Best Fit Selection Output

6. Metin Sezgin Lifecycle of a stroke Data Collection Data Collection Vertex Detection Vertex Detection Polyline Fit Polyline Fit Oval Fit Curve Detection Curve Detection Line Fit Line Fit Complex Fit (lines and curves) Complex Fit (lines and curves) Best Fit Selection Best Fit Selection Output

7. Metin Sezgin Collecting Strokes

8. Metin Sezgin Vertex detection

9. Metin Sezgin Vertex detection Curvature

10. Metin Sezgin Vertex detection Curvature Speed

11. Metin Sezgin Problems with local extrema Input is noisy -Digitization noise -Hand jittering (tremor) Picking all local extrema introduces many false positives Picking the global extremum gives us a single point Picking the n-best extrema doesn’t help either

12. Metin Sezgin Solutions Average-based filtering -Compute threshold based on data average -Partition using this threshold -Select global extremum in each partition

13. Metin Sezgin Average-based filtering Curvature Speed

14. Metin Sezgin Average-based filtering Pros -Reliable (when data isn’t too noisy) -Fast: O(n) -False positives easy to filter out Cons -Still false positives -Need pre-set constant for scaling

15. Metin Sezgin Scale-space filtering Real world data has features at many scales Represent the data multiple different scales Selecting scale(s) at which data is meaningful

16. Metin Sezgin Deriving the scale space Convolve the original signal with Gaussian signals of increasing width Feature points in the curvature scale space

17. Metin Sezgin Selecting a scale

18. Metin Sezgin Selecting a scale

19. Metin Sezgin Scale selected by the algorithm The original stroke Output generated at the scale chosen by our algorithm

20. Metin Sezgin Handling extremely noisy strokes (cont.) Curvature fit using SSF, 9 fpsSpeed fit using SSF, 7 fps

21. Metin Sezgin Combining information sources Individual detection methods may miss points Speed fit missing a point Curvature fit missing a point Need a way to combine information provided by each method Generate a series of hybrid fits and pick the best

22. Metin Sezgin Hybrid fit generation Given N fits generated using N different methods -Within each fit, compute vertex certainties and sort by certainty Initial fit contains points common to all methods Generate N new fits by adding best candidate from each method. Select the one of these with the least error Terminate when error smaller than preset error bound

23. Metin Sezgin Data Collection Data Collection Vertex Detection Vertex Detection Polyline Fit Polyline Fit Oval Fit Curve Detection Curve Detection Line Fit Line Fit Complex Fit (lines and curves) Complex Fit (lines and curves) Best Fit Selection Best Fit Selection Output

24. Metin Sezgin Curve approximation c2c2 c1c1 u v

25. Metin Sezgin Results

26. Metin Sezgin Results

27. Metin Sezgin User study results All users except one perceived the system as more natural. Overall it took users much less time to specify geometries. All subjects except one indicated they would prefer an interface that feels like our system for specifying shapes in a sketchy fashion. Strokes classified correctly 96% of the time

28. Metin Sezgin Related work Existing systems lack -Support for arbitrary shape creation with a single stroke -Automatic feature point detection -Modeless drawing -Natural feel

29. Metin Sezgin Future work Integration with the new sketch recognition architecture Support for over-tracing Further investigation of sketching behavior to enhance/aid feature point detection and stroke classification