Handwritten Mathematical Symbol Recognition for Computer Algebra Applications Xiaofang Xie, Stephen M. Watt Dept. of Computer Science, University of Western Ontario ECCAD 2005 Motivation Provide computer algebra applications a nice pen-based interface Attract more users to Maple, Mathematica. Handwriting recognition is the key to PDA Make mathematics easier input and edited

Math Editor Limitations: cannot cover all math symbols Slow comparing with hand writing

Literature Overview Pattern Matching: feature matching, element matching, elastic matching Neural Network Hidden Markov Model On-line vs. Off-line recognition

Literature Overview(cont.)

Recognizer Architecture Data Collection Preprocessing Feature Extraction Hidden M. Model Grouping Vector Quantization Elastic Matching Combined Classifier Context Info

Preprocessing Resampling Deslanting Smoothing Size Normalization Selectively Chop Head / Tail

Feature Extraction Number of Strokes; Pendown; Pen Pressure Position of end points; Distance to initial. Writing Angle; SgnX; End points directions Point Density; Aspect Ratio Number of Cusps Number of Intersections Number of Loops Velocity; Acceleration

Feature Extraction (Cont.)

Design Hidden Markov Model Decompose symbols into basic elements

Design Hidden Markov Model(Cont.) Multiple Path Hidden Markov Model

Allomorph Analysis Variance makes recognition hard Contribute to design Hmm

Combine Classifiers Classifiers complement each other Analyze the recognition results Detect Correlated Errors Develop Combination Scheme

Context Information Solve Ambiguity Build Dictionary

Partial Results ExperimentNo.of PrototypesRecog. Rate(%) P1:T1,2,3, P1,2:T1,2,3, P1,2,3:T1,2,3, P1,2,3,4:T1,2,3, Recog. Results on 227 mathematical symbols. P represents prototype, T represents test data.

Partial Results(Cont.) ExperimentNo.of Prototypes Candidate Prototypes Percentage Pruned(%) Recog. Rate(%) P1:T P1,2:T P1-3:T P1-4:T Using features to prune prototypes.