Oriented Local Binary Patterns for Offline Writer Identification Handwriting as oriented texture Anguelos Nicolaou, Marcus Liwicki, Rolf Ingolf DIVA Group Fribourg Switzerland

Offline writer identification sample Image to feature vector Vector should “uniquely” identify the handwriting style Measure similarity between handwriting styles Identify writer as the most similar LBP LBP LBP LBP DB Features LBP LBP LBP LBP LBP All image samples in this presentation are taken from Icfhr 2012 competition on writer identification challenge

Local Binary Patterns Introduced by Ojala et al. Each pixel has a circular neighborhood Encode the binary relationship of each pixel in the neighborhood borders Each pixel’s neighborhood is encoded as an integer Bag of words approach Total disregard towards structure between individual patterns

LBP parameters Radius: Sample count: Binary operator: Texture Information Structure Smaller Larger Radius Frequent events Low detail High detail Scarce events Less samples More Samples Sample count

Proposed feature set: LBP tuning 8 samples in the periphery 3 pixel radius Binary function: center equals periphery Histogram of all occurring patterns except of “all white or all black” pattern normalized to a sum of 1

Proposed feature-set: High-order features Derived from the LBP histogram Histogram of rotation invariant hashes Histogram of rotation phases Histogram of border orientations Histogram of border aggregations Histogram of true bit count Histogram of 0-1 transitions (beta function) LBP Histogram High-order Features 255 72 327

Transforming the feature space: Pipeline Overview Feature extraction Only Training Raw Features Train set Raw Features PCA Component Matrix Rebase to components Rebased Features Train set Labeled Features Evolutionary Algorithm Scale Matrix Scale to optimal Scaled Features NN Classification

Transforming the feature space: PCA Changing the basis of the feature space Preserving 99.9% of the information Reducing 327 to 125 dimensions Should be perceived as training With very good generalizing properties

Transforming the feature space: Scale vector optimization Scale each dimension Evolutionary optimization Fitness: minimize Fitness inspired by LDA 2000 Generations 50 individuals per population

Performance measurement Train-set: ICFHR 2012 (100 writers x 4 samples) “Icfhr 2012 com- petition on writer identification challenge” Test-set: ICDAR 2011 (26 writers x 8 samples) “Icdar 2011 writer identification contest” NN classification: Method Soft 10 Hard 7 NN accuracy Tsinghua (2011 winner) 100% 44.1% 99.5% Untrained 99.04% 28.84% 96.63% Trained 50.48% 98.56%

Qualitative experiments: Rotation robustness Rotation correction destroys slant information Tolerance +- 5 degrees Conclusion: Quite robust to rotation -20o -5o 0o

Qualitative experiments: Robustness against scale Typically, methods are sensitive to scale NN classification Dataset: ICHFR 2012 Intolerant of comparing samples at different scales from the db Discriminative ability preserved when rescaling samples and db

Qualitative experiments: Signal quantity requirements NN classification Dataset: 2011 ICDAR Gradually removed connected components from top to bottom For the specific dataset, the method requires 3-4 text-lines. 25% 100%

Qualitative Experiments: Writer vs Writing Style SigWiComp2013 train set 55 writers 3 samples Performance of winner 28% SigWiComp2013. Our system 19.4% Split in left and right halves Task: assign left half to right half Assigning correctly two pieces of the same sample: 86.06% Assigning correctly two pieces of the same writer: 87.27% Conclusion: Our method practically treats different writing styles as different writers

Conclusions: Powerful feature set Good behavior in trained and untrained modalities Perceives handwriting as texture Reasonably robust against geometric distortions As is, probably unfit for biometric applications

Use cases of LBP feature set Thank you for your attention