S EGMENTATION FOR H ANDWRITTEN D OCUMENTS Omar Alaql Fab. 20, 2014

Outline Optical Character Recognition (OCR). OCR for the Historical Documents. Text Lines Segmentation Approaches.  Profile Projection.  Hough Transform.  Level Set Method.  Affinity Propagation.  Steerable Directional Technique.

Optical Character Recognition (OCR) The electronic translation of images into machine-editable text.

Optical Character Recognition (OCR) There are four major stages which must be done in any optical characters recognition: 1) Preprocessing. 2)Segmentation. 3)Feature extraction. 4)Recognition.

Optical Character Recognition (OCR) Preprocessing: – Noise reduction. – Binarization or Gray scale image. – Compression in the amount of data to be analyzed.

Segmentation: – The isolation of various writing units, such as paragraphs, sentences, words, or letters. Optical Character Recognition (OCR)

Representation: – Extracts the most relevant information from the text image which helps the recognition stage to recognize the text. – This information is the features of each symbol that is needed to distinguish it from other symbols. Optical Character Recognition (OCR)

Recognition: – Recognition stage is the last and the main decision making stage. – It is a classification process that identifies each unknown symbol and assigns it into a predefined class. – This classification is based on the extracted features which are the output of the previous stage. Optical Character Recognition (OCR)

Historical documents processing is a challenging task for various reasons: 1) Lack of standard alphabets and presence of unknown fonts. 2) Low quality. OCR for the Historical Documents

3) The lack of constraints on page layout.

OCR for the Historical Documents 4) The complexity of handwriting. 5) The variability of skew between the different text-lines and within the same text-line.

6) Spaces between lines are narrow and variable. 7) The existence of small components. 8) Distinguishing noise from text. OCR for the Historical Documents

Text Lines Segmentation Approaches There are many techniques for text lines segmentation:  Profile Projection.  Hough Transform.  Level Set Method.  Affinity Propagation.  Steerable Directional Technique.

Projection Profile Summing pixel values along the horizontal axis for each y value.

Projection Profile Example:  Input image.

Projection Profile Example:  Skew Correction.

Projection Profile Example:  Horizontal Projection.

Projection Profile Example:  Peaks detection

Example:  Positions for segmentation. Projection Profile

Example:  Image for each text line. Projection Profile

For skewed or fluctuating text lines, the image may be divided into vertical strips. Subdivision the page into columns. Determination of the minimal values of the histograms resulting from horizontal projections for all the columns. Drawing horizontal stroke by means of each minimal value inside a column. The link between these strokes allows the separation of two adjacent lines. Projection Profile

Hough Transform The Hough transform is used for locating straight lines in images. Text line is best align matches the black pixels. Any black pixel has an infinite number of lines that could pass through this pixel.

There are two ways to represent the lines : – y = mx + c – x cos θ + y sin θ = ρ  Each line has a unique value (m, c) or (ρ, θ) which is called accumulator.  There is a vote for the accumulator when the line passes through a black pixel.  The text line is the line that has the maximum accumulator. Hough Transform

Level-set Method Instead of directly segmenting on a binary image, it is converted to a probability map, where each element represents the probability of this pixel belonging to a text line.

Level-set Method The probability map is analyzed using the level set method to segment text lines by determining the boundary of neighboring text lines. The zero value for the boundary, automatically grows, merges, and stops to the final text line boundary.

Connected Components Clustering Grouping many connected components in a cluster by using grouping algorithms, each cluster represents a separate text line.

Affinity Propagation The algorithm first estimates local orientation at each primary component of a word to build a sparse similarity graph.  At each point, the region is divided into five regions.  The Breadth-First Search algorithm is applied to find disjoint sets in the similarity graph.  There exist a path from each element to every other element in the set.

Steerable Directional Local Profile Technique One of the connected components based approaches is steerable directional technique. Adaptive local connectivity map (ALCM) is generated using a steerable directional filter.

Steerable Directional Local Profile Technique Firstly, a steerable filter is used to determine foreground intensity along multiple directions at each pixel while generating the ALCM.

Steerable Directional Local Profile Technique The ALCM is then binarized using an adaptive thresholding algorithm to get a rough estimate of the location of the text lines.

This approach has difficulties and limitations when it comes to the binarization of the ALCM images. Especially when text lines in the document are very close to each other. Steerable Directional Technique

To solve the problem: 1) Steerable dynamic directional filter is applied. Angle value is taken instead of the density value. Steerable Directional Technique

2) apply a mode filter to extract each paragraph in the document and its orientation. Steerable Directional Technique

3) a steerable static directional filter is applied. - the direction of the kernel is taken from the paragraph map.

Steerable Directional Technique 4) Thresholding

Horizontal Projection Technique To use Projection Technique: – First : paragraph segmentation.

Horizontal Projection Technique To use Projection Technique: – Second: Skew Correction.

Horizontal Projection Technique To use Projection Technique: – Third: Horizontal Projection.

Horizontal Projection Technique To use Projection Technique: – Fourth: Profile Analysis. There are some drawbacks makes finding he maximum and the minimum in the profile more complicated.  Short line will provide low peak that might be ignored  very narrow lines, or the lines that including many overlapping components will not produce significant peaks

Horizontal Projection Technique To use Projection Technique: – Fourth: Profile Analysis. To solve this problem, the profile should be smoothed.