Presentation on theme: "Document Processing Methods for Telugu and other SE Asian Scripts Authors: Atul Negi, VSR Sowri, K Mohan Rao Presented by: Atul Negi, Dept of CIS, University."— Presentation transcript:
Document Processing Methods for Telugu and other SE Asian Scripts Authors: Atul Negi, VSR Sowri, K Mohan Rao Presented by: Atul Negi, Dept of CIS, University of Hyderabad
SE Asian Scripts n Complex arrangement of connected components n Problems –difficulty in identifying the words and text line boundaries –touching characters n Nature of scripts: consonants with vowels and large number of distinct symbols
SE Asian scripts-Contd. n SE Asian scripts such as Telugu, Kannada, Simhala are rounded in nature. n We base our work on Telugu Script which is orthographically similar to many SE Asian scripts.
Acchulu Vowel Sound Symbols (16) Hallulu Consonant Sound Symbols (38) Maatra Vowel Sound Modifying Symbols for Hallulu (16) Voththulu Core Consonant Sound Symbols About Telugu Script Consists of Rounded Shapes (no vertical strokes) Characters may be basic vowel/consonant shapes or could be composed by compounding shapes ([NCK 01] shows examples) Example above shows glyphs in bounded boxes in a word pronounced as “Maa-tru-gee-ta”
Some Features of Telugu Script n Telugu is a phonetic script with each character representing a spoken syllable. n Contains curved letters with no vertical linear strokes and shirorekha (head line). n 16 Vowels, 36 consonants, Telugu OCR system [NCK 01] reduced possible 10,000 symbols to about 400 glyphs n Glyph represents a single connected component, but is NOT a character
More Features of Telugu Script n Orthography is compositional with vowel sound symbols (matraas) modifying basic consonants. n Pure consonants sounds can be symbolized as vottus and can be combined with other consonant/vowel modified consonant symbols. n A character is made from a combination of the above n Vottus and matraas can be positioned at locations surrounding the base character
OCR Efforts in Telugu n [RD 77] Rajasekharan and Deekshatulu 1977 n [SSP 95] Sukhswami, Seetharamulu, and Pujari 1995 n [NCK 01] Negi, Chakravarthy and Krishna 2001 n [NCS 02] Negi, Chakravarthy and Suresh Kumar 2002 n [ P 02] Pujari et al 2002 n [C R M N] Chakravarthy et al n [VP 02] Vasantha and Patvardhan 2002 n [ NKC 03] Negi, Kasinadhuni, Chandrakanth 2003 Brief Review Recognition Approaches
Focus on Text Line and Character Segmentation Issues n In this presentation our contribution is focussed on –Text line Extraction: By clustering of connected components based upon their spatial properties. –Character segmentation- Drop Fall method and White stream method
Text Line Segmentation
Motivation Text-line and text column extraction are crucial in PLA (Text Line Segmentation) Affects the word and character level analysis. Helps in logical grouping of individual glyphs into characters. Simplifies the determination of logical sequence of characters. Can be used to reduce the search space of OCR.
Overview of Text Line Segmentation Approaches n Approach as shown in [NKC 03] very complex, high time complexity n Pixel Projection Profile Approaches –Simpler, but do not work well with complex layouts and overlapping lines, or presence of skew n Bounding Box Projection Approaches –More efficient, work well in certain conditions –Limitations due to unevenness of white spacing n Bounding Box Co-ocurrances (this work)
Text Line Segmentation Using BB Projections n Heuristics Based on BB Projections –Concept is to extract adjacent zero BB count scan lines between BB peak lines –White space in between text-lines is broken, uneven and not contiguous because of the vottus and maatras in between text lines. –Touching characters from adjacent text-lines –More heuristics to improve results by estimating interfering characters from BB projections but results are not very good due to difficulty of estimation
Co-occurrence “A measure of OVERLAP between different connected components.” It is based on the spatial relationships of connected components. It’s symmetric in nature. Two types: Horizontal co-occurrence Vertical co-occurrence n Co-occurrence defines 3 different spatial relationships between components.
Horizontal Co-occurrence n Total Inclusion
Horizontal Co-occurrence n Partial-inclusion
Horizontal Co-occurrence n No Relation
Text-line extraction using co- occurrence Text-line extraction problem is formulated as: Identifying all the connected components which belong to the same text-line and obtaining the boundaries of text-lines by considering the bounding boxes of components. Two major steps: Computation of horizontal co-occurrence matrix for each pair of components. Clustering of connected components based on the h- cooccurrence matrix.
Text-line extraction - Clustering Let P,Q be two CC in the document image. P
Text-line extraction - Results
Experimental Results – Hand Written Document Image
Experimental Results – Kannada Document Image
Experimental Results – Tamil Document Image
n Is an operation that seeks to decompose an image of sequence of characters into sub images of individual symbols.
Character Segmentation methods
How can we segment characters? Successful segmentation mainly involves two steps: 1.Locating a segmentation point 2.Generating a segmentation path Drop Fall Methods attempt to do both
Hybrid Drop Fall Method n Segments the characters by following the contour of the image. n Advanced version of Hit and Deflect strategy. n Follows a set of rules that maximizes the chances that it will hit and deflect its way to an accurate path.
Drop Fall builds a path by mimicking an object falling or rolling in between the two characters There are 8 varieties of Drop Fall methods which differ in directions, starting points and set of rules. Path generated by a drop fall can be seen in fig given below
Locating the segmentation point n Pixels are scanned row-by-row until a black boundary pixel with another black boundary pixel to the right of it is detected, where the two pixels are seperated by atleast one white space. n This white pixel is then used as the starting point from which the marble is rolled down
Starting point for Drop fall
Incorrect segmentation of touching characters can be seen in the figure shown below. Incorrect starting points leads to incorrect segmentation path.
Drop Fall Path Generation n The algorithm first looks out for a white pixel in its surroundings and if unable to find a white pixel then only cuts through the black pixel. n The directions that the algorithm will move in according to the present pixel positions and its surroundings is shown below
Top Left Drop Fall n Input: Image n 1.Binarize the input image n 2.Locate the Segmentation point (x, y) using drop fall n 3. Generate the segmentation path using the rules specified in the previous slide. n Output: Segmented Image
Characters segmented using top left Drop Fall: (standard drop fall)
Top left fails to segment the touching characters when the first character contains a Talakattu or is of concave shape. Eg :Incorrect segmentation of Touching characters using Top left drop fall
Top Right Drop Fall n Identical to Top left drop fall except that it initiates from the top-right of the connected component. n Input: Touching character Image n Binarise the input image n Flip the image vertically n Locate the segmentation point n Generate the segmentation path n Re-flip the Image and obtain the segmented image.
Top Right Drop Fall
Bottom Left Drop Fall n Identical to standard drop fall except that it initiates from the bottom left drop fall n Input:Touching characters n Binarise the input image n Flip the image horizontally n Locate the segmentation point n Generate the segmentation path n Re-flip the Image horizontally and obtain the segmented image.
Bottom Left Drop Fall Method Touching Characters segmented using Bottom left drop fall
Bottom left Drop Fall n Fails to segment the touching characters when the bottom half of the first character consists of curves or grooves
Bottom right drop fall Method
Characters segmented using Bottom right drop fall Cases where Bottom right drop fall fails to segment the touching characters
Advanced Drop fall methods n Similar to Drop fall method in locating the segmentation point but while generating the segmentation path follows different set of rules. n While generating the segmentation path it will be look out for white pixels and when unable to find a while pixel it will move for black pixels and when it is on black pixels it will only look for black pixels.
Difference between drop fall and Advanced drop fall segmentation paths
Advanced Top left Drop Fall n Characters using Advanced top left drop fall Incorrectly segmented characters using Advanced top left drop fall
Advanced Top right drop fall n Identical to Top right drop fall except the segmentation path generated is different. n Characters segmented using Advanced top right drop fall
Advanced bottom left drop fall n Characters segmented using Advanced bottom left drop fall Incorrectly segmented characters using Advanced bottom left drop fall
Advanced bottom right drop fall (ABRD) Characters segmented using ABRD Incorrect segmented characters using ABRD
White Stream Method n Used for identifying correct segmentation point n Input: Touching characters n 1.Generate the contour n 2Generate a bounding box n 3.Count the number of white pixels in each column until a black pixel is encountered starting from the bottom of image n 4.find column C with maximum count of white pixel. n 5.output: Segmented characters
White stream Drop Fall Segmentation n In this method depending on the aspect ratios the segmentation is done. Horizontally touching characters are segmented using white stream method for locating the segmentation point and drop fall method to generate the segmentation path n Vertical touching characters are segmented using the column anding and projection profile
White stream drop fall segmentation n Characters segmented using white stream DF
Objective Function Drop Fall Segmentation (OFDS) n Is a Hybrid method (Column Anding + DF) n Input: Touching characters n Calculate the aspect ratio of C.C n Perform Column anding and find the column to be segmented n Generate the segmentation path using DF
OFDS n Characters segmented using OFDS
Future work n Segmenting characters which consists of double and triple touchings. n Find the best path among various drop fall segmentation paths. n Finding the cavities and their positioning in order to segment the touching characters.