Presentation on theme: "Document Processing Methods for Telugu and other SE Asian Scripts"— Presentation transcript:
1Document Processing Methods for Telugu and other SE Asian Scripts Authors: Atul Negi, VSR Sowri, K Mohan RaoPresented by:Atul Negi, Dept of CIS, University of Hyderabad
2SE Asian Scripts Complex arrangement of connected components Problems difficulty in identifying the words and text line boundariestouching charactersNature of scripts: consonants with vowels and large number of distinct symbols
3SE Asian scripts-Contd. SE Asian scripts such as Telugu, Kannada, Simhala are rounded in nature.We base our work on Telugu Script which is orthographically similar to many SE Asian scripts.
4About Telugu Script Acchulu Hallulu Maatra Voththulu Vowel Sound Symbols (16)HalluluConsonant Sound Symbols (38)MaatraVowel Sound Modifying Symbols for Hallulu (16)VoththuluCore Consonant Sound SymbolsConsists of Rounded Shapes (no vertical strokes)Characters may be basic vowel/consonant shapes or could be composed by compounding shapes ([NCK 01] shows examples)Example above shows glyphs in bounded boxes in a word pronounced as “Maa-tru-gee-ta”
5Some Features of Telugu Script Telugu is a phonetic script with each character representing a spoken syllable.Contains curved letters with no vertical linear strokes and shirorekha (head line).16 Vowels, 36 consonants, Telugu OCR system [NCK 01] reduced possible 10,000 symbols to about 400 glyphsGlyph represents a single connected component, but is NOT a character
6More Features of Telugu Script Orthography is compositional with vowel sound symbols (matraas) modifying basic consonants.Pure consonants sounds can be symbolized as vottus and can be combined with other consonant/vowel modified consonant symbols.A character is made from a combination of the aboveVottus and matraas can be positioned at locations surrounding the base character
7Brief Review Recognition Approaches OCR Efforts in TeluguBrief Review Recognition Approaches[RD 77] Rajasekharan and Deekshatulu 1977[SSP 95] Sukhswami, Seetharamulu , and Pujari 1995[NCK 01] Negi, Chakravarthy and Krishna 2001[NCS 02] Negi, Chakravarthy and Suresh Kumar 2002[ P 02] Pujari et al 2002[C R M N] Chakravarthy et al. 2002[VP 02] Vasantha and Patvardhan 2002[ NKC 03] Negi, Kasinadhuni, Chandrakanth 2003
8Focus on Text Line and Character Segmentation Issues In this presentation our contribution is focussed onText line Extraction: By clustering of connected components based upon their spatial properties.Character segmentation- Drop Fall method and White stream method
10MotivationText-line and text column extraction are crucial in PLA (Text Line Segmentation)Affects the word and character level analysis.Helps in logical grouping of individual glyphs into characters.Simplifies the determination of logical sequence of characters.Can be used to reduce the search space of OCR.
11Overview of Text Line Segmentation Approaches Approach as shown in [NKC 03] very complex, high time complexityPixel Projection Profile ApproachesSimpler, but do not work well with complex layouts and overlapping lines, or presence of skewBounding Box Projection ApproachesMore efficient, work well in certain conditionsLimitations due to unevenness of white spacingBounding Box Co-ocurrances (this work)
12Text Line Segmentation Using BB Projections Heuristics Based on BB ProjectionsConcept is to extract adjacent zero BB count scan lines between BB peak linesWhite space in between text-lines is broken, uneven and not contiguous because of the vottus and maatras in between text lines.Touching characters from adjacent text-linesMore heuristics to improve results by estimating interfering characters from BB projections but results are not very good due to difficulty of estimation
13Co-occurrence“A measure of OVERLAP between different connected components.”It is based on the spatial relationships of connected components.It’s symmetric in nature.Two types:Horizontal co-occurrenceVertical co-occurrenceCo-occurrence defines 3 different spatial relationships between components.
19Text-line extraction using co-occurrence Text-line extraction problem is formulated as:Identifying all the connected components which belong to the same text-line and obtaining the boundaries of text-lines by considering the bounding boxes of components.Two major steps:Computation of horizontal co-occurrence matrix for each pair of components.Clustering of connected components based on the h-cooccurrence matrix.
20Text-line extraction - Clustering Let P,Q be two CC in the document image. P<Q, P,QC, PTk, Q ?If h-co-occurrence(P,Q) = total inclusion add Qh-cooccurrence(P,Q) = partial inclusion, add Q *h-cooccurrence(P,Q) = no relation, check next*-conditional to overlap being greater than ½(height)Post processing step
28How can we segment characters? Successful segmentation mainly involves two steps:1.Locating a segmentation point2.Generating a segmentation pathDrop Fall Methods attempt to do both
29Hybrid Drop Fall Method Segments the characters by following the contour of the image.Advanced version of Hit and Deflect strategy.Follows a set of rules that maximizes the chances that it will hit and deflect its way to an accurate path.
30Drop Fall builds a path by mimicking an object falling or rolling in between the two characters There are 8 varieties of Drop Fall methods which differ in directions, starting points and set of rules.Path generated by a drop fall can be seen in fig given below
31Locating the segmentation point Pixels are scanned row-by-row until a black boundary pixel with another black boundary pixel to the right of it is detected, where the two pixels are seperated by atleast one white space.This white pixel is then used as the starting point from which the marble is rolled down
33Incorrect segmentation of touching characters can be seen in the figure shown below. Incorrect starting points leads to incorrect segmentation path.
34Drop Fall Path Generation The algorithm first looks out for a white pixel in its surroundings and if unable to find a white pixel then only cuts through the black pixel.The directions that the algorithm will move in according to the present pixel positions and its surroundings is shown below
35Top Left Drop Fall Input: Image 1.Binarize the input image 2.Locate the Segmentation point (x, y) using drop fall3. Generate the segmentation path using the rules specified in the previous slide.Output: Segmented Image
36Characters segmented using top left Drop Fall: (standard drop fall)
37Top left fails to segment the touching characters when the first character contains a Talakattu or is of concave shape.Eg :Incorrect segmentation of Touching characters using Top left drop fall
38Top Right Drop FallIdentical to Top left drop fall except that it initiates from the top-right of the connected component.Input: Touching character ImageBinarise the input imageFlip the image verticallyLocate the segmentation pointGenerate the segmentation pathRe-flip the Image and obtain the segmented image.
40Bottom Left Drop FallIdentical to standard drop fall except that it initiates from the bottom left drop fallInput:Touching charactersBinarise the input imageFlip the image horizontallyLocate the segmentation pointGenerate the segmentation pathRe-flip the Image horizontally and obtain the segmented image.
41Bottom Left Drop Fall Method Touching Characters segmented using Bottom left drop fall
42Bottom left Drop FallFails to segment the touching characters when the bottom half of the first character consists of curves or grooves
44Characters segmented using Bottom right drop fall Cases where Bottom right drop fall fails to segment the touching characters
45Advanced Drop fall methods Similar to Drop fall method in locating the segmentation point but while generating the segmentation path follows different set of rules.While generating the segmentation path it will be look out for white pixels and when unable to find a while pixel it will move for black pixels and when it is on black pixels it will only look for black pixels.
46Difference between drop fall and Advanced drop fall segmentation paths
47Advanced Top left Drop Fall Characters using Advanced top left drop fallIncorrectly segmented characters using Advanced top left drop fall
48Advanced Top right drop fall Identical to Top right drop fall except the segmentation path generated is different.Characters segmented using Advanced top right drop fall
49Advanced bottom left drop fall Characters segmented using Advanced bottom left drop fallIncorrectly segmented characters using Advanced bottom left drop fall
50Advanced bottom right drop fall (ABRD) Characters segmented using ABRDIncorrect segmented characters using ABRD
51White Stream Method Used for identifying correct segmentation point Input: Touching characters1.Generate the contour2Generate a bounding box3.Count the number of white pixels in each column until a black pixel is encountered starting from the bottom of image4.find column C with maximum count of white pixel.5.output: Segmented characters
52White stream Drop Fall Segmentation In this method depending on the aspect ratios the segmentation is done. Horizontally touching characters are segmented using white stream method for locating the segmentation point and drop fall method to generate the segmentation pathVertical touching characters are segmented using the column anding and projection profile
53White stream drop fall segmentation Characters segmented using white stream DF
54Objective Function Drop Fall Segmentation (OFDS) Is a Hybrid method (Column Anding + DF)Input: Touching charactersCalculate the aspect ratio of C.CPerform Column anding and find the column to be segmentedGenerate the segmentation path using DF
57Future workSegmenting characters which consists of double and triple touchings.Find the best path among various drop fall segmentation paths.Finding the cavities and their positioning in order to segment the touching characters.