AUTOMATIC LANDMARKING OF CEPHALOGRAMS BY CELLULAR NEURAL NETWORKS D. Giordano 1, R. Leonardi 2, F. Maiorana 1, G. Cristaldi 1, M.L. Distefano 2 1 Dipartimento di Ingegneria Informatica 2 Clinica Odontoiatrica II - Policlinico University of Catania Italy

Cephalometric analysis Cephalograms are lateral skull radiographs taken under standard conditions Cephalometric analysis is based on the identification of landmarks, which are used for linear and angular measurements It is important for orthodontic planning and treatment evaluation Literature review Outlines of CNNs. Tool and the CNN templates Experimental evaluation Results Conclusions

A cephalogram

Tracing key anatomical structures

Landmarks identification Baseline for measurements

Approaches to cephalometrics 1.Manual. placing a sheet of acetate over the cephalometric radiograph, tracing salient features, identifying landmarks and measuring distances and angles between landmark location. 2.Computer aided. Landmarks are located manually while these locations are digitized into a computer system. The computer then completes the cephalometric analysis. 3.Completely automated. The computer automatically locates landmarks and performs the cephalometric analysis. AIME 05 Literature review Outlines of CNNs. Tool and the CNN templates Experimental evaluation Results Conclusions

1.speed-up a very time-consuming manual process 2.improve measurements accuracy AIME 05 Why automated landmarking? Literature review Outlines of CNNs. Tool and the CNN templates Experimental evaluation Results Conclusions

PRIOR KNOWLEDGE LEARNING APPROACH AIME 05 Previous approaches to automated landmarking Literature review Outlines of CNNs. Tool and the CNN templates Experimental evaluation Results Conclusions

1.Use of filters to minimize noise and enhance the image, 2.Application of operators for edge detection, 3.On line-following algorithms guided by a prior knowledge, introduced in the system by means of simple ad hoc criteria AIME 05 Approaches based on prior knowledge Literature review Outlines of CNNs. Tool and the CNN templates Experimental evaluation Results Conclusions

Some examples of the techniques that have been used: Neural networks together with genetic algorithms Fuzzy neural networks Active shape models AIME 05 Approaches based on learning and pattern matching Literature review Outlines of CNNs. Tool and the CNN templates Experimental evaluation Results Conclusions

WorkSample size Techniques Parthasarathy et al. (1989) 5Resolution piramid Knowledge based line extractor Tong et al. (1990) 5Resolution pyramid Edge enhancement Knowledge-based extraction Cardillo et al. (1994) 40Pattern matching Rudolph et al. (1998) 14Spatial spectroscopy Statistical pattern recognition Liu et al. (1999) 38Multilayer Perceptron Genetic Algorithms Hutton et al. (2000 ) 63Active Shape Models El-Feghi et al. (2003) 200Fuzzy neural network Innes et al. (2002) 109PCNN pulse coupled neural networks

Limitations of previous approaches 1.Accuracy achieved 2.Performance varying on different landmarks 3.Strongly dependent on the quality of the X-rays Golden standard: landmarks should be located within 1mm tolerance; although 2mm is deemed acceptable for clinical practice AIME 05 Literature review Outlines of CNNs. Tool and the CNN templates Experimental evaluation Results Conclusions

Our approach The proposed method proposed is based on CNN (Cellular Neural Networks) CNNs are an emerging paradigm for image processing CNNs is a powerful computational model equivalent to a Turing Machine AIME 05 Literature review Outline of CNNs Tool and the CNN templates Experimental evaluation Results Conclusions

Cellular Neural Networks CNN consist of computational units (cells) arranged in matrix forms (2D) or cube forms (3D) Each cell is a dynamic unit with an input, an output and a state Each cell is influenced by the input and the output of all neighboring cells within a given radius AIME 05 Literature review Outline of CNNs Tool and the CNN templates Experimental evaluation Results Conclusions

The neighborhood circle of the interacting cells is defined as follows: Nr (i,j) =  C (k,h): max (  k-i ,  h-j  ) ≤ r, 1≤ k≤ M; 1≤ h≤ N  where M and N are the matrix dimensions AIME 05 Circles of influence with radius equal to one for cells Cij, Ci+1, j+1 Literature review Outline of CNNs Tool and the CNN templates Experimental evaluation Results Conclusions Cellular Neural Networks

CNN dynamics are determined by the following equation, where x is the state, y is the output, u is the input, x ij is the generic cell belonging to the matrix I ij is the activating treshhold for each cell. AIME 05 Cellular Neural Networks Literature review Outline of CNNs Tool and the CNN templates Experimental evaluation Results Conclusions

CNN dynamics are determined by the following equation, where x is the state, y is the output, u is the input,: A is known as feedback template B is known as control template AIME 05 Cellular Neural Networks Literature review Outline of CNNs Tool and the CNN templates Experimental evaluation Results Conclusions

Several image processing tasks can be performed by CNNs by programming by templates Library of known templates are available A key advantage is that the inherently parallel architecture of the CNN can be implemented on chips, known as CNN-UM (CNN Universal Machine) chips allowing computation times three orders of magnitude faster than classical methods. Cellular Neural Networks Literature review Outline of CNNs Tool and the CNN templates Experimental evaluation Results Conclusions

In our work we used: A constant treshold for each cell A circle of influence with radius equal to 1 (A, B: 3X3) and with radius equal to 2 (A, B: 5X5) Every cell has an initial state variable equal to zero Contour condition uij = 0 (Dirichlet condition) Input: the image to be processed Symmetrical feedback templates (to ensure steady state) Exploitation of the transient solution  n. of cycles and integration steps are important for landmark identification AIME 05 Cellular Neural Networks Literature review Outline of CNNs Tool and the CNN templates Experimental evaluation Results Conclusions

Our system is based on a software simulator of a CNN of 512X480 cells. It uses different types of CNNs on the scanned cephalogram 1) first to pre-process the image and eliminate the noise, 2) then to ensure that each landmark region is properly highlighted (by appropriate CNN templates) 3) landmark-specific algorithms including expert rules for point identification are then applied and landmarks coordinates computed AIME 05 Tool Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions

AIME 05

The system operates based on two classes of rules Expert rules concerning where landmark should be located, Rules to select the proper CNN template based on local image properties AIME 05 Tool Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions

The tool has been designed to detect 8 landmarks, which are essential to conduct a basic cephalometric analysis: Menton, B point, Pogonion, PM point, A point, Upper incisal, Lower incisal, Nasion. AIME 05 Tool Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions

Why n. of cycles are important AIME 05 Non saturated CNN Output Saturated CNN Output Using images with the same brightness simplifies point extraction and emphasize program correctness Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions

Menton AIME 05 Templates and CNN output for Menton (n.cycles=30) Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions

Gnation and B point Templates and CNN output for Chin Curvature (n.cycles=30) Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions

AIME 05 Up and low Incisors Templates and CNN output for incisors Curvature (n.cycles=60) Good contrast and luminosity Low contrast and luminosity Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions

Nasion White nasion Black nasion Four templates were used Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions

AIME 05

8 landmarks were chosen for preliminary assessment of the method, and a set of 97 digital X-rays was landmarked by an expert orthodontist. Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions Assessment

The first stage assessed the image output of the CNNs, to verify that it included the sought landmark. This was done by visual inspection from the same expert who landmarked the X-rays. Over 97 cases, 29 cases (30%) led to CNN outputs in which some edges were overly eroded. This implies that the number of processing cycles in these cases needs to be reduced. Assessment Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions

The second stage evaluated performance of the developed algorithms for 8 landmarks Sample of 26 cases randomly selected from the previous one after eliminating the cases that had not been taken into consideration by the algorithms. Assessment Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions

The coordinates of each point found by the program were compared to expert landmarking, and the Euclidean distance of the found landmark from the reference one was computed. Assessment Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions

Results LandmarkMean error (mm) MDSD ≤1 (mm) >1;≤2 (mm) Imprecise casesSuccess Rate Success Rate (overall sample) ≤3 (mm) >3 (mm) Upper incisor %8% 4%-96%92% Lower incisor %26% 4% 92%81% Nasion %17% -13%87%81% A Point %21% 17%4%79%73% Menton %7% 4% 92% B Point %8% -21%79%73% Pogonion %8% 11%81% PM Point %8% 15%77% Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions

Work and Ref.Sample size N. Landmarks and accuracyTechniques Parthasarathy et al. (1989) [10] 59 landmarks, 58% < 2mm, (18%<1mm) mean error: 2.06 mm Resolution piramid Knowledge based line extractor Tong et al. (1990) [11] 517 landmarks, 76%< 2mm mean error: 1.33 mm Resolution pyramid Edge enhancement Knowledge-based extraction Cardillo et al. (1994) [13] 4020 landmarks, 75% < 2mm mean error: not reported Pattern matching Rudolph et al. (1998) [14] 1415 landmarks, 13% <2mm mean error: 3,07 mm Spatial spectroscopy Statistical pattern recognition Liu et al. (1999) [6] 3813 landmarks, 23% < 2mm (8% <1mm), mean error: 2,86 mm Multilayer Perceptron Genetic Algorithms Hutton et al. (2000 ) [7] 6316 landmarks, 35% < 2mm (13% < 1mm) mean error: 4,08 Active Shape Models El-Feghi et al. (2003) [16] landmarks, 90% <2mm mean error: not reported Fuzzy neural network Innes et al. (2002) [18] 1093 landmarks, 72% <2mm, mean error: not reported PCNN : pulse coupled neural networks Our Work 268 landmarks, 85%<2mm (73% < 1mm) mean error: 1.07 mm Cellular Neural Networks Knowledge based landmark extraction

The experimental results have shown that of the sought landmarks 85% are within 2mm precision, and remarkabily that 73% are within 1mm. Results Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions

CNNs provide an effective method to pre-process images for automated landmarking They are accurate and flexible (integration of edge based and region based methods) Their hardware implementation affords real-time performance Literature review Outlines of CNNs. CNN prototype and the templates Reports the experimental evaluation Results Conclusions

The approach that we have employed will be further improved by prior classification on the cases based on: 1.Key morphologies of the skull (e.g., byte typology, shape of anatomical structures) 2.X-ray brightness Conclusions Literature review Outline of CNNs Tool and CNN templates Experimental evaluation Results Conclusions

MANY THANKSGrazie