Structural Image Analysis in Investigation of Concrete Warsaw, October 21-23, 2002 Classification of defects on the surface of black ceramics Leszek Chmielewski, Mariusz Nieniewski, Marek Skłodowski, Waldemar Cudny Division of Vision and Measurement Systems (PSWiP) Institute of Fundamental Technological Reserach, PAS (IPPT PAN) Adam Jóźwik Institute of Biocybernetics and Biomedical Engineering, PAS (IBIB PAN)

2/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Schedule l Objects and their defects l Detection of defects l Classification of defects l Training of the classifier l Postprocessing l Performance of the processes l Results Acknowledgements This research was partly supported by the European Commission:  COPERNICUS grant CRASH no. COP ( )  INCO-COPERNICUS grant SQUASH no. ERBIC 15CT ( )

3/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics This is not concrete – this is ferrite l Black ceramics: u ferrite cores u magnets l The material is: u milled u molded u pressed u sintered u ground u transported u... l A large number of various defects can emerge during these processes A pair of ferrite cores

4/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Objects and their defects – called nicely: irregularities l Surfaces: 3 important types of defects: u crack u chip u pull-out l Sometimes difficult to classify even for humans l Tiring quality inspection

5/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Special illumination: controlled light l Tangential, multidirectional light amplifies the visibility of defects l Brightness uniform and independent on distance LED illuminatorFluorescent illuminator

6/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Locating the object in the field of view l Simple morphological operations help to find the region occupied by the object for further processing l Aim: to eliminate bright spots and blobs  In this application a narrow stripe at the edge was excluded from analysis original  thresholded  complemented original  complemented

7/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Detection of irregularities (not defects!) Region of interest is further limited to the irregular part of the surface with the morphological methods  tomorrow’s presentation by prof. Mariusz Nieniewski original  thresholded  elongated irregular  summed

8/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Classification of irregularities – features (1/2) l Each pixel detected in the detection phase is classified with the pattern recognition methods. Pixel = pattern. l Features are calculated for each pixel: functions on pixel neighbourhoods = masks. l Direction invariance of features is obtained by rotation of the mask according to local directionality of texture. Pixel & its mask original maskrotated mask [YBF95]

9/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Classification of irregularities – features (2/2) l brightnesses in the original & rotated mask l statistical moments of order up to R in masks l gradient modulus l 9 classical textural features according to [Law80, Pra91] l textural features based on coocurrence relations [WuCh92] l relative values of brightness function along the red line From 30 to 150 features were used for feature selection. For example: Features as in [Law80, Pra91]: convolve the mask with A 1 – A 9 and take standard deviation of the output values  9 features.

10/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Classification: the K Nearest Neighbour (k-NN) method

11/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics K-NN: Enhancements and speed-ups l With full selection of features and K u Leave-one-out method l Fuzzy version u Fuzzy decisions made crisp in the end l Parallel u Distinct classifiers for each pair of classes l Hierarchical u Advanced version only where classes overlap l Reference set largely reduced u with the modified, bidirectional Hart algorithm  Optimized, low error rate, quick algorithm

12/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics The parallel version of the K Nearest Neighbour method

13/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics K-NN: class overlap as the training criterion (not error)

14/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Training: the training patterns Note: artificial, boundary classes introduced  better accuracy 2479 training patterns can be obtained quite quickly...

15/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Training: first results The system has successfully classified thousands of unknown pixels. Quite satisfactory results can be obtained with just 4 training images training patterns raw classifi- cation results enhanced by local votong

16/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Training: first results – zoom – results of the enhancement This was only a convincing example. The error rate estimated with the leave-one-out method was 3.3%. More training patterns were used in the final system. pixels used in trainingclassified pixels: all / raw / enhanced

17/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Final training results – error estimates l 5903 training patterns l a posteriori error probabilities: pixel classified as class "i" (row) comes in fact from the class "j" (column); in % l overall error: 2.56% l max error: 9% between classes 8 and 9 u cared for by the postprocessing (to some extent)

18/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Classification results – various types of elements (1/4) blue – chip, yellow – good object

19/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Classification results – various types of elements (2/4) brown – irregular, red – crack, green – pull-out, blue – chip, grey – good object.

20/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Classification results – various types of elements (3/4) blue – chip, navy – chip near good, red – crack.

21/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Classification results – various types of elements (4/4) blue – chip, green – pull-out, red – crack !?

22/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Higher level – discern cracks from grinding grooves Classify cracks – red between all irregular regions – green. Limits of the method reached. details in other images

23/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Classification results – rotation (in)variance

24/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Technical data & Performance Resolution A 512*512 pixel camera. Spatial resolutions:  0.05mm/pixel (up to 20*larger magnifications can be attained with normal lenses) Accuracy of results Classification errors: overall up to 4%, inter-class typically 4%, max 10%; Stability of results Detection phase: repeatability not worse than 2-5% in area. Classification phase: repeatability not worse than 10-20% in area, depending of how fast classifier version is used. Processing time [s] (PC, 1000 MHz) only softwaremorphological processor image acquisition detection classification1.00 typically; 10 for v. large defects - 20% of object decision0.1

25/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics Conclusion l Irregularities of flat surface in black ceramics – ferrite cores, magnets – can be detected and classififed l Special lighting system has been designed l Detection of irregularities: u Irregularities in general – dynamic thresholding u Compact irregularities – morphological method u Elongated irregularities – morphological method u General decision on quality of the tested object l Classification of irregularities: u Training by showing examples l Segmentation and measurements u Detailed, quantitative final decision Project www site:

26/25SIAIC 2002, Warsaw L. Chmielewski (PSWiP) et al., Classification of defects on the surface of black ceramics References A. Jóźwik, L. Chmielewski, M. Skłodowski and W. Cudny. A proposition of the new feature space and its use to construction of a fast minimum distance classifier. In Proc. 2nd Polish Conference on Computer Pattern Recognition Systems KOSYR 2001, pages , Miłków, Poland, May 28-31, M. Nieniewski, L. Chmielewski, A. Jóźwik and M. Skłodowski. Morphological detection and feature-based classification of cracked regions in ferrites. MG&V, 8(4): , A. Jóźwik, L. Chmielewski, M. Skłodowski and W. Cudny. Class overlap rate as a design criterion for parallel Nearest Neighbour classifier. In Proc. 1st Polish Conference on Computer Pattern Recognition Systems KOSYR'99 Trzebieszowice, Poland, May 24-27, A. Jóźwik, L. Chmielewski, M. Skłodowski and W. Cudny. A parallel net of (1-NN, k-NN) classifiers for optical inspection of surface defects in ferrites. MG&V, 7(1-2):99-112, G. Vernazza, M. Lugg, T. Postupolski, A. Jóźwik, L. Chmielewski, D. Chetverikov and M. Peri. SQUASH: Standard Compliant Quality Control System for High-Level Ceramic Material Manufacturing. In Proc. INCO-COPERNICUS-INTAS Workshop on Advanced Ceramics and Alloys, pages 35-40, Brussels, Belgium, Mar 12-13, European Commission, Directoriate Generale XII. L. Chmielewski, M. Skłodowski, W. Cudny, M. Nieniewski and A. Jóźwik. Optical system for detection and classification of surface defects in ferrites. In Proc. 3rd Symp. Image Processing Techniques (TPO'97), pages 1-13, Serock, Poland, Oct 29-31, Oficyna Wydawnicza Politechniki Warszawskiej. M. Mari, C. Dambra, D. Chetverikov, J. Verestoy, A. Jóźwik, M. Nieniewski, M. Skłodowski, L. Chmielewski, W. Cudny and M. Lugg. The CRASH Project: Defect Detection and Classification in Ferrite Cores. In A. Del Bimbo, editor, Proc. 9th Int. Conf. Image Analysis and Processing, number 1310 in Lecture Notes in Computer Science, pages (vol. II), Florence, Italy, Sept 17-19, Springer Verlag, Berlin. A. Jóźwik, L. Chmielewski, W. Cudny and M. Skłodowski. A 1-NN preclassifier for fuzzy k-NN rule. In Proc. 13th Int. Conf. Pattern Recognition, pages D D-238, Wien, Austria, Aug 25-29, IAPR, Technical Univ. Vienna. [Law80] K. I. Laws, Textured image segmentation, Univ. of Southern California, Image Processing Institute, USCIPI Report 940, Jan 1980 [Pra91] W. K. Pratt, Digital Image Processing, John Wiley, New York [WuCh92] C-M. Wu, Y-C. Chen, Statistical feature matrix for texture analysis, CVGIP: Graphical Models and Image Processing, 54, 5, 1992, [YBF95] G. Z. Yang, P. Burger, D. N. Firmin, S. R. Underwood, Structure Adaptive Anisotropic Filtering for Magnetic Resonance Image Enhancement, Proc. 6th Int. Conf. CAIP, Prague, Czech Republic, Sept. 6-8, 1995, Lecture Notes on Computer Science. Springer Verlag, 1995.