Presentation is loading. Please wait.

Presentation is loading. Please wait.

!!! Automatic Understanding of the Images Many techniques of medical computer vision, e.g. X-ray images processing, analysis and recognition are now.

Similar presentations


Presentation on theme: "!!! Automatic Understanding of the Images Many techniques of medical computer vision, e.g. X-ray images processing, analysis and recognition are now."— Presentation transcript:

1

2 !!!

3 Automatic Understanding of the Images

4 Many techniques of medical computer vision, e.g. X-ray images processing, analysis and recognition are now very known

5 All such techniques exist because every image can be represented as a two-dimensional function One can find some details of the image on the function structures

6 understanding Automatic understanding of the images (and other signals) is a new scientific idea in the area of computer vision and signal processing. Traditional (and very known) approach in such technique includes: processing digital image processing analysis computer analysis of the image recognition automatic recognition of the image

7 Relations and connections between the traditional sub-items of total computer vision task presents following figure: Real object Registration Digital image Image processing Image description Decision based on the image Analysis Recognition Sometimes image processing task is performed many times until the image of desired quality is obtained! In simple cases Computer graphics Not discussed at this works! Image processing

8 ...automatic understanding „understanding” of the images ? contribution What new contribution is added to this schema by...

9 Let me explain it step by step

10 Stage 0: Stage 0: Registration of the images (or other signals obtaining information) Object under consideration Raw - probably noisy and not very useful signals

11 Stage 1: Stage 1: Preprocessing (mainly filtration) of signals Object under consideration Signals cleaned and enhanced

12 Stage 2: Stage 2: Signal analysis and feature extraction Object under consideration Features and parameters extracted from signals

13 Stage 3: Stage 3: Recognition and classification of signals Object under consideration Object classification and pattern recognition

14 Stage 4: Signal understanding and semantic interpretation Object under consideration Merit sense and semantic content of the object

15 Recent book describing the details:

16 Image processing Image processing helps us to answer the question: how to increase quality and visibility of the image? Image analysis Image analysis helps us to answer the question: which are the exact values of selected features of the image? Image recognition Image recognition helps us to answer the question: to which classes (patterns) do the selected objects on the image belong? Lets try to analyze exact meaning of such items:

17 Automatic image understanding helps us to answer following questions: follows What follows the visualized details ? & meaning What is the meaning of the features extracted from the image? & belong to particular classes What are the results of the fact, that some objects belong to particular classes?

18 „automatic understanding” new The idea of „automatic understanding” is very new, so is necessary whowhen we ought to explain, why this new idea is necessary and who and when can use automatic understanding instead of simple recognition.

19 Let me use for explanation......although the matter under consideration is definitely serious

20 Lets think together... intelligent in sense of semantic content...how to describe criteria for intelligent selection of next pictures similar (in sense of semantic content) to the ones shown on the next slides from a multimedial database.

21

22 Hallo! Honey !!!Splash!!!!!!!

23 this Do you see this body?! Where ??! Brrryyymmss!

24 classical image analysis For solving of selected problem using classical image analysis one needs following operations: Segmentation of all pictures and selection of important objects on all images Extraction and calculation of the main features of the selected objects Objects classification and recognition Selection of features and classes of recognized objects which are the same on all images under consideration Searching in database for images having the same objects with the same features

25 Performing pointed out steps for the first image under consideration we can find: Object number 1: features:... Recognition = Women Object number 2: features:... Recognition = vehicle

26 Performing pointed out steps for next image under consideration we can find: Object number 1: features:... Recognition = Women Object number 2: features:... Recognition = Vehicle Object number 3: features:... Recognition = Man

27 Performing pointed out steps for last image under consideration we can find: Object number 1: features:... Recognition = Women Object number 2: features:... Recognition = Vehicle Object number 3: features:... Recognition = Man Object number 4: features:... Recognition = Man

28 Summarizing our information we can do such induction: WomenVehicleOn all images we can find two objects: Women and Vehicle ManMenOn some images there are also object Man, but not on all - so Men can be automatically considered as not important in searching criteria Women VehicleResult: computer finds and presents as desired output all images with Women and Vehicle

29 For example such an image can be obtained from the multimedial database : (For people form non-communistic countries: It is a very known allegory of the Soviet poster named “all young girls ought to work on tractors”)

30 Let see some real examples...

31 bellow Vehicle In fact the image presented bellow ought to be taken from the Web database although it does not contain Vehicle at all!

32 proper meaning It is proper solution because in fact general meaning of all presented images is covered by the sentence: Now we can see, why (and because of who) the men’s life can be so often shortened!

33

34

35

36

37 I apologize... It was off course only joke (I apologize...) content... because very often images apparently very different in fact can hide the semantically identical the matter although is quite serious...

38 images understanding? What is than the sense of images understanding? present visible Basically we do need a method of extraction of some kind of semantic content, which is present in the image, but is not simply visible on it. This job can be difficult, because the matter of things is often hidden and needs understanding instead of simple recognition!

39 Technically speaking, the difference between image recognition and image understanding includes following assumptions: recognitionfixed number of a priori known classes differentiationin case of recognition we always have a fixed number of a priori known classes and the task demands only extraction of all these features of the image, which are necessary and sufficient for differentiation between classes under consideration. After processing we do obtain the number (or name) of proper class; understanding potential number of possible classes goes to infinity. automatic reasoning processin case of understanding we have not any a priori known classes or templates, because in fact the potential number of possible classes goes to infinity. So after processing we do obtain a description of the image content without using any a priori known classification, because even the criteria of classification are constructed and developed during the automatic reasoning process.

40 The starting point of all our research was analysis of medical images

41 Difficult medical images! Urography of kidney and ureter ERCP image of pancreas Coronarograhpy of heart arteries

42 MRI image of cerebral hemispheres CT image of cerebral hemispheres Actual area of our research: Brain Images

43 Images of M Images of M icroscopic Structure and Tissue

44 Development of proper methods for automatic interpretation of such images is very difficult because of two types of troubles: n First, morphology of health organ is different for every human being, so we have not any kind of template of “proper view” of the analyzed object. n Second, deformations of the organ shape and size can be very different in the form, number and in localization also if the diseases are in fact identical!

45 Lets compare two examples of ERCP images of pancreatic duct with pancreatits (permanent inflammation) The illness is the same, but the images are definitely different! Nobody can show any pattern or template on the image, which may be pointed as a signal of this kind of illness.

46 The same situation we can find when analyzing of ERCP images presented pancreatic cancer. pattern Who can point out here the pattern for recognition ??!

47 Another typical example: cardiological imaging (coronarography) with evident symptoms of serious cardiac illnesses. normal form localization Every patient has quite different shape of normal arteries and definitely another form and localization of pathological changes (stenosis).

48 Similar problems one can find during analysis of X-ray images of urters and kidney pelvis imaging (with pathological narrowing and other possible deformations). Different images, the same illness!

49 n Problem of proper interpretation of such images can not be solved by means of traditional image processing, picture analysis and pattern recognition. n Only way is formulating of new paradigm of images interpretation and development of new method of its full automatisation based on application of advanced tools of artificial intelligence. n Automatic reasoning about images semantic content, performed on the base of picture analysis is called automatic understanding of the image. On the base of analysis of many examples of medical images we try formulate such three assumptions:

50 Fundamental features of the automatic understanding: n We try to simulate natural method of thinking of the physician, who needs to understand the illness before formal diagnosis and selection of treatment. n First we make linguistic description of the of the merit content of the image using special kind of image description language. Thanks to that idea we can describe every image without pointing to any limited number of a priori described classes. n The linguistic description of the image content build such way is the base for understanding of the image for diagnosis or for indexing of multimedial database.

51 New view for analysis and semantic classification of visual data including Understanding Systems and This new schema includes pattern analysis and semantic classification. It can be used for intelligent creation of multimedia representation in Visual Data Understanding Systems for Web applications.

52 When we use traditional pattern recognition all process of the image analysis is based on feed- forward (one-direction flow of signals) scheme. Object Taking of the image (reception) Medical image Analysis X=3,75 y=2,54 z=-8, Recognition Image features Decision Diagnosis: narrowing of left coronary artery!

53 source of knowledge during image understanding we always have two-directional flow of information: input data stream (all features obtained by means of analysis of the image under consideration) must be compared with the stream of demands generated by dedicated source of knowledge. The demands are always connected with particular (selected) hypothesis of semantic interpretation of the image content. In contrast to this simple scheme...

54 understanding two-directional So we assume understanding of the image as two-directional flow of information. Object Image acquisition Medical image Analysis X=3,75 y=2,54 z=-8, Linguistic description Image features Knowledge about images Description of image merit content Demands about image merit content    Cognitive resonance Under- standing of image content Demands are kind of postulates, describing desired values of some (selected) features of the image. The selected parameters of the image under consideration must have desired values when some assumption about semantic interpretation of the image content can be validated as true.

55 image understanding Such structure of the system for image understanding corresponds with one of the very known models of natural visual perception by man, named knowledge based perception.

56 Humans eye can not recognize object if the brain have not a template for such view. knows It remains true although the brain knows the object, but in another view, what means another signals. For example - lets try to recognize the face of a known man.

57 If we have no experience in seeing some kind of images - we can not recognize even very known object! Lets try recognize, who is this man? Do you know them?

58 The right answer is... President George W. Bush works on a Habitat For Humanity house in Tampa, Tuesday, June 5, oficial This is oficial WHITE HOUSE PHOTO by Eric Draper

59 This situation is easier... becuse is typical and known

60 “who is Bush” Perhaps somebody can not recollect, “who is Bush” ? without So, let me present the other experiment without recognition. recognize Perhaps you are thinking: It is not easy recognize somebody on first look?

61 fast Lets explain fast, which is the main difference between such two variants of a very known image? Difficult to tell?

62 And now?

63 generates hypothesis. selected During observation process mans brain in every moment generates hypothesis. Natural perception in fact is not only processing of visual signals obtained by eyes. It is mainly mental cognitive process, based on hypothesis generation and its on-line verification. Verification is performed by permanent comparing of the selected features of the image with the expectations taken from previous visual experience.

64 physicians How and when physicians really can use this technology? T The answer can be formulated on the base of plot in form of T letter

65 T T Fast selection for screening purposes Accurate analysis of the very- -complicated medical cases Indexing of multimedial medical databases More precise analysis for some cases

66 A few more detail information...

67 Screening (with automatic understanding of the images) T T Fast selection of the images for screening purposes

68 Hand-made screening (in theory...) Group of people Physician doing selection Pointing of “suspicion” person for more detail investigation Standard medical investigation

69 Hand-made screening (in practice...) Group of people Physician doing selection Standard medical investigation Everybody OK.!

70 Semi-automatic screening (with use of the automatic understanding of the images)

71 Group of people Physician doing selection Standard medical investigation Algorithm for the automatic understanding of the image contents Alert signal! Very detail analysis Analysis is fast and accurate! clinical Pointing of “suspicion” person for clinical investigation

72 Analysis of difficult diagnostic problems Accurate analysis of the very- -complicated medical cases T T

73 Patients data Surgery ? Pharmacology? ? Patients organ with not known disease Board of experts (physicians)

74 Surgery ? Pharmacology? ! Patients data Algorithm for the automatic understanding of the image contents Suggestion: cancer

75 Searching in mulitimedial medical databases T T Indexing of multimedial medical databases

76 Multimedial database Index file Algorithm for the automatic understanding of the image contents Semantic description of the image contents Identification code Identification Name Illness description Images  #123ABC

77 Multimedial database Index file Algorithm for the automatic understanding of the image contents Semantic description of the image contents Identification code Identification Name Illness description Image User Example Description of the selected case Question

78 Example Multimedial database Index file Semantic description of the image contents Identification code Identification Name Illness description Image User Description of the selected case Question Answer Optimal therapy

79 Example: Semantic information representation in understanding of images with different lesions Images of: urinary tract (left), coronary vessels (upper right) and pancreatic duct (bottom right)

80 cognitive resonance Verification of the hypothesis is based on the process called cognitive resonance parsing In future works we do identify cognitive resonance with parsing of linguistic description of the image content. terminal symbols non- terminal In this parsing process strings of terminal symbols (“statements ” describing features of the image) are converted to single non- terminal symbols, which can be interpreted as semantic description of the merit sense (for example diagnosis).

81 description The graph grammar as a tool for description of the images before its automatic understanding

82 Elementary example of scene analysis using EDT graph grammar (I) Examples of analyzed scenes I II III

83 Elementary example of scene analysis using EDT graph grammar (II) 1. Analysis of scene described by the following primitives: - building - car - tree b d a 2. Definition of connection primitives: p r s t u v x y „Nouns” „Verbs”

84 Elementary example of scene analysis using EDT graph grammar (III) 3. Description of analyzed scenes I II III b ad tv b a d d tv v b a d dd u tv v

85 Elementary example of scene analysis using EDT graph grammar (IV) 4. Graph based description of analyzed scenes I II III b b b a a a d d d d dd t u v tv tv v v b( v d t a( v d)) b( v d t a) b( v d t a( v d u d))

86 iterative The main idea of cognitive resonance is based on iterative performing of such steps: Lets assume semantic description of some image in usual form of the string of terminal symbols:  meaning Working hypothesis nr. 1 about meaning of this image leads to the assumption, that image must include at least one pattern:  We must search linguistic description of the image for localization   Not found! meaning The working hypothesis nr 1 about meaning of the image must be failed!

87 iterative The main idea of cognitive resonance is based on iterative performing of such steps: Lets assume semantic description of some image in usual form of the string of terminal symbols:  meaning Working hypothesis nr. 2 about meaning of this image leads to the assumption, that image must include at least one pattern:  We must search linguistic description of the image for localization   Lets try again...  exactly sure Hypothesis nr. 2 can be now find as a more probable, but we stillane not exactly sure, is the hypothesis true or not, because for its full validation it is necessary to test also another assumptions taken from this hypothesis.

88 The description of the cognitive resonance showed on the previous slide is the most simplified one. In fact the set of methods and formulas used by real parser designed by us especially for this works is much, much more complicated! We show it now on base of some real (and interesting!) examples

89 Computer-Aided Diagnosis of Neoplasm and Pancreatitis n To diagnose lesions of pancreatic ducts characterising neoplasm and chronic pancreatitis - Symptoms of pancreas neoplasm: local stenoses or dilatations of pancreatic duct or cysts on external borders of the pancreatic duct - Symptoms of pancreas neoplasm: local stenoses or dilatations of pancreatic duct or cysts on external borders of the pancreatic duct - Symptoms of chronic pancreatitis: incorrect lateral ramifications and local stenoses or dilatations - Symptoms of chronic pancreatitis: incorrect lateral ramifications and local stenoses or dilatations

90 ERCP Images with Chronic Pancreatitis

91 ERCP Images with Pancreas Neoplasm

92 Computer-Aided Diagnosis of Urinary Tracts n To diagnose lesions pointing to the existence of renal calculi or deposits those obstacles cause artresia of urinary tracts leading to diseases such as extra-renal uraemia or hydronephrosis n To diagnose the correct morphology of renal pelvis and renal calyx with the use of graph grammars

93 RTG Images of Left Renal Pelvises with Ureter Ducts

94 Computer-Aided Diagnosis of Coronary Arteries n To detect lesions characteristic cardiac ischemic states those states are caused by atheromatosis lesions in coronary vessels which cause stenosis of artery lumen n Cardiac ischemic disease can take the form of stable or non-stable angina pectoris or infarct

95 Coronographical Images of Coronary Arteries with Stenoses

96 Parsing we try to describe now on the base of following structure of the whole processing procedure. Our method is based on four consecutive steps:

97 Let me remind you schema of very known structural analysis of images, which is the base and tool for our method. Input Image Image Pre-processing Syntactic Analysis Classification & recognition Image Representation Segmentation Definition of image primitives and relations between them

98 Classification of Pattern Recognition Methods

99 Stages of Preliminary Image Processing n Segmentation and filtration n Skeletonisation n Analysis of real and verification of apparent skeleton ramifications n Smoothing of skeleton n Straightening transformation

100 Examples of preprocessing stages, which must be performed before starting the automatic understanding process Only proper concentrate Only proper initial processing of the image let us concentrate during understanding of the image on its important semantic features, not on noises or artifacts.

101 Method of Segmentation Mikrut Z.: A METHOD OF LINEAR “STAR-SECTIONS” APPLIED FOR OBJECT SEPARATION IN ERCP IMAGES (ICIP’96)

102 Method of Segmentation

103 Mikrut Z.: A METHOD OF LINEAR “STAR-SECTIONS” APPLIED FOR OBJECT SEPARATION IN ERCP IMAGES (ICIP’96) Averaging & median filtration Method of Segmentation

104 Result of Segmentation & Filtration Original image Binary image

105 Classical Pavlidis Algorithm of Skeletonisation Neighborhood templates for skeletal points P - actually considered skeletal point 0 - points from image background 2 – previous skeletal points P P is a skeletal point if at least one from each of X and Y point sets is not a background point

106 Skeletonisation & Smoothing of Skeleton Pavlidis skeletonisation Averaging of skeleton

107 Analysis of Lateral Ramifications Determination of parts of pancreatic duct Detection of apparent ramifications 28% 45% 27%

108 Example of goal-oriented processing: Differentiation between real and apparent lateral ramifications Duct with chronic pancreatitis Duct with pancreas neoplasm

109 Results of artifacts elimination

110 Algorithm of Straightening Transformation(I) Geometric transformation which creates width graphs of the studied structure ERCP image of pancreatic duct with pancreatitis

111 Algorithm of Straightening Transformation(II) A - the skeletal point in which object width is measured B define neighborhood of point A, and determine position of the guiding line C D - result of translation of the line C so that it passes through the skeletal point E - width measuring line, perpendicular to line D F - points of intersection of width measuring line and outer object contour These points are rotated around point A by an angle   which results in object “straightening”

112 Algorithm of Straightening Transformation(III) Algorithm of straightening transformation () { Read object contour; /*creation of object boundaries transformed (i.e. “straightened”)*/ for any skeletal point { Draw the guiding line; Draw the line measuring widths of the object; Find points of intersection of the contour and width measuring line; Determine location of these points in relation to the guiding line; Rotate the contour points around the skeletal point by an angle opposite to an angle between width measuring line and y-axis; } /*linearization of obtained table with border coordinates*/ Sort border points; Remove redundant points in the straightened contour; Improve continuity of the obtained width graph; }

113 Original image Results of Straightening Transformation Width graph Width graph obtained for the duct with chronic pancreatitis Lines determine width profiles Marked areas of real ramification

114 Scheme of Encoding Width Graphs as Terminal Symbols I Approximation of width graph contour by polygon Approximation angles for the lower part of the graph (signs altered) -28, -23, 15, 11, 31, -31, 11, 3, 37, -29, 8, 6, 6, -29, -2, 8, 105, 43, -40, -81, -14, 7, -10, 11, 66, -56, -64, 5, -28 Approximation angles (in degrees) for the upper part of the graph 7, 22, 3, -4, 65, 70, 6, -78, -125, -99, 10, 11, -12, 0, 4, 19, -28, 41, 2, -109, -146, -40, 15, 40, 13, -80, -147, -66, 12, 23, 16, -18, -145, -151, -72, 8

115 Vocabulary of the grammar Scheme of Encoding Width Graphs as Terminal Symbols Image features: angles between segments of polygon line approximating of width graph of the organ Terminal symbols as elements of linguistic description taking into account merit sense of the image Please note unusually form of the vocabulary!

116 Scheme of Encoding Width Graphs as Terminal Symbols Vocabulary of our grammar (conditions for terminal symbols) Approximation angles for the lower part of the graph -28, -23, 15, 11, 31, -31, 11, 3, 37, -29, 8, 6, 6, -29, -2, 8, 105, 43, -40, -81, -14, 7, -10, 11, 66, -56, -64, 5, -28 Approximation angles for the upper part of the graph 7, 22, 3, -4, 65, 70, 6, -78, -125, -99, 10, 11, -12, 0, 4, 19, -28, 41, 2, -109, -146, -40, 15, 40, 13, -80, -147, -66, 12, 23, 16, -18, -145, -151, -72, 8 Sequence of terminal symbols for the lower part of the graph ns, ns, s, s, s, ns, s, p, s, ns, p, p, p, ns, p, p, n, g, ng, ni, ns, p, ns, s, i, ni, ni, p, ns Sequence of terminal symbols for the upper part of the graph p, s, p, p, i, i, p, ni, nn, nn, s, s, ns, p, p, s, ns, g, p, nn, nn, ns, s, g, s, ni, nn, ni, s, s, s, ns, nn, nn, ni, p

117 Scheme of Syntactic Analysis of Pancreatic Duct Syntax analysis diagram for width graphs obtained for pancreatic ducts

118 Attributed Grammar Describing Morphological Lesions in Pancreatic Duct (I) G = (V N, V T, SP, STS) V N – non-terminal symbols set V T – terminal symbols set SP – production set, STS – starting symbol V N = {SYMPTOM, CYST, STENOSIS, DILATATION, BRANCH, HI, LO, P, S, G, I, N, NS, NG, NI, NN} V T = {p, s, ns, g, ng, i, ni, n, nn}

119 Attributed Grammar Describing Morphological Lesions in Pancreatic Duct (II) SP: n SYMPTOM  CYST Symptom=cyst n SYMPTOM  STENOSIS Symptom=stenosis n SYMPTOM  DILATATION Symptom=dilatation n SYMPTOM  BRANCHSymptom=branch Description of recognized pathological lesions

120 Attributed Grammar Describing Morphological Lesions in Pancreatic Duct (III) n CYST  HI P LO | HI S LO | HI NS LO n STENOSIS  NS S | NS G | NS P S | NS P I n STENOSIS  NG S | NI NS I | NI S n DILATATION  S P NG | S G NS | S NS | G NS n BRANCH  I NI | I NS | S NG | G NI | G S NN | S NS NN n BRANCH  N G NG NI | I P NI NN | G P NN | G S NI NN Description of various forms of cysts, stenoses, shapes of branches and dilatations

121 n HI  I | G n LO  NI | NG n N  n | n N w sym := w sym + w n ; h sym := h sym + h n n NN  nn | nn NN w sym := w sym + w nn ; h sym := h sym + h nn n I  i | i I w sym := w sym + w i ; h sym := h sym + h i n NI  ni | ni NI w sym := w sym + w ni n G  g | g G w sym := w sym + w g ; h sym := h sym + h g n NG  ng | ng NG w sym := w sym + w ng n S  s | s S w sym := w sym + w s ; h sym := h sym + h s n NS  ns | ns NS w sym := w sym + w ns n P  p w sym := w sym + w p ; h sym := h sym + h p W sym, h sym denotes the width and the height of the symptom W sym, h sym denotes the width and the height of the symptom Definition of ascending and descending parts of the detected symptoms Attributed Grammar Describing Morphological Lesions in Pancreatic Duct (IV)

122 Sequential transducer foe semantic analysis of the shapes q i – denotes the i th state of transducer, i=1,2,3,4 – number of sinquad Q i / - denotes that in the i th state appear terminal belonging to i th sinquad and no symbol is writing to output ( - an empty symbol) Q i  j/ij - denotes that in the i th state appear terminal belonging to j th sinquad and sequence “ij” is written to output

123 Results of Understanding of the shape of Pancreatic Duct with Chronic Pancreatitis ns, ns, s, s, s, ns, s, p, s, ns, p, p, p, ns, p, p, n, g, ng, ni, ns, p, ns, s, i, ni, ni, p, ns p, s, p, p, i, i, p, ni, nn, nn, s, s, ns, p, p, s, ns, g, p, nn, nn, ns, s, g, s, ni, nn, ni, s, s, s, ns, nn, nn, ni, p Legend BRANCH DILATATION STENOSIS Sequence of terminal symbols for recognized lesions in the upper part of the graph Sequence of terminal symbols for recognized lesions in the lower part of the graph

124 Parser Control Table state 0 $accept : _SYMPTOM $end s shift 18 g shift 14 ii shift 13 n shift 19 ns shift 15 ng shift 16 ni shift 17. error SYMPTOM goto 1 CYST goto 2 STENOSIS goto 3 BRANCH goto 4 DILATATION goto 5 I goto 6 NI goto 10 G goto 7 NG goto 9 S goto 11 NS goto 8 N goto 12 state 1 $accept : SYMPTOM_$end $end accept. error... state 79 BRANCH : I NI S I P NN NS NN_ (35) NN : NN_nn nn shift 73. reduce 35 11/127 terminals, 14/200 nonterminals 57/400 grammar rules, 80/600 states 0 shift/reduce, 0 reduce/reduce conflicts reported 54/0 working sets used memory: states,etc. 969/0, parser 5200/61 57/450 distinct lookahead sets 15 extra closures 109 shift entries, 1 exceptions 62 goto entries 0 entries saved by goto default Optimizer space used: input 330/0, output 5200/ table entries, 0 zero maximum spread: 265, maximum offset: 265

125 Distribution of Sinquads in LSFD procedure Cyst_or_Ramification(): if (there is a terminal which belongs to I st sinquad or p - i k >2) then Symptom = Cyst; else Symptom = Ramification; procedure Thickening_or_Cyst(): if (w sym >3/2*h sym ) then Symptom = Thickening ; else Symptom = Cyst; p- i k stands for the presence of i terminal symbols between p and k terminal element in the input sequence

126 Representation of Lesions in LSFD Lesions are detected by an analysis of transitions between sinquads, e.g. transition indicates appearance of : II  IV lateral ramifications II  I  IV or II  IV cysts and dilatations (also II  III  IV in backward analysis) IV  II and IV  I  II stenoses (also IV  III  II in backward analysis)

127 Results of Recognition in Pancreatic Duct with Chronic Pancreatitis Area of a detected side ramification, thickenings & stenoses Original image Recognized symptoms

128 Area of a detected side ramification, cyst & thickening with stenoses Results of Recognition in Pancreatic Duct with Pancreas Cancer Original image Recognized symptoms

129 Next example Ureters...

130 Results of Straightening Transformation Original image Width graph

131 Scheme of Encoding Width Graphs of Ureters into Terminal Symbols Approximation of width graph Intervals for terminal symbols

132 STS = SYMPTOM SP: SYMPTOM  STENOSIS Symptom=Stenosis SYMPTOM  DILATATION Symptom=Dilatation STENOSIS  NV H V | NV V | NV H DILATATION  V H NV | V NV | V H V  v | v Vw sym := w sym + w v ; h sym := h sym + h v NV  nv | nv NVw sym := w sym + w nv ; h sym := h sym + h nv H  h | h Hw sym := w sym + w h ; h sym := h sym + h h Attributed Grammar Describing Morphological Lesions in Ureters G U = (V N, V T, SP, STS) V N = {SYMPTOM, STENOSIS, DILATATION, H, V, NV} V T = {h, v, nv}

133 Result of Recognition in Upper Segment of Ureter Area of a detected stenosis Original image Recognized lesions

134 Result of Recognition in Upper Segment of Ureter Area of a detected stenosis Original image Recognized lesions

135 Approximation of Renal Pelvis Skeletons

136 G edt =( , r, P, Z)     ,   = {pelvis_renalis, calix_major, calix_minor, papilla_renalis}   = {PELVIS_RENALIS, CALIX_MAJOR,CALIX_MINOR}  = {x, y, z} for y  (-30 , 30  ), x  (30 , 180  ), z  (-30 , -180  ), Z = {PELVIS_RENALIS}P: EDT-Graph Grammar Describing Morphology of Renal Pelvises

137 Grammar Describing Lesions in Coronary Arteries STS = SYMPTOM SP: SYMPTOM  STENOSIS Symptom=Stenosis STENOSIS  NV H V V  v | v Vw sym := w sym + w v ; h sym := h sym + h v NV  nv | nv NVw sym := w sym + w nv ; h sym := h sym + h nv H  h H | w sym := w sym + w h ; h sym := h sym + h h G ca = (V N, V T, SP, STS) V N = {SYMPTOM, STENOSIS, H, V, NV} V T = {h, v, nv, } for h  (-10 , 10  ), v  (11 , 90  ), nv  (-11 , -90  )

138 Result of Recognition in Coronographical Images

139 n As we show, automatic understanding of the images is possible and can be performed at last for selected images by means of: n Preprocessing and defining primitives for analyzed images n Defining proper context-free LALR(1) grammars: W language of shape feature description supporting analysis for Pancreatic Ducts W EDT-type graph grammar to determine the morphology for Renal Pelvis Ureters and Coronary Arteries n Parsing of language descriptions of the images for semantic understanding of the images content Summary

140 Our methods are useful for many kinds of Medical Imagings 1.Image Processing & Analysis 2.Visual Perception 3.CAD - Computer Aided-Diagnosis 4.Image Registration and Data Management 5.PACS - Picture Archiving and Communications Systems 6.3D-Reconstruction 7.Display technologies 8.Telemedicine 9.Therapy planning 10.etc. Medical Imaging = biocybernetics+medical informatics

141 Additional remark: digital images Additional remark: There are many sources of digital images and many databases in Web collecting such data: e-Learning multimedial resources Computer Aided Design Systems (CAD) Geoinformatics systems (digital maps, satellite imaging, GPS) Police, Military and Security personal databases (portraits, fingerprints) Business imaging and statistics, etc.... but this is definitely another story!

142 Selected references... Tadeusiewicz R., Ogiela M.R.: Artificial Intelligence Techniques in Retrieval of Visual Data Semantic Information. In: Menasalvas E., Segovia J., Szczepaniak P.S. (eds.): Advances in Web Intelligence, Lecture Notes in Artificial Intelligence, nr 2663, Springer Verlag, 2003, pp Ogiela M.R., Tadeusiewicz R.: Artificial Intelligence Structural Imaging Techniques in Visual Pattern Analysis and Medical Data Understanding, Pattern Recognition, Elsevier 2003, vol 36/10 pp Ogiela M. R., Tadeusiewicz R.: Syntactic reasoning and pattern recognition for analysis of coronary artery images, International Journal of Artificial Intelligence in Medicine (Elsevier), vol. 26, nr. 1-2, 2002, pp Ogiela M. R., Tadeusiewicz R.: Advanced Image Understanding and Pattern Analysis Methods in Medical Imaging. In: Younan N. (ed.): Signal and Image Processing, IASTED, ACTA Press Anaheim-Calgary-Zurich, 2002, pp Tadeusiewicz R., Ogiela M. R.: Automatic Understanding of Medical Images – New Achievements in Syntactic Analysis of Selected Medical Images, Biocybernetics and Biomedical Engineering, vol. 22, nr 4, 2002, pp

143 Recent book describing the details:


Download ppt "!!! Automatic Understanding of the Images Many techniques of medical computer vision, e.g. X-ray images processing, analysis and recognition are now."

Similar presentations


Ads by Google