1. International Workshop
on Inductive Modeling IWIM’2017
Structure-Oriented Classifiers in Feature Space of Objects Defined by Sets of Measurement in Biology and Medicine
Ievgen Nastenko
Oleksandra Konoval
Olena Nosovets
Volodymyr Pavlov
NTUU ‘KPI’, Kyiv, Ukraine
One of the most important problems in medical science, biology and image processing is classification problem class for objects, which are defined by sets of multidimensional measurement. Set, which characterizes an object, can be a result of original feature measurements, registered in various observation condition or of multiple monitoring.
September 5-8, 2017 Lviv, Ukraine

2. Content Problem statement Problem solution Real task calculation
Results
Conclusions
According to this fact in the paper an optimal model structure of classified objects is suggested to obtain for each class to solve the problem. To construct the structures, a new version of GMDH algorithm was developed having a new combined external criterion that includes accuracy and division ability of objects structure of each class. 2
Lviv IWIM Ukraine

3. Problem statement
The fact of existence of some set of classes is given. It is supposed that any classified object could be given by the set of feature vectors in finite dimensional space. A partial intersection of the sets belonging to different classes is possible. It is necessary to develop the best classification rule for dividing the objects of initial sets.
Fig.1 – Objects in the initial space of characteristics
Fig.2 – Object in the space of model parameters
The challenge was inspired by a real-world classification problem of determining the human cardiovascular system functional state based on the measurements of pulse and pressure in different circumstances. To solve the problem, the best structure was suggested to find using a range of GMDH [1] algorithms. The structure is used to build the models of all the classified objects. Then, the problem solution was proposed to find in the parameters space of the found structure [2, 3]. In the sequel, a new recurrent additive-multiplicative multistage algorithm (RAMMA) of GMDH was developed to determine the optimal structure [4]. The accuracy combined criterion and the divisibility criterion were proposed as external criteria here. Here on this picture
On this slide objects from the initial space of characteristics and objects obtained in the new parameter space example is demonstrated.

4. Problem solution
RAMMA algorithm finds the best structure as a part of complete polynomial. Obtained structures are forming more complicated descriptions in step-by-step manner. The best generalized variable is added into the structure as a part of the polynomial (3) on each stage according to the value of the external criterion.

5. Problem solution RAMMA stages: Initial matrix extension
General variables matrix development
Complete information matrix development
Structures generation and parameters evaluation
The best structure selection based on external criterion values
RAMMA modification includes the following stages.
Initial matrix extension
General variables matrix development
Complete information matrix development
Structures generation
The best structure selection based on external criterion values

6. Problem solution
Therefor, RAMMA algorithm starts with some data preparation. First stem is optional. Initial matrix can be extended by adding some extra variables – input values in different powers (-1, 1/3, -1/3). This approach is used to provide better accuracy of future model.
After extension is made matrix of general variables is found. Each general variable is one of terms of full polynomial and one of possible products of excising variables raised to a power limited by maximum polynomial power 𝑝.

7. Problem solution Z3 Z2 Z1 1 2 3
For example, if there are 𝑚 variables in initial extended matrix then the best way to generate general variables is to find out 𝑆=𝐶_(𝑚+𝑝)^𝑚−1 sequential natural numbers and transform them to (𝑝+1) notation but ignore those of them for which sum of digits is more then 𝑝. Obtained 𝑆 numbers in (𝑝+1) notation are general variables where bit number corresponds to variables from extended matrix and bit value demonstrates power of each extended variable in current general variable.
After general variables matrix was prepared complete information matrix can be calculated. This allows to avoid repeating calculations in further.

8. Problem solution
Model generation on each stage is performed in step-by-step manner. General variable determined at the current stage is joint to previously obtained polynomial.
To form generalized variables and create new structures it is proposed to move through the sequence up in such a way that firstly the numbers with the sum of digits of p = 1 are considered, then p = 2, and so on. There are two parameters in the algorithm that control the number of structures surviving at a stage, F1 and F2. The F1 is used at the first stage and the F2 at the others. Additionally, the number of best structures F is retained and is given as the result when the algorithm stops.

9. Problem solution
The best structure selection based on external criterion values. Previously two variants of external criterion were developed:
Combined accuracy criterion
Divisibility criterion
The best structure selection is based on external criterion values. Previously two variants of external criterion were developed
Combined accuracy criterion
Divisibility criterion

10. Problem solution
Combined accuracy criterion based on relative standard root-mean-square (RMS) errors values
for current model.

11. Problem solution
Second variant of external criterion (9) is based on values of dispersion in classes and between classes. For both of these external criteria structure quality is defined by minimum of criterion value.

12. Real task calculation
Initial data includes 96 patients with variety of measurements of systolic pressure, diastolic pressure and heart rate which belongs to one of 5 classes.
Class No 1 2 3 4 5
Class name
Arterial hypertension
Isolated systolic hypertension
Normal regulation
Heart failure of low degrees
Heart failure of high degrees
Number of objects 10 11 25 22 28
Min number of measurements
Max value of measurements
151
114 42 45 41
Monitoring data of five different groups was used in the research. First group consists of monitoring records for 10 patients with arterial hypertension. Second group represents 11 patients with isolated systolic hypertension. Third group includes 26 patients with normal regulation. Fourth and Fifth groups includes 22 and 28 patients with cardiac insufficiency of low degree and of high degree.

13. Results
Combined accuracy criterion allowed to get average (by classes) classification accuracy of 79,2%. Obtained structure is:
𝐷𝐼𝐴= 𝑎 0 + 𝑎 1 ∙ 1 𝐻𝑅 ∙𝑆𝑌𝑆∙ 3 𝑆𝑌𝑆
Classifier for second class (Isolated systolic hypertension) is:
𝑌=1,25964− 50,8624 𝑥 𝑎 0
67,7% classification accuracy. 100% correctly classified “own" objects.
Divisibility criterion allowed to get average (by classes) classification accuracy of 82,5%. Obtained structure is:
𝐷𝐼𝐴= 𝑎 0 + 𝑎 1 ∙ 𝑆𝑌𝑆 2 𝐻𝑅
𝑌=−0, , ∙ 𝑥 𝑎 ,6143∙ 𝑥 𝑎 1
73,96% classification accuracy. 90,9% correctly classified “own” objects.
Classifiers system construction was performed in the objects parameters space using the algorithms of GMDH Shell software [5]. The classifiers system obtained applying the combined accuracy criterion allows to get average (by classes) classification accuracy of 79.2%. When the divisibility criterion was used in RAMMA, the accuracy was 82.5%.

14. Results
In general case, the classes specificity, obviously, may require describing different classes objects by different structures, despite the quite high classification indices obtained. This calls to suggest new approaches to obtaining optimal classes structures as well as ways of their application to objects classification.
A new variant of external criterion was proposed. It takes into account the RMS and the dispersion components and should be calculated separately for each class.
Where 𝛽 defines the weighs ratio of the criterion components.
Alpha coefficient allows accounting the object dispersion relative to the classes centers with a different power when the structure is determined.

15. Results The model obtained for the second class is:
𝐷𝐼𝐴= 𝑎 0 + 𝑎 1 ∙ 1 3 𝐻𝑅 ∙ 𝑆𝑌𝑆 2
Classifier for second class (Isolated systolic hypertension) is:
𝑌=0, 𝑎 0 ∙ 𝑎 1 ∙1,34134∙ 𝑎 1 ∙1,34134∙ 10 7
83,1% classification accuracy. 90,9% correctly classified “own” objects.
The appropriate model was found and classification was performed for one of the worst classes – second (Isolated systolic hypertension). New criterion allows to obtain 81,3% classification accuracy. This is 13,6% more accurate result than for combined accuracy criterion, and 7,3% better result than divisibility criterion allows to obtain.

16. Conclusions
A new approach to solving the classification problem for objects that are given by a set of observations was proposed. The approach is based on conversion of the problem solution from the initial features space into the space of objects model structure parameters of identifiable class.
To implement the idea, a new version of RAMMA algorithm was developed. The algorithm uses an external criterion that combines the accuracy of object representation by a model and divisibility capabilities of the structure in the model parameters space of the classified objects.
This new approach allowed to get on specific class 81,3% classification accuracy which is 13,6% and 7,3% more accurate result then previous criteria allowed to get.
A new approach to solving the classification problem for objects that are given by a set of observations was proposed. The approach is based on conversion of the problem solution from the initial features space into the space of classified objects model structure parameters.
To implement the idea, a new version of RAMMA algorithm was developed. The algorithm uses an external criterion that combines the accuracy of object representation by a model and divisibility capabilities of the structure in the model parameters space of the classified objects.
This new approach allowed to get on specific class 81,3% classification accuracy which is 13,6% and 7,3% more accurate result then previous criteria allowed to get.

17. Thank you for attention! E-mail: alexandra.konoval@gmail.com17
Lviv IWIM Ukraine