PANN Testing

Training Time and Error Benchmarking Target error: 0.02 (Mean Squared Error)

Training Time and Error Benchmarking Progress ANN reduces its training error to desired minimum in less than one second. Perceptron and Back Propagation ANNs have substantial error, which reduces slowly. Perceptron and Back Propagation ANNs show no tendency to reach target error. Tested training set: 30,000 images 60 Perceptron 50 40 Mean Squared Error (x 100) 30 Back propagation 20 Progress ANN 10 Target error: 0.02 50 250 1250 6250 31250 156250 781250 Time (milliseconds)

Progress P-Net Creator NeuroSolution Data Manager Additional Comparison to Alternatives Progress P-network NeuroSolutions Parameters Progress P-network NeuroSolution Data Manager Standard deviation 0.0035 0.011 Working time 3.297 sec 1938 sec = 32 minutes 18 sec Number of epochs 8 44000 Progress P-Net Creator NeuroSolution Data Manager

Training advantage factor = Additional Comparison to Alternatives Number of: records, images, lines, samples, data set and sample data are used synonymously. Comparison of training time between IBM SPSS Statistics 22 and P-network, for the same problem, tested on Apple iMac 27" 3.5 GHz quad-core Intel Core i7 8GB of 1600MHz DDR3 memory; SSD 14000 IBM SPSS Statistics 22 Exponential growth of training time 12000 10000 Network Images Training time Progress P-network 7000 4 sec. IBM SPSS Statistic 22 13400 = 3h. 43min. 8000 Time in seconds 6000 4000 2000 𝟏𝟑𝟒𝟎𝟎 𝒔𝒆𝒄 𝟒 𝒔𝒆𝒄 =𝟑𝟑𝟓𝟎 Training advantage factor = Number of images 1000 2000 3000 4000 5000 6000 7000 4 Progress P-network Linear growth of training time 3 Time in seconds 2 1 Number of images 1000 2000 3000 4000 5000 6000 7000 с Progress, Inc. 2013

PANN vs. Classical Neural Network Theoretically unlimited intelligence Long training time PANN Practically unlimited intelligence Quick training Human brain Training Time 10 1013 1012 1011 1010 109 107 108 106 105 104 103 102 Classical neural network PANN Hardware 1 hour 1 minute 24 hour 1 month 1 year 1 million years 1 thousand years Network Intelligence 1014 Network Intelligence is proportional to the number of elements and problems at hand Seconds PANN Software PANN requires minutes (at most - hours) to reach the level of network intelligence unreachable for classical neural networks for thousands of years с Progress, Inc. 2013

Test on images compression Files with images from CIFAR-10 test site No compression 20000 MSE:0.013 MSE:0.006 15000 MSE:0.005 MSE:0.011 Economy of disk space Number of color images in a file 10000 MSE:0.008 MSE:0.004 PANN with 50 weights on a synapse PANN with 100 weights on a synapse 5000 1000 100 1 50 100 150 200 250 300 File size on disk, MB

#2 GPUs breakthrough - 2 Amdahl’s Law and P-net PANN simple matrix algebra mathematics allows 100% parallel processing. Thus, speed increases linearly with additional GPUs and CPUs. Progress US patent application 15/449,614 that covers matrix algebra application with PANN, is filed on March 3, 2017. P-net application This allows building computers and other electronics with: very high processing speed, and reduced number of GPUs and CPUs

Trading speed: Comparison of PANN and nVidia cuDNN #2 GPUs breakthrough - 2 Trading speed: Comparison of PANN and nVidia cuDNN PANN training speed on CPU and GPUs is thousand times higher than of existing ANN. Allows to: Improve ANN training speed thousands times Build supercomputer on GPUs Build Hypercomputer on GPUs PANN provides 60 (threads on GPU) x = 201 000 x. Acceleration is proportional to the number (N) of GPUs: 201 000 x N

Comparison of P-net with CPU/GPU Training time, in msec inputs outputs images CPU GPU Log CPU time Log GPU time difference times 10.00 - 3.00 100.00 2.00 3.10 0.30 0.49 - 1.10 0.65 1 000.00 321.00 19.90 2.51 1.30 301.10 16.13 5 000.00 7 872.00 271.00 3.90 2.43 7 601.00 29.05 CPU - central processing unit GPU - graphics processing unit 1 sec = 1000 msec Testing computer with CPU speed/ GPU speed = 4 time, log

Comparison of P-net with CPU/GPU training time, in msec inputs outputs images CPU, ms GPU log CPU time log GPU time difference times 100.00 10.00 2.00 69.10 0.30 1.84 - 67.10 0.03 1 000.00 28.00 69.30 1.45 - 41.30 0.40 100 000.00 3 719.00 86.40 3.57 1.94 3 632.60 43.04 500 000.00 18 440.00 125.30 4.27 2.10 18 314.70 147.17 CPU - central processing unit GPU - graphics processing unit time, log