1. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 1 / 45 A General Neural Network Model for Estimating Telecommunications Network Reliability Instructor : Frank Yeong-Sung Lin, Ph.D. Students: D96725001 陳君銘, D97725002 黃健誠 2009-06-08 Journal: IEEE Transactions on Reliability, Vol. 58, No. 1, March 2009. Authors: Fulya Altiparmak, Berna Dengiz, and Alice E. Smith, Senior Member of IEEE.

2. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 2 / 45 Author Home page for Professor Smith: http://www.eng.auburn.edu/~aesmith/ Alice E. Smith is Professor and Chair of the Industrial and Systems Engineering Department at Auburn University.

3. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 3 / 45 References D. W. Coit and A. E. Smith, “Solving the redundancy allocation problem using a combined neural network/genetic algorithm approach,” Computers and Operations Research, vol. 23, no. 6, pp.515–526, 1996. B. Dengiz, F. Altiparmak, and A. E. Smith, “Efficient optimization of all-terminal reliable networks, using an evolutionary approach,” IEEE Trans. on Reliability, vol. 46, pp.18–26, 1997. C. Srivaree-ratana, and A. E. Smith, “Estimating All-Terminal Network Reliability Using a Neural Network,” Proceedings of the 1998 IEEE International Conference on Systems, Man, and Cerbernetics, San Diego, CA, 1998, vol. 5, pp.4734-4740. C. Srivaree-ratana, A. Konak, and A. E. Smith, “Estimation of All-terminal network reliability using an artificial neural network,” Computers and Operations Research, vol. 29, pp. 849–868, 2002. F. Altiparmak, B. Dengiz, and A. E. Smith, “Reliability estimation of computer communication networks: ANN models,” Proceedings Eighth IEEE International Symposium on Computers and Communication (IEEE ISCC’03), Antalya-Kemer, Turkey, 2003, vol. 2, pp.1353–1358.

4. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 4 / 45 Outline Abstract Introduction Artificial Neural Networks The General ANN Method Application to Real Communications Systems Conclusions Appendix Demo Comments

5. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 5 / 45 Reliability( 可靠度 ) The Advisory Group on Reliability of Electronic Equipment (AGREE), established on August 21, 1952. AGREE, “Reliability of Military Electronic Equipment,” Advisory Group on the Reliability of Electronic Equipment (AGREE), U.S. Government Printing Office, Washington, DC. Foundation work on reliability theory, June 4, 1957. Reliability is defined as the ability of a system or component to perform its required functions under stated conditions for a specified period of time. 「產品於既定的時間內，在特定的使用環境條件下，執行 特定的功能，成功完成工作目標的機率」。

6. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 6 / 45 Outline Abstract Introduction Artificial Neural Networks The General ANN Method Application to Real Communications Systems Conclusions Appendix Demo Comments

7. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 7 / 45 Abstract This paper proposes a new encoding method for using neural network models to estimate the reliability of telecommunications networks with identical link reliabilities. Drawback of previous approaches: Long vector length of the inputs required to represent the network link architecture. The specificity of the neural network model to a certain system size. This study demonstrates both the precision of the neural network estimate of reliability, and the ability of the neural network model to generalize to a variety of network sizes (small to large scale communications networks).

8. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 8 / 45 Outline Abstract Introduction Artificial Neural Networks The General ANN Method Application to Real Communications Systems Conclusions Appendix Demo Comments

9. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 9 / 45 Introduction The exact calculation of all-terminal network reliability is an NP-hard problem, with computational effort growing exponentially with the number of nodes and links in the network. There exists no algorithm with a polynomial time to compute all-terminal network reliability. Because of the impracticality of calculating all-terminal network reliability for networks of moderate to large size, Monte Carlo simulation methods to estimate network reliability and upper and lower bounds to bound reliability have been used as alternatives. Valiant, L.G., “The Complexity of Enumeration and Reliability Problems,” SIAM Journal on Computing, Vol. 8, Issue 3, pp.410-421, 1979. Provan, J. S. and M. O. Ball, “The complexity of counting cuts and of computing the probability that a graph is connected,” SIAM Journals on Computing, Vol. 12, Issue 4, pp. 777–788, 1983.

10. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 10 / 45 Introduction (cont’d) In this study, a generalized artificial neural network (General ANN) is proposed to estimate all-terminal network reliability for networks. This study use an input encoding that, unlike previous approaches, does NOT rely on a vector of all possible links between nodes. Advantages: The first is that a single ANN model can be used for multiple network sizes and topologies. The second advantage is that the input information to the ANN is compact, which makes the method tractable, even for large sized networks.

11. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 11 / 45 Outline Abstract Introduction Artificial Neural Networks The General ANN Method Application to Real Communications Systems Conclusions Appendix Demo Comments

12. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 12 / 45 Artificial Neural Network An ANN has the ability to learn relationships between given sets of input and output data by changing the weights. This process is called: training the ANN (Back Propagation).

13. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 13 / 45 Artificial Neural Network (cont’d) The performance of the ANN model is a function of several design parameters such as the number of hidden layers, the number of hidden neurons in each hidden layer, the size of the training set, and the training parameters. The technique of k-fold cross validation is particularly useful because it makes the most of a limited size data set. The data set is divided (randomly) into multiple sets (cross validation would include sets, where is the data set size, while grouped cross validation would include sets).

14. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 14 / 45 Outline Abstract Introduction Artificial Neural Networks The General ANN Method Application to Real Communications Systems Conclusions Appendix Demo Comments

15. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 15 / 45 The General ANN Method This study identified compact, easily calculated measures of network connectivity and reliability as the candidate set of inputs: ND: of each node, 0 if the node is not present ND min : minimum node degree of the network ND med : median node degree of the network ND max : maximum node degree of the network LR: link reliability NL: number of links C: link connectivity UB: network reliability upper bound

16. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 16 / 45 The General ANN Method 5 input configurations were studied: 1) ND, LR, UB 2) ND, NL, LR, UB 3) ND, C, LR, UB 4) ND, NL, C, LR, UB 5) ND min, ND med, ND max, NL, C, LR, UB.

17. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 17 / 45 The General ANN Method (cont’d) 5 input configurations: 22 23 24 7 Input neurons

18. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 18 / 45 The General ANN Method (cont’d) The output of the ANN is the estimation of all-terminal network reliability (one real valued neuron). The target network reliability of each network is estimated using a Monte Carlo simulation method. This study used randomly generated data sets for training and validation considering five different link reliabilities (0.80, 0.85, 0.90, 0.95, and 0.99), and five different link connectivity values (1 to 5), so that there are 25 design points. RMSE: root mean squared error. MAD: mean absolute deviation

19. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 19 / 45 The General ANN Method (cont’d) A preliminary experimental study, the number of hidden neurons, and training data size were set to 15, and 2400, respectively. The model was validated using 5-fold cross validation, where each validation network was trained and tested using 2400, and 600 observations, respectively. A final application network was trained using all members of the data set, i.e. 3000 observations, and its validation was inferred using the average of the prediction error of the five validation networks.

20. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 20 / 45 Comparison of Input Data Groupings This study give the results of 5-fold cross validation for the neural networks considering the 5 different configurations. The ERROR used to calculate RMSE is the difference between the Monte Carlo simulation, and the Neural Network estimation of the all-terminal network reliability. RMSE: root mean squared error

21. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 21 / 45 Comparison of the General ANN (GNN), and Specific ANN (SNN) Models All RMSE values of the General ANN are smaller than the specific ANN, and the UB. RMSE: root mean squared error MAD: mean absolute deviation

22. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 22 / 45 The Performance of the General Model on New Network Sizes This table shows no systematic error patterns in terms of network size. It appears that the General ANN can be used to estimate all- terminal network reliability for any network size from 10 to 20 nodes with similar estimation error.

23. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 23 / 45 Scaling Up to Large Networks This table gives 5-fold cross validation results for large networks (30, 35, 40 nodes). While the average RMSE value is 0.04325 for the General ANN, it is 0.08504 for the upper bound.

24. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 24 / 45 Scaling Up to Large Networks (cont’d) This table gives the RMSE, and MAD values. See that there are no systematic error patterns in terms of network size. These results show that the scale up of the General ANN approach is good, and that this approach is viable for networks of realistic size.

25. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 25 / 45 Outline Abstract Introduction Artificial Neural Networks The General ANN Method Application to Real Communications Systems Conclusions Appendix Demo Comments

26. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 26 / 45 Application to Real Communications Systems This study considered three real networks to better investigate the effectiveness of the General ANN approach. These are Authorized licensed Arpanet, the European Optical Network, and the communications network of Gazi University.

27. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 27 / 45 Application to Real Communications Systems (cont’d) The General ANN estimates of system reliability were compared with the actual system reliability (using Monte Carlo simulation), the upper bound of system reliability, and the estimate of system reliability using a specific ANN developed for that network architecture. The General ANN developed expressly for performed very well, better than both the UB, and the Specific ANN.

28. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 28 / 45 Outline Abstract Introduction Artificial Neural Networks The General ANN Method Application to Real Communications Systems Conclusions Appendix Demo Comments

29. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 29 / 45 Conclusions This study presented a novel method of encoding communications networks for ANN that accommodates networks of varying node and link sizes. Single ANN model can be used for multiple network sizes and topologies. In design optimization, one might use the General ANN for screening many designs to gauge the trade off between system reliability and cost. An exact method or computationally laborious Monte Carlo simulation should be used on the final few candidate network designs to ascertain the precise system reliability.

30. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 30 / 45 Outline Abstract Introduction Artificial Neural Networks The General ANN Method Application to Real Communications Systems Conclusions Appendix Demo Comments

31. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 31 / 45 Appendix This study must note that there are different network topologies which yield the same values of the inputs to the General ANN. Authors studied this aspect by generating some differing networks with the same number of nodes, links, node degrees, link reliabilities, etc.; but which have different all terminal network reliability.

32. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 32 / 45 Appendix The General ANN approach CANNOT discriminate among networks with the same topological inputs.

33. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 33 / 45 Appendix The reliabilities estimated by the UB, Monte Carlo simulation (MC), the General ANN (GNN) approach, and a specific ANN (SNN) trained for that topology.

34. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 34 / 45 Outline Abstract Introduction Artificial Neural Networks The General ANN Method Application to Real Communications Systems Conclusions Appendix Demo Comments

35. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 35 / 45 Demo ANN – Using Qnet

36. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 36 / 45 Qnet Training Mode Network Design ：決定隱藏層 數及輸入、輸出層單元數，且 可決定轉換函數。

37. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 37 / 45 Qnet (cont’d) Training Data ：「 Input Data File 」選取文字檔案位置， 並選取輸入層起始欄位， 如果資料非經過正規化處 理，則不勾選「 Normalize Input 」。

38. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 38 / 45 Qnet (cont’d) Input Data File

39. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 39 / 45 Qnet (cont’d) Training Parameters

40. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 40 / 45 Qnet (cont’d) Training

41. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 41 / 45 Qnet (cont’d) Result ： 前為目標值；後為預測值

42. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 42 / 45 Qnet (cont’d) Recall Mode

43. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 43 / 45 Outline Abstract Introduction Artificial Neural Networks The General ANN Method Application to Real Communications Systems Conclusions Appendix Demo Comments

44. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 44 / 45 Comments This paper presented a novel method of encoding communications networks for ANN. Experimental results shows that the General ANN is equivalent or superior in estimation accuracy to ANN models developed for a specific sized network. The study elaborated more detail about the experimental results of the proposed method and the compared methods. The experiment is fair to use k-fold cross validation. The authors considered three real networks to better investigate the effectiveness of the General ANN approach.

45. 20090608.ppt © NTU-IM-OPLab. (Chien-Cheng Huang) 45 / 45 Thank you !