1. Some Applications of Neural Networks in Plasma Physics and Fusion Materials Modelling R. Kemp 1, G. Cottrell 2, and H. K. D. H Bhadeshia 1 University of Cambridge, Department of Materials Science and Metallurgy, Phase Transformations & Complex Properties Research Group. E-mail: rk237@cam.ac.uk 2 EURATOM/UKAEA Fusion Association, Culham Science Centre This work is sponsored by the EPSRC and Culham Labs. For more information see www.msm.cam.ac.uk/phase-trans and www.fusion.org.ukwww.msm.cam.ac.uk/phase-transwww.fusion.org.uk Introduction There are problems in plasma physics and controlled-fusion research involving a large number of significant variables; the phenomena may also be extremely complex. Examples include : the prediction of confinement properties and disruptions in a tokamak, and the behaviour of candidate fusion materials under 14 MeV neutron irradiation from D-T reactions in the plasma. Models based on fundamental principles may be overwhelmed by the complexity, requiring simplification to such an extent that they cease to be effective. Simplification can also lead to a loss of information that may be vital to the original issue. In particular, subtle nonlinear interactions between the important variables may be lost. In these cases, artificial neural networks (ANNs) can be used to capture relationships between controlling variables in complex empirical multi-variate data and hence provide quantitative predictions. In addition, ANNs typically have fast processing speeds and so could be applied to control problems involving real-time measurement and feedback. Real- time temperature measurements, for example, have been taken for temperature diagnostics in JET, and could be used as inputs for an ANN. References C M Bishop, Neural Networks For Pattern Recognition, Edition 1 (1995), Clarendon Press E E Bloom, The Challenge of Developing Structural Materials for Fusion Power Systems, J. Nucl. Mater. 258 - 263 (1998) 7 - 17 D MacKay, Information Theory, Inference, and Learning Algorithms, 1st Edition (2004) L Allen and C M Bishop, Neural Network Approach to Energy Confinement Scaling in Tokamaks, Plasma Phys. Control. Fusion 34 (7) (1992) 1291 - 1302 J Svensson et al, Analysis of JET Charge Exchange Spectra Using Neural Networks, Plasma Phys. Control. Fusion. 41 (1999) 315 - 338 A schematic diagram of a three layer feed- forward network of the type used in MacKay’s framework. The model’s complexity is controlled by the number of neurons in the second layer, known as hidden units. Modelling Fusion Materials Neural network modelling has been successfully used to predict material properties of conventional steels. However, such predictions require large databases. For fusion materials, there is much ongoing work to find these data, and neural network models can help to pinpoint the most fruitful areas of research by predicting relationships between inputs and properties. For example, a model trained on radiation hardening in reduced activation ferritic/martensitic (RAFM) steels could be used to optimise experiments in the proposed IFMIF facility. It can be demonstrated that changes in yield stress have a strong dependence on dose rate, and occur fastest at low doses. However, the uncertainty is very large in higher dose, fusion-relevant regimes. Identifying these regions of maximum uncertainty, in combination with identifying regions of rapid variation in the target property, can be used to compose a maximally informative experimental matrix. Neural Network Structure A sufficiently complex ANN can ‘mimic’ any other continuous differentiable function. Thus, ANNs provide a highly flexible non-linear regression method, which avoids many of the problems encountered in linear regression. They are, however, susceptible to overfitting and there is nothing within their structure which restricts unjustified extrapolation. These problems are avoided here by employing MacKay’s Bayesian framework which elegantly addresses both of these difficulties. Predictions made using ANN models constructed in this way are accompanied by a measure of modelling uncertainty, which increases where data are particularly noisy or sparse. In addition, the algorithm has the option of being trained assuming a specified level of noise in the input data, which can also prevent overfitting and produce a more general model. Modelling Plasma Physics Using ANNs to model problems in plasma physics is not a new idea. However, previous approaches have had difficulty quantifying the effects of noise in the input dataset, or in extrapolation between data clusters or beyond the bounds of the training dataset. To avoid these problems, roundabout techniques such as the use of “novelty-detection” algorithms to identify such situations have been used. Nevertheless, ANNs have been shown to be more effective - producing a better, more flexible fit - than linear regression in modelling behaviour such as energy confinement scaling in tokamaks, and charge exchange spectra. It is hoped that a Bayesian approach will allow the refinement of models such as these, potentially providing a powerful diagnostic and control tool. ANN predictions for radiation hardening of F82H, a RAFM steel. The data plotted were “unseen” - they were not included in the training database. The large modelling uncertainty for the irradiated predictions reflects the sparse nature of the training database. Graph of global confinement time,  E, against the paramter  for ANN predictions (solid line) and linear regression (dotted line). The ANN model shows behaviour which is not contained in the linear regression model (from Allen and Bishop (1992)). Conclusions Bayesian neural networks offer advantages over linear regression and other neural network techniques. Briefly, these advantages are: Very flexible fit to data Automatic detection of noisy or sparse datasets No need to make assumptions about functional form of fitting curve It is anticipated that applying this approach to complex non-linear modelling problems will yield many benefits.