Presentation on theme: "Introduction to Business Analytics"— Presentation transcript:
1Introduction to Business Analytics Chapter 6: Neural Networks for Data MiningMatthew J. LiberatoreThomas CoghlanFall 2008
2Learning ObjectivesUnderstand the concept and different types of artificial neural networks (ANN)Learn the advantages and limitations of ANNUnderstand how backpropagation neural networks learnUnderstand the complete process of using neural networksAppreciate the wide variety of applications of neural networks
3Using Neural Networks to Predict Beer Flavors with Chemical Analysis Why is beer flavor important to the profitability of Coors?What is the objective of the neural network used at Coors?Why were the results of the Coors neural network initially poor, and what was done to improve the results?What benefits might Coors derive if this project is successful?What modification would you provide to improve the results of beer flavor prediction?
4Business Applications of Artificial Neural Networks (ANN) Many applications across all areas of businesstarget customers (CRM)bank loan approvalhiringstock purchasetrading electricityapproving loan applicationsfraud preventionpredicting bankruptcytime series forecasting
5What are Artificial Neural Networks? Artificial Neural Networks (ANN) are biologically inspired and attempt to build computer models that operate like a human brainThese networks can “learn” from the data and recognize patterns
6Basic Concepts of Neural Networks Biological and artificial neural networksNeuronsCells (processing elements) of a biological or artificial neural networkNucleusThe central processing portion of a neuronDendriteThe part of a biological neuron that provides inputs to the cell
7Basic Concepts of Neural Networks Biological and artificial neural networksAxonAn outgoing connection (i.e., terminal) from a biological neuronSynapseThe connection (where the weights are) between processing elements in a neural network
10Relationship Between Biological and Artificial Neural Networks Soma – NodeDendrites – InputAxon – OutputSynapse – WeightANNs typically have much fewer neurons than humans
11Basic Concepts of Neural Networks BackpropagationThe best-known learning algorithm in neural computing. Learning is done by comparing computed outputs to desired outputs of historical casesNetwork structure (three layers)InputIntermediate (hidden layer)Output
13Basic Concepts of Neural Networks Network information processingInputsOutputsConnection weightsSummation function (combination function)Transformation function (activation function)
14Basic Concepts of Neural Networks Loan Processing ExampleInputs – income level, age, home ownership nodesOutputs – approval/disapproval of loan nodes
15Basic Concepts of Neural Networks Connection weightsThe weight associated with each link in a neural network modelstrength of the data transferred between layers in the networkThey are assessed by neural networks learning algorithms
16Basic Concepts of Neural Networks Transformation function (activation function)maps the summation (combination) function onto a narrower range ( 0 to 1 or -1 to 1) to determine whether or not an output is produced (neuron fires)The transformation occurs before the output reaches the next level in the networkSigmoid (logical activation) function: an S-shaped transfer function in the range of zero to one –exp(x)/(1-exp(x))Threshold value is sometimes used instead of a transformation functionA hurdle value for the output of a neuron to trigger the next level of neurons. If an output value is smaller than the threshold value, it will not be passed to the next level of neurons
19Learning in ANN Learning algorithm The training procedure used by an artificial neural networkSupervised learningA method of training artificial neural networks in which sample cases are shown to the network as input and the weights are adjusted to minimize the error in its outputs
21Learning in ANN How a network learns Backpropagation The best-known supervised learning algorithm in neural computing. Learning is done by comparing computed outputs to desired outputs of historical cases
22Learning in ANN How a network learns Procedure for a learning algorithmInitialize weights with random values and set other parametersRead in the input vector and the desired outputCompute the actual output via the calculations, working forward through the layersCompute the errorChange the weights by working backward from the output layer through the hidden layers
24Error calculation and weights At each hidden node and target node: compute:Linear combination function: C = w0 + w1x1 +…+ wnxnLogistic activation function: L = exp(C)/(1+exp(C)At the target node compute Bernoulli error function: sum errors over all observations, where the error is -2 ln (L) if there is a response, or -2 ln (1 – L) if there is no responseIn the first iteration, random weights are usedIn subsequent iterations, the weights are changed by a small amount so that the error is reducedThe process continues until the weights cannot be reduced further
25Developing Neural Network–Based Systems Data collection and preparationThe data used for training and testing must include all the attributes that are useful for solving the problemRecall the bankruptcy prediction problem we modeled using logistic regression -- The same data can be used to train a neural network:working capital/total assets (WC/TA)retained earnings/total assets (RE/TA)earnings before interest and taxes/total assets (EBIT/TA)market value of equity/total debt (MVE/TD)sales/total assets (S/TA)
26Developing Neural Network–Based Systems Selection of network structureDetermination of:Input nodesOutput nodesNumber of hidden layersNumber of hidden nodesFor the bankruptcy problem (and all of our examples) we have one hidden layerThe Bankruptcy problem has ten nodes in the hidden layer – sometimes one might experiment with the number of nodes
28Developing Neural Network–Based Systems Learning algorithm selectionIdentify a set of connection weights that best cover the training data and have the best predictive accuracyNetwork trainingAn iterative process that starts from a random set of weights and gradually enhances the fitness of the network model and the known data setThe iteration continues until the error sum is converged to below a preset acceptable level
29Developing Neural Network–Based Systems TestingBlack-box testingComparing test results to actual resultsThe test plan should include routine cases as well as potentially problematic situationsIf the testing reveals large deviations, the training set must be reexamined, and the training process may have to be repeatedMight compare ANN results with other methods such as logistic regression
30Developing Neural Network–Based Systems Implementation of an ANNImplementation often requires interfaces with other computer-based information systems and user trainingOngoing monitoring and feedback to the developers are recommended for system improvements and long-term successIt is important to gain the confidence of users and management early in the deployment to ensure that the system is accepted and used properly
32Neural Networks In SAS Enterprise Miner 5.3 In your bankrupt project, create a new diagram called bankrupt_neuralDrag the bankrupt data node onto your diagramFrom the Model tab, drag the Neural Network node onto the diagram and connectConnect the data node to the Neural Network Node
33Highlight the Neural Network node Highlight the Neural Network node. In the property panel window, set model selection criterion to average error
34In the Property Panel window, click on the square to the right of network and change the defaults for the Target Layer Combination, Activation, and Error functions as indicated. Note that we are using the default of 3 hidden units (nodes).
35The results show an excellent fit with the cumulative lift equal to the best cumulative lift, no misclassifications, and an average error nearly zero.
36In the Property Panel click on the box to the right of Exported Data to see the individual predictions and probabilities. The logistic activation function at the target level provides the probabilities, like those obtained from logistic regression
37Similar to what we did with logistic regression add the bankruptscore data node and the score node to the diagram as shown.
38After running the score node, the output shows that 6 firms are predicted to go bankrupt (vs. 4 under logistic regression)
39For details about the individual predictions, highlight the Score node and on the left-hand panel click on the square to the right of Exported Data. Then in the box that appears click on the row whose Port entry is Score. Then click on Explore.