1. Lecture 11, CS5671 Secondary Structure Prediction Progressive improvement –Chou-Fasman rules –Qian-Sejnowski –Burkhard-Rost PHD –Riis-Krogh Chou-Fasman rules –Based on statistical analysis of residue frequencies in different kinds of secondary structure –Useful, but of limited accuracy

2. Lecture 11, CS5672 Qian-Sejnowski Pioneering NN approach Input: 13 contiguous amino acid residues Output: Prediction of secondary structure of central residue Architecture: –Fully connected MLP –Orthogonal encoding of input, –Single hidden layer with 40 units –3 neuron output layer Training: –Initial weight between -0.3 and 0.3 –Backpropagation with the LMS (Steepest Descent) algorithm –Output: Helix xor Sheet xor Coil (Winner take all)

3. Lecture 11, CS5673 Qian-Sejnowski Performance –Dramatic improvement over Chou-Fasman –Assessment Q = 62.7% (Proportion of correct predictions) Correlation coefficient (Eq 6.1) –Better parameter because »It considers all of TP, FP, TN and FN »Chi-squared test can be used to assess significance –C  = 0.35; C  = 0.29, C c = 0.38; Refinement –Outputs as inputs into second network (13X3 inputs, otherwise identical) –Q = 64.3%; C  = 0.41; C  = 0.31, C c = 0.41

4. Lecture 11, CS5674 PHD (Rost-Sander) Drawback of QS method –Large number of parameters (10 4 versus 2 X 10 4 examples) leads to overfitting –Theoretical limit on accuracy using only sequence per se as input~ 68% Key aspect of PHD: Use evolutionary information –Go beyond single sequence by using information from similar sequences (Enhance signal-noise ratio; “Look for more swallows before declaring summer”) through multiple sequence alignments –Prediction in context of conservation (similar residues) within families of proteins –Prediction in context of whole protein

5. Lecture 11, CS5675 PHD (Rost-Sander) Find proteins similar to the input protein Construct a multiple sequence alignment Use frequentist approach to assess position-wise conservation Include extra information (similarity) in the network input –Position-wise conservation weight (Real) –Insertion (Boolean); Deletion (Boolean) Overfitting minimized by –Early stopping and –Jury of heterogeneous networks for prediction Performance –Q = 69.7%; C  = 0.58; C  = 0.5, C c = 0.5

6. Lecture 11, CS5676 PHD input Fig 6.2

7. Lecture 11, CS5677 PHD architecture Fig 6.2

8. Lecture 11, CS5678 Riis-Krogh NN Drawback of PHD –Large input layer –Network globally optimized for all 3 classes; scope for optimizing wrt each predicted class Key aspects of RK –Use local encoding with weight sharing to minimize number of parameters –Different network for prediction of each class

9. Lecture 11, CS5679 RK architecture (Fig 6.3)

10. Lecture 11, CS56710 RK architecture (Fig 6.4)

11. Lecture 11, CS56711 Riis-Krogh NN Find proteins similar to the input protein Construct a multiple sequence alignment Use frequentist approach to assess position-wise conservation BUT first predict structure of each sequence separately, followed by integration based on conservation weights

12. Lecture 11, CS56712 Riis-Krogh NN Architecture –Local encoding Each amino acid represented by analog value (‘real correlation’, not algebraic) Weight sharing to minimize parameters Extra hidden layer as part of input –For helix prediction network, use sparse connectivity based on known periodicity –Use ensembles of networks differing in architecture for prediction (hidden units) –Second integrative network used for prediction Performance –Q = 71.3%; C  = 0.59; C  = 0.5, C c = 0.5 –Corresponds to theoretical upper bound for a contiguous window based method

13. Lecture 11, CS56713 NN tips & tricks Avoid overfitting (avoid local minima) –Use the fewest parameters possible Transform/filter input Use weight sharing Consider partial connectivity –Use large number of training examples –Early stopping –Online learning as opposed to batch/offline learning (“One of the few situations where noise is beneficial”) –Start with different values for parameters –Use random descent (“ascent”) when needed

14. Lecture 11, CS56714 NN tips & tricks Improving predictive performance –Experiment with different network configurations –Combine networks (ensembles) –Use priors in processing input (Context information, non-contiguous information) –Use appropriate measures of performance (e.g., correlation coefficient for binary output) –Use balanced training Improving computational performance –Optimization methods based on second derivatives

15. Lecture 11, CS56715 Measures of accuracy Vector of TP, FP, TN, FN is best, but not very intuitive measure of distance between data (target) and model (prediction), and restricted to binary output Alternative: Single measures (transformation of above vector) Proportions based on TP, FP, TN, FN –Sensitivity (Minimize false negatives) –Specificity (Minimize false positives) –Accuracy (Minimize wrong predictions)

16. Lecture 11, CS56716 Measures of error/accuracy L p distances (Minkowski distances) –(  i |d i - m i | p ) 1/p –L 1 distance = Hamming/Manhattan distance =  i |d i - m i | –L 2 distance = Euclidean/Quadratic distance = (  i |d i - m i | 2 ) 1/2 Pearson correlation coefficient –  I (d i – E[d])(m i - E[m])/  d  m –(TP.TN – FP.FN)/(TP+FN)(TP+FP)(TN+FP)(TN+FN) Relative entropy (déjà vu) Mutual information (déjà vu aussi)