1. Reverse engineering gene and protein regulatory networks using Graphical Models. A comparative evaluation study. Marco Grzegorczyk Dirk Husmeier Adriano Werhli

3. Systems biology Learning signalling pathways and regulatory networks from postgenomic data

5. possibly completely unknown

6. E.g.: Flow cytometry experiments data Here: Concentrations of (phosphorylated) proteins

7. possibly completely unknown E.g.: Flow cytometry experiments data Machine Learning statistical methods

8. true network extracted network Is the extracted network a good prediction of the real relationships?

9. true network extracted network biological knowledge (gold standard network) Evaluation of learning performance

10. Reverse Engineering of Regulatory Networks Can we learn network structures from postgenomic data themselves? Are there statistical methods to distinguish between direct and indirect correlations? Is it worth applying time-consuming Bayesian network approaches although computationally cheaper methods are available? Do active interventions improve the learning performances? –Gene knockouts (VIGs, RNAi)

11. direct interaction common regulator indirect interaction co-regulation

12. Reverse Engineering of Regulatory Networks Can we learn network structures from postgenomic data themselves? Are there statistical methods to distinguish between direct and indirect correlations? Is it worth applying time-consuming Bayesian network approaches although computationally cheaper methods are available? Do active interventions improve the learning performances? –Gene knockouts (VIGs, RNAi)

13. Relevance networks Graphical Gaussian models Bayesian networks Three widely applied methodologies:

14. Relevance networks Graphical Gaussian models Bayesian networks

15. Relevance networks (Butte and Kohane, 2000) 1.Choose a measure of association A(.,.) 2.Define a threshold value t A 3.For all pairs of domain variables (X,Y) compute their association A(X,Y) 4. Connect those variables (X,Y) by an undirected edge whose association A(X,Y) exceeds the predefined threshold value t A

16. Relevance networks (Butte and Kohane, 2000)

17. 1.Choose a measure of association A(.,.) 2.Define a threshold value t A 3.For all pairs of domain variables (X,Y) compute their association A(X,Y) 4. Connect those variables (X,Y) by an undirected edge whose association A(X,Y) exceeds the predefined threshold value t A

18. direct interaction common regulator indirect interaction co-regulation

19. Pairwise associations without taking the context of the system into consideration direct interaction pseudo-correlations between A and B E.g.:Correlation between A and C is disturbed (weakend) by the influence of B

20. 12 X 21 X 21 ‘direct interaction’ ‘common regulator’ ‘indirect interaction’ X 21 12 strong correlation σ 12

21. Relevance networks Graphical Gaussian models Bayesian networks

22. Graphical Gaussian Models 2 2 1 1 direct interaction Partial correlation, i.e. correlation conditional on all other domain variables Corr(X 1,X 2 |X 3,…,X n ) But usually: #observations < #variables strong partial correlation π 12

23. Shrinkage estimation of the covariance matrix (Schäfer and Strimmer, 2005) with where 0<λ 0 <1 estimated (optimal) shrinkage intensity, is guaranteed

24. direct interaction common regulator indirect interaction co-regulation

25. Graphical Gaussian Models direct interaction common regulator indirect interaction P(A,B)=P(A)·P(B) But: P(A,B|C)≠P(A|C)·P(B|C)

26. Further drawbacks Relevance networks and Graphical Gaussian models can extract undirected edges only. Bayesian networks promise to extract at least some directed edges. But can we trust in these edge directions? It may be better to learn undirected edges than learning directed edges with false orientations.

27. Relevance networks Graphical Gaussian models Bayesian networks

28. A CB D EF NODES EDGES Marriage between graph theory and probability theory. Directed acyclic graph (DAG) represents conditional independence relations. Markov assumption leads to a factorization of the joint probability distribution:

29. Bayesian networks versus causal networks Bayesian networks represent conditional (in)dependency relations - not necessarily causal interactions.

30. Bayesian networks versus causal networks A CB A CB True causal graph Node A unknown

31. Bayesian networks A CB D EF NODES EDGES Marriage between graph theory and probability theory. Directed acyclic graph (DAG) represents conditional independence relations. Markov assumption leads to a factorization of the joint probability distribution:

32. Bayesian networks Parameterisation: Gaussian BGe scoring metric: data~N(μ,Σ) with normal-Wishart distribution of the (unknown) parameters, i.e.: μ~N(μ*,(vW) -1 ) and W~Wishart(T 0 )

33. Bayesian networks BGe metric: closed form solution

34. Learning the network structure n46810 #DAGs5433,7· 10 6 7,8 ·10 11 4,2 ·10 18 Idea: Heuristically searching for the graph M* that is most supported by the data P(M*|data)>P(graph|data), e.g. greedy search graph → score BGe (graph)

35. MCMC sampling of Bayesian networks Better idea: Bayesian model averaging via Markov Chain Monte Carlo (MCMC) simulations Construct and simulate a Markov Chain (M t ) t in the space of DAGs {graph} whose distribution converges to the graph posterior distribution as stationary distribution, i.e.: P(M t =graph|data) → P(graph|data) t → ∞ to generate a DAG sample: G 1,G 2,G 3,…G T

36. Order MCMC (Friedman and Koller, 2003) Order MCMC generates a sample of node orders from which in a second step DAGs can be sampled: ABDEFC ABDEFC AFDEBC AFDEAC → DAG Acceptance probability (Metropolis Hastings) G 1,G 2,G 3,…G T DAG sample

37. Equivalence classes of BNs A B C A B A B A B C C C A B C completed partially directed graphs (CPDAGs) A C B v-structure P(A,B)=P(A)·P(B) P(A,B|C) ≠ P(A|C)·P(B|C) P(A,B)≠P(A)·P(B) P(A,B|C)=P(A|C)·P(B|C)

38. CPDAG representations A BC D EF DAGs CPDAGs Utilise the CPDAG sample for estimating the posterior probability of edge relation features: where I(G i ) is 1 if the CPDAG G i contains the directed edge A → B, and 0 otherwise A BC D EF

39. CPDAG representations A BC D EF A BC D EF DAGs CPDAGs Utilise the DAG (CPDAG) sample for estimating the posterior probability of edge relation features: where I(G i ) is 1 if the CPDAG of G i contains the directed edge A → B, and 0 otherwise A BC D EF interpretation superposition

40. Interventional data AB AB AB inhibition of A AB down-regulation of Bno effect on B A and B are correlated

41. Evaluation of Performance Relevance networks and Graphical Gaussian models extract undirected edges (scores = (partial) correlations) Bayesian networks extract undirected as well as directed edges (scores = posterior probabilities of edges) Undirected edges can be interpreted as superposition of two directed edges with opposite direction. How to cross-compare the learning performances when the true regulatory network is known? Distinguish between DGE (directed graph evaluation) and UGE (undirected graph evaluation)

42. data Probabilistic inference - DGE true regulatory network Thresholding edge scores TP:1/2 FP:0/4 TP:2/2 FP:1/4 concrete network predictions lowhigh

43. data Probabilistic inference - UGE undirected edge scores add up scores of directed edges with opposite direction skeleton of true regulatory network

44. data Probabilistic inference - UGE skeleton of true regulatory network undirected edge scores add up scores of directed edges with opposite direction + + +

45. data Probabilistic inference skeleton of true regulatory network undirected edge scores

46. data Probabilistic inference skeleton of true regulatory network Thresholding undirected edge scores TP:1/2 FP:0/1 TP:2/2 FP:1/1 highlow concrete network (skeleton) predictions

47. Evaluation 1: AUC scores Area under Receiver Operator Characteristic (ROC) curve AUC=0.5AUC=10.5<AUC≤1 sensitivity inverse specificity

48. Evaluation 2: TP scores We set the threshold such that we obtained 5 spurious edges (5 FPs) and counted the corresponding number of true edges (TP count).

49. Evaluation 2: TP scores 5 FP counts

50. Evaluation 2: TP scores 5 FP counts BN GGM RN

52. Evaluation On real experimental cytometric from the RAF signalling pathway for which a gold standard network is known On synthetic data simulated from this gold- standard network topology

53. Evaluation On real experimental cytometric from the RAF signalling pathway for which a gold standard network is known On synthetic data simulated from the gold- standard network topology

54. Evaluation: Raf signalling pathway Cellular signalling cascade which consists of 11 phosphorylated proteins and phospholipids in human immune systems cell Deregulation  carcinogenesis Extensively studied in the literature  gold standard network

55. ‘gold standard RAF pathway‘ according to Sachs et al. (2004)

56. Raf pathway 11 nodes (proteins) and 20 directed edges

57. Data Intracellular multicolour flow cytometry experiments concentrations of 11 proteins 5400 cells have been measured under 9 different cellular conditions (cues) We decided to downsample our test data sets to 100 instances - indicative of microarray experiments

59. Two types of experiments

61. Evaluation On real experimental data, using the gold standard network from the literature On synthetic data simulated from this gold- standard network

62. Raf pathway

63. Gaussian simulated data

64. Netbuilder simulated data DNA + TF`s→ DNA●TF → mRNA → protein

65. DNA + TF A DNA●TF A DNA + TF B DNA●TF B Netbuilder simulated data Steady-state approximation KAKA KBKB

66. Generating data using Netbuilder tool The main idea of Netbuilder is instead of solving the steady state approximation to ODEs explicitly, we approximate them with a qualitatively equivalent combination of multiplications and sums of sigmoidal transfer functions. Netbuilder simulated data

67. Experimental Results

68. Synthetic data, observations

69. Synthetic data, interventions

70. Cytometry data, observations

71. Cytometry data, interventions

74. How can we explain the difference between synthetic and real data ?

75. Raf pathway

77. Regulation of Raf-1 by Direct Feedback Phosphorylation. Molecular Cell, Vol. 17, 2005 Dougherty et al Disputed structure of the gold- standard network

78. Complications with real data Interventions are not “ideal” owing to negative feedback loops. Putative negative feedback loops: Can we trust our gold-standard network?

79. Stabilisation through negative feedback loops inhibition

80. Conclusions 1 BNs and GGMs outperform RNs, most notably on Gaussian data. No significant difference between BNs and GGMs on observational data. For interventional data, BNs clearly outperform GGMs and RNs, especially when taking the edge direction (DGE score) rather than just the skeleton (UGE score) into account.

81. Conclusions 2 Performance on synthetic data better than on real data. Real data: more complex Real interventions are not ideal Errors in the gold-standard network

82. Additional analysis I: Raf pathway

85. CPDAGs of networks 3/20 directed edges13/16 directed edges ORIGINAL MODIFIED

86. ORIGINALMODIFIED Gaussian Netbuilder

87. Some additional analysis II

88. Thank you

89. References: Butte, A.S. and Kohane, I.S. (2003): Relevance networks: A first step toward finding genetic regulatory networks within microarray data. In Parmigiani, G., Garett, E.S., Irizarry, R.A. und Zeger, S.L. editors, The analysis of Gene Expression Data, pages 428-446. Springer. Doughtery, M.K. et al. (2005): Regulation of Raf-1 by Direct Feedback Phosphorylation. Molecular Cell, 17, 215-227. Friedman, N. and Koller, D. (2003): Being Bayesian about network structure. Machine Learning, 50:95-126. Madigan, D. and York, J. (1995): Bayesian graphical models for discrete data. International Statistical Review, 63:215-232. Sachs, K., Perez, O., Pe`er, D., Lauffenburger, D.A., Nolan, G.P. (2004): Protein-signaling networks derived from multiparameter single-cell data. Science, 308:523-529. Schäfer, J. and Strimmer, K. (2005): A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Statistical Applications in Genetics and Molecular Biology, 4(1): Article 32.