3 Fold ChangeBased on the expression index, select genes with high fold change (e.g., R/G > 3)Advantage:IntuitiveLarger fold change may indicate greater biological impact.DrawbackReliable estimates are difficult to get.1st approach is sensitive to noise; 2nd approach loses quantitative information
4 Fold change is noisy Log-transformed expression in replicate #2 Noise is very high at low intensity.Log-transformed expression in replicate #1
5 SAMSignificance Analysis of Microarrays (SAM) considers a signal-to-noise ratio.whered(i) can large if either the signal is large or the noise is low. Therefore, it is different from fold change.Genes are ranked by d(i). The top candidates genes correspond to most positive or negative d(i).(Tusher et al. 2001)
6 Permutation test 2 1 “I” 2 5 I 3 3 4 4 5 6 U “U” 6 1 If a gene expresses at the same level in I and U conditions, then then relabeling the arrays will not affect the result of the value of d.
7 SAMTo test for statistical significance, arrays are randomly permuted.For each permutation, compute and rank the result dp(i).CalculateIdea is that for truly differentially expressed genes, d(i) should be greater than dE(i).Select those d(i) that are different from dE(i) more than a threshold level D.
9 Statistical hypothesis testing Null hypothesis H0: there is no association between the expression levels and the sample groups.Alternative hypothesis H1: there is association.Differentially expressed genes Rejection of null hypothesis.Genes are selected regardless of fold change.The last feature is often not desirable.
10 Single Hypothesis Testing Calculate the value a test statistic.IF the value is very unlikely given the null hypothesis H0, THENH0 is rejected and H1 is accepted.The gene is differentially expressed.ELSEH0 is not rejected.The gene is not differentially expressed.
13 p-valueThe p-value is the probability of obtaining a result at least as extreme as a given data point.It is also the minimum significance level required to reject H0.
14 Choice of test statistic Standard t-testAssume that yij are Gaussian distributed, then ti is given by the student-t distribution.A p-value is calculated from t-distribution with the 2n-2 degree of freedom.Issues:When n is small the denominator is an unreliable estimate of the variance.The assumption that yij are Gaussian is often violated in real data.
15 Variance shrinkageBasic idea: The variance at different genes should be correlated. If the data are noisy, then they are likely to be noisy everywhere. Thus one can use the information from other genes to estimate the variance at a given gene.
16 Variance shrinkage (Smyth 2004) Assume and where d0 and s02 correspond to the pooled data.ThenModify the t-statistic by replacing si2 withThe new statistic obeys t-distribution with d0 + di degrees of freedom.
17 Permutation test 2 1 “normal” 2 5 normal 3 3 4 4 5 6 cancer “cancer” 6 If H0 is correct, then relabeling the arrays will not affect the result of the test statistic.
18 Permutation p-value Permutation-test For the b-th permutation, b = 1, …, B,Permute the n columns (array labels) of the data matrix X.Compute test statistics t1,b, …, tm,b for each hypothesis (whether the m-th gene is not differentially expressed).The permutation distribution of the test statistic Ti for hypothesis Hi, ti,1, …, ti,B. For two-sided alternative hypotheses, the permutation p-value for hypothesis Hi iswhere I(.) is the indicator function, equaling 1 if the condition in parenthesis is true, and 0 otherwise.
19 Permutation p-value permutation distribution t-distribution scaled H0 is correctH0 is rejected
20 Multiple hypothesis testing Microarray experiments measure expression levels of thousand of genes.The hypothesis testing procedure is applied once for each gene.A large number of false positives may result.Cutoff at p = 0.05 for 6000 genes6000 X 0.05 = 300 genes falsely rejectedIf number of real target ~ 100, then most rejected genes are false targets.
24 Bonferroni correction Let m be the total number of tests. Reject hypothesis at a/m instead of a.Strong control of FWER.Too conservative.
25 Adjusted p-valueThe adjusted p-value for a single hypothesis Hj is the nominal level of the entire test procedure at which Hj would just be rejected, given the values of all test statistics involved.Example: pi = If rejecting all hypotheses with cutoff p < pi leads to FDR = 0.2, then the adjusted p-value is 0.2.The adjusted p-value is dependent on the specific test procedure.
26 Adjusted p-valueThe adjusted p-value for Bonferroni correction is.
27 False Discovery RateFWER aims at requiring no false positive at all. This is often too stringent in practice.False discovery rate (FDR) is proposed by Benjamini and Hochberg (1995). The idea is to allow a few false positives while enhancing the power.
28 Control of FDR, BH-procedure Find ordered observed p-values, andLet k be the largest i for whichReject all H1, …, Hk.(Benjamini and Hochberg, 1995)
29 Control of FDR, BH-procedure Find ordered observed p-values, andLet k be the largest i for whichReject all H1, …, Hk.Strongly controls FDRAlso weakly controls FWER(Benjamini and Hochberg, 1995)
30 Positive false discovery rate (pFDR) Better power than FDR procedure.Estimate
31 Estimation of p0(t)Under the null hypothesis, p-value is uniformly distributed.
32 Estimation of p0(t) Procedure: Choose 0 < l < 1 Assume pi is uniformly distributed at p > l.Then estimate asl
40 SAMTo test for statistical significance, arrays are randomly permuted.For each permutation, compute and rank the result dp(i).CalculateIdea is that for truly differentially expressed genes, d(i) should be greater than dE(i).Select those d(i) that are different from dE(i) more than a threshold level D.
41 Estimation of FDR in SAM R ≈ #(genes called significant)V ≈ #(genes called significant in permutation tests)FDR ≈ V/RPower of SAM is better than fold change criteria.
42 Data: Apo AI experiment 8 mice in treatment group (apo AI knockout); 8 mice in control group (normal)16 arrays: Cy5 – mRNA from trt or control mice; Cy3 – mRNA from pooled control mice.6356 genes.Want to detect differentially (trt vs control mice) expressed genes.
44 Cutoff value vs top genes Each metric can be viewed as a monotonic transformation of another.The only difference is the cutoff values are different.All statistical hypothesis testing methods are equivalent in terms of selecting the top k genes, for a fixed k.