A General Decomposition of Dependence.

Dependence among the rows of E and among the rows of $R=X-\widehat{B}S$ are types of multivariate dependence among vectors. The standard approach for modeling multivariate dependence is to estimate a population-level parameterization of the dependence and then include estimates of these parameters when performing inference (30). For example, if the e_i are assumed to be Normally distributed with the columns of E being independently and identically distributed random m-vectors, then one would estimate the m × m covariance matrix which parameterizes dependence across the rows of E. One immediate problem is that because n m, it is computationally and statistically problematic to estimate the covariance matrix (31).

A key feature is that, in the multiple testing scenario, the dimension along which the sampling occurs is different than the dimension along which the multivariate inference occurs. In terms of our notation, the sampling occurs with respect to the columns of X, whereas the multiple tests occur across the rows of X. This sampling-to-inference structure requires one to develop a specialized approach to multivariate dependence that is different from the classical scenarios. For example, the classical construction and interpretation of a P value threshold is such that a true null test is called significant with P value ≤ α at a rate of α over many independent replications of the study. However, in the multiple testing scenario, the P values that we utilize are not P values corresponding to a single hypothesis test over m independent replications of the study. Rather, the P values result from m related variables that have all been observed in a single study from a single sample of size n. The “sampling variation” that forms the backbone of most statistical thinking is different in our case: we observe one instance of sampling variation among the variables being tested. Therefore, even if each hypothesis test's P value behaves as expected over repeated studies, the set of P values from multiple tests in a single study will not necessarily exhibit the same behavior. Whereas this phenomenon prevents us from invoking well-established statistical principles, such as the classical interpretation of a P value, the fact that we have measured thousands of related variables from this single instance of sampling variation allows us to capture and model the common sources of variation across all tests. Multiple testing dependence is variation that is common among hypothesis tests.

Thus, rather than proposing a population-level approach to this problem (which includes the population of all hypothetical studies that could take place in terms of sampling of the columns of X), we directly model the random manifestation of dependence in the observed data from a given study, by aggregating the common sampling variation across all tests' data. Including this information in the model during subsequent significance analyses removes the dependence within the study. Therefore dependence is removed across all studies, providing study-specific and population-level solutions. To directly model the random manifestation of dependence in the observed data, we do the following: (i) additively partition E into dependent and independent components, (ii) take the singular value decomposition of the dependent component, and (iii) treat the right singular values as covariates in the model fitting and subsequent hypothesis testing. To this end, we provide the following result, which shows that any dependence can be additively decomposed into a dependent component and an independent component. It is important to note that this is both for an arbitrary distribution for E and an arbitrary (up to degeneracy) level of dependence across the rows of E.

Proposition 1. Let the data corresponding to multiple hypothesis tests be modeled according to Eq.1. Suppose that for each e_i, there is no Borel measurable function g such that e_i = g(e₁, …, e_i−1, e_i+1, …, e_m) almost surely. Then, there exist matrices Γ_m×r, G_r×n (r ≤ n), and U_m×n such that

where the rows of U are jointly independent random vectors so that

Also, for all i = 1, 2, …, m, u_i ≠ 0 and u_i = h_i(e_i) for a non-random Borel measurable function h_i.

A formal proof of Proposition 1 and all subsequent theoretical results can be found in the supporting information (SI) Appendix. Note that if we let r = n and then set U = 0 or set U equal to an arbitrary m × n matrix of independently distributed random variables, then the independence of the rows of U is trivially satisfied. However, our added assumption regarding e_i allows us to show that a nontrivial U exists where u_i ≠ 0 and u_i = h_i(e_i) for a deterministic function h_i. In other words, u_i is a function of e_i in a nondegenerate fashion, which means that U truly represents a row-independent component of E. The intuition behind these properties is that our assumption guarantees that e_i does indeed contain some variation that is independent from the other tests. For hypothesis tests where there does exist a Borel measurable g such that e_i = g(e₁, …, e_i−1, e_i+1, …, e_m), then the variation of e_i is completely dependent with that of the other tests' data. In this case, one can set u_i = 0 and the above decomposition is still meaningful.

The decomposition of Proposition 1 immediately indicates one direction to take in solving the multiple testing dependence problem, namely to account for the ΓG component, thereby removing dependence. To this end, we now define a “dependence kernel” for the data X.

Definition: An r × n matrix G forms a dependence kernel for the high-dimensional data X, if the following equality holds:

where the rows of U are jointly independent as in Proposition 1.

In practice, one would be interested in minimal dependence kernels, which are those satisfying the above definition and having the smallest number of rows, r. Proposition 1 shows that at least one such G exists with r ≤ n rows. As we discuss below in Scientific Applications, the manner in which one incorporates additional information beyond the original observations to estimate and utilize Γ and G is context specific. In the SI Appendix, we provide explicit descriptions for two scientific applications, latent structure as encountered in genomics and spatial dependence as encountered brain imaging. We propose a new algorithm for estimating G in the genomics application and demonstrate that it has favorable operating characteristics.

Dependence Kernel Accounts for Dependence.

An important question arises from Proposition 1. Is including G, in addition to S(Y), in the model used to perform the hypothesis tests sufficient to remove the dependence from the tests? If this is the case, then only an r × n matrix must be known to fully capture the dependence. This is in contrast to the m(m−1)/2 parameters that must be known for a covariance matrix among tests, for example. To put this into context, consider a microarray experiment with 1,000 genes and 20 arrays. In this case, the covariance has ~500,000 unknown parameters, whereas G has, at most, 400 unknown values. The following two results show that including G in addition to S(Y) in the modeling is sufficient to remove all multiple hypothesis testing dependence.

Corollary 1. Under the assumptions of Proposition 1, all population-level multiple testing dependence is removed when conditioning on both Y and a dependence kernel G. That is,

If instead of fitting model 1, suppose that we instead fit the decomposition from Proposition 1, where we assume that S and G are known:

It follows that estimation-level multiple testing independence may then be achieved.

Proposition 2. Assume the data for multiple tests follow model 1, and let G be any valid dependence kernel. Suppose that model 3 is fit by least squares, resulting in residuals $r_i=x_i-{\widehat{b}}_{i}S-{\widehat{\gamma }}_{i}G$. When the row space jointly spanned by S and G has dimension less than n, the residuals r₁, r₂, …, r_m are jointly independent given S and G, the ${\widehat{b}}_1,{\widehat{b}}_2,\dots ,{\widehat{b}}_m$ are jointly independent given S and G, and

The analogous results hold for the residuals and parameter estimates when fitting the model under the constraints of the null hypothesis.

Since G will be unknown in practice, the practical implication of this proposition is that we have to estimate only the relatively small r × n matrix G well in order to account for all of the dependence, while the simple least-squares solution to Γ suffices. When the row space jointly spanned by S and G has dimension equal to n, then the above proposition becomes trivially true. However, if we assume that S, G, and Γ are known, then the analogous estimation-level independence holds. In this case, we have to estimate Γ and G well in order to account for dependence. These (m + r)n parameters are still far smaller than the unknown m(m - 1)/2 parameters of a covariance matrix, for example.

Spatial Dependence.

Spatial dependence usually arises as dependence in the noise because of a structural relationship among the tests. In this case, we will consider the e_i of model 1 to simply represent “noise,” an example being the spatial dependence for noise that is typically assumed for brain-imaging data (6–8). In this setting, the activity levels of thousands of points in the brain are simultaneously measured, where the goal is to identify regions of the brain that are active. A common model for the measured intensities is a Gaussian random field (6). It is assumed that the Gaussian noise among neighboring points in the brain are dependent, where the covariance between two points in the brain is usually a function of their distance.

In Fig. 2 A and B, we show two datasets generated from a simplified 2-dimensional version of this model. It can be seen that the manifestation of dependence changes notably between the two studies, even though they come from the same data generating distribution. Using model 3 for each dataset, we removed the ΓG term. In both cases, the noise among points in the 2-dimensional space becomes independent and the P value distributions of points corresponding to true null hypotheses follow the Uniform distribution. It has been shown that null P values following the Uniform(0,1) distribution is the property that confirms that the assumed null distribution is correct (22). Additionally, it can be seen that the null P values from the unadjusted data fluctuate substantially between the two studies, and neither follows the Uniform(0,1) null distribution. This is due to varying levels of correlation between S and G from model 3. In one case, S and G are correlated producing spurious signal among the true null hypotheses; this would lead to a major inflation of significance. In the other case, they are uncorrelated leading to a major loss of power. By accounting for the ΓG term, we have resolved these issues.

Open in a separate window

Fig. 2.

Simulated examples of multiple testing dependence. A and B consist of spatial dependence examples as simplified versions of that encountered in brain imaging, and C and D consist of latent structure examples as encountered in gene expression studies. In all examples, the data and the null P values are plotted both before and after subtracting the dependence kernel. The data are plotted in the form of a heat map (red, high numerical value; white, middle; blue, low). The signal is clearer and the true null tests' P values are unbiased after the dependence kernel is subtracted. (A and B) Each point in the heat map represents the data for one spatial variable. The two true signals are in the diamond and circle shapes, and there is autoregressive spatial dependence between the pixels. (A) An example where the spatial dependence confounds the true signal, and the null P values are anticonservatively biased. (B) An example where the spatial dependence is nearly orthogonal to the true signal, and the null P values are conservatively biased. (C and D) Each row of the heat map corresponds to a gene's expression values, where the first 400 rows are genes simulated to be truly associated with the dichotomous primary variable. Dependence across tests is induced by common unmodeled variables that also influence expression, as described in the text. (C) An example where dependence due to latent structure confounds the true signal, and the null P values are anticonservatively biased. (D) An example where dependence due to latent structure is nearly orthogonal to the true signal, and the null P values are conservatively biased.

We thank the editor and several anonymous referees for helpful comments. This research was supported in part by National Institutes of Health Grant R01 HG002913.