ЖУРНАЛ ОБЩЕЙ БИОЛОГИИ, 2007, том 68, № 5, с. 332-340
УДК 57.087.2:519.233.3
ON THE POWER OF SOME BINOMIAL MODIFICATIONS OF THE BONFERRONI MULTIPLE TEST
© 2007 r. A. T. Teriokhin, T. de Meeûs, J.-F. Guégan
Génétique et Évolution des Maladies Infectieuses, UMR 2724IRD-CNRS, Centre IRD 911 Avenue Agropolis, BP 64501, 34394 Montpellier Cedex 5, France Faculty of Biology, Lomonosov Moscow State University Leninskie Gory, Moscow 119992, Russia e-mail: terekhin_a@mail.ru
Widely used in testing statistical hypotheses, the Bonferroni multiple test has a rather low power that entails a high risk to accept falsely the overall null hypothesis and therefore to not detect really existing effects. We suggest that when the partial test statistics are statistically independent, it is possible to reduce this risk by using binomial modifications of the Bonferroni test. Instead of rejecting the null hypothesis when at least one of n partial null hypotheses is rejected at a very high level of significance (say, 0.005 in the case of n = 10), as it is prescribed by the Bonferroni test, the binomial tests recommend to reject the null hypothesis when at least k partial null hypotheses (say, k = [n/2]) are rejected at much lower level (up to 30-50%). We show that the power of such binomial tests is essentially higher as compared with the power of the original Bonferroni and some modified Bonferroni tests. In addition, such an approach allows us to combine tests for which the results are known only for a fixed significance level. The paper contains tables and a computer program which allow to determine (retrieve from a table or to compute) the necessary binomial test parameters, i.e. either the partial significance level (when k is fixed) or the value of k (when the partial significance level is fixed).
An environmental factor, x, can often influence independently n variables yx, y2, ..., yn that describe the state of a population. For example, the presence of a pollutant may increase the frequencies of several diseases. Suppose that we know, in the form of p-values1, p1, p2, ..., pn, the results of testing n partial null hypotheses, H01, H02, ..., H0n, postulating that the factor x has no effect on the variables y1, y2, ..., yn, correspondingly. Then the problem arises how to combine these results to verify the overall null hypothesis, H0, that the factor has no effect. The simplest way is to reject H0 if at least one partial hypothesis is rejected at some given level of significance, a', i.e. when the probability of the partial mistake is not greater than a' (when the partial hypothesis events are statistically independent). But this test procedure is misleading because its significance level, an, may be much greater then a'. For example, for a' = 0.05 and an = 10 we would obtain an unacceptably great value a10 = 1 -- (1 - 0.05)10 = 0.40.
To avoid such a high risk of rejecting falsely the overall null hypothesis H0, a number of procedures (multiple tests) were proposed for combining the results of partial tests in such a way that the overall significance an be not greater than a given significance level, say, a = 0.05. The
1 A p-value is the probability that a test statistic will be equal to or greater than the currently observed statistic under assumption that the null hypothesis, i.e. the hypothesis being tested, is true. The smaller the p-value, the greater the confidence with which the test rejects the null hypothesis.
most known multiple test is based on the Bonferroni inequality
an < na'
where a' is the significance level of each partial test (Morrison, 2004; Couples et al., 1984; Meinert, 1986; Hochberg, Tamhane, 1987; Westfall, Young, 1993; Bland, Altman, 1995). The inequality expresses a simple tenet of probability theory: the probability that one of several events occurs can not exceed the sum of probabilities of all those events. It follows from this inequality that if we use for partial tests the significance level a' = = a/n (Bonferroni correction for multiplicity2) then the overall significance an will be not greater than required significance level a.
However, the power (i.e., the probability of detecting the really existing effect by rejecting the false null hypothesis) of such a Bonferroni multiple test, as well as that of some of its modifications (Holm, 1979; Simes, 1986; Hochberg, 1988; Rom, 1990; Zhang et al., 1997; Roth, 1999), is rather low (Blair et al., 1996; Morikawa et al., 1997; Legendre P., Legendre L., 1998; Ryman, Jonde, 2001). It is a reason why some researchers (Roth-man, 1990; Perneger, 1998; Bender, Lange, 1999) suggest rather to combine the results of partial tests at an in-
2 The Bonferroni correction was proposed by Carlo Bonferroni (Bonferroni, 1935) for the case when several dependent or independent statistical tests are performed simultaneously (because it does not follow from a given significance level holding for each individual comparison that the same does hold for the set of all comparisons).
formal level instead of to apply a multiplicity correction. Another way to improve the situation and to rescue some empirically discovered effects falsely rejected by the Bonferroni test is to use more powerful multiple tests.
The intuitive idea underlying our approach is that when the really existing effect is expressed rather weakly but in all partial tests, the power of Bonferroni test, yn, equal to the probability of obtaining at least one test with the significance level less than a/n may be very low and, on the contrary, in this case the probability that at least some number k (k > 1) of n tests are significant at a level a (greater than a/n or even greater than a) may be much higher. Taking k and a' such that the overall probability an is not greater than the desired overall significance, a, and rejecting the null hypothesis each time when there are at least k tests significant at the level a', one can obtain a multiple test with overall significance not greater than a and with the power greater than that of the Bon-ferroni multiple test. It is natural to consider the multiple tests of this type as binomial modifications of the Bon-ferroni test because the values of k and a' ensuring the desired overall significance a can be easily found (under the assumption of independence of partial tests) by means of the well-known formula for binomial probabilities. We will see that, indeed, the binomial multiple tests may have a power notably exceeding the power of the Bonferroni test and its former modifications.
It is essential that we assume independence of partial tests to construct the binomial tests. In practice, however, the partial tests may be both independent (when they are based on different sets of data) and less or more dependent (when they are based on the same set of data, say, when performing multiple comparisons). We illustrate therefore the consequences of attenuating the restriction of independence.
There are at least two principles for determining the values of k and a' for binomial multiple tests. First, we may fix arbitrarily the value of k and search for the largest value of a' that provides for the chosen overall significance an not greater than the required value a. We may set k equal, say, to 2 and then calculate the corresponding value of a'. Also we may, for any given n, set k equal to some fraction of n, for example, equal to k = [n/2] (the integer part of n/2). Second, we may fix arbitrarily the value of a' and search for the smallest value of k which provides that the obtained overall significance an is not greater that the given level a, say, a = 0.05. In particular, a' can be set equal to a, i.e. we can set a' = a = 0.05 (Prugnolle et al., 2002) and then calculate the corresponding value of k which, evidently, depends on the required overall significance a, on the chosen significance of partial tests a', and on the total number of partial tests n. But we may also fix a' at any other level, say, at 0.10, 0.25 or even 0.50 and calculate the corresponding value of k for the chosen level of significance.
There are other modifications of the standard Bonfer-roni multiple test, mainly based on the ranking the partial p-values. Holm (1979) proposed a sequential multiple
testing procedure (see also Rice, 1989). The procedure consists of a stepwise comparison of successively increasing partial p-values, p1, p2, ..., pn, with successively greater partial significance levels, a/n, a(n - 1), a/(n - 2), ..., a/1. If pi > a/n, then the overall null hypothesis H0 is not rejected and the procedure is stopped; otherwise, H0 is rejected. Inequality p1 < a/n means also that the partial alternative should be rejected and we may pass to the next comparison. This stepwise process continues until the step i where the inequality pt < a/(n - i + 1) is fulfilled.
In fact, it is only the first step of this procedure, concerning the overall null hypothesis, that is of interest for us. The binomial multiple tests we will consider do not test partial hypotheses and, in this sense, they give less information as compared with sequential tests. In principle, this is not even necessary that the overall alternative hypothesis is formulated as a falsity of all partial null hypotheses. But even in the case when the alternative hypothesis is formulated as the falsity of only a part of partial null hypotheses, it is very important to have a powerful test for the overall null hypothesis because falsely accepting the overall null hypothesis prevents automatically any further testing of partial hypotheses.
Note also that Holm's procedure has the same power as the simple Bonferroni test because its first step is the same as that in the Bonferroni test. We will therefore use for comparison another sequential modification of the Bonferroni test developed by Simes (1986) in which the overall null hypothesis is rejected if at least one of inequalities pi < ia/n, i = 1, 2, ..., n, holds. Though th
Для дальнейшего прочтения статьи необходимо приобрести полный текст. Статьи высылаются в формате PDF на указанную при оплате почту. Время доставки составляет менее 10 минут. Стоимость одной статьи — 150 рублей.