Topic Review - Newest First (only newest 5 are displayed)

rqayyum

Multiple hypotheses testing

A recent article in NEJM addresses a very important issue of the false positive results due to multiple statistical tests on the same data and specifically in the case of multiple subgroup analyses of randomized clinical trials. This is an important issue which is frequently ignored by both investigators and editors. This article explains, very concisely, how to interpret subgroup analyses.
Here is its link
http://content.nejm.org/cgi/content/extract/354/16/1667
As this article explains:
“When treatments have identical efficacy, the probability of finding at least one “statistically significant” interaction test when 10 independent interaction tests are undertaken is 40 percent. The more subgroup analyses conducted, the higher the probability of one or more chance findings that may be misinterpreted as clinically directive.”

One way of correcting for this problem is to decrease p-value from its commonly used p-value of 0.05. The most conservative correction for multiple hypothesis testing problem is Bonferroni’s. In its very simple form, Bonferroni’s correction can be written as 0.05/K, where K is number of hypothesis tested. Using this correction, one can get a p-value against which all subgroup p-values should be compared. There are other less conservative, but more commonly used, corrections for multiple testing problem such as Holm’s correction.

I have hardly ever seen correction for multiple hypotheses testing even in well-recognized medical journals. On the other hand, it is extremely important to remember this problem not only in planning a study but also in interpreting the significance of multiple p-values in any paper.