Fisher’s Combination of p-values corrected for file-drawer effect.

Insert commas.

Fisher’s combination.
Run a JavaScript program.

In the following sections are examples showing use of the programs. The user is respectfully invited to try out the examples or to use any others. The only thing to remember is: follow the grammatical rules of JavaScript. (This is because the “eval” method of JavaScript is used in picking up the data from the text area.) In particular, remember that starting an integer with a zero will force the use of base 8.

Which browsers?

Modern browsers such as Safari 3, Microsoft Internet Explorer 6, Mozilla Firefox 1.5, Mozilla 1.7, Netscape 7, and Opera 8 can use this page. Netscape 4 is out of date and cannot use this page. To top.

Fisher’s combination.

We think of independent experiments where the “p” values are continuously distributed from zero to unity. Let us assume in the present file that all the null hypotheses are true, so that each p is distributed according to a uniform distribution. The negated logarithms of the p values have gamma distributions, so their sum has a gamma distribution. Then we integrate to get the right-hand tail area of the gamma curve. The integral will be the p value for the whole collection of p values. (Fisher multiplied the logarithms by two, but he was using chi-square instead of gamma. Otherwise, we follow him.)

Here are six fictitious independent p values: .06, .07, .08, .11, .47, and .51 . Not one of them is 5% or less, so not one of the six experiments could get published in a learned journal. We hope that combining the p values will get the resulting p down to 5% or less. To prepare the p values for the program on this page we put them into a tableau:

[
1.0,
[ .06, .07, .08, .11, .47, .51 ]
]

The “1.0” on the line before the p values will be explained later. The reader is respectfully invited to select the tableau with the mouse, copy it to the clipboard, clear the upper text area if necessary, paste into that text area, and click on the “Fisher’s combination” button. The computer will print the number

0.025534324485258473

in the lower text area. The number is less than 5%, so maybe a paper ought to be written and sent in. (The p values in the tableau need not be sorted into order. Shuffling them will make no difference.) To top.

Correcting for file-drawer effect.

In the previous section, the p values were known only to the experimenter. In what is called “meta-analysis,” p values found published in journals are looked at by a person unconnected with the experiments belonging to the p values. Perhaps that person finds these in various journals: .04, .04, .03, .03, .02, .01, .01, and .01 . It would be tempting, but wrong, to put these into a tableau as in the previous section, and to run the program on them. The computer would by mistake print the number “2.087676626068954e-7”. The number looks impressive, but it is the result of a fallacy. The trouble is that p values published in journals are always 5% or less, so they are not distributed uniformly from zero to unity. Rather, they are distributed uniformly from zero to 5%. (This is called the “file-drawer effect” or the “publication bias.”) It is necessary to divide each p value by 5% before using Fisher’s formula, so as to have the correct domain. The division should be done by a computer, not by us humans. Let us therefore make our tableau look like

[
.05,
[ .04, .04, .03, .03, .02, .01, .01, .01 ]
]

That is, the “1.0” on the line before the p values has been replaced by a “.05”. Again the reader is respectfully invited to select the tableau with the mouse, to copy to the clipboard, to clear the upper text area if necessary, to paste into that text area, and to click on the “Fisher’s combination” button. This time the computer will print the number

0.5670778879922392

Now the number looks much less impressive. To top.

Run a JavaScript program.

While building this page I needed a way to run little JavaScript programs, so I constructed the “Run a JavaScript program” button. When I was done I left the button so users can practice JavaScript programming with it. If a program is in the upper text area, the button will run it. Here is a trivial example:

var x=[];
for(var j=0;j<10;j++)x[j]=j;
x;

The user is respectfully invited. To top.

Bibliography.

It is usual on the web to refer to Fisher RA: Statistical Methods for Research Workers. 4th edition. London: Oliver and Boyd; 1932. I am not able to check this reference for Fisher’s combination formula, but I have read about Fisher’s method on many web sites.

A readable and helpful paper on the file-drawer effect is http://www.scientificexploration.org/jse/articles/pdf/14.1_scargle.pdf.

No idea in the present file belongs to me. I have merely tried to use what I got from these two statisticians. Any mistakes that I may have made are my own, not Fisher’s or Scargle’s. To top.

Insert commas.

It may be that the user is bringing in data copied from other web pages, or from files, and is pasting the data from the clipboard to the upper text area. In that case the numbers are perhaps separated by blanks or tabulation characters or the like instead of by commas. The “Insert commas” button is meant to change such other separators to commas. The button does not always guess rightly what is expected of it, so the user is respectfully asked to look at the button’s work to make sure everything is as desired. It is a good idea to type the left and right square brackets before clicking the “Insert commas” button. To top.

License, revision date, and e-mail address.

All of this file is in the public domain. The date of this revision is 30 April 2008. Criticism both constructive and destructive comes to me, Harold Kaplan,

       at        dot        
smtw2gh  toadmail   com

To top.
Harold Kaplan’s statistics.htm
John C. Pezzullo’s page

Other links for this page.