The maimed Martian, credible intervals and bias against benefit

Kault, David (2017) The maimed Martian, credible intervals and bias against benefit. Evidence-Based Medicine, 22 (2). pp. 49-53.

PDF (Author Accepted Version) - Accepted Version
Download (310kB) | Preview
[img] PDF (Published Version) - Published Version
Restricted to Repository staff only

View at Publisher Website:


A sad little story about a maimed Martian astronaut is used to illustrate a method of improving confidence interval (CI) calculations. CIs in medical statistics are currently calculated from the data available in a clinical trial or meta-analysis considered in isolation from all other information available on earth. Likewise, the Martian in the story uses only information available to it, in isolation from further information from earth. However, there is further objective knowledge available to people on earth to improve the Martian's estimate. In the same way, we have objective prior knowledge available to us outside of the current clinical trial results which we can use to improve CI calculations. This prior knowledge is incorporated into the CI calculations using Bayesian methods. The objective prior knowledge that is available is the fact that there were researchers who felt it worthwhile to conduct the trial and journal editors who felt it worthwhile publishing the results. It is shown here that the use of this information contracts the width of the log CI by a factor of about three quarters on average. Unlike standard CIs, these new intervals also have the advantage of being directly interpretable in terms of probabilities. These probabilities also enable calculation of improved point estimates. These calculations are applied to 100 randomly selected Cochrane systematic reviews and show serious problems in assessing medical treatments. For treatments not involving new drugs or devices, it is shown that there is evidence of a bias towards a negative assessment. The calculations here make a quantitative adjustment for publication bias. They show that the proportion of negative assessments do not reflect an appropriate adjustment for publication bias.

Item ID: 47377
Item Type: Article (Research - C1)
ISSN: 1473-6810
Keywords: credible intervals, Bayesian statistics, false negative results, publication bias
Related URLs:
Additional Information:

The author's Accepted Version (post-refereeing) is freely available for download above and includes the tables and mathematical appendices. This additional material is also available in the publisher's version but only online.

The author's "Credible" program is available from the Related URLs link below. "Credible" is a calculator which processes conventional statistics to improve estimates of relative risks.

Date Deposited: 20 Feb 2017 06:33
FoR Codes: 49 MATHEMATICAL SCIENCES > 4905 Statistics > 490502 Biostatistics @ 100%
SEO Codes: 92 HEALTH > 9202 Health and Support Services > 920204 Evaluation of Health Outcomes @ 100%
Downloads: Total: 548
Last 12 Months: 64
More Statistics

Actions (Repository Staff Only)

Item Control Page Item Control Page