Post Hoc Power Calculations: An Inappropriate Method for Interpreting the Findings of a Research Study

Article Figures & Data

Download figure
Open in new tab
Download powerpoint
Figure 1.
Illustration of how to assess the likelihood of a clinically meaningful difference (CMD) based on 95% confidence limits for a scenario of comparing a binary outcome between 2 groups. Examples are provided with and without the occurrence of a statistically significant difference (ie, P < 0.05), and all assume that an OR of 1.5 indicates a CMD in this example, which is shown with a solid horizontal line. ORs are represented by solid black square points, and 95% CIs are represented by vertical lines. A dashed horizontal line is provided for an OR of 1, indicating no difference between groups. CI: confidence interval; OR: odds ratio.

Table 1.

Illustration of the 2 types of statistical errors.

		Truth
		No Difference^{^a} Between Groups or Association Between Variables	Difference^{^a} Between Groups or Association Between Variables
Results of the research study	No statistically significant difference between groups or association between variables	True negative finding	False negative finding (ie, type II error)^{^b}
	Statistically significant difference between groups or association between variables	False positive finding (ie, type I error)	True positive finding

↵^aThe difference is a prespecified value that is the alternative hypothesis of the statistical test.
↵^bNote that statistical power is equal to 1 minus the probability of a type II error.

Table 2.

Suggested best practices regarding statistical power analysis.

Best Practice	Should Be Avoided
Always consider performing a power analysis before any research study. Ideally, a statistical expert should be consulted to aid in such analysis. Conclusions should be made based on examination of effect sizes (eg, mean differences, odds ratios, hazard ratios) and 95% confidence intervals for those effect sizes, in conjunction with P values.	Performing power analysis for any reason other than to help guide sample size decisions for a future study. Using power analysis in order to help interpret the results of an already completed study.