We theoretically analyze the problem of testing for p ‐hacking based on distributions of p ‐values across multiple studies. We provide general results for when such distributions have testable restrictions (are non‐increasing) under the null of no p ‐hacking. We find novel additional testable restrictions for p ‐values based on t ‐tests. Specifically, the shape of the power functions results in both complete monotonicity as well as bounds on the distribution of p ‐values. These testable restrictions result in more powerful tests for the null hypothesis of no p ‐hacking. When there is also publication bias, our tests are joint tests for p ‐hacking and publication bias. A reanalysis of two prominent data sets shows the usefulness of our new tests.

A flourishing empirical literature investigates the prevalence of p-hacking based on the distribution of p-values across studies. Interpreting results in this literature requires a careful understanding of the power of methods for detecting p-hacking. We theoretically study the implications of likely forms of p-hacking on the distribution of p-values to understand the power of tests for detecting it. Power can be low and depends crucially on the p-hacking strategy and the distribution of true effects. Combined tests for upper bounds and monotonicity and tests for continuity of the p-curve tend to have the highest power for detecting p-hacking.