Power & Error Explorer

Trade one risk for another

Hold n, effect, and variability fixed. Move a right-tail cutoff farther right: α falls, β rises, and power falls. Two-sided cutoffs move symmetrically, with α/2 in each tail.

More information separates the worlds

At fixed α, larger samples narrow the sampling distributions. For an effect in the chosen direction, power increases. A larger effect or less population variability also makes it easier to detect.

Try quadrupling n: the standard error halves. The raw horizontal scale stays fixed until you change the effect/variability or fit the axes.

A decision is not a verdict on truth

Failing to reject H₀ does not prove H₀. α is not the probability H₀ is true, and power is not the probability a significant result is correct. These are long-run decision rates conditional on a model.

Power needs an effect to detect

Specify a scientifically meaningful effect before collecting data. There is no single power for a study without that choice. Do not plug an observed effect back in to claim “observed” or post-hoc power; report estimates and uncertainty instead.

Sampling distributions are not populations

Each point on these curves represents a possible sample mean or difference in sample proportions across repeated studies. Increasing n reduces uncertainty in that estimate; it does not reduce the population’s σ.

Choose the right model

The mean model assumes independent observations and known σ, with a normal population (exact) or an adequate normal approximation. The proportions model assumes independent groups and independent Bernoulli observations with adequate expected counts.

Unknown-σ mean tests use t distributions; variance tests use χ² under normality. Their power requires different alternative distributions. They are not implemented here. A rule such as n > 30 is not a universal guarantee.