The trade
Change alpha from 0.05 to 0.01. The threshold moves right, the grey tail shrinks — and the orange tail grows. Beta rises, power falls.
Move it the other way and the reverse happens.
You cannot reduce both by moving the line. That is not a limitation of the method; it is what "the distributions overlap" means. Any threshold is a statement about which error you would rather make.
What actually helps
Two things, and only two.
More data. Drag n upward. Both curves narrow, because their spread is the [standard error](sampling_distributions.html) and that falls like sqrt(n). The overlap shrinks, and both error rates can fall at once. This is the only lever that improves both without assuming anything.
A larger true effect. Drag the effect control. The curves separate. You do not usually control this, but it is why detecting a large effect needs so much less data than detecting a small one.
Choosing alpha honestly
The convention is 0.05, and treating it as fixed is the mistake. The right value depends on which error costs more.
A screening test for a treatable disease. A false negative means missing a case; a false positive means an unnecessary follow-up. Loosen alpha, gain power.
A criminal conviction. The system explicitly prefers false negatives, which is what "beyond reasonable doubt" encodes.
Particle physics. The five-sigma standard is alpha near 3 in 10 million, because the field has been burned by fluctuations and there is enough data to afford it.
A/B testing a button colour. A wrong call is cheap. Standard thresholds are generous, and the real risk is running many tests and picking winners.
Power, before the experiment
Power analysis is the calculation nobody enjoys and everybody should do: given a target power — usually 80% — an alpha, and the smallest effect worth detecting, how large a sample is required?
Set the effect to 0.2 and watch what n has to be before power reaches 80%. The required sample grows roughly with the inverse square of the effect size, which is why detecting small effects is so expensive and why so much published work is underpowered.
An underpowered study is worse than no study. It usually fails to find a real effect — and when it does find one, the estimate is inflated, because only an unusually large sample fluctuation could have crossed the threshold at that sample size. That is the winner's curse, and it is a substantial part of why published effects shrink on replication.
Where it goes wrong
Interpreting a non-significant result as no effect. Report the power, or the interval.
Running the power analysis afterwards. Post-hoc power computed from the observed effect is a restatement of the p-value and carries no new information.
Fixing alpha at 0.05 regardless of costs. It is a convention.
Peeking at results and stopping when significant. This inflates the true Type I rate far above the nominal one. Sequential testing methods exist and correct for it.