Which statement describes the relationship between experiment-wise alpha and per-test alpha?

Which statement describes the relationship between experiment-wise alpha and per-test alpha?

• A. The experiment-wise alpha level is typically greater than the per-test alpha.
• B. They are always equal.
• C. The per-test alpha is always greater than the experiment-wise.
• D. There is no systematic relationship.

Answer: (A)

When you run multiple tests, the chance of at least one false positive across all tests adds up. To keep that overall risk at a desired level, you set a smaller threshold for each individual test—the per-test alpha—often by dividing the overall (experiment-wise) alpha by the number of tests. This means the experiment-wise alpha stays at the chosen overall level, while each test uses a more stringent criterion. So, the experiment-wise alpha is typically greater than the per-test alpha. They’d be equal only if there’s a single test. If you don’t adjust for multiple tests, there isn’t a consistent relationship to rely on, but with adjustment, the per-test alpha shrinks as the number of tests grows, maintaining the overall limit.

When you run multiple tests, the chance of at least one false positive across all tests adds up. To keep that overall risk at a desired level, you set a smaller threshold for each individual test—the per-test alpha—often by dividing the overall (experiment-wise) alpha by the number of tests. This means the experiment-wise alpha stays at the chosen overall level, while each test uses a more stringent criterion. So, the experiment-wise alpha is typically greater than the per-test alpha. They’d be equal only if there’s a single test. If you don’t adjust for multiple tests, there isn’t a consistent relationship to rely on, but with adjustment, the per-test alpha shrinks as the number of tests grows, maintaining the overall limit.