What is the experiment-wise alpha level?

What is the experiment-wise alpha level?

• A. The total probability of a Type I Error that is accumulated from all of the individual tests in the experiment.
• B. The alpha value used for a single test only.
• C. The probability of a Type II Error across the experiment.
• D. The nominal alpha for the last test.

Answer: (A)

The experiment-wise alpha level is the overall risk of making at least one Type I error across all the statistical tests in the study. When you run multiple tests, false positives can accumulate, so the chance of at least one false positive becomes larger than the alpha you set for a single test. If you perform m independent tests at a per-test alpha, the approximate probability of at least one Type I error across the whole set is 1 minus (1 minus alpha_per_test) raised to the m. Researchers often choose a desired experiment-wise alpha (like 0.05) and adjust the per-test alpha accordingly, for example using the Bonferroni method: alpha_per_test = alpha_experiment / m. The other options describe the error rate for a single test, the Type II error, or the last test, which do not capture the idea of the overall, across-the-board false-positive risk.

The experiment-wise alpha level is the overall risk of making at least one Type I error across all the statistical tests in the study. When you run multiple tests, false positives can accumulate, so the chance of at least one false positive becomes larger than the alpha you set for a single test. If you perform m independent tests at a per-test alpha, the approximate probability of at least one Type I error across the whole set is 1 minus (1 minus alpha_per_test) raised to the m. Researchers often choose a desired experiment-wise alpha (like 0.05) and adjust the per-test alpha accordingly, for example using the Bonferroni method: alpha_per_test = alpha_experiment / m. The other options describe the error rate for a single test, the Type II error, or the last test, which do not capture the idea of the overall, across-the-board false-positive risk.