P o w e r, type one error rate, beta

                                                     True State of the World

Power refers to the probability of a test appropriately rejecting the null hypothesis.  In other words rejecting Ho because it is false. 

Because a type II error, or failure to reject a false null hypothesis, is known as beta, power must equal    1 - beta.

A test with High Power is what you want as a test developer.  Your test will have more clout if it has a higher probability of detecting a difference between the compared means.  A test with Low Power is not a good instrument to detect differences between compared means. 

Most of us are familiar with Type I and Type II errors.  Type I errors occurs when we find a difference that is not really there.  Type II errors occur when we fail to see differences where there actually is a difference.  How does Power relate to this?  Power is the probability that we will avoid a Type II error.

Power analysis is used to determine how large an N an experiment needs and to evaluate the worth of an experiment that retains the null hypothesis.

There are three factors that determine the power of an experiment:    

1.  Effect size:  The larger the effect size, the more likely you are to reject Ho.  In other words the larger the difference between the means of the things you are measuring the larger the effect size.          

2.  The standard error of a difference:  The smaller Sxbar1 - xbar2 becomes the larger t gets and the more likely to reject Ho.  A larger sample size and a sample with very little variability will produce a smaller standard error.               

  3.  Alpha:  As you all know the higher the alpha level, the more likely you are to reject the null hypothesis.  The conventional alpha level is .05.