Power vs. Confidence Level

Statistical power is defined as the probability that a test will correctly reject a false null hypothesis. In other words, it is the likelihood of detecting an effect (e.g. a real difference between two groups) if one actually exists. It is typically set to 80% meaning that 80% of the time we will can correctly detect a difference between the groups. It is also a critical component of calculating the correct sample size for an experiment. Let's say if we conduct an experiment on a very small sample size, we run the risk of not being able to detect an effect when it is there. On the other hand, if we choose too large of a sample size, we are wasting resources and money on the experiment. Therefore, using the right statistical power and using that to calculate the right sample size is very important for the success of an experiment.

Confidence levels are a cornerstone of statistical inference which provides a framework of understanding the reliability of our estimates. By definition, confidence level is the percentage of times the confidence interval would capture the true, unknown population parameter if you repeated the sampling process many times. It quantifies how certain you are that the range (confidence interval) you calculated from your sample contains the true population mean. Typical confidence level values are 95% and 99%. Higher confidence level indicates more certainty but less precision (i.e. wider interval) versus lower confidence level indicates less certainty but more precision.