Easy Epidemiology for Everyone
Tracy Powell, MPH
ISDH Field Epidemiologist, District 4
The last couple of newsletters described case control and cohort studies. The next step is looking behind the studies at the p-value and confidence interval. This month focuses on the p-value.
You might ask about the p-value when discussing which variables or exposures are significant in a study or outbreak investigation. In public health, the p-value, or probability value, is used to determine if the observed differences between groups (such as ill and not ill) are true differences. Epidemiologists are taught to look at the p-value when evaluating significance and testing hypotheses. Although the significance level is arbitrary, a p-value of less than or equal to 5 percent (0.05) is widely accepted. Smaller or larger percentages may be used if justification, such as previous studies or clinical significance, is warranted. For the purpose of this article, a p-value of ≤0.05 will be considered statistically significant, as most public health professionals use this value as a standard.
The p-value indicates how often you would expect to see the observed outcomes just as a result of chance. P-values less than or equal to 5 percent are considered “statistically significant”, which means the result is not likely attributable to chance. Events that have a high probability of occurring (p >0.05) are considered more common and, therefore, not significant. P-values range from 0-1 and are usually expressed as p <0.05.
P-value Example
This example will assist in understanding how to interpret a p-value. The table contains hypothetical data on Disease X rates for 2007 per 100,000 people in Indiana by county. The far right column displays the p-value for the disease rate for each county compared to the disease rate for the state. A p-value of less than 0.05 indicates a statistically significant difference between the annual disease rate for the county compared to the state rate.
As the table shows, four counties have p-values less than 0.05. County 1, County 3, and County 5 have rates that are significantly higher than the state rate. County 4 has a rate that is significantly lower than the state rate. County 2 and County 6 do not have statistically significant p-values, p = 0.28 and p = 0.15 respectively. Therefore, the rates for County 2 and County 6 are not statistically significantly different than the state rate.
The p-value does have limitations. A non-significant p-value does not necessarily mean the outcome is not clinically important. Similarly, there is a possibility that an outcome can be statistically significant and not be practical. The “common sense” component of public health practice is an essential part of decision-making and should not be limited by a p-value. To further explain this limitation, consider two studies. The first study looks at a low-fat diet and breast cancer in women with prior breast cancer. The second study looks at a low-fat diet and breast cancer in women without prior breast cancer. The p-value in the first study was significant (p = 0.03) and the p-value in the second study was not significant (p = 0.07). However, in both studies, the low-fat diet groups developed fewer new breast cancers. Although the second study was not significant, the result of fewer new breast cancers was clinically significant.
Another limitation of the p-value is that the value can be influenced by the number of subjects. If there are too few people in the study groups, the p-value may be unstable. There is no way to distinguish when no difference exists between groups (no significance) and when there are not enough people in the study groups.
The p-value is important but should not be the sole determination of significance. A better assessment would also include appropriate confidence intervals (along with an odds ratio or risk ratio), which will be discussed in the next issue of the Indiana Epidemiology Newsletter.
References
- Kuzma, Jan. Basic Statistics for the Health Sciences. 3rd Edition. 1998.
- Prentice RL, Caan B, Chlebowski RT, et al. Low-Fat Dietary Pattern and Risk of Invasive Breast Cancer, JAMA. 2006; 295:626-642.
- Chlebowski RT, Blackburn GL, Thomson CA, et al. Dietary fat reduction and breast cancer outcome: interim efficacy results from the Women's Intervention Nutrition Study (WINS). J Natl Cancer Inst. 2006; Vol. 98, No. 24, pp 1767-1776.
Stay Connected
