# p-value interpretation and how to write analysis For example, suppose that a vaccine study produced a P value of 0.04. This P value indicates that if the vaccine had no effect, you’d obtain the observed difference or more in 4% of studies due to random sampling error.
There are several reasons why P values can’t be the error rate.

One commonly used significance level is 0.05. If the investor finds that the p-value is less than 0.05, then there is evidence against the null hypothesis. As a result, the investor would reject the null hypothesis and accept the alternative hypothesis. The smaller the p-value, the greater the evidence against the null hypothesis. Thus, if the investor finds that the p-value is 0.001, there is strong evidence against the null hypothesis, and the investor can confidently conclude the portfolio’s returns and the S&P 500’s returns are not be equivalent.
P-values are calculated using p-value tables or spreadsheets/statistical software. Because different researchers use different levels of significance when examining a question, a reader may sometimes have difficulty comparing results from two different tests. P-values provide a solution to this problem. A p-value less than 0.05 (typically в‰¤ 0.05) is statistically significant. It indicates strong evidence against the null hypothesis, as there is less than a 5% probability the null is correct (and the results are random). Therefore, we reject the null hypothesis, and accept the alternative hypothesis.

By Saul McLeod, published 2019

The P value means the probability, for a given statistical model that, when the null hypothesis is true, the statistical summary would be equal to or more extreme than the actual observed results . Therefore, P values only indicate how incompatible the data are with a specific statistical model (usually with a null-hypothesis). The smaller the P value, the greater statistical incompatibility of the data with the null hypothesis. What is important is that P values do not focus on the study hypothesis but on the null hypothesis.
Department of Anesthesiology and Pain Medicine, Seoul National University Bundang Hospital, Seongnam, Korea. A colleague just shared your blog with me and after 2 posts I’m hooked. I will read more today.
I use ttest and pvalues in the domain of web and app A/B testing and I’ve read everything I could find online but I still wasn’t sure I understood.
I built an A/A simulator in python and I got a lot more statistically significant results than 5% so I’m confused.
Just for clarity I call an A/A test a randomise observation where both series use the same success rate in %.