Not even sure if I am using the correct terminology.
If p value < alpha/significance level, then reject, if p value > alpha/significance level, then do not reject.
However, there is another problem where I calculated a “test statistic” for a paired comparison test (different in mean minus mean difference in returns divided by standard error). The value was 3.4, which was greater than the t-statistic for 24 degress of freedom at the 1% level of significance (2.807).
Because the test statistic is higher, it rejects the null hypothesis that the mean returns of the two portfolios differ from each other.
I’m relying on my college memory here so unsure why there is a disconnect between the p value and the test statistic?
P value is the smallest number that the null hypothesis can be rejected. so if the significant level alpha is greater than P, then reject Ho. If P is greater than alpha, then fail to reject Ho.
If the test statistic is greater than the critical value, then reject Ho
If the test stat is less than CV, then fail to reject reject Ho
The P-value is just the probability of that result given the null hypothesis. If the p-value of the test-statistic is lower than the rejection point, you can reject Ho.
Essentially, the probability of a sample mean has to be so low given your original assumptions that your original assumptions were clearly wrong.
The p-value is the probability of observing a test statistic of that magnitude or greater when the null is actually true. In essence, it’s a measure of how far away you are from the null.
If you’ve set (as an example) a 5% level of significance as your critical level, to reject the null, you’d have to see a value of the test statistic large enough that, if the null is true, you’d only see it 5% of the time by chance.
If your test statistic resulted in a p-value of (as an example) 3%, that means that you’re far enough away from the null that you’d only see that value (or greater) by chance 3% of the time. So, in some sense, you’re even farther away from the null than you need to be to reject at the 5% level. You’re not, however, far enough away to reject at the 1% level.
busprof hit it on the head. Remember that the definition of a p-value refers to the probability of obtaining the current value and those values that are more extreme than the given value, assuming a true null (really, our current results and more crazy results would be relevant as evidence against the null, so we want everything).
Adding on this a little, think of the p-value as a measure of incompatibility of the observed data with the null hypothesis. A smaller p-value indicates less compatibility of the data with the null hypothesis. Using this measure of disagreement, we can say whether or not it is reasonable to reject the null hypothesis (i.e. to say “The data are really weird if the assumed null is true, so I’m skeptical that the null is accurate.”). The test statistics (t-values, z, F, etc.) do the same thing, except they get larger in magnitude as the data become less compatible with the null.