Today I updated my Tutorial on Correlation Coefficients to include a new section on the effect of sample size on the significance of correlation coefficients. This was motivated by some comments from search engine marketers on correlation strengths. (http://searchenginewatch.com/3641002). The new material might help those interested in learning whether a reported correlation coefficient is statistically different from zero. It is given below. Enjoy it.
The problem with correlation strength scales is that these say nothing about how the size of a sample impacts the significance of a correlation coefficient. This is a very important issue that is now addressed.
Consider three different correlation coefficients: 0.50, 0.35, and 0.17. Assume that we want to test that there is no significant relationship between the two variables at hand. The null hypothesis (H0) to be tested is that these r values are not statistically different from zero (rho = 0). How to proceed?
As recommended by Stevens (17), for rho = 0, H0 can be tested using a two tailed (i.e.,two sided) ttest at a given confidence level, usually at a 95% level. If t_{calculated} ≥ t_{table}, H0 is rejected. However, if t_{calculated} < t_{table} H0 is not rejected and there is no significant correlation between variables.
Here t_{calculated} is computed as r/SEr = r*SQRT[((n – 2)/(1 – r^{2}))] while t_{table} values are obtained from the literature (http://en.wikipedia.org/wiki/Student%27s_tdistribution#Table_of_selected_values ). Table 2 summarizes the result of testing the null hypothesis at different sample size values.
Table 2. H_{0} tests at different sample sizes; twotailed, 95% confidence. 
n 
df = n – 2 
r 
SE_{r} 
t(calc) 
t (0.95) 
Reject (H_{0} : rho = 0)? 
5 
3 
0.50 
0.50 
1.000 
3.182 
don’t reject 
10 
8 
0.50 
0.31 
1.633 
2.306 
don’t reject 
12 
10 
0.50 
0.27 
1.826 
2.228 
don’t reject 
14 
12 
0.50 
0.25 
2.000 
2.179 
don’t reject 
20 
18 
0.50 
0.20 
2.449 
2.101 
reject 
30 
28 
0.50 
0.16 
3.055 
2.048 
reject 
40 
38 
0.50 
0.14 
3.559 
2.024 
reject 
50 
48 
0.50 
0.13 
4.000 
2.011 
reject 

5 
3 
0.35 
0.54 
0.647 
3.182 
don’t reject 
10 
8 
0.35 
0.33 
1.057 
2.306 
don’t reject 
12 
10 
0.35 
0.30 
1.182 
2.228 
don’t reject 
14 
12 
0.35 
0.27 
1.294 
2.179 
don’t reject 
20 
18 
0.35 
0.22 
1.585 
2.101 
don’t reject 
30 
28 
0.35 
0.18 
1.977 
2.048 
don’t reject 
40 
38 
0.35 
0.15 
2.303 
2.024 
reject 
50 
48 
0.35 
0.14 
2.589 
2.011 
reject 

5 
3 
0.17 
0.57 
0.299 
3.182 
don’t reject 
10 
8 
0.17 
0.35 
0.488 
2.306 
don’t reject 
12 
10 
0.17 
0.31 
0.546 
2.228 
don’t reject 
14 
12 
0.17 
0.28 
0.598 
2.179 
don’t reject 
20 
18 
0.17 
0.23 
0.732 
2.101 
don’t reject 
30 
28 
0.17 
0.19 
0.913 
2.048 
don’t reject 
40 
38 
0.17 
0.16 
1.063 
2.024 
don’t reject 
50 
48 
0.17 
0.14 
1.195 
2.011 
don’t reject 
The table addresses at which size level an r value is high enough to be statistically significant.
For n = 14, all three r values (0.50, 0.35, and 0.17) are not statistically different from zero.
For n = 30, r = 0.50 is statistically different from zero while r = 0.35 and r = 0.17 are not.
Conversely, r = 0.50 is not statistically different from zero when n is equal or less than 14 while r = 0.35 is not different from zero when n is equal or less than 30.
Finally, r = 0.17 is not statistically different from zero at any of the sample sizes tested.