Scottish independence polls: Update & plus

Hi,

***Please note that I made a mistake. All the analysis are conducted since the beginning of January and not August as I stated in my post. *** Please see my next post for an update.
From now on, I concentrate my analysis on the polls published since the beginning of August in order to give more weight to the recent polls in the estimation of change. In this post, I also come back on the generation gap that I have analyzed in a previous post.

The next graph shows the recent evolution of the situation. It includes the YouGov and ICM polls published in the last hours. We see a concentration of estimates in the last few days: Most of the polls have put the two sides very close.

If we attribute non-disclosers proportionally to each side, we get the following portrait. It is slightly modified compared to my preceding post. Because of the two last posts, it no longer shows the two sides at par. The analysis estimates the Yes side at about 47.5% and the no at 52.5%. The spread of estimates has decreased and the estimates seem more similar across pollsters.

However, there is still a very significant difference between the opt-in web polls and the other polls. Using regression analysis, I have the estimate of support for the Yes side (with proportional attribution of non disclosers) at 42.8% at the beginning of August, and almost three points more when the polls are  opt-in panel polls, and gaining on average .136 points per week for the last 6 weeks (0.8 points total). Therefore, opt-in panels have the Yes at about 46.6% and the other polls at 43.6%.

Like in my preceding post, I estimate the results with a non proportional attribution of non-disclosers, giving 67% of them to the No side and 33% to the Yes side (see here for an explanation). This gives the following graph. With this more realistic attribution, the Yes side is closer to 45%

The generation gap

I have looked at the generation gap in a preceding post. It is very interesting to update on this gap for comparative purposes. In the 1995 referendum in Quebec, people less than 55 years old were more favorable to sovereignty at the beginning of the campaign and they were also the only one to move towards the Yes side during the campaign. The 55 years and older age group did not change its mind -- at least on average -- during the campaign.

The following graph allows to compare the 16-34 age old group to the total. Unfortunately, since pollsters do not use the same age categories, it is the only comparison that I can perform (Notice that for YOUGOV, the age group is in fact 16-39 and was 18-39 in the polls conducted before the August 12-15 poll). The graph confirms that, on average, support for independence is higher and has increased slightly more rapidly among the younger age group. However, notice that the estimates (in light blue) vary much. In August, for example, some polls have put support for independence in the younger age group as low as 37% while others put it at 60%.

Conclusion

We have to be very careful not to interpret random variation as movement. There is no doubt that there has been an increase in support for the Yes side during the last three weeks. However, if we take all the polls into account, the Yes side does not have the support of a majority of voters right now. The next polls will be crucial to estimate whether the increase in support has reached a ceiling or not.

In addition, let us stress the fact that when the difference between the two sides is estimated at less than six points, it is not appropriate -- and even a false assertion -- to say that "The Yes side is ahead for the first time" or "the No side is still ahead". This is what the margin of error means. And it is even more so when polls are not conducted using random samples of respondents like opt-in panels do. In this latter case, some speak about a credibility interval that would be similar to the margin of error. However, empirically, there is evidence that, in addition to that random error, there is some systematic bias in opt-in panels. In this case, the bias is likely to produce an overestimation of the Yes side. If the results show that it is not the case, it will be a very interesting topic for research.