Brexit: It's all about modes?

Hi,

We are close to entering the last week of the campaign. I will present an update of my last week analysis but, for this post, I will mainly focus on the major differences between modes.

First an update

As we can see in this first graph, the progression of the leave side went on during last week. However, it is important to notice that the Stay side remained stable. Its support did not decrease. What happens in that the progression of the Leave side seems to come almost entirely from a decrease in the proportion of respondents who say that they don't know how to vote or that they will not vote. This proportion, as noticed in a previous post, varies substantially, from3% to 15% during the last week.

If we attribute non-disclosers -- i.e. don't knows and will not vote -- proportionally like all the pollsters do, we get the following graph. We see that support for Leave has now pass support for Stay, as others have shown.

However, as I explained in preceding posts, empirically it is much more sound to attribute non-disclosers non proportionally, attributing more of them to the status quo side, in this case, Stay. If I keep the same non-proportional attribution that I used before -- 67% to stay and 33% to leave -- I get the following portrait of the situation. Support for Stay is slightly ahead of support of Leave, by about three points (it was five points last week).

However, it is very relevant to ask whether the portrait traced by polls is the same for telephone and Web opt-in polls.

It's all about modes

The first question is whether there is a difference between modes, controlling for change over time. In order to check for that, I perform a regression. The conclusions are:

•  As we can easily see with the form of the curves in the preceding graphs, change over time is in the form of a reverse U (quadratic).

• For Stay with proportional attribution of non-disclosers, after taking into account change over time, web polls give on average 5.1 points less to Stay than telephone polls. The mode, by itself, explains 45% of the variation between polls (which is huge!).
• Using non-proportional attribution, the difference between modes is slightly corrected. Support for Stay in Web opt-in polls is 3.66 points lower on average than in telephone polls and mode explains 26% of the variation between polls.

 The second question is whether the two modes trace the same portrait of change over time. The simple answer is no. The first graph shows chance in support for Stay with proportional attribution of non-disclosers according to mode. It shows that, while telephone polls have estimated a steady decrease in support for Stay since the beginning of 2016, the web opt-in polls trace quite a different portrait. Is is only recently that they show a small decrease. These quite different portraits however converge to a similar estimate -- close to 50% -- in the last few days. The same thing happened in a way in the Scottish referendum where the difference between modes disappeared in the last weeks before election day.

If we use non-proportional attribution of non-disclosers, the portrait is similar but the endpoint estimate is slightly different, at 51.5%. However, since non-proportional attribution corrects for some of the differences between modes, there is no difference left in estimates according to mode.

 Conclusion

Although the polls using different modes of administration do not trace the same portrait of change in support over time, it seems that, at the end, they tend to agree. So, as of now, we do not have to start a battle on who's right.

It seems to me that referendums on "independence" somewhat look alike, if one compares Quebec 1995, Scotland 2014 and the actual Brexit. During the campaign, the "change" side always gains support and, in the days before election, comes close to 50 percent . In Quebec and in Scotland, the "Law of even polls" was respected. This is, when the two sides are at par, the status quo side is likely to win. Why is that? We may speculate. It is possible that people who are for status quo tend less to reveal their preferences to pollsters or are less present in the samples. It is also possible that some people who are in favour of change are afraid of what could happen if change win by a tiny margin. they may therefore change their minds at the last minute. Anyhow, it is easier and less consequential to tell a pollster that you are going to vote for change than to do it for real. And "the message" is sent to leaders nonetheless.

Another nine days to go to see whether what happened in Quebec and Scotland will happen for the Brexit. We know however that the situation is somewhat different, in particular in the sociodemographic profile of the supporters for the two sides. Support for change in the actual campaign comes more from older people, who tend to turnout in larger proportions.



**Notice on methodology: In the graphs, each point represents a poll estimate positioned at the middle of the fieldwork; lines represent the likely change in support estimated using Loess (Epanechnikov, 0.65).

 For methodologists and other interested people

A question to ask is whether there is more variation according to mode and whether there is variation within mode. The next graph shows a box-and whiskers plot of the variation according to mode in support for Stay with proportional attribution of non-disclosers. The graph again illustrates that support for Stay is estimated higher by telephone polls. However, there is not much difference between modes in the level of variation and not that many polls differ significantly from other polls using the same mode. Two poll estimates by Survation are significantly higher than the other web opt-in polls and one YouGov poll is somewhat lower. For telephone polls, ICM and ORB each have two polls that are somewhat low.

 A similar graph done with estimates using non-proportional attribution of non-disclosers show a similar portrait. However, this procedure reduces variance among telephone polls and now show two Ipsos-Mori and one ComRes polls somewhat higher than the other telephone polls.

A general conclusion from this analysis would be that the major difference between modes is in the estimates -- in this case the median estimate -- not in variation. And there is not much difference within modes either.