lundi 19 mars 2018

Russia: The day after


Finally, how did the polls fare in the Russian presidential election? First, let us revisit the graphs that I presented on Friday, based on all the polls conducted since the end of December. On average, the polls estimated Putin's support one week before the election -- there was a ban on publication of the polls during the last week -- slightly under 80%. The results published by the electoral Commission according to the newspaper Le Monde show that Putin received around 76.6% of the vote. Therefore,  the polls estimates are within the margin of error. The support of the two main opponents, Grudinin from the Communist Party, and Zhirinovski, an ultranationalist candidate, was generally estimated at around 6%. Zhirinovski's support was well estimated: he received 5.6% of the vote. However, Grudinin finished second, with 11.8% of the vote. He managed to get twice the support he was estimated to have.

I do not have the sample size for these surveys but if they had 1000 respondents, the margin of error for Putin would have been 2.5% and for Grudinin or Zhirinovski, around 1.5%. Therefore, we may conclude that the only support that was not estimated accurately is Grudinin's. It reminds of the difficulties pollster had to estimate the support for the Communist Party -- and extreme left in general -- in some European countries during the Cold War, a difficulty that has since been replaced by a similar difficulty to estimate the extreme-right. Only one poll had put Grudinin at more than 10%, the one conducted by CIPKR at the end of February.

The support for all the other candidates, including Xenia Sobchak, was generally estimated  to be under 2%. Sobchak had two higher estimates, one at 2.3% by WCIOM, and one at 4.3% by CIPKR. She got 1.5%. Navalny's support was estimated at 1% at the end of December, just before he was banned from running. Of course, we do not know how he would have fared if he had run, but from the beginning, in this election, there were  two "main opponents" to Putin according to the polls, i.e., Grudinin and Zhirinovski.

What about pollsters?

The following graph compares the estimates from the different pollsters. It shows that, while FOM estimated Putin's support at around 75%, WCIOM tended to slightly overestimate the support, putting it at more than 80%.

However, in the exit polls, WCIOM had a slightly lower estimate of Putin's support, at 73.9% compared to 76.3% for FOM (see below the estimate from WCIOM). Both estimates were within the margin of error or close. And the estimates for Grudinin and Zhirinovski are quite accurate. It is rather impressive given the difficulty to conduct exit polls with a population scattered over such a large territory. In Canada, with a similar scattered population over a large territory, pollsters do not conduct exit polls.

Finally, the turnout was 67.5%. It is quite close to the estimates that I saw from WCIOM, i.e., 70% of respondents reporting that they would go vote for sure. As elsewhere, there is a tendency for respondents to over report their intention to vote or their vote -- or else, people who do not vote tend not to answer polls.


Finally, the pollsters fared rather well in their estimation of the vote, except for Grudinin, whom they clearly underestimated. However, since the polls could not be published during the last week, one could speculate that there was some movement during that week. Such an eventuality is unlikely however since there was no huge movement during the whole campaign.The exit polls were very good. However the pollsters will have to try to understand how come they underestimated voting intention for the Grudinin.

vendredi 16 mars 2018

Russia: The Day before


Although the results of the Russian presidential election are quite known in advance, it is  nevertheless interesting to see what the polls tell and how they will fare.

I took all the results of all the polls referred to on Wikipedia since the end of December 2017. I added more recent polls that I found on the pollsters' web sites. There is an embargo on the publication of polls in Russia during the last week of the Campaign. The last poll I found was conducted by FOM and ended on March 11.

There are two "serious pollsters" who conducted polls almost every week, i.e. WCIOM and FOM. In addition two polls were conducted by the "Center of research on political culture of Russia" (CIPKR) and three polls were conducted by people around Navalny. These polls seem to have been conducted by Navalny's partisans and it is not sure at all that the methodology is scientific. However, since the results of these polls are not different from the results of other polls, there is no serious justification not to keep them in.The statistical procedure that I use is the same as the one I used in many recent elections, i.e., local regression (loess).

The first graph shows voting intentions for the main candidates, including non-disclosers, i.e., respondents who say that they will abstain, that they will void their ballot or that they do not know whom they will vote for. It shows that voting intentions are quite stable with Putin always close to 70%. The two other most important candidates, Grudinin and Zhirinovski get less than 5% and the rest of the candidates together get also less than 5%. The non-disclosers are also quite stable, at a level somewhat lower than 20%.

However, since the proportion of non-disclosers varies with pollsters and with polls, it is necessary to exclude them in order to examine the proportion of support for each candidate after excluding the non-disclosers. In this way the total support for all the candidates is 100% for all polls. The next figure shows the support for the same candidates excluding the non-disclosers. It shows Putin's support around 80%, perhaps decreasing a bit in the last weeks. The support for Grudinin and Zhirinovski is stable at 5% as is the support for the total of the other candidates.

Finally, one interesting question is whether the pollsters give the same estimation of the support for Putin. The following figure presents the estimates from WCIOM, FOM, CIPKR and FBK. There are trend lines only for WCIOM and FOM because they are the only two who conducted enough polls. In the following graph, the blue line is the trend line from the WCIOM polls and the green line, from the FOM polls, We also see two violet larger dots illustrating the CIPKR estimates, and three larger red dots illustrating the FBK (Navalny's group) estimates. The figure shows that the CIPKR tended to put Putin lower than the other pollsters and that FOM estimates that support for Putin is decreasing (from 83% in December to 75% on March 11). On the contrary, WCIOM estimates that Putin's support is stable at around 82%. In the end, there is a difference of 6-7 points between the estimates.


It will be interesting to see whether the Russian pollsters, particularly WCIOM and FOM managed to predict the support for the different candidates well. It is problematic that there is a ban on the publication of polls during the last week since we will not be able to estimate the reliability of polls and we cannot know whether the trend estimated by FOM went on during the last days or else, if support for Putin is stable, as estimated by WCIOM.

Unfortunately, I do not read Russian well enough (I read the numbers, at least 😊 and I get help from google translate) to examine the possible differences in the polls' methodology. I am open to receiving more information, in French or in English, on this question.

The main question on Sunday will be the turnout. In the WCIOM polls, about 70% say that they will go vote for sure. These figures are close to what we get in many established democracies. However, research shows that turnout is higher when elections are close races, which is not the case here. It will be a challenge to mobilize voters.

lundi 24 avril 2017

France, le jour d'après, the day after

Bonjour, Hi,

(English follows in the same message).

Juste un mot pour faire un bilan de la performance des sondeurs, une excellente performance comme tout le monde l'a remarqué puisque les résultats sont à l'intérieur de la marge d'erreur pour tous les principaux candidats, comme le montre le graphique suivant (utilisant les sondages de la dernière semaine seulement) tiré du blogue de PJ Fournier, avec sa collabaration: .

Nous avions prédit une possible surestimation de Mélenchon. Cela ne s'est pas produit, fort probablement parce que des partisans de Hamon sont passés à Mélenchon à la dernière heure en voyant sa performance. Mélenchon avait fait des gains aux dépens de Hamon tout le long de la campagne. L'appui à Le Pen avait été un peu surestimé mais, comme ses appuis étaient en déclin dans les derniers jours, comme le vote a lieu trois jours après les derniers sondages, la prédiction est à peu près parfaite.

Just a word about the pollsters' performance in the recent French election. This was an excellent performance as everybody noticed since the results for all the main candidates are within the margin of error of the polls, as shown in the graph (using only the polls conducted during the last week). The graph comes from PJ Fournier's blog, with his collaboration:

We had predicted an overestimation of Mélenchon. It did not happen, likely because at the last minute, supporters of Hamon decided to vote for him (he had made gains from Hamon during the whole campaign). Le Pen has been somewhat overestimated but, since support for her was declining during the last week, and since the last polls were conducted three days before the vote, the prediction is quite perfect.

samedi 22 avril 2017

Tomorrow, the French presidential election

First a word to say that there is a new feature in this blog, for this election. This blog is a collaboration with P.J. Fournier (, which allows for a combination of competence. 

And so, today’s topic, what should we expect for the French presidential election tomorrow?

Let us examine first the polls published during the last weeks. This first graph show change in support for the five main candidates since the beginning of March. It shows what other researchers, pollsters and media alike have shown. Support for Macron and Le Pen has varied a little since the beginning of the campaign. Support for Macron is quite stable. Meanwhile, support for Le Pen decreases somewhat. Support for Fillon has been stable lately while Hamon and Mélenchon exchange support. At the end, Mélenchon is now tied with Fillon at 20%. These estimates are dependent upon all the polls published since the beginning of March. 

If we use only the polls conducted since the beginning of April, the analysis is less dependent upon older polls. Is the portrait different? Not really. The conclusions are the same. All the polls show Macron slightly ahead of Le Pen, with Fillon and Mélenchon tied in 3rd and 4th place. 

And what about the margin of error?

If we take the margin of error of each poll, we could conclude, as some have done, that « everything can happen”, this because support for the four main candidates are usually within the margin of error for individual polls. This interpretation is not adequate. The margin of error for the all the polls combined is much smaller. In a way, it is as if we could combine all the samples or if we took into account the fact that the current estimations are dependent upon the previous ones. It is not as if « anything » can happen. Since the method used in the preceding graphs does not allow to show the margin of error of the estimation of the regression lines, the following graph will allow to show this in a more evident way. 

Since there has been lots of movement in the two last weeks, it is possible to estimate the margin of error for all the polls published during that period, in a way as if they had all been conducted at the same time. However it gives more weight to the last polls (using a squared weighting) in order to compensate. It is a conservative estimation of the margin of error. The following graph allows for a visual portrait of the confidence intervals for each candidate’s support. It takes into account all the polls conducted from April 8 to 21. The intervals are not large since there is not much variation between the estimates of the various pollsters because of the methodology they use (see later on in this post).

What can we conclude? If the polls are reliable, it is impossible to be sure who, between Macron and Le Pen, will finish first because statistically they get equal support (the confidence intervals overlap). The same thing happens with Fillon and Mélenchon. However, the graph also shows that the confidence intervals for Fillon and Le Pen slightly overlap. Le Pen could get a score as low as 20.7% while Fillon could go as high as 21%, i.e. he could finish second. On the contrary, Mélenchon is significantly lower than all the other main candidates. In short, if we rely on polls, the two who are most likely to finish first are Macron and Le Pen but the possibility of a Macron-Fillon 2nd round also exists. We have tried other hypotheses using different periods and weights and we get similar results.

Are polls reliable?

This is « the question ». In order to answer this question, one has two rely on two types of information, i.e. the methodological ones and history.

First, methodological information. What is interesting in the French situation is that the pollsters have to file methodological information and their data with the Commission des sondages, a government body. The Commission’s experts can check the data and decide whether the estimates match the data for each poll. The methodological “Notices” are available for everybody to consult on the Commission’s web site here:

One author of this blog, Claire Durand, had examined these files for the 2002 presidential election (see: and the 2007 election In 2002, when respondents were asked whom they had voted for at the preceding election, only around 5% reported having voted for Jean-Marie Le Pen, the extreme-right candidate who had received 15% at the 1997 election. In 2007, JM Le Pen had been over-estimated at 14% while he received 10% of the vote. Just after the first round, only 3% to 7% reported having voted for him. In short, support for JM Le Pen was largely under reported and this caused problems in the estimation of the vote. It was necessary to multiply those who declared having voted for Le Pen by 2-3 times and even more. When we examine the files at the Commission des sondages this year, the situation is very different. Support for Marine Le Pen in the 2012 election is only slightly under reported. Support for the two main candidates, Hollande and Sarkozy, tends to be over-reported, a situation that is usual for that type of question. 

This year, pollsters like Opinion Way and IFOP present the results of its polls, before any weighting or adjustment, and after weighting and adjustment, this for the whole sample and for those who declare being sure to vote. These estimations show that weighting and adjustments have a very small impact on estimations. The information from the other pollsters are less detailed but the information we could consult allow to be confident that the samples are quite representative socio-demographically and socio-politically.  It is thus possible to conclude that a catastrophe like in 2002 is not likely to occur this year. Reports of past vote are very accurate. This could be due to the fact that most polls are now self-administered (web polls) and also to the fact that support for Marine Le Pen is much less “shameful” than support for her father.

However, adjusting using report of past vote that is used in France (and also by many pollsters in the UK) will normally tend to produce an underestimation of the candidates whose share is increasing compared to the previous election and overestimate those whose share is decreasing, this because report of past vote is not very reliable and is even less reliable when time has passed. In short, people tend to adjust their memory on their current voting intention (see:
Given this information and the fact that the data is checked by the Commission’s experts, herding – like what had occurred in the French election of 2002 where five out of six estimates of support for Jospin were similar – would be almost impossible and could be detected. The low variance in estimates is likely due to adjustments using recall of previous votes, a procedure that mathematically reduces variance. 

Now a word about history. Historically, in the 1st round of the French presidential elections, right wing candidates tend to be under-estimated and left-wing candidates over-estimated. In addition, in elections in general, support for small candidates tend to be over-estimated, either because supporters tend to vote less or because they finally decide to cast a “strategic” vote for one of the candidates in the lead. What happened in 2012? The vote for Hollande and Sarkozy has been estimated almost perfectly ( However, the vote for Marine Le Pen had been lightly underestimated (one-two points lower than her final score of 17.9%) and support for Mélenchon had been quite overestimated, at 14-15 %, while he received 11.1% of the vote. 

What are the consequences for tomorrow’s election? Le Pen, who has slightly increase her support compared to 2012 and who has been historically underestimated, could be underestimated on Sunday. As for Fillon, it is more difficult to reach a conclusion. He does not have a positive image in the media and the right-wing candidates tend to be underestimated. However, since he has less support than Sarkozy at the preceding election, adjustment using vote recall could lead to an overestimation. If these two possible effects cancel out, his support is very well estimated. For Mélenchon and Hamon, likely overestimation, Mélenchon because he is on the “far-left” and he has been substantially overestimated in 2012, Hamon, because small candidates tend to be overestimated.  And what about Macron? Well, for Macron, we have no reliable information that would allow to devise whether his support is adequately estimated.

In conclusion, unless there is a catastrophe like in 2002, which is very unlikely given the methodological information provided, the highest probability is Macron – Le Pen for the 2nd round. All the analyses lead to this conclusion. A poll conducted in the middle of last week has shown Le Pen, Fillon and Mélenchon tied but the next polls did not show similar estimates. In fact, the most recent polls all showed increase support for Macron. The second possibility would be Macron-Fillon for the 2nd round. It is not likely but it cannot be totally excluded. Why not a Macron-Mélenchon? In addition to what has already been mentioned, we should add that the Brexit reminded us that “old” people win consultations, and “old” people support Fillon more than Mélenchon. Is it possible that the polls go wrong? In such an election, it is possible, but it is unlikely.