In a previous post we recreated the Guardian/Yougov poll but with scientific methods. Showing that the support for Ukraine is not dropping sharply in Sweden as they falsely stated. Also rising the question if the poll using unscientific methods is so wrong compared to thw Swedish population, it may very well be the same for other countries polled in the article, like UK, Germany and France.
We did however change the question a bit. The Yougov poll most likely had a “don’t know” alternative, because the two presented data points do not add up to 100 %. Indicating that the respondent could choose a third alternative. Here it should be noted that the presentation of the poll in the Guardian is very unclear. Hiding the “don’t know” answer hides the opinion from the poll of a quarter of the people responding to the questionnaire. It is not the population, because the nonscientific method used hinder the article to claim that the poll is giving a view of the population of the country.
Using a scientifically proven method shows how much it differ. See chart below. Gallup Nordics daughter company Novus is using a probability based panel. Yougov is using an open access panel. The latter introducing so many unknowns in the sample that it is not possible to claim that the result is showing the view of the population, the only real claim you can do is to say so many of the email addresses that clicked the link also answered like this. Because it is also not possible to say that one email address equal one person. Definitely not one person in one country. But not more on that in this text.
In our first recreation of the poll we did not include a don’t know alternative. forcing the respondent to make a decision. The result surprised me that the “negotiation” alternative got a relatively high support. We could have asked the respondent to give a graded answer, that would have been more interesting, but then we would come to far away from the one we wanted to test. We also could have included a “don’t know” alternative, but since that was not presented in the article as even existing, we chose to force the respondent to make a decision, it is a well known topic to all, although complicated in practice, it is possible to answer by making a more or less educated guess. However it bothered me that we did not have the don’t know alternative as well.
So we recreated the poll once again. 1014 interviews during jan 16 – jan 21 2025 in Novus probability based panel, since being probability based it give the opinion of the Swedish population between 18-85 years old.
As you can see in the chart above, when introducing “don’t know” as an alternative, the swedes who being forced to chose clearly move away from a negotiation solution. The don’t know is not evenly spread between supporting Ukraine and a negotiation solution.
Support for Ukraine drops 3 percentage points,within the margin of error. (since this is probability based sample we can count on the margin of error, that is not possible in the Yougov poll, but that is for another blogpost in the future I think).
So the support for Ukraine is basically the same in both polls. But, the support for a negotiation solution drop 10 points. That is statistically significant. The drop is there because the support for that solution is much weaker when presented with a don’t know alternative. 13 % of the Swedish population chose “don’t know”. 10 % in favor of now choosing than a negotiation alternative.
Swedes have grown up with threats and bullying from Russia, but still wanting to be neutral. But the invasion of Ukraine shifted the minds of Swedes in a record time. The support for joining NATO that Novus researched was a historically fast shift in an opinion in a country. And there is a lasting majority for NATO in Sweden as a direct reaction to the invasion of Ukraine. One of the reasons I reacted to the Guardian/Yougov poll. I was certain it was not correctly describing the view of the swedes.
But I was also surprise over the support for a negotiation solution from a quarter of the Swedish adult population. Now I see what happened, that support is much lower, but there are 10 % being forced to chose based on the knowledge they have now, then would choose a negotiation solution. But add to that, the fact that no one in Sweden is talking about a negotiation solution, there is no information of what that would be. And also that there is a overwhelming support from the swedes for Ukraine, it is also a politically dead issue to try to push that opinion in the public.
I would have hoped that someone in for instance France, Germany or UK did a similar study. Because I know that the problem with the poll in the Guardian is using a nonscientific method. I also know that when we replicate weird nonscientific polls using proper methods we almost always see huge differences. Differences that never occur when replicating polls based on scientific methods.
So here we look at the Guardian/Yougov with the missing 26 % to sum up to 100 % in a don’t know answer. Compared to Novus poll also with a don’t know answer. This shows two completely different views on the Ukraine war. One based on scientific methods and the other on nonscientific methods…
During almost 15 years that we have looked at nonscientific polls vs scientific we see that: If there is an outlier there is always the nonscientific poll. Should not be a surprise, BUT still credible news media use nonscientific polls as being newsworthy… How can something you do not know to be true be news? I am pretty sure then it is just gossip or rumors… Or just fake news…
Torbjörn Sjöström
CEO Novus and Chairman Gallup Nordic
Leave a Reply