POLLS - Why are they wrong?

Why are polls wrong?

  • Too small a sample

    Votes: 16 47.1%
  • Biased poll

    Votes: 22 64.7%
  • Vague Questions

    Votes: 13 38.2%
  • Lack of participation

    Votes: 15 44.1%
  • False answers

    Votes: 10 29.4%
  • Results are too close

    Votes: 5 14.7%
  • Other

    Votes: 8 23.5%

  • Total voters
    34

ThirstyBarbarian

Well-Known Member
Joined
Feb 11, 2013
Messages
10,927
Reaction score
5,058
Actually, I did watch it (well, most of it). I felt there were various issues in their methodology and obviously had an agenda.
In terms of split tickets, I consider myself "middle of the road". I will vote for a moderate politician (of either party) over an extremist from the other party (again, of either party).
But I agree, my goal was to understand why in 2020 and 2022 there were so many surprises and I do not want to go down the political path related to 2000 mules or other voter fraud discussions.....

I think Thirsty's comments about many low quality polls used to create a narrative for the various talking heads is germane.

Related to that, if there is a close race, and polls show the spotted candidate is slightly ahead, does that cause the striped base to work harder to catch up? Or if the polls are strong for the spotted candidate, might more of those supporters actually skip voting because of a "sure thing" (think Dewey vs Truman)?

Do poll results (or at least the narratives) create a feedback loop that then influences voters and impacts the poll results?

I think there can be a feedback loop where past polling results can affect future polling results and election results to a small degree, but maybe not make a significant difference. I think it probably only comes into play when the margins are already pretty big.

I think it can affect polling by changing the likelihood that certain political demographics will respond to a poll. If polls show your candidate is doing well and you feel really good about their prospects in the election, you might be more inclined to answer a poll than you would if polling showed your candidate down. So polling that causes an enthusiasm gap might affect later polling results by widening the margin a bit.

And an enthusiasm gap can also possibly affect actual election results. If you don’t think your candidate can possibly win, you might not bother voting at all. But I think that would likely just come into play when the margin is already pretty big.

There’s a theory that polling and the election narrative leading up to the 2016 election might have affected the actual outcome, which would be kind of an exception to the rule. The idea is that the 2016 election was an odd one because it was so much of a lesser-of-two-evils election. Some elections are like that, but 2016 was extreme. Many voters were not voting for a candidate as much as voting against the other one. There was a core of Clinton supporters excited to be voting for a female president, and there was a core of Trump supporters excited to be voting for an anti-establishment outsider. But many rank and file Democrats and Republicans were not enthusiastic about their own party’s nominee, and were more motivated by strong dislike for the other party’s nominee. As the election got closer, polling was pretty tight, but it showed Clinton ahead by a few points. Analysts ran the polling data through their Monte Carlo sims, and came up with something like a 70% likelihood that Clinton would win (meaning a 30% chance Trump would win). And then the narrative developed that Clinton pretty much had it in the bag, ignoring the fact that 30% of simulations had Trump winning. The theory is that some Democrats felt like they didn’t need to turn out and vote for Clinton, which they weren’t very enthusiastic about doing, because the narrative they were hearing was that Trump was going to lose, which was all they really cared about. Wrong! Some people believe that if people had a better understanding of what the polls really meant, and how close the race really was, they may have been more motivated to turn out, and the results might have been different. It’s an interesting idea, I’m not sure I believe it, and we will never really know.
 

lakeroadster

Improvise, Adapt and Overcome
TRF Supporter
Joined
Mar 3, 2018
Messages
6,427
Reaction score
6,603
Location
Central Colorado
One clear result of the 2022 midterms was that candidates who promoted 2020 election denial, conspiracy theories, support for January 6th insurrectionists, or the idea that if they won they would somehow overturn or “decertify” the 2020 results or refuse to certify future results — those candidates did not poll well leading up to the 2022 midterms, and they lost.

I don’t think a single candidate promoting these ideas while running for statewide offices like Governor, Lieutenant Governor, Secretary of State, Attorney General, or Senator won. They all lost, and many of them lost very winnable seats that their party would probably have won with less wacko candidates. Polling showed that voters are sick of the BS and lies, and the election results bear it out. That’s another issue where you can see ticket splitting in the 2022 election where an election denier lost while another less cuckoo candidate from the same party won in the same state. Deniers underperformed non-deniers. It’s clearly a losing issue.
So... you didn't watch it either I'm guessing. :dontknow:
 

ThirstyBarbarian

Well-Known Member
Joined
Feb 11, 2013
Messages
10,927
Reaction score
5,058
So... you didn't watch it either I'm guessing. :dontknow:

LOL! No!

I did watch the January 6 committee hearing in which Trump- appointed Attorney General Bill Barr laughed his ass off at the idea that “2,000 Mules” would have changed his mind. So that was kind of funny.

Nope. I’m full up on conspiracy theories. I’d suggest that the way you handle election disputes in this country is you put your claims and evidence and proposed remedies into a court case and run it through the legal system and abide by the results. That’s not partisan, that’s the way it works. Were the claims in “2,000 mules” ever taken to court? What happened?

Anyway, trying to avoid partisan politics, and getting back to the polling topic — this election denialism issue does not poll well. And the election results reflect the polling in that it’s an issue that loses elections. Voters are tired of the BS, lies, anger, outrage, and extremism. They say so in polls, and they back it up with votes.

In the recent election, in Arizona there were cases of people staking out ballot drop boxes, dressed in tactical gear, carrying rifles, taking pictures and video, and shouting at voters, “You’re a mule!” That’s the kind of extremist behavior inspired by these conspiracy theories, and the “mules” language is straight from “2,000 Mules”. How did that play out? People don’t like it. Extremism, election denial, attempted intimidation rallied the opposition and cost extreme candidates winnable seats for Governor, Secretary of State, Attorney General, and Senator.
 

lakeroadster

Improvise, Adapt and Overcome
TRF Supporter
Joined
Mar 3, 2018
Messages
6,427
Reaction score
6,603
Location
Central Colorado
LOL! No!

I did watch the January 6 committee hearing in which Trump- appointed Attorney General Bill Barr laughed his ass off at the idea that “2,000 Mules” would have changed his mind. So that was kind of funny.

Nope. I’m full up on conspiracy theories. I’d suggest that the way you handle election disputes in this country is you put your claims and evidence and proposed remedies into a court case and run it through the legal system and abide by the results. That’s not partisan, that’s the way it works. Were the claims in “2,000 mules” ever taken to court? What happened?

Anyway, trying to avoid partisan politics, and getting back to the polling topic — this election denialism issue does not poll well. And the election results reflect the polling in that it’s an issue that loses elections. Voters are tired of the BS, lies, anger, outrage, and extremism. They say so in polls, and they back it up with votes.

In the recent election, in Arizona there were cases of people staking out ballot drop boxes, dressed in tactical gear, carrying rifles, taking pictures and video, and shouting at voters, “You’re a mule!” That’s the kind of extremist behavior inspired by these conspiracy theories, and the “mules” language is straight from “2,000 Mules”. How did that play out? People don’t like it. Extremism, election denial, attempted intimidation rallied the opposition and cost extreme candidates winnable seats for Governor, Secretary of State, Attorney General, and Senator.

It's not a theory... when there is documented evidence and it's telling when folks are called extremists... for doing research and then presenting that research so others can make up their own minds.
 

Marc_G

Well-Known Member
TRF Supporter
Joined
Jun 5, 2010
Messages
8,338
Reaction score
2,452
Location
Indianapolis Metro Area
Several articles here may have valuable insights for those interested in the polling accuracy topic:

 

ThirstyBarbarian

Well-Known Member
Joined
Feb 11, 2013
Messages
10,927
Reaction score
5,058
It's not a theory... when there is documented evidence and it's telling when folks are called extremists... for doing research and then presenting that research so others can make up their own minds.

When they take their documented evidence of crimes and fraud to court, I’ll listen. Until then, it’s getting lumped in with all the other BS and lies that previously failed in court.

What has happened since the 2020 election is that a lot of people made a lot of bogus claims in public where their free speech rights allow them to lie pretty much with impunity, but in court, where evidence is required, they didn’t make the same claims, because it could get them into legal trouble for lying in court. Or they would make the claim, but not provide evidence, and their case would get thrown out.

We are supposed to be a rule-of-law society, so take it to court. That applies to everyone, regardless of party.

Anyway, “2,000 Mules” is definitely off topic and veering into dangerous forbidden subject matter, so I already regret engaging on it. I’m done with that. Back to polling…
 

ThirstyBarbarian

Well-Known Member
Joined
Feb 11, 2013
Messages
10,927
Reaction score
5,058
Several articles here may have valuable insights for those interested in the polling accuracy topic:


One thing I really like about FiveThirtyEight is that they don’t get out ahead of the data, and they do a good job of explaining their methods and the limits of what can really be learned from polling.

This is the article I had in mind when I wrote some of my earlier replies. https://fivethirtyeight.com/features/2022-polling-error/
 

ThirstyBarbarian

Well-Known Member
Joined
Feb 11, 2013
Messages
10,927
Reaction score
5,058

When you cut the sentences in half and delete the rest of the context, it reads funny. But the post is not partisan, and it’s clear the second sentence is changing the subject away from politics and back to polling.

What do you have to say about polling? You haven’t said anything about the topic yet.
 

Funkworks

Low Earth Orbit, obstructing Earth's view of Venus
Joined
Jul 28, 2018
Messages
4,625
Reaction score
5,233
Do poll results (or at least the narratives) create a feedback loop that then influences voters and impacts the poll results?
Yes they can. People can change their minds unpredictably for different reason. I'd guess that marketing polls are more reliable than political polls.
 

Tyeeking

Well-Known Member
Joined
Jun 27, 2019
Messages
289
Reaction score
323
When you cut the sentences in half and delete the rest of the context, it reads funny. But the post is not partisan, and it’s clear the second sentence is changing the subject away from politics and back to polling.

What do you have to say about polling? You haven’t said anything about the topic yet
You reference a partisan committee as something you watched rather than a partisan movie and then say the post is non partisan. Ok 😉

We all have partisan bias. Sometimes the tell is in what we say, other times how we say it, and sometimes what we don’t say.

As for polling . . . I tend to take them with a grain of salt. The polls were all over the place this last election cycle so by selectively citing specific polls one can argue they were accurate or that they were not. There were enough of both to go around for which ever argument one is inclined to make.
 

Marc_G

Well-Known Member
TRF Supporter
Joined
Jun 5, 2010
Messages
8,338
Reaction score
2,452
Location
Indianapolis Metro Area
You reference a partisan committee as something you watched rather than a partisan movie and then say the post is non partisan. Ok 😉

We all have partisan bias. Sometimes the tell is in what we say, other times how we say it, and sometimes what we don’t say.

As for polling . . . I tend to take them with a grain of salt. The polls were all over the place this last election cycle so by selectively citing specific polls one can argue they were accurate or that they were not. There were enough of both to go around for which ever argument one is inclined to make.

I agree any one poll can be very far off, which is why I only paid attention to meta-polls such as 538. It blended the polls with weighting based on historical data and other data-driven corrections. Their approach is historically pretty accurate and it performed well the last few cycles.
 

ThirstyBarbarian

Well-Known Member
Joined
Feb 11, 2013
Messages
10,927
Reaction score
5,058
You reference a partisan committee as something you watched rather than a partisan movie and then say the post is non partisan. Ok 😉

We all have partisan bias. Sometimes the tell is in what we say, other times how we say it, and sometimes what we don’t say.

As for polling . . . I tend to take them with a grain of salt. The polls were all over the place this last election cycle so by selectively citing specific polls one can argue they were accurate or that they were not. There were enough of both to go around for which ever argument one is inclined to make.

Well, I disagree the committee is partisan or that my watching it was partisan, and I brought it up only to mention the comments made by a Trump appointed official about “2,000 Mules”. Am I being too Trumpy, bringing up comments by Trump-appointed AG Bill Barr? ;)

I acknowledged it was getting too close to partisan politics and tried to get back on track:

Anyway, trying to avoid partisan politics, and getting back to the polling topic — this election denialism issue does not poll well.

You cut off the part of my sentence about getting back on topic, so that’s your dishonest framing of what I said.

Anyway…trying to get back on topic AGAIN, I agree, you definitely SHOULD take polls with a grain of salt! At least take the narratives that accompany polling with a grain of salt. Elections have been so close in recent cycles that the polls don’t tell us much about outcomes, other than it’s going to be close, and either side could win. That’s not very satisfying to people who want some kind of certainty about the direction of an election, so they form narratives that try to make sense of something unpredictable.

Politicians and their supporters try to get out ahead of the data and frame a narrative they think might be helpful to them, and the media like to settle on a narrative as well so they sound like they know what they are talking about. But if you listen to the polling experts and analysts, they are typically not pushing those narratives or making those kinds of definitive conclusions.

If polls are showing one candidate or party favored by 2 points, and the margin of error is +/- 5 points, what does that tell you? Pretty much nothing. You would rather be up by 2, obviously, but the margin of error means the race is really a tossup.

So what if you are looking at the House of Representatives and there are 60 competitive seats with a 1, 2 or 3-point lead, and they are all leaning the same direction toward the polka dot party? If they all go that way, there will be a polka dot wave of historic proportions! But those polling leads all have a +/- 5 margin, so they really could go either way.

You can be guaranteed in that scenario, there will be a narrative about the coming polka dot wave, even though it’s not supported by data. That’s pretty much what happened in the 2022 midterms. Polls were not necessarily wrong — the narrative was wrong, despite warnings by 538 and others that the data weren’t supporting that level of confidence in a wave.
 

teepot

Well-Known Member
TRF Supporter
Joined
Dec 7, 2019
Messages
3,361
Reaction score
3,604
Location
Pahrump, Nevada
I remember a guy I went to college with. He was taking a statistics classes. He said once that he could use the same statistics and argue either side of the debate and win the arguments. I wouldn't trust a poll for a lot of the reasons put forward here.
 

ThirstyBarbarian

Well-Known Member
Joined
Feb 11, 2013
Messages
10,927
Reaction score
5,058
So now, with the 2022 elections not even fully finalized, we are already getting polling related to the 2024 election, something which, when I polled myself (sample size 1), proved to be very unpopular!
 

bjphoenix

Well-Known Member
TRF Supporter
Joined
May 3, 2011
Messages
1,558
Reaction score
641
They don't ask everybody so they have to ask a representative group. It is hard to find a representative group and even harder because you don't know who is going to vote and who is going to stay home. Also I wonder if people when asked give the same answer as when they are actually voting.
 

OverTheTop

Well-Known Member
TRF Supporter
Joined
Jul 9, 2007
Messages
7,978
Reaction score
6,274
Location
Melbourne Australia
We had state elections here last weekend. Polling showed the results were likely very close and possibly a hung parliament with minorities holding controlling votes. The previous Labor government, who had been in power during covid, absolutely romped home, with 52 seats compared to the Coalition (Liberal and National Parties combined) of 25. It was called as close before the counting due to the vociferous people on media and social media. In actual fact the silent majority spoke volumes that were not heard in the leadup to the election.
 

les

Well-Known Member
TRF Supporter
Joined
Jan 19, 2009
Messages
3,199
Reaction score
952
They don't ask everybody so they have to ask a representative group. It is hard to find a representative group and even harder because you don't know who is going to vote and who is going to stay home.

In actual fact the silent majority spoke volumes that were not heard in the leadup to the election.

These comments appear to be related to too small a sample and/or lack of participation.
Lack of participation leads to too small a sample group, or potentially creates a bias based on who is willing to participate..........
 

DEmery

Well-Known Member
Joined
May 16, 2015
Messages
292
Reaction score
273
Location
Red Lodge, MT
I think lack of participation is the biggest factor. Ever since Brendan Eich lost his job as CEO of Monzilla for his history of donating to support Prop 8, I saw the power of the liberal mob. Why would I give answers to a pollster that has my phone number?
 

dr wogz

Fly caster
Joined
Feb 5, 2009
Messages
8,825
Reaction score
4,714
Location
Land of Poutine!
60% of the time, I'm right all the time! :D

I think it also comes to those who not only interpret the polls, but it's also those who already have their minds made up. so, the polls are right, if they agree with them, or are wrong, if they don't.
 
Top