How well did polls predict the midterm winners?

CATHY WURZER: Well, the polls were pointing to strong Republican gains both nationally and here in Minnesota that didn't materialize. We're going to talk a bit now about the polls during this election cycle. How accurate were they? Joining us is Craig Helmstetter, who is the head of the APM Research Lab, which is part of our parent company, American Public Media Group. How tired are you, Craig? I bet you were up all night long watching.

CRAIG HELMSTETTER: Yeah. Well, fortunately, there was an extra Red Bull hanging around the newsroom this morning, so I'm all right.

CATHY WURZER: Good. Excellent Well, the polls predicted this red wave that didn't happen. So what exactly happened?

CRAIG HELMSTETTER: Well, I have been parsing those numbers a little bit this morning, and I'm going to start off in a cantankerous way for you this morning, Cathy, and challenge the premise of your question. Actually, as I look at the polls, I really don't think it's fair to say that the polls predicted a red wave. I think that did become a narrative, and maybe some polls did tend in that direction.

But there are places like FiveThirtyEight that put together an average or an aggregation of these poll questions that asked people, would you prefer a Democrat or a Republican in Congress? This is the so-called generic ballot question that many pollsters ask, and that would signal a red wave if there were one.

But instead what we really find when we look at those generic ballot questions across the country-- Republicans were indeed ahead from the beginning of the year through the middle of July, and then Republicans and Democrats were tied for a bit. And then Democrats emerged as favored by the public. Democrats were actually ahead from August until October.

And then in the last few weeks, Republicans gained the advantage, so there was this back and forth. If there were a red wave, it would have been constantly Republicans ahead in these pollings. And furthermore, when they were changing places, it was really switching only by a couple of percentage points. So I really don't think it's fair to say that there was a red wave predicted by the polls.

CATHY WURZER: All right. Here in Minnesota, the SurveyUSA/KSTP poll had Tim Walz up by eight in their last poll. The Minnesota poll, which you had a hand in, had Walz up by seven. The outlier was that Alpha News/Trafalgar poll which showed the gubernatorial race neck and neck, right?

And it appears the first two polls were pretty close to the final mark, but these polls showed Attorney General Ellison and State Auditor Blaha in trouble and losing by several points potentially. What happened there? Parse that for me.

CRAIG HELMSTETTER: Yeah. Well, first of all, some general caveats just about polling in general. One thing is that polls are a snapshot in time, and things can happen. The horse race analogy is quite apt. You can see that your horse is ahead, but that horse can stumble and fall before it gets to the finish line.

Secondly, the other thing that's good about the horse race analogy is that they do a photo finish where you can precisely see who won, and that's what the election results are. But prior to that with the surveys and polls, we're kind of sitting in the stands with a pair of binoculars, and things are a little bit fuzzy. There's a margin of error around the poll results that come out.

So they're not necessarily predictive, but you are right. This KSTP poll was done in late October, so you think it should be fairly close. And it's done by a pollster with a high rating. But what happened there-- if we look closely at the attorney general race, that had Schultz up by seven points over Ellison.

But that margin of error was a 3.9 percentage point margin of error, so it was just barely beyond that margin of error. In addition, 9% indicated that they were undecided in that poll. So when you take into account the undecideds plus the margin of error, there was plenty of room for it to turn out in the sort of 50/50 way that the actual election results came out.

In addition to that, the two prior public and nonpartisan polls that were done in October-- they had the race virtually tied, as did the September poll that we did here with MPR News, Star Tribune, and KARE 11. So if you just take the one data point from the most recent poll and ignore margins of error and undecided, you might think that the polls were off. But in fact, when you look at the whole picture, I think they were pretty close.

CATHY WURZER: And the exact same thing happened, as you say, with the state auditors' race?

CRAIG HELMSTETTER: That's right. Same thing happened right there.

CATHY WURZER: Interesting. I'm wondering here-- the other big race to watch was that second district congressional race between incumbent Angie Craig and Tyler Kistner, and Democrat Craig won by some 5 percentage points in that match-up with Tyler Kistner. What were the polls saying there?

CRAIG HELMSTETTER: Yeah, congressional districts are even harder to pull than statewide races. They're smaller areas. And so they're much more expensive. And we really only had one nonpartisan public poll that was done in that region.

Again, that was another KSTP poll, and that was done in mid-October. And in that poll, the two were statistically tied. There was really no difference. I think, technically, they had Craig up by 1 percentage points. But again, that margin of error, that means that they're tied. Interestingly, I'll also point out, however, that related to polling, there are these election forecasters.

CATHY WURZER: Yes.

CRAIG HELMSTETTER: I already mentioned FiveThirtyEight, Cook Political Report does this, Inside Elections, POLITICO, and so on. We tracked six election forecasters on our Minnesota poll watch site. And of the six, four of them kind of wimped out and just called it a toss up and failed to make a call.

But The Economist and FiveThirtyEight, they were predicting that Craig would win. And some of their predictions are in part based on polling, as well as other dynamics. And then they went out on a limb, and they were right. Craig did come out ahead.

CATHY WURZER: Say, I'm going to throw this question at you maybe out of left field, but you know your business. How difficult is it getting to get decent polls in that so many people don't want to answer their phones? If they do, they're kind of really skittish about answering questions. How tough is it nowadays?

CRAIG HELMSTETTER: It is really, really tough, and it's getting tougher all the time. That's why as somebody who does research, who does survey research, who's a member of the American Association for Public Opinion Research, I was as nervous about the polls as I was about the election outcomes during last night's race. And as we've just discussed, the polls aren't perfect. But they do help us provide insights into what the public is thinking. It helps us provide a voice for Americans in a different way than just interviewing people on the streets. Hopefully, it's a good mix and continues to be a strong input into the sort of work that we do here at the APM Research Lab and with you at Minnesota Public Radio news.

CATHY WURZER: I'm really glad that you had an opportunity to come on and talk a little bit more about this. We appreciate what you did during this election cycle. Thanks much, get some sleep.

CRAIG HELMSTETTER: Oh, you too, Cathy, thank you.

CATHY WURZER: You're welcome. Craig Helmstetter is the head of APM Research Lab.