Here's How All The Pollsters Blew It On The Election's Most Shocking Near-Upset

If this week's elections had taken place three or four days later, Quentin Kidd might have nailed the most shocking race of the midterms when no one else was even close.

Kidd, the director of the Wason Center for Public Policy at Christopher Newport University, began conducting a final poll of Virginia's Senate race 13 days before Tuesday's election. His pollsters were in the field for six nights. And on the final night of polling, he came away with a result that shocked him: Sen. Mark Warner, the Democratic candidate in the race, was leading Republican Ed Gillespie by just 3 points.

"I could see this race closing," Kidd told Business Insider.

However, his time in the field was up, and he could not go solely on the results of a one-night sample of 111 people.

CNU's final poll of the Virginia Senate race, which was conducted over six nights, found Warner up 7 points on Gillespie. Ultimately, Kidd - like pretty much every Virginia pollster and staffers on both campaigns - failed to clearly identify just how quickly the race was tightening.

When the results began trickling in from Virginia on Tuesday night, the political world watched in shock as Gillespie jumped out to a big early lead and threatened to pull off the biggest upset of the night.

By the end of the evening, Warner had the slimmest of leads. As of Thursday, according to the Associated Press, Warner had earned 49.16% of the vote to Gillespie's 48.39%. Warner was ahead by about 16,700 votes out of more than 2.1 million cast.

Dr. Harry Wilson, director of the Institute for Policy and Opinion Research at Roanoke College in Virginia, was among many pollsters who predicted a wide lead for Warner. He found the Democrat ahead by 13 points in his final poll conducted between Oct. 20 and 25.

"The polling industry as a whole, we've got some soul-searching to do individually and collectively," Wilson told Business Insider on Thursday. "I don't know where we go from here. I don't know that any of us have a good idea of what happened."

The results mirrored the shock that buzzed through Virginia in June, when Republican candidate Dave Brat pulled off the stunner of the election cycle and knocked off then-House Majority Leader Eric Cantor in the 7th District's GOP primary.

Warner has claimed victory in the race and is confident he won another six-year term, but his win is still hanging in the balance. Local electoral boards are "canvassing" the results, which essentially means they are double-checking the numbers as well as ruling on provisional ballots. According to sources close to both campaigns, this process should end Friday.

The state board of elections will certify the final results at its meeting on Nov. 25. Gillespie has a few options: If the eventual official results reveal a vote differential of less than 1%, he can ask for a recount. If the vote differential amounts to less than 0.5%, the state will pay for the recount - otherwise, Gillespie's campaign will have to pay.

"We will be watching the results closely so that we can ensure Virginians have confidence in the accuracy of the results. It was an honor to run, and I will respect the decision reached by Virginia's voters," Gillespie said in a statement.

However, most sources on the ground in Virginia who spoke to Business Insider - even Republicans - seem to think Gillespie will not request a recount unless the margins tighten to below the 0.5% threshold. Though they doubt he will be able to mount a successful recount bid, Republicans point out Gillespie is now in great shape to make a run for governor in 2017.

"Ed won't win the Senate, but he laid the foundation for a gubernatorial run that in my opinion will be successful," one Republican operative told Business Insider.

Warner would have been an unlikely victim in a year that one source close to his campaign called "a Republican tidal wave." Gillespie's campaign knew it was going to be different from 2008, a year in which President Barack Obama first ran for president and Warner, a popular former governor of the state, won all but six of the state's 133 cities and counties.

Gillespie entered the Senate race in January, giving Warner a legitimate challenge. However, the incumbent, a popular senator whose approval and favorability ratings ran far above those of Obama in the state, still appeared safe heading into election night. The Real Clear Politics average of polling found Warner almost 10 points ahead of Gillespie.

Some Republicans in Virginia made the case to GOP organizations and outside groups that they should invest in Virginia as the race tightened. Instead, the National Republican Senatorial Committee put its money in North Carolina, a state it won, and Louisiana, where GOP candidate Bill Cassidy is favored to beat incumbent Sen. Mary Landrieu in a runoff.

Some Democrats, meanwhile, have criticized Warner, a self-described "radical centrist" who ran a campaign built on touting bipartisan credentials. However, a source close to Warner's campaign credited that bipartisan focus as essential during a race in which Gillespie tried furiously to tie him to Obama.

"Senator Warner was the only Democrat in the South to win in [the] year of a Republican tidal wave," the source told Business Insider, adding: "He outperformed President Obama's approval rating by 9 points by campaigning on his record of bipartisanship and willingness to reach across the aisle to tackle our debt and deficit in order to invest in our future. In the end, he's still standing in the ring and is going to be strong positioned to be a key bipartisan deal maker in the new Senate. That's good for Virginia, and good for the country."

One Republican operative disputed that notion, telling Business Insider the "hyper-partisan climate" was especially noticeable in Virginia and decimated Warner.

"Mark Warner is the centrist's centrist, and he didn't change that for the campaign. Conversely, Ed beat the anti-Obama drum and successfully tied Warner to him," the operative said by way of explaining Gillespie's surprisingly close finish.

With the final result still in limbo, the campaigns and number crunchers were still scratching their heads two days later and trying to figure out how they got it wrong. Multiple pollsters told Business Insider the only explanation for the tightened race was that a large part of the electorate sat on the fence until the very last minute and then broke for Gillespie.

The large number of undecided voters was something Kidd, of Christopher Newport University, saw throughout the campaign. In January, he found a 20-point difference between the candidates, with a large chunk of undecideds. In midsummer, he found a similar number of voters who had not made up their minds. After Labor Day, when it happened again, he decided to tighten up his likely-voter screen.

Kidd added two additional screening questions. To determine whether voters were likely to show up at the polls, his field workers now asked four questions: how much interest they had in the race, how much attention they paid to the candidates, how likely they said they were going to vote, and how certain they were to act on their impression of the race.

"The Republican electorate has been extremely unstable the last couple of election cycles, and that's what's made the electorate kind of difficult to poll," Kidd said. "To some extent - this isn't an excuse for the entire industry getting some of these elections off - but in Virginia, I think we had a large section of voters move late and make their vote choice. And they all seemed to go for Ed Gillespie."

The polling firm Vox Populi came the closest to identifying the tight status of the race in the moments before Election Day. Last Friday, the firm provided to the conservative news site The Daily Caller the results of a poll that showed Gillespie down only 4 points, a survey that was dismissed by journalists and politicos as an outlier.

Brent Seaborn, a pollster at Vox Populi, told Business Insider he was confident the firm's poll was correct, as it was consistent with the firm's previous findings. He went on the record with friends and colleagues saying Virginia would be a lot closer than it looked.

But even Seaborn said polling over the past three election cycles, at least, has become more and more difficult.