As an expert witness for Whole Foods, Kellyanne Conway's testimony was deemed 'fundamentally flawed' and thrown out

Since becoming a regular on television as President Donald Trump's campaign manager and later top counselor, Kellyanne Conway has raised doubts about the veracity and credibility of her answers.

But while some television hosts like MSNBC's Mike Brzezinski no longer trust Conway's answers enough to put her on television, she isn't the first to reject the Trump counselor for a lack of credibility.

In 2007, Conway - then a Republican pollster running the research firm The Polling Company - was hired as an expert witness by Whole Foods in a lawsuit filed by the Federal Trade Commission in Washington.

The FTC sued Whole Foods that year to block the upscale grocery chain's purchase of Wild Oats, a North American grocery store chain known similarly for its organic products, arguing the merger would reduce competition for organic goods and increase food prices in several markets. To support this, the FTC produced a series of internal company emails and public blog posts that showed the company's desire for Wild Oats' "monopoly" over their local markets.

During the case, Whole Foods retained Conway to design a survey to support the testimony of David Scheffman, a former FTC official with experience testifying on behalf of companies in anti-trust suits. He argued that Whole Foods and Wild Oats shoppers frequently shopped at other grocery chains, and that these other supermarkets competed for the business of crossover customers.

In other words, Whole Foods and Wild Oats couldn't raise prices without losing customers to competing grocers, voiding the FTC's concerns about a monopoly.

But it didn't exactly pan out for Conway or the mega-supermarket chain.

The DC District Court threw out Conway's survey, saying it would not give "any weight or consideration" to it because of its methodology.

The court found the FTC's expert witness Kent Van Liere presented compelling evidence about the study's drawbacks, which the agency said suffered "serious and fatal design flaws, poor execution, and conclusions that are inconsistent with the survey's results," and failed to "meet basic standards" of survey research.

"My overall opinion in this matter is that Ms. Conway's survey methodology and procedures are fundamentally flawed and render her data and results unreliable. In addition, it is my opinion that her survey does not provide a reliable basis to assess the issues associated with consumer perceptions of the substitutability of products and services across food retailers," Van Liere testified, according to July 2007 court documents.

Van Liere cited Conway's response rate, which was "so low that her results cannot be considered reliable," and included "unqualified respondents," who could not "understand or could not complete the questions accurately." Her response rate, according to Van Liere, was between 6% and 0.5% of every caller the pollsters attempted to reach, and over 20% did not actually reside in the zip codes designated in the survey.

The court also agreed that the survey was too complicated.

The FTC argued that survey respondents said they shopped mostly at Whole Foods or Wild Oats, but responded to later questions saying they "purchased most or all of the same product categories from other food retailers."

"Inconsistent survey results reveal the presence of underlying flaws in the survey design itself, such as the presentation of questions that are confusing, complex, and poorly worded, which here led to unreliable results," the FTC argued in its July motion to exclude Conway's testimony.

Further, court documents noted that Scheffman himself didn't ultimately rely on Conway's report, despite the fact that it was conducted on his behalf and cost Whole Foods over a quarter of a million dollars, according to court documents filed in July 2007.

Despite the dismissal of the survey, some experts argued that the court was probably too harsh on Conway's survey.

"It makes sense that you would have to cast a very wide net to find enough people who were somewhat regular shoppers at these stores, so the response rate doesn't necessarily bother me," Tom Jensen, director of Public Policy Polling, told Business Insider. "I'm used to that kind of thing if we're trying to survey people who listen to a particular radio station, for instance."

But Jensen said the results suggested the court was correct that the survey was confusing.

He said it was "very curious" that Conway's survey claimed to interview respondents who were in the designated zip codes, yet included respondents who didn't live in the zip codes as well.

"The survey is unreasonably complicated for most poll respondents to be able to reliably respond to," Jensen said. "There's no way people know off the top of their head how much they spend at different grocery stores on a monthly basis for different categories of products, so I agree that it's a stretch to make very broad assertions on that data."

The survey's rejection by the court wasn't a rebuke of Whole Foods' position writ large - the court found the testimony from another Whole Foods expert witness, Jon Stanton, relevant in its description of grocery store categories.

The case itself became less relevant as other grocery stores cashed in on the organic food trend and Whole Foods attempted to lower prices on some goods as consumer spending dipped during the beginning of the recession in 2008. The two parties settled in 2009, with Whole Foods agreeing to shut down over a dozen stores and sell the remaining locations.

The settlement may have been a blessing in disguise for Whole Foods: According to a 2009 Reuters report, many of the stores Whole Foods was forced to sell were not high-performing locations.

For her part, Conway's political career has largely been dedicated to polling.

Before managing Trump's campaign, Conway for decades worked in Republican circles as a pollster, with a particular focus on female voters. Her group, The Polling Company, made almost $2 million conducting surveys for candidates like Sen. Ted Cruz, former candidate Ben Carson, and Trump during the 2016 election, double the amount from any previous campaign cycle.

Poor polling clearly frustrates Conway - The New Yorker reported that during a presidential debate last September, Conway chided an assistant tweeting unscientific polls from the then-campaign manger's Twitter account.

Conway, Whole Foods, and The Polling Company all did not respond to Business Insider's request for comment.