- Federal surveys measuring prices, job openings, and other stats have been seeing declining response rates.
- That's concerning for markets that have become ultra-sensitive to economic data because it affects Fed policy.
Inflation. Jobs. Home prices. There's lots of data on the health of our economy. But what if those numbers aren't giving us an accurate picture of what's happening?
Surveys that gather market-moving statistics are capturing a narrower slice of the population. For example, the response rate on surveys tracking the consumer price index sank from 67% in 2016 to 53% this year, according to the Bureau of Labor Statistics. For job openings, the rate tumbled from 66% to 31%.
That's concerning for markets, which are trying to anticipate the Federal Reserve's next data-driven move and have become a lot more twitchy about economic reports.
"They're not giving us as accurate a picture," former Fed economist Claudia Sahm told Business Insider. "And again, where one would start to see this first is the precision of the estimates slips, and you get movements from months to months that aren't reality."
If the quality of the surveys gets worse, variability will get bigger, she added. "And then that means that markets are going to get knocked around more and react to things that aren't really there."
What this means for investors
The quality of the data is especially critical now as the Fed approaches a pivotal point: shifting from rate hikes to cuts.
Falling response rates can increase the uncertainty about the economy, Goldman Sachs said in a report last month, pointing out larger-than-usual data revisions. In fact, job openings have been revised by an average of 180,000 over the last few years, more than triple what was typical six years ago.
At the same time, the market has become much more sensitive to individual data releases in recent years, as they weigh on Fed policy, Goldman added.
Just last month, the stock market rocketed higher after a cooler-than-expected inflation report. It also nosedived in September and surged in July on other data.
With markets swinging wildly after every statistical bump and wiggle, Sahm said investors should always be cautious — but not suspicious — when reacting to the latest numbers.
"Don't overreact to some sharp turn, because it might not be real or it could turn back really fast," she added.
Why are response rates falling?
Response rates have been in decline for a host of reasons, such as people becoming more private and suspicious of officials asking a bunch of personal questions. Remote work during the pandemic affected rates too.
There are also more "gatekeepers" who prevent statisticians from reaching respondents. Think administrative assistants, voicemail, email, smart doorbells, and locked buildings.
The Bureau of Labor Statistics is well aware of the declining response rates, which it frequently publishes online. In March, the BLS said it was maintaining the quality of the data by expanding sample sizes and reaching out to more respondents via phone calls.
In spite of the decline, government statistics are still considered the highest quality data that's out there.
That's because it's hard to beat the scope of the federal surveys that reach hundreds of thousands of people, according to Gerald Perrins, a top official overseeing consumer price data at BLS.
"Although some of the response rates might be declining, our response rates are relatively high considering the large sample sizes that we do," he told Business Insider.
Take the CPI. Almost every month, government economists and statisticians set out to collect price data on everything from eggs to MRIs and cable TV subscriptions. In 2022, the CPI included data from an average 259,504 respondents in US cities.
"Our goal is to put the best possible gold standard data out there," Perrins said.