The median home in the US requires a down payment of $78,400. First-time buyers are worried they can't swing it.

• The median home price in the US is sitting at a record high of $392,000 and mortgage rates are on the rise.
• First-time buyers are getting more nervous about their ability to afford a home.

The median home price in the US is sitting at a record high of $392,000, which means homebuyers are scraping together a down payment of roughly $78,000 just to get in the door.

The typical advice to pay 20% down means many would-be buyers are daunted by the prospect.

According to a TD Bank survey of more than 1,000 hopeful buyers, 29% are uncertain if right now is a good time to purchase a home. Affordability seems to be the biggest hang up as 67% cited it as a concern, while 46% said saving for a down payment was a roadblock to purchasing their first home.

"First-time homebuyers are feeling anxious and unprepared in this highly competitive housing market," researchers wrote, adding that 65% of those surveyed said home prices have climbed too high.

In February, the US median listing price spiked at an "unusually-fast" pace to a new record, signaling that the upcoming spring homebuying season is likely to be ripe with competition. And in March, the benchmark interest rate rose for the first time since the start of the pandemic. Further hikes are imminent throughout the year as the Fed attempts to cool inflation. In a housing market deprived of significant inventory, it's no surprise that personal finances have become a chief source of worry for first-time buyers.

"Low inventory, steep competition, and high prices in many areas across the US are fueling fears for buyers and creating an even more daunting undertaking for those entering the market for the first time," TD researchers wrote.

Rising costs are pushing first-time buyers out of the market

For people like Courtney Zaelit, a first-time homebuyer from Bedford, Texas, the uptick may be just enough to keep them from competing in this year's oversaturated and overpriced market. Zaelit, who began her home buying search in 2020, made an offer on her dream home but was outbid by another buyer.

"I felt defeated," Zaelit told Insider. "We knew there were a few offers but we didn't anticipate the seller holding out for the best offer. I know many are looking at numbers and not need. Who can blame them? It just gives us first-time homebuyers a disadvantage."

Frustrated with the homebuying process, Zaelit and her family have returned to renting. This is a familiar tale for many first-time buyers, especially as the demographic seems to be having the most difficulty at finding affordable options in this year's white hot housing market.

"The combination of higher home prices, rising construction costs and moderately higher interest rates will exacerbate housing affordability conditions and increasingly push prospective buyers out of the market in the coming months," Robert Dietz, National Association of Homebuilders chief economist, previously told Insider.

With odds stacked against them, first-time home buyers are becoming increasingly pessimistic about purchasing a home. TD Banks survey claims that 54% of its respondents now view the homebuying process as painful and 64% do not believe it is a good time to buy.

"Whether you enter now or in a few months, you're going to be faced with a competitive market," Steve Kaminski, TD Bank head of US residential lending, said in a statement, adding that the housing market has become "undoubtedly a seller's market."