ChatGPT planned a family vacation to Costa Rica. A travel adviser found 3 glaring reasons why AI won't replace experts anytime soon.

• KimKim's head of product asked ChatGPT to plan a Costa Rica vacation for her family of four.
• "It was super easy to use," she told Insider. "On the surface it looks like a great itinerary."

When Yenyi Fu asked the popular AI chatbot ChatGPT to plan a 10-day vacation in Costa Rica for her family, she was initially blown away by the results.

"For someone who doesn't know the destination well, it seems pretty amazing," she told Insider.

Fu, head of product at KimKim, an online travel agency, is no stranger to planning trips. But after showing the chatbot's itinerary to a local travel adviser, she found it contained several logistical errors and lacked a level of personalization that a family of four typically requires.

The software suggested spending two days in five different locations throughout Costa Rica: San Jose, Arenal Volcano National Park, Monteverde Cloud Forest Reserve, Manuel Antonio National Park, and the Guanacaste Province.

While there was nothing wrong with the locations themselves, travel adviser Brittany McNamara immediately found three issues with the plan.

"It's too much for a family with young kids to go to five locations in 10 days," Fu told Insider, adding that the travel specialist recommended traveling to three destinations, tops.

Additionally, some people would advise against spending more than a day in the city of San Jose with young children, she said, particularly if you're looking for a more nature-oriented vacation.

But the largest issue McNamara found was that ending the trip in Guanacaste doesn't necessarily make logistical sense due to its distance from Manuel Antonio and the airport.

With McNamara's recommendations in mind, Fu asked ChatGPT to remove Guanacaste from the itinerary and come back with a revised schedule. In response, the chatbot suggested traveling north to Arenal, then south to Manuel Antonio and ending the trip back north in Monteverde.

While McNamara said this was a slight improvement over the original itinerary, she noted that she typically recommends starting in Arenal, then Monteverde, and ending in Manuel Antonio in order to minimize driving time. She also said most families prefer front-loading their trips with activities and ending them on the beach to unwind.

With an itinerary finalized, Fu asked the ChatGPT to recommend some hotels for her family. The chatbot provided a list of high-end resorts that McNamara said "are not geared towards families or children." And on two occasions, ChatGPT recommended adult-only hotels.

Artificial intelligence won't replace travel specialists any time soon

The confusing suggestions are an example of "hallucinations," or when a large-language model like ChatGPT embeds realistic-sounding falsehoods into its responses. The danger there, Fu said, is that as someone who is unfamiliar with Costa Rica, she would have happily gone along with the advice and potentially not realized the chatbot's mistakes until it was too late.

But despite its pitfalls, Fu said she thoroughly enjoyed using ChatGPT and found it "super easy to use," especially when it came to asking clarifying questions.

"It is definitely amazing to look at but if you dig into it, it doesn't have the knowledge," she told Insider. "It's kind of a funny thing — it has so much knowledge based on all of the internet but because there's so many different ways to do this, it's not really able to figure out the right way to organize this trip for a family of four."

She also pointed out that the chatbot doesn't have access to real-time hotel prices or flights, and can't actually book a vacation for you the same way a travel adviser would.

"Just by itself, I don't think it's going to replace travel advisers' jobs," she said. "There's no ability to help with the logistics, which is what a travel adviser is really helpful for — not just the advice, but also the execution."

ChatGPT appears to agree. When I asked the chatbot if it believes it's better than a travel adviser at planning vacations, its response was surprisingly self-aware.

"I do not have personal experiences of traveling, so I am not able to offer first-hand recommendations or personal opinions," the chatbot wrote. "While I can be a useful tool to gather information and provide ideas for a trip, a travel adviser's expertise and personal experience may be invaluable for planning a truly customized and memorable vacation."