Don't worry — the machines won't be as smart as humans for a long time, because they've still got a lot to learn

• AI has dominated the agenda at the World Economic Forum in Davos this year.
• Experts have been adamant that AI – in its current form at least – is pretty limited, despite the hype.

AI might have been the talk of the town in Davos this year, but some experts in attendance had a pretty sobering message for everyone there: AI still has a long way to go to get real smart.

It's easy to understand why AI was top of the agenda at the World Economic Forum in Switzerland. After all, it's in the midst of a hype cycle that would make Web3 blush.

In the year since world leaders last convened for the annual fair, Big Tech titans like Google and Microsoft have scrambled to match OpenAI's ChatGPT, while Bill Gates has touted the technology's world-changing capabilities.

But despite all the hype, AI experts have been adamant this week that AI – in its current form anyway – is pretty limited in scope. Especially if the end goal of the field is to create artificial general intelligence. Here's why.

AI is scratching the surface

During a panel discussion on Tuesday on generative AI, experts first pointed to data challenges that need to be overcome to make today's AI a lot smarter.

Daphne Koller, a computer scientist and MacArthur "genius," told the panel that "we're only starting to scratch the surface of the data that are available."

Much of today's most popular AI models, such as OpenAI's GPT-4, are trained on what's publicly available on the internet. The kind of data Koller would like AI to handle goes beyond that.

For one, there's a world of data that can come from so-called "embodied AI." This is AI embedded into agents, like robots, that can interact with the physical environment. Today's chatbots don't really get much of that data.

Right now, there are specific instances in which AI interacts with this kind of environment to collect data. Consider the ways autonomous cars pick up and analyze data about road traffic or the way AI is used to detect early signs of retinal diseases.

The only problem is, an all-purpose AI model that can analyze and process all of this data, on top of data from the internet, doesn't yet exist in any meaningful way.

Data that comes from experimentation is lacking too.

As Koller noted, the ability to "experiment with this world" is part of what makes humans so effective at learning. AI's ability to do this, by comparison, is currently lacking.

One solution to this data issue is giving machines the chance to create their own synthetic data – rather than just rely on data created by humans that are fed to it from the web.

"If we want these machines to grow, we need to give them the ability not just 'in silico' talk to each other … but really to experiment with the world and generate the kind of data that helps them continue to grow and develop," she said.

The architecture problem

The other problem experts pointed to revolves around architecture.

For Yann LeCun, chief AI scientist at Meta, it's clear that autoregressive large language models (LLMs) – the models underpinning today's AI chatbots – are in need of "some new architectures" to reach the next level of intelligence.

Right now, AI models like LLMs work by taking a piece of text, for example, corrupting it by removing words, and then getting the models to reconstruct the full text. LeCun notes that they're pretty good at doing this with text, but images or video? Forget it.

"I take an image corrupted by removing some pieces, and then train some big neural net to recover the image. And that doesn't work, or it doesn't work very well," the Meta scientist said.

It's worth noting that AI models exist today that are pretty effective at generating images, but these are text-to-image models, like Midjourney and Stable Diffusion. OpenAI also has an AI model called DALL-E for image generation that's separate from GPT-4.

For LeCun, the path forward to an AI model that does it all may not lie in the stuff everyone's currently obsessing over.

"There is no real solution yet, but the things that are most promising at the moment, at least the things that work for image recognition – I'm going to surprise everybody – are not generative, okay," he said.

Koller also sees issues with today's LLMs. In her view, today's iterations of these models aren't, for instance, very good at understanding basic cognitive logic, like cause and effect.

"They are entirely predictive engines; they're just doing associations," she said.

This isn't the first time doubts have been raised about the capacity of today's AI models.

A pre-print paper submitted to ArXiv by a trio of Google researchers in November found that the transformer technology underneath LLMs was not very good at generalizing beyond its existing data set. If AGI is the big goal, that's not very promising.

This is not to say today's LLMs are useless. Taiwanese computer scientist and 01.AI founder Kai-Fu Lee, who was also on the panel, spoke of their "incredible commercial value." His company achieved a $1 billion valuation less than eight months after launch.

"They solve real problems, they can generate content, they dramatically improve our productivity, they're being deployed everywhere," he said.

Are they on the verge of making machines as smart as humans, though? Not in their current form at least.