A Wharton business school professor said Bing will get 'much higher grades' than ChatGPT

• UPenn professor Ethan Mollick expects Bing's AI chatbot to "receive much higher grades" than ChatGPT.
• Mollick compared the chatbots' essays and found that Bing generates "much higher quality" responses.

A Wharton business school professor expects Microsoft's new Bing AI search engine to "receive much higher grades" than OpenAI's ChatGPT.

Ethan Mollick, who teaches a class on entrepreneurship at the University of Pennsylvania, asked Bing to "write a 1000-word essay on how innovations are adopted in a specific industry or organization of your choice" as part of an experiment to compare Bing with ChatGPT.

More specifically, Mollick asked Bing to analyze the process of adopting new technologies — and the benefits and challenges that come along with it — through the lens of the diffusion of innovation theory. A prompt to suggest recommendations to "overcome the barriers to innovation" followed.

He ended his request asking Bing to include an introduction, body paragraphs, a conclusion, and references in the essay based on the specific bits of information he wanted for each section.

After feeding the same prompt to ChatGPT, he compared the essays generated by both chatbots. Bing wrote an essay on how innovation is disrupting the e-commerce industry, using Amazon as a case study. ChatGPT's essay looked at how innovation affects the automobile industry, focusing on electric vehicles.

The difference between the two was "quite large" Mollick concluded. ChatGPT wrote "an empty shell of an essay," he said, which seemed to have answered the prompt in a vague way. Bing, on the other hand, generated at least 3 pages of "much higher quality" responses that included real-world examples of innovation such as Amazon Prime and one-click ordering. Correct citations were also made.

More testing confirmed to Mollick that Bing may be more powerful than ChatGPT, claim that Microsoft has made in the past. In another test, Mollick asked Bing to create a syllabus for his MBA entrepreneurship class, resulting in a week-by-week breakdown of lesson plans and topics of discussion, according to screenshots he posted on Twitter. He found that the readings were, in-fact, real.

He followed up with a request to add readings authored by him and found that Bing "did an impressive job" at tweaking the syllabus to include his books and assigning the right chapters to their respective topics.

Still, Mollick recognized that the Bing contains "hiccups and glitches." When he pushed Bing to include his readings throughout the syllabus, it fabricated books that he didn't write.

He also noticed that Bing's reference links don't always include the statistics mentioned in its responses to queries like "give me 10 ideas for a new business for a doctor who doesn't want to practice medicine." It did, however, explain each statistic with the correct link and answer when asked.

"The educational hand-wringing about what tests it can pass and what assignments it can do is likely to start again, with more ominous answers," Mollick concluded.

Mollick didn't respond to Insider's immediate request for comment.

AI-generated errors haven't stopped Mollick from seeing chatbots as a useful learning tool. While many educators are worried tools like ChatGPT will encourage cheating and plagiarism, Mollick requires his students to use ChatGPT to help with their classwork and improve their writing. His new AI policy holds students accountable for any inaccuracies the bot spits out.

Despite a widespread school ban of ChatGPT, Mollick isn't the only educator who sees AI chatbots as a learning opportunity.

The New York City school system is training computer science teachers to take a "critical computing" approach in their lessons to evaluate emerging tech like ChatGPT and its societal consequences. Some Bronx high school teachers have even taught ChatGPT-generated lessons as part of a class experiment.

Mollick said he only recently started to toy with the new Bing. Still, he believes that "the improvements are already huge" in comparison to ChatGPT.

But he remains skeptical about AI and its disruption to the classroom.

"The solution is the same (we need to work with AI in classes, not against it), but the threat to traditional essay-writing and test-taking is even more acute," he said.