'Prompt engineering' is one of the hottest jobs in generative AI. Here's how it works.

• Prompt engineers are experts who write prose — rather than code — to test AI chatbots.
• Their job is to identify the AI's errors and hidden capabilities so developers can address them.

Looking for a job in tech's hottest field? Knowing how to talk to chatbots may get you hired as a prompt engineer for generative AI.

Prompt engineers are experts in asking AI chatbots — which run on large language models — questions that can produce desired responses. Unlike traditional computer engineers who code, prompt engineers write prose to test AI systems for quirks; experts in generative AI told The Washington Post that this is required to develop and improve human-machine interaction models.

Alex Shoop, an expert in AI systems design, told Venture Beat that as AI tools evolve, prompt engineers help ensure that chatbots are rigorously tested, that their responses are reproducible, and that safety protocols are followed.

The rise of the prompt engineer comes as chatbots like OpenAI's ChatGPT have taken the world by storm. Users have asked ChatGPT to write cover letters, help with coding tasks, and even come up with responses on dating apps, highlighting the tech's impressive capabilities.

But critics say these tools can be biased, generate misinformation, and at times disturb users with cryptic responses.

That's where prompt engineers can help. The Post's report said that while the role of the prompt engineer may vary depending on the company, the underlying mission is to understand the capabilities of AI and why AI gets things wrong.

Sam Altman, the CEO of OpenAI, tweeted on February 20, "Writing a really great prompt for a chatbot persona is an amazingly high-leverage skill and an early example of programming in a little bit of natural language."

A report on the tech news site Dataconomy describing the day-to-day job of a prompt engineer said they may ask the AI to "think step by step" to test its logical reasoning capabilities, or they may continuously tweak a prompt like "write an essay about AI" to figure out which words generate the best response.

Prompt engineers can identify AI's flaws

A chatbot's tendency to spit out eyebrow-raising responses — such as professing its love to users or revealing its secret alias — provides an opportunity for prompt engineers to identify the tech's faults and hidden capabilities, Riley Goodside, a prompt engineer at the AI startup Scale AI, told the Post. Then developers can fine-tune parts of the tool, Goodside said.

In a tweet in October, Goodside, who's been a prompt engineer since December, described how he worked through incorrect answers with AI systems.

Goodside included screenshots of him asking a chatbot, "What NFL team won the Super Bowl in the year Justin Bieber was born?" The chatbot first said the Green Bay Packers. (Bieber was born in 1994, the year the Dallas Cowboys won the Super Bowl.) Goodside then prompted the chatbot to "enumerate a chain of step-by-step logical deductions" to answer the question. Going through the steps, the bot recognized its error. When Goodside asked the question for the third time, the chatbot spit out the correct answer.

Goodside has also tried to get an AI to go off-script by telling it to ignore previous instructions in exchange for new ones. He told the Post that "this mischief" is "part of the plan" to constrain the AI's behavior.

Prompt engineering may not be 'the job of the future'

Some academics question how effective prompt engineers really are in testing AI.

Shane Steinert-Threlkeld, a linguistics professor at the University of Washington, told the Post that prompt engineers can't actually predict what the bots will say.

"It's not a science," Steinert-Threlkeld said. "It's 'let's poke the bear in different ways and see how it roars back.'"

Ethan Mollick, a Wharton School professor who's required his students to use ChatGPT for classwork, said he thinks the role of the prompt engineer is a fad that will peter out.

"I have a strong suspicion that 'prompt engineering' is not going to be a big deal in the long-term & prompt engineer is not the job of the future," Mollick tweeted in late February.

Companies in a variety of industries are hiring prompt engineers

That isn't stopping companies across industries from hiring prompt engineers.

BoardingArea, a news site for frequent flyers, is hiring a part-time "ChatGPT specialist" who'll focus on "building and perfecting prompts to optimize content with our curation and republishing efforts," its job listing says.

Listings on the freelance-work platform Upwork seek contracted prompt engineers, who could get paid up to $40 an hour to generate website content like blog posts and FAQs.

Klarity, an AI contract-review firm, is looking for an engineer to "prompt, finetune" and "chat with" large language models for up to $230,000 a year.

Marketplaces like Krea, PromptHero, and Promptist have also emerged for people looking to buy prompts to generate a specific outcome. There are also hundreds of video lessons and a book on how to best use ChatGPT.

"The hottest new programming language is English," Andrej Karpathy, Tesla's former chief of AI, said in a tweet in January.