Prompt training technique helps users speak AI’s language

Today’s generative artificial intelligence models can create everything from images to computer applications, but the quality of their output depends largely on the prompt a human user provides.

Carnegie Mellon University researchers have proposed a new approach for teaching everyday users how to create these prompts and improving their interactions with generative artificial intelligence models.

The method, called Requirement-Oriented Prompt Engineering (ROPE), shifts the focus of prompt writing from clever tricks and templates to clearly stating what the AI should do. As large language models (LLMs) improve, the importance of coding skills may wane while expertise in prompt engineering could rise.

“You need to be able to tell the model exactly what you want. You can’t expect it to guess all your customized needs,” said Christina Ma, a Ph.D. student in the Human-Computer Interaction Institute (HCII). “We need to train humans in prompt engineering skills. Most people still struggle to tell the AI exactly what they want. ROPE helps them do that.”

Prompt engineering refers to the precise instructions — the prompts —a user gives a generative AI model to produce a desired output. The better a user is at prompt engineering, the more likely an AI model will produce what the user intended.

In “What Should We Engineer in Prompts? Training Humans in Requirement-Driven LLM Use,” appearing in the Association for Computing Machinery’s Transactions on Computer-Human Interaction, researchers describe their ROPE paradigm and a training module they created to teach and assess the method.

ROPE is a human-LLM partnering strategy where humans maintain agency and control of the goals by specifying requirements for LLM prompts. The paradigm focuses on the importance of crafting accurate and complete requirements to achieve better results, especially for complex, customized tasks.

To test ROPE, the researchers asked 30 people to write prompts for an AI model to complete two separate tasks as a pretest: create a tic-tac-toe game and design a tool to help people develop content outlines. Half of the participants then received training through ROPE and the rest watched a YouTube tutorial on prompt engineering. The groups then wrote prompts for a different game and a different chatbot as a posttest.

When researchers compared the results of the exercises, they found that participants who received the ROPE training outperformed the people who watched the YouTube tutorial. Scores from pretest to posttest rose by 20% for people who received the ROPE training and only 1% for those who did not.

“We not only proposed a new framework for teaching prompt engineering but also created a training tool to assess how well participants do and how well the paradigm works,” said Ken Koedinger, a University Professor in the HCII. “It’s not just our opinion that ROPE works. The training module backs it up.”

Generative AI models have already altered the content of introductory programming and software engineering courses as traditional programming evolves into natural language programming. Instead of writing software, an engineer can write a prompt directing AI to develop the software.

This paradigm shift could create new opportunities for students, allowing them to work on more complex development tasks earlier in their studies and advancing the field.

The researchers did not design ROPE solely for software engineers. As humans continue to integrate AI into daily life, clearly communicating with machines will become an important aspect of digital literacy. Armed with knowing how to write successful prompts and an AI model up to the task, people without coding or software engineering backgrounds can create applications that will benefit them.

“We want to empower more end users from the general public to use LLMs to build chatbots and apps,” Ma said. “If you have an idea, and you understand how to communicate the requirements, you can write a prompt that will create that idea.”

The researchers have open-sourced their training tools and materials, aiming to make prompt engineering more accessible to nonexperts.