This AI research proposes a framework for modeling user interactions with LLM using Norman’s seven action stages.

Smart writing aids have been extensively researched for many different writing goals and activities. The focus of recent advances in writing helpers has been large language models (LLMs), which allow people to produce material in response to a message by providing its purpose. Important developments in LLM such as ChatGPT and its use in common products highlight its potential as writing aids. However, human-computer interface with these wizards reveals significant usability issues, including consistency and fluency of model output, reliability, property of the material created, and predictability of model performance.

While some of the interactive components of writing assistants have been covered in previous posts, there still needs to be a focused attempt to satisfy end-to-end writing goals and approach their interactions from a usability perspective. These issues often cause users to need help using the tools successfully to achieve their writing goals, and sometimes cause users to give up altogether. Researchers from McGill University and the Université de Montréal examine the interface design of LLM-supported intelligent writing assistants, emphasizing human activities and drawing influence from previous research and design literature. They also suggest using Norman’s seven action phases as a design paradigm for LLM-compliant intelligent writing aids and discussing the usability implications.

A cyclical cognitive model known as Norman’s seven action phases is frequently used to understand users’ thought processes and associated physical activities. It is mainly used to inform the design of the system interface. The seven action steps are (a) develop objectives, (b) plan, (c) specify, (d) perform, (e) perceive, (f) interpret, and (g) compare, as shown in Figure 1. Plan The phases of , specify and execute constitute the execution phase of the interaction, and the phases of perceiving, interpreting and comparing constitute the evaluation phase. User interactions are based on a mental model of the system that they developed from past assumptions. They claim that this paradigm allows for the creation and testing of interfaces that facilitate detailed interactions with LLMs at various stages.

They suggest that efficient LLM-based writing assistance should answer the relevant questions for the various stages to inform the design and provide the user with essential skills. They provide an example that was heavily influenced by their early effort to use the OpenAI Codex to write software tutorials to further clarify their point. In a typical interaction, the user begins by deciding on a primary goal, such as creating a lesson on how to use matplotlib to plot data points. They then break the goal down into manageable components to help them determine how to approach the writing helper.

The main objective, for example, can be divided into three sub-objectives:

• Creating Tutorial Sections
• Provide proper instructions for installing libraries in various contexts.
• Produce and explain code snippets
• Increase the readability of the tutorial

Although more limited in scope and may come after several action framework cycles, each step in this situation can also be considered a secondary goal. When customers ask the writing wizard for help, they often describe and then complete their request through the interface, for example, “Write a code snippet to plot a scatterplot using matplotlib given the data points in a list of Python and provide an explanation of the code.”

The Take action stage may include various interface capabilities for changing and updating prompts, while the specific stage may have systems for recommending alternative prompts to the model. The execution stage is influenced by users’ previous conceptual models, their work and domain experience, and both. When the writing assistant produces output, the user reads, understands, and adjusts their pre-existing mental models following her knowledge and skill. For example, a user with substantial experience with matplotlib would be better able to detect any unexpected material or errors in the resulting code. Additionally, it might be necessary to run existing unit tests or run the produced code snippet in an IDE to compare the results with resources in other contexts.

They argue that applying Norman’s seven stages of action as a paradigm for researching user behavior with LLM-based writing aids can provide a useful foundation for conducting and designing detailed interactions throughout the goal formulation, execution, and evaluation phases. . It is possible to identify important interactions and direct the design of a writing assistant to help with the work of creating tutorials by asking pertinent questions at each step. It is possible to solve particular usability problems in the design of LLM-based writing tools by analyzing the devices and their characteristics through the interaction design dimensions described by the framework. More ambitiously, they target understudied areas of study in human-LLM interactions, such as alignment with user preferences, effective prompt design, and explainability and interpretation of model output.

review the Paper. Don’t forget to join our 20k+ ML SubReddit, discord channel, and electronic newsletter, where we share the latest AI research news, exciting AI projects, and more. If you have any questions about the article above or if we missed anything, feel free to email us at [email protected]

🚀 Check out 100 AI tools at AI Tools Club