SaySelf: A machine learning training framework that teaches LLMs to express more accurate and detailed confidence estimates

Language learning models (LLMs), who are very good at reasoning and giving good answers, are sometimes honest about their mistakes and tend to freak out when asked questions they haven't seen before. When responses are more than a single token, it becomes much more important to determine how to obtain reliable confidence estimates from LLMs.

Training-based and encouragement-based approaches have been used in the past to build confidence in LLMs. Prompt-based approaches, for example, use specific prompts to create confidence ratings or consistency of responses as an indication of confidence. To train LLMs to be confident, training-based methods create custom data sets for adjustments. However, these techniques often produce simplistic or less than ideal confidence estimates, which do not accurately represent the degrees of certainty of the models.

A new study by Purdue University, the University of Illinois Urbana-Champaign, the University of Southern California, and the Hong Kong University of Science and technology presents SaySelf, a training framework for LLMs that helps them produce confidence estimates with greater precision and accuracy. Significantly, unlike previous work, SaySelf allows LLMs to provide self-reflective rationales that show where they lack knowledge and explain their confidence estimates. To achieve this, researchers use a pre-built LLM (such as GPT4) to automatically generate a model-tailored dataset, which can then be used for supervised fitting. They randomly sample several chains of reasoning, which are sequences of tokens that represent the LLM's thought process, from the LLMs for each query. After that, the reasoning chains are grouped into groups based on their semantic similarity, and an example of each group is saved.

From a first-person point of view, GPT-4 is asked to examine cases chosen from different groups and summarize uncertainty about specific knowledge in plain language. Researchers calibrate LLMs' confidence estimate on each response using reinforcement learning to ensure accurate confidence estimates. They devise a payment system that discourages LLMs from making overconfident predictions and punishes them when they are wrong. Various question-answering tasks that require extensive knowledge, such as complex medical diagnoses or legal case analysis, are used to evaluate SaySelf in the experiments in this study. The study demonstrates that SaySelf maintains task performance while dramatically reducing confidence calibration errors. It is possible to further improve calibration performance with the developed self-reflective foundations, which also successfully capture internal uncertainty.

The following examples are incomplete about how this work could impact relevant academic research and practical applications: (1) From an LLM alignment point of view, ai can benefit from a transparent trust statement that includes explanations. (2) LLMs can improve their interaction and performance by following self-reflective foundations to execute additional activities, such as requesting external tools or making clarification queries.

Once the SaySelf training process is complete, the team expects to see encouraging developments in training procedures, such as proactive learning algorithms that improve the learning outcomes of LLMs through their interactions with people.

Review the Paper. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on twitter.com/Marktechpost”>twitter. Join our Telegram channel, Discord channeland LinkedIn Grabove.

If you like our work, you will love our Newsletter..

Don't forget to join our 43k+ ML SubReddit | Also, check out our ai Event Platform