Guided reasoning: a new approach to improve the intelligence of multi-agent systems

Gregor Betz from Logikon ai, KIT introduces guided reasoning. A system with more than one agent is a guided reasoning system if one agent, called a guide, primarily works with the other agents to improve its reasoning. A multi-agent system with one guide agent and at least one client agent is called a guided reasoning system if the guide works with the clients in a planned and principal manner to get them to reason in a way that follows a certain method M. One way to describe the reasoning method M is with standards and criteria, clear examples, or detailed rules and instructions. Guided reasoning methods include a coach helping a business unit do a SWOT analysis, a child helping his grandmother solve a crossword puzzle problem, and a Socratic dialogue.

At first glance, the argument for ai-guided reasoning rests on these assumptions:

1. ai should give the correct answers and explain them.
2. ai systems can only honestly explain their answers if they are based on clear thinking.
3. Incorrect reasoning makes it difficult for ai systems to give the correct answers.
4. Great experts in a field do not always know how to use advanced thinking techniques.

The cognitive specialization principle says that to create explainable and accurate ai systems, more ai experts for reasoning methods (meta-reasoning specialists) should be brought in who can work with experts in other domains. Guided reasoning is a good design technique for advanced GenAI applications because it facilitates the division of cognitive labor.

Logikon's standard way of using guided reasoning is that when customer agents are faced with a decision problem, they are told to carefully examine and evaluate the pros and cons.

• Step 1: The guided reasoning method is initiated when the user query is submitted. This can be done immediately when the client model calls a tool usage method or if the user specifically requests it.
• Step 2: The client presents the problem statement to the guide. The guide's key role is to meticulously organize the thinking steps that will be used to find the answer, providing a clear structure to the process. Step 3: The guide may ask the client questions.
• Step 4: The guide obtains the client's responses.
• Step 5: Responses are further processed and reviewed.

The guide sets the rules for the thinking process and manages the workflow, either statically or dynamically. The guide rewrites the problem differently after obtaining the problem statement (in step 2). Steps 3 and 4 allow the client to respond to the different problem statements without depending on each other. This is called a “chain of thought.” The guide compares the possible answers to determine whether the client understands the problem and what he or she should say in response. The client is given a well-written explanation and a summary of the thinking process (protocol). If the ai ​​has not developed consistent lines of reasoning and answers to similar problem statements, the client can respond to the user's first question.

After receiving the problem statement, the coach instructs the client to think of different ways to solve the problem and lists the pros and cons of each possible solution. The coach uses the thought trail produced in this way as a starting point for further analysis. In particular, through a series of steps outlined below, he or she creates an informal argument map that clarifies the different arguments presented during the brainstorming and shows how they connect to the competing response options, directly or indirectly.

• A single statement shows each case for the informal argument map.
• The guide then uses the argument map to have the client evaluate the arguments in a planned way.
• The client is tasked with evaluating the persuasiveness of claim C by examining all the pros and cons that have been deemed reasonable.
• This backward, argument-by-argument review begins with the leaf nodes of the argument map and ends with a check of how plausible the main claims are.

The figure above shows the steps that users take to put together a controversial argument as a fuzzy argument map. This is how Logikon typically performs forward reasoning by weighing pros and cons. Each step in Logikon's Python program corresponds to a different analyst class. Analyst classes primarily use internal LLM processes to create the necessary logical artifacts.

• IssueBuilder takes the reasoning trail of rough thinking and, with the help of expert LLMs, describes the main problem that the text is about, which is usually a new way of stating the original problem.
• The ProsConsBuilder uses the reasoning trails to build a multi-rooted pros and cons list that addresses the main problem already identified. This method consists of several steps: First, all reason statements relevant to the problem are extracted from the reasoning trail, regardless of their valence. In the second step, these reasons are combined into one or more pros and cons lists. This is the only step where the central root statements are found and aggregated. The final pros and cons lists are checked for duplicates and thoroughness (based on the reasons given at the beginning) and modified if necessary.
• RelevanceNetworkBuilder uses a set of cue templates to determine the likelihood that two reason statements are relevant to each other, and to any pair of reason statements and main claims. This creates a complete graph of all reason statements and main claims, with weighted support and attack ratios. (Two main claims are believed to maximally contradict each other.)
• FuzzyArgmapBuilder takes the entire graph and uses an optimal branching method to build a tree that connects all argument nodes to the strongest edges. It then adds more edges with weights above a given level. This process results in a fuzzy argument map, which is then exported in several useful formats. The purpose of FuzzyArgmapBuilder is to provide a complete and visually intuitive representation of the argumentation process, making it easier to understand and analyze.

Take a look at the Paper and ai/logikon” target=”_blank” rel=”noreferrer noopener”>GitHub. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on twitter.com/Marktechpost”>twitter and LinkedInJoin our Telegram Channel. If you like our work, you will love our fact sheet..

Don't forget to join our SubReddit of over 50,000 ml

Dhanshree Shenwai is a Computer Science Engineer with extensive experience in FinTech companies spanning the Finance, Cards & Payments and Banking space and is keenly interested in the applications of artificial intelligence. She is excited to explore new technologies and advancements in today’s ever-changing world, making life easier for everyone.