This artificial intelligence paper from Arizona State University examines whether large language models (LLMs) can reason and plan.

Large Language Models (LLM) are the latest and most amazing introduction in the field of artificial intelligence (ai). Large volumes of textual data from the Internet have been used to train these supercharged n-gram models, which have captured a large amount of human knowledge. Many have been surprised by its language generation and text completion capabilities, which display linguistic behaviors in text completion systems.

It is useful to consider LLMs as non-veridical mass memories, similar to an external cognitive system for the human race to understand. Word-by-word reconstruction of text prompt completions has been performed using LLM, which operate more probabilistically than typical databases that accurately index and retrieve data. Because of this technique, known as approximation retrieval, LLMs are excellent at creating unique completions based on the information they receive rather than ensuring full answers are memorized.

There have been concerns about whether LLMs can go beyond language production to tasks involving thinking and planning, which are generally linked to higher-order cognitive processes. Unlike people or conventional ai systems, LLMs are not predisposed to principled reasoning, which often includes intricate computational inferences and searches of any form during training or operation.

A team of researchers has recently studied whether LLMs can reason and plan. It is reasonable to ask whether LLMs are really capable of reasoning from first principles or simply copying reasoning by remembering patterns. Making this distinction is essential since pattern recognition is not the same as logical problem solving. It becomes more difficult to distinguish between true problem solving and memorization as LLMs are trained on large banks of questions.

The results of attempts to assess the thinking skills of LLMs have been inconsistent. First, evidence on planning problems, such as those generated by the International Planning Competition, refuted anecdotal claims about the planning capabilities of LLMs. Subsequent studies with more recent versions of LLM, such as GPT-3.5 and GPT-4, indicated some progress in plan generation, although accuracy varied by domain.

The team has shared that honing LLMs in planning problems, by helping them make better guesses, is one way to improve their planning performance, but even so, this approach essentially turns planning problems into exercises in memory-based retrieval. instead of actual planning.

Another approach is to provide LLMs with cues or recommendations so that they can iteratively improve their early predictions about the plans. Although this method could increase performance, it raises concerns about the certification of final answers, the difference between manual and automatic prompts, and whether the prompts actually increase LLM problem knowledge or simply motivate them to try again.

The best course of action is to use an external model-based plan checker to activate the LLM and validate the accuracy of the solutions, which will provide a robust test and critique generation system. On the other hand, repeated human insistence runs the risk of the Clever Hans effect, in which human input influences LLM estimates. It is questionable whether LLMs can improve through iterative self-criticism because there is no evidence to support the idea that LLMs are more adept at validating solutions than creating them.

In short, although LLMs are remarkably good at language production, there is little evidence to support the claim that they are capable of true reasoning or planning. Their ability to generate ideas and possible solutions is one of their strongest points, and can be useful in organized frameworks that have external verification procedures.

Review the Paper. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on twitter.com/Marktechpost”>twitter. Join our Telegram channel, Discord channeland LinkedIn Grabove.

If you like our work, you will love our Newsletter..

Don't forget to join our 39k+ ML SubReddit