Meet ChatDB: A framework that augments LLMs with token memory in the form of databases

Large language models such as GPT-4 and PaLM 2 have become a crucial part of contemporary AI systems, revolutionizing their understanding of natural language processing and changing several industries. Despite great strides in understanding and producing contextually appropriate responses, LLMs still have certain drawbacks. The fact that multi-turn interactions with language models generate many tokens that easily exceed the input token limit of LLMs is one of the key issues. GPT-4, for example, is limited to 32,000 tokens. LLMs must maintain contextual information throughout the encounter and produce responses based on the information collected.

However, simply concatenating all the contextual information and stuffing it into the LLMs can easily overwhelm the LLMs’ processing capabilities and accumulate errors, causing the model to lose the thread of the conversation and produce less accurate responses. Some neural memory mechanisms have been explored to overcome the limited token input problem of LLM. The memory components serve as a storage and retrieval system for relevant information from previous interactions. However, augmenting LLMs with conventional neural memory generally leads to difficulties in storing, retrieving, and manipulating historical information in memory, especially for tasks that require complex, multi-hop reasoning.

Two main reasons are that they don’t retain historical data in a structured way and they don’t manipulate it symbolically, since they all rely on vector similarity calculations, which can be erroneous and cause error accumulation. Researchers from Tsinghua University, Beijing Academy of Artificial Intelligence, and Zhejiang University advocate the use of databases as innovative symbolic memory for LLMs to solve the above problems. ChatDB is the name of the entire framework. Figure 1 below shows the two parts that make up ChatDB: an LLM driver and its memory. Memory read and write operations are controlled by the LLM driver, which can be any widely used LLM.

LLM memory, which can be symbolic, non-symbolic or a hybrid of the two, is charged with keeping track of the past and disseminating the data needed to help the LLM react to human input. ChatDB emphasizes the use of databases as symbolic memory, which allows for the organized storage of historical data through the execution of figurative language, namely SQL commands. The LLM created these SQL statements. A database can be used as symbolic memory in situations that require exact logging, updating, querying, deleting, and analysis of historical data. For example, a store manager must keep track of daily sales figures. Therefore, using arrays or plain text as memory is inappropriate.

However, using a database as external symbolic memory is quite appropriate. The database uses SQL commands to perform precise actions, such as inserting, deleting, updating, and selecting data. As a result, they were using databases as external symbolic memory that ensures correctness and efficiency in managing and manipulating historical data, greatly improving the performance of LLMs in situations that require very precise and lengthy data capture and processing. Within the ChatDB framework, they suggest the memory string strategy to more skillfully use external symbolic memory, further increasing the reasoning ability of LLMs.

User input is converted to a sequence of buffer operation stages via the memory string technique, which produces the desired outputs. A complex problem is divided into several memory operation stages using the memory chain technique, which considerably reduces the difficulty of solving the problem. Each intermediate step in ChatDB involves one or more SQL statements. The LLM field benefits greatly from their ChatDB. First, they suggest adding databases to LLMs as their external symbolic memory. This would allow for organized archiving of historical data and allow complicated and symbolic data manipulations using SQL statements.

Second, they can effectively manipulate memory by transforming user input into multi-step buffer operations using their memory string technique. This improves the efficiency of ChatDB and allows you to manage complicated multi-table database transactions with more precision and stability. Finally, their research shows that adding symbolic memory to LLMs improves multi-hop reasoning skills and reduces error backlog, allowing ChatDB to perform better on a synthetic dataset than ChatGPT.

review the Paper and Project. Don’t forget to join our 23k+ ML SubReddit, discord channel, and electronic newsletter, where we share the latest AI research news, exciting AI projects, and more. If you have any questions about the article above or if we missed anything, feel free to email us at [email protected]

🚀 Check out 100 AI tools at AI Tools Club