Large language models (LLMs) demonstrated impressive learning capabilities in a few opportunities, quickly adapting to new tasks with just a handful of examples.
However, despite their advances, LLMs still face limitations in complex reasoning involving chaotic contexts overloaded with unconnected facts. To address this challenge, researchers have explored techniques such as chain of thought that guide models to analyze information incrementally. However, on their own, these methods struggle to fully capture all critical details in vast contexts.
This paper proposes a technique that combines thread-of-thought (ToT) prompts with a retrieval augmented generation (RAG) framework that accesses multiple knowledge graphs in parallel. While ToT acts as the “backbone” of reasoning that structures thought, the RAG system expands the available knowledge to fill gaps. Parallel querying of multiple information sources improves efficiency and coverage compared to sequential retrieval. Taken together, this framework aims to improve the understanding and problem-solving skills of LLMs in chaotic contexts, getting closer to human cognition.
We begin by outlining the need for structured reasoning in chaotic environments where relevant and irrelevant facts are mixed. Below we present the design of the RAG system and how it expands the accessible knowledge of an LLM. We then explain the integration of ToT prompts to methodically guide the LLM through a step-by-step analysis. Finally, we discuss optimization strategies such as parallel retrieval to efficiently query multiple knowledge sources at the same time.
Through a conceptual explanation and Python code examples, this article illuminates a novel technique for orchestrating the strengths of an LLM with complementary external knowledge. Creative integrations like this highlight promising directions for overcoming inherent model limitations and improving ai reasoning capabilities. The proposed approach aims to provide a generalizable framework that can be improved as LLMs and knowledge bases evolve.
