Microsoft Researchers Introduce InsightPilot: An LLM-Based Automated Data Exploration System

Data exploration is an important step in data analysis that extracts key information through multiple steps such as filtering, sorting, grouping, etc. It helps discover patterns in the data set and reveal potential relationships between variables. However, this process is generally interactive and requires the user to manually explore the data, making the process time-consuming and requiring domain expertise.

Although there are different tools for general data exploration, they often do not consider the user's intent or the characteristics of the data set, resulting in irrelevant insights. Furthermore, LLM hallucinations are a notorious problem that causes LLMs to generate unreliable content. To address the shortcomings of existing models, Microsoft researchers have published vision pilot, a system that automates the data exploration process using LLM. The system provides LLMs with accurate information to avoid hallucinations and presents a compact abstraction of the data set to reduce computational costs, allowing the LLM to better answer users' questions.

InsightsPilot consists of the following three components:

1. A user interface that allows users to ask questions in natural language and also display analysis results.
2. An LLM that makes it easy to explore data by selecting the appropriate analysis based on context.
3. An insights engine that does the analysis and presents the results in natural language.

Initially, a user poses a query in the interface and the information engine generates preliminary information. Depending on the context, the LLM identifies the most relevant insights and continues querying the engine for more details about them. For example, a user can ask about trends in students' science scores and then, based on initial insights, the LLM can query the engine to perform additional analysis, such as comparing scores or finding outliers. As long as the exploration is not complete, the interaction between the LLM and the engine continues, and at the end of the data exploration step, the engine presents the most important insights in the form of a coherent report, which is then displayed to the user via of the interface.

To evaluate its performance, researchers conducted user studies to simulate real-world use cases for InsightPilot. Four data science participants were asked to pose three questions and the system was evaluated on metrics such as relevance, completeness, and understandability. The results show that InsightPilot consistently outperformed both OpenAI Code Interpreter and Langchain Pandas Agent.

A case study based on an automobile sales data set was also conducted to evaluate the performance of InsightPilot. By asking about the overall trend of Toyota's car sales, the system not only identified 'Camry' as the key driver of Toyota's sales, but also compared Toyota's sales with Honda's and also provided other data interesting.

Although InsightPilot performs better than other state-of-the-art systems, it often produces vague responses that require manual evaluation. Therefore, it is essential to test its effectiveness on different real-life data sets. However, it is an effective method for obtaining information from a data set using natural language queries and has the potential to streamline the exploratory data analysis process and save time and effort. More research is needed to ensure that the method can be implemented in real-world scenarios and reinforce efficiency and data-driven decision making.

Review the Paper. All credit for this research goes to the researchers of this project. Also, don't forget to join. our 34k+ ML SubReddit, 41k+ Facebook community, Discord channel, and Electronic newsletterwhere we share the latest news on ai research, interesting ai projects and more.

If you like our work, you'll love our newsletter.

<!– ai CONTENT END 2 –>