Secure AccountantAI Chatbot: Lili's Journey with Amazon Bedrock

This post was written in collaboration with Liran Zelkha and Eyal Solnik of Lili.

Small business owners tend to prioritize the operational aspects of their businesses over administrative tasks, such as financial record keeping and bookkeeping. While hiring a professional accountant can provide valuable guidance and expertise, it can be cost prohibitive for many small businesses. Additionally, the availability of accountants may not always be in line with business owners’ immediate needs, leaving them with unanswered questions or delayed decision-making processes.

In the rapidly evolving world of large language models (LLM) and generative artificial intelligence (ai), Lily Lili recognized an opportunity to use this technology to address the financial advisory needs of her small business clients. Using Anthropic’s Claude 3 Haiku on amazon Bedrock, Lili developed an intelligent AccountantAI chatbot capable of providing on-demand accounting advice tailored to each client’s unique financial history and business requirements. The AccountantAI chatbot functions as a virtual assistant, offering affordable and easily accessible financial guidance, allowing small business owners to focus on their core expertise while ensuring the financial health of their operations.

About Lili

Lili is a financial platform designed specifically for businesses, offering a combination of advanced business banking with integrated accounting and tax preparation software.

By consolidating financial tools into one easy-to-use interface, Lili streamlines and simplifies business finance management and makes it an attractive solution for business owners looking for a centralized and efficient way to manage their financial operations.

In this post, we’ll explore how Lili, a financial platform purpose-built for enterprises, used amazon Bedrock to build a secure and intelligent AccountantAI chatbot for small business owners. amazon Bedrock is a fully managed service that offers a selection of high-performance foundation models (FMs) from leading ai companies like Anthropic, Meta, Mistral ai, Stability ai, Cohere, AI21 Labs, and amazon through a single API, along with a broad set of capabilities you need to build generative ai applications with security, privacy, and responsible ai.

Solution Overview

The AccountantAI chatbot provides small business owners with accurate and relevant financial accounting advice in a secure manner. To achieve this, the solution is designed to address two key requirements:

• Validation of questions: Implement security measures to ensure that the user input is a valid and legitimate financial accounting question. This step helps filter out irrelevant or inappropriate queries, maintaining system integrity.
• Enriching the context:Expand the user's question with relevant contextual data, such as up-to-date accounting information and user-specific financial data. This step ensures that the chatbot's responses are tailored to the individual user's financial and business situation, providing more personalized and actionable advice.

To address the two key requirements of question validation and context enrichment, the AccountantAI solution employs a two-stage architecture comprising an ingestion workflow and a retrieval workflow.

Ingestion Workflow

The ingestion workflow is an offline process that prepares the system to handle customer queries. For this stage, Lili prepared a comprehensive workflow golden collection of financial accounting questions, drawn from common queries and real-world questions from their customer base over the years. This diverse and high-quality collection serves as a reference corpus, ensuring that the chatbot can handle a wide range of relevant queries. The ingestion workflow transforms these curated questions into vector embeddings using the amazon Titan Text Embeddings model API. This process is performed over AWS PrivateLink for amazon Bedrock, a protected and private connection in your VPC. The vector embeddings are persisted in the application’s in-memory vector store. These vectors will help validate user input during the retrieval workflow.

Each selected vector embedding is combined with a corresponding message template that was evaluated during testing as the most effective.

Example of a message template

Provides context about the agent's role as Lili's ai assistant for financial questions and outlines the general guidelines applied to all queries.



Provides details on Lili platform.



Lists out all of Lili's product features in detail. This section aims to explain Lili's features in detail, ensuring that answers are aligned with the Lili platform. For instance, when addressing questions about tax reduction management, highlight the relevant features that Lili offers, which customers should be familiar with.



Outlines the required formatting for the response to ensure it meets the expected structure.



Data relevant to answering the customer's question.



Specific accounting knowledge that is relevant to the question and the model is not familiar with, such as updated data for 2024.



Contains the user's actual question.



Provides the core instructions on how to approach answering the question appropriately and meet expectations. It also defines the steps in providing a detailed and high-quality answer.



Important guidelines to remind the agent and make sure it follows them, such as the exact format of the answer.

Recovery Workflow

Lili's chatbot web interface allows users to submit queries and receive real-time responses. When a customer asks a question, it is sent to the backend system for processing.

1. The system first converts the query into a vector embedding using the amazon Titan Text Embeddings model API, accessed securely through PrivateLink.
2. The system then performs a similarity search on the precomputed embeddings in the golden collection, to find the most relevant matches for the user's query. The system evaluates the similarity scores of the search results against a predetermined threshold. If the user's query returns matches with low similarity scores, it is considered ill-formed or unclear, and the user is prompted to rephrase or refine their query.
3. However, if the user's query produces matches with high similarity scores, it is considered a legitimate query. In this case, Lili's backend system continues processing using the reference question that has the highest similarity score to the user's query.
4. Based on the golden question with the highest similarity score, the system retrieves the corresponding message template.

This template is supplemented with up-to-date accounting information and customer-specific financial data from external sources such as amazon RDS for MySQL. The resulting contextualized message is sent to Anthropic’s Claude 3 Haiku on amazon Bedrock, which generates a customized response that addresses the customer’s query within their unique business context.

As model providers continually improve their offerings with innovative updates, amazon Bedrock simplifies the ability to adopt emerging advancements in generative ai across multiple model providers. This approach has proven its benefits since AccountantAI’s initial launch. Lili moved from Anthropic’s Claude Instant to Claude 3 within two weeks of its official release in the amazon Bedrock environment and three weeks after its general availability.

Lili selected Anthropic’s Claude family of models for AccountantAI after reviewing industry benchmarks and conducting her own quality assessment. Anthropic’s Claude on amazon Bedrock consistently outperformed other models in understanding financial concepts, generating consistent natural language, and providing accurate, personalized recommendations.

Following the initial release of AcountantAI, amazon Bedrock introduced Anthropic’s Claude 3 Haiku model, which Lili evaluated against Anthropic’s Claude Instant version. Anthropic’s Claude 3 Haiku model demonstrated significant improvements across three key evaluation metrics:

• Quality – Anthropic Claude 3 Haiku produced higher quality results, providing more detailed and better worded responses compared to its predecessor.
• Response time – Anthropic Claude 3 Haiku exhibited a 10 to 20 percent improvement in response times over Claude Instant, offering faster performance.
• Cost – Anthropic Claude 3 Haiku on amazon Bedrock is the most cost-effective option. For example, it costs up to 68 percent less per 1000 input/output tokens compared to Anthropic Claude Instant, while offering higher levels of intelligence and performance. See amazon-bedrock-anthropic-ai-claude-3″ target=”_blank” rel=”noopener”>Anthropic's Claude 3 models on amazon Bedrock for more information.

For customers like Lili, this underscores the importance of having access to a fully managed service like amazon Bedrock, which offers a variety of high-performance foundation models to meet diverse enterprise ai needs. There is no one-size-fits-all model, and the ability to select from a variety of cutting-edge FMs is crucial for organizations looking to utilize the latest advances in generative ai effectively and cost-effectively.

Conclusion

AccountantAI, available exclusively to Lili customers, reduces the need to hire a professional accountant. While professional accountants can provide valuable guidance and expertise, their services can be prohibitively expensive for many small businesses. AccountantAI has already answered thousands of questions, delivering real value to businesses and providing quality answers to financial, tax and accounting queries.

Lili uses amazon Bedrock to easily, securely, and reliably access high-performance foundational models from leading ai companies to integrate accounting knowledge at scale with unique client data. This innovative solution delivers economic insights to optimize cash flow, streamline tax planning, and enable informed decision making to drive growth. AccountantAI closes the accounting resource gap and democratizes access to high-quality financial intelligence for all businesses.

Explore Lili’s AccountantAI powered by amazon Bedrock to gain affordable and accessible financial intelligence for your business today, or use amazon Bedrock Playgrounds to experiment with running inferences on different models on your data.

About the authors

Doron Bleiberg is a Senior Solutions Architect for AWS Startups helping fintech customers on their journey to the cloud.

Liran Zelkha is co-founder and CTO of Lili, leading our development and data efforts.

Eyal Solnik is the Head of Data at Lili and leads our AccountantAI product.