QA RAG with Self-Assessment II
For this variation, we made a change to the evaluation procedure. In addition to the question-answer pair, we also pass the retrieved context to the LLM evaluator.
To achieve this, we add an additional itemgetter function in the second RunnableParallel to collect the context string and pass it to the new qa_eval_prompt_with_context request template.
rag_chain = (
RunnableParallel(context = retriever | format_docs, question = RunnablePassthrough() ) |
RunnableParallel(answer= qa_prompt | llm | retrieve_answer, question = itemgetter("question"), context = itemgetter("context") ) |
qa_eval_prompt_with_context |
llm_selfeval |
json_parser
)
Deployment flowchart:
One of the common pain points when using a chain implementation like LCEL is the difficulty in accessing intermediate variables, which is important for debugging pipelines. We looked at some options where we can still access any intermediate variables we are interested in using LCEL manipulations.
Using RunnableParallel to transfer intermediate outputs
As we saw earlier, RunnableParallel allows us to carry multiple arguments to the next step in the chain. We then use this RunnableParallel capability to move the required intermediate values to the end.
In the following example, we modify the original self-assessment RAG string to generate the recovered context text along with the final self-assessment result. The main change is that we add a RunnableParallel object to each step of the process to carry the context variable forward.
In addition, we also use the itemgetter function to clearly specify the inputs for the following steps. For example, for the last two RunnableParallel objects, we use element getter('input') to ensure that only the input argument from the previous step is passed to the LLM/Json parser objects.
rag_chain = (
RunnableParallel(context = retriever | format_docs, question = RunnablePassthrough() ) |
RunnableParallel(answer= qa_prompt | llm | retrieve_answer, question = itemgetter("question"), context = itemgetter("context") ) |
RunnableParallel(input = qa_eval_prompt, context = itemgetter("context")) |
RunnableParallel(input = itemgetter("input") | llm_selfeval , context = itemgetter("context") ) |
RunnableParallel(input = itemgetter("input") | json_parser, context = itemgetter("context") )
)
The output of this string looks like this:
A more concise variation:
rag_chain = (
RunnableParallel(context = retriever | format_docs, question = RunnablePassthrough() ) |
RunnableParallel(answer= qa_prompt | llm | retrieve_answer, question = itemgetter("question"), context = itemgetter("context") ) |
RunnableParallel(input = qa_eval_prompt | llm_selfeval | json_parser, context = itemgetter("context"))
)
Using global variables to save intermediate steps
This method essentially uses the principle of a logger. We introduce a new function that saves its input to a global variable, thus allowing us to access the intermediate variable through the global variable.
global contextdef save_context(x):
global context
context = x
return x
rag_chain = (
RunnableParallel(context = retriever | format_docs | save_context, question = RunnablePassthrough() ) |
RunnableParallel(answer= qa_prompt | llm | retrieve_answer, question = itemgetter("question") ) |
qa_eval_prompt |
llm_selfeval |
json_parser
)
Here we define a global variable called context and a function called save_context which saves its input value in the global context variable before returning the same input. In the chain we add the save_context It works as the last step of the context recovery step.
This option allows you to access any intermediate step without making major changes to the chain.
Using callbacks
Attaching callbacks to your chain is another common method used to record values of intermediate variables. There is a lot to cover on the topic of callbacks in LangChain, so I will cover this in detail in a different post.
