Using a multimodal document machine learning model to query your documents | by Eivind Kjosbakken | April 2024

Harness the power of the mPLUG-Owl document understanding model to ask questions about your documents

ai aj ak al am an ao ap aq ar jl” data-testid=”authorName” href=”https://oieivind.medium.com/?source=post_page—–9bb3a2199380——————————–” rel=”noopener follow”>Eivind Kjosbakken

·

Continue

Published in

Towards data science

·

9 minutes of reading

·

6 hours ago

—

<button aria-controls="postFooterSocialMenu" aria-expanded="false" aria-label="Share Post" data-testid="headerSocialShareButton" class="af fj ah ai aj ak al mx an ao ap ew my mz mn na nb nc nd ne s nf ng nh ni nj nk nl u nm nn no”>

Share

This article will discuss the x-PLUG/mPLUG-DocOwl?tab=readme-ov-file” rel=”noopener ugc nofollow” target=”_blank”>Alibaba Document Understanding Model, recently published with model weights and data sets. It is a powerful model capable of performing various tasks such as answering questions about documents, extracting information, and embedding documents, making it a useful tool when working with documents. This article will deploy the model locally and test it on different tasks to give an opinion on its performance and usefulness.

· Motivation
· Tasks
· Running the model locally
· Model test
∘ Data
∘ Testing the first leftmost receipt:
∘ Testing the second rightmost receipt:
∘ Testing the first leftmost class note:
∘ Testing the second rightmost class note
· My thoughts on the model.
· Conclusion