Jina AI Introduces Jina-CLIP v2: A 0.9B Multilingual Multimodal Embedding Model Connecting Images with Text in 89 Languages

In an interconnected world, effective communication in multiple languages ​​and media is increasingly important. Multimodal ai faces challenges in combining images and text for seamless retrieval and understanding in different languages. Existing models often perform well in English, but struggle with other languages. Additionally, simultaneously handling high-dimensional data for text and images has been computationally intensive, limiting applications for non-English speakers and scenarios requiring multilingual contexts.

Jina-CLIP v2: A multilingual multimodal embedding model 0.9B

Jina ai has introduced Jina-CLIP v2, a 0.9 billion multilingual multimodal embedding model that connects images with text in 89 languages. Jina-CLIP v2 supports a wide range of languages, addressing limitations that previously restricted access to advanced multimodal ai technologies. It handles images with a resolution of 512 × 512 and processes text with up to 8000 tokens, providing an efficient solution for linking images and multilingual text. Additionally, it offers Matryoshka representations that reduce embeddings to 64 dimensions for both text and images, ensuring more efficient embeddings while preserving essential contextual information.

Technical details

Jina-CLIP v2 stands out for its flexibility and efficiency. It allows the generation of embeddings not only at large dimensional scales but also at smaller scales, with its Matryoshka rendering function that reduces embeddings to 64 dimensions. This allows users to tune the integration process to meet specific requirements, whether for computationally intensive deep learning tasks or lightweight mobile applications. Additionally, the model's text encoder can function independently as a dense retrieval, matching the performance of jina-embeddings-v3, the current leader in multilingual embeddings under 1 billion parameters in the Multilingual Text Embeddings Benchmark (MTEB). The versatility to perform retrieval and classification tasks makes Jina-CLIP v2 suitable for a variety of use cases, from multilingual search engines to contextual recommender systems.

Jina-CLIP v2 represents an important step toward reducing bias in language models, particularly for users who rely on less-spoken languages. In evaluations, the model performed well on multilingual retrieval tasks, demonstrating its ability to match or exceed the performance of specialized text models. Its use of Matryoshka representations ensures that embedding calculations can be performed efficiently without sacrificing accuracy, allowing deployment in resource-constrained environments. Jina-CLIP v2's ability to connect text and images in 89 languages ​​opens new possibilities for businesses and developers to create ai that is accessible to diverse users while maintaining contextual accuracy. This can significantly impact e-commerce applications, content recommendation, and visual search systems, where language barriers have traditionally posed challenges.

Conclusion

Jina-CLIP v2 is a significant advancement in multilingual multimodal models, addressing both linguistic diversity and technical efficiency in a unified approach. By enabling effective connectivity of images and text in 89 languages, Jina ai is contributing to more inclusive ai tools that transcend linguistic boundaries. Whether for retrieval or classification tasks, Jina-CLIP v2 offers flexibility, scalability and performance that allows developers to create robust and efficient ai applications. This development is a step forward in making ai accessible and effective for people around the world, fostering cross-cultural interaction and understanding.

Verify <a target="_blank" href="https://jina.ai/news/jina-clip-v2-multilingual-multimodal-embeddings-for-text-and-images/” target=”_blank” rel=”noreferrer noopener”>the details here. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on <a target="_blank" href="https://twitter.com/Marktechpost”>twitter and join our Telegram channel and LinkedIn Grabove. If you like our work, you will love our information sheet.. Don't forget to join our SubReddit over 55,000ml.

(FREE VIRTUAL CONFERENCE ON ai) SmallCon: Free Virtual GenAI Conference with Meta, Mistral, Salesforce, Harvey ai and More. Join us on December 11 for this free virtual event to learn what it takes to build big with small models from ai pioneers like Meta, Mistral ai, Salesforce, Harvey ai, Upstage, Nubank, Nvidia, Hugging Face and more.