Med-Gemini achieves 91.1% accuracy in medical diagnosis

Researchers from Google and DeepMind have introduced Med-Gemini, a new generation of ai models designed specifically for medical applications. Leveraging the strengths of Gemini 2023 models, renowned for language processing, multimodal understanding, and long-context reasoning, Med-Gemini significantly enhances these capabilities for healthcare applications.

Med-Gemini's superiority is demonstrated through evaluation of 14 medical benchmarks, where it achieves new, state-of-the-art performance on 10 benchmarks, often significantly outperforming GPT-4 models. In particular, in MedQA (USMLE), Med-Gemini achieved an accuracy of 91.1%, outperforming previous models by 4.6%.

<figure class="wp-block-embed is-type-rich is-provider-twitter wp-block-embed-twitter“>

The creation of Med-Gemini

Med-Gemini opens interesting doors for ai in medicine. It can help physicians address complex diagnoses, engage in informative medical dialogue, and efficiently analyze large amounts of data within electronic medical records. The researchers achieved this specialization through innovative techniques:

• Self-training with web search integration: Med-Gemini can access and integrate up-to-date medical information from the web, ensuring your knowledge stays up to date.
• Multimodal adjustment: The model can be adapted to incorporate new medical data formats, making it future-proof.
• Custom encoders: Med-Gemini can process various types of data, including text, images, videos, and even sensor readings from medical equipment.

Med-Gemini Capabilities

Med-Gemini is presented as a family of high-capacity multimodal medical models based on Gemini. The clinical reasoning capabilities of the models are improved through self-training and web search integration, while multimodal performance is improved through fine-tuning and custom encoders.

Med-Gemini models achieve state-of-the-art (SoTA) performance in 10 of 14 medical benchmarks spanning text, multimodal, and long-context applications, outperforming the GPT-4 family of models in all benchmarks in which a direct comparison can be made. made.

The bar chart below demonstrates the relative percentage gains of the models over previous SoTA across all benchmarks. Especially in the MedQA benchmark (USMLE) a new SoTA is achieved, surpassing the previous best (Med-PaLM 2) by a significant margin of 4.6%.

Furthermore, re-annotation of the data set with expert clinicians reveals that 7.4% of the questions are considered unsuitable for evaluation due to missing key information, having incorrect answers, or supporting multiple plausible interpretations. These data quality issues are taken into account to characterize model performance more accurately.

Med-Gemini models excel in multi-modal and long-context capabilities, as evidenced by their SoTA performance on several benchmarks, including needle-in-a-haystack retrieval from long, deidentified medical records, and data benchmarks. answering questions in medical videos.

Beyond benchmarks, Med-Gemini's real-world potential is demonstrated through quantitative evaluation of medical summaries, referral letter generation, and medical simplification tasks where models outperform human experts, plus of qualitative examples of multimodal medical dialogue.

Safety and precision remain paramount

The article emphasizes the importance of safety and precision in medical applications. Researchers recognize the need for specialized techniques such as prompting and tuning to ensure responsible development of ai in this critical domain.

One such technique is the “uncertainty-guided search strategy.” This allows Med-Gemini to access and integrate relevant web search results during complex clinical reasoning tasks, resulting in more nuanced and reliable results.

Also Read: Top 7 ai Healthcare Solution Providers

Dialogue example

You can find the research paper here.

Our opinion

Med-Gemini's multimodal capabilities open doors to more natural and comprehensive interactions between healthcare providers and patients. Clinicians can take advantage of the model's ability to analyze various types of data, while the model itself can interact in a more conversational manner, requesting additional information to obtain a more complete picture.

This development adds to Google's growing portfolio of healthcare-focused ai models, including Med-PaLM 2, AlphaFold, and Flan-PaLM. Med-Gemini represents a significant step forward in ai-based healthcare, paving the way for a future with improved diagnostics, personalized medicine, and better communication between patients and providers.

Follow us Google news to stay up to date with the latest innovations in the world of ai, data science and GenAI.