Google DeepMind Research Introduces AMIE (Articulate Medical Intelligence Explorer): A Research AI System Based on Large Language Models (LLM) for Diagnostic Medical Reasoning and Conversations

Communication between doctor and patient is essential to providing effective and compassionate care. A medical interview is “the most powerful, sensitive and versatile instrument at the doctor's disposal,” according to studies. It is believed that clinical history accounts for between 60 and 80% of diagnoses in certain contexts.

Advances in general-purpose large language models (LLMs) have shown that ai systems can reason, plan, and include relevant context to maintain genuine conversations. The development of fully interactive conversational ai is within our reach thanks to this breakthrough, which opens up new potential for ai in healthcare. Conversations between patients and their caregivers can be natural and useful for diagnosis, and artificial intelligence systems involved in healthcare would understand clinical language and intelligently gather information even in the face of uncertainty.

Although LLMs can codify clinical knowledge and answer precise medical questions in a single turn, their conversational skills have been honed for industries other than healthcare. Previous research in health-related LLM has not yet compared the capabilities of ai systems with those of experienced physicians or conducted a comprehensive analysis of their ability to take a patient's medical history and engage in a diagnostic discussion.

Researchers at Google Research and DeepMind have developed an artificial intelligence system called AMIE (Articulate Medical Intelligence Explorer), designed to take a patient's medical history and talk to a doctor about possible diagnoses. Several real-world data sets were used to build AMIE. These data sets include answers to medical questions with multiple-choice questions, medical reasoning with extended questions vetted by experts, note summaries from electronic health records (EHRs), and interactions from large-scale recorded medical conversations. The AMIE training task mix included medical question answering tasks, reasoning, summary activities, and conversation production.

However, two major obstacles make passively collecting and transcribing real-world dialogue from in-person clinical visits impractical for training LLMs in medical conversations: (1) actual data from real-life conversations are not always comprehensive or scalable because they do not cover all possible medical conditions and scenarios; (2) real-life conversation data is often noisy because it contains slang, slang, sarcasm, interruptions, grammatical errors, and implicit references. As a result, AMIE's experience, capacity and relevance may be limited.

The team devised a personal game-based simulated learning environment for diagnostic medical dialogues in a virtual care environment to overcome these constraints. This allowed them to expand AMIE's knowledge and capabilities to various medical conditions and settings. In addition to the static corpus of medical QA, reasoning, summarization, and real-world dialogue data, the researchers used this environment to incrementally refine AMIE with a dynamic set of simulated dialogues.

To evaluate diagnostic conversational medical ai, they created a pilot evaluation rubric that includes both doctor- and patient-centered criteria for taking a patient's history and their diagnostic reasoning, communication skills, and empathy.

The team created and operated a blinded remote OSCE trial with 149 case scenarios from clinicians in India, the United Kingdom, and Canada. This allowed them to compare AMIE with PCPs in a balanced and randomized manner during consultations with verified patient actors. Compared to PCPs, AMIE demonstrated greater diagnostic accuracy across several metrics, including accuracy of the first and top three spots on the differential diagnosis list. Compared to PCPs, AMIE was considered better on 28 of 32 evaluation axes from the specialist physician's perspective and noninferior on the remaining 26 evaluation axes from the patient actor's perspective.

In their paper, the team highlights critical limitations and offers key next steps for clinical translation of AMIE to the real world. A major limitation of this research is the fact that they used a text chat platform, which PCPs for remote consultations were not accustomed to, but which allowed for potentially large-scale interaction between patients and LLMs specializing in diagnostic conversations.

Review the Paper and ai-system-for-diagnostic_12.html” target=”_blank” rel=”noreferrer noopener”>Blog. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on Twitter. Join our 36k+ ML SubReddit, 41k+ Facebook community, Discord channeland LinkedIn Grabove.

If you like our work, you will love our Newsletter..

Don't forget to join our Telegram channel

<!– ai CONTENT END 2 –>