Microsoft Researchers Introduce RadEdit: Biomedical Vision Models for Stress Testing Using Diffusion Image Editing to Remove Dataset Bias

Biomedical vision models are increasingly used in clinical settings, but a major challenge is their inability to generalize effectively due to data set changes—Discrepancies between training data and real-world scenarios. These changes arise from differences in image acquisition, changes in disease manifestations, and population variation. As a result, models trained with limited or biased data sets often perform poorly in real-world applications, posing a risk to patient safety. The challenge lies in developing methods to identify and address these biases before models are deployed in clinical settings, ensuring they are robust enough to handle the complexity and variability of medical data.

Current strategies to address changes in data sets often involve the use of synthetic data generated by deep learning models such as GANs and diffusion models. While these approaches have shown promise in simulating new scenarios, they are plagued by several limitations. Methods such as LANCE and DiffEdit, which attempt to modify specific features within medical images, often introduce unwanted changes, such as altering unrelated anatomical features or introducing visual artifacts. These inconsistencies reduce the reliability of these techniques in stress testing models for real-world medical applications. For example, a single skin-based approach like DiffEdit fights against spurious correlations, causing key features to be altered incorrectly, limiting its effectiveness.

A team of researchers from Microsoft Health Futures, the University of Edinburgh, the University of Cambridge, the University of California and Stanford University propose Rada novel diffusion-based image editing approach specifically designed to address the shortcomings of previous methods. RadEdit uses multiple image masks to precisely control which regions of a medical image are edited while preserving the integrity of surrounding areas. This multi-masking framework ensures that spurious correlations, such as the coexistence of chest drains and pneumothorax on chest radiographs, are avoided, maintaining visual and structural consistency of the image. RadEdit's ability to generate high-fidelity synthetic data sets allows you to simulate changes in real-world data sets, thereby exposing failure modes in biomedical vision models. This proposed method presents a significant contribution to stress testing models under conditions of acquisition, manifestation and population changes, offering a more accurate and robust solution.

RadEdit is based on a latent diffusion model trained on more than 487,000 chest x-ray images from large data sets, including MIMIC-CXR, ChestX-ray8, and CheXpert. The system takes advantage of dual masks: an edit mask for regions to be modified and a preservation mask for areas that must remain unchanged. This design ensures that edits are localized without disrupting other critical anatomical structures, which is crucial in medical applications. RadEdit uses the BioViL-T model, a domain-specific language and vision model for medical images, to evaluate the quality of your edits through image and text alignment scores, ensuring that synthetic images accurately represent medical conditions without introduce visual artifacts.

The evaluation of RadEdit demonstrated its effectiveness in stress testing biomedical vision models in three dataset change scenarios. In it acquisition shift In testing, RadEdit exposed a significant performance drop on a weak COVID-19 classifier, with accuracy dropping from 99.1% on biased training data to just 5.5% on synthetic test data, revealing the dependence of the model on confounding factors. For change of manifestationwhen the pneumothorax was removed while chest drains were preserved, the classifier's accuracy fell from 93.3% to 17.9%, highlighting its inability to distinguish between disease and treatment artifacts. In it population change In this scenario, RadEdit added anomalies to radiographs of healthy lungs, causing substantial decreases in the performance of the segmentation model, particularly in Dice scores and error metrics. However, more robust models trained with diverse data showed greater resilience across all turns, underscoring RadEdit's ability to identify model vulnerabilities and assess robustness under various conditions.

In conclusion, RadEdit represents an innovative approach to testing biomedical vision models by creating realistic synthetic data sets that simulate critical changes in the data sets. By leveraging multiple masks and advanced diffusion-based editing, RadEdit mitigates the limitations of previous methods, ensuring that edits are precise and artifacts are minimized. RadEdit has the potential to significantly improve the robustness of medical ai models, improving their real-world applicability and ultimately contributing to safer and more effective healthcare systems.

look at the Paper and Details. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on twitter.com/Marktechpost”>twitter and join our Telegram channel and LinkedIn Grabove. Don't forget to join our SubReddit over 50,000ml.

Subscribe to the fastest growing ML newsletter with over 26,000 subscribers

Aswin AK is a Consulting Intern at MarkTechPost. He is pursuing his dual degree from the Indian Institute of technology Kharagpur. He is passionate about data science and machine learning, and brings a strong academic background and practical experience solving real-life interdisciplinary challenges.