Federated learning has emerged as an approach for collaborative training between medical institutions while preserving data privacy. However, the nature of non-IID data, arising from differences in institutional specializations and regional demographics, creates significant challenges. This heterogeneity leads to client drift and suboptimal performance of the overall model. Existing federated learning methods primarily address this problem through model-centric approaches, such as modifying local training processes or global aggregation strategies. Still, these solutions often offer marginal improvements and require frequent communication, which increases costs and raises privacy concerns. As a result, there is a growing need for robust and efficient communication methods that can handle severe non-IID scenarios effectively.
Recently, data-centric federated learning methods have attracted attention to mitigate data-level divergence through virtual data synthesis and sharing. These methods, including FedGen, FedMix, and FedGAN, attempt to approximate real data, generate virtual representations, or share data trained by GAN. However, they face challenges such as low-quality synthesized data and redundant knowledge. For example, mixed approaches can distort data, and random selection for data synthesis often leads to repetitive and less meaningful updates to the global model. Additionally, some methods introduce privacy risks and remain ineffective in environments with limited communication. Addressing these issues requires advanced synthesis techniques that ensure high-quality data, minimize redundancy, and optimize knowledge extraction, enabling better performance in non-IID conditions.
Researchers at Peking University propose FedVCK (Federated Learning through Valuable Condensed Knowledge), a data-centric federated learning method designed for collaborative analysis of medical images. FedVCK addresses non-IID challenges and minimizes communication costs by condensing each customer's data into a small, high-quality data set using latent distribution constraints. A model-driven approach ensures that only essential, non-redundant knowledge is selected. On the server side, relational supervised contrastive learning improves global model updates by identifying strict negative classes. Experiments demonstrate that FedVCK outperforms state-of-the-art methods in predictive accuracy, communication efficiency, and privacy preservation, even under limited communication budgets and severe non-IID scenarios.
FedVCK is a federated learning framework that comprises two key components: client-side knowledge condensation and server-side relational supervised learning. On the client side, it uses distribution matching techniques to condense critical knowledge from local data into a small, learnable data set, guided by latent distribution constraints and importance sampling of difficult-to-predict samples. This ensures that the condensed data set addresses gaps in the overall model. The international model is updated on the server side using cross-entropy loss and prototype-based contrastive learning. Improves class separation by aligning features with their prototypes and moving them away from hard, negative classes. This iterative process improves performance.
The proposed FedVCK method is a data-centric federated learning approach designed to address the challenges of non-IID data distribution in collaborative medical image analysis. It was evaluated on various datasets, including colon pathology, retinal OCT scans, abdominal CT scans, chest x-rays, and general datasets such as CIFAR10 and ImageNette, spanning various resolutions and modalities. The experiments demonstrated the superior accuracy of FedVCK on all data sets compared to nine benchmark federated learning methods. Unlike model-centric methods, which showed mediocre performance, or data-centric methods, which struggled with synthesis quality and scalability, FedVCK efficiently condensed high-quality knowledge to improve overall model performance while maintaining low communication costs and robustness in severe non-IID scenarios. .
The method also demonstrated significant privacy preservation, as demonstrated by membership inference attack experiments, where it outperformed traditional methods such as FedAvg. With fewer rounds of communication, FedVCK reduced the risks of temporary attacks, offering better defense rates. Furthermore, ablation studies confirmed the effectiveness of its key components, such as model-guided selection, which optimized knowledge condensation for heterogeneous data sets. Extending their evaluation to natural data sets further validated its generality and robustness. Future work aims to expand the applicability of FedVCK to additional data modalities, including 3D CT scans, and improve condensation techniques for greater efficiency and effectiveness.
Verify he Paper. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on <a target="_blank" href="https://twitter.com/Marktechpost”>twitter and join our Telegram channel and LinkedIn Grabove. Don't forget to join our SubReddit over 60,000 ml.
Trending: LG ai Research launches EXAONE 3.5 – three frontier-level bilingual open-source ai models that deliver unmatched instruction following and broad context understanding for global leadership in generative ai excellence….
Sana Hassan, a consulting intern at Marktechpost and a dual degree student at IIT Madras, is passionate about applying technology and artificial intelligence to address real-world challenges. With a strong interest in solving practical problems, he brings a new perspective to the intersection of ai and real-life solutions.
<script async src="//platform.twitter.com/widgets.js” charset=”utf-8″>
