UniMedSeg: Unified Multi-Modal Medical Image Segmentation via Uncertainty Guided Encoding and Frequency-Spatial Dual-Prompt Decoding
Text-driven medical image segmentation demands unified models based on multi-modal learning, capable of processing diverse imaging modalities while providing spatially reliable prediction estimates for clinical practice. Nevertheless, existing vision-language model solutions suffer from poor domain generalization and exhibit limited capability for uncertainty estimation to model the cumulative…
UniMedSeg is a novel unified vision-language framework designed for multi-modal medical image segmentation. This cutting-edge model harnesses the power of frozen pre-trained vision-language backbones, enabling it to process a variety of imaging modalities in a cohesive manner.
One of the key features of UniMedSeg is its modality-aware adaptation, which ensures the model can effectively handle diverse imaging data. This capability is crucial for achieving reliable clinical outcomes.
The framework also incorporates cross-layer uncertainty-guided encoding. This mechanism is designed to suppress misleading cross-modal features, preventing the model from making inaccurate predictions based on unreliable information. By doing so, it ensures that the final segmentation results are based on the most trustworthy and relevant data.
A standout feature of UniMedSeg is its frequency-spatial dual-prompt decoder. This innovative component disentangles anatomical structure and boundary details in the frequency domain, using dual textual prompts. The result is a more nuanced and accurate representation of the medical images being analyzed.
To further enhance the model's performance, UniMedSeg employs auxiliary losses. These losses sharpen boundary representations and calibrate pixel-wise uncertainty maps. This is particularly important in clinical settings, where precise risk estimation can be a matter of life and death.
The effectiveness of UniMedSeg has been demonstrated through extensive experiments. The results show that the model maintains competitive in-domain segmentation performance, even when dealing with diverse imaging data. Moreover, UniMedSeg shows improved domain generalization, meaning it can effectively handle medical images from different sources or hospitals. Perhaps most importantly, the model provides clinically useful spatial risk estimation, which can aid clinicians in making more informed decisions.
Written by urgent.news from bioRxiv's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
