Wang, Hao, Ahn, Euijoon, Bi, Lei, and Kim, Jinman (2025) Self-supervised multi-modality learning for multi-label skin lesion classification. Computer Methods and Programs in Biomedicine, 265. 108729.

Preview

PDF (Published Version) - Published Version Available under License Creative Commons Attribution. Download (3MB) | Preview

Abstract

BACKGROUND: The clinical diagnosis of skin lesions involves the analysis of dermoscopic and clinical modalities. Dermoscopic images provide detailed views of surface structures, while clinical images offer complementary macroscopic information. Clinicians frequently use the seven-point checklist as an auxiliary tool for melanoma diagnosis and identifying lesion attributes. Supervised deep learning approaches, such as convolutional neural networks, have performed well using dermoscopic and clinical modalities (multi-modality) and further enhanced classification by predicting seven skin lesion attributes (multi-label). However, the performance of these approaches is reliant on the availability of large-scale labeled data, which are costly and time-consuming to obtain, more so with annotating multi-attributes

METHODS: To reduce the dependency on large labeled datasets, we propose a self-supervised learning (SSL) algorithm for multi-modality multi-label skin lesion classification. Compared with single-modality SSL, our algorithm enables multi-modality SSL by maximizing the similarities between paired dermoscopic and clinical images from different views. We introduce a novel multi-modal and multi-label SSL strategy that generates surrogate pseudo-multi-labels for seven skin lesion attributes through clustering analysis. A label-relation-aware module is proposed to refine each pseudo-label embedding, capturing the interrelationships between pseudo-multi-labels. We further illustrate the interrelationships of skin lesion attributes and their relationships with clinical diagnoses using an attention visualization technique.

RESULTS: The proposed algorithm was validated using the well-benchmarked seven-point skin lesion dataset. Our results demonstrate that our method outperforms the state-of-the-art SSL counterparts. Improvements in the area under receiver operating characteristic curve, precision, sensitivity, and specificity were observed across various lesion attributes and melanoma diagnoses.

CONCLUSIONS: Our self-supervised learning algorithm offers a robust and efficient solution for multi-modality multi-label skin lesion classification, reducing the reliance on large-scale labeled data. By effectively capturing and leveraging the complementary information between the dermoscopic and clinical images and interrelationships between lesion attributes, our approach holds the potential for improving clinical diagnosis accuracy in dermatology.

Item ID:	86087
Item Type:	Article (Research - C1)
ISSN:	1872-7565
Copyright Information:	© 2025 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
Funders:	Australian Research Council (ARC)
Projects and Grants:	ARC DP200103748
Date Deposited:	14 Jul 2025 23:53
FoR Codes:	46 INFORMATION AND COMPUTING SCIENCES > 4603 Computer vision and multimedia computation > 460306 Image processing @ 40% 46 INFORMATION AND COMPUTING SCIENCES > 4603 Computer vision and multimedia computation > 460308 Pattern recognition @ 30% 40 ENGINEERING > 4003 Biomedical engineering > 400304 Biomedical imaging @ 30%
SEO Codes:	20 HEALTH > 2002 Evaluation of health and support services > 200206 Health system performance (incl. effectiveness of programs) @ 100%
	More Statistics