EXPLAINABLE MULTIMODAL DEEP LEARNING FOR EARLY ORAL CANCER DETECTION: INTEGRATING SELF-SUPERVISED REPRESENTATION LEARNING, ENSEMBLE TRANSFER LEARNING, AND CLINICAL CONTEXT

Abstract

Early detection of oral cancer, particularly oral squamous cell carcinoma (OSCC), remains challenging because clinical appearance can vary substantially across patients and lesions, while histopathological assessment may be affected by observer variability and differences in image quality. This study presents an explainable multimodal deep learning framework that combines clinical images, histopathological information, and patient-level contextual variables with self-supervised representation learning and ensemble transfer learning. The framework is designed to learn robust visual representations from unlabeled data before downstream supervised classification, thereby reducing dependence on extensive manual annotation. Complementary feature representations from multiple modalities are fused to capture lesion appearance, tissue-level structure, and contextual patient information. Transfer learning using established convolutional architectures provides a strong initialization, while ensemble prediction is used to improve stability across heterogeneous cases. To support clinical interpretability, Gradient-weighted Class Activation Mapping (Grad-CAM) is used to localize image regions influencing predictions, and SHAP is used to quantify the contribution of input features. In the supplied experimental manuscript, the reported system achieved 92% accuracy, 89% precision, 90% recall, 89% F1-score, and an AUROC of 0.92, with a reported confusion matrix of 950 true-negative/benign cases, 50 false positives, 40 false negatives, and 960 true-positive/malignant cases. Five-fold cross-validation was also reported as a robustness assessment. The proposed framework therefore provides a potentially useful direction for computer-assisted oral cancer screening, while further independent, multicenter validation and modality-specific ablation studies remain necessary before clinical deployment.