ROBUSTNESS OF CROSS-PROJECT DEFECT PREDICTION TO MISSING AND NOISY SOFTWARE METRICS

Abstract

Cross-project defect prediction (CPDP) is usually evaluated on complete, clean software metrics, whereas measurement tools in practice may omit or corrupt some metrics. We examine how simulated missing and noisy target-project metrics affect threshold-independent discrimination and ranking, and fixed-threshold classification, and whether source-only missingness-augmented training changes these outcomes. Using the five AEEEM projects (5,371 modules), we hold out each project in turn and train on the other four. Three Random Forest configurations are compared: standard, missingness-aware, and source-only missingness-augmented. Target metrics are perturbed by random missingness (missing completely at random, MCAR; 5–30%), noise (5–30%), and systematic dropout of the five most important metrics. At 30% MCAR, augmented training lowers mean ROC-AUC (area under the receiver operating characteristic curve) by 0.0088 (0.7301 vs 0.7389) but raises the Matthews correlation coefficient (MCC) by 0.0632 (0.3020 vs 0.2388) and balanced accuracy by 0.0389 (0.5864 vs 0.5475); the MCC and balanced-accuracy gains occur in all five projects. Average precision (AP; +0.0010) and Precision@20% (+0.0070) show no consistent change. With five independent projects, the smallest attainable two-sided exact p-value is 0.0625, so all findings are exploratory. Gains in threshold-dependent metrics did not translate into ranking gains, and part of the MCC gain also appears on clean data. The study is a limited-scope robustness evaluation, not a new algorithm; generalisation beyond AEEEM and Random Forest is untested.