PESSIMISTIC ON AVERAGE, OPTIMISTIC IN THE TAIL: AN AUDIT OF CROSS-PROJECT ISSUE RESOLUTION-TIME PREDICTION

Abstract

Issue resolution-time predictors are usually evaluated within projects, whereas a model deployed on a new or data-sparse project may need histories from other projects. The operational risk of interest is severe underestimation: an issue whose actual resolution time exceeds twice its prediction. This paper reports a staged audit of that risk using TAWOS v1.1 (351,369 resolved issues from 39 Jira projects), leakage-safe creation-time reconstruction of mutable issue fields, and three fixed-hyperparameter regressors: Ridge, Extra Trees, and histogram gradient boosting. A nine-project discovery stage tested whether cross-project deployment systematically increases the Severe Underestimation Rate (SUR) relative to within-project models on the same chronological test issues. No model met the frozen rule (median cross-minus-within SUR from -0.027 to +0.004), and the stage returned STOP_OR_REDESIGN. A redesigned hypothesis, that temporal resolution-time shift conditions the cross-versus-within SUR difference, was tested on 18 untouched projects and did not replicate: correlations were weakly negative for all three models (rho = -0.10 to -0.30; Holm-adjusted p = 1.0). Post-confirmatory analysis showed a different failure mode. Cross-project predictions were often more pessimistic for typical issues, yet predicted log-duration distributions were strongly compressed, with median IQR ratios of 0.22 to 0.41 relative to observed outcomes. Consequently, the longest 10% of issues were underestimated by more than a factor of two at median rates of 0.98 to 1.00 under both regimes. In this setting, average or median bias concealed severe upper-tail risk; resolution-time predictors should therefore be evaluated with asymmetric and tail-stratified error alongside conventional accuracy measures.