A simulation study to assess the sensitivity of concordance measures to the added predictive ability in survival prediction models

Abstract

Survival prediction models are often used in healthcare to estimate the prognosis of patients, guide treatment decisions and allocate resources effectively. When developing a survival prediction model or updating the model with a new predictor or novel marker, it is important to evaluate their performance with measures that facilitate natural and intuitive interpretations and are sensitive to the correct value added by the new predictor. Concordance statistic (C-statistic) is frequently used to assess the predictive performance, especially the discriminatory power of the models. Although multiple estimators for C-statistic, such as Harrell’s, Uno’s, and Gonen & Heller’s estimators, are available in the literature, their performance under different survival data conditions, such as varying levels of censoring, and the added predictive value from new predictor remains unclear. To address these aspects, this paper first showed an application of some popular C-statistics using two different datasets to describe how these C-statistics can be estimated and interpreted in practice, and secondly investigated their comparative performance using an extensive simulation study. The aim is to evaluate the robustness of these measures to varying degrees of censoring and their sensitivity to the added predictive value of a new predictor in the model, providing practical recommendations for their use. The findings revealed that Gonen & Heller’s C- statistic was comparatively more robust to increasing levels of censoring than both Harrell’s and Uno’s estimators, with Uno’s estimator performing moderately better than Harrell’s. Additionally, Gonen & Heller’s estimator proved to be more sensitive to the added predictive value of a new predictor, regardless of the type of predictor or the level of censoring. The paper concludes with recommendations for selecting the most effective C-statistics to evaluate the performance of survival prediction models across various real-world data scenarios.

Journal of Statistical Research 2024, Vol. 58, No. 2, pp.369-383