Heinrich-Heine-Universität Düsseldorf

Evaluating machine learning models that generate time-dependent predictions in clinical settings remains challenging, as conventional metrics such as time-dependent AUC rely on observed event information and do not account for undocumented intermediate clinical events. In this work, we propose a novel evaluation framework based on class--cluster alignment. The method uses over-time clustering to identify structural patterns in such predictions and introduces alignment-based scores that quantify temporal separation between outcome groups. We evaluate the proposed framework on real-world clinical data from mortality prediction after allogeneic hematopoietic stem cell transplantation. Results show that the proposed scores capture temporally consistent separation patterns that complement conventional evaluation metrics. Overall, the proposed framework provides a novel perspective for evaluating time-dependent risk estimates in longitudinal clinical settings.