A Dual-Metric Framework for Rare-Event Prediction of Myocardial Infarction Complications: Discrimination, Threshold-Selected Detection, and Calibrated Multiclass Mortality

Most studies of myocardial infarction (MI) complications report a single performance metric, usually ROC-AUC, and identify a single winning model, yet under severe class imbalance, the best-ranking model is often not the one that would raise a clinical alert at a usable threshold. This paper presents a dual-metric evaluation framework that reports ranking performance (threshold-free ROC-AUC) and positive-class detection (threshold-dependent F1, average precision, and sensitivity and positive predictive value at thresholds chosen inside training data only) as complementary rather than interchangeable summaries, applied to 11 binary in-hospital complications and an 8-category lethal-outcome variable using 111 features (admission plus early-course variables). Five classifiers were compared through a unified, leakage-controlled pipeline with a single held-out 20% test set and stratified 10-fold cross-validation on the development data. Because cross-validation folds from one dataset are not independent experimental units, the primary model comparison treats the 11 binary clinical targets as the units of comparison (Friedman test with Holm-corrected Wilcoxon tests and Nemenyi critical differences). Across those 11 targets, Random Forest ranked first on discrimination (mean AUC 0.715; average rank 1.27; best on 9 of 11) but last on default-threshold detection (mean F1 0.020; average rank 4.55; F1 = 0 on 10 of 11 targets), whereas Logistic Regression ranked first on detection (best F1 on all 11; mean F1 0.211; average rank 1.00). Targets-as-units Friedman tests were significant on both axes (AUC chi-square = 28.87, p < 0.001; F1 chi-square = 32.32, p < 0.001). At F1-optimal thresholds selected inside training folds, mean sensitivity/PPV was 0.322/0.163 for Logistic Regression versus 0.107/0.202 for Random Forest; mean average precision was 0.225 for Random Forest and 0.217 for Logistic Regression. For multiclass mortality, XGBoost led held-out discrimination (macro-AUC 0.798, weighted F1 0.836), but balanced accuracy was only 0.26 and macro-F1 0.31, so the minority death classes remain hard to detect. SHAP rankings were consistent with established risk factors (face validity, not causal validation). Selecting a single best model without specifying the clinical operating point is unsound for rare-event MI complications; the framework matches models to ranking versus threshold-dependent detection and reports calibration and uncertainty for those tasks.