A Multi-Dimensional Safety Readiness Index for Machine Learning-Based Industrial Control System Anomaly Detectors
Ashwajit Warwatkar
11/08/2026
Purpose: Industrial control systems (ICS) require anomaly detectors that perform reliably under realistic deployment conditions, yet current evaluation practice relies almost exclusively on event-F1 computed under artificially inflated attack priors. This paper addresses this gap by introducing the Safety Readiness Index (SRI), a five-dimensional composite evaluation framework designed to assess detector quality across performance, robustness, uncertainty quantification, explainability, and fault tolerance.
Methods: The SRI framework derives dimension weights from Fussell-Vesely fault-tree importance analysis applied to IEC 62443 and NIST SP 800-82 guidelines. We evaluate five unsupervised anomaly detectors, Isolation Forest, One-Class SVM, XGBoost (lag-1 residual forecaster), LSTM Autoencoder, and USAD, on the SWaT water treatment benchmark, which comprises 51 sensors and 9 documented attack events. We compare two aggregation rules, Weighted Linear Sum and Geometric Mean, and conduct attribution localization studies against ground-truth attacked sensors.
Conclusions: The choice of aggregation rule, not model capability, determines the top-ranked model: Weighted Linear Sum ranks XGBoost first on individual dimensions yet USAD leads overall, whereas Geometric Mean collapses XGBoost to last place due to its zero Robustness score. At a realistic attack prior of 10⁻⁶, XGBoost precision collapses to 7.4×10⁻⁵ while reconstruction-based models retain approximately 0.59 true positive rate under strict false-alarm constraints. Attribution localization reveals hit@1 equals zero for all five models, indicating that existing explainability metrics overstate interpretability by approximately 83%. We further prove via symbolic computation that any safety aggregator simultaneously satisfying strict Pareto-monotonicity and a zero-floor axiom is unsatisfiable. All findings, including negative results, are reported as observed.