Detecting Human Gesture Anomalies to Enhance Road Safety of Autonomous Vehicles Using Machine Learning Models

This research paper uses Machine Learning (ML) models to detect human gesture anomalies to enhance Autonomous Vehicle safety. Specifically, it addresses the following research question: Can ML models trained on normal traffic control gestures reliably detect anomalous gestures and generalize across datasets? We hypothesize that ML models trained on normal traffic control gestures can detect abnormal poses and can transfer this recognition ability across datasets. This study investigates this hypothesis through three experiments: (1) anomaly detection on a Traffic Control Gestures (TCG) dataset using an autoencoder and reconstruction error as the anomaly proxy, (2) supervised classification of known pose categories to test the model’s ability to understand semantically meaningful information, and (3) cross-dataset generalization by converting a Chinese Traffic Police (CTP) gestures dataset to the European TCG-compatible pose feature for evaluation and generalization. Statistical analysis of the highest performing autoencoder confirmed a statistically significant separation of the reconstruction errors between the normal and Out-of-Vocabulary (anomalous) frames (Mann-Whitney U=5,938,675,636.5; p<0.001). The MLP model using MeanPose plus standard deviation features achieved 87.1% accuracy and an F1 score of 0.821 for semantic pose information. In the cross-dataset generalization, the model regained anomaly separability between CTP “inactive” or normal, and “active” or active poses used as proxy anomalies (anomalous poses), achieving an ROC-AUC of 0.727. Together, these results suggest that the model demonstrated a clear separability between pose types, an understanding of pose gesture information, and a meaningful cross-dataset adaptation after calibration