Impact of Dataset Diversity on Traffic Sign Detection Models Under Harsh Visual Conditions

Object detection models represent a deeply important element of autonomous vehicles. These systems rely on computer vision in order to detect, classify, and respond to important road marks. In particular, traffic sign detection models are known to be highly accurate, yet their performance continues to face limitations when exposed to harsh visual conditions such as fog, poor lighting, or snow. While dataset diversity remains a possible solution to addressing this problem, the field lacks quantitative studies which establish a clear relationship between the diversity of datasets and model performance.

This study aimed at identifying whether increasing the diversity of training datasets could improve the model accuracy for detecting signs in harsh visual conditions. All images were extracted from the Mapillary Traffic Sign Dataset, and assigned four environmental metrics: brightness, contrast, blur, and harshness. Then, images were grouped to construct three different training datasets of increasing diversity: LOW, MEDIUM, and HIGH. In order to quantify these levels, each dataset was assigned a Vendi score to calculate a numeric diversity measurement based on the different image characteristics. In addition, a HARSH test dataset was constructed by taking the photos with the most visually challenging conditions. Three object detection models, YOLOv5s, YOLOv7-tiny, and YOLOv8n were tested on the three training datasets, then all evaluated on the HARSH set, measuring their performance using precision, recall, F1-score, and mAP@50.

Results demonstrated no significant or consistent correlation, refuting the initial hypothesis that there would be a clear relationship leading to an increase in accuracy. These findings demonstrate diversity alone is not sufficient enough to improve model robustness, as dataset size and model architecture may play a more significant role.