New York Times Headlines Sentiment Analysis: Classification, Regression, and Transformer Models
Pari Bhandari
17/02/2026
Sentiment analysis is an important component of natural language processing, intending to determine the emotional tone of a text. This study investigates whether fine-tuned transformers outperform traditional machine learning models (classification and ordinal regression) on lexicon-labeled sentiment headlines with a severe class imbalance. The first section uses frozen BERT embeddings as input features for classification and ordinal regression models such as Random Forest, KNeighbors, Support Vector Machine, and Linear Regression, with hyperparameters optimized via GridSearchCV using 5-fold cross-validation on the training set. The second section directly employs a text-to-text transformer T5-small model evaluated in zero-shot and fine-tuned settings. Contrary to expectations, results demonstrate that Random Forest Classifier achieves the highest test macro F1 performance over fine-tuned T5-small and all regression models. While KNeighbors excelled in minority classes performance despite overfitting, all model families generally struggled to generalize across minority sentiment classes due to dataset imbalance. Fine-tuned T5-small, particularly at 5 epochs, achieves balanced performance across all sentiment labels, but collapses under premature early stopping. These findings challenge transformer superiority for short-text sentiment analysis, suggesting that BERT embeddings and stratified optimization are essential for achieving competitive performance on imbalance datasets. Further improvements could be achieved through larger models, hybrid embedding-transformer approaches, optimized hyperparameters, and balanced datasets.