Can Data Augmentation Change the Best-Performing Classifier? Evidence From a Machine Learning Study on Sarcopenia

Abstract

Introduction

One of the M-Brain project's aims is to identify biomarkers that can distinguish between healthy and pathological ageing. The choice of classifier can have a significant impact on predictive performance due to the challenges posed by limited sample size, class imbalance and heterogeneous features. Using data augmentation techniques can improve the performance of different machine learning algorithms.

Objectives

Compare different supervised classification models to assess whether the best-performing classifier on real data remained the most effective after data augmentation.

Methods

We used bioelectrical impedance analysis (BIA) parameters and muscle thickness data measured by ultrasound collected. The original dataset included 50 subjects (20 diagnosed with sarcopenia according to EWGSOP2 criteria, and 30 without) and 17 variables (nine for BIA and eight for ultrasound). We trained five supervised classification algorithms—Logistic Regression with L2 regularization (LR-L2), K-Nearest Neighbors (KNN), Support Vector Machine (SVM) with a radial basis function kernel, Linear Discriminant Analysis, and Random Forest—on the original dataset. The dataset was subsequently augmented using jittering, a technique that generates synthetic samples through small random perturbations of the training data. To preserve the underlying data structure, synthetic instances were generated using the learned weights of each classifier. Any new instance was added to the dataset according to a given threshold. The five algorithms were then re-evaluated to assess the impact of data augmentation on classification performance. Model validation was performed using Leave-One-Out Cross-Validation, and performance by balanced accuracy, recall, precision, and F1-score measures.

Results

In the real-data analysis, KNN showed the best overall performance, achieving a balanced accuracy of 0.650, recall of 0.600, precision of 0.571, and F1-score of 0.585. All classifiers performance improved after data augmentation, although gains were not consistent across all models and settings. The KNN + SVM pipeline performed best, i.e. the feature weights derived from KNN guided the selection of the most suitable synthetic instances generated through jittering, when we used a conservative threshold (1.0–0.7). The five classifiers were then trained and evaluated on the augmented dataset, and SVM achieved the best performance, reaching a balanced accuracy of 0.795, recall of 0.741, precision of 0.769, and F1-score of 0.755. Surprisingly, the LR-L2 + KNN pipeline achieved the highest classification metrics at the lower threshold (0.6–0.5), reaching a balanced accuracy of 0.807, recall of 0.815, precision of 0.733, and F1-score of 0.772 at a threshold of 0.5.

Conclusions

The results suggest that the classifier that performs best on the original dataset may not remain the best after data augmentation. KNN achieved the highest performance on the original data, whereas SVM outperformed all other classifiers after synthetic data generation. However, when the threshold for generating synthetic instances was relaxed, KNN once again became the top-performing classifier. These findings underscore the importance of reevaluating classification models after data augmentation rather than assuming that the model selected on the original dataset will continue to perform best. They also highlight the importance of balancing class distributions and verifying that the statistical properties of the original variables are preserved following data augmentation.