Machine learning predictions for microbial eukaryotic plankton: implications from unevenly structured data

Machine learning models provide a scalable approach for predicting the diversity of eukaryotic microbial plankton from environmental predictors. However, the extent to which these models generalize to data outside the training set remains poorly quantified. In this study, XGBoost was used to predict the 18S rRNA gene Shannon Diversity Index (SDI) from seven environmental predictors derived from satellite and model data. Surface samples were collected between 2001 and 2025 at two fixed stations in the northwestern Mediterranean (BBMO and SOLA), one fixed station in the northern Adriatic Sea (VIDA), and during the HOTMIX expedition, which sampled an east-west open-sea transect across the Mediterranean Sea. Model performance was assessed using standard repeated K-fold cross-validation (CV), Leave-One-Dataset-Out CV (LODO-CV), and a blocked spatiotemporal CV that combined LODO with temporal forward chaining. Under standard K-fold CV, the model showed moderate performance (R² = 0.44, RMSE = 0.59). In contrast, performance declined substantially under LODO-CV (R² = 0.09, RMSE = 0.73), with uniformly low per-dataset generalization, a pattern also observed with blocked spatiotemporal CV. VIDA and HOTMIX sampled environmental regimes distinct from those at BBMO and SOLA, which may partly explain their poor transferability. Additionally, BBMO and SOLA, despite similar environmental conditions, exhibited poor transferability, indicating that technical differences among independently collected 18S rRNA datasets likely constrain transferability, although their effects cannot be disentangled from environmental variation. Overall, these results highlight the limitations of imbalanced training data and underscore the importance of spatially explicit evaluation, protocol standardization, and environmentally representative coverage.