Is Overlapping Training Data Impacting Your Student ML Results?
I’m working on a student machine learning/computer vision project and recently realized that my validation set was not completely independent from my training set.
The project is more focused on comparing different experimental conditions rather than maximizing benchmark performance, but I’m concerned about the implications of this oversight.
From a research or academic perspective:
How serious is train/validation overlap in a student project?
Does it invalidate the entire project or mainly affect the reliability of the reported performance numbers?
If the main goal is comparing different experimental setups under the same evaluation procedure, are those comparisons still useful?
If you discovered this late in the project timeline, what would be the most reasonable way to address it?
I’m trying to understand how researchers, reviewers, and professors would view this situation.
[link] [comments]
Want to read more?
Check out the full article on the original site