Struggling with Overfitting on Medical Imaging Task [D]

Hi everyone,

I’m working on a 2-class classification problem (LCA vs. RCA coronary arteries) using 2D X-ray angiograms. I’m currently stuck in a cycle of extreme overfitting and could use some advice on my training strategy.

The Setup:

• Dataset: Small (~900 training frames from ~300 unique DICOMs).
• Architecture: InceptionV3 (PyTorch).
• Input: Grayscale .npy arrays converted to 3-channel, resized to 299x299.
• Current Strategy: Transfer learning from ImageNet. I’ve tried full unfreezing and partial unfreezing (last blocks).

The Problem: My training accuracy hits ~95-99% within a few epochs, but validation accuracy peaks early (around 74-79%) and then collapses toward 30-40% as the model starts memorizing the specific textures of the training patients.

What I’ve Tried So Far:

1. Normalization: Standard ImageNet mean/std (applied at load time).
2. Class Weights: Handled 2:1 imbalance (LCA:RCA).
3. Regularization: Added Dropout (tried 0.3 to 0.6) and Weight Decay (1e-4).
4. Augmentation: Flips, 25deg rotations, and translation.
5. Schedulers: ReduceLROnPlateau (factor 0.5, patience 8).

Would love any insights or papers you'd recommend for small-sample medical classification. Thanks!