September 4, 2026•1 min read•from InfoQ

Presentation: From S3 to GPU in One Copy: Rethinking Data Loading for ML Training

Onur Satici explains how Vortex, an open-source columnar file format under the Linux Foundation, revolutionizes high-throughput data loading. He details how cascading lightweight encodings, layout-based segment pruning, and zero-copy memory pipelines eliminate CPU/NVMe bottlenecks to stream S3 data straight to GPUs at speeds up to 60 Gbps without requiring upfront data reprocessing.

By Onur Satici

Want to read more?

Check out the full article on the original site

View original article→

Tagged with

#Vortex

#columnar file format

#data loading

#GPU

#S3

#machine learning (ML)

#high-throughput

#zero-copy

#memory pipeline

#encoding

#segment pruning

#bottlenecks

#NVMe

#Linux Foundation

#open-source

#data reprocessing

#streaming

#60 Gbps

#lightweight encodings

#layout-based