Pandas API for DuckDB, PostgreSQL & ClickHouse — keeping computation inside the database[P]
I've been building memFrame — an open-source dataframe API that compiles operations to SQL.
The idea:
**Python/DataFrame API → SQL → DuckDB / PostgreSQL / ClickHouse**
Instead of pulling data into Python and doing everything in pandas, memFrame tries to keep computation inside the database.
Currently it has operations for inspection, selection, cleaning, statistics, arithmetic, visualization, etc.
I'm intentionally releasing analytics operations incrementally , because first I want the currently released features to be well tested in public and once all the issues on the current features are solved then — groupby, window functions, sorting, filtering are on the way.
It comes with built-in multiagent architecture for chatting with your data in natural languages.
Would love some technical feedback & suggestions to improve this from people working with pandas/SQL/DuckDB/Postgres.
If the idea sounds useful, a ⭐ helps a lot.
[link] [comments]
Want to read more?
Check out the full article on the original site