Vaex
Data
dataframe library
SHOW HN2019
264points
26comments
#126of 2,195 in 2019
Top 6%of 2019 by points
How HN took it
CuriousMostly questions and neutral comments
5 positive, 11 neutral or asking, 0 mixed, 2 critical, out of 18 comments about the project. Thread score +0.13.
Performance: 1 for, 1 againstExecution: 2 for, 0 againstIdea: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Fast performance on large data compared with pandas-style workflows
From 3 comments
No Python looping happening in Vaex :), otherwise, we wouldn't get this performance. We are also working on GraphQL support, with a Hasura-like API:…
maartenbreddels on HNIve been switching an reporting system that did analysis in postgres, to analysis in pandas (mostly business-stats type summaries). It feels like growing wings and a jetpack.…
mooneater on HNAh, of cause. It makes sense you don't loop in Python. This all seems pretty interesting. I will give it a go.
drblah on HN
Pandas-like API that is easy to use
From 2 comments
It is not similar to Dask, but similar to dask.dataframe. Dask.dataframe is built on top of Pandas, but that also means it inherits its issue, like memory usage, and performance.…
maartenbreddels on HNDask’ s compatibility with Pandas makes it ideal. Very easy to use.
floki999 on HN
Objections
Analysis of the example mixes in an unrealistic interpretation of cash tips
From 1 comment
> 99.97% of the passengers that pay by cash do not leave tips. Correction - 99.97% of cash tips don't go recorded. Why pay tax if they never saw it happen? Compare this to: >…
darkstar999 on HN
Requests
Benchmarks against Postgres and Spark
From 2 comments
How would this compare to ingesting this in a locally running rdbms, like postgres, and applying some judicious indexing? Or maybe a local spark cluster? I'm tempted to benchmark…
isoprophlex on HNThink you'd want to compare with a columnar database since this is an analytic workload, not a transactional one. And a 4 node Spark cluster with Parquet or Arrow files and a…
kthejoker2 on HN
Support for full Dask integration
From 1 comment
It is not similar to Dask, but similar to dask.dataframe. Dask.dataframe is built on top of Pandas, but that also means it inherits its issue, like memory usage, and performance.…
maartenbreddels on HN
Questions
How does it compare with Postgres, Spark, Dask or Xarray?
From 2 comments
How would this compare to ingesting this in a locally running rdbms, like postgres, and applying some judicious indexing? Or maybe a local spark cluster? I'm tempted to benchmark…
isoprophlex on HNVaex seems to be very similar to Dask and Xarray. Which one to choose?
jononor on HN
Alternatives named
Dask dataframe
From 2 comments
Vaex seems to be very similar to Dask and Xarray. Which one to choose?
jononor on HNDask’ s compatibility with Pandas makes it ideal. Very easy to use.
floki999 on HN
Postgres
From 2 comments
How would this compare to ingesting this in a locally running rdbms, like postgres, and applying some judicious indexing? Or maybe a local spark cluster? I'm tempted to benchmark…
isoprophlex on HNI am currently doing an internship as part of my masters degree where I am analyzing ~30 GB of data. I'm using Postgres + Python and it is working quite well, even on my 2014…
drblah on HN
Spark
From 2 comments
How would this compare to ingesting this in a locally running rdbms, like postgres, and applying some judicious indexing? Or maybe a local spark cluster? I'm tempted to benchmark…
isoprophlex on HNThink you'd want to compare with a columnar database since this is an analytic workload, not a transactional one. And a 4 node Spark cluster with Parquet or Arrow files and a…
kthejoker2 on HN
Xarray
From 1 comment
Vaex seems to be very similar to Dask and Xarray. Which one to choose?
jononor on HN
Pandas
From 1 comment
It is not similar to Dask, but similar to dask.dataframe. Dask.dataframe is built on top of Pandas, but that also means it inherits its issue, like memory usage, and performance.…
maartenbreddels on HN