vaex

SKILL.md

Vaex

Overview

Vaex is a high-performance Python library designed for lazy, out-of-core DataFrames to process and visualize tabular datasets that are too large to fit into RAM. Vaex can process over a billion rows per second, enabling interactive data exploration and analysis on datasets with billions of rows.

When to Use This Skill

Use Vaex when:

Processing tabular datasets larger than available RAM (gigabytes to terabytes)

Performing fast statistical aggregations on massive datasets

Creating visualizations and heatmaps of large datasets

Building machine learning pipelines on big data

Converting between data formats (CSV, HDF5, Arrow, Parquet)

Needing lazy evaluation and virtual columns to avoid memory overhead

Working with astronomical data, financial time series, or other large-scale scientific datasets

Core Capabilities