Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WASM build using DuckDB WASM engine #394

Open
jonmmease opened this issue Sep 23, 2023 · 0 comments
Open

WASM build using DuckDB WASM engine #394

jonmmease opened this issue Sep 23, 2023 · 0 comments
Labels
enhancement New feature or request

Comments

@jonmmease
Copy link
Collaborator

In apache/datafusion#7633 I worked through which datafusion-* crates are currently compatible with wasm-pack. These are:

  • datafusion-common with default-features = false
  • datafusion-expr
  • datafusion-physical-expr
  • datafusion-optimizer
  • datafusion-sql

The main datafusion crate (called datafusion-core in the repo) is not yet compatible with wasm. This should eventually be doable, but there are issues to work out regarding dependencies with native build requirements.

The current vegafusion-wasm crate depends on vegafusion-core, which in turn depends on datafusion-common. The main reason to depend on this crate currently is for the ScalarValue support.

I realized that the vegafusion-datafusion-udf, vegafusion-dataframe, datafusion-sql (without the datafusion-conn feature enabled), and datafusion-runtime crates all only depend only on the datafusion crates that are compatible with WASM. The only vegafusion dependency on the core datafusion crate is datafusion-sql with the datafusion-conn feature flag enabled. This means that it should be possible to compile the vegafusion-runtime to WASM and use an alternative WASM-compatible connection implementation.

One obvious candidate for a WASM-compatible connection would be a connection to DuckDB WASM. This would be very similar to how the vegafusion Python API includes support for implementing connections in Python.

@jonmmease jonmmease added the enhancement New feature or request label Sep 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant