Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(treesitter): add support for wasm parsers #28415

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

lewis6991
Copy link
Member

@lewis6991 lewis6991 commented Apr 19, 2024

Problem

Treesitter parsers are currently distributed as compiled shared objects which need to be compiled by the users (with the help of nvim-treesitter).

Solution

Allow loading wasm built parsers.

Notes:

  • For now this is an optional build-time feature (via ENABLE_WASMTIME) which will be dynamically linked.
    • Builds with make CMAKE_EXTRA_FLAGS=-DENABLE_WASMTIME=ON DEPS_CMAKE_FLAGS=-DENABLE_WASMTIME=ON
  • Treesitter requires being built with TREE_SITTER_FEATURE_WASM and also requires wasmtime at build time. So no [uv_]dlopen 😢
  • wasmtime has includes that require C11 (static_assert).

@lewis6991 lewis6991 force-pushed the tswasm branch 2 times, most recently from 7264715 to be7cc1c Compare April 19, 2024 15:19
@lewis6991 lewis6991 added DO NOT MERGE Nothing to see here; move along needs:discussion For PRs that propose significant changes to some part of the architecture or API labels Apr 19, 2024
@clason clason added this to the 0.11 milestone Apr 19, 2024
@lewis6991 lewis6991 force-pushed the tswasm branch 5 times, most recently from c2519a9 to 31f9b03 Compare April 20, 2024 17:27
@lewis6991 lewis6991 force-pushed the tswasm branch 2 times, most recently from 7b627ac to ca9e93d Compare April 22, 2024 07:55
@lewis6991 lewis6991 removed the DO NOT MERGE Nothing to see here; move along label Apr 22, 2024
cmake/Deps.cmake Outdated Show resolved Hide resolved
cmake/FindWasmtime.cmake Outdated Show resolved Hide resolved
cmake.deps/deps.txt Outdated Show resolved Hide resolved
@lewis6991 lewis6991 force-pushed the tswasm branch 3 times, most recently from 9f77fb6 to 2ecad21 Compare April 22, 2024 08:51
@lewis6991 lewis6991 marked this pull request as ready for review April 22, 2024 11:36
@github-actions github-actions bot requested review from bfredl and clason April 22, 2024 11:41
@clason
Copy link
Member

clason commented Apr 22, 2024

Is this something we want to add to BUILD.md (yet)?

@github-actions github-actions bot requested a review from clason May 8, 2024 08:38
@dundargoc
Copy link
Member

This is done. There are some followup work to be done such as making wasmtime work on debug windows as well as making the importing of the library more robust, but I will punt these tasks to a future date.

@dundargoc dundargoc marked this pull request as draft May 8, 2024 20:26
@github-actions github-actions bot removed request for bfredl and clason May 8, 2024 20:26
@dundargoc dundargoc marked this pull request as ready for review May 8, 2024 20:26
@github-actions github-actions bot requested review from bfredl and clason May 8, 2024 20:26
@@ -109,7 +109,14 @@ function M.add(lang, opts)
path = paths[1]
end

vim._ts_add_language(path, lang, symbol_name)
if vim.endswith(path, '.wasm') then
Copy link
Member

@justinmk justinmk May 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be case-insensitive?

return data;
}

static const char *wasmerr_to_str(TSWasmErrorKind werr)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this match the convention of the existing os_strerror, uv_strerror ?

Suggested change
static const char *wasmerr_to_str(TSWasmErrorKind werr)
static const char *wasm_strerr(TSWasmErrorKind werr)

src/nvim/lua/treesitter.c Outdated Show resolved Hide resolved
@dundargoc dundargoc marked this pull request as draft May 16, 2024 16:20
@github-actions github-actions bot removed request for bfredl and clason May 16, 2024 16:20
@dundargoc dundargoc force-pushed the tswasm branch 2 times, most recently from b3c1cd3 to 1c74802 Compare May 16, 2024 16:27
@dundargoc dundargoc marked this pull request as ready for review May 16, 2024 17:28
@dundargoc dundargoc force-pushed the tswasm branch 3 times, most recently from a3b394e to f857699 Compare May 26, 2024 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants