Experimenting With the Built-in Tree-sitter Support in Emacs

Update: 22nd January: As a commenter implies, things are in a fair bit of flux still. I have been following development and while the existing treesit-enabled modes are bare-bones, new hooks and variables are being added all the time. You are unlikely to want to completely switch at this stage, but the future is very promising.

The following posts are a great place to start: overview, note in particular major-mode-remap-alist, details.

For another example of what can be unlocked, check out this minimalist expand-region replacement.

A quick post today, mostly my working notes.

I have a Friday habit of building Emacs from the master branch, and reading the additions to etc/NEWS1. The last few years have been particularly active, with support for built-in LSP (via eglot), native compilation, improved long-lines performance, etc.

The biggest news recently has been the incorporation of treesitter support. This has already been available via dynamic modules, but is now available natively, with a bunch of language modes updated with support for it as well. I won't dwell on what treesitter is, but benefits include much improved font-lock and code-folding support and the potential for semantic navigation, etc.

I wanted to try this out, but because tree-sitter itself is an external library, and new enough not to be available in my distribution's package manager, it requires a bit more leg-work. Because I'm just experimenting (and don't use an immutable distribution2), I didn't fancy installing the library, so had a few more hoops to add and jump through. More detailed notes are available here.

  1. git clone https://github.com/tree-sitter/tree-sitter

  2. cd tree-sitter; make all

  3. Now we need to configure Emacs --with-tree-sitter, but because it's not installed we need to tell it the library to link against (and where to find it), and where to find the dev headers. Ignoring other options3, for me this looks like TREE_SITTER_CFLAGS=-I~/Projects/tree-sitter/lib/include TREE_SITTER_LIBS="-L~/Projects/tree-sitter -ltree-sitter" ./configure --with-tree-sitter

  4. Now, that's dynamically linked, so to run it we still need to modify the environment so it can find the library: LD_LIBRARY_PATH=~/Projects/tree-sitter ~/Projects/emacs/src/emacs Update: I now also needed LD_LIBRARY_PATH set to build as well, because building involves starting a bare-bones Emacs in order to dump. Not sure if I missed this earlier, or the build process has changed.

Now you have an Emacs with built-in support, and you can verify by evaluating (treesit-available-p), but it doesn't get you very far because the language-specific information is still missing. To see this, open a supported mode, say a javascript file, and run M-x js-ts-mode. You will probably see a warning like "Cannot activate tree-sitter, because language definition for javascript is unavailable (not-found): (libtree-sitter-javascript libtree-sitter-javascript.so) No such file or directory".

To resolve this we need the language-specific modules (the equivalent of https://github.com/emacs-tree-sitter/tree-sitter-langs for the dynamic-module project), which you can obtain from https://github.com/casouri/tree-sitter-module/. These can either be built, or just downloaded from the releases; you can extract them into /.emacs.d/tree-sitter (see variable treesit-extra-load-path for details or customisation). Now if you try again — no restart necessary — your new mode should load successfully!

So far I haven't observed any issues from having both the built-in and the dynamic-module support. You will see that the built-in support uses the prefix treesit- while the third-party package uses tree-sitter-. The updated modes contain -ts-, eg python-ts-mode, js-ts-mode, etc.


1

My git alias dlp = diff ORIG_HEAD..FETCH_HEAD is handy here.

3

My full line is: TREE_SITTER_CFLAGS=-I~/Projects/tree-sitter/lib/include TREE_SITTER_LIBS="-L~/Projects/tree-sitter -ltree-sitter" ./configure --with-native-compilation --without-mailutils --with-json --with-xwidgets --with-xinput --with-tree-sitter


comments powered by Disqus